Building Quality Gates and Automated Decisions in CI/CD Pipelines. Lessons Learned with 200,000 Regressions.

location_city Chicago schedule Mar 29th 02:50 - 03:30 PM CDT place Sauganash West

Today’s modern development disciplines -- whether Agile, Continuous Integration (CI) or Continuous Delivery (CD) -- have completely transformed how teams develop and deliver applications faster. But in order to compete in today’s fast-paced digital economy, companies must also test faster and test continuously. Teams must make fast and reliable decisions by tracking test results of thousands tests for hundreds of runs triggered by code changes. 

But what if we could leverage Machine Learning to do the analysis and help team make rule-based decisions using a concept named Quality Gates?

In this presentation, I will share learnings derived from regressions in 200,000 test cases, and how we use Machine Learning (ML) to triage failed reports and reduce team effort on results analysis. How we organized a Quality Gates and formed a rule-based engine to identify Go and No-Go decisions for a particular set. And how we automate release pipelines by use of meaningful feedback from the automated testing results.

You will learn how to build Quality Gates in ReportPortal, how to integrate feedback within CI/CD pipelines. How to utilize ML analysis in triaging of failed reports. And finally make a leap forward to Continuous Testing.


Outline/Structure of the Presentation

  1. Learnings and pain points derived from 200,000 regression sets
  2. Minimization of effort to triage failed reports
  3. Extension of automated test cases with attribute, related to components, priority, severity, flows
  4. Forming the dashboards for component health checks and environment checks
  5. Building Quality Gates based on rules and test case attributes.
  6. Organizing sequences of pipelines in Ci/CD tool based on automated QG decision
  7. Outcome and time savings

Learning Outcome

You will learn how to :

  1. Build Quality Gates in ReportPortal
  2. Integrate test automation feedback within CI/CD pipelines
  3. Extend automated test cases with attributes for detailed structuring
  4. Utilize ML analysis for triaging of failed reports

Target Audience

QA Managers, QA Leads, Testing team leads, mid - senior test automation engineers

schedule Submitted 7 months ago

  • Maksim Sadym

    Maksim Sadym / Sri Harsha - WebDriver BiDi + Selenium. Why and How?

    40 Mins

    We are presenting a hands-on talk with demonstrating Selenium work over WebDriver BiDi to ChromeDriver.

    1. Short introduction into WebDriver BiDi
    2. Hands-on demonstration
  • Rahul Kumar

    Rahul Kumar - Healenium : The self healing test automation framework

    Rahul Kumar
    Rahul Kumar
    Quality Architect
    schedule 7 months ago
    Sold Out!
    40 Mins
    Deep Dive Session

    As part of   Helanium committee from EPAM , i will give the insight and techniques we use to create a selenium based framework leveraging power of machine learning techniques where an automation framework can heal itself .

    It improves stability of Selenium based test by minimizing maintenance time of test automation code .Healing is performed at run time so that UI changes will not break the test cases . Only product related changes will causes test to fail .

    Healenium support advance selenium features like parallelisation , remote execution , iframes , Js actions and selenium based framework such as selenide. It also works for appium based test automation .

    Most suitable for projects where :

    1. Frequent UI changes or weak locator.

    2. UI generated by CMS .

    3. Automation based on Selenium and Webdriver.

    4. Localization testing

    5. Testing under IE.


  • Christian Bromann

    Christian Bromann - Test Authoring made easy: the new WebdriverIO VS Code Extension

    Christian Bromann
    Christian Bromann
    Software Engineer
    schedule 7 months ago
    Sold Out!
    40 Mins

    When it comes to creating a solid set of tests to cover the quality of your application, test authoring is one of the most important processes and providing an environment where tests can be written efficiently will make you not only write better tests but also make you want to write more.

    The new VS Code extension for WebdriverIO makes it very easy to write and debug tests within your favorite IDE. It provides a nice UI to set up your environment as well as run your tests. With various debugging capabilities you can halt the test at any time to debug the state of your application and probe it by sending more commands.

    Join Christian Bromann, Founding Engineer at Stateful, in this session and explore all features of the new VS Code Extension for WebdriverIO.