Building Quality Gates and Automated Decisions in CI/CD Pipelines. Lessons Learned with 200,000 Regressions.
Today’s modern development disciplines -- whether Agile, Continuous Integration (CI) or Continuous Delivery (CD) -- have completely transformed how teams develop and deliver applications faster. But in order to compete in today’s fast-paced digital economy, companies must also test faster and test continuously. Teams must make fast and reliable decisions by tracking test results of thousands tests for hundreds of runs triggered by code changes.
But what if we could leverage Machine Learning to do the analysis and help team make rule-based decisions using a concept named Quality Gates?
In this presentation, I will share learnings derived from regressions in 200,000 test cases, and how we use Machine Learning (ML) to triage failed reports and reduce team effort on results analysis. How we organized a Quality Gates and formed a rule-based engine to identify Go and No-Go decisions for a particular set. And how we automate release pipelines by use of meaningful feedback from the automated testing results.
You will learn how to build Quality Gates in ReportPortal, how to integrate feedback within CI/CD pipelines. How to utilize ML analysis in triaging of failed reports. And finally make a leap forward to Continuous Testing.
Outline/Structure of the Presentation
- Learnings and pain points derived from 200,000 regression sets
- Minimization of effort to triage failed reports
- Extension of automated test cases with attribute, related to components, priority, severity, flows
- Forming the dashboards for component health checks and environment checks
- Building Quality Gates based on rules and test case attributes.
- Organizing sequences of pipelines in Ci/CD tool based on automated QG decision
- Outcome and time savings
Learning Outcome
You will learn how to :
- Build Quality Gates in ReportPortal
- Integrate test automation feedback within CI/CD pipelines
- Extend automated test cases with attributes for detailed structuring
- Utilize ML analysis for triaging of failed reports
Target Audience
QA Managers, QA Leads, Testing team leads, mid - senior test automation engineers
Links
Also known as Dzmitry Humianiuk.
Public Speaking experience:
- Selenium Camp 2020, Ukraine, Kyiv talk "test-gap-analysis-and-minimization-of-regression-suites-via-test-to-code-mapping-with-drill4j"
- DelEx Conf 2020, Belarus Talk: "Self-healing test automation with Healenium. Test Gap analysis and minimization of regression suite with drill4j"
- SeleniumConf 2019, London, UK Talk: "Using ML to find value in your automated tests"
- Automation in Action 2019, Ukraine, Kyiv Talk "App code coverage by automated tests – easy! We made test-to-code mapping open-sourced tool"
- SeleniumCamp 2019, Ukraine, Kyiv Talk: "Building a self-service marketplace for Test Data"
- DelEx Conf 2019, Belarus Talk: "Building self-managing Test Data marketplace"
- [email protected] TestingCup 2018, Poland Talk: "Machine Learning usage for test fail categorization"
- top [email protected] DelEx Conf 2018, Belarus Talk: "The applicability of machine learning and neural networks in test automation"
- TestCon Moscow 2018,
- SeleniumCamp 2018, Kiev, Ukraine
- Belgrade TestConference 2017, Belgrade, Serbia, http://bg-testconference.rs
- Top [email protected] TestConf 2017, Vilnius http://www.testcon.lt/
- Top [email protected] QAFest 2017, Kiev, Ukraine, http://qafest.com/
- EPAM SEC2017, Budapest, Hungary https://events.epam.com/events/sec-2017
- BAQ Conference 2017, Lviv, Ukraine http://baq.dakiry.com.ua
- top [email protected] SeleniumCamp2017, Kiev http://seleniumcamp.com
- top [email protected] TestCon2017, Moscow http://testconf.ru
- EPAM SEC 2016, 2014 (best speaker award)
- SQA Days #17, #20 (Minsk, Belarus) http://sqadays.com
schedule Submitted 7 months ago
People who liked this proposal, also liked:
-
keyboard_arrow_down
Maksim Sadym / Sri Harsha - WebDriver BiDi + Selenium. Why and How?
40 Mins
Presentation
Intermediate
We are presenting a hands-on talk with demonstrating Selenium work over WebDriver BiDi to ChromeDriver.
- Short introduction into WebDriver BiDi
- Hands-on demonstration
-
keyboard_arrow_down
Rahul Kumar - Healenium : The self healing test automation framework
40 Mins
Deep Dive Session
Intermediate
As part of Helanium committee from EPAM , i will give the insight and techniques we use to create a selenium based framework leveraging power of machine learning techniques where an automation framework can heal itself .
It improves stability of Selenium based test by minimizing maintenance time of test automation code .Healing is performed at run time so that UI changes will not break the test cases . Only product related changes will causes test to fail .
Healenium support advance selenium features like parallelisation , remote execution , iframes , Js actions and selenium based framework such as selenide. It also works for appium based test automation .
Most suitable for projects where :
1. Frequent UI changes or weak locator.
2. UI generated by CMS .
3. Automation based on Selenium and Webdriver.
4. Localization testing
5. Testing under IE.
-
keyboard_arrow_down
Christian Bromann - Test Authoring made easy: the new WebdriverIO VS Code Extension
40 Mins
Presentation
Beginner
When it comes to creating a solid set of tests to cover the quality of your application, test authoring is one of the most important processes and providing an environment where tests can be written efficiently will make you not only write better tests but also make you want to write more.
The new VS Code extension for WebdriverIO makes it very easy to write and debug tests within your favorite IDE. It provides a nice UI to set up your environment as well as run your tests. With various debugging capabilities you can halt the test at any time to debug the state of your application and probe it by sending more commands.
Join Christian Bromann, Founding Engineer at Stateful, in this session and explore all features of the new VS Code Extension for WebdriverIO.