Leveraging Artificial Intelligence to create self-healing tests

A key requirement for successful test automation is to get past the brittle or fragile nature of test scripts. Any Selenium (or Appium) developer has encountered the dreaded "NoSuchElement Exception". A locator that worked yesterday may fail today. What's a test engineer to do?

Fortunately the field of AI provides promising solutions and allows for the creation of self-healing tests. Tests that can find elements across all environments. Tests that can learn from "human-in-the-loop" intervention and work perfectly thereafter. Imagine automated tests that "just work"!

This session will look at how to apply the latest in AI and Machine Learning technologies to improve your test scripts. With the plethora of new open source AI libraries made available by companies such as Google, the ability to leverage AI in your applications is more accessible than ever.

This session will be a primer on AI technologies and how they can be utilized for perfect test automation.

 
 

Outline/Structure of the Tutorial

  • Introduction to AI
    • Neural Network primer
    • Understanding Convolution Neural Networks (CNN), Fully convolution networks (FCN), Deconvolution Neural Networks (DNN), Generative Adversarial Networks (GAN)
  • Machine Learning and Image libraries
    • Open source image libraries such as Open CV
    • Open source machine learning libraries such as TensorFlow
  • Using AI to make test cases more robust
    • How to apply ML libraries to solve common test case problems
  • How self-learning allows test cases to improve over time
  • Continuing your AI Journey

Learning Outcome

Participants will walk away with a good understanding of the various AI models and libraries used today. Moreover, attendees will also have a pragmatic understanding of how these AI models can be applied to improve automated tests.

Target Audience

QA engineers, Developers

Prerequisites for Attendees

This course will address utilizing AI to address common pitfalls in automated test cases. As such, participants should already have a good working knowledge of Appium or Selenium.

  • Selenium or Appium experience
schedule Submitted 2 months ago

Public Feedback

comment Suggest improvements to the Speaker

  • Liked Shweta Sharma
    keyboard_arrow_down

    Shweta Sharma / Nikita Jain - Accessibility testing 101

    45 Mins
    Talk
    Beginner

    "This world is such a beautiful place to live in." If you can read the first sentence without any screen readers or assistance, you're privileged. As technologists, shouldn’t we be more empathetic towards differently-abled people and make all parts of our website accessible to them? In my humble opinion, the true power of technology can be identified when it reaches out to people of all kinds having different physical or psychological challenges. We not only legally bound to provide Accessibility but also it should be considered as our moral responsibility.

    As testers, we have a wonderful opportunity to contribute to Accessibility by ensuring that the site is accessible in many different ways. Although it is impossible to identify all the issues that exist around Accessibility in the world, we are lucky enough to still understand a majority of them. With this understanding, there have been many measures taken in order to make your site accessible. But, don’t forget - we are QA engineers. We got to ensure that the site is accessible as per the standards set by WCAG 2.0 (AA) by testing for accessibility using various tools and techniques.

  • Liked Prateek Sethi
    keyboard_arrow_down

    Prateek Sethi - Enhance Test Automation ROI with Docker based Selenium Environment

    Prateek Sethi
    Prateek Sethi
    QA Lead
    3PillarGlobal
    schedule 3 days ago
    Sold Out!
    20 Mins
    Talk
    Intermediate

    Quick Description :

    To support the continuously evolving software industry, DevOps and Agile practices have established themselves as a big helping hand. With ever-increasing large scale digital transformation projects, organizations realize that the only way to shorten the product development cycles is to automate the processes involved in the delivery pipeline.

    There are many tools and technologies to establish DevOps, but one of the most important ones is containerization, which gives flexibility in application packaging and automating a number of processes involved in the delivery pipeline. Containers have all the needed configurations and dependencies, for faster delivery of committed items to the customer. Docker is an open-source solution that makes it easier to create, deploy and run applications securely using containers. Therefore, quick feedback and testing applications with different environments can be achieved using Selenium with Docker.

    My Proposal :

    We can reduce the significant effort in provisioning, maintaining, running the Test Automation on different environment setup using, Docker with Test Automation. Some of the benefits are:

    Benefits of using Docker with Selenium:

    1. Test an application on multiple browsers and their different versions without installing them on multiple machines
    2. To test multiple application builds concurrently without updating the code for different environments
    3. To test your applications on multiple devices/OS/Browser combinations on a single machine.As Selenium within Docker will make it much quicker to test different environments by using the preconfigured containers.
    4. Also, I will cover topics like how to set up the Selenium multi-environment setup using Docker concepts like Compose file, etc.

    Best practices using Selenium with Docker

    1. Common Dos and Don'ts for this setup.
    2. How to avoid over maintenance of your Test Harness.
    3. Infrastructure setup

    Increase the power of your Test Automation Setup with Kubernates:

    How to handle Selenium grid-based dockerize nodes with Kubernates.

  • Liked Sameer Arora
    keyboard_arrow_down

    Sameer Arora - Triggering alerts on Slack along with screenshots in case of test case failure

    45 Mins
    Demonstration
    Beginner

    One of the biggest limitations with most of the frameworks in selenium is that we need to wait till the end to get the final test case report. Only then we can raise the alert that a particular functionality is broken. By then, it may be too late! In addition to this, in most of the frameworks, the HTML report generated is hosted on our company's private network and we need to connect to the VPN if we are outside the office to open the report and check the related screenshots to see which test case has failed and why.

    So, why not raise an alert as soon as the test case fails? And why not attach a screenshot along with that alert so that all the stakeholders can actually see what has failed in the application?

    In this session, we will learn a simple yet a really useful way of sending an alert along with the failed test case screenshot by integrating slack with selenium which will help us alert all the stakeholders as soon as an automated test case fails so that everyone can react to it immediately and the damage will be minimised.

  • Liked Amit Rawat
    keyboard_arrow_down

    Amit Rawat - Is Puppeteer better than Selenium

    45 Mins
    Demonstration
    Intermediate

    Puppeteer is a Node js library (developed by Google Chrome team) to control Chrome and Firefox and is getting lot of traction recently because of its amazing capabilities. It has already become so popular that it has got 50K+ stars on Github against Selenium's 15K+ stars.

    In the last Google I/O event, this tool's capability has been showcased and it has been perceived as the next generation Web Test Automation Tool.

    Is Puppeteer better than Selenium? The answer is 'No', and I will cover 'why' in detail during this talk. I will show some live examples to demonstrate that Selenium can also do all those advance things which Puppeteer promises to do.

  • Liked Abhijeet Vaikar
    keyboard_arrow_down

    Abhijeet Vaikar - End-end test code as a first class citizen

    45 Mins
    Case Study
    Intermediate

    "All tests in today's automated regression run have been marked as Untested. What happened?"

    "No notifications are being sent for test runs on the channel"

    "I pulled latest code, and the framework dependency shows compilation error"

    "What does this new method in the framework do?"

    How often do you hear such things within your team?

    As Quality champions, we need to walk the talk. When we expect our developers to write quality code, write unit tests, build features without introducing bugs, the onus lies on us (as test engineers) to do the same. With almost every test engineering team writing automated tests to check functionality of their products and services, it becomes very important to ensure that the test automation framework and the test scripts are bug-free and follow good standards of software engineering.

    It cannot be stressed enough that test automation code should be as good as production code. In order to build production-quality test automation framework and scripts, a number of steps can be taken at:

    1. Code & System Level

    2. Process & People Level

    Our test engineering team went through a transition from having random & unexpected failing test runs to having greater confidence in the quality of the tests. Learn from this case study of our journey to ensure that end-end UI automated tests are built with quality in mind. We will also see demonstration of some of the use cases.

  • Liked Artem Sokovets
    keyboard_arrow_down

    Artem Sokovets - Motion Up! Next level of automation testing

    45 Mins
    Case Study
    Intermediate

    In the test automation industry, the Page Object pattern has been used for more than 8 years. During this time, Java has been updated from JDK7 to JDK13. It's time to update the Page Object Pattern. What do you think about what next POM representation ?

    Do you really know what application coverage brings your UI tests? - Not, I will show you how we can calculate it. Yes, one way is using the JoCoCo plugin, but there are other ways that exist.

    What about the test report tools? Most of your know Allure report, but do you hear about Allure Server?

    If you have an interest in the above themes come to my talk.

  • Liked Ivan Krutov
    keyboard_arrow_down

    Ivan Krutov - Workshop: Efficient Selenium Infrastructure with Selenoid

    Ivan Krutov
    Ivan Krutov
    Developer
    Aerokube
    schedule 1 week ago
    Sold Out!
    480 Mins
    Workshop
    Intermediate

    Selenoid is an alternative lightning fast open-source Selenium protocol implementation running browsers and Android emulators inside Docker containers. It is distributed with a set of ready-to-use Docker images corresponding to the majority of popular browser versions, has a one-command installation utility and works slightly more efficiently than traditional Selenium Grid.

    This tutorial shows how to build efficient and scalable browser automation infrastructure using Selenoid and related tools. You will be taught why running browsers in containers is so efficient, how to easily install Selenoid and use its powerful features.

  • Liked Benjamin Bischoff
    keyboard_arrow_down

    Benjamin Bischoff - Simplifying your test runs with ‘make’

    20 Mins
    Case Study
    Intermediate

    The ‘make’ command has been around since 1976 and was originally used to build executable programs. In this talk I want to show how this powerful tool can greatly reduce complexity and eliminate stumbling blocks of running your automated tests. This applies to both CI pipelines and local test runs on developer or QA machines.

  • Liked Babu Narayanan Manickam
    keyboard_arrow_down

    Babu Narayanan Manickam - Deep Learning Based Selenium Test Failure-triage Classification Systems

    20 Mins
    Talk
    Intermediate

    Problem Statement:

    While running thousands of automated test scripts on every nightly test schedule, we see a mixed test result of pass and failures. The problem begins when there is a heap of failed tests, we caught in the test-automation trap: unable to complete the test-failure triage from a preceding automated test run before the next testable build was released.

    Deep Learning Model:

    The classification was achieved by introducing Machine Learning in the initial months and followed by Deep Learning algorithms into our Selenium, Appium automation tests. Our major classification was based on the failed test cases: Script, Data, Environment, and Application Under Test and that internally had hundreds of sub-classifications.

    To overcome this problem, we started to build and train an AI using Deep Learning, which simulates a human to categorize the test case result suite. Based on the test result failure for each test, the AI model would predict an outcome through API, categorizes and prioritize on the scale of 0 to 1. Based on the prediction model, the algorithm takes appropriate response actions on those test cases that are failed like re-run test again or run for different capabilities. We kick-started this by gathering the historical data set of 1.6 million records, which was collected over a 12 months period, data including the behavior of the test case execution and the resulting suite.

    This Deep Learning-based algorithm has been provided the quality to break down the new defects based on its category, and a classification score is given on a scale of 0-1. We’ve also established a cutoff threshold based on its accuracy of improving, and to group the failed test cases based on its similarity. Classification of the test cases is done in high granularity for sophisticated analysis, and our statistical report states that the classification of the defects has been increased with 87% accuracy over a year. The system has been built based on the feedback adapting models, where for each right classification it will be rewarded and for the wrong, a penalty is given. So whenever receiving a penalty the system will automatically enhance itself for the next execution.

    The algorithm has a powerful model for detecting false-positive test results calculated using the snapshot comparisons, test steps count, script execution time and the log messages. Also, the model has been built with other features like – duplicate failure detection, re-try algorithms and defect logging API, etc.

    The entire classification system has been packaged and deployed in the cloud where it can be used as a REST service. The application has been built with its own reinforcement learning where it uses the classification score to enhance itself and this is programmed to perform in an inconclusive range.

    In sum, this deep learning solution can assist all Selenium testers to classify their test results in no-time and can assist to take next steps automatically and allow team could focus its efforts on new test failures.

    Link: https://github.com/testleaf-software/reinforced-selenium-test-execution

  • Liked Gaurav Singh
    keyboard_arrow_down

    Gaurav Singh - How to build an automation framework with selenium : patterns and practices

    Gaurav Singh
    Gaurav Singh
    Product Engineer
    Go-Jek
    schedule 1 month ago
    Sold Out!
    45 Mins
    Talk
    Beginner

    With an ever increasing no of businesses being conducted on web the testing need to write automated tests for the app's UI is something that can never be ignored. As you all know Selenium provides an API that enables us to do this quite effectively.

    However, when tasked with setting up the automation framework, there are a lot of questions that arise in the minds of aspiring test developers regardless of what level they are in their career.


    Some of such questions are:

    1. How does one actually go about the business of building a robust and effective automation framework on top of selenium?
    2. What are the elementary building blocks to include in the framework that an aspiring automation developer should know of?
    3. How should we model our tests? XUnit style vs BDD?
    4. Are their good practices, sensible design patterns and abstractions that we can follow in our code?
    5. What are some of the anti patterns/common mistakes we should avoid

    A lot of literature, documentation and blogs exists on these topics on the web already.

    However In this talk,

    I would combine this existing knowledge and my years of experience in building automation frameworks and breakdown these elements and walk you through exactly the sort of decisions/considerations and practices that you can take while starting to implement or improve the UI automation for your team.

    Hope to see you there!

  • Liked Martin Schneider
    keyboard_arrow_down

    Martin Schneider / Prabhagharan D K - Building and scaling a virtual Android and iOS device lab

    45 Mins
    Case Study
    Intermediate

    Virtual mobile devices (emulators/simulators) are a cost-effective and straightforward alternative to testing on physical devices. We showcase how to set-up and scale an Android emulator farm using Appium, Docker and SQS and how it fits into our larger testing and quality strategy.
    Maintaining physical test devices for mobile automation can be expensive and time-consuming. On top of the initial investment, you need to consider maintenance cost, replacement devices and efforts for manual scaling. On the other side of the spectrum, cloud providers take care of these restrictions, but their services can come at a hefty price tag, especially when your use-case requires a large number of devices. We present a middle path and demonstrate how to use virtual devices to build a reliable and scalable in-house device lab using Docker and Appium.

  • Liked Denys Pavlenko
    keyboard_arrow_down

    Denys Pavlenko - Integrating Selenium tests into GitLab CI

    Denys Pavlenko
    Denys Pavlenko
    QA EM
    Personio GmbH (Munich)
    schedule 1 week ago
    Sold Out!
    45 Mins
    Demonstration
    Intermediate

    GitLab is the first single application for the entire DevOps cycle. Only GitLab enables Concurrent DevOps, unlocking organizations from the constraints of the toolchain. GitLab provides unmatched visibility, higher levels of efficiency, and comprehensive governance. This makes the software lifecycle 200% faster, radically improving the speed of business.

    Taking into account the fast growth of GitLab in recent years and it’s prominent advantages, you might be required to start running Selenium tests in GitLab or even migrate your end2end tests from your previous CI tool to GitLab.

    Step by step guide will provide you the main challenges, key configurations, pitfalls in order to integrate and successfully run Selenium tests in GitLab CI. The wrapper around the Selenium to be presented is going to be Protractor with a built-in Jasmine test runner.

    The presentation will also showcase some main tips and tricks we’ve faced during this integration at our fast-growing startup in Munich, Germany.

  • Liked Srinivasu gangam
    keyboard_arrow_down

    Srinivasu gangam - Zero Touch Automation using NLP (Natural language processing) & AI

    Srinivasu gangam
    Srinivasu gangam
    Sr Manager
    Cotiviti India
    schedule 1 month ago
    Sold Out!
    45 Mins
    Demonstration
    Intermediate

    Problem Statement:

    As part of SDLC process:

    1. Is your product quality impacted due to a smaller number of QA resources available in the team?
    2. Are you waiting for QA resources to certify your code every time when you deploy? Is this impacting your product lead time (Speed to Market)?
    3. Is your Product delivery timelines are impacted due to last minute defects identified?
    4. Do you have your QA resources only in one location, but you want to “follow-the-sun” approach for Software delivery across multiple locations?
    5. Do you have manual testers who are not skilled in programming, but you want them to execute automated test scripts w/o any training efforts and automation setup?
    6. Would you like your team more agile and cross functional with Delivery?
    7. Would you like to increase your QA team’s productivity while they invest more time in script development rather than script execution?

    If answer is ‘Yes’ for above questions, "Zero touch automation" is the solution for above challenges that we have been facing part of SDLC.

    Solution: Zero touch automation with cutting-edge technologies

    In this session, I will cover how we solved this problem using innovative solutions, Cutting-edge technologies like NLP (Natural language processing), AI & Cloud solutions.

    You will learn how AI, NLP integrated with core automation components to achieve Zero touch automation.

    This solution is not just revolutionary, it is paradigm shift in test automation to get results to your email with detailed analysis of failure categorization with recommended actions to users.

    I will also cover how E2E automation will be driven with decisions taken by machines based on what user is looking for . There is no manual intervention in this process. NLP and AI play key role to help machines to take decisions.

    We will also cover how we empowered developer/release manager/any team member/Manager to trigger the scripts from their cell phone and get the detailed execution report without having any automation software installed in their computer or Phone.

    We will be demonstrating how the request will be initiated from User, understand the need from user using NLP & AI , Fetching the code from bitbucket to select appropriate automation scripts , running them on Selenoid/docker server , storing results to MongoDB , receiving email with test results and Failure analysis.

    What is the value of zero touch automation?

    1. Enable speed to market: Now that Developers does not need to wait for QA resource, Changes can be certified quickly and ready to push to production. Lead time will be significantly reduced.
    2. Increase quality: Now that test automation is easy and it can run multiple times in each environment, most of the defects will be uncovered and addressed before code goes to production.
    3. Ease of test execution: Test execution will be very easy, no automation or framework setup required from user side. Test execution can be done 24*7.
    4. Productivity: Increase QA team’s Productivity to focus more on script development rather than focusing on script execution and failure analysis .
  • Liked Syam Sasi
    keyboard_arrow_down

    Syam Sasi - When ansible meets selenium grid - Story of building a stable local iOS simulator farm

    Syam Sasi
    Syam Sasi
    Quality Engineer
    Carousell
    schedule 3 weeks ago
    Sold Out!
    45 Mins
    Case Study
    Beginner

    Running parallel automation tests for iOS apps is always tricky since iOS simulators need Apple hardware. There are many docker based solutions available for building a local device lab, but fails when it come into the perspective of apple iOS simulators.

    The mighty selenium grid is a good is a choice to the above problem, but configuring the grid and nodes is a tedious task if you want to scale up the infrastructure .

    In this talk , I will explain

    • How Ansible scripts helped to set up the setup the selenium grid and node configuration in 60 seconds.
    • How to customise selenium grid to make it more stable.

  • Liked Arun Prakash
    keyboard_arrow_down

    Arun Prakash / Oleg Shuliak - Scalable & Lightning fast Test Infrastructure in AWS

    45 Mins
    Demonstration
    Intermediate

    Quality Assurance is an important term in the lifecycle of a software product. When it comes to QA, there are a lot of layers of testing which includes but not limited to Unit testing, Integration testing, UI testing. If we consider the test pyramid, although UI testing forms only a small percentage of the tests, it's an important factor in deciding the quality of the product. One of the main pain points in having a UI test is the execution time. To overcome this, we now have the tests that are executed in parallel. While designing a test infrastructure for executing the UI tests in parallel, there are a lot of factors that need to be considered. Some of them are

    • Scalability
    • Reliability
    • Resource constraints

    Taking into account the different tools which are available now in the market, one of the most efficient tool which satisfies the above criteria is "Selenoid"

    Selenoid is a powerful Go implementation of original Selenium hub code. It is using Docker to launch browsers. One of the main advantages of selenoid is that its' lightning fast. Selenoid launches a new container for each new browser request and stops it after the session is closed. A certain browser version, the required version of the web driver which supports this browser and all necessary dependencies form each container. Selenoid also support most of the popular browsers such as Chrome, firefox etc.

    This presentation will be to show case how to create a scalable selenoid infrastructure in AWS cloud with the help of lambda function and kubernetes and this is being used at our fast growing startup based on Munich, Germany. We will also show case the comparison between different tools that are available in the market.

  • Liked Nishil Patel
    keyboard_arrow_down

    Nishil Patel - Tesbo : An Awesome Test Automation Framework with Plain English Text But not BDD

    Nishil Patel
    Nishil Patel
    Founder
    QAble
    schedule 2 weeks ago
    Sold Out!
    45 Mins
    Demonstration
    Beginner

    Tesbo is a hybrid Test Framework built top on the Selenium which allows you to use the plain English language to define the test which helps you to reduce the development and maintenance time and efforts. sounds similar to the BDD framework (i.e. Cucumber, Specflow, Gauge etc...) ??? But No It Isn't... It's much better than Cucumber.

    In this session, we will look into why we have developed Tesbo and how it will reduce the development and maintenance times of web and mobile test script.

    This session will focus on the

    • Introduction of Tesbo
    • Advantages of Tesbo
    • Feature description
    • Test Development and Execution demo
    • Future plans for Tesbo
  • Liked Rahul Yadav
    keyboard_arrow_down

    Rahul Yadav - Resilience Testing - How Chaos Engineering is helping in building resilient systems

    Rahul Yadav
    Rahul Yadav
    Senior consultant
    Apps associates
    schedule 1 week ago
    Sold Out!
    20 Mins
    Talk
    Beginner

    With the amount of digital transactions happening day by day, businesses cannot afford to have downtimes. Imagine being presented with an error screen or a message saying - System is getting upgraded/ System failed when browsing a video on any OTT.

    In order to have services up 24X7 we need to know the failure points of our systems and how to mitigate it gracefully. Chaos Engineering and Testing helps us identify that and allows businesses to built scalable, fault tolerant and resilient systems .

  • Liked Shi Ling Tai
    keyboard_arrow_down

    Shi Ling Tai - Start with the scariest code

    Shi Ling Tai
    Shi Ling Tai
    CEO
    UI-licious
    schedule 3 weeks ago
    Sold Out!
    20 Mins
    Talk
    Beginner

    "We should start from unit tests"

    "No, integration tests are better!"

    "Should we use tool A or tool B?"

    I see this debate play out all the time, and I've been there before.

    A lot of teams get lost trying to figure out where and how to get started with testing. It is really intimidating because there's so much to test, and so many ways to do testing. The worst that could happen is inaction from the decision paralysis.

    My rule of thumb is "Start with the scariest code". Let me share with you my process for prioritising what to test and figuring out how to test it.