location_city Virtual Platform schedule Sep 11th 12:30 - 01:15 PM place Online Meeting 2 people 56 Interested

Problem Statement

Though Selenium taking most of the stake in the UI test automation tool market comfortably, there were always challenges that were for selenium test automation engineers are handicapped with especially when dealing with modern JS technologies. For example,

  • dealing with DOM elements to solve the stale / loading / non-interactable elements,
  • handling full screenshots to know how the elements at the left, bottom, etc,
  • measuring the performance of request and response resources at different speeds,
  • monitoring the memory of the pages, controls, etc,
  • attaching to an existing browser for debugging the failed scripts and many more.

These problems were largely resolved with the integration of selenium with devtool protocols. And that makes the selenium engineer's life merrier than before.

Why this proposal can be different from others?

  1. Our solution can be executed as independent with chrome dev tools or with selenium. That gives the power back to the automation engineer to choose what and how to debug/run their tests.
  2. We used this solution for one of our largest enterprise customers and moved this solution to public repository this week (for this conference and beyond). With that said, we tested reasonably with more than 10,000+ test scripts and more than 1M tests.
  3. The present solution is completely (100%) packed with all Chrome Dev Tools API in Java and with that said, any Java Selenium automation engineer can bind in minutes for their existing code base with no additional dependencies.
  4. Finally, we love to present at the local home to start our selenium conference campaign. Where else?


The present proposal largely connected with Chrome and Selenium in Java language. However, there is no limitation to expand the boundaries for other language bindings and browsers.

Google Chrome, the most picked browser for browsing, which makes it the primary concentration for developers and testers. DevTools is one such boon for developers, testers especially the new aged test automation engineers. With that said, we built the following design pattern to allow chrome dev tools API to marry Selenium using debugger address / remote targets.

Selenium Devtools


Outline/Structure of the Demonstration

  • Introduction to Devtools (5 Minutes)
  • Deep dive towards Chrome Devtools Protocols (5 Minutes)
  • Selenium Integration with Chrome Devtools (with Selenium code base and alternative) (5 Minutes)
  • Integration solutions of Selenium with Chrome Devtools: (20 Minutes)
    • DOM changes detected using Event Listeners and how that solves stale element, not clickable element exceptions
    • Layer changes listeners that solves click interception exceptions
    • Highlighting elements to allow better debugging
    • Full snapshots to perform quick post-test reviews
    • Network monitors that measure performance, bottlenecks
    • Every chrome driver actions in the browser through the element monitoring controls
    • and many more
  • Our present challenges, the roadmap to next version (5 Minutes)
  • Q & A (5 Minutes)

Learning Outcome

Integrate Devtools and Implement in your existing Selenium Java tests in no time.

Target Audience

All Selenium automation engineers, managers and others

Prerequisites for Attendees




schedule Submitted 1 year ago

Public Feedback

    • Simon Stewart

      Simon Stewart - Selenium: State of the Union

      45 Mins

      What's the latest in the world of Selenium since the last SeleniumConf? In this talk, we'll cover Selenium 4, how the project is run, how you can participate, reminiscing on the Old Days, and reflect on something that at first doesn't appear to be related to Selenium, yet is intimately related to it.

    • Virender Singh

      Virender Singh - Rise of Shadow DOM - Lets solve it through WebDriver

      45 Mins

      Shadow DOM is kind of web standard that developers use to encapsulate their custom HTML code and style components so that no other global style components can override their code. It ensures that a component will work in any environment, even if other CSS or JavaScript is run on the page i.e. Shadow DOM hides away the complexity of the components from the rest of the page because it renders separately from main DOM.

      Selenium legacy selectors are not able to interact these custom elements under the shadow DOM so we need java script to interact with shadow DOM elements and we also require JavascriptExecutor interface for their execution.

      We will also talk about the WebdriverIO tool v5.5.0, which implicitly support Shadow DOM elements.

    • Gaurav Singh

      Gaurav Singh - How to build an automation framework with Selenium : Patterns and practices

      Gaurav Singh
      Gaurav Singh
      Test automation Lead
      schedule 1 year ago
      Sold Out!
      45 Mins

      With an ever increasing no of businesses being conducted on web the testing need to write automated tests for the app's UI is something that can never be ignored. As you all know Selenium provides an API that enables us to do this quite effectively.

      However, when tasked with setting up the automation framework, there are a lot of questions that arise in the minds of aspiring test developers regardless of what level they are in their career.

      Some of such questions are:

      1. How does one actually go about the business of building a robust and effective automation framework on top of selenium?
      2. What are the elementary building blocks to include in the framework that an aspiring automation developer should know of?
      3. How should we model our tests? XUnit style vs BDD?
      4. Are there good practices, sensible design patterns and abstractions that we can follow in our code?
      5. What are some of the anti patterns/common mistakes we should avoid

      A lot of literature, documentation and blogs exists on these topics on the web already.

      However In this talk,

      I would combine this existing knowledge and my years of experience in building automation frameworks and breakdown these elements and walk you through exactly the sort of decisions/considerations and practices that you can take while starting to implement or improve the UI automation for your team.

      Hope to see you there!

    • Tomasz Konieczny

      Tomasz Konieczny - Serverless - how to speed up tests over 300 times and achieve continuous feedback?

      Tomasz Konieczny
      Tomasz Konieczny
      Senior QA Engineer
      schedule 1 year ago
      Sold Out!
      45 Mins

      Automated tests can provide results faster and it’s possible to execute them more frequently than manual ones. That allows to test earlier in the development process, decrease overall time needed for tests and what is probably the most important it’s possible to release and deliver business value faster and more frequently.

      But what if we have more and more tests and even automated execution of them takes too much time - 10 minutes... 30 minutes... maybe even hours? Should we consider the ability to execute full tests set just a few times a day as something normal? Is adding more compute resources the only option to reduce the execution time? Or maybe there are too many high-level tests and some of them should be replaced by low-level ones according to tests pyramid? Is the tests pyramid still valid in the cloud world?

      During the presentation you will see how the serverless cloud services like AWS Lambda may be used to run tests in the highly parallelized environment that can speed up execution even hundreds of times.

    • Babu Narayanan Manickam

      Babu Narayanan Manickam - Deep Learning Based Selenium Test Failure-triage Classification Systems

      45 Mins

      Problem Statement:

      While running thousands of automated test scripts on every nightly test schedule, we see a mixed test result of pass and failures. The problem begins when there is a heap of failed tests, we caught in the test-automation trap: unable to complete the test-failure triage from a preceding automated test run before the next testable build was released.

      Deep Learning Model:

      The classification was achieved by introducing Machine Learning in the initial months and followed by Deep Learning algorithms into our Selenium, Appium automation tests. Our major classification was based on the failed test cases: Script, Data, Environment, and Application Under Test and that internally had hundreds of sub-classifications.

      To overcome this problem, we started to build and train an AI using Deep Learning, which simulates a human to categorize the test case result suite. Based on the test result failure for each test, the AI model would predict an outcome through API, categorizes and prioritize on the scale of 0 to 1. Based on the prediction model, the algorithm takes appropriate response actions on those test cases that are failed like re-run test again or run for different capabilities. We kick-started this by gathering the historical data set of 1.6 million records, which was collected over a 12 months period, data including the behavior of the test case execution and the resulting suite.

      This Deep Learning-based algorithm has been provided the quality to break down the new defects based on its category, and a classification score is given on a scale of 0-1. We’ve also established a cutoff threshold based on its accuracy of improving, and to group the failed test cases based on its similarity. Classification of the test cases is done in high granularity for sophisticated analysis, and our statistical report states that the classification of the defects has been increased with 87% accuracy over a year. The system has been built based on the feedback adapting models, where for each right classification it will be rewarded and for the wrong, a penalty is given. So whenever receiving a penalty the system will automatically enhance itself for the next execution.

      The algorithm has a powerful model for detecting false-positive test results calculated using the snapshot comparisons, test steps count, script execution time and the log messages. Also, the model has been built with other features like – duplicate failure detection, re-try algorithms and defect logging API, etc.

      The entire classification system has been packaged and deployed in the cloud where it can be used as a REST service. The application has been built with its own reinforcement learning where it uses the classification score to enhance itself and this is programmed to perform in an inconclusive range.

      In sum, this deep learning solution can assist all Selenium testers to classify their test results in no-time and can assist to take next steps automatically and allow team could focus its efforts on new test failures.

      Link: https://github.com/testleaf-software/reinforced-selenium-test-execution

    • Gopinath Jayakumar

      Gopinath Jayakumar / Babu Narayanan Manickam - Selenium Internals in Python Programming - Workshop

      90 Mins

      Are you keen to know Selenium 4.x(experimental) deeper with internal architecture and workings with its dearest drivers and its API in Python programming? Are you in Selenium with other language bindings and willing to cross-skill in Python to make up for the deep learning/machine learning/data science projects?

      If one of the above is yes, this workshop will get your hands dirty with practical exposure with several webdriver commands in modern applications and of course, the course is designed to assist all levels of test automation engineers experience.

      In sum, all attendees of this workshop will sign off with a working selenium codebase in python with git repo and best practices to deploy using pytest framework on their projects.

    • Babu Narayanan Manickam

      Babu Narayanan Manickam / Sarath Muthuswamy - Reinforced Selenium test selection using Recommender Systems

      20 Mins
      Case Study

      Problem Statement

      One of our enterprise oracle customers has more than 30,000+ UI & API regression tests for end to end Apps Integration testing. The average time it takes to run these tests (depends on parallelization) is anywhere between 11-12 hours.

      With considerable time to complete all regression tests, the objective was to find a working AI model to identify the probable failing test scripts and prioritize them early along with the most critical business automated tests.


      Selecting the most promising selenium test scripts to detect application defects may be harder considering the uncertainties on the impact of committed code changes and breakage of many traceability links between code and automated tests. The design was to automatically choose the test scripts and prioritize in CI tool with the goal to minimize round-trip-times between code commits and developer feedback on failed test scripts.

      In our devops environment, where new test scripts are created and obsolete tests are deleted constantly, the reinforced method learns to prioritize error-prone test scripts higher under the guidance of a reward function and by observing previous test cycles from historical data. By applying reinforced learning techniques on data extracted, it is evident that reinforcement learning enables better automatic adaptive test script selection and prioritization in CI and automated regression testing.

      As the first step, we created a predictive model that estimates the probability of each test failing for a newly proposed build. Instead of defining the model manually, we built it by using a large data set containing results of tests on historical builds/releases and then applying the standard machine learning technique - gradient-boosted decision tree.


      • Code changes based on build metadata
      • Code Owner Information
      • Historical test runs

      With this model, we can analyze a particular code change to find all potentially impacted tests that transitively depend on modified files, and then estimate the probability of that test detecting a regression introduced by the change. Based on those estimates, the system selects the tests that are most likely to fail for a particular change. The diagram below shows which tests (shown in blue) would be chosen for a change affecting the two files from the previous example, where the likelihood of each considered is represented by a number between zero and one.

      In sum, Our recommender system with reinforcement learning algorithms helps to identify the priority test scripts in runtime, the model helps to move up or down the test scripts from the sequences set at the beginning.

      Our github repo: https://github.com/testleaf-software/reinforced-selenium-test-execution

      Process Diagram: https://github.com/testleaf-software/reinforced-selenium-test-execution/blob/master/re-inforcement%20process.png