Problem Statement

Though Selenium taking most of the stake in the UI test automation tool market comfortably, there were always challenges that were for selenium test automation engineers are handicapped with especially when dealing with modern JS technologies. For example,

  • dealing with DOM elements to solve the stale / loading / non-interactable elements,
  • handling full screenshots to know how the elements at the left, bottom, etc,
  • measuring the performance of request and response resources at different speeds,
  • monitoring the memory of the pages, controls, etc,
  • attaching to an existing browser for debugging the failed scripts and many more.

These problems were largely resolved with the integration of selenium with devtool protocols. And that makes the selenium engineer's life merrier than before.

Why this proposal can be different from others?

  1. Our solution can be executed as independent with chrome dev tools or with selenium. That gives the power back to the automation engineer to choose what and how to debug/run their tests.
  2. We used this solution for one of our largest enterprise customers and moved this solution to public repository this week (for this conference and beyond). With that said, we tested reasonably with more than 10,000+ test scripts and more than 1M tests.
  3. The present solution is completely (100%) packed with all Chrome Dev Tools API in Java and with that said, any Java Selenium automation engineer can bind in minutes for their existing code base with no additional dependencies.
  4. Finally, we love to present at the local home to start our selenium conference campaign. Where else?


The present proposal largely connected with Chrome and Selenium in Java language. However, there is no limitation to expand the boundaries for other language bindings and browsers.

Google Chrome, the most picked browser for browsing, which makes it the primary concentration for developers and testers. DevTools is one such boon for developers, testers especially the new aged test automation engineers. With that said, we built the following design pattern to allow chrome dev tools API to marry Selenium using debugger address / remote targets.

Selenium Devtools


Outline/Structure of the Demonstration

  • Introduction to Devtools (5 Minutes)
  • Deep dive towards Chrome Devtools Protocols (5 Minutes)
  • Selenium Integration with Chrome Devtools (with Selenium code base and alternative) (5 Minutes)
  • Integration solutions of Selenium with Chrome Devtools: (20 Minutes)
    • DOM changes detected using Event Listeners and how that solves stale element, not clickable element exceptions
    • Layer changes listeners that solves click interception exceptions
    • Highlighting elements to allow better debugging
    • Full snapshots to perform quick post-test reviews
    • Network monitors that measure performance, bottlenecks
    • Every chrome driver actions in the browser through the element monitoring controls
    • and many more
  • Our present challenges, the roadmap to next version (5 Minutes)
  • Q & A (5 Minutes)

Learning Outcome

Integrate Devtools and Implement in your existing Selenium Java tests in no time.

Target Audience

All Selenium automation engineers, managers and others

Prerequisites for Attendees


schedule Submitted 7 months ago

Public Feedback

comment Suggest improvements to the Speaker
  • Sai Krishna
    By Sai Krishna  ~  6 months ago
    reply Reply

    Hi Gopinath and Babu,

    Thanks for submitting a proposal on CDP. definitely an interesting one. 
    Can we please see a video of you talking?


    • Anand Bagmar
      By Anand Bagmar  ~  6 months ago
      reply Reply

      Hi Gopinath and Babu,

      As requested by Sai, can we please see a video of both of you talking? Also, will you be sharing this via slides or demo or a combination? If slides / combination, can you please share some reference slides as well?

      Also, I see you have added reference to a recently created github repo ( Is this the Selenium - DevTools integration that you have created? If yes, can you also add some sample tests to the repo for easier understanding and usage?

      Regards, Anand

  • Maaret Pyhajarvi
    By Maaret Pyhajarvi  ~  6 months ago
    reply Reply

    What makes this a two person talk? My experience has been that talks with two people need a special connection between the two presenters so that the story is not complete without one of the pair. Is there such a connection and how could you make it visible in the proposal?

    • Gopinath Jayakumar
      By Gopinath Jayakumar  ~  6 months ago
      reply Reply

      Your are right, Maaret Pyhajarvi. Myself and Babu are working together at TestLeaf for 10 years and have been co-creating and co-presenting many open-source projects together. 

      In this context of Conference paper, Myself and Babu have built more than 800 java wrapper classes specific to Chrome DevTools API that can also integrate with Selenium. 

      With that said, when presented in stage together, it gives the comfort and completion to deliver the maximum experience on DevTools integration.

      If you need additional details, we are happy to provide.

      Thanks for asking and looking forward to the Conference.

  • Liked Babu Narayanan Manickam

    Babu Narayanan Manickam - Deep Learning Based Selenium Test Failure-triage Classification Systems

    45 Mins

    Problem Statement:

    While running thousands of automated test scripts on every nightly test schedule, we see a mixed test result of pass and failures. The problem begins when there is a heap of failed tests, we caught in the test-automation trap: unable to complete the test-failure triage from a preceding automated test run before the next testable build was released.

    Deep Learning Model:

    The classification was achieved by introducing Machine Learning in the initial months and followed by Deep Learning algorithms into our Selenium, Appium automation tests. Our major classification was based on the failed test cases: Script, Data, Environment, and Application Under Test and that internally had hundreds of sub-classifications.

    To overcome this problem, we started to build and train an AI using Deep Learning, which simulates a human to categorize the test case result suite. Based on the test result failure for each test, the AI model would predict an outcome through API, categorizes and prioritize on the scale of 0 to 1. Based on the prediction model, the algorithm takes appropriate response actions on those test cases that are failed like re-run test again or run for different capabilities. We kick-started this by gathering the historical data set of 1.6 million records, which was collected over a 12 months period, data including the behavior of the test case execution and the resulting suite.

    This Deep Learning-based algorithm has been provided the quality to break down the new defects based on its category, and a classification score is given on a scale of 0-1. We’ve also established a cutoff threshold based on its accuracy of improving, and to group the failed test cases based on its similarity. Classification of the test cases is done in high granularity for sophisticated analysis, and our statistical report states that the classification of the defects has been increased with 87% accuracy over a year. The system has been built based on the feedback adapting models, where for each right classification it will be rewarded and for the wrong, a penalty is given. So whenever receiving a penalty the system will automatically enhance itself for the next execution.

    The algorithm has a powerful model for detecting false-positive test results calculated using the snapshot comparisons, test steps count, script execution time and the log messages. Also, the model has been built with other features like – duplicate failure detection, re-try algorithms and defect logging API, etc.

    The entire classification system has been packaged and deployed in the cloud where it can be used as a REST service. The application has been built with its own reinforcement learning where it uses the classification score to enhance itself and this is programmed to perform in an inconclusive range.

    In sum, this deep learning solution can assist all Selenium testers to classify their test results in no-time and can assist to take next steps automatically and allow team could focus its efforts on new test failures.