Why Building Record/Playback Tools Is So Hard

location_city Portland, OR schedule Sep 9th 11:35 AM - 12:20 PM IST place Grand Ballroom 1

Almost every manual QA started with Selenium IDE or at least tried it at some point.. retention rate is close to 0.

In this talk we will depict the challenges that Test Automation Developers face. We'll categorise the challenges, name them, and see the skillset required to overcome the difficulties.

We'll compare the different Record/Playback tools and see where they excel, and where we must resort to traditional solutions (e.g. generic programming languages, and the Page Object design).

This talk will discuss topics such as Reuse of code, Component Isolation, Separation of Concerns, APIs, where test fragility stem from and how to overcome it, etc..


Outline/Structure of the Talk

0-2: About and thank you

2-5: Intro

5-20: The challenges of a Test Automation Developer 

20-40: Go over the challenges, and for each one show how to implement it

40-45: Questions

Learning Outcome

For beginners: learn the different challenges of a developer - reuse of code, ease of use, separation of concerns, isolation, minimizing fragility, importance of good locators.

For advanced: the challenges of building a tool in this domain, and specifically, a tool for developers.

Target Audience

Beginners; Advanced developers keen on developing tools for developers



schedule Submitted 7 years ago

  • Jonathan Lipps

    Jonathan Lipps - The Mobile JSON Wire Protocol

    Jonathan Lipps
    Jonathan Lipps
    Project Lead
    schedule 7 years ago
    Sold Out!
    45 Mins

    The JSON Wire Protocol (JSONWP) is the version of the WebDriver spec currently implemented by all the Selenium clients. It defines an HTTP API that models the basic objects of web automation---sessions, elements, etc... The JSON Wire Protocol is the magic that powers Selenium's client/server architecture, enables services like Selenium Grid or Sauce Labs to work, and gives you the ability to write your test scripts in any language.

    The JSONWP has served Selenium faithfully for a number of years, but the future of automated testing lies beyond the borders of the web browser. Mobile automation is an essential ingredient in any build, and tools like Appium or Selendroid have made it possible to run tests against mobile apps using the JSONWP. The JSONWP's current incarnation isn't enough to automate all the new behaviors that mobile apps support, however. Complex gestures, multiple device orientations, airplane mode, and the ability to use both native and web contexts, for example, are all essential to mobile automation.

    For this reason the leaders of the Selenium project, in concert with other Selenium-based projects like Appium and Selendroid, met to discuss the future of the JSONWP. We've been working on its next version, called the "Mobile JSON Wire Protocol" (MJSONWP). Appium and Selendroid already implement much of the MJSONWP spec. In this talk I'll dive into the specifics of the MJSONWP extensions, how they relate to the original JSONWP, and how the Selenium clients have begun to implement them.

    Finally, I will talk about the future of the MJSONWP and how it's related to the current and future versions of the WebDriver spec. I'll share how you can get help with the creation of the MJSONWP, and discuss issues with the authors of the new spec before the API is set in stone. We need the help of everyone who's involved in mobile automation to come up with the best and most future-proof version of the MJSONWP. Ultimately, your understanding of how Selenium works will be improved, and you'll have a much better handle on how projects like Appium and Selenium work together to make sure you have the best automation methods available.

  • Ragavan Ambighananthan

    Ragavan Ambighananthan - Distributed Automation Using Selenium Grid / AWS / Autoscaling

    45 Mins

    Speed of UI automation has always been an issue when it comes to Continuous Integration / Continuous Delivery. If UI automation suite takes 3 hours to complete, then any commit happens during this time will not be visible in test environment, because the next deployment will happen only after 3 hours. 

    With 2000+ developers and average 250+ checkins per day, the above issues is replicated 250+ times every day. This is not productive and feedback cycle is super slow!

    Another issue is , with 35+ different project teams using 10 or more different jenkins jobs to run their UI automation. So many jobs means (350+), individual teams need to go through the pain of managing their own jenkins job, its a duplicate effort and waste of time. Automation teams need to spend time on writing reliable automation and not managing jenkins jobs.

    Solution is to reduce the UI automation run time from hours to minutes and also use only handful of jobs to run the Distributed Automation!

    Goal: To run all UI automation scenarios within the time take by the longest test case

  • Tanay Nagjee

    Tanay Nagjee - Run your Selenium tests in a fraction of the time

    45 Mins

    Comprehensive functional testing is a widely accepted best practice and Selenium is the prevalent tool of choice. But long-running Selenium test suites cannot be fit into short continuous integration (CI) cycles and are often run once a night or less. Running functional tests less frequently means bugs are discovered later than they should. Tanay Nagjee will discuss how Selenium test suites can be parallelized at a very fine granularity and included in CI builds. By leveraging a cluster of compute horsepower (on-premise and/or cloud), large Selenium suites can execute in a fraction of the time by smartly parallelizing the individual tests and running them on individual *cores* (not hosts). Tanay will outline the approach and tools to achieve these results with Selenium, and will present a live demonstration.

  • Dilip S

    Dilip S - TestComplete supports Selenium – Other Commercial tools will follow soon?

    Dilip S
    Dilip S
    Associate Architect
    schedule 7 years ago
    Sold Out!
    45 Mins

    Oliver Wendell Holmes once said: I would not give a fig for the simplicity this side of complexity, but I would give my life for the simplicity on the other side of complexity.

    Tool evangelists around the world have been using this phrase for selling their products. They make it a point to look into customer’s eyes and say “Scalability, you know, that’s the main problem with Selenium. What about 3 years down the line, when you have multiple applications in your landscape and Selenium does not support it?”

    But they simply ignore the fact, knowingly off course, that this ever belting technology world is going where Selenium is right now. Applications are shrinking into browser windows and changing tracks to align itself to this mobile era.

    Even commercial tool like TestComplete from SmartBear has started supporting Selenium and many will follow soon. Reason for this change is not only that most of the organizations are preferring open source tools like Selenium for starting point of their automation activities but also the fact that Selenium by far has proven itself to be one of the best automation tools when it comes to mobile or browser based desktop automation.

    Here our aim is to display how seamlessly Selenium integrates with TestComplete and QAComplete and it is for us to understand that it is not Selenium which needs other tools to extend it but it is the other way round.

  • vishnu nallani chekravarthula

    vishnu nallani chekravarthula - Extending Selenium Element Locator Strategies – Element Filtering

    45 Mins

    Element Locator strategies for Selenium WebDriver are highly flexible, and have been later inherited by many commercial tools. Although the locator strategies are flexible, they are also limited in a sense that, Selenium WebDriver does not currently allow its users to identify/filter UI elements with multiple locator strategies(at a time), as many commercial tools do.

    The solution discussed in this article describes a library that allows Selenium WebDriver users to extend the Selenium element locator strategies for Element Filtering and few use cases for the library.

    The solution approach allows users to continue to use the existing UI Element definitions in their tests, and extend them, using the By reference. The library will replace the existing Selenium WebDriver “By” reference.

    Filtering based on multiple locator strategies

    There are various scenarios where to uniquely identify an UI element, a complex XPath has to be written. However, the element can be identified uniquely using multiple locator strategies for the UI Element. The UI Elements can also be filtered, when there are multiple matches in a page. This is the UI Element recognition mechanism used in many commercial test automation tools.

    The algorithm for filtering UI Elements based on multiple locator strategies is based on priority of locator strategies. The priority of locator strategies when filtering is:

    1. ID
    2. Name
    3. TagName
    4. ClassName
    5. XPATH
    6. LinkText and PartialLinkText
    7. CSS

    The By.elementFilter method takes multiple locator strategies, and searches the page for elements matching a particular locator strategy/property, and checks if it is a unique match on the page, if not then it uses the next locator strategy passed to it and so on.

    This method is also very helpful when the application undergoes constant changes and UI Elements might have either of XPATH, ID , NAME, TagName, ClassName etc still unchanged. That way, it helps reduce a lot of maintenance effort in Selenium WebDriver implementations which is due to UI element changes.

    Filtering based on Index

    When there are multiple similar UI Elements in a page, such as cells in a grid/table, it makes sense to identify objects based on their Index based on their appearance on the web page.

    The By.indexFilter method allows users to define an UI Element based on its Index of occurrence of the UI Element. The Index starts from 1.

    Filtering based on relative element

    When a UI element cannot be identified uniquely and reliably by any of its properties, but has some elements in its hierarchy or relative to a particular element, this method can be used to identify the element

    The By.relationFilter method allows users to define an UI Element in relation to another element. The relation can be defined as “Left”, ”Right”, ”Top”, ”Bottom”, ”Child”, ”Parent”

    Filtering for Tables

    When dealing specifically with Tables, which have the

    html tag, the By.tableFilter method allows the user to quickly identify specific cells in the table, without having to write complex XPaths or logics to achieve the same.

    The By.tableFilter method allows users to define a cell in the table with Row,Column numbers. This allows users to directly use the UI Element in their code instead of writing their logic each time. This also increases efficiency and readability of the code.

  • Dipesh Bhatewara
    Dipesh Bhatewara
    Staff Engineer
    VMware Software
    schedule 7 years ago
    Sold Out!
    45 Mins

    In modern cloud aeon, customers are delivered with UI and API for most of the Enterprise Products. The QE teams have to test the product for UI and APIs from functional as well as non functional perspective. For these they typically end up writing different test suites.

    There are lot of challenges one faces when automating test suites for these different purposes, like

    1. How do you integrate UI automation seamlessly with different test frameworks/suites?
    2. How do you test all these requirements (UI, API, Performance etc.) effectively without a lot of redundant or duplicate code?
    3. How do you use the best fit language/technology for UI automation and still do not impact the grand test automation strategies?

    There is an interesting solution to conquer these challenges which I have recently implemented in one of my projects. 

    The idea is to segregate tests from the UI Automation OR webdriver code. This code can be exposed as a web service APIs. Such service allows us to write common tests for UI as well as API. Tests written for API testing can be reused for UI testing with very minimal configuration modification like pointing to appropriate web service. These UI Automation APIs can also be called easily from any performance test requiring UI interaction.

    This approach will enable any test suite using any language or technology in your organization to reuse UI automation seamlessly. This will also make lot of things straight and simple for execution and management across the verticals.


    We will deep dive into the technical solution for this in this talk.