Leveraging Artificial Intelligence to create self-healing tests

A key requirement for successful test automation is to get past the brittle or fragile nature of test scripts. Any Selenium (or Appium) developer has encountered the dreaded "NoSuchElement Exception". A locator that worked yesterday may fail today. What's a test engineer to do?

Fortunately the field of AI provides promising solutions and allows for the creation of self-healing tests. Tests that can find elements across all environments. Tests that can learn from "human-in-the-loop" intervention and work perfectly thereafter. Imagine automated tests that "just work"!

This session will look at how to apply the latest in AI and Machine Learning technologies to improve your test scripts. With the plethora of new open source AI libraries made available by companies such as Google, the ability to leverage AI in your applications is more accessible than ever.

This session will be a primer on AI technologies and how they can be utilized for perfect test automation.

 
 

Outline/Structure of the Tutorial

  • Introduction to AI
    • Why is AI promising in the field of Testing?
    • Using AI to make test cases more robust
  • How to apply ML libraries to solve common test case problems
  • How self-learning allows test cases to improve over time
  • A review of the major AI tools in the market today that you can use immediately

Learning Outcome

Participants will walk away with a good understanding of the various AI products and libraries used today. Moreover, attendees will also have a pragmatic understanding of how these AI models can be applied to improve automated tests.

Target Audience

QA engineers, Developers

Prerequisites for Attendees

This course will address utilizing AI to address common pitfalls in automated test cases. As such, participants should already have a good working knowledge of Appium or Selenium.

  • Selenium or Appium experience
schedule Submitted 5 months ago

Public Feedback

comment Suggest improvements to the Speaker
  • Pallavi ...
    By Pallavi ...  ~  2 months ago
    reply Reply

    Hello Khanh

    Regarding the libraries and solutions which you would be showcasing to the audience, will it be required for them to know a certain programming language like will those libraries will be from python or java? 

    b. does the ML/AI tools showcases belong to open source world or commercial world? 

    c. In the outline and structure can you please provide a time breakup of the tutorial talk. 

    • Khanh Do
      By Khanh Do  ~  1 month ago
      reply Reply

      Hello!!

      a) The focus is on the capability of the library or solution and is largely language agnostic. While my examples will likely be in Java, the participants will not need to really understand the language. It is largely invoking calls into the library which is generally self-explanatory in any language.All that said, there will be very little code referenced in the session.

      b) I want to show two open source/free libraries, and then talk about and show 1 or 2 commercial solutions which are easier for non-programmers to use.

      c) Here is the proposed breakdown:

      • Introduction to AI - 5min
      • Why is AI promising in the field of Testing? - 5min
      • Using AI to make test cases more robust - 5min
      • How to apply ML libraries to solve common test case problems - 5min
      • How self-learning allows test cases to improve over time - 5min
      • A review of the major AI tools in the market today that you can use immediately - 20-25min
      • Q&A (time remaining)

      I'd like to reiterate the point that I hope participants will be able to leave my session knowing where AI is being used to improve automation testing, and what some of the open source and commercial solutions look like on the market today. 

      • Pallavi ...
        By Pallavi ...  ~  1 month ago
        reply Reply

        Thanks for the information.

  • Deepti Tomar
    By Deepti Tomar  ~  3 months ago
    reply Reply

    Hello Khanh,

    Thanks for your proposal!
    Could you please update the Outline/Structure section of your proposal with a time-wise breakup of how you plan to use 45 mins for the topics you've highlighted?

    And, to help the program committee understand your presentation style, can you provide a link to your past recording or record a small 1-2 mins trailer of your talk and share the link to the same? 

    Thanks!

    • Khanh Do
      By Khanh Do  ~  2 months ago
      reply Reply
    • Khanh Do
      By Khanh Do  ~  3 months ago
      reply Reply

      Hello Deepti - thanks for the follow-up and apologies for the delay. I will put a few slides together that not only provides more information around the topics and time allocation, but also to serve as a 1-2 min trailer of my talk. I will get it posted next week.

      Thanks again for the opportunity.

  • Robin Gupta
    By Robin Gupta  ~  3 months ago
    reply Reply

    The workshop can be made interactive/slightly more hands on as there are going to be intimidating topics for the entry level engineers. Also, the topic of the workshop is on singular topic, but the same is not covered in a focused fashion in the content/outline. It would be great , if we can focus on one problem statement, setup a solution (with some hands on) and cover the boundaries/challenges as the topic at hand is very broad to be covered in a 90 minute session

    • Khanh Do
      By Khanh Do  ~  3 months ago
      reply Reply

      Hi Robin - thanks for the comments. Based on the feedback received, I've opted to make it more pragmatic/practical and less theoretical. This will arm testers with solutions they can use immediately. I'll keep theory only to a minimum necessary to help testers know which solutions to select for which particular testing problem they're facing.

  • Maaret Pyhajarvi
    By Maaret Pyhajarvi  ~  3 months ago
    reply Reply

    The topic of this is interesting, but reading the details it leaves me concerned on what are the actual contents - would help to clarify. The description has so many conceptual things mentioned that the practical side of *applying* this to a testing problem looks very light. At first you mention specifically the locator problem and that there is an AI solution to it (are you planning on demoing one?) but later your description of how the session is structured is again very general. 

    Also, 1.5 hours workshop where I would think workshop = hands-on for audience with this many points to cover raises concerns. 

    Would clarifying what the audience does in the workshop be possible? Or, could this be a talk if it has no audience participation, specifically on the applying ML on testing problem with an example? 

    • Khanh Do
      By Khanh Do  ~  3 months ago
      reply Reply

      Hi Mareet - thank you for taking the time to review my proposal and for your feedback.

      I wanted to take a dual-approach to this. One, and the primary focus, is to show in highly practical terms how AI can make your test scripts less brittle, and I will do this by demoing current products on the market. Second though is I wanted to also provide a theoretical understanding of how AI works in the testing domain. This is to empower people to have an understanding behind-the-scenes versus just using commercial products.This is not a deep dive but I think that AI is going to be one of the most important skills over the next decade, and that a good introduction and basic understanding of how it works will empower testers to understand where and how they can apply AI to make them even better test engineers.

      In summary, I'd say it's 70% practical - seeing it in action - and 30% an introduction to AI in testing generally. 

      Finally, a talk may be a more accurate description than a workshop. Participants will see live demonstrations but most likely will not be required to have active participation or exercises.

      Please let me know if you have more questions. Based on your feedback, I will try highlight this in my proposal.

      Thanks again

      • Maaret Pyhajarvi
        By Maaret Pyhajarvi  ~  3 months ago
        reply Reply

        90 minute session without audience hands on is very heavy and I would find hard time selecting that to my suggestions. Is there a 45 minute version of this (demo talk) that you imagine you could find in you? Also, is there a video of your past speaking engagements available somewhere?

        • Khanh Do
          By Khanh Do  ~  3 months ago
          reply Reply

          Based on the feedback, I've updated the synopsis and outline to take a far more pragmatic approach, within a shorter timeframe. We'll first explore the promise of AI and then look at how solutions on the market today can help testers. This steps away from heavy theory and looks at the popular open source and commercial products available to testers today. Thanks for the guidance and helping to shape the presentation.


  • Liked Shweta Sharma
    keyboard_arrow_down

    Shweta Sharma / Nikita Jain - Accessibility testing 101

    45 Mins
    Talk
    Beginner

    "This world is such a beautiful place to live in." If you can read the first sentence without any screen readers or assistance, you're privileged. As technologists, shouldn’t we be more empathetic towards differently-abled people and make all parts of our website accessible to them? In my humble opinion, the true power of technology can be identified when it reaches out to people of all kinds having different physical or psychological challenges. We not only legally bound to provide Accessibility but also it should be considered as our moral responsibility.

    As testers, we have a wonderful opportunity to contribute to Accessibility by ensuring that the site is accessible in many different ways. Although it is impossible to identify all the issues that exist around Accessibility in the world, we are lucky enough to still understand a majority of them. With this understanding, there have been many measures taken in order to make your site accessible. But, don’t forget - we are QA engineers. We got to ensure that the site is accessible as per the standards set by WCAG 2.0 (AA) by testing for accessibility using various tools and techniques.

  • Liked Abhijeet Vaikar
    keyboard_arrow_down

    Abhijeet Vaikar - End-end test code as a first class citizen

    45 Mins
    Case Study
    Intermediate

    "All tests in today's automated regression run have been marked as Untested. What happened?"

    "No notifications are being sent for test runs on the channel"

    "I pulled latest code, and the framework dependency shows compilation error"

    "What does this new method in the framework do?"

    How often do you hear such things within your team?

    As Quality champions, we need to walk the talk. When we expect our developers to write quality code, write unit tests, build features without introducing bugs, the onus lies on us (as test engineers) to do the same. With almost every test engineering team writing automated tests to check functionality of their products and services, it becomes very important to ensure that the test automation framework and the test scripts are bug-free and follow good standards of software engineering.

    It cannot be stressed enough that test automation code should be as good as production code. In order to build production-quality test automation framework and scripts, a number of steps can be taken at:

    1. Code & System Level

    2. Process & People Level

    Our test engineering team went through a transition from having random & unexpected failing test runs to having greater confidence in the quality of the tests. Learn from this case study of our journey to ensure that end-end UI automated tests are built with quality in mind. We will also see demonstration of some of the use cases.

  • Liked Rajdeep varma
    keyboard_arrow_down

    Rajdeep varma - The Joy Of Green Builds - Running Tests Smartly

    45 Mins
    Talk
    Intermediate

    So you have got a few UI tests and they are running in parallel, great! However, life will not be so sweet once these 'a few' turns into 'a lot'. We grew from a few to 1500 UI tests (although not particularly proud of this number, there are situations and reasons)

    We started with a simple parallel distribution of tests 3 years ago. As test count increased failure count run time increased along with increased flaky tests. Mobile tests had their own challenges (eg. device dropping-off, random wi-fi issues, etc) To keep up with this, we created a queue and workers based solution which could distribute the tests more efficiently (https://github.com/badoo/parallel_cucumber). Over time, we made more improvements, in particular:

    • Segregation of failures based on infrastructure issues and re-queue the tests
    • If a device/emulator malfunction, rescue the tests to another device
    • Repeating a single test on 100s of the worker in parallel to detect flakiness
    • Repeat a test if a known network issue
    • Terminating the build early if more than a certain number of tests have failed
    • Health check of each device, before each test to ensure reliability
    • Muting a test if failure is known, and highlight outdated mutes if the related task is fixed

    In this talk, I will talk about the initial challenges with running UI tests in parallel (Selenium and Appium), how we approached the queue based solution and continuous improvement of this solution; finally, how attendees can use it at their workplace or create their own solution based on our learnings.

  • Liked Rabimba Karanjai
    keyboard_arrow_down

    Rabimba Karanjai - Testing Web Mixed Reality Applications: What you need to know for VR and AR

    Rabimba Karanjai
    Rabimba Karanjai
    Researcher
    Mozilla
    schedule 3 months ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    There are already over 200 million users consuming VR applications by 2018. And with Google, Mozilla pushing WebXR capabilities in browser and vendors like BBC, Amnesty International, Universal, Disney, Lenskart and a lot of them adopting them to their websites, we will soon see a huge rise i demand for Web VR and Mixed Reality applications.

    But how do you test them in scale? How do you define "smooth" as opposed to just responsive?

    In this talk I will go over some key details about the WebXR specification. The work that Mozilla, Google and the W3C Immersive Web Group is doing. The differences between testing a regular web page and a Mixed Reality enabled one. What to watch for and how you can automate it.

  • Liked Krishnan Mahadevan
    keyboard_arrow_down

    Krishnan Mahadevan - My experiments with Grid

    45 Mins
    Tutorial
    Intermediate

    Everyone starts off with a simple grid setup which involves a hub and one or more nodes.

    This traditional setup is a good start but the moment one starts to get serious with the selenium grid and decide to house their own selenium grid for their local executions, that is when issues start.

    My experiences with the Selenium grid in the past couple of years has led me to get introduced some of the most prevalent problems with maintaining an in-house selenium grid.

    • Nodes get unhooked randomly due to network glitches.
    • Nodes introduce false failures due to memory leaks.
    • Selenium Grid running out of capacity.
    • Nodes require OS upgrades/patches etc.
    • Needing to deal with auto upgrades by browsers (especially chrome and firefox)

    Some of these issues I managed to fix by building a "Self Healing" Grid wherein the nodes automatically get restarted after they have serviced "n" tests. But that still didn’t solve many of these other problems.

    That was when I felt, what if there was an on-demand selenium grid.

    What if the Grid could do the following ?

    • The Grid auto scales itself in terms of the nodes based on the current load.
    • The Grid does not require a lot of infrastructure to support it.
    • The Grid can plug itself into some of the cloud providers or leverage a solution such as Docker so that the nodes can be spun and shutdown at will.

    That was how the idea of "Just Ask" an on-demand grid was born.

    Just-Ask is an on-demand grid. It has no nodes attached to it.

    It’s designed to spin off nodes on demand, run test against the newly spun off node and after test runs to completion, clean-up the node as well. The node can be backed by anything. It could be Docker (or) it could be a VM running on any of the popular clouds.

    The session aspires to walk the audience through with my experiments with the selenium grid, my learnings on the selenium grid internals and how I used all of that knowledge to build my own On Demand Selenium Grid. What better avenue to share these learnings than a Selenium Conference.

    The session will introduce the audience to the grid internals and their concepts such as

    • What is a Selenium Remote Proxy ? What is it used for? What can you do with it?
    • What is a Hub (or) Node level Servlet ? When would you need one ?
    • All of this followed by a quick demo on "Just Ask", the on-demand grid that I have built and open sourced here: https://github.com/rationaleEmotions/just-ask

  • Liked Gaurav Singh
    keyboard_arrow_down

    Gaurav Singh - How to build an automation framework with selenium : patterns and practices

    Gaurav Singh
    Gaurav Singh
    Product Engineer
    Go-Jek
    schedule 5 months ago
    Sold Out!
    45 Mins
    Talk
    Beginner

    With an ever increasing no of businesses being conducted on web the testing need to write automated tests for the app's UI is something that can never be ignored. As you all know Selenium provides an API that enables us to do this quite effectively.

    However, when tasked with setting up the automation framework, there are a lot of questions that arise in the minds of aspiring test developers regardless of what level they are in their career.


    Some of such questions are:

    1. How does one actually go about the business of building a robust and effective automation framework on top of selenium?
    2. What are the elementary building blocks to include in the framework that an aspiring automation developer should know of?
    3. How should we model our tests? XUnit style vs BDD?
    4. Are there good practices, sensible design patterns and abstractions that we can follow in our code?
    5. What are some of the anti patterns/common mistakes we should avoid

    A lot of literature, documentation and blogs exists on these topics on the web already.

    However In this talk,

    I would combine this existing knowledge and my years of experience in building automation frameworks and breakdown these elements and walk you through exactly the sort of decisions/considerations and practices that you can take while starting to implement or improve the UI automation for your team.

    Hope to see you there!

  • Liked Syam Sasi
    keyboard_arrow_down

    Syam Sasi - When Ansible meets Selenium Grid - Story of building a stable local iOS simulator farm

    Syam Sasi
    Syam Sasi
    Senior Software Engineer
    Carousell
    schedule 4 months ago
    Sold Out!
    45 Mins
    Case Study
    Beginner

    There are dozens of matured docker based solutions available in the market for Android automation testing,

    but

    what about iOS testing?

    It’s hard to create and maintain the docker based solution for iOS testing since it demands the Xcode and for the optimal performance the system needs the Apple certified hardware.

    Combining Ansible with Selenium Grid yields a powerful combination because it allows us to set-up our grid and nodes in just a few seconds. In this talk I will demonstrate how to use simulators to build a reliable and scalable in-house iOS simulator lab using Ansible, Selenium Grid and Appium.

  • Liked Shi Ling Tai
    keyboard_arrow_down

    Shi Ling Tai - Start with the scariest feature - how to prioritise what to test

    Shi Ling Tai
    Shi Ling Tai
    CEO
    UI-licious
    schedule 4 months ago
    Sold Out!
    20 Mins
    Talk
    Beginner

    It can be intimidating for inexperienced teams embarking on their test automation journey for an existing code base. There is so much to test, and so many ways to test. I often see teams stuck with debating on where to start and what tools to use and best practices:

    "We should start from unit tests"

    "No, integration tests are better!"

    "Should we use tool A or tool B?"

    I see this play out all the time, and I've been there before. And the worst that could happen is decision paralysis and inaction.

    The bigger question really is "What to test?".

    My rule of thumb is "Start with the scariest code". I'll share with you my framework for evaluating the ROI of writing a test for a feature and prioritising what to test.

  • Liked Amit Rawat
    keyboard_arrow_down

    Amit Rawat - Is Puppeteer better than Selenium

    45 Mins
    Demonstration
    Intermediate

    Puppeteer is a Node js library (developed by Google Chrome team) to control Chrome and Firefox and is getting lot of traction recently because of its amazing capabilities. It has already become so popular that it has got 50K+ stars on Github against Selenium's 15K+ stars.

    In the last Google I/O event, this tool's capability has been showcased and it has been perceived as the next generation Web Test Automation Tool.

    Is Puppeteer better than Selenium? The answer is 'No', and I will cover 'why' in detail during this talk. I will show some live examples to demonstrate that Selenium can also do all those advance things which Puppeteer promises to do.

  • Liked Diego Molina
    keyboard_arrow_down

    Diego Molina - Test Automation is not just coding

    Diego Molina
    Diego Molina
    Sr. Software Engineer
    Sauce Labs
    schedule 3 months ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    We learn more from our failures than our successes. I want to share one of my failure stories, where I learned that coding is not the most important task in Test Automation.

    A failure taught me that coding is not the most important task in Test Automation. I fixed a bug, added tests, and shipped it to production. After that, a bug report came in showing that the fix created a new bug. I felt bad, I did not understand how that happened. A hotfix solved the issue, but the root cause was not addressed: what did I miss?

    By taking a step back, I understood the situation, it all happened by overseeing basic concepts in testing (like understanding how the system works). The problem was that coding had more priority than creating a test plan. After this, I prioritized tasks better to avoid this situation to happen again.

    This talk shows that testing concepts are more important than ever, in a time where tools promise to do everything, we focus less on what and how to test, and more in using tools to test. It outlines how a test strategy can leverage a continuous testing setup. Finally, it shows that failing is ok, but failing again for the same reasons is not.

  • Liked Denys Pavlenko
    keyboard_arrow_down

    Denys Pavlenko - Integrating Selenium tests into GitLab CI

    45 Mins
    Demonstration
    Intermediate

    GitLab is the first single application for the entire DevOps cycle. Only GitLab enables Concurrent DevOps, unlocking organizations from the constraints of the toolchain. GitLab provides unmatched visibility, higher levels of efficiency, and comprehensive governance. This makes the software lifecycle 200% faster, radically improving the speed of business.

    Taking into account the fast growth of GitLab in recent years and it’s prominent advantages, you might be required to start running Selenium tests in GitLab or even migrate your end2end tests from your previous CI tool to GitLab.

    Step by step guide will provide you the main challenges, key configurations, pitfalls in order to integrate and successfully run Selenium tests in GitLab CI. The wrapper around the Selenium to be presented is going to be Protractor with a built-in Jasmine test runner.

    The presentation will also showcase some main tips and tricks we’ve faced during this integration at our fast-growing startup in Munich, Germany.

  • Liked Babu Narayanan Manickam
    keyboard_arrow_down

    Babu Narayanan Manickam - Deep Learning Based Selenium Test Failure-triage Classification Systems

    45 Mins
    Talk
    Intermediate

    Problem Statement:

    While running thousands of automated test scripts on every nightly test schedule, we see a mixed test result of pass and failures. The problem begins when there is a heap of failed tests, we caught in the test-automation trap: unable to complete the test-failure triage from a preceding automated test run before the next testable build was released.

    Deep Learning Model:

    The classification was achieved by introducing Machine Learning in the initial months and followed by Deep Learning algorithms into our Selenium, Appium automation tests. Our major classification was based on the failed test cases: Script, Data, Environment, and Application Under Test and that internally had hundreds of sub-classifications.

    To overcome this problem, we started to build and train an AI using Deep Learning, which simulates a human to categorize the test case result suite. Based on the test result failure for each test, the AI model would predict an outcome through API, categorizes and prioritize on the scale of 0 to 1. Based on the prediction model, the algorithm takes appropriate response actions on those test cases that are failed like re-run test again or run for different capabilities. We kick-started this by gathering the historical data set of 1.6 million records, which was collected over a 12 months period, data including the behavior of the test case execution and the resulting suite.

    This Deep Learning-based algorithm has been provided the quality to break down the new defects based on its category, and a classification score is given on a scale of 0-1. We’ve also established a cutoff threshold based on its accuracy of improving, and to group the failed test cases based on its similarity. Classification of the test cases is done in high granularity for sophisticated analysis, and our statistical report states that the classification of the defects has been increased with 87% accuracy over a year. The system has been built based on the feedback adapting models, where for each right classification it will be rewarded and for the wrong, a penalty is given. So whenever receiving a penalty the system will automatically enhance itself for the next execution.

    The algorithm has a powerful model for detecting false-positive test results calculated using the snapshot comparisons, test steps count, script execution time and the log messages. Also, the model has been built with other features like – duplicate failure detection, re-try algorithms and defect logging API, etc.

    The entire classification system has been packaged and deployed in the cloud where it can be used as a REST service. The application has been built with its own reinforcement learning where it uses the classification score to enhance itself and this is programmed to perform in an inconclusive range.

    In sum, this deep learning solution can assist all Selenium testers to classify their test results in no-time and can assist to take next steps automatically and allow team could focus its efforts on new test failures.

    Link: https://github.com/testleaf-software/reinforced-selenium-test-execution

  • Liked Ivan Krutov
    keyboard_arrow_down

    Ivan Krutov - Workshop: Efficient Selenium Infrastructure with Selenoid

    Ivan Krutov
    Ivan Krutov
    Developer
    Aerokube
    schedule 4 months ago
    Sold Out!
    480 Mins
    Workshop
    Intermediate

    Selenoid is an alternative lightning fast open-source Selenium protocol implementation running browsers and Android emulators inside Docker containers. It is distributed with a set of ready-to-use Docker images corresponding to the majority of popular browser versions, has a one-command installation utility and works slightly more efficiently than traditional Selenium Grid.

    This tutorial shows how to build efficient and scalable browser automation infrastructure using Selenoid and related tools. You will be taught why running browsers in containers is so efficient, how to easily install Selenoid and use its powerful features.

  • Liked Srinivasu gangam
    keyboard_arrow_down

    Srinivasu gangam - Zero Touch Automation using NLP (Natural language processing) & AI

    Srinivasu gangam
    Srinivasu gangam
    Sr Manager
    Cotiviti India
    schedule 5 months ago
    Sold Out!
    45 Mins
    Demonstration
    Intermediate

    Problem Statement:

    As part of SDLC process:

    1. Is your product quality impacted due to a smaller number of QA resources available in the team?
    2. Are you waiting for QA resources to certify your code every time when you deploy? Is this impacting your product lead time (Speed to Market)?
    3. Is your Product delivery timelines are impacted due to last minute defects identified?
    4. Do you have your QA resources only in one location, but you want to “follow-the-sun” approach for Software delivery across multiple locations?
    5. Do you have manual testers who are not skilled in programming, but you want them to execute automated test scripts w/o any training efforts and automation setup?
    6. Would you like your team more agile and cross functional with Delivery?
    7. Would you like to increase your QA team’s productivity while they invest more time in script development rather than script execution?

    If answer is ‘Yes’ for above questions, "Zero touch automation" is the solution for above challenges that we have been facing part of SDLC.

    Solution: Zero touch automation with cutting-edge technologies

    In this session, I will cover how we solved this problem using innovative solutions, Cutting-edge technologies like NLP (Natural language processing), AI & Cloud solutions.

    You will learn how AI, NLP integrated with core automation components to achieve Zero touch automation.

    This solution is not just revolutionary, it is paradigm shift in test automation to get results to your email with detailed analysis of failure categorization with recommended actions to users.

    I will also cover how E2E automation will be driven with decisions taken by machines based on what user is looking for . There is no manual intervention in this process. NLP and AI play key role to help machines to take decisions.

    We will also cover how we empowered developer/release manager/any team member/Manager to trigger the scripts from their cell phone and get the detailed execution report without having any automation software installed in their computer or Phone.

    We will be demonstrating how the request will be initiated from User, understand the need from user using NLP & AI , Fetching the code from bitbucket to select appropriate automation scripts , running them on Selenoid/docker server , storing results to MongoDB , receiving email with test results and Failure analysis.

    What is the value of zero touch automation?

    1. Enable speed to market: Now that Developers does not need to wait for QA resource, Changes can be certified quickly and ready to push to production. Lead time will be significantly reduced.
    2. Increase quality: Now that test automation is easy and it can run multiple times in each environment, most of the defects will be uncovered and addressed before code goes to production.
    3. Ease of test execution: Test execution will be very easy, no automation or framework setup required from user side. Test execution can be done 24*7.
    4. Productivity: Increase QA team’s Productivity to focus more on script development rather than focusing on script execution and failure analysis .
  • Liked Naveen Khunteta
    keyboard_arrow_down

    Naveen Khunteta - Best Practices to implement the test automation framework starting from Design - To -> Infrastructure - To -> Execution.

    45 Mins
    Talk
    Intermediate

    Best Practices - How to get the best 'Return ON Investment' (ROI) from your Test Automation.

    This has been observed that, most of the test frameworks wont be able to survive due to lack of expertise, no maintenance, no best practices being followed, and finally your test automation will be dead after few months, and there is no "Return ON Investment" from this. This is the most common problem, most of the companies are struggling and finally back to square to the Manual testing.

    My proposal : HOW to leverage your test automation in terms of best practices, best ROI, and how to adopt best automation culture in your organisation.
    I strongly propose some of the important points/suggestions to achieve this in your Organisation/Team.
    1. Test Automation Practices:
    • Design Patterns (Web/Mobile/API)
    • What to Automate/Not to Automate
    2. Common Automation Frameworks at Org Level:
    • How to design Generic Utilities, Libraries and different Components, which can be suitable for all the teams in the same Org.
    • Best practices to design your Tests (Automation).
      • Common Design Patterns
      • Common application level and Page libraries
      • Best Practices to use Assertions in your Tests (How and What to write for assertions). Most of the people don't write proper assertions and this is making your test unreliable and no defects found during execution.
    3. Inclusion of API/Backend libraries in your UI test automation as an external Maven/Gradle Dependencies to avoid un-necessary tasks, some of the important points to be considered here:
    • User Creation from APIs (No need to automate user creation from web/app for all the test cases)
    • API tests are stabled most of the time
    • API calls takes lesser time as compared to web, hence include API calls in your UI/App framework to save time.
    • Less flaky test

    4. Best Code Review Process (Do not merge your code into Master without proper Code Review)

    • Implement PR (PULL Request) Process
    • Static Code Analysis using SonarQube, Cobertura, JACOCO etc..
    • Get the benefits of Best Test Automation Quality Matrices
    • Sometimes, Manual (Functional Tester) should review your code (Assertion, test steps and use cases) to get the best coverage
    5. Quality is A Team responsibility:
    • Developers, POs, Manual QEs and Automation engineers should be included to get an overview of test automation coverage.
    6. Maintenance of the Frameworks
    • After couple of months, it should not make your life miserable if you don't maintain your libraries and framework properly.
    • Do not use Hard Coded values, make it simple and Generic.
    7. Infrastructure Setup for Test Design and Test Execution:
    • Proper Browser - OS lab setup
    • Proper Mobile Labs setup with different Devices - IOT, iOS, Android, iPad, Tablets
    • Proper CI - CD common configuration using Jenkins, Dev Ops, AWS, Docker and Cloud setup
    • Handling multiple Docker nodes using Kubernates (use of Selenoid, GRID on Cloud)
  • Liked Praveen Umanath
    keyboard_arrow_down

    Praveen Umanath - State-of-the-art test setups: How do the best of the best test?

    20 Mins
    Talk
    Intermediate

    The best engineering teams are releasing code hundreds of times in a day. This is supported by a test setup that is not just fast, but robust and accurate at the same time.

    We look at data (anonymized) from millions of tests running on BrowserStack, to figure out the very best test setups. We also analyze the testing behavior of these companies—how do they test, how frequently do they test, how many device-browser-OS combinations do they test on. Do they gain speed by running more parallels or leaner test setups?

    Finally, we see how these steps help these teams to test faster and release continuously, and how it ties-in to the larger engineering strategy.

  • Liked Artem Sokovets
    keyboard_arrow_down

    Artem Sokovets - Three approaches to the next level of automation testing

    45 Mins
    Case Study
    Intermediate

    In my talk, I want to share several approaches that help to improve automation testing in your company.

    Do you want to spend less time developing UI automated tests? Maybe you want to reduce boilerplate code in UI tests? Or you think it is time to improve Page Object pattern that has been used for more than 8 years?

    My first point will be about how the interface can help us to represent a new vision of Page Object and what benefits we could get from it.

    The second - modern methods for coverage measuring are good only for development. However, the amount of useful information from coverage reports is extremely small.

    It will a journey about autotest coverage. Imagine, that your project manager asks you - Do we have autotests for these buttons (forms, pages) or what autotests cover these Rest APIs with 'special' query parameters? I will show you two simple solutions that allow you to evaluate the coverage of tests written by test automation specialists.

    a) API tests. There is a swagger and during his execution, we also collect a swagger. We count coverage as a diff of two swaggers. We can see what is covered, what is not and what data used.

    b) Web tests. During the execution of tests, we collect information about elements, pages, and actions. As a result, we have such a format: page -> elements -> actions -> tests. All information representing in Chrome Plugin, that I will share with you.

    And last, I am going to consider what cons have current TMS (QA time management systems) and what to do if you want to make an automation-first strategy on your project? Moreover, you look for one TMS for developers and QAs/or you want to save the Allure report of all autotests in one place?

    Here you are! Allure Server has already released. I will overview the most important features and how to start using it from scratch.

    If you have an interest in the above themes come to my talk.

  • Liked Benjamin Bischoff
    keyboard_arrow_down

    Benjamin Bischoff - Simplifying your test runs with ‘make’

    45 Mins
    Case Study
    Intermediate

    The ‘make’ command has been around since 1976 and was originally used to build executable programs. In this talk I want to show how this powerful tool can greatly reduce complexity and eliminate stumbling blocks of running your automated tests. This applies to both CI pipelines and local test runs on developer or QA machines.

  • Liked Sameer Arora
    keyboard_arrow_down

    Sameer Arora - Triggering alerts on Slack along with screenshots in case of test case failure

    Sameer Arora
    Sameer Arora
    Senior Software Test Engineer
    Gaana
    schedule 5 months ago
    Sold Out!
    45 Mins
    Demonstration
    Beginner

    One of the biggest limitations with most of the frameworks in selenium is that we need to wait till the end to get the final test case report. Only then we can raise the alert that a particular functionality is broken. By then, it may be too late! In addition to this, in most of the frameworks, the HTML report generated is hosted on our company's private network and we need to connect to the VPN if we are outside the office to open the report and check the related screenshots to see which test case has failed and why.

    So, why not raise an alert as soon as the test case fails? And why not attach a screenshot along with that alert so that all the stakeholders can actually see what has failed in the application?

    In this session, we will learn a simple yet a really useful way of sending an alert along with the failed test case screenshot by integrating slack with selenium which will help us alert all the stakeholders as soon as an automated test case fails so that everyone can react to it immediately and the damage will be minimised.

  • Liked Varadharajan Srinivasan
    keyboard_arrow_down

    Varadharajan Srinivasan - How do we test billion data set in 1hour

    45 Mins
    Talk
    Intermediate

    We are now living in the age of Big Data. What makes Big Data so special is its ability to help the common man process petabytes of data to build innovative solutions. Cloud has played a pivotal role in building this data economy by making infrastructure more cost effective and enabling easy scalability.

    Today, the abundance of data helps business make intelligent decisions through business process automation. Data drives all our digital interactions, starting from products that we see on our Amazon home page, to our Netflix recommendations, to the advertisement on Google search results, to even the Friends that we get connected to on Facebook. All these systems are powered by data that is collected from the various system, cleaned, transformed and stored into Big Data solutions. Testing of these data systems have become very complex and requires a new approach. This paper highlights various aspects of Big Data Testing including:

    • Complexity in testing Big Data systems

    • Cloud based tools for testing Big Data cross various stages

    • Simplification of test processes and visualization through dashboards

    • Testing Big Data in the Agile/ DevOps