Everyone starts off with a simple grid setup which involves a hub and one or more nodes.

This traditional setup is a good start but the moment one starts to get serious with the selenium grid and decide to house their own selenium grid for their local executions, that is when issues start.

My experiences with the Selenium grid in the past couple of years has led me to get introduced some of the most prevalent problems with maintaining an in-house selenium grid.

  • Nodes get unhooked randomly due to network glitches.
  • Nodes introduce false failures due to memory leaks.
  • Selenium Grid running out of capacity.
  • Nodes require OS upgrades/patches etc.
  • Needing to deal with auto upgrades by browsers (especially chrome and firefox)

Some of these issues I managed to fix by building a "Self Healing" Grid wherein the nodes automatically get restarted after they have serviced "n" tests. But that still didn’t solve many of these other problems.

That was when I felt, what if there was an on-demand selenium grid.

What if the Grid could do the following ?

  • The Grid auto scales itself in terms of the nodes based on the current load.
  • The Grid does not require a lot of infrastructure to support it.
  • The Grid can plug itself into some of the cloud providers or leverage a solution such as Docker so that the nodes can be spun and shutdown at will.

That was how the idea of "Just Ask" an on-demand grid was born.

Just-Ask is an on-demand grid. It has no nodes attached to it.

It’s designed to spin off nodes on demand, run test against the newly spun off node and after test runs to completion, clean-up the node as well. The node can be backed by anything. It could be Docker (or) it could be a VM running on any of the popular clouds.

The session aspires to walk the audience through with my experiments with the selenium grid, my learnings on the selenium grid internals and how I used all of that knowledge to build my own On Demand Selenium Grid. What better avenue to share these learnings than a Selenium Conference.

The session will introduce the audience to the grid internals and their concepts such as

  • What is a Selenium Remote Proxy ? What is it used for? What can you do with it?
  • What is a Hub (or) Node level Servlet ? When would you need one ?
  • All of this followed by a quick demo on "Just Ask", the on-demand grid that I have built and open sourced here: https://github.com/rationaleEmotions/just-ask

 
 

Outline/Structure of the Tutorial

  • 10 - Mins - Grid customization concepts
  • 10 - Mins - Components of On-demand Grid
  • 15 - mins - On-demand Grid demo
  • 5 - mins Q & A

Learning Outcome

After this session, the audience would be able to understand:

  1. How does the Selenium Grid work ?
  2. What are the various ways of customizing a selenium grid ?
  3. Get a better understanding on the Selenium Grid internals ?

Target Audience

Anyone who is interested in the Grid or is involved in maintaining a Grid infrastructure to support their test automation

Prerequisites for Attendees

Basic understanding and introduction to the Grid should suffice.

schedule Submitted 4 months ago

Public Feedback

comment Suggest improvements to the Speaker
  • Anand Bagmar
    By Anand Bagmar  ~  3 months ago
    reply Reply

    Hi Krishnan,

    Just-ask seems very interesting. I have a few questions -

    • Will this be a live demo or a video recording?
    • What aspects are you going to demo?
    • Do you any specific requirements for the talk / demo?

    Lastly, can you please share any reference slides and video from any of your past talks? You can edit the proposal and update the links there itself. Thanks.

    Anand

    • Krishnan Mahadevan
      By Krishnan Mahadevan  ~  3 months ago
      reply Reply

      Anand,

      My answers.

      1. I intend to keep this as a live demo. But as you know, demos usually conk only at the time when you really want to show it to a bunch of people :) So I will have a video recording of the demo as a back-up just in case its required.

      2. Aspects of the demo - Its mainly to showcase what Just-ask can do, and while we are at it, also show some of the aspects of the code, that are related to the concepts of the Selenium Grid (For e.g., a custom servlet and a custom proxy)

      3. Apart from the usual stuff like projector, I dont have anything specific as a requirement. 

      I am a first time speaker Anand. Never done any talks in all these 10 years of me using Selenium and contributing in my own ways. This would be my first maiden presentation.

      I am yet to work on creating the slides. I will be doing that soon. Just want to ensure that I start doing it, if this proposal does get selected :) I hope that is fine.

      • Anand Bagmar
        By Anand Bagmar  ~  3 months ago
        reply Reply

        Hi Krishnan,

        Firstly - I appreciate you submitting to speak at Selenium Conference and wish you best of luck from a selection perspective!

        To assist in making that happen, in response to your points, please see some follow-up thoughts below:

        #1 - good idea to have a back-up plan. Note that the video you prepare as a back-up should have the details clear in them - which will be visible also to the last person on the far-side of the room

        #2 - Can you list / share what are the different aspects / features / capabilities you will demo? That will also help you ensure the time-management is done correctly

        #3 - For the live demo, do you need internet connectivity?

        Since this would be your first presentation, can you record a 2-3 min video clip summarizing this talk and share it with us? Also, would not be a bad idea to start putting your thoughts in form of slides / or any other way you want to present it and share that as well. The final version can be updated closer to the conference date. This aspect will ensure you also get the structure and content right in terms of time, and help the program committee take an informed decision.

         

        • Krishnan Mahadevan
          By Krishnan Mahadevan  ~  3 months ago
          reply Reply

          Anand,

          Thanks for the detailed feedback. It helps.

          1. I will work on creating a video that would show what I intend to demo.
          2. I intend to showcase the following aspects in the demo:
            1. What can just-ask do as a tool (Run one test case that will spin off a docker container, run the test against it and then close the browser and thus clean up the container as well)
            2. Show how a custom proxy looks like.
            3. Show how a custom servlet looks like.
            4. How to bundle the custom proxy and custom servlet, and spin off the Hub using this.
          3. For the live demo, no i dont need any internet connectivity. I think I should be able to do with just a locally hosted sample web app, that can be spun off. 

          For the 2-3 min video, can you please let me know on the following:

          1. How much time do I have before I have to submit this video ?
          2. How do I share the created video with you folks ?

          For the presentation, I will start working on creating it soon and share that as well. For the presentation :

          1. How much time do I have before I have to submit the presentation ?
          2. How do I share the created presentation with you folks ?

          Thank you so much for offering to help with all these valuable inputs. Appreciate it.

          • Anand Bagmar
            By Anand Bagmar  ~  3 months ago
            reply Reply

            The sooner you can share the video & reference slides, the better it will be. We do understanding this is work in progress, but every bit of information will help.

            You could put the video on youtube, or share it via google drive. Similarly for the slides, google slides may be a good idea to keep working on the slides for the talk.

            Regarding your other clarifications, please do edit and update the proposal where applicable with the information you have provided.

            • Krishnan Mahadevan
              By Krishnan Mahadevan  ~  3 months ago
              reply Reply

              Thanks Anand. I will get to work on this immediately and try and submit this stuff at the earliest. Never done video recording, so it would be a good learning exercise I guess. 

              • Anand Bagmar
                By Anand Bagmar  ~  3 months ago
                reply Reply

                Thanks Krishnan!

            • Krishnan Mahadevan
              By Krishnan Mahadevan  ~  3 months ago
              reply Reply

              Anand, I have updated the proposal with a 3 min video. Please let me know if that would suffice


  • Liked Abhijeet Vaikar
    keyboard_arrow_down

    Abhijeet Vaikar - End-end test code as a first class citizen

    45 Mins
    Case Study
    Intermediate

    "All tests in today's automated regression run have been marked as Untested. What happened?"

    "No notifications are being sent for test runs on the channel"

    "I pulled latest code, and the framework dependency shows compilation error"

    "What does this new method in the framework do?"

    How often do you hear such things within your team?

    As Quality champions, we need to walk the talk. When we expect our developers to write quality code, write unit tests, build features without introducing bugs, the onus lies on us (as test engineers) to do the same. With almost every test engineering team writing automated tests to check functionality of their products and services, it becomes very important to ensure that the test automation framework and the test scripts are bug-free and follow good standards of software engineering.

    It cannot be stressed enough that test automation code should be as good as production code. In order to build production-quality test automation framework and scripts, a number of steps can be taken at:

    1. Code & System Level

    2. Process & People Level

    Our test engineering team went through a transition from having random & unexpected failing test runs to having greater confidence in the quality of the tests. Learn from this case study of our journey to ensure that end-end UI automated tests are built with quality in mind. We will also see demonstration of some of the use cases.

  • Liked Rajdeep varma
    keyboard_arrow_down

    Rajdeep varma - The Joy Of Green Builds - Running Tests Smartly

    45 Mins
    Talk
    Intermediate

    So you have got a few UI tests and they are running in parallel, great! However, life will not be so sweet once these 'a few' turns into 'a lot'. We grew from a few to 1500 UI tests (although not particularly proud of this number, there are situations and reasons)

    We started with a simple parallel distribution of tests 3 years ago. As test count increased failure count run time increased along with increased flaky tests. Mobile tests had their own challenges (eg. device dropping-off, random wi-fi issues, etc) To keep up with this, we created a queue and workers based solution which could distribute the tests more efficiently (https://github.com/badoo/parallel_cucumber). Over time, we made more improvements, in particular:

    • Segregation of failures based on infrastructure issues and re-queue the tests
    • If a device/emulator malfunction, rescue the tests to another device
    • Repeating a single test on 100s of the worker in parallel to detect flakiness
    • Repeat a test if a known network issue
    • Terminating the build early if more than a certain number of tests have failed
    • Health check of each device, before each test to ensure reliability
    • Muting a test if failure is known, and highlight outdated mutes if the related task is fixed

    In this talk, I will talk about the initial challenges with running UI tests in parallel (Selenium and Appium), how we approached the queue based solution and continuous improvement of this solution; finally, how attendees can use it at their workplace or create their own solution based on our learnings.

  • Liked Virender Singh
    keyboard_arrow_down

    Virender Singh - Rise of Shadow DOM - Lets solve it through WebDriver

    45 Mins
    Demonstration
    Beginner

    Shadow DOM is kind of web standard that developers use to encapsulate their custom HTML code and style components so that no other global style components can override their code. It ensures that a component will work in any environment, even if other CSS or JavaScript is run on the page i.e. Shadow DOM hides away the complexity of the components from the rest of the page because it renders separately from main DOM.

    Selenium legacy selectors are not able to interact these custom elements under the shadow DOM so we need java script to interact with shadow DOM elements and we also require JavascriptExecutor interface for their execution.

    We will also talk about the WebdriverIO tool v5.5.0, which implicitly support Shadow DOM elements.

     
     
  • Liked Rabimba Karanjai
    keyboard_arrow_down

    Rabimba Karanjai - Testing Web Mixed Reality Applications: What you need to know for VR and AR

    Rabimba Karanjai
    Rabimba Karanjai
    Researcher
    Mozilla
    schedule 4 months ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    There are already over 200 million users consuming VR applications by 2018. And with Google, Mozilla pushing WebXR capabilities in browser and vendors like BBC, Amnesty International, Universal, Disney, Lenskart and a lot of them adopting them to their websites, we will soon see a huge rise i demand for Web VR and Mixed Reality applications.

    But how do you test them in scale? How do you define "smooth" as opposed to just responsive?

    In this talk I will go over some key details about the WebXR specification. The work that Mozilla, Google and the W3C Immersive Web Group is doing. The differences between testing a regular web page and a Mixed Reality enabled one. What to watch for and how you can automate it.

  • Liked Vinayaka Mayura G G
    keyboard_arrow_down

    Vinayaka Mayura G G - Search Relevancy Testing: QA in Machine Learning Models

    Vinayaka Mayura G G
    Vinayaka Mayura G G
    QA
    ThoughtWorks
    schedule 4 months ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    The adoption of Artificial Intelligence is getting more traction, it is in need to enhance QA capabilities to cope up with these skills. Machine Learning is used extensively in retail applications for solving complex problems, one of them is solving the search relevancy. Showing the appropriate results for the user is important for the conversion rate to go high. As Machine Learning poses different challenges such as a Test Oracle, Fairness, Correctness and Robustness to do QA, We may need to follow different approaches and testing techniques to do the QA for Machine Learning models.

    Different Machine learning types such as Supervised and Unsupervised Models have different characteristics and are used for different types of problems. Though these solves different complex problems, Machine learning Models also a unit of software code that needs to be verified as a normal software system. When a Machine learning model is seen as a whole system, it may look complex and unsolvable. We can group them into small modules and verify for quality. Black box and White box testing techniques can be applied to verify the functionality. Data, Feature Engineering and Algorithms are the major part of the Machine Learning model. We will see how we applied different techniques to validate these.

    This talk is focused on viewing the Machine Learning software as a whole and performing the Quality Analysis for it. We look at how different is testing a machine learning model from typical software testing. We will discuss the challenges that came across, the Process involved in building an ML model. We take an example of Search Relevance for an explanation. We will dive into the areas where quality is assessed. The significant factors considered here are measuring Accuracy and Efficiency. We will look into the different black box testing techniques for different Algorithms. Let us also see how traditional testing is different from testing machine learning applications. I will go through different black-box testing techniques with examples following a live demo.

  • Liked Zachary Attas
    keyboard_arrow_down

    Zachary Attas - Selenium For All - Setting Your Team Up For Success So Anyone Can Understand and Write Tests

    Zachary Attas
    Zachary Attas
    Sr. Test Engineer
    ReviewTrackers
    schedule 4 months ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    Are you the only one contributing to your test suite, because others feel too intimidated to contribute to it?

    Have you ever looked at a test fail and not known where to investigate first?

    This talk will cover techniques for uncomplicating your test suites, so they can clearly test exactly what's needed to be tested, allowing others to contribute tests. When the process of writing tests is shared, everyone has an impact on quality, and that can be quite contagious for a company's culture.

    In addition, this talk will cover techniques for failing transparently, so you will no longer feel stressed triaging test fails. Quality is something shared by the entire team, so anyone should be able to triage test fails. When tests fail for a specific, clear reason, you'll be thanking your past self for writing logic to fail transparently, saving you half a day going down a rabbit hole.

  • Liked Jesus Sanchez Martinez
    keyboard_arrow_down

    Jesus Sanchez Martinez - Test and monitor one website is not that hard, but what if you need to do it to over 40 websites?

    Jesus Sanchez Martinez
    Jesus Sanchez Martinez
    QA Manager
    Onestic
    schedule 4 months ago
    Sold Out!
    45 Mins
    Talk
    Advanced

    Onestic QA department made and maintained test suites, which was a huge bottleneck in our development process.

    In order to solve it, management bought us an idea: our developers must be able to build their tests using a DSL framework without friction. QA maintain a big library that provides resources to developers. From there, they were free to extend this library in each project. With all of this we have a CI process, a 1 hour execution to monitor our results and of course our bot, SpongeBot.

    SpongeBot can check 40+ e-commerce sites, with 4+ environments, for desktop and mobile platforms always available for developers.

    With this solution we achieve to decentralize the work, add value testing production and the confidence in SpongeBot notifications if something goes wrong.

  • Liked Gaurav Singh
    keyboard_arrow_down

    Gaurav Singh - How to build an automation framework with selenium : patterns and practices

    Gaurav Singh
    Gaurav Singh
    Product Engineer
    Go-Jek
    schedule 5 months ago
    Sold Out!
    45 Mins
    Talk
    Beginner

    With an ever increasing no of businesses being conducted on web the testing need to write automated tests for the app's UI is something that can never be ignored. As you all know Selenium provides an API that enables us to do this quite effectively.

    However, when tasked with setting up the automation framework, there are a lot of questions that arise in the minds of aspiring test developers regardless of what level they are in their career.


    Some of such questions are:

    1. How does one actually go about the business of building a robust and effective automation framework on top of selenium?
    2. What are the elementary building blocks to include in the framework that an aspiring automation developer should know of?
    3. How should we model our tests? XUnit style vs BDD?
    4. Are there good practices, sensible design patterns and abstractions that we can follow in our code?
    5. What are some of the anti patterns/common mistakes we should avoid

    A lot of literature, documentation and blogs exists on these topics on the web already.

    However In this talk,

    I would combine this existing knowledge and my years of experience in building automation frameworks and breakdown these elements and walk you through exactly the sort of decisions/considerations and practices that you can take while starting to implement or improve the UI automation for your team.

    Hope to see you there!

  • Liked Moshe Milman
    keyboard_arrow_down

    Moshe Milman - Fixing Your Automation Challenges in the Era of CI/CD

    Moshe Milman
    Moshe Milman
    COO & Co-founder
    Applitools
    schedule 4 months ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    Continuous Delivery is now the holy grail of IT organizations, but most companies are still struggling with the transition into shorter release cycles and faster, more frequent deployments. A key challenge that companies are facing in that transition lies around test automation, and in this talk we will cover these challenges in details and demonstrate how successful companies are addressing it.
    The key takeaways from this session include:
    1) Testability hacks + best practices
    2) Tips for reducing your testing matrix to support the automation speed required for frequent builds/releases
    3) Examples of team structure/architecture from leading companies to support better alignment around automation
    4) Guidelines for picking the right open source tools / frameworks
    5) Learnings/Issues to avoid when designing your automation architecture
  • Liked Syam Sasi
    keyboard_arrow_down

    Syam Sasi - When Ansible meets Selenium Grid - Story of building a stable local iOS simulator farm

    Syam Sasi
    Syam Sasi
    Senior Software Engineer
    Carousell
    schedule 4 months ago
    Sold Out!
    45 Mins
    Case Study
    Beginner

    There are dozens of matured docker based solutions available in the market for Android automation testing,

    but

    what about iOS testing?

    It’s hard to create and maintain the docker based solution for iOS testing since it demands the Xcode and for the optimal performance the system needs the Apple certified hardware.

    Combining Ansible with Selenium Grid yields a powerful combination because it allows us to set-up our grid and nodes in just a few seconds. In this talk I will demonstrate how to use simulators to build a reliable and scalable in-house iOS simulator lab using Ansible, Selenium Grid and Appium.

  • Liked Shi Ling Tai
    keyboard_arrow_down

    Shi Ling Tai - Start with the scariest feature - how to prioritise what to test

    Shi Ling Tai
    Shi Ling Tai
    CEO
    UI-licious
    schedule 4 months ago
    Sold Out!
    20 Mins
    Talk
    Beginner

    It can be intimidating for inexperienced teams embarking on their test automation journey for an existing code base. There is so much to test, and so many ways to test. I often see teams stuck with debating on where to start and what tools to use and best practices:

    "We should start from unit tests"

    "No, integration tests are better!"

    "Should we use tool A or tool B?"

    I see this play out all the time, and I've been there before. And the worst that could happen is decision paralysis and inaction.

    The bigger question really is "What to test?".

    My rule of thumb is "Start with the scariest code". I'll share with you my framework for evaluating the ROI of writing a test for a feature and prioritising what to test.

  • Liked Amit Rawat
    keyboard_arrow_down

    Amit Rawat - Is Puppeteer better than Selenium

    45 Mins
    Demonstration
    Intermediate

    Puppeteer is a Node js library (developed by Google Chrome team) to control Chrome and Firefox and is getting lot of traction recently because of its amazing capabilities. It has already become so popular that it has got 50K+ stars on Github against Selenium's 15K+ stars.

    In the last Google I/O event, this tool's capability has been showcased and it has been perceived as the next generation Web Test Automation Tool.

    Is Puppeteer better than Selenium? The answer is 'No', and I will cover 'why' in detail during this talk. I will show some live examples to demonstrate that Selenium can also do all those advance things which Puppeteer promises to do.

  • Liked Diego Molina
    keyboard_arrow_down

    Diego Molina - Test Automation is not just coding

    Diego Molina
    Diego Molina
    Sr. Software Engineer
    Sauce Labs
    schedule 4 months ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    We learn more from our failures than our successes. I want to share one of my failure stories, where I learned that coding is not the most important task in Test Automation.

    A failure taught me that coding is not the most important task in Test Automation. I fixed a bug, added tests, and shipped it to production. After that, a bug report came in showing that the fix created a new bug. I felt bad, I did not understand how that happened. A hotfix solved the issue, but the root cause was not addressed: what did I miss?

    By taking a step back, I understood the situation, it all happened by overseeing basic concepts in testing (like understanding how the system works). The problem was that coding had more priority than creating a test plan. After this, I prioritized tasks better to avoid this situation to happen again.

    This talk shows that testing concepts are more important than ever, in a time where tools promise to do everything, we focus less on what and how to test, and more in using tools to test. It outlines how a test strategy can leverage a continuous testing setup. Finally, it shows that failing is ok, but failing again for the same reasons is not.

  • Liked Sanjay Kumar
    keyboard_arrow_down

    Sanjay Kumar - ChroPath- Smarter way of writing automation script!!

    Sanjay Kumar
    Sanjay Kumar
    ChroPath Creator
    AutonomIQ
    schedule 6 months ago
    Sold Out!
    45 Mins
    Case Study
    Intermediate
    • Want to save 70-80% manual effort of automation script writing!!
    • Wasting time in verifying xpath one by one?
    • Want to complete automation script without wasting much time?
    • Are you still wasting time in writing english manual test cases?
  • Liked Christian Bromann
    keyboard_arrow_down

    Christian Bromann - The Nuts and Bolts of WebdriverIO

    Christian Bromann
    Christian Bromann
    Software Engineer
    Sauce Labs
    schedule 4 months ago
    Sold Out!
    90 Mins
    Workshop
    Beginner

    There are thousands ways if not more to setup your automation testing environment. It is often crucial when it comes to stability and maintainability. While you can build a custom framework that fits your needs there are often already sophisticated setups or frameworks available that get you up and running very quickly.

    WebdriverIO is one of these frameworks that is written in Node.js. It gets you up and running within minutes and allows you to scale up your test suite while maintaining your execution time by running tests in parallel. With its huge project community it is an ideal choice for small as well as big projects that use modern frameworks such as React, Angular or Vue.js.

    In this workshop you will learn everything you need to know to run a successful, stable and maintainable WebdriverIO framework. It starts with an introduction to the project and the basic concepts and will end with a sophisticated framework that includes testing strategies like Frontend Performance Testing as well as complex browser interaction with Puppeteer.

  • Liked Dawid Pacia
    keyboard_arrow_down

    Dawid Pacia - Build your own Internet of Continuously Delivered Things

    Dawid Pacia
    Dawid Pacia
    Test Automation Manager
    Brainly
    schedule 4 months ago
    Sold Out!
    45 Mins
    Case Study
    Intermediate

    It is predicted that till 2025 there will be over 75 billion devices spewing 180 zettabytes of data and generating up to 6 trillion dollars. That enormous increase force companies to introduce a continuous approach to deliver the product as fast as possible and be able to compete on the market.


    The main question is how to test application for end user among so much hardware equipment and ecosystems combining HW, FW, mobile devices and complex backend architecture? Considering all factors and possible obstacles is it for companies a real “A New Hope” for companies or just simply “Mission Impossible”?


    I will take the participants on a journey to the IoT world. It will be a talk about the challenges that any tester will face at some point. I will present the dangers, risks and snares but also good practices and practical approach to mobile E2E test automation for the IoT solutions in CI approach.


    Technical examples will be presented using Python languages and supported by physical devices (mobile phones and IoT equipment).

  • Liked Pallavi ...
    keyboard_arrow_down

    Pallavi ... - Learn Selenium with JavaScript

    480 Mins
    Workshop
    Beginner

    Selenium is a popular open source test automation tool, so popular it has its own conference event which run for 3 whole days across continents and is attended by hundreds of enthusiastic testers, and people wanting to know more about testing and automation. In my little experience of coaching for last 7 years, i feel what english language has done for common man[those from non english speaking countries] in earning bread and butter and giving them an elite status, selenium has done for manual testers.

    Anyone and everyone wants to learn Selenium. The magic of selenium or the flexibility is that it can be learnt with your choice of programming languages. I have taught people Selenium with Java, C#, Python and Ruby largely in past these years. I wish to delve into the world of Javascript now and see what this has to offer to us. So this workshop is for those open minded souls out there who wish to pick up another language and learn, unlearn, relearn selenium with it.

  • Liked Srinivasan Sekar
    keyboard_arrow_down

    Srinivasan Sekar / Sai Krishna - Testing And Observability in an Integrated Microservices environment

    45 Mins
    Case Study
    Intermediate

    Leading-edge applications are dynamic and adaptive in capabilities that require people to use increasingly dexterous tools and supporting infrastructure, including microservices. All of these applications leverage data in new ways. Decoration and tagging of data with intelligent meta-data have become more important than data itself. To keep up with evolving needs, enterprise devs across industries are shifting from traditional app architectures in favor of more fluid architecture for building data-centric applications.

    Microservices break traditionally structured applications into manageable pieces that can be developed and maintained independently. microservices are often decoupled, allowing for updates with little to no downtime, as the other components can continue running.

    Moving to distributed Microservices ecosystem brings its own challenges; Among them is the loss of visibility into the system, and the complex interactions now occurring between services. Monitoring these applications only reports the health of it but Observability provides granular insights about the behavior of the system along with rich content. In this talk, we will cover the difference of Monitoring and Observability, data path engineering challenges, pillars of observability, distributed tracing of various microservices, testing in distributed microservices ecosystem, automated observability, etc.

  • Liked Anton Angelov
    keyboard_arrow_down

    Anton Angelov - Combining Load and Functional Testing- Reusing WebDriver Tests for Load Testing

    Anton Angelov
    Anton Angelov
    CTO
    Automate The Planet
    schedule 4 months ago
    Sold Out!
    45 Mins
    Demonstration
    Advanced

    Typically, industry software testing practice is to separate load testing from functional testing. Different teams with different skills and expertise do their testing at different times, and each evaluates the results against its own criteria.

    Anton Angelov will present to you how you can reuse your functional tests for load testing. We will review the design of a load testing engine that will reuse the same functional tests and at the end of the execution, generate a comprehensive report. We will also talk about how to utilize the new features in Selenium 4.0 in this load testing solution. As part of the reviewed tooling, there will be a discussion about the best possible checks you can do in your coded load testing library and the essential load testing metrics you need to collect during the run.

  • Liked Babu Narayanan Manickam
    keyboard_arrow_down

    Babu Narayanan Manickam - Deep Learning Based Selenium Test Failure-triage Classification Systems

    45 Mins
    Talk
    Intermediate

    Problem Statement:

    While running thousands of automated test scripts on every nightly test schedule, we see a mixed test result of pass and failures. The problem begins when there is a heap of failed tests, we caught in the test-automation trap: unable to complete the test-failure triage from a preceding automated test run before the next testable build was released.

    Deep Learning Model:

    The classification was achieved by introducing Machine Learning in the initial months and followed by Deep Learning algorithms into our Selenium, Appium automation tests. Our major classification was based on the failed test cases: Script, Data, Environment, and Application Under Test and that internally had hundreds of sub-classifications.

    To overcome this problem, we started to build and train an AI using Deep Learning, which simulates a human to categorize the test case result suite. Based on the test result failure for each test, the AI model would predict an outcome through API, categorizes and prioritize on the scale of 0 to 1. Based on the prediction model, the algorithm takes appropriate response actions on those test cases that are failed like re-run test again or run for different capabilities. We kick-started this by gathering the historical data set of 1.6 million records, which was collected over a 12 months period, data including the behavior of the test case execution and the resulting suite.

    This Deep Learning-based algorithm has been provided the quality to break down the new defects based on its category, and a classification score is given on a scale of 0-1. We’ve also established a cutoff threshold based on its accuracy of improving, and to group the failed test cases based on its similarity. Classification of the test cases is done in high granularity for sophisticated analysis, and our statistical report states that the classification of the defects has been increased with 87% accuracy over a year. The system has been built based on the feedback adapting models, where for each right classification it will be rewarded and for the wrong, a penalty is given. So whenever receiving a penalty the system will automatically enhance itself for the next execution.

    The algorithm has a powerful model for detecting false-positive test results calculated using the snapshot comparisons, test steps count, script execution time and the log messages. Also, the model has been built with other features like – duplicate failure detection, re-try algorithms and defect logging API, etc.

    The entire classification system has been packaged and deployed in the cloud where it can be used as a REST service. The application has been built with its own reinforcement learning where it uses the classification score to enhance itself and this is programmed to perform in an inconclusive range.

    In sum, this deep learning solution can assist all Selenium testers to classify their test results in no-time and can assist to take next steps automatically and allow team could focus its efforts on new test failures.

    Link: https://github.com/testleaf-software/reinforced-selenium-test-execution