Run tests at scale with on-demand Selenium Grid using AWS Fargate

If CI/CD is the backbone of any modern DevOps environment, then testing is crucial for CI/CD success. Speed and stability of tests can be a major challenge for any successful CI/CD implementation, and these factors are governed to a huge extent by the testing infrastructure.

For our DevOps approach to be successful, we needed a test infrastructure that is highly available, stable and also rapidly scalable under load. To achieve this, we built Selenium Grid using AWS services that can be spun up on-demand, supports auto-scaling and can be easily disposed of after use, using docker and AWS container orchestration service, Fargate.

Benefits and Takeaways:

1. Speedy execution by running tests in 1000s of browsers. This can be a very cost-effective solution as cloud-based Grid can be switched off when not needed.
2. Lower maintenance overhead since infra does not need to be managed or patched from time to time in case of Fargate
3. Easy to change browser versions, count etc. with just a small change in CloudFormation template.
4. Auto-scaling logic is integrated. That means, more browsers can be added on-the-fly under heavy load.


Outline/Structure of the Talk

  • Brief overview of the problems faced with scaling tests, maintaining infra etc., and how docker and cloud can be used to solve them
  • Detailed technical walkthrough of the implementation
  • How it integrates with CI/CD pipelines

Learning Outcome

  • Learn how AWS and docker can be combined to finish 1000s of UI tests within minutes
  • Modifications required to enable running tests from any client
  • Concepts of HA and scalability are covered
  • Use the solution presented or modify it as per your organizational needs

Target Audience

Anyone interested in a technical deep-dive into how the solution came together and the many scaling problems it aims to resolve

Prerequisites for Attendees

Good if the audience has familiarity with Selenium Grids in general or even industry solutions for browser testing such as Sauce Labs or Browser Stack.

schedule Submitted 10 months ago

Public Feedback

    • Krishnan Mahadevan

      Krishnan Mahadevan - My experiments with Grid

      45 Mins

      Everyone starts off with a simple grid setup which involves a hub and one or more nodes.

      This traditional setup is a good start but the moment one starts to get serious with the selenium grid and decide to house their own selenium grid for their local executions, that is when issues start.

      My experiences with the Selenium grid in the past couple of years has led me to get introduced some of the most prevalent problems with maintaining an in-house selenium grid.

      • Nodes get unhooked randomly due to network glitches.
      • Nodes introduce false failures due to memory leaks.
      • Selenium Grid running out of capacity.
      • Nodes require OS upgrades/patches etc.
      • Needing to deal with auto upgrades by browsers (especially chrome and firefox)

      Some of these issues I managed to fix by building a "Self Healing" Grid wherein the nodes automatically get restarted after they have serviced "n" tests. But that still didn’t solve many of these other problems.

      That was when I felt, what if there was an on-demand selenium grid.

      What if the Grid could do the following ?

      • The Grid auto scales itself in terms of the nodes based on the current load.
      • The Grid does not require a lot of infrastructure to support it.
      • The Grid can plug itself into some of the cloud providers or leverage a solution such as Docker so that the nodes can be spun and shutdown at will.

      That was how the idea of "Just Ask" an on-demand grid was born.

      Just-Ask is an on-demand grid. It has no nodes attached to it.

      It’s designed to spin off nodes on demand, run test against the newly spun off node and after test runs to completion, clean-up the node as well. The node can be backed by anything. It could be Docker (or) it could be a VM running on any of the popular clouds.

      The session aspires to walk the audience through with my experiments with the selenium grid, my learnings on the selenium grid internals and how I used all of that knowledge to build my own On Demand Selenium Grid. What better avenue to share these learnings than a Selenium Conference.

      The session will introduce the audience to the grid internals and their concepts such as

      • What is a Selenium Remote Proxy ? What is it used for? What can you do with it?
      • What is a Hub (or) Node level Servlet ? When would you need one ?
      • All of this followed by a quick demo on "Just Ask", the on-demand grid that I have built and open sourced here: