From Test Pyramid to Test Cup : Evolution of Modern Testing Stack

We all have read or heard about test automation pyramid .But is it really supposed to be a pyramid ? or it's just a metaphor? lets find out

At this talk we will share our learnings from implementing test automation pyramid for years and its variations finally arriving to test automation cup modal focusing on

1.Need for growing up from automation testing pyramid

2.How the modern testing stack has evolved

3.How to choose and design your automation testing stack

4.Real World examples from testing to validate our test cup model

5.How to move from Pyramid to test cup and Our lessons learned

6.Sharing Best practices ,limitations and patterns for success


Outline/Structure of the Case Study

This talk/case study will demonstrate how to leverage modern test stack for moving to test automation cup model with key focus on:

1.Problems with Test Automation Pyramid
2.Need for growing up from automation testing pyramid
3.Examples with steps to Implement automation cup model with open source project.

4.Live Lessons learned from our past journey
5.Best practices ,limitations and patterns for automation cup model success.

Learning Outcome

1.Automation cup model, how it works and why its important to testing
2.Modern test stack with automation cup model and key improvements
3.Steps to Align your existing tests with automation cup model
4.Integrate automation cup model to drive Continuous Delivery along with DevOps tools
5.Ready to start examples for taking your projects ahead with automation cup model

Target Audience

Testers,Developers and Product Managers



schedule Submitted 1 year ago

Public Feedback

    • Simon Stewart

      Simon Stewart - Selenium: State of the Union

      45 Mins

      What's the latest in the world of Selenium since the last SeleniumConf? In this talk, we'll cover Selenium 4, how the project is run, how you can participate, reminiscing on the Old Days, and reflect on something that at first doesn't appear to be related to Selenium, yet is intimately related to it.

    • Jim Evans
      Jim Evans
      Selenium Project
      schedule 1 year ago
      Sold Out!
      45 Mins

      Since nearly the very beginning, the software industry has prized the concept of the visionary, the extraordinary creator, the lone person who relentlessly pursues their goal against impossible odds, all to help make the world a better place through technology. The industry also reveres the concept of dedication to the software above all else in the lives of those visionaries. Even those who have chosen to shepherd large portions of popular open-source projects are often seen as people who have a special quality about them that places them above other developers. These concepts are myths and oftentimes dangerous ones.

    • Rajdeep varma

      Rajdeep varma - The Joy Of Green Builds - Running Tests Smartly

      Rajdeep varma
      Rajdeep varma
      Automation Lead
      schedule 1 year ago
      Sold Out!
      45 Mins

      So you have got a few UI tests and they are running in parallel, great! However, life will not be so sweet once these 'a few' turns into 'a lot'. We grew from a few to 1500 UI tests (although not particularly proud of this number, there are situations and reasons)

      We started with a simple parallel distribution of tests 3 years ago. As test count increased failure count run time increased along with increased flaky tests. Mobile tests had their own challenges (eg. device dropping-off, random wi-fi issues, etc) To keep up with this, we created a queue and workers based solution which could distribute the tests more efficiently ( Over time, we made more improvements, in particular:

      • Segregation of failures based on infrastructure issues and re-queue the tests
      • If a device/emulator malfunction, rescue the tests to another device
      • Repeating a single test on 100s of the worker in parallel to detect flakiness
      • Repeat a test if a known network issue
      • Terminating the build early if more than a certain number of tests have failed
      • Health check of each device, before each test to ensure reliability
      • Muting a test if failure is known, and highlight outdated mutes if the related task is fixed

      In this talk, I will talk about the initial challenges with running UI tests in parallel (Selenium and Appium), how we approached the queue based solution and continuous improvement of this solution; finally, how attendees can use it at their workplace or create their own solution based on our learnings.

    • Shama Ugale

      Shama Ugale - Webdriver connector for Botium - Tool for testing Conversational UI

      Shama Ugale
      Shama Ugale
      Sr. QA Consultant
      schedule 1 year ago
      Sold Out!
      45 Mins

      Last year was dominated by the smart devices & voice based home assistants. These use the conversational interfaces unlike other applications to interact, built using advanced algorithms, ranging from natural language processing to AI/ML techniques. They are constantly learning by themselves improving the interactions with the user bringing up the challenge of non-deterministic output. To such interfaces, natural language is the input & we humans love having alternatives & love our synonyms, express using emojis gifs & pictures. Testing in this context moves to clouds of probabilities.

      Unfortunately Selenium cannot be used to automate such systems and hence Botium was designed.
      In this session I will cover the Selenium driver for Botium to automate E2E tests on Web UI and mobile along with testing strategy, testing NLP models & automating these tests to the CI/CD build pipelines with a DialogFlow based 'Coffee-Shop bot' as an example during my demo.