Serverless - how to speed up tests over 300 times and achieve continuous feedback?
Automated tests can provide results faster and it’s possible to execute them more frequently than manual ones. That allows to test earlier in the development process, decrease overall time needed for tests and what is probably the most important it’s possible to release and deliver business value faster and more frequently.
But what if we have more and more tests and even automated execution of them takes too much time - 10 minutes... 30 minutes... maybe even hours? Should we consider the ability to execute full tests set just a few times a day as something normal? Is adding more compute resources the only option to reduce the execution time? Or maybe there are too many high-level tests and some of them should be replaced by low-level ones according to tests pyramid? Is the tests pyramid still valid in the cloud world?
During the presentation you will see how the serverless cloud services like AWS Lambda may be used to run tests in the highly parallelized environment that can speed up execution even hundreds of times.
Outline/Structure of the Talk
- Introduction
- Advantages of Automated Tests
- Problems
- Possible workarounds
- Case study: 500 E2E Selenium tests, 20 seconds each
- Parallel test execution (AWS Lambda) - implementation
- Summary
Learning Outcome
How to speed up test execution even hundreds of times with serverless cloud services like AWS Lambda?
Target Audience
People related to Test Automation
Video
Links
Previous talks:
- TestWarez 2019 - “Serverless - Automated Tests of web apps in Cloud world”,
- PyCode 2019 - “Serverless - Automated Tests of web apps in Cloud world”,
- Testaton 2019 - „Docker z perspektywy QA” ("Docker from QA's perspective"),
- Dev.js Warszawa #10 - “Test Environments Management with Docker”,
- 4Developers 2019 - "Insecure by design",
- ConSelenium 2019 - „Docker z perspektywy QA” ("Docker from QA's perspective"),
- TestCon Moscow 2019 - "Test Environments Management with Docker",
- Testing Stage 2019 - "Insecure by design",
- Warsaw IT Days 2019 - „Docker z perspektywy QA” ("Docker from QA's perspective"),
- TestFest 2019 - „Docker z perspektywy QA” ("Docker from QA's perspective"),
- What The [email protected] 2018 - "Insecure by design",
- Testing United Bratislava 2018 - "Insecure by design",
- TestWarez 2018 - "Insecure by design",
- Silesia Blockchain Meetup #4 - "Blockchain scaling - Lightning Network",
- Test Camp 2018 - "Insecure by design",
- Devoxx Poland 2018 - "Blockchain scaling - Lightning Network",
- Noc Informatyka 1.1 - "Insecure by design",
- ConSelenium 2018 - „Architektura frameworka testowego - E2E aplikacji webowych” ("Test framework architecture - web applications' E2E"),
- TestWarez 2017 - „Architektura frameworka testowego - E2E aplikacji webowych” ("Test framework architecture - web applications' E2E"),
- Uszanowanko Programowanko Meetup #19 - „Architektura frameworka testowego” ("Test framework architecture"),
- TestWarez 2016 - „Testowanie algorytmów wizyjnych” ("Testing vision algorithms"),
- Quality Excites 2016 - „Testowanie algorytmów wizyjnych” ("Testing vision algorithms")
schedule Submitted 2 years ago
People who liked this proposal, also liked:
-
keyboard_arrow_down
Simon Stewart - Selenium: State of the Union
45 Mins
Keynote
Intermediate
What's the latest in the world of Selenium since the last SeleniumConf? In this talk, we'll cover Selenium 4, how the project is run, how you can participate, reminiscing on the Old Days, and reflect on something that at first doesn't appear to be related to Selenium, yet is intimately related to it.
-
keyboard_arrow_down
Gopinath Jayakumar / Babu Narayanan Manickam - Expanding boundaries of WebDriver with DevTools Integration
Gopinath JayakumarSoftware ArchitectTestLeaf SoftwareBabu Narayanan ManickamDirectorQeagle Assuranceschedule 2 years ago
45 Mins
Demonstration
Beginner
Problem Statement
Though Selenium taking most of the stake in the UI test automation tool market comfortably, there were always challenges that were for selenium test automation engineers are handicapped with especially when dealing with modern JS technologies. For example,
- dealing with DOM elements to solve the stale / loading / non-interactable elements,
- handling full screenshots to know how the elements at the left, bottom, etc,
- measuring the performance of request and response resources at different speeds,
- monitoring the memory of the pages, controls, etc,
- attaching to an existing browser for debugging the failed scripts and many more.
These problems were largely resolved with the integration of selenium with devtool protocols. And that makes the selenium engineer's life merrier than before.
Why this proposal can be different from others?
- Our solution can be executed as independent with chrome dev tools or with selenium. That gives the power back to the automation engineer to choose what and how to debug/run their tests.
- We used this solution for one of our largest enterprise customers and moved this solution to public repository this week (for this conference and beyond). With that said, we tested reasonably with more than 10,000+ test scripts and more than 1M tests.
- The present solution is completely (100%) packed with all Chrome Dev Tools API in Java and with that said, any Java Selenium automation engineer can bind in minutes for their existing code base with no additional dependencies.
- Finally, we love to present at the local home to start our selenium conference campaign. Where else?
Solution:
The present proposal largely connected with Chrome and Selenium in Java language. However, there is no limitation to expand the boundaries for other language bindings and browsers.
Google Chrome, the most picked browser for browsing, which makes it the primary concentration for developers and testers. DevTools is one such boon for developers, testers especially the new aged test automation engineers. With that said, we built the following design pattern to allow chrome dev tools API to marry Selenium using debugger address / remote targets.
-
keyboard_arrow_down
Virender Singh - Rise of Shadow DOM - Lets solve it through WebDriver
45 Mins
Demonstration
Beginner
Shadow DOM is kind of web standard that developers use to encapsulate their custom HTML code and style components so that no other global style components can override their code. It ensures that a component will work in any environment, even if other CSS or JavaScript is run on the page i.e. Shadow DOM hides away the complexity of the components from the rest of the page because it renders separately from main DOM.
Selenium legacy selectors are not able to interact these custom elements under the shadow DOM so we need java script to interact with shadow DOM elements and we also require JavascriptExecutor interface for their execution.
We will also talk about the WebdriverIO tool v5.5.0, which implicitly support Shadow DOM elements.
-
keyboard_arrow_down
Gaurav Singh - How to build an automation framework with Selenium : Patterns and practices
45 Mins
Talk
Beginner
With an ever increasing no of businesses being conducted on web the testing need to write automated tests for the app's UI is something that can never be ignored. As you all know Selenium provides an API that enables us to do this quite effectively.
However, when tasked with setting up the automation framework, there are a lot of questions that arise in the minds of aspiring test developers regardless of what level they are in their career.
Some of such questions are:
- How does one actually go about the business of building a robust and effective automation framework on top of selenium?
- What are the elementary building blocks to include in the framework that an aspiring automation developer should know of?
- How should we model our tests? XUnit style vs BDD?
- Are there good practices, sensible design patterns and abstractions that we can follow in our code?
- What are some of the anti patterns/common mistakes we should avoid
A lot of literature, documentation and blogs exists on these topics on the web already.
However In this talk,
I would combine this existing knowledge and my years of experience in building automation frameworks and breakdown these elements and walk you through exactly the sort of decisions/considerations and practices that you can take while starting to implement or improve the UI automation for your team.
Hope to see you there!
-
keyboard_arrow_down
Benjamin Bischoff - Smoke tests and mirrors - magic principles in test automation
45 Mins
Talk
Beginner
Having been a hobbyist magician for 34 years, I started noticing that some close-up and stage magic principles and theories don't only apply to magic tricks but also to software testing.
An example of this is the application of the KISS principle (Keep It Simple, Stupid) in software development. This helps to avoid unnecessary code complexity and make it easy to understandable and follow. In magic, this applies as well. Magicians strive for clarity and simplicity by creating effects that are easy to follow for spectators.
In this sessions, I will try to connect those two worlds and thus combine two of my passions. Also, I will try my best to illustrate the magic side of things with demonstrations. This is a challenge I set for myself - let’s see how it goes.
-
keyboard_arrow_down
Tomasz Wojciechowski / Tomasz Wojtuń - How to run 2500 selenium tests in an hour.
Tomasz WojciechowskiSr. QA EngineerStepStone Services Sp.z o.o.Tomasz WojtuńDevOps EngineerStepStoneschedule 2 years ago
45 Mins
Case Study
Advanced
If you are tired of waiting for your selenium tests status and old fasioned ways of reporting tests status, you are in a good place.
We will show you our test environment setup and how we run over 2500 tests in about an hour. Not cool enough? Take a look how we make reports via slack.
Do you want to check if we can be even more faster?
-
keyboard_arrow_down
Tomasz Wojciechowski - Quality Assurance is more than testing
45 Mins
Talk
Intermediate
Have you ever heard "you are just a tester"? Have you ever felt being a tester is just a stop for "better" position? Are you afraid someday test automation will take your job? Do you feel you could bring more value for your team but don't know how?
Hi, I'm BeardedQA and I have some answers.
During this talk, I'll share my experience about moving from Manual Tester to Quality Assurance Engineer position, what quality assurance really is and how to give more value for your team.
Public Feedback