Hyper-Parallelism - The Panacea of Automated Testing?
When it comes to most things in life, most people tend to think more is better. But does this maxim hold true to automated testing? Should you test every possible browser/OS combination possible with every functional workflow because an executive thinks it’s a good idea? Does this mean you need to build the biggest Selenium you can grid to test on the most OS and browser combinations as possible? Or maybe even leverage a 3rd party infrastructure solution?
Testing on as many platforms as possible may not always be the best approach to test execution, even though it may seem that way at first. The best approach is to test strategically, not just indiscriminately testing as much as possible just because you can. Sometimes this means going big and testing at a massively parallel scale, others times this may not be the best approach.
When considering how much to parallelize your tests, there are many things to think about, including how well your framework supports parallel execution, how robust your execution environment is, and how much load your non-prod environments can handle. All of these factors will impact how to parallelize your tests.
I will discuss the best approach to determine how to optimize test execution parallelization, both in terms of what considerations and tradeoffs to make and also how to set implement parallelized testing common frameworks.
Topics that I will cover include:
- Use of Google Analytics and other site data to drive platform testing needs
- Test structure & framework choice and their impact running tests in parallel
- Situations when massively parallel testing is appropriate & pitfalls of over-parallelization
- Determining best approaches and coverage models for unit, smoke, integration, and regression testing.
- A brief demonstration of parallelization approaches in several common frameworks, taking theory and putting it into action
Outline/Structure of the Talk
- How to determine what platforms your users require
- How to determine what platforms to test on
- Gaining an understanding of your capacity to test in parallel
- Signs to look for when testing in parallel, i.e., are you testing too broadly
- Test structure to optimize parallel efficiency
- How to parallelize in common frameworks (demo)
Attendees should take away:
- How to strategically test in parallel, more isn't always better
- Things to avoid when testing parallel - common pitfalls
- How to structure your tests to optimize efficiency for parallalization
QA Engineers, infrastructure people who maintain non-prod systems
Prerequisites for Attendees
- Selenium knowledge
- An understanding of parallel testing and how to implement in frameworks
schedule Submitted 4 years ago
People who liked this proposal, also liked:
Dan Rabinovitz - From Point & Click to Pinch & ZoomDan RabinovitzSr. Solution Engineer in Sales ConnecticutSauce Labs
schedule 4 years agoSold Out!
In today’s fast paced world, if your mobile website or native app isn’t working as well as it should, you’re probably losing business. It’s important for your mobile website to function as well as your desktop website while offering similar capabilities.
However, when moving from traditional to mobile tests, it’s not always as straightforward as simply changing selectors to find new elements IDs.. There are a number of questions you need to ask yourself. How do you find new element IDs? Are you embedding another app (like a Google Map)? Does your mobile website use geo location? Android or iOS? How do you test for all of this? These questions, along with many others, lead to some very specific considerations when considering mobile testing strategy.
All of these differences require a different approach to selectors. During this session, I will explore the answers to a few key questions:
– How to determine if your selectors are the same between desktop and mobile web.
– Finding selectors in native and hybrid apps using Appium.
– Understanding when the switch needs to be made between native and web contexts.
– Determining an execution engine once your selector strategy is set, and deciding between emulators, simulators or real devices.