Reliable. Scalable. Lightning fast. Running 1 000 000 Android tests with Selenoid.
Mobile platforms nowadays give more than a half of the worldwide Internet traffic. Being the most widespread platform, Android is slightly more difficult to work with from test automation perspective than traditional desktop browser automation. Last autumn in Berlin I have shown Selenoid - an open-source Selenium protocol implementation that dramatically simplifies working with Selenium for desktop browsers. In this talk I would like to demostrate how it is possible to have the same experience on Android platform for mobile application and mobile web testing.
Outline/Structure of the Talk
- What are the main Android automation components
- How Android automation works
- Why real devices are not reliable
- Android emulator limitations and requirements
- How Docker solves common Android automation issues
- Where to get ready to use Android images
- What Selenoid is and how it works
- How to use Selenoid for efficient Android automation, hardware requirements
- Demo
- Notes on scaling this solution
Learning Outcome
- What are the most common Android testing infrastructure pitfalls
- Why reliable Android testing solution can only be built with Android emulators
- Why Docker isolation is needed to run Android emulators in parallel
- What Selenoid is and how to easily start testing on Android platform with it
- How to create a big Android cluster with Selenoid and easily scale it
Target Audience
Test automation engineers, infrastructure teams and anybody working with Selenium.
Prerequisites for Attendees
Basic knowledge about Selenium, Docker and mobile platforms.
Video
Links
- SeleniumConf Berlin 2017 talk about scalable Selenium infrastructure: https://www.youtube.com/watch?v=TGjpc32my0Y
- QAFest 2017 talk about Selenoid (in Russian): https://youtu.be/wAKcBinMn6o
- Other talks do not have public videos for the moment
- Slideshare page: https://www.slideshare.net/IvanKroutov
- Medium.com article about recent Selenoid features: https://medium.com/@ivan_krutov/selenium-growing-muscles-9df57ee77a3a
schedule Submitted 3 years ago
People who liked this proposal, also liked:
-
keyboard_arrow_down
Srinivasan Sekar / Sai Krishna - Advanced Appium
Srinivasan SekarLead ConsultantThoughtWorksSai KrishnaLead ConsultantThoughtWorksschedule 3 years ago
480 Mins
Workshop
Intermediate
At this workshop, you'll learn about advanced concepts in Appium. We will learn how to write a single script for Android, iOS and mobile web apps. Once our tests are established, we will work on framework design and report. We will also have a detailed look at how can we automate various mobile gestures using TouchActions and mobile endpoints in Appium and running distributed tests in parallel on various emulators and simulators.
-
keyboard_arrow_down
Srinivasan Sekar / Sai Krishna - Code Once Test Anywhere: On Demand Private Appium Device Cloud using ATD
Srinivasan SekarLead ConsultantThoughtWorksSai KrishnaLead ConsultantThoughtWorksschedule 3 years ago
45 Mins
Case Study
Beginner
Mobile Test Automation is increasingly becoming very important. Almost all web applications are responsive these days and it's very important to test how the application works across devices. The same is true with the native application as well. At the same time, the number of devices and the custom OS versions on devices are also vast. This means that it's harder for a tester to manually run the automated tests over a list of devices to get device coverage and quicker results over every feature development.
We came up with a solution of executing tests in distributed or parallel fashion across remote devices from anywhere in the network using Appium Test Distribution. Same framework is officially used by Appium members for Beta testing of Appium.
USP of ATD over other Market Solutions:
- Device Cloud:
- Setup Devices anywhere within a network, ATD executes remotely without Grid
- Never worry about device location in network.
- Plug and Play:
- Connect your android/iOS devices or Emulators/Simulators and just execute tests.
- Multiple TestRunner:
- TestNG and Cucumber
- Parallel Test Execution:
- Runs across all connected iOS, Android real devices and Simulators/Emulators
- Test Coverage:
- Parallel(Run entire suite across all devices, which gives device coverage)
- Distribute(Run tests across devices to get faster feedback).
- Device Management:
- Manage devices remotely using Device Manager.
- Reporting:
- Covers detailed crashes logs from Android and iOS.
- Appium Servers logs
- Screenshots on Failures and On Demand Video logs
- Reporting Trends for multiple builds
- Manual Access to Remote Devices - OpenSTF support
Who loves/uses ATD?
ThoughtWorks, CeX, Jio, TravelStart, M800, Reward Gateway and lot more.
- Device Cloud:
-
keyboard_arrow_down
Rajdeep varma - Android Application Backdoor via Appium
45 Mins
Demonstration
Advanced
Application Backdoor via Appium
There's a shift towards open-source mobile test automation tools happening today among developers and QAs. Whether it be Appium, Calabash or anything else: all are good, with some major limitations.
While a chosen tool may work well when you first start using it, things can quickly get out of hand with changing business requirements. We started using Calabash at Badoo when there was no Appium. Given the capability of Appium to drive the whole device, we started automation of new apps with Appium. However, we realized a powerful feature was missing in Appium for Android! : The ability to call Application code from automation code like Calabash Backdoors.
As Appium UiAutomator server is based on instrumentation, we modified it such that we could instrument our app under test. This gave us the power to access context of Application under test and invoke public methods of Activity using reflection APIs. We use these methods to setup app state, seed DB OR even enable/disable some client-side A/B tests. This makes our application more testable and our tests more predictable.
This talk is going to be about how I achieved the above solution and benefits of backdoors.
There will be a small demo and code!
-
keyboard_arrow_down
Nalilnikanth Meesala / Srinivasan Sekar - New hope in Selenium Docker
Nalilnikanth MeesalaSr. Quality AnalystThoughtWorksSrinivasan SekarLead ConsultantThoughtWorksschedule 3 years ago
45 Mins
Demonstration
Advanced
Idea behind this talk is to describe how to create a short lived containers for each test and scale to large Selenium Cluster using Selenoid. What is standard Selenium architecture and why it is not suitable for big clusters, Also states issues in maintaining a large Docker Selenium Grid / Selenium Clusters. In this session we will also be talking about the challenges we faced in using official selenium docker images and how we scaled up regression execution time from 3:3 (3 hours to 3 mins). Followed by a demo of how Selenoid server solves the problem with docker. How it works, how to use inside big Selenium cluster, where else it could be used.
-
keyboard_arrow_down
Ivan Krutov - Selenoid: get rid of Selenium server!
90 Mins
Tutorial
Beginner
Browser tests are known to be the flakiest ones. This is partly because browser infrastructure is complicated to maintain. But the second reason is – main stream browser automation tools such as Selenium server are far from being efficient.
As your company grows – your browser automation infrastructure should easily adapt to increased loads and new requirements. Unfortunately this is not so easy to get really scalable Selenium cluster using only the main stream tools. But there are some new open-source tools and approaches that can be used to deliver really scalable Selenium cluster.
This tutorial covers an emerging browser automation tool – Selenoid, a truly efficient replacement of the standard Selenium server. I will explain how it works and why it is so fast. I will show in details features like video recording, sending logs and statistics to centralized storage and how to easily visualize this data.
-
keyboard_arrow_down
Nalilnikanth Meesala / Prasad Mudedla - Continuous Security with Selenium tests
Nalilnikanth MeesalaSr. Quality AnalystThoughtWorksPrasad MudedlaQuality ConsultantThoughtWorksschedule 3 years ago
45 Mins
Demonstration
Intermediate
We have come to a place where we run our selenium tests on pipelines every day.
Now are these tests confined to test only the functionality of the application?
No, we can use this tests to do much more. Let's test security with those same tests.Turn your selenium tests in to Security tests, get security testing in to your CI pipelines just by tweaking a bit of the selenium integration suite. Find and report security issues that exists which can be caught way before paying off for it.
In this Demo we will introduce a framework that will help your selenium tests turn into security tests and run on pipelines.
With ZAP API + selenium and CI pipelines, find security issues on the application see a red or green security pipeline. Not just that fetch the reports and take necessary actions on every build.
The talk covers different ways that ZAP tests the web application for security(automated with selenium tests). We will see what all of them can be integrated with selenium and can make most out the suite.
As an add we will see how your API can be tested for security using ZAP API :)
Tools used:
1. Selenium Web driver framework.
2. ZAP framework integrated with selenium framework.
3. GO (for CI/CD)
4. Maven for building and fetching reports. -
keyboard_arrow_down
Artem Eroshenko - Allure Framework: new generation test report tool
45 Mins
Talk
Beginner
Everyone involved in the automated testing has to examine the autotests results. Sometimes it's hard to understand whats going on in your test run. Is test failed because of test problem or product bug? Which tests are unstable (flaky)? How many new problems discovered in last run? Why current test run took more time than the previous one?
In this talk Artem Eroshenko, Allure Core Developer, will describe how to solve mentioned problems using Allure Framework, present a new major version of Allure and how to integrate Allure with other tools.
New Allure version brings lot of improvements to the report. Now you can adapt Allure to suite your needs using a plugin system. Besides that, it has a lot of interesting features: test history and retries, problem categories, environments, test analytics and trends, popular tools integrations and so on.
Public Feedback