Reliable. Scalable. Lightning fast. Running 1 000 000 Android tests with Selenoid.

schedule Jun 29th 11:30 AM - 12:15 PM place Kalinga Hall 2 people 32 Interested

Mobile platforms nowadays give more than a half of the worldwide Internet traffic. Being the most widespread platform, Android is slightly more difficult to work with from test automation perspective than traditional desktop browser automation. Last autumn in Berlin I have shown Selenoid - an open-source Selenium protocol implementation that dramatically simplifies working with Selenium for desktop browsers. In this talk I would like to demostrate how it is possible to have the same experience on Android platform for mobile application and mobile web testing.

 
10 favorite thumb_down thumb_up 4 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

  • What are the main Android automation components
  • How Android automation works
  • Why real devices are not reliable
  • Android emulator limitations and requirements
  • How Docker solves common Android automation issues
  • Where to get ready to use Android images
  • What Selenoid is and how it works
  • How to use Selenoid for efficient Android automation, hardware requirements
  • Demo
  • Notes on scaling this solution

Learning Outcome

  • What are the most common Android testing infrastructure pitfalls
  • Why reliable Android testing solution can only be built with Android emulators
  • Why Docker isolation is needed to run Android emulators in parallel
  • What Selenoid is and how to easily start testing on Android platform with it
  • How to create a big Android cluster with Selenoid and easily scale it

Target Audience

Test automation engineers, infrastructure teams and anybody working with Selenium.

Prerequisite

Basic knowledge about Selenium, Docker and mobile platforms.

schedule Submitted 7 months ago

Comments Subscribe to Comments

comment Comment on this Submission
  • Anand Bagmar
    By Anand Bagmar  ~  7 months ago
    reply Reply

    Hi Ivan,

    Seems like an interesting proposal. Would you be doing any demos as well?

    Also, you seems to be talking about a lot of things - Android automation components, automation, emulators Vs real, docker & Selenoid. Would it not help to focus on few items from this list so that you can get enough time to get deep into the core areas of your proposal?

    • Ivan Krutov
      By Ivan Krutov  ~  7 months ago
      reply Reply

      My current idea is to briefly describe how Android automation works and certainly spend most of the time showing the new solution. So if you take a look at the outline - the first half could take e.g. 12-15 minutes, and the second half - the rest of the talk. This second part will certainly include the demonstration - more probably in the form of one or two recorded videos with my comments (like it was in Berlin).

      • Anand Bagmar
        By Anand Bagmar  ~  7 months ago
        reply Reply

        Thanks for clarification. One more question please - the talk title seems to focus on scaling to run 1000000 Android tests. However, that seems to be the last thing you cover in your outline. Shouldn't there be more focus on the scaling aspect in that case?

        • Ivan Krutov
          By Ivan Krutov  ~  7 months ago
          reply Reply

          Scaling the cluster is important but useless without efficient worker nodes. The overall talk is about creating a scalable solution. For example ready to use Android images for different Android versions are also a part of scalability. Because otherwise you have no way to run different Android versions on the same host simultaneously. The last talk section describes how to run Ggr load balancer in order to use more than one host with Android. So what I'm trying to say - every section of the talk is about scalability and efficiency.


  • Liked Srinivasan Sekar
    keyboard_arrow_down

    Srinivasan Sekar / Sai Krishna - Advanced Appium

    Srinivasan Sekar
    Srinivasan Sekar
    Sr. Consultant
    ThoughtWorks
    Sai Krishna
    Sai Krishna
    Lead Consultant
    ThoughtWorks
    schedule 7 months ago
    Sold Out!
    480 Mins
    Workshop
    Intermediate

    At this workshop, you'll learn about advanced concepts in Appium. We will learn how to write a single script for Android, iOS and mobile web apps. Once our tests are established, we will work on framework design and report. We will also have a detailed look at how can we automate various mobile gestures using TouchActions and mobile endpoints in Appium and running distributed tests in parallel on various emulators and simulators.

  • Liked Srinivasan Sekar
    keyboard_arrow_down

    Srinivasan Sekar / Sai Krishna - Code Once Test Anywhere: On Demand Private Appium Device Cloud using ATD

    45 Mins
    Case Study
    Beginner

    Mobile Test Automation is increasingly becoming very important. Almost all web applications are responsive these days and it's very important to test how the application works across devices. The same is true with the native application as well. At the same time, the number of devices and the custom OS versions on devices are also vast. This means that it's harder for a tester to manually run the automated tests over a list of devices to get device coverage and quicker results over every feature development.

    We came up with a solution of executing tests in distributed or parallel fashion across remote devices from anywhere in the network using Appium Test Distribution. Same framework is officially used by Appium members for Beta testing of Appium.

    USP of ATD over other Market Solutions:

    • Device Cloud:
      • Setup Devices anywhere within a network, ATD executes remotely without Grid
      • Never worry about device location in network.
    • Plug and Play:
      • Connect your android/iOS devices or Emulators/Simulators and just execute tests.
    • Multiple TestRunner:
      • TestNG and Cucumber
    • Parallel Test Execution:
      • Runs across all connected iOS, Android real devices and Simulators/Emulators
    • Test Coverage:
      • Parallel(Run entire suite across all devices, which gives device coverage)
      • Distribute(Run tests across devices to get faster feedback).
    • Device Management:
      • Manage devices remotely using Device Manager.
    • Reporting:
      • Covers detailed crashes logs from Android and iOS.
      • Appium Servers logs
      • Screenshots on Failures and On Demand Video logs
      • Reporting Trends for multiple builds
    • Manual Access to Remote Devices - OpenSTF support

    Who loves/uses ATD?

    ThoughtWorks, CeX, Jio, TravelStart, M800, Reward Gateway and lot more.

  • Liked Rajdeep
    keyboard_arrow_down

    Rajdeep - Android Application Backdoor via Appium

    Rajdeep
    Rajdeep
    Sr. Test Automation Engineer
    Badoo
    schedule 8 months ago
    Sold Out!
    45 Mins
    Demonstration
    Advanced

    Application Backdoor via Appium

    There's a shift towards open-source mobile test automation tools happening today among developers and QAs. Whether it be Appium, Calabash or anything else: all are good, with some major limitations.

    While a chosen tool may work well when you first start using it, things can quickly get out of hand with changing business requirements. We started using Calabash at Badoo when there was no Appium. Given the capability of Appium to drive the whole device, we started automation of new apps with Appium. However, we realized a powerful feature was missing in Appium for Android! : The ability to call Application code from automation code like Calabash Backdoors.

    As Appium UiAutomator server is based on instrumentation, we modified it such that we could instrument our app under test. This gave us the power to access context of Application under test and invoke public methods of Activity using reflection APIs. We use these methods to setup app state, seed DB OR even enable/disable some client-side A/B tests. This makes our application more testable and our tests more predictable.

    This talk is going to be about how I achieved the above solution and benefits of backdoors.

    There will be a small demo and code!

  • Liked NaliniKanth M
    keyboard_arrow_down

    NaliniKanth M / Srinivasan Sekar - New hope in Selenium Docker

    45 Mins
    Demonstration
    Advanced

    Idea behind this talk is to describe how to create a short lived containers for each test and scale to large Selenium Cluster using Selenoid. What is standard Selenium architecture and why it is not suitable for big clusters, Also states issues in maintaining a large Docker Selenium Grid / Selenium Clusters. In this session we will also be talking about the challenges we faced in using official selenium docker images and how we scaled up regression execution time from 3:3 (3 hours to 3 mins). Followed by a demo of how Selenoid server solves the problem with docker. How it works, how to use inside big Selenium cluster, where else it could be used.

  • Liked Ivan Krutov
    keyboard_arrow_down

    Ivan Krutov - Selenoid: get rid of Selenium server!

    Ivan Krutov
    Ivan Krutov
    Developer
    Aerokube
    schedule 7 months ago
    Sold Out!
    90 Mins
    Tutorial
    Beginner

    Browser tests are known to be the flakiest ones. This is partly because browser infrastructure is complicated to maintain. But the second reason is – main stream browser automation tools such as Selenium server are far from being efficient.

    As your company grows – your browser automation infrastructure should easily adapt to increased loads and new requirements. Unfortunately this is not so easy to get really scalable Selenium cluster using only the main stream tools. But there are some new open-source tools and approaches that can be used to deliver really scalable Selenium cluster.

    This tutorial covers an emerging browser automation tool – Selenoid, a truly efficient replacement of the standard Selenium server. I will explain how it works and why it is so fast. I will show in details features like video recording, sending logs and statistics to centralized storage and how to easily visualize this data.

  • Liked NaliniKanth M
    keyboard_arrow_down

    NaliniKanth M / Prasad Mudedla - Continuous Security with Selenium tests

    45 Mins
    Demonstration
    Intermediate

    We have come to a place where we run our selenium tests on pipelines every day.
    Now are these tests confined to test only the functionality of the application?
    No, we can use this tests to do much more. Let's test security with those same tests.

    Turn your selenium tests in to Security tests, get security testing in to your CI pipelines just by tweaking a bit of the selenium integration suite. Find and report security issues that exists which can be caught way before paying off for it.
    In this Demo we will introduce a framework that will help your selenium tests turn into security tests and run on pipelines.
    With ZAP API + selenium and CI pipelines, find security issues on the application see a red or green security pipeline. Not just that fetch the reports and take necessary actions on every build.
    The talk covers different ways that ZAP tests the web application for security(automated with selenium tests). We will see what all of them can be integrated with selenium and can make most out the suite.
    As an add we will see how your API can be tested for security using ZAP API :)

    Tools used:
    1. Selenium Web driver framework.
    2. ZAP framework integrated with selenium framework.
    3. GO (for CI/CD)
    4. Maven for building and fetching reports.

  • Liked Artem Eroshenko
    keyboard_arrow_down

    Artem Eroshenko - Allure Framework: new generation test report tool

    Artem Eroshenko
    Artem Eroshenko
    Developer
    Qameta Software
    schedule 7 months ago
    Sold Out!
    45 Mins
    Talk
    Beginner

    Everyone involved in the automated testing has to examine the autotests results. Sometimes it's hard to understand whats going on in your test run. Is test failed because of test problem or product bug? Which tests are unstable (flaky)? How many new problems discovered in last run? Why current test run took more time than the previous one?

    In this talk Artem Eroshenko, Allure Core Developer, will describe how to solve mentioned problems using Allure Framework, present a new major version of Allure and how to integrate Allure with other tools.

    New Allure version brings lot of improvements to the report. Now you can adapt Allure to suite your needs using a plugin system. Besides that, it has a lot of interesting features: test history and retries, problem categories, environments, test analytics and trends, popular tools integrations and so on.