From in-house testing to the cloud - how to scale your device farm

Instead of buying test devices and maintaining them on-premise, there is an ongoing trend to facilitate cloud providers for UI test automation. This talk will explain the basics and provide an overview of the advantages and limitations of this approach. It will also tell the story of how we moved from running a handful of tests against four of our own test devices to executing 100+ scenarios (and counting) on AWS Device Farm, Browserstack and a local simulator farm.

 
 

Outline/Structure of the Case Study

  • What are the options for end-to-end mobile testing?
  • Simulators vs. real devices
  • In-house vs. cloud farm
  • Code examples and demos (Java + Appium)
  • Pricing models
  • Tool integration and API support
  • Criteria for vendor selection

Learning Outcome

Attendees will learn about the pros and cons of using a commercial cloud testing platform vs. building their own in-house solutions. They will get an overview of the current market situation and the main cloud testing solutions which support Appium. They will also receive a simple set of criteria for selecting a suitable vendor for their own use-cases.

Target Audience

Everyone looking into scaling up their device farm

schedule Submitted 7 months ago

Public Feedback

comment Suggest improvements to the Speaker

  • Liked Srinivasan Sekar
    keyboard_arrow_down

    Srinivasan Sekar / Sai Krishna - Native mobile commands in Appium

    45 Mins
    Tutorial
    Intermediate

    Apple and Google’s test automation framework does not natively support W3C standards for few web driver spec implementations directly for e.g TouchActions interface in XCTest, etc. Although test automation frameworks support a rich set of those functions specific to platforms, Appium does provide ways to directly invoke these functions e.g gestures, biometric handling, etc.

    Many special behaviors and workarounds are made available and achieved only through executing platform-specific native commands of Appium. For instance, there are 100+ issues been reported on date picker or handling picker wheel in the appium organization but it can be achieved quite easily by executing native mobile commands.

    There are so many that testers might not get chance to go through each one of these and get acquainted with all of those. Native mobile commands help to handle much complex use cases like biometric handling, talking to Siri, performance profiling, etc quite easily.

  • Liked Jonah Stiennon
    keyboard_arrow_down

    Jonah Stiennon - All Desired Capabilities

    Jonah Stiennon
    Jonah Stiennon
    Partner
    Cloud Grey
    schedule 9 months ago
    Sold Out!
    90 Mins
    Tutorial
    Intermediate

    Appium has so many desired capabilities! How many exactly? Let's count!

    Join Appium contributor Jonah Stiennon as he iterates through every desired capability supported by Appium.

    Many special behaviors and specific workarounds are made available only through desired capabilities. Often the key to selecting an element reliably, launching an app, or avoiding a timeout, is picking the right set of desired capabilities. There's so many that testers don't get the chance to sit down and become acquainted with them all.

    There's too many to easily memorize and documentation can be sparse. Jonah will introduce novel ways to classify, visualize, and organize all of the desired capabilities, making it easier to find ones which can be useful.

  • 45 Mins
    Case Study
    Intermediate

    Experimentation and quick feedback is the key to success of any product, while of course ensuring a good quality product with new and better features is being shipped out at a decent / regular frequency to the users.

    In this session, we will discuss how to enable experimentation, get quick feedback and reduce risk for the product by using a case study of a media / entertainment domain product, used by millions of users across 10+ countries - i.e. - we will discuss Testing Strategy and the Release process an Android & iOS Native app - that will help enable CI & CD.

    To understand these techniques, we will quickly recap the challenges and quirks of testing Native Apps and how that is different than Web / Mobile Web Apps.

    The majority of the discussion will focus on different techniques / practices related to Testing & Releases that can be established to achieve our goals, some of which are listed below:

    • Functional Automation approach - identify and automate user scenarios, across supported regions
    • Testing approach - what to test, when to test, how to test!
    • Manual Sanity before release - and why it was important!
    • Staged roll-outs via Google’s Play Store and Apple’s App Store
    • Extensive monitoring of the release as users come on board, and comparing the key metrics (ex: consumer engagement) with prior releases
    • Understanding Consumer Sentiments (Google’s Play Store / Apple’s App Store review comments, Social Media scans, Issues reported to / by Support, etc.)

  • Liked Sravan Kumar
    keyboard_arrow_down

    Sravan Kumar - Bootcamp to understand Appium android internals

    Sravan Kumar
    Sravan Kumar
    Sr. Software Engineer
    Badoo
    schedule 9 months ago
    Sold Out!
    45 Mins
    Tutorial
    Intermediate

    Appium is a world's most popular open source mobile test automation framework developed using WebDriver protocol and I am fortunate enough to get a chance to contribute to appium-uiautomator2-server and appium-uiautomator2-driver modules.

    The goal of the session is to help the Appium community to have a better understanding of Appium and how it works.

    In this session, I will be discussing Appium architecture specifically towards its Android modules(UiAutomator2 and Espresso), how the communication happens between Appium modules and how to add new handler/API in appium

  • Liked Sai Krishna
    keyboard_arrow_down

    Sai Krishna / Srinivasan Sekar - Life Cycle of an Appium command

    45 Mins
    Talk
    Beginner

    Every command you write using Appium Client Library will be converted to JSON Wire Protocol over HTTP which in turn will be passed to the drivers(AndroidDriver, IOSDriver). The respective driver doesn’t just send or execute the command directly on the device. Appium has a Lifecycle of modules which sends the command to the devices.

    Abstract:

    The goal of the session is to help the Appium community to have a better understanding of Appium and how it works internally.

    Will discuss on following aspects:

    • What happens under the hood when the session is created, mainly how appium connect to the corresponding device, installing the app, launching the app and the different steps involved in the process.

    • Command life cycle - How client request converted into an HTTP request, how received request passes through appium drivers(ios-driver & AndroidDriver) and the respective driver don’t just send or execute the command directly on the device. Appium has a Lifecycle of modules which sends the command to the devices.

    • How the final response passed to the client through different layers.
  • Liked Aisyah Dzulqaidah
    keyboard_arrow_down

    Aisyah Dzulqaidah - Cloud Based Mobile Application Testing with Appium

    Aisyah Dzulqaidah
    Aisyah Dzulqaidah
    TEST ENGINEER
    BBM
    schedule 9 months ago
    Sold Out!
    20 Mins
    Demonstration
    Beginner

    Testing our mobile application using real device is not scalable and need high maintenance. As the test is bigger , we need to make our test is parallel to make it faster and efficient. We also need to test it agains many variety of device and OS version. Running testing using device cloud is the solution, so in this session I will show you how to setup and run parallel automation testing using Android device emulator and iOS simulator. In this case, I wil show the demonstration using Genymotion Cloud, Appium, and Robot Framework.

  • Liked Anand Bagmar
    keyboard_arrow_down

    Anand Bagmar - Build your own MAD-LAB - for Mobile Test Automation for CD

    45 Mins
    Case Study
    Intermediate

    In this age of a variety of cloud-based-services for virtual Mobile Test Labs, building a real-(mobile)-device lab for Test Automation is NOT a common thing – it is difficult, high maintenance, expensive! Yet, I had to do it!

    Attend this talk to understand the Why, What and How I built my own MAD-LAB (Mobile Automation Devices LAB). The discussion will include –

    • Understanding the context,
    • Why no cloud-based solution worked for me,
    • The automation strategy for Android / iOS / Web platform,
    • The tech-stack (cucumber-jvm / Appium), and,
    • The core implementation to build your own MAD-LAB! (already open-sourced)
    • How this results in Continuous Delivery (CD)