In the modern era, we have many different cloud testing services to choose from. These cloud services are useful and help reduce the burden of building and maintaining your own Selenium Grid environment. However, there are many scenarios in which you need your tests running locally and quickly, such as you work for the government (or agency), you have sensitive software/data you cannot expose to the cloud, or service costs are too expensive for your organization.

This presentation will feature getting started with setting up your own mobile device grid, running your tests in parallel, running in CI (Jenkins), and the lessons I have learned along the way.

 

 
3 favorite thumb_down thumb_up 2 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

  •  Getting Started
    • Downloading and running a Grid Hub.
    • Getting the needed information about your devices.
    • Connecting your mobile devices (Appium nodes) to the Hub.
  • Running a test on the Grid.
    • Running your tests in parallel.
    • Running your tests distributed.
  • Setting up Jenkins for CI
    • Installing helpful Jenkins plugins (grid, allure)
    • Trigger your tests on commits.
  • Reporting
    • Capturing critical test data. (logcat, appium, screenshots, video)
    • Feeding critical data into Allure reports.

Learning Outcome

Participants will walk away with the understanding of how to start a Selenium Hub, connecting the device Nodes, and running Appium tests in parallel or distributed with Ruby and how to capture critical data from devices. Participants will also gain insight on the challenges and lessons I learned during this endeavour.

Target Audience

Anyone interested in building their own mobile device grid!

schedule Submitted 2 years ago

Comments Subscribe to Comments

comment Comment on this Proposal
  • Anand Bagmar
    By Anand Bagmar  ~  2 years ago
    reply Reply

    will you be doing this as a demo or a talk?

    • Justin Ison
      By Justin Ison  ~  2 years ago
      reply Reply

      I will be doing both. I'll walk through the process of setup and demoing as I go. I'll try to do a live demo if the wifi agrees. if not, I'll have pre-recorded videos.


  • Liked Anand Bagmar
    keyboard_arrow_down

    Anand Bagmar - To Deploy or Not-to-Deploy - decide using TTA's Trend & Failure Analysis

    Anand Bagmar
    Anand Bagmar
    Director - Quality
    Vuclip Inc.
    schedule 2 years ago
    Sold Out!
    45 mins
    Demonstration
    Intermediate

    The key objectives of organizations is to provide / derive value from the products / services they offer. To achieve this, they need to be able to deliver their offerings in the quickest time possible, and of good quality!

    In order for these organizations to to understand the quality / health of their products at a quick glance, typically a team of people scramble to collate and collect the information manually needed to get a sense of quality about the products they support. All this is done manually.

    So in the fast moving environment, where CI (Continuous Integration) and CD (Continuous Delivery) are now a necessity and not a luxury, how can teams take decisions if the product is ready to be deployed to the next environment or not?

    Test Automation across all layers of the Test Pyramid (be it Selenium-based UI tests, or, xUnit based unit tests, or, Performance Tests, etc.) is one of the first building blocks to ensure the team gets quick feedback into the health of the product-under-test. 

    The next set of questions are:
        •    How can you collate this information in a meaningful fashion to determine - yes, my code is ready to be promoted from one environment to the next?
        •    How can you know if the product is ready to go 'live'?
        •    What is the health of you product portfolio at any point in time?
        •    Can you identify patterns and do quick analysis of the test results to help in root-cause-analysis for issues that have happened over a period of time in making better decisions to better the quality of your product(s)?

    The current set of tools are limited and fail to give the holistic picture of quality and health, across the life-cycle of the products.

    The solution - TTA - Test Trend Analyzer

    TTA is an open source product that becomes the source of information to give you real-time and visual insights into the health of the product portfolio using the Test Automation results, in form of Trends, Comparative Analysis, Failure Analysis and Functional Performance Benchmarking. This allows teams to take decisions on the product deployment to the next level using actual data points, instead of 'gut-feel' based decisions.

  • moiz
    moiz
    Software Engineer
    Saucelabs
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Beginner

    Appium, often dubbed "Selenium for mobile", at heart its a web server written in NodeJs. Its architecture is modular, which means that it is composed of many small, independently maintained and tested modules. Testing Appium is challenging, but clearly very important, since thousands of users depend on it for their testing. Appium also has all the usual challenges of a large open source project, for example, ensuring consistency of JavaScript code style across hundreds of contributors. It's important to have high-quality and readable code.

     
    I will be discussing approaches to and strategies for testing these kinds of large, modular applications. On the Appium team, we use a combination of unit, functional, and integration tests. Modern services like GitHub, Travis CI, and Sauce Labs make it possible for large open source projects to be tested thoroughly, keeping the code and the app at high quality. I will also discuss the use of tools like JSLint and Gulp, which help prevent code style issues.
     
    Testing the tool which is used for testing is clearly very important. This talk aims to showcase how testing should be approached for large, modular projects which has many collaborators.
  • Liked rubytester
    keyboard_arrow_down

    rubytester - Docker Selenium. Getting Started

    45 mins
    Demonstration
    Beginner
    • `docker-selenium` project is about packaging selenium grid as docker containers (https://github.com/seleniumhq/docker-selenium)
      To me this means I don't have to build any selenium infrastructure machines. I just run the provided images by docker-selenium project (https://hub.docker.com/r/selenium/).
    • I don't have to install selenium jar, java, browsers and other runtime dependencies. They are already built in a docker image and I can just run them as either selenium grid with hub and nodes or as standalone selenium on any docker engine enabled vm. 

    In this talk/demo/case study I will show you how you can use `docker-selenium` project to build several pipelines starting from running on your local dev box to public cloud for quick tests and finally to a stable private cloud for your team.

     

  • Liked Rémi
    keyboard_arrow_down

    Rémi - Mobile end to end testing at scale: stable, useful, easy. Pick three.

    Rémi
    Rémi
    Software engineer
    Facebook
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Beginner

    This talk is about how Facebook turned a great idea with a terrible track record into a great tool for thousands of developers.

    The promise of E2E testing — complex, real-world test scenarios from the point of view of and end user — is appealing.
    Many attempts have been made over the years at automating large parts of companies' and developers' testing and release processes, yet most of these efforts ended up in bitter and hard learned lessons about the inherent challenges of the whole approach.

    My work at Facebook over the last two years has been making mobile end to end testing at scale a reality.
    When others said it couldn't be done, or fell by the wayside, we relentlessly pushed forward, solving problems deemed intractable, and finding new, untold vistas of horror before us

    We've come a long way: E2E testing is now an integral part of Facebook's mobile development and release process.
    We'll cover what challenges we faced, and how we chose to solve or make them irrelevant.

  • Liked Jonathan Lipps
    keyboard_arrow_down

    Jonathan Lipps - The Mobile JSON Wire Protocol

    45 mins
    Talk
    Intermediate

    The JSON Wire Protocol (JSONWP) is the version of the WebDriver spec currently implemented by all the Selenium clients. It defines an HTTP API that models the basic objects of web automation---sessions, elements, etc... The JSON Wire Protocol is the magic that powers Selenium's client/server architecture, enables services like Selenium Grid or Sauce Labs to work, and gives you the ability to write your test scripts in any language.

    The JSONWP has served Selenium faithfully for a number of years, but the future of automated testing lies beyond the borders of the web browser. Mobile automation is an essential ingredient in any build, and tools like Appium or Selendroid have made it possible to run tests against mobile apps using the JSONWP. The JSONWP's current incarnation isn't enough to automate all the new behaviors that mobile apps support, however. Complex gestures, multiple device orientations, airplane mode, and the ability to use both native and web contexts, for example, are all essential to mobile automation.

    For this reason the leaders of the Selenium project, in concert with other Selenium-based projects like Appium and Selendroid, met to discuss the future of the JSONWP. We've been working on its next version, called the "Mobile JSON Wire Protocol" (MJSONWP). Appium and Selendroid already implement much of the MJSONWP spec. In this talk I'll dive into the specifics of the MJSONWP extensions, how they relate to the original JSONWP, and how the Selenium clients have begun to implement them.

    Finally, I will talk about the future of the MJSONWP and how it's related to the current and future versions of the WebDriver spec. I'll share how you can get help with the creation of the MJSONWP, and discuss issues with the authors of the new spec before the API is set in stone. We need the help of everyone who's involved in mobile automation to come up with the best and most future-proof version of the MJSONWP. Ultimately, your understanding of how Selenium works will be improved, and you'll have a much better handle on how projects like Appium and Selenium work together to make sure you have the best automation methods available.

  • 45 mins
    Demonstration
    Intermediate

    How many times do we test the same things at multiple layers, multiple levels, adding time to the build process and testing cycle, delaying the feedback?

    We know what to test and how to test, but what is the right place to test it?

    In this workshop, we will demonstrate how as QA’s we can identify what tests can be classified as unit tests, integration test and functional test. Using a case study, we will see how each component can be tested as part of unit testing; the integration of different parts and the functioning of a software system as a whole and how functional tests fit into this big picture. We will then bring all these tests together to understand and build the testing pyramid and how it enables us to build the right testing framework with fewer selenium i.e functional tests.

  • Liked James Eisenhauer
    keyboard_arrow_down

    James Eisenhauer - An Introduction to the World of Node, Javascript & Selenium

    45 mins
    Talk
    Beginner

    Ever wanted to write Selenium code in Node.js?  There seems to be a new javascript library written every hour!  Entering the world of Node.js can be a daunting task.  This session will teach you everything you need to know to make the right decisions when selecting what libraries you should implement on your new Node.js Selenium project and what the possible challenges will be.

     

     

  • Liked Russell Rutledge
    keyboard_arrow_down

    Russell Rutledge - Blazing Fast UI Validation - 5000 Reliable Tests in 10 Minutes on One Machine

    Russell Rutledge
    Russell Rutledge
    Senior Technical Lead
    Nike
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Advanced

    A big blocker for putting a website on truly continuous production delivery is the amount of time it take to validate that the site works correctly.  Tests themselves take time to run, and test results are unreliable to the point where it takes a human to investigate and interpret them.  When counting the time that it takes to both run and interpret results, test runs for an enterprise web site can take an entire day from inception to useful result.

    This session describes common points of failure in test execution that add both latency and unreliability and what can be done to overcome them while still preserving the value of UI validation.  We'll discuss why, after addressing these concerns, UI can be unblocked to reliably field thousands of validation scenarios on a local machine in a matter of minutes. 

  • Liked Titus Fortner
    keyboard_arrow_down

    Titus Fortner - What Are We Testing, Anyway?

    45 mins
    Talk
    Intermediate

    Testing strategies and the role of DOM to Database Testing in a world of micro services and client side MVCs.

    The trends in software development are making UI testing increasingly difficult. Sites are leveraging more dynamic interactions and moving toward Single Page Applications. Gone are the days when the term “and the page finishes loading” makes any sense. This shift is dramatically increasing the number of flaky tests as well as the costs of such testing relative to the benefits, leaving many organizations wondering if they are worth doing at all. 

    The approach to testing that is “good enough” for any given organization is going to vary by context. In this talk, I’ll cover some different testing options and the advantages and disadvantages to each. We’ll discuss the dangers of mocking and stubbing, the problems with relying on testing journeys, and dealing with bloated test suites that are difficult to maintain.

    Another trend in software development is away from monolithic architectures and toward micro services and service oriented architectures. This approach provides opportunities for decreasing the costs and overhead of UI testing while still maintaining all of the benefits of DOM to Database verification.

  • Liked vishnu nallani chekravarthula
    keyboard_arrow_down

    vishnu nallani chekravarthula - Extending Selenium Element Locator Strategies – Element Filtering

    45 mins
    Talk
    Intermediate

    Element Locator strategies for Selenium WebDriver are highly flexible, and have been later inherited by many commercial tools. Although the locator strategies are flexible, they are also limited in a sense that, Selenium WebDriver does not currently allow its users to identify/filter UI elements with multiple locator strategies(at a time), as many commercial tools do.

    The solution discussed in this article describes a library that allows Selenium WebDriver users to extend the Selenium element locator strategies for Element Filtering and few use cases for the library.

    The solution approach allows users to continue to use the existing UI Element definitions in their tests, and extend them, using the By reference. The library will replace the existing Selenium WebDriver “By” reference.

    Filtering based on multiple locator strategies

    There are various scenarios where to uniquely identify an UI element, a complex XPath has to be written. However, the element can be identified uniquely using multiple locator strategies for the UI Element. The UI Elements can also be filtered, when there are multiple matches in a page. This is the UI Element recognition mechanism used in many commercial test automation tools.

    The algorithm for filtering UI Elements based on multiple locator strategies is based on priority of locator strategies. The priority of locator strategies when filtering is:

    1. ID
    2. Name
    3. TagName
    4. ClassName
    5. XPATH
    6. LinkText and PartialLinkText
    7. CSS

    The By.elementFilter method takes multiple locator strategies, and searches the page for elements matching a particular locator strategy/property, and checks if it is a unique match on the page, if not then it uses the next locator strategy passed to it and so on.

    This method is also very helpful when the application undergoes constant changes and UI Elements might have either of XPATH, ID , NAME, TagName, ClassName etc still unchanged. That way, it helps reduce a lot of maintenance effort in Selenium WebDriver implementations which is due to UI element changes.

    Filtering based on Index

    When there are multiple similar UI Elements in a page, such as cells in a grid/table, it makes sense to identify objects based on their Index based on their appearance on the web page.

    The By.indexFilter method allows users to define an UI Element based on its Index of occurrence of the UI Element. The Index starts from 1.

    Filtering based on relative element

    When a UI element cannot be identified uniquely and reliably by any of its properties, but has some elements in its hierarchy or relative to a particular element, this method can be used to identify the element

    The By.relationFilter method allows users to define an UI Element in relation to another element. The relation can be defined as “Left”, ”Right”, ”Top”, ”Bottom”, ”Child”, ”Parent”

    Filtering for Tables

    When dealing specifically with Tables, which have the

    html tag, the By.tableFilter method allows the user to quickly identify specific cells in the table, without having to write complex XPaths or logics to achieve the same.

    The By.tableFilter method allows users to define a cell in the table with Row,Column numbers. This allows users to directly use the UI Element in their code instead of writing their logic each time. This also increases efficiency and readability of the code.

  • 45 mins
    Talk
    Intermediate

    Responsive Website Design have enabled mobile phones and tablets to fundamentally changed how we interact with the internet. Now we have instant access to any website we choose to visit and this causes headaches for testers, especially automated testers.

    This changes how automation, specifically Selenium, is implemented as the test suite needs to be maintainable, which is difficult and will get unruly if not maintained.

    The talk will be specifically about responsive websites however the same techniques can be applied to native app testing.

    Utilizing a test case generator allows for the test conditions(browser, OS and resolution) to exist outside of the test itself allows a single test to be able to test against all testing combinations without having to code for the other options explicitly. With the different options outside of the test the driver is easily instantiated and the browser windows are modified prior to test execution.

    As a sole(or two man team) automated test engineer for 4 years in over 5 projects these are my tools and techniques on how to make your automated test suite not only maintainable but adaptable to any device that you need to test with minimal overhead.

  • Liked Bhumika S
    keyboard_arrow_down

    Bhumika S - The 10 Myths of Testing that we need to Break

    45 mins
    Talk
    Beginner

    This talk aims to break 10 (amongst many) Myths that desperately need to be broken around Software testing and Testers. These so-called myths, have been around for so long that they are almost the norm. Breaking these myths is necessary to help budding as well as experienced Quality Analysts and teams, roll out better quality software products and build better team practices as well.

  • Liked Jason Watt
    keyboard_arrow_down

    Jason Watt - Challenges of the Mobile Cloud

    Jason Watt
    Jason Watt
    Senior Software Engineer
    Salesforce
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    Creating a mobile app is now the new cross platform problem. The major mobile platforms tend to gear their development tool chain towards individuals and their workstations.  But what if you want to introduce a CI solution to this environment? What if your app is launching on more than one platform and there's a team of 20+ developers working on it? What if your tests are more than just Selenium based?

    This is normally where you can look to the cloud for scale but mobile has a ton of challenges to do so.  Come and learn from some of the challenges and pitfalls I've encountered while working towards this goal.

  • Liked Sarah Thompson
    keyboard_arrow_down

    Sarah Thompson - DevOps meets QA - Using Puppet to set up and manage your Selenium Grid

    Sarah Thompson
    Sarah Thompson
    QE UI Toolsmith
    Puppet Labs
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    For testers, setting up and maintaining a Selenium Grid infrastructure can be timely and costly. A lot of the time, we are asked to do this as part of our day to day job when we really want to focus on testing the product!

    There are some great cloud based alternatives out there that allow you to easily run your tests on a wide range of Operating Systems and against multiple browser types (at a price).

    But what if you already have plenty of devices available within your own company (be it physical machines, virtual machines or cloud based resources) and you want to be able to setup and manage your own grid infrastructure:

    • to allow you greater control over the configuration (perhaps you want to have a headless browser like PhantomJS available on your grid)
    • to run your tests faster (the cloud based alternatives are a good bit slower for obvious reasons)
    • or to simply save money.
  • Liked Arjun Rajeev
    keyboard_arrow_down

    Arjun Rajeev - Beacon - End to End Automation with selenium

    45 mins
    Talk
    Advanced

    When we talk about enterprise level automation where in there are multiple technology stack involved it becomes exceedingly difficult to automate every application using the same tool. Selenium web driver has become a defacto standard when it comes to web application automation and no body would prefer any other tool over it when it comes to web. But what about the other applications, We may have some SAP/ some .NET applications that are linked to the web application and needs to be automated as well. Lets go for some other tool here, but oops this tool does not talk java (which is what the automation team uses here). Basically we need to maintain two different automation teams with two different automation framework which is defenitely and overhead.

     

    The talk is about

    1. How to over come this scenario

    2. Rabbit MQ

    3. Integrating the MQ server with Grid

    4. Other tool (other language binding) integrated to selenium

    5. Building java COM bridge

    6. Developing custom bindings to your tools which by default does not have them

  • Liked Herman Ching
    keyboard_arrow_down

    Herman Ching - Using genetic algorithm to do exploratory testing with cucumber and selenium.

    Herman Ching
    Herman Ching
    QA Engineer
    Lab49
    schedule 2 years ago
    Sold Out!
    45 mins
    Experience Report
    Advanced

    Writing selenium tests has an inherent problem in which we can only automate what we know the steps for. It is primarily used for regression and can scale very quickly. We generally optimize regression for sprint's relevant features or have smoke tests to get quicker feedback. In addition negative testing is very expensive to write. This entire process is very manual and the maintenance of tests become more costly than the actual writing of tests. 

    We can change this. By using genetic algorithm and cucumber steps we can create new feature files to discover where in the application may be be problematic. The goal of each individual would be to find the most defect. The two individuals that find the most defects then create children that follows the sequences of how new individuals are form through genetic algorithms. They will breed the next generation of testers which will focus on problematic areas. 

    Let's find out when and if this strategy can be deemed effectively. 

  • Liked Anthony Browness
    keyboard_arrow_down

    Anthony Browness - Dependency Injection in the Page Object Model

    45 mins
    Talk
    Advanced

    The Page Object Model is awesome, but you can make it even better with Dependency Injection.  I'll show you how to get started and some benefits of using Spring IOC in your Page Objects.  Though the concepts of this talk can be extended to any language, the examples will be shown in Java.

  • Anand Bagmar
    Anand Bagmar
    Director - Quality
    Vuclip Inc.
    schedule 2 years ago
    Sold Out!
    45 mins
    Case Study
    Advanced

    Page-Object pattern is very commonly used when implementing Automation frameworks. However, as the scale of the framework grows, there is a limitation on how much reusability really happens. It inherently becomes very difficult to separate the test intent from the business domain.

    Learn more about this problem, and the solution (by means of real-project examples) - Business Layer - Page - Object pattern, which has helped me keep my code DRY.