Looking ahead: testing responsive, mobile, and native apps with Selenium

Selenium made test automation easier and affordable for many software development teams, but it had many limitations.  It was limited to DOM manipulation in the browser, it depended on explicit waits.  

Webdriver helped overcome some of these deficiencies and took Selenium to the next level.  Other extensions like Appium have enabled us to use the familiar Selenium API for testing mobile apps.  Proprietary frameworks allow you to integrate Selenium with native extensions and ALM tools.  But a new category of apps is coming with responsive UIs, rich client side Javascript frameworks, touch screens (with pinch/zoom, swipe, rotation, etc) and interact with native device features (such as GPS, accelerometer, local storage) and apps are becoming a collection of interactive services.  

Is Selenium becoming outdated?  What can we do to keep up with these new interfaces and architectures?

In this talk, we'll discuss some of the challenges and limitations facing testers using Selenium with this new generation of apps.  We'll cover some of the solutions people are using today, and propose a new way to address these issues and others going forward.

 
7 favorite thumb_down thumb_up 2 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

  • Background
    • Proprietary test automation
    • Selenium - the cure for Mercury Poison
    • Selenium Grid - cross browser testing
    • WebDriver - a better Selenium
    • Appium - selenium for mobile
    • Native and proprietary extensions
  • The problems
    • Native Elements
    • AJAX
    • Responsive websites & single-page-apps
    • Mobile apps
    • Rich Internet Apps
    • Tablets and touch screens
    • Web Services
    • Mashups & SaaS
  • Webdriver
    • Why WebDriver?
    • WebDriverBackedSelenium
    • Vendor specific WebDrivers
    • Json Wire Protocol
  • Appium & Sauce Labs
    • Mobile API
    • Mobile Devices
    • Cloud accessible Devices
  • How to deal with challenges 
    • Touch Screen UIs
    • Responsive pages
    • Single Page Apps
    • Native Elements
    • Web Services
  • New ideas
    • Looking back at the old Selenium command, locator, argument API
    • Expanding user actions
    • API for new Drivers
    • Separating concerns 
    • Network Proxy, UI Proxy
    • Mockability

Learning Outcome

You'll be aware of some of the challenges facing test automation with mobile and responsive apps and learn of some existing solutions.  You'll come away with ideas for how we can improve Selenium and take it to the next level.   

Target Audience

Experienced testers familiar with Selenium who are bumping up against some of it's limitations

schedule Submitted 2 years ago

Comments Subscribe to Comments

comment Comment on this Proposal
  • Dave Haeffner Test
    By Dave Haeffner Test  ~  1 year ago
    reply Reply

    Can you please attach a rough time allotment to each of your bullet segments (e.g., Background: 2-4 minutes, The problems: 5 minutes, etc.).

    • Aaron Evans
      By Aaron Evans  ~  1 year ago
      reply Reply

      Heh, you got me Dave.  I put in way too much material and need to pare it down.


  • Anand Bagmar
    Anand Bagmar
    schedule 2 years ago
    Sold Out!
    45 mins
    Demonstration
    Intermediate

    Typically in organizations, there are multiple projects / products. These products may be of implemented using tech-stacks over many years. Yet - they interact with each other in some way. To manage the complexity around Test Automation, many organizations prefer to have a common Test Automation solution across these products in an effort to build, standardize and maintain the framework.

    However, this is not a good idea! With this approach one potentially ends up having to compromise on the quality of automation that can be done for each product, limited by the toolset.

    The better approach would be to use the tools and technologies that are "right" for each product. This does have other disadvantages, but you would ensure each product is well tested! The only missing piece which remains is that these different products talk with each other. You need to test the integration between them in an automated way to verify all is well.

    "TaaS" is an open-source product solution that allows you do achieve the "correct" way of doing integration testing across a variety of products via Test Automation.

    Example: 

    For one set of products, Selenium-based toolset may be the right choice, where as for legacy reasons, QTP may be used for some other product. With TaaS - you will be able to automate the Integration Testing between these products, by re-using the tests already implemented in the individual product suites.

     

  • moiz
    moiz
    schedule 1 year ago
    Sold Out!
    45 mins
    Talk
    Beginner

    Appium, often dubbed "Selenium for mobile", at heart its a web server written in NodeJs. Its architecture is modular, which means that it is composed of many small, independently maintained and tested modules. Testing Appium is challenging, but clearly very important, since thousands of users depend on it for their testing. Appium also has all the usual challenges of a large open source project, for example, ensuring consistency of JavaScript code style across hundreds of contributors. It's important to have high-quality and readable code.

     
    I will be discussing approaches to and strategies for testing these kinds of large, modular applications. On the Appium team, we use a combination of unit, functional, and integration tests. Modern services like GitHub, Travis CI, and Sauce Labs make it possible for large open source projects to be tested thoroughly, keeping the code and the app at high quality. I will also discuss the use of tools like JSLint and Gulp, which help prevent code style issues.
     
    Testing the tool which is used for testing is clearly very important. This talk aims to showcase how testing should be approached for large, modular projects which has many collaborators.
  • 45 mins
    Talk
    Intermediate

    There has been a recent explosion in second-screen technologies such as Chromecast, but designing test automation for second-screen applications is far from straightforward. This new paradigm lacks major automated tool support, and coordinating test execution across multiple devices is tricky and error-prone.

    Our automation solution uses WebdriverJS and WebSockets to perform end-to-end test automation that covers our web player controller and second screen application.

    Learn about our approach to second-screen automation which we’ve used to build a reactive, responsive test suite. We’ll describe our solutions to synchronizing test flow between the controller and target device, validation on the device, targeting different integration components, and device management.

  • James Farrier
    James Farrier
    Xiaoxing Hu
    Xiaoxing Hu
    schedule 2 years ago
    Sold Out!
    45 mins
    Demonstration
    Intermediate

    If a test fails in the woods and no one is there to see it does anyone care, does anyone even notice. What happens when failing tests become the norm and you can't see the wood from the trees? 

     

    After watching last years Allure Report presentation I was inspired.  Selenium tests (and automation tests in general) are often poorly understood by the team as a whole.  Reports/emails go unread with tests failing becoming an expected outcome rather than a glaring red flag.  We looked at what Allure brought to the table and from that base created a dashboard which was designed to:

    • Display the results of test runs in a way that was useful to managers, testers and the rest of the development team.  Including tools to filter out specific test runs and view the overall trend of the test run results.
    • Make debugging tests easier by grouping errors, displaying history of test results, filtering tests and offering visual comparison of test runs.
    • Help mitigate the problems flaky tests cause with test run result reporting (say that three times fast).
    • Help with our mobile device certification process, by easily providing a view to compare test runs across devices.

    Since it's creation the dashboard has been used and praised by managers through to developers.  With our full suite of tests from unit to integration to selenium and appium being stored on the dashboard.  We've managed to:

    • Decrease the time taken to debug test cases.
    • Increase the visibility of all our test suites, with managers having a better idea of how our selenium test suite is progressing and testers better understanding the coverage of unit tests.
    • Focus the organization on quality.

    We are working with legal at present to have this project open sourced and available to all prior to Selenium Conf 2015.

  • Liked Ragavan Ambighananthan
    keyboard_arrow_down

    Distributed Automation Using Selenium Grid / AWS / Autoscaling

    Ragavan Ambighananthan
    Ragavan Ambighananthan
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Advanced

    Speed of UI automation has always been an issue when it comes to Continuous Integration / Continuous Delivery. If UI automation suite takes 3 hours to complete, then any commit happens during this time will not be visible in test environment, because the next deployment will happen only after 3 hours. 

    With 2000+ developers and average 250+ checkins per day, the above issues is replicated 250+ times every day. This is not productive and feedback cycle is super slow!

    Another issue is , with 35+ different project teams using 10 or more different jenkins jobs to run their UI automation. So many jobs means (350+), individual teams need to go through the pain of managing their own jenkins job, its a duplicate effort and waste of time. Automation teams need to spend time on writing reliable automation and not managing jenkins jobs.

    Solution is to reduce the UI automation run time from hours to minutes and also use only handful of jobs to run the Distributed Automation!

    Goal: To run all UI automation scenarios within the time take by the longest test case

  • Liked Sveta Kostinsky
    keyboard_arrow_down

    Selenium Today vs. Selenium Tomorrow: Digital as the Convergence of Mobile & Web Programs

    Sveta Kostinsky
    Sveta Kostinsky
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Beginner

    Today, mobile is increasingly trumping web as the most important brand engagement point; enterprises are moving away from mobile and web projects independent of each other. The rapid adoption of responsive web encourages teams to discover one approach to measuring software quality regardless of form factors.

     

    Selenium is current market leading solution for web testing, but how does it stand with mobile? The truth is that working with Selenium presents a few challenges, including:

    • Building and maintaining an internal structure to support it
    • Bridging an architectural gap
    • Requirements demand support for unattended test execution
    • Lack of real network conditions for mobile testing

     

    There is a solution to address these challenges!

    Let’s work through a demo and show how to test mobile & web in parallel with Selenium

  • Surendran Ethiraj
    Surendran Ethiraj
    schedule 1 year ago
    Sold Out!
    45 mins
    Demonstration
    Intermediate

    The evolution of Test Automation started with automation tools that had record and playback features.  This allowed Automation Testers to record and structure the script in such a way that it could be reused. Tools like Selenium, which provided APIs, could interact with different browsers. The Automation Testers could use these APIs to interact with web applications. Additionally, it was possible to develop frameworks for reusing each of the components of the framework. Currently, the focus has shifted more and more towards the designing of frameworks rather than just the tools, so that the testing framework could be integrated with test management applications and continuous integration tools to aid test-driven development.

    With that background, we have come up with certain Value Added Services (VAS), a step ahead of developing functional automation scripts. Imagine creating an Automation Framework which will not just check for the functionality of the application, but also check for security, page performance, page layout, accessibility and have an output that can be a trigger to other aspects of testing.

    This paper presents three of the Value Added Services that we offer. We are working on creating many more such services on top of the Automation Framework.

  • Dipesh Bhatewara
    Dipesh Bhatewara
    schedule 1 year ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    In modern cloud aeon, customers are delivered with UI and API for most of the Enterprise Products. The QE teams have to test the product for UI and APIs from functional as well as non functional perspective. For these they typically end up writing different test suites.

    There are lot of challenges one faces when automating test suites for these different purposes, like

    1. How do you integrate UI automation seamlessly with different test frameworks/suites?
    2. How do you test all these requirements (UI, API, Performance etc.) effectively without a lot of redundant or duplicate code?
    3. How do you use the best fit language/technology for UI automation and still do not impact the grand test automation strategies?

    There is an interesting solution to conquer these challenges which I have recently implemented in one of my projects. 

    The idea is to segregate tests from the UI Automation OR webdriver code. This code can be exposed as a web service APIs. Such service allows us to write common tests for UI as well as API. Tests written for API testing can be reused for UI testing with very minimal configuration modification like pointing to appropriate web service. These UI Automation APIs can also be called easily from any performance test requiring UI interaction.

    This approach will enable any test suite using any language or technology in your organization to reuse UI automation seamlessly. This will also make lot of things straight and simple for execution and management across the verticals.

     

    We will deep dive into the technical solution for this in this talk.

  • Anand Bagmar
    Anand Bagmar
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Beginner

    With all the focus on Test practices and tools getting better for the present, it is easy to forget about the Future of Testing, especially in terms of Tools and Infrastructure? A question that always comes up in my mind - “Are we so caught up in the past and present, that we will not be as effective in the future?”

    In this talk, we will go on a journey to figure out what new challenges are coming up in the future, and more importantly, what do we need to do next to prepare for it. Also, just preparing for the future is not sufficient. We have an opportunity to stretch beyond our current set of skills, capabilities and boundaries to influence out future! The question is - will we make use of this opportunity? Will Selenium help us take the step in the right direction? Or, will it hold us back?

  • Liked Jason Watt
    keyboard_arrow_down

    Challenges of the Mobile Cloud

    Jason Watt
    Jason Watt
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    Creating a mobile app is now the new cross platform problem. The major mobile platforms tend to gear their development tool chain towards individuals and their workstations.  But what if you want to introduce a CI solution to this environment? What if your app is launching on more than one platform and there's a team of 20+ developers working on it? What if your tests are more than just Selenium based?

    This is normally where you can look to the cloud for scale but mobile has a ton of challenges to do so.  Come and learn from some of the challenges and pitfalls I've encountered while working towards this goal.

  • Liked Sarah Thompson
    keyboard_arrow_down

    DevOps meets QA - Using Puppet to set up and manage your Selenium Grid

    Sarah Thompson
    Sarah Thompson
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    For testers, setting up and maintaining a Selenium Grid infrastructure can be timely and costly. A lot of the time, we are asked to do this as part of our day to day job when we really want to focus on testing the product!

    There are some great cloud based alternatives out there that allow you to easily run your tests on a wide range of Operating Systems and against multiple browser types (at a price).

    But what if you already have plenty of devices available within your own company (be it physical machines, virtual machines or cloud based resources) and you want to be able to setup and manage your own grid infrastructure:

    • to allow you greater control over the configuration (perhaps you want to have a headless browser like PhantomJS available on your grid)
    • to run your tests faster (the cloud based alternatives are a good bit slower for obvious reasons)
    • or to simply save money.
  • Liked Satyajit Malugu
    keyboard_arrow_down

    Adapting testing and automation for Mobile First

    Satyajit Malugu
    Satyajit Malugu
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    Mobile-first is a paradigm shift in developing software where products are created with an emphasis on mobile experiences, rather than desktop. Mobile sites, native apps become the new centerfold for the company around which various other initiatives and goals are driven. This paradigm has been in adoption by major industry players including Google, facebook, Airbnb, ebay, shopify etc.

     

    From a development standpoint it is not really different from traditional websites after you get UX but testing is quite different. You have to consider various devices, screen sizes, performance and visual testing. Automation should focus on tackling the most time consuming manual aspects which include functionality, visual parity and device compatibility.

     

    Our $5B company has been in a mobile-first transformation this year, spanning many legacy products and teams. As a senior SDET leading the testing efforts I have been involved in various phases of this transformation. I believe this talk would give practical insights in mobile web development and testing.

  • Liked Manoj Pahuja
    keyboard_arrow_down

    Measuring the metrics to make them reliable, fast and valuable- What, why and How?

    Manoj Pahuja
    Manoj Pahuja
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    UI Tests are fragile by nature. The reliability and execution times are common challenges. If you add scaling UI tests for multiple runs, multiple times a day and environmental instabilities, it becomes very difficult to find the right information related to these tests. Running through and Understanding test results across different builds takes a lot of time. You get too many false alarms and now you have reached a point where nothing is an alarm. Get to know some practical approaches and success stories on how you can distribute all tests across multiple machines, VMs, cloud services and still be able to combine all the results in a easy to read and understandable report. Fix reliability and improve speed of your tests.

  • Liked Dmytro Zharii
    keyboard_arrow_down

    Smoke Tests and Page Objects: Let's do the right thing fast!

    Dmytro Zharii
    Dmytro Zharii
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    "Write the Smoke Tests" -- this is the ultimate answer for the questions:
    - How do I start a new test automation project?
    - What is the way to test that my PageObjects are still working correctly after the "nightly" product changes?
    - What is the fastest way for my Test Automation Framework to bring the real value for my teammates and managers?

    Can you imagine you can generate the Page Objects code in a fast and easy way with SWD Page Recorder?
    Can you imagine you can generate the Smoke Tests for your PageObjects and the Application Under Test by pressing one button?

    Let me show how you can increase your productivity in average 45% by using good tooling, patterns and practices during your daily work.

  • Liked manjyot singh
    keyboard_arrow_down

    WebDriver - Partners in crime

    manjyot singh
    manjyot singh
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    Talk is about :

    1) How neatly we can design our tests using page object patterns.

    2) How easy it is for users belonging to different roles to understand the object of tests using BDD i.e. cucumber.

    3) How handy it is to use test framework like TestNG along with WebDriver.

    4) How easy it is to maintain dependencies of your test framework using build tools i.e. Maven.

    I don't want to get into serious crime of boring the audience with the details that are not at all required/ necessary but to target the areas which beginners and experts are interested in.

  • Liked Herman Ching
    keyboard_arrow_down

    Using genetic algorithm to do exploratory testing with cucumber and selenium.

    Herman Ching
    Herman Ching
    schedule 2 years ago
    Sold Out!
    45 mins
    Experience Report
    Advanced

    Writing selenium tests has an inherent problem in which we can only automate what we know the steps for. It is primarily used for regression and can scale very quickly. We generally optimize regression for sprint's relevant features or have smoke tests to get quicker feedback. In addition negative testing is very expensive to write. This entire process is very manual and the maintenance of tests become more costly than the actual writing of tests. 

    We can change this. By using genetic algorithm and cucumber steps we can create new feature files to discover where in the application may be be problematic. The goal of each individual would be to find the most defect. The two individuals that find the most defects then create children that follows the sequences of how new individuals are form through genetic algorithms. They will breed the next generation of testers which will focus on problematic areas. 

    Let's find out when and if this strategy can be deemed effectively. 

  • Liked Anthony Browness
    keyboard_arrow_down

    Dependency Injection in the Page Object Model

    Anthony Browness
    Anthony Browness
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Advanced

    The Page Object Model is awesome, but you can make it even better with Dependency Injection.  I'll show you how to get started and some benefits of using Spring IOC in your Page Objects.  Though the concepts of this talk can be extended to any language, the examples will be shown in Java.

  • Anand Bagmar
    Anand Bagmar
    schedule 2 years ago
    Sold Out!
    45 mins
    Case Study
    Advanced

    Page-Object pattern is very commonly used when implementing Automation frameworks. However, as the scale of the framework grows, there is a limitation on how much reusability really happens. It inherently becomes very difficult to separate the test intent from the business domain.

    Learn more about this problem, and the solution (by means of real-project examples) - Business Layer - Page - Object pattern, which has helped me keep my code DRY.