Mobile Test Automation is increasingly becoming very important. Almost all web applications are responsive these days and it's very important to test how the application works across devices. The same is true with the native application as well. At the same time, the number of devices and the custom OS versions on devices are also vast. This means that it's harder for a tester to manually run the automated tests over a list of devices to get device coverage and quicker results over every feature development.

We came up with a solution of executing tests in distributed or parallel fashion across remote devices from anywhere in the network using Appium Test Distribution. Same framework is officially used by Appium members for Beta testing of Appium.

USP of ATD over other Market Solutions:

• Device Cloud:
  • Setup Devices anywhere within a network, ATD executes remotely without Grid
  • Never worry about device location in network.
• Plug and Play:
  • Connect your android/iOS devices or Emulators/Simulators and just execute tests.
• Multiple TestRunner:
  • TestNG and Cucumber
• Parallel Test Execution:
  • Runs across all connected iOS, Android real devices and Simulators/Emulators
• Test Coverage:
  • Parallel(Run entire suite across all devices, which gives device coverage)
  • Distribute(Run tests across devices to get faster feedback).
• Device Management:
  • Manage devices remotely using Device Manager.
• Reporting:
  • Covers detailed crashes logs from Android and iOS.
  • Appium Servers logs
  • Screenshots on Failures and On Demand Video logs
  • Reporting Trends for multiple builds
• Manual Access to Remote Devices - OpenSTF support

Who loves/uses ATD?

ThoughtWorks, CeX, Jio, TravelStart, M800, Reward Gateway and lot more.

Mobile platforms nowadays give more than a half of the worldwide Internet traffic. Being the most widespread platform, Android is slightly more difficult to work with from test automation perspective than traditional desktop browser automation. Last autumn in Berlin I have shown Selenoid - an open-source Selenium protocol implementation that dramatically simplifies working with Selenium for desktop browsers. In this talk I would like to demostrate how it is possible to have the same experience on Android platform for mobile application and mobile web testing.

Application Backdoor via Appium

There's a shift towards open-source mobile test automation tools happening today among developers and QAs. Whether it be Appium, Calabash or anything else: all are good, with some major limitations.

While a chosen tool may work well when you first start using it, things can quickly get out of hand with changing business requirements. We started using Calabash at Badoo when there was no Appium. Given the capability of Appium to drive the whole device, we started automation of new apps with Appium. However, we realized a powerful feature was missing in Appium for Android! : The ability to call Application code from automation code like Calabash Backdoors.

As Appium UiAutomator server is based on instrumentation, we modified it such that we could instrument our app under test. This gave us the power to access context of Application under test and invoke public methods of Activity using reflection APIs. We use these methods to setup app state, seed DB OR even enable/disable some client-side A/B tests. This makes our application more testable and our tests more predictable.

This talk is going to be about how I achieved the above solution and benefits of backdoors.

There will be a small demo and code!

Autonomous cars were a Scifi dream not 10 years ago. A computer driving a car? No way. But it did happen, and is happening. And if scientists do it for a complicated task such as driving, can they do it for automated regression testing? In this talk we explore what is being done in the field today, but also speculate about the future: we introduce the 6 levels of autonomous testing (that correspond to the 5 levels of autonomous driving), and try and figure out what kind of help current AI techniques can bring to automated testing.

For so many years, testers have focused on functionality, ensuring the applications are working properly, stable, and reliable. However, in today’s world, with so many competing applications, products, and software packages, it is critical that testers also examine the UI/UX element of each deliverable. If your organization is not building UI/UX testing into your test planning, then you are increasing the risk that your product may be left behind by competitors. And if you have not experienced the art and craft of UI/UX test planning and execution, then this workshop will help you and your organization learn the proper methods to do so.

There are very few known techniques that can accurately and consistently shape a good User Interface (UI) or User Experience (UX). While most of the companies are spending a lot of time and energy deciding the colors and bars on the screen, frankly beauty comes second. It’s also a known fact that users resist change. So how can you test for the acceptability of these changes in a way that’s beneficial to your company in terms of revenue, inbound marketing, and customer acquisition, without offending customers to the point that they make a massive exodus and go to your competitor?

The goal to success is making the customers happy and pleased with the product, while ensuring they do not feel foolish or confused. In this workshop, we will go through case studies of real world apps and stories of evolving UI and UX. We will observe how that impacts the User Experience for better or worse. We will look at building UX testing strategy and implementing UX testing techniques. We will look at the popular tools used to perform UX testing and how to best use them in each phase of not just testing but UX designing overall. Some of the tools to be discussed are - testflight / heap/hubspot / eyetracking tools / A/B testing / Screen recording tools.

Idea behind this talk is to describe how to create a short lived containers for each test and scale to large Selenium Cluster using Selenoid. What is standard Selenium architecture and why it is not suitable for big clusters, Also states issues in maintaining a large Docker Selenium Grid / Selenium Clusters. In this session we will also be talking about the challenges we faced in using official selenium docker images and how we scaled up regression execution time from 3:3 (3 hours to 3 mins). Followed by a demo of how Selenoid server solves the problem with docker. How it works, how to use inside big Selenium cluster, where else it could be used.

Many good product and project teams try to understand the “whys” when they’re asked to build a new feature. But, how do we know whether our reasons are actually good enough?

Are we tracking customer behaviour and feedback to get more insights into building the “right product”?

Do we have analytics? Are we monitoring issues/crashes and user suggestions? Are we automating these processes?

In this talk, we will share how we did our analysis in order to get better insights for building the right product and adding more value to our customers. We will share our experience, challenges, etc. We will also talk about a small utility (Sentiment Analyser) we developed which helps us to generate meaningful reports from user's feedback on daily basis.

We will also cover how we’ve automated analytics and understanding the sentiments of our customers.

There will be a demo for Analytics Automation and Sentiment Analyser.

How to build a good quality product is not a new topic. Proper usage of methodologies, processes, practices, collaboration techniques can yield amazing results for the team, the organisation, and for the end-users of your product.

While there is a lot of emphasis on the processes and practices side, one aspect that is still spoken about "loosely" - is the feedback loop from your end-users to making better decisions.

SO, What is this feedback loop? Is it a myth? How do you measure it? Is there a "magic" formula to understand this data received? How to you add value to your product using this data?

In this interactive session, we will use a case study of a B2C entertainment-domain product (having millions of consumers) as an example to understand and also answer the following questions:

• The importance of knowing your Consumers
• How do you know your product is working well?
• How do you know your Consumers are engaged with your product?
• Can you draw inferences and patterns from the data to reach of point of being able to make predictions on Consumer behaviour, before making any code change?

Continuous testing is an integral part of continuous delivery pipeline. When it comes to mobile application, the testing become increasingly complex.

As part of our internal quarter hackathon at Carousell, we developed an automation testing framework and device lab in 48 hours which has both parallel and distributed mode of running.

We will share about how you can set up a similar device lab in your organization and the best practices to be followed.

With the exponential increase of smart devices, businesses are bound to make their websites more responsive for the best user experience on all devices that can lead to increased sale and conversion. Moreover responsiveness is a major factor in Google's search ranking system. This inherently implies that the tester has to validate the application on different device dimension which is tiresome and error-prone if done manually. Here Galen Framework, an open source automated layout testing technique built on top of selenium, comes to the rescue of the testers.

In this workshop, we will learn how Galen empowers the QA, how to write layout tests using Galen Specs, use Galen Java API and ultimately run them on a CI server by integrating the Galen tests with selenium tests.

We will also learn few best practices such as - sharing a common locator repository between Galen and Selenium tests for better maintenance and writing Galen tests in plain english so that UX Designers, Business Analysts & Developers too can contribute to the tests.

Docker can be used to containerize Selenium tests and can be made to run against applications in another container. These tests can be further run on Selenium grid without the need of physical machines or VMs using Docker Selenium. Multiple and different versions of browsers, headless mode etc can be run inside containers and can be used to run tests in parallel. All these can be achieved with minimal changes to the existing Selenium tests.

The org culture mantra is usually considered a silicon valley mumbo jumbo. Although many companies and teams rant about culture, very few in my opinion take it seriously. In this talk I would like to discuss the importance of Org culture and how it makes or breaks businesses and teams. I will lay out some key pointers that will help the influencers in the audience make decisions about their org or team culture. The talk will be in the context of culture in tech companies.

In this era of automation where everyone is focused on Agile and DevOps, learn how you can migrate from xvfb (X virtual framebuffer which enabled you to run browser tests without a display on a CI server) to Headless Chrome while you use Protractor.
Attend this talk to understand how I migrated to headless chrome and the pain points. The discussion will include:

• things you can take care of when migrating
• checking versions compatibility for migration
• things you will need to fix

Talk Description Automation provides an immense amount of value in preventing regressions and helping to deliver quality software. As an organization’s automation grows and grows, it requires continuous maintenance so that tests remain fast, reliable, and valuable. If not scaled efficiently, an organization’s automation suite will turn into a messy, uncontrollable beast. Having a lean test suite will help to combat this. In this session will present methods to keep automated test suites lean and mean, so they always provide quick and accurate feedback to the software delivery team. Using a few examples, she will discuss a wide range of ideas including evaluating a test's value, parallelizing tests, and producing consistent results. Session attendees will walk away with strategies and practices to scale their test automation over time in a highly efficient and maintainable way.

We have come to a place where we run our selenium tests on pipelines every day.
Now are these tests confined to test only the functionality of the application?
No, we can use this tests to do much more. Let's test security with those same tests.

Turn your selenium tests in to Security tests, get security testing in to your CI pipelines just by tweaking a bit of the selenium integration suite. Find and report security issues that exists which can be caught way before paying off for it.
In this Demo we will introduce a framework that will help your selenium tests turn into security tests and run on pipelines.
With ZAP API + selenium and CI pipelines, find security issues on the application see a red or green security pipeline. Not just that fetch the reports and take necessary actions on every build.
The talk covers different ways that ZAP tests the web application for security(automated with selenium tests). We will see what all of them can be integrated with selenium and can make most out the suite.
As an add we will see how your API can be tested for security using ZAP API :)

Tools used:
1. Selenium Web driver framework.
2. ZAP framework integrated with selenium framework.
3. GO (for CI/CD)
4. Maven for building and fetching reports.

While automated functional tests using appium acts as safety net, there is something missing. Often, we find issues like:

“Sign-In button has shifted a bit and user have to scroll down to see it”

“We asked to change background color of screen-X, it got changed for Screen-Y screen as well”

“Why has fonts of this link become so tiny?”

For mobile apps, look and feel of an application is extremely important. With long running projects, any minor refactoring can change your app’s looks. functional testing tools like Appium can not help us in this case.

Most of the clients are not wanting to use any cloud based solutions, reason been cost and security concerns.

Solution was to build a in-house tool which is Nakal

github.com/saikrishna321/nakal_java

Targeting both mobile devices and desktop browsers has become a necessity for successful tech applications. Whether an existing web application triggers the development of native apps or a desktop version is introduced to supplement the smartphone experience, supporting both worlds is a reality for a majority of businesses.
To maximise user experience, an application should behave as identical as possible across all platforms. However, more often than not versions for different platforms are built by different teams with different designers, possibly across multiple sites (even countries, continents, timezones) each with their own tech-stack and development process. In such a set-up, using different tools for automated testing could come naturally and add to the challenges of building a single product.
We argue that a common approach to test automation is critical. Not only will it save time and development costs, it can also increase communication and improve collaboration between teams and be a great facilitator for setting the focus on the product as a whole rather than separate deliverables for each platform. In this sense, it can serve as an umbrella across all involved teams.
We will demonstrate a simple JAVA-based solution using Selenium, Appium and Cucumber to combine testing for different platforms (namely Android, iOS and Web) within the same framework. We will talk about best practices in designing an extendable and easy-to-use test framework and how we can facilitate cloud-based services to integrate our tests into a CI pipeline.

QAOps is Continuous Testing Strategies when
frequent software delivery matters.

It is no surprise that automation and orchestration
make life very easy, be it a small organization or
a large-scale industry that houses hundreds of
servers. IT automation is essentially the ability to
orchestrate and integrate tools,
people, and processes through a certain workflow. I will be using Ansible and Docker to showcase the same and writing ansible tests on top of that.

Ansible - Ansible is an IT automation tool which can configure systems, deploy software, and orchestrate more advanced IT tasks such as continuous deployments. Ansible is actually designed to be a “fail-fast” and ordered system, therefore it makes it easy to embed testing directly in Ansible playbooks.

Docker - A docker container is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, settings. Containers isolate software from its surroundings, for example differences between development and staging environments and help reduce conflicts between teams running different software on the same infrastructure.