"All tests in today's automated regression run have been marked as Untested. What happened?"

"No notifications are being sent for test runs on the channel"

"I pulled latest code, and the framework dependency shows compilation error"

"What does this new method in the framework do?"

How often do you hear such things within your team?

As Quality champions, we need to walk the talk. When we expect our developers to write quality code, write unit tests, build features without introducing bugs, the onus lies on us (as test engineers) to do the same. With almost every test engineering team writing automated tests to check functionality of their products and services, it becomes very important to ensure that the test automation framework and the test scripts are bug-free and follow good standards of software engineering.

It cannot be stressed enough that test automation code should be as good as production code. In order to build production-quality test automation framework and scripts, a number of steps can be taken at:

1. Code & System Level

2. Process & People Level

Our test engineering team went through a transition from having random & unexpected failing test runs to having greater confidence in the quality of the tests. Learn from this case study of our journey to ensure that end-end UI automated tests are built with quality in mind. We will also see demonstration of some of the use cases.

Shadow DOM is kind of web standard that developers use to encapsulate their custom HTML code and style components so that no other global style components can override their code. It ensures that a component will work in any environment, even if other CSS or JavaScript is run on the page i.e. Shadow DOM hides away the complexity of the components from the rest of the page because it renders separately from main DOM.

Selenium legacy selectors are not able to interact these custom elements under the shadow DOM so we need java script to interact with shadow DOM elements and we also require JavascriptExecutor interface for their execution.

We will also talk about the WebdriverIO tool v5.5.0, which implicitly support Shadow DOM elements.

There are dozens of matured docker based solutions available in the market for Android automation testing,

but

what about iOS testing?

It’s hard to create and maintain the docker based solution for iOS testing since it demands the Xcode and for the optimal performance the system needs the Apple certified hardware.

Combining Ansible with Selenium Grid yields a powerful combination because it allows us to set-up our grid and nodes in just a few seconds. In this talk I will demonstrate how to use simulators to build a reliable and scalable in-house iOS simulator lab using Ansible, Selenium Grid and Appium.

The adoption of Artificial Intelligence is getting more traction, it is in need to enhance QA capabilities to cope up with these skills. Machine Learning is used extensively in retail applications for solving complex problems, one of them is solving the search relevancy. Showing the appropriate results for the user is important for the conversion rate to go high. As Machine Learning poses different challenges such as a Test Oracle, Fairness, Correctness and Robustness to do QA, We may need to follow different approaches and testing techniques to do the QA for Machine Learning models.

Different Machine learning types such as Supervised and Unsupervised Models have different characteristics and are used for different types of problems. Though these solves different complex problems, Machine learning Models also a unit of software code that needs to be verified as a normal software system. When a Machine learning model is seen as a whole system, it may look complex and unsolvable. We can group them into small modules and verify for quality. Black box and White box testing techniques can be applied to verify the functionality. Data, Feature Engineering and Algorithms are the major part of the Machine Learning model. We will see how we applied different techniques to validate these.

This talk is focused on viewing the Machine Learning software as a whole and performing the Quality Analysis for it. We look at how different is testing a machine learning model from typical software testing. We will discuss the challenges that came across, the Process involved in building an ML model. We take an example of Search Relevance for an explanation. We will dive into the areas where quality is assessed. The significant factors considered here are measuring Accuracy and Efficiency. We will look into the different black box testing techniques for different Algorithms. Let us also see how traditional testing is different from testing machine learning applications. I will go through different black-box testing techniques with examples following a live demo.

• Want to save 70-80% manual effort of automation script writing!!
• Wasting time in verifying xpath one by one?
• Want to complete automation script without wasting much time?
• Are you still wasting time in writing english manual test cases?

Selenium is a popular open source test automation tool, so popular it has its own conference event which run for 3 whole days across continents and is attended by hundreds of enthusiastic testers, and people wanting to know more about testing and automation. In my little experience of coaching for last 7 years, i feel what english language has done for common man[those from non english speaking countries] in earning bread and butter and giving them an elite status, selenium has done for manual testers.

Anyone and everyone wants to learn Selenium. The magic of selenium or the flexibility is that it can be learnt with your choice of programming languages. I have taught people Selenium with Java, C#, Python and Ruby largely in past these years. I wish to delve into the world of Javascript now and see what this has to offer to us. So this workshop is for those open minded souls out there who wish to pick up another language and learn, unlearn, relearn selenium with it.

'Testing is fun' - Michael Bodnarchuk, creator of CodeceptJS
I am pretty sure it is. But how many of us actually feel that way?
Set aside Testers with Development background and you will find testers struggling to get a good hold of different automation tools available in the market. I am no exception.

In the past five years, Automation is the buzz word in the Testing World. Fresh graduates or young professionals are absorbing the automation journey in their career path. But you will find very few Senior Testers or Lead Testers doing a good job at this new trend to succeed in the industry. They have either moved to managerial roles or on the Business side of the organization. While there is no harm in that but we are headed towards a time where we can not escape whatever automation brings to our industry.
Thus, let me introduce to you an Acceptance testing Framework which is - 'CodeceptJS'. It is an end-to-end NodeJS TestFramework.
You have the liberty to use any of the helpers -
1. Classic Selenium +WebriverIO
2. Protractor
3. Nightmare
4. Puppeteer
Few really useful features being -
-Interactive Debug
-Parallel Execution & multiple browsers
-Web & Mobile Testing
-Convert your scripts written in one of the helpers into another
-Scenario driver & cucumber-like BDD

In this talk, I will demonstrate how to get started with codeceptJS automated testing & take advantage of the userfriendly and multiple features to write, read and debug automation scripts.

Software Craftsmanship is a practice of continuous development and helping others learn the craft. As the world is evolving towards the era of scripting language, we will be sharing our experience of how we have grown as a team of all QA with programming language background to the niche typescript world contributor.
During this talk, we will be focussing on the shift of existing selenium to the actual implementation of Protractor test API. How and when to use a protractor? When to pick protractor as an automation tool for building a framework.

By the end of the session, one will be able to understand how can we start working on end to end tests for any JavaScript and TypeScript based application using Protractor, Cucumber framework, and Page Object Model design pattern.

We will also talk about configuration files, cucumber, typescript, promise handling in protractor.

Problem Statement:

As part of SDLC process:

1. Is your product quality impacted due to a smaller number of QA resources available in the team?
2. Are you waiting for QA resources to certify your code every time when you deploy? Is this impacting your product lead time (Speed to Market)?
3. Is your Product delivery timelines are impacted due to last minute defects identified?
4. Do you have your QA resources only in one location, but you want to “follow-the-sun” approach for Software delivery across multiple locations?
5. Do you have manual testers who are not skilled in programming, but you want them to execute automated test scripts w/o any training efforts and automation setup?
6. Would you like your team more agile and cross functional with Delivery?
7. Would you like to increase your QA team’s productivity while they invest more time in script development rather than script execution?

If answer is ‘Yes’ for above questions, "Zero touch automation" is the solution for above challenges that we have been facing part of SDLC.

Solution: Zero touch automation with cutting-edge technologies

In this session, I will cover how we solved this problem using innovative solutions, Cutting-edge technologies like NLP (Natural language processing), AI & Cloud solutions.

You will learn how AI, NLP integrated with core automation components to achieve Zero touch automation.

This solution is not just revolutionary, it is paradigm shift in test automation to get results to your email with detailed analysis of failure categorization with recommended actions to users.

I will also cover how E2E automation will be driven with decisions taken by machines based on what user is looking for . There is no manual intervention in this process. NLP and AI play key role to help machines to take decisions.

We will also cover how we empowered developer/release manager/any team member/Manager to trigger the scripts from their cell phone and get the detailed execution report without having any automation software installed in their computer or Phone.

We will be demonstrating how the request will be initiated from User, understand the need from user using NLP & AI , Fetching the code from bitbucket to select appropriate automation scripts , running them on Selenoid/docker server , storing results to MongoDB , receiving email with test results and Failure analysis.

What is the value of zero touch automation?

1. Enable speed to market: Now that Developers does not need to wait for QA resource, Changes can be certified quickly and ready to push to production. Lead time will be significantly reduced.
2. Increase quality: Now that test automation is easy and it can run multiple times in each environment, most of the defects will be uncovered and addressed before code goes to production.
3. Ease of test execution: Test execution will be very easy, no automation or framework setup required from user side. Test execution can be done 24*7.
4. Productivity: Increase QA team’s Productivity to focus more on script development rather than focusing on script execution and failure analysis .

When it comes to creating a project we always search for ideas that are open-source, easily scalable, ready to use.

We as humans always have a tendency to grow and expand and when it comes to customizable infrastructure it's a big Yes.

In this talk, we will be focussing on how we can save the cost of actual servers with easily scalable and manageable open-source software using an open-source tool - selenium grid. How we set up the nodes and hub altogether to make cross-browser, multi-platform execution takes place.

This talk will answer the questions "Will you be able to run your automation scripts without connecting the USB cable to your device in a limited range?"

If we have to ask ourselves where is the mobile and when is it required? So the answer would be its 'Everywhere'!!

It's next to possible to imagine a day without looking at mobile or using apps for that matter- Google, Facebook, Whatsapp, etc.

When it comes to using it so testing comes hand-in-hand with it. As we develop applications over a mobile device, be it any platform ios, Andriod and windows we shall also think about usability and end-to-end testing.

In this talk, we will discuss how Appium worked as a bridge between devices and controllers.

Everyone starts off with a simple grid setup which involves a hub and one or more nodes.

This traditional setup is a good start but the moment one starts to get serious with the selenium grid and decide to house their own selenium grid for their local executions, that is when issues start.

My experiences with the Selenium grid in the past couple of years has led me to get introduced some of the most prevalent problems with maintaining an in-house selenium grid.

• Nodes get unhooked randomly due to network glitches.
• Nodes introduce false failures due to memory leaks.
• Selenium Grid running out of capacity.
• Nodes require OS upgrades/patches etc.
• Needing to deal with auto upgrades by browsers (especially chrome and firefox)

Some of these issues I managed to fix by building a "Self Healing" Grid wherein the nodes automatically get restarted after they have serviced "n" tests. But that still didn’t solve many of these other problems.

That was when I felt, what if there was an on-demand selenium grid.

What if the Grid could do the following ?

• The Grid auto scales itself in terms of the nodes based on the current load.
• The Grid does not require a lot of infrastructure to support it.
• The Grid can plug itself into some of the cloud providers or leverage a solution such as Docker so that the nodes can be spun and shutdown at will.

That was how the idea of "Just Ask" an on-demand grid was born.

Just-Ask is an on-demand grid. It has no nodes attached to it.

It’s designed to spin off nodes on demand, run test against the newly spun off node and after test runs to completion, clean-up the node as well. The node can be backed by anything. It could be Docker (or) it could be a VM running on any of the popular clouds.

The session aspires to walk the audience through with my experiments with the selenium grid, my learnings on the selenium grid internals and how I used all of that knowledge to build my own On Demand Selenium Grid. What better avenue to share these learnings than a Selenium Conference.

The session will introduce the audience to the grid internals and their concepts such as

• What is a Selenium Remote Proxy ? What is it used for? What can you do with it?
• What is a Hub (or) Node level Servlet ? When would you need one ?
• All of this followed by a quick demo on "Just Ask", the on-demand grid that I have built and open sourced here: https://github.com/rationaleEmotions/just-ask

Virtual mobile devices (emulators/simulators) are a cost-effective and straightforward alternative to testing on physical devices. We showcase how to set-up and scale an Android emulator farm using Appium, Docker and SQS and how it fits into our larger testing and quality strategy.
Maintaining physical test devices for mobile automation can be expensive and time-consuming. On top of the initial investment, you need to consider maintenance cost, replacement devices and efforts for manual scaling. On the other side of the spectrum, cloud providers take care of these restrictions, but their services can come at a hefty price tag, especially when your use-case requires a large number of devices. We present a middle path and demonstrate how to use virtual devices to build a reliable and scalable in-house device lab using Docker and Appium.

It is always challenging for us while dealing with Xpath and Css in iOS Mobile Automation.Even we can't think to do iOS native app mobile automation without them .

So problems we need to face are

• High Maintenance
• High Execution Time
• Complex installation
• Additional Web server

Do we have some solutions for this problem - The answer is Yes !!! -> KIF(Keep It Functional)

Quick Description :

To support the continuously evolving software industry, DevOps and Agile practices have established themselves as a big helping hand. With ever-increasing large scale digital transformation projects, organizations realize that the only way to shorten the product development cycles is to automate the processes involved in the delivery pipeline.

There are many tools and technologies to establish DevOps, but one of the most important ones is containerization, which gives flexibility in application packaging and automating a number of processes involved in the delivery pipeline. Containers have all the needed configurations and dependencies, for faster delivery of committed items to the customer. Docker is an open-source solution that makes it easier to create, deploy and run applications securely using containers. Therefore, quick feedback and testing applications with different environments can be achieved using Selenium with Docker.

My Proposal :

We can reduce the significant effort in provisioning, maintaining, running the Test Automation on different environment setup using, Docker with Test Automation. Some of the benefits are:

Benefits of using Docker with Selenium:

1. Test an application on multiple browsers and their different versions without installing them on multiple machines
2. To test multiple application builds concurrently without updating the code for different environments
3. To test your applications on multiple devices/OS/Browser combinations on a single machine.As Selenium within Docker will make it much quicker to test different environments by using the preconfigured containers.
4. Also, I will cover topics like how to set up the Selenium multi-environment setup using Docker concepts like Compose file, etc.

Best practices using Selenium with Docker

1. Common Dos and Don'ts for this setup.
2. How to avoid over maintenance of your Test Harness.
3. Infrastructure setup

Increase the power of your Test Automation Setup with Kubernates:

How to handle Selenium grid-based dockerize nodes with Kubernates.

XPath Generator is an API developed in Java which is free and opensource. It helps any test automation enthusiastic people to capture xPaths and other Selenium locators for any given web URL. It will generate the optimised xPath and show how it was derived.

A key requirement for successful test automation is to get past the brittle or fragile nature of test scripts. Any Selenium (or Appium) developer has encountered the dreaded "NoSuchElement Exception". A locator that worked yesterday may fail today. What's a test engineer to do?

Fortunately the field of AI provides promising solutions and allows for the creation of self-healing tests. Tests that can find elements across all environments. Tests that can learn from "human-in-the-loop" intervention and work perfectly thereafter. Imagine automated tests that "just work"!

This session will look at how to apply the latest in AI and Machine Learning technologies to improve your test scripts. With the plethora of new open source AI libraries made available by companies such as Google, the ability to leverage AI in your applications is more accessible than ever.

This session will be a primer on AI technologies and how they can be utilized for perfect test automation.

Did you know that Tesla can deploy firmware updates to its Model S seconds after a developer updates a module to improve the car’s acceleration performance or refine the car’s console?

Streamlined development-to-deployment efficiency arises elegantly from a smart automated testing strategy. In a world where time is precious spending hours to execute the tests is an

outdated approach and hence it is necessary to fill the gap using the latest technology. Our idea is to build an infrastructure that supports faster execution of the automated tests.

We plan to set up a selenium hub and selenium nodes in docker containers and once the nodes are registered to the hub ,initiate the process of test execution. This will parallelize the

tests and establish 1:1 mapping between the tests and containers .

Basic flow:

1.Automation framework will count the tests to be executed.

2.Set up the infrastructure of hub and nodes containers based on the count.

3.Check if the hub is ready to accept request.

4.Parallelize tests and send them to the hub for execution.

5.After completion of the execution kill the containers.

Benefits:

1.Reduce Execution time significantly.

2.Disposable runtime environments.

3.Enhance development and release cycle.

4.Lets new features and improvements reach the customer faster.

5.Save time and money of the company in long run.

6. No requirement for any reserved infrastructure

7. Cloud native and portable

Most of the time, Non-Functional reports are generated from different teams at the last moment of the release and so are chances of last-minute delays occur. So, how about keeping a track on NFT reports during the test automation execution.