"This world is such a beautiful place to live in." If you can read the first sentence without any screen readers or assistance, you're privileged. As technologists, shouldn’t we be more empathetic towards differently-abled people and make all parts of our website accessible to them? In my humble opinion, the true power of technology can be identified when it reaches out to people of all kinds having different physical or psychological challenges. We not only legally bound to provide Accessibility but also it should be considered as our moral responsibility.

As testers, we have a wonderful opportunity to contribute to Accessibility by ensuring that the site is accessible in many different ways. Although it is impossible to identify all the issues that exist around Accessibility in the world, we are lucky enough to still understand a majority of them. With this understanding, there have been many measures taken in order to make your site accessible. But, don’t forget - we are QA engineers. We got to ensure that the site is accessible as per the standards set by WCAG 2.0 (AA) by testing for accessibility using various tools and techniques.

Shadow DOM is kind of web standard that developers use to encapsulate their custom HTML code and style components so that no other global style components can override their code. It ensures that a component will work in any environment, even if other CSS or JavaScript is run on the page i.e. Shadow DOM hides away the complexity of the components from the rest of the page because it renders separately from main DOM.

Selenium legacy selectors are not able to interact these custom elements under the shadow DOM so we need java script to interact with shadow DOM elements and we also require JavascriptExecutor interface for their execution.

We will also talk about the WebdriverIO tool v5.5.0, which implicitly support Shadow DOM elements.

The adoption of Artificial Intelligence is getting more traction, it is in need to enhance QA capabilities to cope up with these skills. Machine Learning is used extensively in retail applications for solving complex problems, one of them is solving the search relevancy. Showing the appropriate results for the user is important for the conversion rate to go high. As Machine Learning poses different challenges such as a Test Oracle, Fairness, Correctness and Robustness to do QA, We may need to follow different approaches and testing techniques to do the QA for Machine Learning models.

Different Machine learning types such as Supervised and Unsupervised Models have different characteristics and are used for different types of problems. Though these solves different complex problems, Machine learning Models also a unit of software code that needs to be verified as a normal software system. When a Machine learning model is seen as a whole system, it may look complex and unsolvable. We can group them into small modules and verify for quality. Black box and White box testing techniques can be applied to verify the functionality. Data, Feature Engineering and Algorithms are the major part of the Machine Learning model. We will see how we applied different techniques to validate these.

This talk is focused on viewing the Machine Learning software as a whole and performing the Quality Analysis for it. We look at how different is testing a machine learning model from typical software testing. We will discuss the challenges that came across, the Process involved in building an ML model. We take an example of Search Relevance for an explanation. We will dive into the areas where quality is assessed. The significant factors considered here are measuring Accuracy and Efficiency. We will look into the different black box testing techniques for different Algorithms. Let us also see how traditional testing is different from testing machine learning applications. I will go through different black-box testing techniques with examples following a live demo.

Puppeteer is a Node js library (developed by Google Chrome team) to control Chrome and Firefox and is getting lot of traction recently because of its amazing capabilities. It has already become so popular that it has got 50K+ stars on Github against Selenium's 15K+ stars.

In the last Google I/O event, this tool's capability has been showcased and it has been perceived as the next generation Web Test Automation Tool.

Is Puppeteer better than Selenium? The answer is 'No', and I will cover 'why' in detail during this talk. I will show some live examples to demonstrate that Selenium can also do all those advance things which Puppeteer promises to do.

• Want to save 70-80% manual effort of automation script writing!!
• Wasting time in verifying xpath one by one?
• Want to complete automation script without wasting much time?
• Are you still wasting time in writing english manual test cases?

Selenium is a popular open source test automation tool, so popular it has its own conference event which run for 3 whole days across continents and is attended by hundreds of enthusiastic testers, and people wanting to know more about testing and automation. In my little experience of coaching for last 7 years, i feel what english language has done for common man[those from non english speaking countries] in earning bread and butter and giving them an elite status, selenium has done for manual testers.

Anyone and everyone wants to learn Selenium. The magic of selenium or the flexibility is that it can be learnt with your choice of programming languages. I have taught people Selenium with Java, C#, Python and Ruby largely in past these years. I wish to delve into the world of Javascript now and see what this has to offer to us. So this workshop is for those open minded souls out there who wish to pick up another language and learn, unlearn, relearn selenium with it.

Selenoid is an alternative lightning fast open-source Selenium protocol implementation running browsers and Android emulators inside Docker containers. It is distributed with a set of ready-to-use Docker images corresponding to the majority of popular browser versions, has a one-command installation utility and works slightly more efficiently than traditional Selenium Grid.

This tutorial shows how to build efficient and scalable browser automation infrastructure using Selenoid and related tools. You will be taught why running browsers in containers is so efficient, how to easily install Selenoid and use its powerful features.

Best Practices - How to get the best 'Return ON Investment' (ROI) from your Test Automation.

This has been observed that, most of the test frameworks wont be able to survive due to lack of expertise, no maintenance, no best practices being followed, and finally your test automation will be dead after few months, and there is no "Return ON Investment" from this. This is the most common problem, most of the companies are struggling and finally back to square to the Manual testing.

My proposal : HOW to leverage your test automation in terms of best practices, best ROI, and how to adopt best automation culture in your organisation.

I strongly propose some of the important points/suggestions to achieve this in your Organisation/Team.

1. Test Automation Practices:

• Design Patterns (Web/Mobile/API)
• What to Automate/Not to Automate

2. Common Automation Frameworks at Org Level:

• How to design Generic Utilities, Libraries and different Components, which can be suitable for all the teams in the same Org.
• Best practices to design your Tests (Automation).
  • Common Design Patterns
  • Common application level and Page libraries
  • Best Practices to use Assertions in your Tests (How and What to write for assertions). Most of the people don't write proper assertions and this is making your test unreliable and no defects found during execution.

3. Inclusion of API/Backend libraries in your UI test automation as an external Maven/Gradle Dependencies to avoid un-necessary tasks, some of the important points to be considered here:

• User Creation from APIs (No need to automate user creation from web/app for all the test cases)
• API tests are stabled most of the time
• API calls takes lesser time as compared to web, hence include API calls in your UI/App framework to save time.
• Less flaky test

4. Best Code Review Process (Do not merge your code into Master without proper Code Review)

• Implement PR (PULL Request) Process
• Static Code Analysis using SonarQube, Cobertura, JACOCO etc..
• Get the benefits of Best Test Automation Quality Matrices
• Sometimes, Manual (Functional Tester) should review your code (Assertion, test steps and use cases) to get the best coverage

5. Quality is A Team responsibility:

• Developers, POs, Manual QEs and Automation engineers should be included to get an overview of test automation coverage.

6. Maintenance of the Frameworks

• After couple of months, it should not make your life miserable if you don't maintain your libraries and framework properly.
• Do not use Hard Coded values, make it simple and Generic.

7. Infrastructure Setup for Test Design and Test Execution:

• Proper Browser - OS lab setup
• Proper Mobile Labs setup with different Devices - IOT, iOS, Android, iPad, Tablets
• Proper CI - CD common configuration using Jenkins, Dev Ops, AWS, Docker and Cloud setup
• Handling multiple Docker nodes using Kubernates (use of Selenoid, GRID on Cloud)

In my talk, I want to share several approaches that help to improve automation testing in your company.

Do you want to spend less time developing UI automated tests? Maybe you want to reduce boilerplate code in UI tests? Or you think it is time to improve Page Object pattern that has been used for more than 8 years?

My first point will be about how the interface can help us to represent a new vision of Page Object and what benefits we could get from it.

The second - modern methods for coverage measuring are good only for development. However, the amount of useful information from coverage reports is extremely small.

It will a journey about autotest coverage. Imagine, that your project manager asks you - Do we have autotests for these buttons (forms, pages) or what autotests cover these Rest APIs with 'special' query parameters? I will show you two simple solutions that allow you to evaluate the coverage of tests written by test automation specialists.

a) API tests. There is a swagger and during his execution, we also collect a swagger. We count coverage as a diff of two swaggers. We can see what is covered, what is not and what data used.

b) Web tests. During the execution of tests, we collect information about elements, pages, and actions. As a result, we have such a format: page -> elements -> actions -> tests. All information representing in Chrome Plugin, that I will share with you.

And last, I am going to consider what cons have current TMS (QA time management systems) and what to do if you want to make an automation-first strategy on your project? Moreover, you look for one TMS for developers and QAs/or you want to save the Allure report of all autotests in one place?

Here you are! Allure Server has already released. I will overview the most important features and how to start using it from scratch.

If you have an interest in the above themes come to my talk.

When it comes to creating a project we always search for ideas that are open-source, easily scalable, ready to use.

We as humans always have a tendency to grow and expand and when it comes to customizable infrastructure it's a big Yes.

In this talk, we will be focussing on how we can save the cost of actual servers with easily scalable and manageable open-source software using an open-source tool - selenium grid. How we set up the nodes and hub altogether to make cross-browser, multi-platform execution takes place.

If you are tired of waiting for your selenium tests status and old fasioned ways of reporting tests status, you are in a good place.

We will show you our test environment setup and how we run over 2500 tests in about an hour. Not cool enough? Take a look how we make reports via slack.

Do you want to check if we can be even more faster?

One of the biggest limitations with most of the frameworks in selenium is that we need to wait till the end to get the final test case report. Only then we can raise the alert that a particular functionality is broken. By then, it may be too late! In addition to this, in most of the frameworks, the HTML report generated is hosted on our company's private network and we need to connect to the VPN if we are outside the office to open the report and check the related screenshots to see which test case has failed and why.

So, why not raise an alert as soon as the test case fails? And why not attach a screenshot along with that alert so that all the stakeholders can actually see what has failed in the application?

In this session, we will learn a simple yet a really useful way of sending an alert along with the failed test case screenshot by integrating slack with selenium which will help us alert all the stakeholders as soon as an automated test case fails so that everyone can react to it immediately and the damage will be minimised.

Everyone starts off with a simple grid setup which involves a hub and one or more nodes.

This traditional setup is a good start but the moment one starts to get serious with the selenium grid and decide to house their own selenium grid for their local executions, that is when issues start.

My experiences with the Selenium grid in the past couple of years has led me to get introduced some of the most prevalent problems with maintaining an in-house selenium grid.

• Nodes get unhooked randomly due to network glitches.
• Nodes introduce false failures due to memory leaks.
• Selenium Grid running out of capacity.
• Nodes require OS upgrades/patches etc.
• Needing to deal with auto upgrades by browsers (especially chrome and firefox)

Some of these issues I managed to fix by building a "Self Healing" Grid wherein the nodes automatically get restarted after they have serviced "n" tests. But that still didn’t solve many of these other problems.

That was when I felt, what if there was an on-demand selenium grid.

What if the Grid could do the following ?

• The Grid auto scales itself in terms of the nodes based on the current load.
• The Grid does not require a lot of infrastructure to support it.
• The Grid can plug itself into some of the cloud providers or leverage a solution such as Docker so that the nodes can be spun and shutdown at will.

That was how the idea of "Just Ask" an on-demand grid was born.

Just-Ask is an on-demand grid. It has no nodes attached to it.

It’s designed to spin off nodes on demand, run test against the newly spun off node and after test runs to completion, clean-up the node as well. The node can be backed by anything. It could be Docker (or) it could be a VM running on any of the popular clouds.

The session aspires to walk the audience through with my experiments with the selenium grid, my learnings on the selenium grid internals and how I used all of that knowledge to build my own On Demand Selenium Grid. What better avenue to share these learnings than a Selenium Conference.

The session will introduce the audience to the grid internals and their concepts such as

• What is a Selenium Remote Proxy ? What is it used for? What can you do with it?
• What is a Hub (or) Node level Servlet ? When would you need one ?
• All of this followed by a quick demo on "Just Ask", the on-demand grid that I have built and open sourced here: https://github.com/rationaleEmotions/just-ask

It is always challenging for us while dealing with Xpath and Css in iOS Mobile Automation.Even we can't think to do iOS native app mobile automation without them .

So problems we need to face are

• High Maintenance
• High Execution Time
• Complex installation
• Additional Web server

Do we have some solutions for this problem - The answer is Yes !!! -> KIF(Keep It Functional)

Quick Description :

To support the continuously evolving software industry, DevOps and Agile practices have established themselves as a big helping hand. With ever-increasing large scale digital transformation projects, organizations realize that the only way to shorten the product development cycles is to automate the processes involved in the delivery pipeline.

There are many tools and technologies to establish DevOps, but one of the most important ones is containerization, which gives flexibility in application packaging and automating a number of processes involved in the delivery pipeline. Containers have all the needed configurations and dependencies, for faster delivery of committed items to the customer. Docker is an open-source solution that makes it easier to create, deploy and run applications securely using containers. Therefore, quick feedback and testing applications with different environments can be achieved using Selenium with Docker.

My Proposal :

We can reduce the significant effort in provisioning, maintaining, running the Test Automation on different environment setup using, Docker with Test Automation. Some of the benefits are:

Benefits of using Docker with Selenium:

1. Test an application on multiple browsers and their different versions without installing them on multiple machines
2. To test multiple application builds concurrently without updating the code for different environments
3. To test your applications on multiple devices/OS/Browser combinations on a single machine.As Selenium within Docker will make it much quicker to test different environments by using the preconfigured containers.
4. Also, I will cover topics like how to set up the Selenium multi-environment setup using Docker concepts like Compose file, etc.

Best practices using Selenium with Docker

1. Common Dos and Don'ts for this setup.
2. How to avoid over maintenance of your Test Harness.
3. Infrastructure setup

Increase the power of your Test Automation Setup with Kubernates:

How to handle Selenium grid-based dockerize nodes with Kubernates.

Add robotic arms to your testing skills

Marvel universe has always fascinated me, the next gen AI bots like JARVIS and FRIDAY used in Marvel cinematic world had been inspiring me a lot. When I watch them, I even envisioned myself to be a creator of one such AI for self-use.

With more experience and exposure in my field (Digital Quality Space), my vision had turned into a quest, my desire turned out to become “EiNo” (Virtually Interactive Bot in Testing).

EiNo with NLP enabled, would help any tester to perform testing related activities, enable self-learning systems and use analytical tools to predict and recommend. EiNo ensures standardization of scripts and reduce manual efforts.

EiNo aids users to

• Automatic Framework Development
• Execute on specific browser and OS combination
• Ensures Auto Healing of automation scripts
• Answers to some fundamental testing related queries
• Validates the web application with respect to NFR (Non-Functional Requirements)
  • End User performance
  • Accessibility
  • SEO
  • Security
• Ensures proper code quality is maintained
• Integrates with Test Management tools like JIRA, QTest …etc
• Creates business flows from production logs

These functionalities are a call away ensuring testing becomes easier and faster.

EiNo do not stop here to ensure we test smarter, EiNo works with time series databases to gather the data, machine learn and represent it, in a meaningful way. So as to predict the future areas to focus on your application development.

This as wholesome package makes tester to test with right tools so that the testing becomes reliable, easy, smart and of course fast.

I would this abstract by quoting that, a good product quality would achieved with a good fail first strategy and a shift left approach mindset, EiNo becomes the right platform for the same.

A key requirement for successful test automation is to get past the brittle or fragile nature of test scripts. Any Selenium (or Appium) developer has encountered the dreaded "NoSuchElement Exception". A locator that worked yesterday may fail today. What's a test engineer to do?

Fortunately the field of AI provides promising solutions and allows for the creation of self-healing tests. Tests that can find elements across all environments. Tests that can learn from "human-in-the-loop" intervention and work perfectly thereafter. Imagine automated tests that "just work"!

This session will look at how to apply the latest in AI and Machine Learning technologies to improve your test scripts. With the plethora of new open source AI libraries made available by companies such as Google, the ability to leverage AI in your applications is more accessible than ever.

This session will be a primer on AI technologies and how they can be utilized for perfect test automation.

1. Are you Confident about your test coverage?
2. Do you have millions of permutations and combinations to be covered in testing?
3. is automation the solution to cover million combinations?
4. why can not my system decide what test cases to be executed to give complete test coverage?
5. As part of CI/CD , instead of executing same test cases irrespective changes going in, would you like to execute more appropriate test cases ?

I will be presenting quick demo on statistical approach and strategy which can help us to generate appropriate number of test cases automatically based on the impacted areas and business metrics . Let me share an example below.

Let’s say we are testing any eCommerce application Flipcart or amazon. We will have to cover multiple combinations like different categories of items, different payment types, shipping types, Promotions, user types etc.. . I will use little bit of sets theory here.

Let us take below 5 sets as an example. Assume that there is new payment type is getting introduced as “Paypal” as enhancement. If we need to make sure this new payment type is working with all products, all shipments types , all user types etc… we will have to cover all permutations and combinations which is nothing but Cartesian product of all these sets. The Cartesian product of below 5 sets is - 6*3*2*4*3 =432 test cases.

Do we really need to execute 432 test cases just for one new payment type ? we just need to execute 24 test case to give confident test coverage .This can be done with OATS tool which is developed with intelligence with Business user Metrics + Orthogonal array testing approach ( OATS) approach . OATS tool will generate required number of test cases with simple input and one click .

There are 2 key areas of input that we consider before we generate optimized tests.

1. Business user metrics - Example : if you look at the below payment set , we would like to know what % of customers using each payment type in overall successful transactions. let us assume 60% Credit card payments , 20 % paytm and 20% rest of payment types . These metrics will help tool to give more weightage to Credit card test cases over other payment types. This make sense because we need to focus more on the areas our end users are using.

2. Impacted areas: let us take an example that there is new payment type (AmazonPay) is getting introduced as en enhancement. We should give more weightage to payment set and make sure all existing payment types are tested with different combinations.

Set1 - PAYMENT = { Cash, Credit card , Paytm , AmazonPay, Debit card}- # elements - 6

Set2 – SHIPPING – {Shiptoday, Prime Shipping, ship Tomorrow}-# elements - 3

Set3 – USER TYPE - { Guest User , Logged in user }-# elements - 2

Set4 – PRODUCT TYPE – {Electronics, Groceries, Books , Stationary}- # elements - 4

Set5 – PROMO TYPE – { Promo1 , Promo2 , Promo3}-# elements – 3

I will be covering how this can be done in detail during presentation along with Tool Demo which was built in with an intelligence using sets theory . you will get an answer on why only 24 test cases are good enough instead of 432 test cases for the change new payment type got introduced.

Advantages :

1. Optimized coverage with less test cases

2. Tester will be confident about test coverage

3. This tool output can be integrated with Selenium key word driven framework . All this happens during run time w/o manual intervention.

4. This strategy is good fit for automation testing .

5. if any combinations are invalid to test, those combinations will be excluded while generating tests.

6. There are tools in the market to provide statistics about untested code (Ex: JaCoCO ). These statistics can be used as input for OATS tool. We have not done this yet.

Limitations:

1. We have seen great value (reduced bug escape) applying this strategy to automated tests. However there is limitation in applying the same strategy to manual exploratory , Adhoc testing.

2. We have not tried this strategy for performance testing yet.

3. There are corner scenarios where human can only think will not be covered by this tool . These scenarios to be appended to generated test cases .

Tesbo is a hybrid Test Framework built top on the Selenium which allows you to use the plain English language to define the test which helps you to reduce the development and maintenance time and efforts. sounds similar to the BDD framework (i.e. Cucumber, Specflow, Gauge, etc...) ??? But No It Isn't... It's much better than Cucumber.

In this session, we will look into why we have developed Tesbo and how it will reduce the development and maintenance times of web and mobile test script.

This session will focus on the

• Introduction of Tesbo
• Advantages of Tesbo
• Feature description
• Test Development and Execution demo
• Future plans for Tesbo