Problem Statement

Though Selenium taking most of the stake in the UI test automation tool market comfortably, there were always challenges that were for selenium test automation engineers are handicapped with especially when dealing with modern JS technologies. For example,

• dealing with DOM elements to solve the stale / loading / non-interactable elements,
• handling full screenshots to know how the elements at the left, bottom, etc,
• measuring the performance of request and response resources at different speeds,
• monitoring the memory of the pages, controls, etc,
• attaching to an existing browser for debugging the failed scripts and many more.

These problems were largely resolved with the integration of selenium with devtool protocols. And that makes the selenium engineer's life merrier than before.

Why this proposal can be different from others?

1. Our solution can be executed as independent with chrome dev tools or with selenium. That gives the power back to the automation engineer to choose what and how to debug/run their tests.
2. We used this solution for one of our largest enterprise customers and moved this solution to public repository this week (for this conference and beyond). With that said, we tested reasonably with more than 10,000+ test scripts and more than 1M tests.
3. The present solution is completely (100%) packed with all Chrome Dev Tools API in Java and with that said, any Java Selenium automation engineer can bind in minutes for their existing code base with no additional dependencies.
4. Finally, we love to present at the local home to start our selenium conference campaign. Where else?

Solution:

The present proposal largely connected with Chrome and Selenium in Java language. However, there is no limitation to expand the boundaries for other language bindings and browsers.

Google Chrome, the most picked browser for browsing, which makes it the primary concentration for developers and testers. DevTools is one such boon for developers, testers especially the new aged test automation engineers. With that said, we built the following design pattern to allow chrome dev tools API to marry Selenium using debugger address / remote targets.

Selenium Devtools

"This world is such a beautiful place to live in." If you can read the first sentence without any screen readers or assistance, you're privileged. As technologists, shouldn’t we be more empathetic towards differently-abled people and make all parts of our website accessible to them? In my humble opinion, the true power of technology can be identified when it reaches out to people of all kinds having different physical or psychological challenges. We not only legally bound to provide Accessibility but also it should be considered as our moral responsibility.

As testers, we have a wonderful opportunity to contribute to Accessibility by ensuring that the site is accessible in many different ways. Although it is impossible to identify all the issues that exist around Accessibility in the world, we are lucky enough to still understand a majority of them. With this understanding, there have been many measures taken in order to make your site accessible. But, don’t forget - we are QA engineers. We got to ensure that the site is accessible as per the standards set by WCAG 2.0 (AA) by testing for accessibility using various tools and techniques.

With an ever increasing no of businesses being conducted on web the testing need to write automated tests for the app's UI is something that can never be ignored. As you all know Selenium provides an API that enables us to do this quite effectively.

However, when tasked with setting up the automation framework, there are a lot of questions that arise in the minds of aspiring test developers regardless of what level they are in their career.

Some of such questions are:

1. How does one actually go about the business of building a robust and effective automation framework on top of selenium?
2. What are the elementary building blocks to include in the framework that an aspiring automation developer should know of?
3. How should we model our tests? XUnit style vs BDD?
4. Are there good practices, sensible design patterns and abstractions that we can follow in our code?
5. What are some of the anti patterns/common mistakes we should avoid

A lot of literature, documentation and blogs exists on these topics on the web already.

However In this talk,

I would combine this existing knowledge and my years of experience in building automation frameworks and breakdown these elements and walk you through exactly the sort of decisions/considerations and practices that you can take while starting to implement or improve the UI automation for your team.

Hope to see you there!

Browser tests are known to be the flakiest ones. This is partly because browser infrastructure is complicated to maintain. But the second reason is – mainstream browser automation tools such as Selenium server are far from being efficient.

During my previous talks I was speaking about Selenoid - a truly efficient replacement of the standard Selenium server. This year I would like to do a live demonstration how to organize a fault-tolerant and easily scalable Selenium cluster using virtual machines in the cloud. I will start by setting up Selenoid and show its powerful features like video recording, live tests debugging, manual testing and many more. Then I will configure Selenoid to send logs and recorded videos to S3-compatible storage. Finally we will run a Ggr load balancer instance allowing to use all running Selenoid nodes and organize a single entry point to the cluster.

Onestic QA department made and maintained test suites, which was a huge bottleneck in our development process.

In order to solve it, management bought us an idea: our developers must be able to build their tests using a DSL framework without friction. QA maintain a big library that provides resources to developers. From there, they were free to extend this library in each project. With all of this we have a CI process, a 1 hour execution to monitor our results and of course our bot, SpongeBot.

SpongeBot can check 40+ e-commerce sites, with 4+ environments, for desktop and mobile platforms always available for developers.

With this solution we achieve to decentralize the work, add value testing production and the confidence in SpongeBot notifications if something goes wrong.

Are you the only one contributing to your test suite, because others feel too intimidated to contribute to it?

Have you ever looked at a test fail and not known where to investigate first?

This talk will cover techniques for uncomplicating your test suites, so they can clearly test exactly what's needed to be tested, allowing others to contribute tests. When the process of writing tests is shared, everyone has an impact on quality, and that can be quite contagious for a company's culture.

In addition, this talk will cover techniques for failing transparently, so you will no longer feel stressed triaging test fails. Quality is something shared by the entire team, so anyone should be able to triage test fails. When tests fail for a specific, clear reason, you'll be thanking your past self for writing logic to fail transparently, saving you half a day going down a rabbit hole.

Automated tests can provide results faster and it’s possible to execute them more frequently than manual ones. That allows to test earlier in the development process, decrease overall time needed for tests and what is probably the most important it’s possible to release and deliver business value faster and more frequently.

But what if we have more and more tests and even automated execution of them takes too much time - 10 minutes... 30 minutes... maybe even hours? Should we consider the ability to execute full tests set just a few times a day as something normal? Is adding more compute resources the only option to reduce the execution time? Or maybe there are too many high-level tests and some of them should be replaced by low-level ones according to tests pyramid? Is the tests pyramid still valid in the cloud world?

During the presentation you will see how the serverless cloud services like AWS Lambda may be used to run tests in the highly parallelized environment that can speed up execution even hundreds of times.

The adoption of Artificial Intelligence is getting more traction, it is in need to enhance QA capabilities to cope up with these skills. Machine Learning is used extensively in retail applications for solving complex problems, one of them is solving the search relevancy. Showing the appropriate results for the user is important for the conversion rate to go high. As Machine Learning poses different challenges such as a Test Oracle, Fairness, Correctness and Robustness to do QA, We may need to follow different approaches and testing techniques to do the QA for Machine Learning models.

Different Machine learning types such as Supervised and Unsupervised Models have different characteristics and are used for different types of problems. Though these solves different complex problems, Machine learning Models also a unit of software code that needs to be verified as a normal software system. When a Machine learning model is seen as a whole system, it may look complex and unsolvable. We can group them into small modules and verify for quality. Black box and White box testing techniques can be applied to verify the functionality. Data, Feature Engineering and Algorithms are the major part of the Machine Learning model. We will see how we applied different techniques to validate these.

This talk is focused on viewing the Machine Learning software as a whole and performing the Quality Analysis for it. We look at how different is testing a machine learning model from typical software testing. We will discuss the challenges that came across, the Process involved in building an ML model. We take an example of Search Relevance for an explanation. We will dive into the areas where quality is assessed. The significant factors considered here are measuring Accuracy and Efficiency. We will look into the different black box testing techniques for different Algorithms. Let us also see how traditional testing is different from testing machine learning applications. I will go through different black-box testing techniques with examples following a live demo.

Puppeteer is a Node js library (developed by Google Chrome team) to control Chrome and Firefox and is getting lot of traction recently because of its amazing capabilities. It has already become so popular that it has got 50K+ stars on Github against Selenium's 15K+ stars.

In the last Google I/O event, this tool's capability has been showcased and it has been perceived as the next generation Web Test Automation Tool.

Is Puppeteer better than Selenium? The answer is 'No', and I will cover 'why' in detail during this talk. I will show some live examples to demonstrate that Selenium can also do all those advance things which Puppeteer promises to do.

We learn more from our failures than our successes. I want to share one of my failure stories, where I learned that coding is not the most important task in Test Automation.

A failure taught me that coding is not the most important task in Test Automation. I fixed a bug, added tests, and shipped it to production. After that, a bug report came in showing that the fix created a new bug. I felt bad, I did not understand how that happened. A hotfix solved the issue, but the root cause was not addressed: what did I miss?

By taking a step back, I understood the situation, it all happened by overseeing basic concepts in testing (like understanding how the system works). The problem was that coding had more priority than creating a test plan. After this, I prioritized tasks better to avoid this situation to happen again.

This talk shows that testing concepts are more important than ever, in a time where tools promise to do everything, we focus less on what and how to test, and more in using tools to test. It outlines how a test strategy can leverage a continuous testing setup. Finally, it shows that failing is ok, but failing again for the same reasons is not.

• Want to save 70-80% manual effort of automation script writing!!
• Wasting time in verifying xpath one by one?
• Want to complete automation script without wasting much time?
• Are you still wasting time in writing english manual test cases?

There are thousands ways if not more to setup your automation testing environment. It is often crucial when it comes to stability and maintainability. While you can build a custom framework that fits your needs there are often already sophisticated setups or frameworks available that get you up and running very quickly.

WebdriverIO is one of these frameworks that is written in Node.js. It gets you up and running within minutes and allows you to scale up your test suite while maintaining your execution time by running tests in parallel. With its huge project community it is an ideal choice for small as well as big projects that use modern frameworks such as React, Angular or Vue.js.

In this workshop you will learn everything you need to know to run a successful, stable and maintainable WebdriverIO framework. It starts with an introduction to the project and the basic concepts and will end with a sophisticated framework that includes testing strategies like Frontend Performance Testing as well as complex browser interaction with Puppeteer.

Leading-edge applications are dynamic and adaptive in capabilities that require people to use increasingly dexterous tools and supporting infrastructure, including microservices. All of these applications leverage data in new ways. Decoration and tagging of data with intelligent meta-data have become more important than data itself. To keep up with evolving needs, enterprise devs across industries are shifting from traditional app architectures in favor of more fluid architecture for building data-centric applications.

Microservices break traditionally structured applications into manageable pieces that can be developed and maintained independently. microservices are often decoupled, allowing for updates with little to no downtime, as the other components can continue running.

Moving to distributed Microservices ecosystem brings its own challenges; Among them is the loss of visibility into the system, and the complex interactions now occurring between services. Monitoring these applications only reports the health of it but Observability provides granular insights about the behavior of the system along with rich content. In this talk, we will cover the difference of Monitoring and Observability, data path engineering challenges, pillars of observability, distributed tracing of various microservices, testing in distributed microservices ecosystem, automated observability, etc.

It is predicted that till 2025 there will be over 75 billion devices spewing 180 zettabytes of data and generating up to 6 trillion dollars. That enormous increase force companies to introduce a continuous approach to deliver the product as fast as possible and be able to compete on the market.

The main question is how to test application for end user among so much hardware equipment and ecosystems combining HW, FW, mobile devices and complex backend architecture? Considering all factors and possible obstacles is it for companies a real “A New Hope” for companies or just simply “Mission Impossible”?

I will take the participants on a journey to the IoT world. It will be a talk about the challenges that any tester will face at some point. I will present the dangers, risks and snares but also good practices and practical approach to mobile E2E test automation for the IoT solutions in CI approach.

Technical examples will be presented using Python languages and supported by physical devices (mobile phones and IoT equipment).

Selenium is a popular open source test automation tool, so popular it has its own conference event which run for 3 whole days across continents and is attended by hundreds of enthusiastic testers, and people wanting to know more about testing and automation. In my little experience of coaching for last 7 years, i feel what english language has done for common man[those from non english speaking countries] in earning bread and butter and giving them an elite status, selenium has done for manual testers.

Anyone and everyone wants to learn Selenium. The magic of selenium or the flexibility is that it can be learnt with your choice of programming languages. I have taught people Selenium with Java, C#, Python and Ruby largely in past these years. I wish to delve into the world of Javascript now and see what this has to offer to us. So this workshop is for those open minded souls out there who wish to pick up another language and learn, unlearn, relearn selenium with it.

Typically, industry software testing practice is to separate load testing from functional testing. Different teams with different skills and expertise do their testing at different times, and each evaluates the results against its own criteria.

Anton Angelov will present to you how you can reuse your functional tests for load testing. We will review the design of a load testing engine that will reuse the same functional tests and at the end of the execution, generate a comprehensive report. We will also talk about how to utilize the new features in Selenium 4.0 in this load testing solution. As part of the reviewed tooling, there will be a discussion about the best possible checks you can do in your coded load testing library and the essential load testing metrics you need to collect during the run.

Software Craftsmanship is a practice of continuous development and helping others learn the craft. As the world is evolving towards the era of scripting language, we will be sharing our experience of how we have grown as a team of all QA with programming language background to the niche typescript world contributor.
During this talk, we will be focussing on the shift of existing selenium to the actual implementation of Protractor test API. How and when to use a protractor? When to pick protractor as an automation tool for building a framework.

By the end of the session, one will be able to understand how can we start working on end to end tests for any JavaScript and TypeScript based application using Protractor, Cucumber framework, and Page Object Model design pattern.

We will also talk about configuration files, cucumber, typescript, promise handling in protractor.

Problem Statement:

As part of SDLC process:

1. Is your product quality impacted due to a smaller number of QA resources available in the team?
2. Are you waiting for QA resources to certify your code every time when you deploy? Is this impacting your product lead time (Speed to Market)?
3. Is your Product delivery timelines are impacted due to last minute defects identified?
4. Do you have your QA resources only in one location, but you want to “follow-the-sun” approach for Software delivery across multiple locations?
5. Do you have manual testers who are not skilled in programming, but you want them to execute automated test scripts w/o any training efforts and automation setup?
6. Would you like your team more agile and cross functional with Delivery?
7. Would you like to increase your QA team’s productivity while they invest more time in script development rather than script execution?

If answer is ‘Yes’ for above questions, "Zero touch automation" is the solution for above challenges that we have been facing part of SDLC.

Solution: Zero touch automation with cutting-edge technologies

In this session, I will cover how we solved this problem using innovative solutions, Cutting-edge technologies like NLP (Natural language processing), AI & Cloud solutions.

You will learn how AI, NLP integrated with core automation components to achieve Zero touch automation.

This solution is not just revolutionary, it is paradigm shift in test automation to get results to your email with detailed analysis of failure categorization with recommended actions to users.

I will also cover how E2E automation will be driven with decisions taken by machines based on what user is looking for . There is no manual intervention in this process. NLP and AI play key role to help machines to take decisions.

We will also cover how we empowered developer/release manager/any team member/Manager to trigger the scripts from their cell phone and get the detailed execution report without having any automation software installed in their computer or Phone.

We will be demonstrating how the request will be initiated from User, understand the need from user using NLP & AI , Fetching the code from bitbucket to select appropriate automation scripts , running them on Selenoid/docker server , storing results to MongoDB , receiving email with test results and Failure analysis.

What is the value of zero touch automation?

1. Enable speed to market: Now that Developers does not need to wait for QA resource, Changes can be certified quickly and ready to push to production. Lead time will be significantly reduced.
2. Increase quality: Now that test automation is easy and it can run multiple times in each environment, most of the defects will be uncovered and addressed before code goes to production.
3. Ease of test execution: Test execution will be very easy, no automation or framework setup required from user side. Test execution can be done 24*7.
4. Productivity: Increase QA team’s Productivity to focus more on script development rather than focusing on script execution and failure analysis .