Patterns to Whole-Team Test Automation Transformation
Looking back at test automation in a product development team for describing patterns of success for research purposes, we identified themes where the experienced success significantly differed from what the literature at large was describing. With those lessons, I moved to a new organization and took upon myself to facilitate a transformation to whole-team test automation over multiple teams, one at a time, one after the other. In this talk, we will revisit the research from one organization two years ago with lessons from another in the last two years.
In this talk, I will introduce you to my core patterns to practice-based test automation transformation. I can't promise a recipe I would apply, as my recipe changes and adapts as I work through teams, but I can promise experiences of the same patterns working on multiple occasions as well as examples of how my go-to patterns turned out inapplicable. We'll discuss moving from specialist language to generalist language, visualizing testing debt and coverage, using visualization to showcase progress made of continuous flow of small changes, choosing to release based on automation no matter how little test automation there is, and growing individual competencies by sharing YOUR screen when working together.
Outline/Structure of the Case Study
I never know such a thing in advance. I start talks by collecting a coherent set of ideas that I believe belong together and only build the talk further if there is a stage for it. I have build some hundreds of talks, and believe this one is worthwhile a stage.
Learning Outcome
- What practices to use to increase your test automation impact and relevance in team
- How to move from subteam to whole team contributions on test automation
- What does success in test automation look like in organization, incrementally
Target Audience
managers, testers, developers
Prerequisites for Attendees
You would most likely pick up different things having different things to start with. But no particular knowledge requirement. Having experience / ideas on working on software development would be of use.
Video
Links
https://maaretp.com/myTalks.html has links to some of my past public talks. The research two years ago is https://www.academia.edu/49495368/Test_Automation_Process_Improvement_in_a_DevOps_Team_Experience_Report
schedule Submitted 10 months ago
People who liked this proposal, also liked:
-
keyboard_arrow_down
Tariq King - From Rigidity to Resiliency: An Evolutionary Tale of Selenium-Based Frameworks
45 Mins
Talk
Intermediate
Prior to Selenium, engineering teams around the globe were struggling to build and maintain their own web testing solutions. Most frameworks at that time neither supported testing the front-end behavior of web applications, nor executing tests across different browsers and operating systems. This forced teams to fill that gap with homegrown solutions and they were doing so in silos, without the support and backing of the open-source community or browser development companies. As we know, Selenium changed all of that and WebDriver has been a saving grace for development teams. However, the rise of Selenium unveiled a hidden set of web UI test automation challenges, many of which are still prevalent today.
Join Tariq King as he takes you on a journey to address those challenges through an evolutionary tale of test automation framework development. Based on a true story, the tale starts with replacing a rigid homegrown solution with a more flexible internal Selenium-Based Domain Specific Language (DSL), and then extending that into an external DSL with powerful test management and software process integrations. The next step in the evolution incorporates AI-based element selection and self-healing features into Selenium. Curious about where the story goes or how it ends? Tariq will share his thoughts on the future of web UI test automation with Selenium.
-
keyboard_arrow_down
Diego Molina - Fix a Bug, Become a Committer
480 Mins
Workshop
Beginner
Have you ever wondered how you can contribute to the Selenium project and become part of the team?
Do you wonder how the different Selenium projects work under the covers?
Selenium is a set of solutions that enables you to automate all major browsers with WebDriver, scale up and run tests in parallel through the Grid, record and play back user interactions with the IDE, and share all this knowledge though the Selenium website and documentation.
However, only a handful of volunteers work to maintain this gigantic effort. If you would like to enhance your own Selenium experience while contributing back to the software that has defined so many of our careers, come to this workshop. In it we'll dissect the different elements of Selenium, dive into its internals, learn how it was built and how to make changes to it, and even fix or improve something that you can contribute on the same day!
-
keyboard_arrow_down
Manoj Kumar / Puja Chiman Jagani - Driving Observability with Selenium Grid 4
Manoj KumarVP, Developer RelationsLambdaTestPuja Chiman JaganiTeam LeadBrowserstackschedule 9 months ago
480 Mins
Workshop
Intermediate
Selenium Grid can be a bit daunting to get up and running. Starting it is quite easy, but using it effectively can require pulling in third-party tools. In this workshop we’ll cover how to run your grid effectively, using best practices culled from several large grid installations.
-
keyboard_arrow_down
Saikat Sengupta / Mayank Joshi - WebAuthn Automation using Selenium
Saikat SenguptaSoftware EngineerMastercardMayank JoshiManager, Software EngineeringMastercardschedule 10 months ago
45 Mins
Demonstration
Beginner
The Web Authentication API (a.k.a WebAuthn) is a specification written by the W3C and FIDO Alliance. The API allows servers to register and authenticate users using public-key cryptography instead of passwords. This helps to strengthen the security posture by going password-less and enabling phishing protection. We can automate WebAuthn-based sites using Virtual Authenticators, and being W3C compliant, Selenium 4 makes it really easy (The Virtual Authentication feature in Selenium 4 is currently available only in Java and in a patch release of Python).
During the talk, we will explain what WebAuthn is and what virtual authenticators are with some quick demonstrations. Then, we will showcase how using Selenium 4 we can easily automate such cases. We are also going to cover the implementation in Selenium 3 (in case someone is still stuck with the old version :)
-
keyboard_arrow_down
David Burns - Working in the Shadow (DOM) - How to test Web Components
45 Mins
Talk
Beginner
As Web Components gain more traction in the web to help websites become more performant it has created a bit of a blind spot on how to test these new features.
In this talk, David will walk you through what Web Components are and how they are benefiting the web as we move forward. He will then walk you through how we can go about testing these and how, while well-meaning, other frameworks are making things a little harder to test. He will also go through some of the performance aspects of the Web Components and how this translates to your tests.
-
keyboard_arrow_down
Shivani Gaba - Free Your Frontend Client from Backend Cage
45 Mins
Talk
Intermediate
Testing your web application end to end through the GUI using Selenium represents your real users on your real system. However, it may not be the only scope in which you wish to test your UI.
Are you:
- facing issue that frontend is ready to be tested but the APIs it depends on aren’t available yet?
- unable to test user interface in scenarios for invalid responses or 5XX errors?
- experiencing flaky UI tests with slow API responses?
- noticing trouble to test different user workflows due to lack of control over third-party APIs?
- seeing data changes break your tests while functionality works?
If you find these situations relatable, then this talk is for you! In this talk, we look at API mocking as a mechanism to enable testing of frontend in isolation.Using a real-life example application, we’ll explore how to handle complex scenarios and form an effective testing strategy. You will gain insights on how, when, and—most importantly—why we should mock APIs. Together we will discover how development and testing can benefit from mocks.
Please note: at the end of this talk, you will have full access to the example application used during the talk for trying out mocking for themselves. -
keyboard_arrow_down
Puja Chiman Jagani - Selenium has a new trick up its sleeve to track failures
45 Mins
Talk
Beginner
As our systems and tests grow more and more complex we need to make sure that we have the tools to capture the root causes without spending hours or days chasing them down. This is where Observability becomes our best friend. Observability allows us to see what is going on inside a system based on what we think is crucial without trawling through logs! Just like any piece of software should be robust, scalable, maintainable, and reliable, it should also be observable. Observability makes the journey from identifying unexpected problems to identifying the root cause easier.
To do so, the code should record as much useful granular information as possible. Metrics, logs, and traces are three known ways of encapsulating granular information. They are the primary sources of information to help determine the state of the system at any given point in time.
Selenium 4 introduced a fully distributed Grid with multiple components that communicate over the network. Troubleshooting and diagnosing problems in this setup is a challenge. To tackle this, Selenium integrated OpenTelemetry’s tracing and event logs. This feature is now available out of the box by default when using Selenium.
The users now have more power in their hands!I will dive into Selenium's observability journey by discussing:
-
What is observability?
-
Need for observability
-
Understanding the three pillars of observability: Metrics, Logging, and Tracing
-
Generating telemetry data alone does not suffice. It is a process from design to deployment.
-
Full-stack tracing in Selenium (Grid and Java client library)
-
Explain how we, at BrowserStack, are benefiting and exposing this information to our users.
-
-
keyboard_arrow_down
Narmadha Ramasamy / Sameena Syed - Functional Programming in Test Automation
Narmadha RamasamySenior ConsultantThoughtworksSameena SyedSenior consultantThoughtworksschedule 10 months ago
90 Mins
Workshop
Intermediate
Functional programming(FP) exists from as long as object oriented programming. But most of the testing frameworks are written in OOPS style rather than FP. A typical automation tester knows little or nothing about it. Functional programming is a programming paradigm in which everything is written in terms of functions. This makes the code more modular, bug-free and parallelizable. Hence, it is important to first understand functional programming before avoiding it.
In this workshop, we will introduce functional programming in testing world. We will help you understand some of the hard concepts of FP, like Currying, Higher order functions, and demonstrate how they can be used in writing an automation framework which is scalable and easier to maintain.
We'll use Selenium 4.0 to automate a few UI scenarios, covering functional programming concepts and a code walkthrough to its object-oriented counterpart. This will entice viewers to choose programming style based on its benefits rather than its simplicity.
-
keyboard_arrow_down
Sowmya Sridharamurthy - Automation in Digital transformation
45 Mins
Talk
Advanced
Businesses are rapidly changing and in constant look out for various revenue streams. The fight against disruption has driven businesses to reconsider their priorities and value steam. Now more than ever the need for automation is in rise. However, digital transformation does not end at automating the application development, it spans much beyond the teams reach.
To support this transformation curve, test engineers/automation engineers may have to do some heavy lifting and load sharing. What do I mean by this?, Let me ask you few questions
1. What is the scope of your automation?
2. What is your understanding of "Performance improvement"?
3. What is the reach of your Test strategy or in specific automation test strategy?
If the answer for all the above 3 questions starts and ends with "Application"/"solution offering" you are assigned to work on, then it's a good time to take a step back and visualise . Restricting yourself to partial play does not help business drive this transformation at its best.
In this talk, I would like you to walk through with me to understand how automation can rise above and beyond a limited scope enabling the businesses for success and imprinting oneself with greater learning opportunities. Its a Win-Win
-
keyboard_arrow_down
Eran Kinsbruner - Comparing Selenium 4, Cypress and Playwright Frameworks
45 Mins
Talk
Intermediate
As the cross-browser testing landscape evolves, there is a stabilization around 3 main leading frameworks that are used the most. Some are more adopted by frontend developers, while some are adopted by SDETs. To better make a selection or even choose more than a single framework you need to realize the core differences and use cases as well as benefits each framework provides to the practitioner.
In this session, Eran Kinsbruner, Chief DevOps evangelist, best-selling author from Perforce (Perfecto and Blazemeter specifically) will go through the core differences and advantages of all 3 frameworks with focus on Selenium 4 vs. the latest versions of Cypress and Playwright and provide a nice outline to better guide test automation engineers as they look for their next test framework.
-
keyboard_arrow_down
Parasar Saha - Cloud First Automation Approach – Road to Scalable and Faster Test Automation Pipelines
45 Mins
Talk
Advanced
Cloud has changed the way the software world works. Companies have accelerated their development and deployment cycles with 3x or more speed by adopting the cloud. But Testing in most organizations either is running locally or not taking advantage of the full potential of cloud infrastructure. Testing is the home-alone kid, left to spend the Christmas weekend without the bells the whistles that cloud has to offer.
Digital transformation in IT and Agile delivery model are pushing the testing teams to complete their testing in shorter testing windows. 26 % of the organization are moving to daily releases. 70% of the organization have adopted Agile in delivery. Even after having a high amount of automation coverage teams are struggling to keep up with the pace of testing asked by the business teams.
This talk is about how you can change the story and accelerate cloud adoption in your testing organization with Selenium-based frameworks. Through cloud adoption you benefit from box features of the cloud: On-demand scaling, Cost optimation, Security, Reliability, Geo -routing of your test infrastructure. Cloud adoption doesn’t need to limit to the execution of selenium tests but can help you in test management of automated selenium tests, test data generation, test reporting, test artifact store, and many more. It is not enough to have an Automation First Approach these days, you also need to have Cloud-First Approach for test infrastructure to get the best value from automation.
The talk elaborates on factors to consider while building up the cloud-based testing pipeline and how they will benefit your organization in terms of time to market, cost optimization, and raising the bar of quality.
Finally talks about the moonshot of having a Testing Cloud – AWS of Testing and what will it take to reach there as a testing community.
Take your testing to the cloud, and fly your flag high in quality!
-
keyboard_arrow_down
Michael Mintz - There's no time to write tests! Give me a recorder for generating them!
20 Mins
Case Study
Beginner
Time is short. You've been assigned to write hundreds (or even thousands) of tests in a small amount of time. Is there a faster way to create those tests than just writing out all the code yourself? Is there a faster way to determine the best CSS selectors to use in your tests than right-clicking and inspecting every element on a page that you need to interact with? If so, then the good news is that you may be able to record the actions that you need for tests, and then export those actions into a runnable script. The most popular tool used for this so far is Selenium IDE.
What are the drawbacks of using Selenium IDE (and other tools) for generating tests quickly:
- The selectors that are picked could be long, messy, and/or inefficient.
- The generated scripts can have a lot of extra lines of code. (Unnecessary imports, etc.)
- The generated scripts can be flaky when they run.
- Generating scripts like this could lead to a lot of duplicate code, which would be better off getting placed in reusable methods.
Is there another open source recorder alternative that can get around some of these limitations? Yes, there is: The SeleniumBase Recorder for Python. Demo included!
-
keyboard_arrow_down
Amuthan Sakthivel - CLEAN TEST DESIGN PRACTICES FOR EFFECTIVE SELENIUM AUTOMATION FRAMEWORK
45 Mins
Demonstration
Intermediate
Selenium is an amazing library for UI Automation. However, using it in a project needs a proper test design, a good approach and the best framework. I have listed some of the challenges that most people face during automation and will also brief on how we can solve those problems with effective design and approach.
Key Challenges :
1. Field level validations on a form containing several fields. (Many people will ignore these tests in automation as it may increase the number of lines of code and number of tests. Maintaining them is a difficult task)
2. Verifying the state of the web element before operating on it. (It is imperative to check whether a web element is present or visible or clickable or needs a scroll to operate. Also, different elements need different explicit wait times. Most probably we will have number of methods like waitForElementToBeClickable, waitForElementToBeVisible. This again results in increased lines of code. )
3. Assertion of multiple components on a page. (Sometimes we want to validate several items on a page and writing methods like getTitle, isCompanyLogoPresent, isFooterMenuPresent either results in multiple tests or poor test code spoiling the readability.)
4. CI/CD integration. ( Most of the companies were using Jenkins as their CI/CD tool to schedule tests and this is most probably maintained by Devops team. To set up Jenkins job we need a lot of permission. At the worst we need a machine/infra to run and schedule our tests)
How we can solve these commonly occurring problems?1. With the advent of functional programming, we can pass different behaviours to the test methods. In the demo, I will use BiPredicate Interface implementations to solve this problem with clean design.
2. Annotations in Java is very powerful but hardly used in Test Automation Frameworks. I will use reflections and annotation to solve this problem with a much cleaner design.
3. We can leverage Custom Validator classes and AssertJ to write some effective readable tests.
4. We can leverage Github Actions and the Github runner to set up Selenium Grid Infrastructure and run our tests without any additional infra.
Tech Stack : Java, Functional Interfaces, Selenium, AssertJ, Github Actions
-
keyboard_arrow_down
Sidhartha Shukla - Continuous Testing With Selenium Automation
20 Mins
Talk
Intermediate
Continuous testing is the process of executing automated test cases as part of the software delivery pipeline to obtain immediate feedback on the business risks and add deployment blocker in Pipelines in case of any Error during test suite execution. “Continuous Testing allows any change made in the code to be tested immediately. This avoids the problems created by having “bigbang” testing left to the end of the cycle such as release delays and quality issues.
Continuous testing is decisive for Full CI-CD in the project and to achieve it, we need to make our Automation very robust and impeccable.To implement continuous testing, we have to automate tests and create test suites(e.g. Regression, Smoke, Sanity, BVT, End to End, Functional). To design the automation suite, we will use the most stable automation framework (SELENIUM). Selenium allows the tests to be automated quickly, and execution is done as soon as a piece of code is ready at their respective stages of the release pipeline. So we will show in this topic that how quickly we can automate using selenium and it can be integrated into our our existing pipeline with ease. One point is preeminent to note that "Automation is the most integral part of continuous testing", and Selenium is the framework which helps us to achieve it.
Continuous testing can be achieved using any CI-CD like Bamboo, CircleCI, Amazon CDK pipeline, and the best part is that Selenium can be integrated with all the CI-CD easily and efficiently. So just to give the audience a proper visualization of Continuous testing, we will use Jenkins as our CI-CD tool.
Target Plan of Action :==> Define Scope and Estimation.
==> Identify important features and scenarios, which will be part of Smoke/Sanity/BVT suite.
==> Develop Automation Framework with Selenium
==> Design test suite.
==> Automate Deployment process for both DEV/QA/UAT environment.
==> Create Jenkins job for automation suite by integrating pom.xml with Jenkins.
==> Create pipeline with deployment and automation jobs.
==> Integrate Email notification for Jenkins job execution status.
==> Add GIT web hook to auto trigger jobs with code push.
Benefit Of Continous Testing:
==> We can add Deployment blocker in Pipelines in case of any Error during suite execution
==> More Test Reliability
==> Faster Release Rate
==> Reduce Costs
==> Easy Maintenance and Updates
==> Auto triggered jobs
==> Testing with more accuracy
==> Automated Email Notification for deployment and automation suite execution
==> Flawless and quick deployment
Advance Plan of Action:==> Automation regression suite with maximum coverage
==> Use script for job creation in Jenkins
==> Code quality with sonarqube integration
==> Slack notification for jenkins job execution report
==> Coverage report
==> Stabilize automation test scripts
-
keyboard_arrow_down
Dimpy Adhikary / Rashmi Konda - UI Testing Journey Towards Selenium 4
Dimpy AdhikaryQuality AnalystThoughtworksRashmi KondaTest ArchitectHappiest Minds Technologiesschedule 10 months ago
45 Mins
Demonstration
Beginner
Selenium 4 was released last year with new features and improvements. Being a selenium user it is high time to adapt to writing our automated tests in selenium 4 way. Here are some key questions we can ask before migrating selenium 3 tests to selenium 4.
- What does it take to migrate automated tests written using selenium 3 to selenium 4?
- Is it simple enough to start with when we have lots and lots of tests written using selenium 3?
- Is there any additional prerequisite or dependencies we may need to keep in mind?
- What are the challenges, errors and deprecation messages we may come across?
Migrating tests is a painful and time consuming process if not planned well. Thanks to the flexible architecture of selenium and detailed documentations, the process is actually simple if you are using any one of the officially supported client language language binding. In this presentation we are going to walk you all through the entire process of migrating selenium tests from 3 to 4 with key focus on the changes and new implementations.
-
keyboard_arrow_down
Tom HENRICKSEN - Humans are hard, Code is easy
45 Mins
Talk
Beginner
Are you a frustrated developer who feels like they know enough? However, the success you thought you would have is out of reach. You see others who make better strides but why? Is it a skills gap? Come learn how to set yourself apart as a developer and learn the skills of influence and collaboration.
-
keyboard_arrow_down
Gaurav Patel - Test Automation Using UiPath
45 Mins
Demonstration
Intermediate
UiPath Test Suite is not just the solution for Software testing team to test their software applications but its also a solution for RPA team to help them to test their robotic Process Automation more effectively and efficiently. RPA developers can now create tests straight from their workflows, data-drive those tests and execute them leveraging UiPath Orchestrator. Test Suite help RPA teams reduce robot maintenance allowing them to focus on building new robots.
-
keyboard_arrow_down
Chandrakanth Donagar - Instant Selenium Cucumber Automation using AI
45 Mins
Case Study
Beginner
Problem Statement:
On usual procedure we need to write the step definition and data storage classes wrt PM model. For each of the feature steps we need to map that to relative stepdefinition i.e method which performs action.
This eventually manual and time consuming effort.
Solution is to automate cucumber test script generation using NLP technique. Tensor flow uses features files and converts the features files to required step definition methods(Java).Separate set of classes will be generated as per the application UI pages. Object repository will be created dynamically when user supplies the steps in feature file.
Tool scans the html page and retrieves all required locators using intelligent locators generating strategy(to achieve this we need to supply strong data as part of feature files)
This eventually leads to no code automation using cucumber with web driver.