We learn more from our failures than our successes. I want to share one of my failure stories, where I learned that coding is not the most important task in Test Automation.

A failure taught me that coding is not the most important task in Test Automation. I fixed a bug, added tests, and shipped it to production. After that, a bug report came in showing that the fix created a new bug. I felt bad, I did not understand how that happened. A hotfix solved the issue, but the root cause was not addressed: what did I miss?

By taking a step back, I understood the situation, it all happened by overseeing basic concepts in testing (like understanding how the system works). The problem was that coding had more priority than creating a test plan. After this, I prioritized tasks better to avoid this situation to happen again.

This talk shows that testing concepts are more important than ever, in a time where tools promise to do everything, we focus less on what and how to test, and more in using tools to test. It outlines how a test strategy can leverage a continuous testing setup. Finally, it shows that failing is ok, but failing again for the same reasons is not.

 
 

Outline/Structure of the Talk

  • My background and how I ended up in testing
    • I knew how to code, but I didn’t know how to test
  • Everyone makes mistakes, even the so-called “experts”
    • How I failed to fix a bug
  • Learning from your mistakes and planning before automating
    • Understanding the system under test and its important flows
    • Designing a simple plan to cover the important flaws
    • How test automation comes at the end, not at the beginning
  • How I succeeded to fix a bug
    • A practical step by step that shows how planning is executed and the main feature flows are covered
    • How creating automated tests using a test framework saves time
    • How planning for testing can help to build a continuous testing pipeline
  • Conclusions
    • Effort in testing is reduced when testing the right thing
    • Coding is important but not the most important task in testing
    • Seeing our failures as an opportunity to learn

Learning Outcome

  • How to view our failures as something positive by identifying the problem and what went wrong in the solution
  • What are the tasks for Test Automation and how to prioritize them
  • How to start from a problem statement and develop each the Test Automation tasks to solve the problem
  • Get a Continuous Testing setup as a result of solving the correct problem in Test Automation

Target Audience

Automation Engineers, QA Automation, Test Managers, Testers

Prerequisites for Attendees

Attendants should have a notion of testing, automation, and pipelines (continuous integration).

schedule Submitted 1 month ago

Public Feedback

comment Suggest improvements to the Speaker
  • Pallavi R Sharma
    By Pallavi R Sharma  ~  1 week ago
    reply Reply

    Hi Diego

    This looks like a very relevant topic specially in today's times when everyone is jumping to learn a programming language and a tool and overlooking that knowing the art of testing is as well crucial. 

    I find a few things in structure of talk and learning outcome a bit confusing with the title, as in are we going to learn how to automate better or how to test better before we automate test? or what all automation techniques helps in testing better and why are they necessary or all of it

    sorry if i sound confused, i am trying to understand your talk and its content. its sounds very interesting and im intrigued. 

    • Diego Molina
      By Diego Molina  ~  4 days ago
      reply Reply

      Hi Pallavi,

      The main idea of the talk is to focus on what to test in order to test better before we even start writing automation code. To give context around this, I put myself upfront through a failure I had in my current job, where I fixed a bug and wrote a few automated tests before understanding first what should be tested.

      The talk also mentions some relevant points on what to take into account when writing automation code, so that part is also covered, nevertheless (as mentioned above), the focus is on taking one step back and figuring out what to test before deciding if we need to automate something or not.

      No need to be sorry, your question makes a lot of sense and it also points out that I should make that more clear.

      Please let me know if I was able to answer properly or if I need to elaborate more.


  • Liked Simon Stewart
    keyboard_arrow_down

    Simon Stewart - Selenium: State of the Union

    45 Mins
    Keynote
    Intermediate

    What's the latest in the world of Selenium since the last SeleniumConf? In this talk, we'll cover Selenium 4, how the project is run, how you can participate, reminiscing on the Old Days, and reflect on something that at first doesn't appear to be related to Selenium, yet is intimately related to it.

  • Liked Krishnan Mahadevan
    keyboard_arrow_down

    Krishnan Mahadevan - My experiments with Grid

    45 Mins
    Tutorial
    Intermediate

    Everyone starts off with a simple grid setup which involves a hub and one or more nodes.

    This traditional setup is a good start but the moment one starts to get serious with the selenium grid and decide to house their own selenium grid for their local executions, that is when issues start.

    My experiences with the Selenium grid in the past couple of years has led me to get introduced some of the most prevalent problems with maintaining an in-house selenium grid.

    • Nodes get unhooked randomly due to network glitches.
    • Nodes introduce false failures due to memory leaks.
    • Selenium Grid running out of capacity.
    • Nodes require OS upgrades/patches etc.
    • Needing to deal with auto upgrades by browsers (especially chrome and firefox)

    Some of these issues I managed to fix by building a "Self Healing" Grid wherein the nodes automatically get restarted after they have serviced "n" tests. But that still didn’t solve many of these other problems.

    That was when I felt, what if there was an on-demand selenium grid.

    What if the Grid could do the following ?

    • The Grid auto scales itself in terms of the nodes based on the current load.
    • The Grid does not require a lot of infrastructure to support it.
    • The Grid can plug itself into some of the cloud providers or leverage a solution such as Docker so that the nodes can be spun and shutdown at will.

    That was how the idea of "Just Ask" an on-demand grid was born.

    Just-Ask is an on-demand grid. It has no nodes attached to it.

    It’s designed to spin off nodes on demand, run test against the newly spun off node and after test runs to completion, clean-up the node as well. The node can be backed by anything. It could be Docker (or) it could be a VM running on any of the popular clouds.

    The session aspires to walk the audience through with my experiments with the selenium grid, my learnings on the selenium grid internals and how I used all of that knowledge to build my own On Demand Selenium Grid. What better avenue to share these learnings than a Selenium Conference.

    The session will introduce the audience to the grid internals and their concepts such as

    • What is a Selenium Remote Proxy ? What is it used for? What can you do with it?
    • What is a Hub (or) Node level Servlet ? When would you need one ?
    • All of this followed by a quick demo on "Just Ask", the on-demand grid that I have built and open sourced here: https://github.com/rationaleEmotions/just-ask

  • Liked Marcus Merrell
    keyboard_arrow_down

    Marcus Merrell / Diego Molina / Manoj Kumar - Selenium Grid

    480 Mins
    Workshop
    Advanced

    Selenium Grid can be a bit daunting to get up and running. Starting it is quite easy, but using it effectively can require pulling in third-party tools. In this workshop we’ll cover how to run your grid effectively, using best practices culled from several large grid installations.

  • Liked Christian Bromann
    keyboard_arrow_down

    Christian Bromann - The Nuts and Bolts of WebdriverIO

    Christian Bromann
    Christian Bromann
    Software Engineer
    Sauce Labs
    schedule 1 month ago
    Sold Out!
    90 Mins
    Workshop
    Beginner

    There are thousands ways if not more to setup your automation testing environment. It is often crucial when it comes to stability and maintainability. While you can build a custom framework that fits your needs there are often already sophisticated setups or frameworks available that get you up and running very quickly.

    WebdriverIO is one of these frameworks that is written in Node.js. It gets you up and running within minutes and allows you to scale up your test suite while maintaining your execution time by running tests in parallel. With its huge project community it is an ideal choice for small as well as big projects that use modern frameworks such as React, Angular or Vue.js.

    In this workshop you will learn everything you need to know to run a successful, stable and maintainable WebdriverIO framework. It starts with an introduction to the project and the basic concepts and will end with a sophisticated framework that includes testing strategies like Frontend Performance Testing as well as complex browser interaction with Puppeteer.

  • Liked Dawid Pacia
    keyboard_arrow_down

    Dawid Pacia - Build your own Internet of Continuously Delivered Things

    Dawid Pacia
    Dawid Pacia
    Test Automation Manager
    Brainly
    schedule 1 month ago
    Sold Out!
    45 Mins
    Case Study
    Intermediate

    It is predicted that till 2025 there will be over 75 billion devices spewing 180 zettabytes of data and generating up to 6 trillion dollars. That enormous increase force companies to introduce a continuous approach to deliver the product as fast as possible and be able to compete on the market.


    The main question is how to test application for end user among so much hardware equipment and ecosystems combining HW, FW, mobile devices and complex backend architecture? Considering all factors and possible obstacles is it for companies a real “A New Hope” for companies or just simply “Mission Impossible”?


    I will take the participants on a journey to the IoT world. It will be a talk about the challenges that any tester will face at some point. I will present the dangers, risks and snares but also good practices and practical approach to mobile E2E test automation for the IoT solutions in CI approach.


    Technical examples will be presented using Python languages and supported by physical devices (mobile phones and IoT equipment).

  • Liked Virender Singh
    keyboard_arrow_down

    Virender Singh - Rise of Shadow DOM - Lets solve it through Selenium & WebDriverIO

    45 Mins
    Demonstration
    Beginner

    Shadow DOM is kind of web standard that developers use to encapsulate their custom HTML code and style components so that no other global style components can override their code. It ensures that a component will work in any environment, even if other CSS or JavaScript is run on the page i.e. Shadow DOM hides away the complexity of the components from the rest of the page because it renders separately from main DOM.

    Selenium legacy selectors are not able to interact these custom elements under the shadow DOM so we need java script to interact with shadow DOM elements and we also require JavascriptExecutor interface for their execution.

    We will also talk about the WebdriverIO tool v5.5.0, which implicitly support Shadow DOM elements.

  • Liked Srinivasan Sekar
    keyboard_arrow_down

    Srinivasan Sekar / Sai Krishna - Testing And Observability in an Integrated Microservices environment

    45 Mins
    Case Study
    Intermediate

    Leading-edge applications are dynamic and adaptive in capabilities that require people to use increasingly dexterous tools and supporting infrastructure, including microservices. All of these applications leverage data in new ways. Decoration and tagging of data with intelligent meta-data have become more important than data itself. To keep up with evolving needs, enterprise devs across industries are shifting from traditional app architectures in favor of more fluid architecture for building data-centric applications.

    Microservices break traditionally structured applications into manageable pieces that can be developed and maintained independently. microservices are often decoupled, allowing for updates with little to no downtime, as the other components can continue running.

    Moving to distributed Microservices ecosystem brings its own challenges; Among them is the loss of visibility into the system, and the complex interactions now occurring between services. Monitoring these applications only reports the health of it but Observability provides granular insights about the behavior of the system along with rich content. In this talk, we will cover the difference of Monitoring and Observability, data path engineering challenges, pillars of observability, distributed tracing of various microservices, testing in distributed microservices ecosystem, automated observability, etc.

  • Liked Anton Angelov
    keyboard_arrow_down

    Anton Angelov - Combining Load and Functional Testing- Reusing WebDriver Tests for Load Testing

    Anton Angelov
    Anton Angelov
    CTO
    Automate The Planet
    schedule 1 month ago
    Sold Out!
    45 Mins
    Demonstration
    Advanced

    Typically, industry software testing practice is to separate load testing from functional testing. Different teams with different skills and expertise do their testing at different times, and each evaluates the results against its own criteria.

    Anton Angelov will present to you how you can reuse your functional tests for load testing. We will review the design of a load testing engine that will reuse the same functional tests and at the end of the execution, generate a comprehensive report. We will also talk about how to utilize the new features in Selenium 4.0 in this load testing solution. As part of the reviewed tooling, there will be a discussion about the best possible checks you can do in your coded load testing library and the essential load testing metrics you need to collect during the run.

  • Liked Naveen Khunteta
    keyboard_arrow_down

    Naveen Khunteta - Best Practices to implement the test automation framework starting from Design - To -> Infrastructure - To -> Execution.

    45 Mins
    Talk
    Intermediate

    Best Practices - How to get the best 'Return ON Investment' (ROI) from your Test Automation.

    This has been observed that, most of the test frameworks wont be able to survive due to lack of expertise, no maintenance, no best practices being followed, and finally your test automation will be dead after few months, and there is no "Return ON Investment" from this. This is the most common problem, most of the companies are struggling and finally back to square to the Manual testing.

    My proposal : HOW to leverage your test automation in terms of best practices, best ROI, and how to adopt best automation culture in your organisation.
    I strongly propose some of the important points/suggestions to achieve this in your Organisation/Team.
    1. Test Automation Practices:
    • Design Patterns (Web/Mobile/API)
    • What to Automate/Not to Automate
    2. Common Automation Frameworks at Org Level:
    • How to design Generic Utilities, Libraries and different Components, which can be suitable for all the teams in the same Org.
    • Best practices to design your Tests (Automation).
      • Common Design Patterns
      • Common application level and Page libraries
      • Best Practices to use Assertions in your Tests (How and What to write for assertions). Most of the people don't write proper assertions and this is making your test unreliable and no defects found during execution.
    3. Inclusion of API/Backend libraries in your UI test automation as an external Maven/Gradle Dependencies to avoid un-necessary tasks, some of the important points to be considered here:
    • User Creation from APIs (No need to automate user creation from web/app for all the test cases)
    • API tests are stabled most of the time
    • API calls takes lesser time as compared to web, hence include API calls in your UI/App framework to save time.
    • Less flaky test

    4. Best Code Review Process (Do not merge your code into Master without proper Code Review)

    • Implement PR (PULL Request) Process
    • Static Code Analysis using SonarQube, Cobertura, JACOCO etc..
    • Get the benefits of Best Test Automation Quality Matrices
    • Sometimes, Manual (Functional Tester) should review your code (Assertion, test steps and use cases) to get the best coverage
    5. Quality is A Team responsibility:
    • Developers, POs, Manual QEs and Automation engineers should be included to get an overview of test automation coverage.
    6. Maintenance of the Frameworks
    • After couple of months, it should not make your life miserable if you don't maintain your libraries and framework properly.
    • Do not use Hard Coded values, make it simple and Generic.
    7. Infrastructure Setup for Test Design and Test Execution:
    • Proper Browser - OS lab setup
    • Proper Mobile Labs setup with different Devices - IOT, iOS, Android, iPad, Tablets
    • Proper CI - CD common configuration using Jenkins, Dev Ops, AWS, Docker and Cloud setup
    • Handling multiple Docker nodes using Kubernates (use of Selenoid, GRID on Cloud)
  • Liked Arjan Blok
    keyboard_arrow_down

    Arjan Blok - TypeScript: your gateway drug to the JavaScript world

    Arjan Blok
    Arjan Blok
    Test Engineer
    Mendix
    schedule 1 month ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    A big part of the testing community is using Java based tooling to do automation. Selenium Java bindings and RestAssured are goto testing libraries for a lot of us. Yet most of the applications we are testing nowadays are written in JavaScript; the language of the web. I speak from experience when I say that writing your automation code in the same language as your application under test is written in will bring you a lot of benefits. It creates a common ground for knowledge sharing, problem solving and ultimately: developers willing to write and debug integration tests.

    However, as a tester, programming might not be second nature to you and learning a new untyped language like JavaScript, with all its quirks, can look like a daunting task. Enter TypeScript: a typed superset of JavaScript which will instantly make you feel at home!

  • Liked Wim Selles
    keyboard_arrow_down

    Wim Selles - Do you know the dependency of your dependencies dependency?

    Wim Selles
    Wim Selles
    Solution Architect
    Sauce Labs
    schedule 1 month ago
    Sold Out!
    45 Mins
    Case Study
    Beginner

    Nowadays a lot of people are using JavaScript based automation frameworks, and why not? They are the closest language to what your developers are using and it's easy to use. You just

    • Install NodeJS on your local machine
    • npm install this-module
    • npm install that-module,
    • script a test-case
    • npm run test

    And you're done, easy peasy, isn’t it? But when you look at your project you see something called

    • node_modules
    • package.json
    • package-lock.json

    And you hear your developers talk about

    • ES6, ES7
    • TypeScript
    • Polyfills
    • NodeJS LTS, 8, 9, 10, 11, 12, 13 (and counting)
    • Promises, asynchronous behavior
    • ...

    And you are like……………...

    If you recognize yourself in this then this presentation might be useful for you, because during this presentation I will cover NodeJS, the biggest dependency of your dependencies dependency.

  • Liked Varadharajan Srinivasan
    keyboard_arrow_down

    Varadharajan Srinivasan - How do we test billion data set in 1hour

    45 Mins
    Talk
    Intermediate

    We are now living in the age of Big Data. What makes Big Data so special is its ability to help the common man process petabytes of data to build innovative solutions. Cloud has played a pivotal role in building this data economy by making infrastructure more cost effective and enabling easy scalability.

    Today, the abundance of data helps business make intelligent decisions through business process automation. Data drives all our digital interactions, starting from products that we see on our Amazon home page, to our Netflix recommendations, to the advertisement on Google search results, to even the Friends that we get connected to on Facebook. All these systems are powered by data that is collected from the various system, cleaned, transformed and stored into Big Data solutions. Testing of these data systems have become very complex and requires a new approach. This paper highlights various aspects of Big Data Testing including:

    • Complexity in testing Big Data systems

    • Cloud based tools for testing Big Data cross various stages

    • Simplification of test processes and visualization through dashboards

    • Testing Big Data in the Agile/ DevOps

  • Liked Zachary Attas
    keyboard_arrow_down

    Zachary Attas - Selenium For All - Setting Your Team Up For Success So Anyone Can Understand and Write Tests

    Zachary Attas
    Zachary Attas
    Sr. Test Engineer
    ReviewTrackers
    schedule 1 month ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    Are you the only one contributing to your test suite, because others feel too intimidated to contribute to it?

    Have you ever looked at a test fail and not known where to investigate first?

    This talk will cover techniques for uncomplicating your test suites, so they can clearly test exactly what's needed to be tested, allowing others to contribute tests. When the process of writing tests is shared, everyone has an impact on quality, and that can be quite contagious for a company's culture.

    In addition, this talk will cover techniques for failing transparently, so you will no longer feel stressed triaging test fails. Quality is something shared by the entire team, so anyone should be able to triage test fails. When tests fail for a specific, clear reason, you'll be thanking your past self for writing logic to fail transparently, saving you half a day going down a rabbit hole.

  • Liked Jesus Sanchez Martinez
    keyboard_arrow_down

    Jesus Sanchez Martinez - Test and monitor one website is not that hard, but what if you need to do it to over 40 websites?

    Jesus Sanchez Martinez
    Jesus Sanchez Martinez
    QA
    Onestic
    schedule 1 month ago
    Sold Out!
    45 Mins
    Talk
    Advanced

    Onestic QA department made and maintained test suites, which was a huge bottleneck in our development process.

    In order to solve it, management bought us an idea: our developers must be able to build their tests using a DSL framework without friction. QA maintain a big library that provides resources to developers. From there, they were free to extend this library in each project. With all of this we have a CI process, a 1 hour execution to monitor our results and of course our bot, SpongeBot.

    SpongeBot can check 40+ e-commerce sites, with 4+ environments, for desktop and mobile platforms always available for developers.

    With this solution we achieve to decentralize the work, add value testing production and the confidence in SpongeBot notifications if something goes wrong.

  • Liked Gaurav Singh
    keyboard_arrow_down

    Gaurav Singh - How to build an automation framework with selenium : patterns and practices

    Gaurav Singh
    Gaurav Singh
    Product Engineer
    Go-Jek
    schedule 3 months ago
    Sold Out!
    45 Mins
    Talk
    Beginner

    With an ever increasing no of businesses being conducted on web the testing need to write automated tests for the app's UI is something that can never be ignored. As you all know Selenium provides an API that enables us to do this quite effectively.

    However, when tasked with setting up the automation framework, there are a lot of questions that arise in the minds of aspiring test developers regardless of what level they are in their career.


    Some of such questions are:

    1. How does one actually go about the business of building a robust and effective automation framework on top of selenium?
    2. What are the elementary building blocks to include in the framework that an aspiring automation developer should know of?
    3. How should we model our tests? XUnit style vs BDD?
    4. Are there good practices, sensible design patterns and abstractions that we can follow in our code?
    5. What are some of the anti patterns/common mistakes we should avoid

    A lot of literature, documentation and blogs exists on these topics on the web already.

    However In this talk,

    I would combine this existing knowledge and my years of experience in building automation frameworks and breakdown these elements and walk you through exactly the sort of decisions/considerations and practices that you can take while starting to implement or improve the UI automation for your team.

    Hope to see you there!

  • Liked Moshe Milman
    keyboard_arrow_down

    Moshe Milman - Fixing Your Automation Challenges in the Era of CI/CD

    Moshe Milman
    Moshe Milman
    COO & Co-founder
    Applitools
    schedule 2 months ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    Continuous Delivery is now the holy grail of IT organizations, but most companies are still struggling with the transition into shorter release cycles and faster, more frequent deployments. A key challenge that companies are facing in that transition lies around test automation, and in this talk we will cover these challenges in details and demonstrate how successful companies are addressing it.
    The key takeaways from this session include:
    1) Testability hacks + best practices
    2) Tips for reducing your testing matrix to support the automation speed required for frequent builds/releases
    3) Examples of team structure/architecture from leading companies to support better alignment around automation
    4) Guidelines for picking the right open source tools / frameworks
    5) Learnings/Issues to avoid when designing your automation architecture
  • Liked Khanh Do
    keyboard_arrow_down

    Khanh Do - Leveraging Artificial Intelligence to create self-healing tests

    Khanh Do
    Khanh Do
    QA Architect
    Kobiton
    schedule 3 months ago
    Sold Out!
    45 Mins
    Tutorial
    Intermediate

    A key requirement for successful test automation is to get past the brittle or fragile nature of test scripts. Any Selenium (or Appium) developer has encountered the dreaded "NoSuchElement Exception". A locator that worked yesterday may fail today. What's a test engineer to do?

    Fortunately the field of AI provides promising solutions and allows for the creation of self-healing tests. Tests that can find elements across all environments. Tests that can learn from "human-in-the-loop" intervention and work perfectly thereafter. Imagine automated tests that "just work"!

    This session will look at how to apply the latest in AI and Machine Learning technologies to improve your test scripts. With the plethora of new open source AI libraries made available by companies such as Google, the ability to leverage AI in your applications is more accessible than ever.

    This session will be a primer on AI technologies and how they can be utilized for perfect test automation.

  • Liked adam goucher
    keyboard_arrow_down

    adam goucher - Running Selenium Grid in AWS

    adam goucher
    adam goucher
    Everything
    Element 34
    schedule 2 months ago
    Sold Out!
    480 Mins
    Workshop
    Intermediate

    The workshop is the full day version of my Selenium Conf London talk where you will securely and efficiently build a modern Selenium Grid in AWS. Topics will include;

    - Infrastructure as Code using Terraform

    - Designing your Selenium Grid for the AWS Cloud

    - Security Fundamentals

    - Maximum result for minimum costs

    - And of course, run a script to prove it all works

    This is a hands-on workshop where each participant will create their own grid in the AWS cloud using starter scripts provided. As a result, all participants will need to have created a new AWS account prior to this workshop which they have full control over and a laptop.

    Note: Participants will accrue a small amount of AWS charges for the infrastructure they build during the day, but it will be minor.