In this era of Digital transformation, clients have been demanding shorter and quicker releases. Shorter and quicker releases mean your team should not only be able to develop them at the required pace but also test and release them at a sustainable pace. User interface plays an important role in the client's business and there are organizations that release new features, fancy CSS regularly that support multiple browsers, multiple Operating systems, mobile devices. Verifying the frontend on this browser/device/OS matrix by humans is not only extremely time consuming but also prone to human errors. In fact, testing by humans should primarily focus on discovery leaving the repetitive and error-prone tasks to tools. Hence, automating Visual tests is becoming less of an optional activity and more of a must-have activity within the team. Ensuring visually perfect user experiences is equally important as to have the functionality working.

 
 

Outline/Structure of the Talk

I would be talking about the following topics:

  • What is Automated Visual testing?

  • Is there a need to automate visual tests? Why?

  • How does the basic algorithm work for any tool supporting automated visual tests? Demonstration on:

    • Create a test

    • Execute a test

    • Interpret the results of a test

  • Challenges with automating Visual tests

    • Anti-aliasing

    • Dynamic Content

    • False positives

  • Can the above challenges be addressed?

    • Are there any open source/commercial tools available to address these challenges?

    • Demonstration - How do they address them?

  • What should be the strategy to design and execute the Automated Visual Test Suite?

    • Capturing screenshots for - Full page, element, viewport

    • Is it necessary to execute each and every test on your browser/device matrix?

    • Demonstration - Can we organize our Test Suites to optimize the execution?

  • Challenges faced and potential solutions for introducing Automated Visual testing in your CI/CD pipeline

  • A quick comparison of tools available in the market - Use what when?

  • How to get started with this type of Automated testing?

    • No coding knowledge/experience - Don't worry, there are config-based, record and playback tools available

    • Good coding knowledge/experience - Wonderful, leverage your skills to address the challenges

Learning Outcome

At the end of the session, attendees will learn about:

  • What is Automated Visual regression testing?

  • How does it work?

  • What are the advantages of using it?

  • What are the challenges associated with it and potential solutions?

  • What are the good practices associated?

  • What are the tools available in the market?

  • When to use what?

  • How can you get started with it?

  • Introducing Visual automation in your CI cycle

Target Audience

QA / Test Automation engineers who want to reduce the time spent on testing on various devices and browsers, Front end developers who want to ensure that new changes don't introduce regression before they push their code and Project Managers who want to introduce this technique in their organization

Prerequisites for Attendees

There are no prerequisites for this session.

schedule Submitted 7 months ago

Public Feedback

comment Suggest improvements to the Speaker
  • Anand Bagmar
    By Anand Bagmar  ~  5 months ago
    reply Reply

    Hi Shweta,

    I have a question regarding your proposed structure:

    You mentioned:

    • Demo using Visual regression service by WebdriverIO

      • Parallel execution on multiple browsers

      • Support for tests using Mocha, Jasmine and Cucumber framework

      • Organizing Test Suites

      • Report generation

      • Run tests on Cloud

     

    How are the aspects of parallel execution, mocha, jasmine, cucumber, organizing test suites, reports and cloud related to your topic title - "Advanced automated visual regression testing"? Are we mixing 2 different things here?

    Regards,

    Anand

    • Shweta Sharma
      By Shweta Sharma  ~  5 months ago
      reply Reply

      Hi Anand,

      Thank you for looking into the proposal and pouring in your thoughts.

      Although executing these types of tests provide good confidence when executed on a single browser, however, leveraging the existence of these tests to run on several devices and browsers will further justify the efforts and provide maximum ROI.

      Organizing Test Suite & Parallel execution - Time is a major factor to be considered while executing Automated scripts. While planning to execute them on several devices/browsers, there are some good practices that we should follow like Organizing Test Suite based on your history of issues encountered on specific browsers/devices, understanding the most important business needs and so on in order to decide what tests executed on selected browsers and which ones get executed across the entire matrix.

      Running tests on the cloud - Another pain point that is encountered is screenshots captured from a machine and then being compared on another machine might run into false issues depending on the machine configuration different people have. Running these tests on the cloud every time, which is not only for capturing the baselines shots but also comparing them avoid these false issues. This solution is not only applicable to execute tests on a particular environment but also comes handy when the developer wants to test his/her changes locally before committing.

      Support for testing using a set of different testing frameworks- This was to cover a wider range of audience.

      Anand, I'm glad through your question, I could elaborate my thoughts/ideas in detail which was actually missing in the proposal.

      Please do let me know if you have more questions.

      Thanks,

      Shweta.

      • Anand Bagmar
        By Anand Bagmar  ~  5 months ago
        reply Reply

        Hi Shweta,

        Based on your clarifications and additional details, there are a few things I suggest:

        1. Focus on advanced visual testing. The current structure sort of implies that you are talking about webdriver.io functionality, instead of visual testing. Once visual testing is integrated in your framework, does it matter how it is going to work when tests run in parallel, or in the cloud, etc.?
        2. What do you mean by "Browser Extensions for Applitools?"
        3. What are the challenges of automating visual tests?
        4. How does webdriver.io / Applitools / any other tool you refer to in your proposal overcome the challenges of visual testing?

        In summary, focusing on #3 and #4 above would make this a very strong proposal.

        Regards,

        Anand

        • Shweta Sharma
          By Shweta Sharma  ~  5 months ago
          reply Reply

          Hello Anand,

          Based on our discussion, I have modified the Outline/Structure of the Talk section of this proposal. Please let me know if it looks OK to you.

          Thank you once again for pointing out great gaps in the proposal. It is a learning that I am going to take forward across all my proposals now.

          Regards,
          Shweta.

          • Anand Bagmar
            By Anand Bagmar  ~  5 months ago
            reply Reply

            Thanks for the update Shweta!

        • Shweta Sharma
          By Shweta Sharma  ~  5 months ago
          reply Reply

          Hi Anand,

          1. I completely agree that the current outline doesn't imply to talk about Visual testing challenges and how the tools that I decide to talk about actually help overcome these challenges. However, I had mentioned the point of the challenges in the Learning Outcome and also I would demonstrate how these challenges are resolved by a few tools and good practices.
          2. I meant the Chrome and Firefox browser plugins that are available by Applitools.
          3. Points 3 & 4 in your previous comment is what the talk would focus on, not to forget on the Good Practices too. How do you suggest we proceed? Do you want me to add those points as a comment or is there a way that I can edit the proposal to better reflect on what you and I are talking about (which is the same thing)?

          Thanks,

          Shweta.


  • Liked Shweta Sharma
    keyboard_arrow_down

    Shweta Sharma / Nikita Jain - Accessibility testing 101

    45 Mins
    Talk
    Beginner

    "This world is such a beautiful place to live in." If you can read the first sentence without any screen readers or assistance, you're privileged. As technologists, shouldn’t we be more empathetic towards differently-abled people and make all parts of our website accessible to them? In my humble opinion, the true power of technology can be identified when it reaches out to people of all kinds having different physical or psychological challenges. We not only legally bound to provide Accessibility but also it should be considered as our moral responsibility.

    As testers, we have a wonderful opportunity to contribute to Accessibility by ensuring that the site is accessible in many different ways. Although it is impossible to identify all the issues that exist around Accessibility in the world, we are lucky enough to still understand a majority of them. With this understanding, there have been many measures taken in order to make your site accessible. But, don’t forget - we are QA engineers. We got to ensure that the site is accessible as per the standards set by WCAG 2.0 (AA) by testing for accessibility using various tools and techniques.