The Test Automation Pyramid is not a new concept.

The top of the pyramid is our UI / end-2-end functional tests - which simulate end-user behavior and interactions with the product-under test.

While Automation helps validate functionality of your product, aspects of UX validations can only be seen and captured by the human eye and is hence mostly a manual activity. This is an area where AI & ML can truly help.

With everyone wanting to be Agile, make quick releases, the look & feel / UX validation, which is a slow, and error-prone activity, quickly becomes a huge bottleneck. In addition, with any UX related issues propping up cause huge brand-value and revenue loss, may lead to social-trolling and worse - dilute your user-base.

In this session, we will explore why Automated Visual Validation is now essential in your Automation Strategy and also look at some popular tools that can help solve this problem.

 
 

Outline/Structure of the Demonstration

  • Recap of Test Automation Pyramid
  • What is Visual / UX Testing?
  • Challenges of Visual / UX Testing
  • Solutions / Options for Automated Visual Testing with pros & cons

Learning Outcome

  • Importance of Visual Testing
  • Challenges of Automated Visual Testing
  • Techniques to include Visual Testing part of your Testing Strategy
  • Different options with examples and demos, for automating Visual Testing

Target Audience

QA, Dev, Product Owners, Product Managers, Managers, Automation, SDET, DevOps, Manual testers

Prerequisites for Attendees

None

schedule Submitted 3 months ago

Public Feedback

comment Suggest improvements to the Speaker
  • Maaret Pyhajarvi
    By Maaret Pyhajarvi  ~  1 month ago
    reply Reply

    I am unsure what the focus of this talk is. The title talks about visual validation whereas the description and outline seem to take the talk more towards the whole test automation pyramid and understanding what kinds of tests can find what kinds of problems. Is your intent here addressing more on how the introduction of new kinds of tools and techniques moves the whole pyramid (like adding visual verification tools has moved people's focus in what they test in 'acceptance tests' and how adding contract tests has done the same but for a different layer of the pyramid) or what would you want to see the audience take out of it and do differently? Use visual testing in the first place? Avoid using it when it is not the most important aspect?

    • Anand Bagmar
      By Anand Bagmar  ~  2 weeks ago
      reply Reply

      Good way to look at this Maaret. I will restructure this proposal to focus on why visual testing is essential in specific contexts, where it does not make sense, and different ways to implement it. Thanks for your feedback on this.

  • Pallavi Sharma
    By Pallavi Sharma  ~  1 month ago
    reply Reply

    Hello Anand

    Can you help me understand, would you be covering the UI validation for only mobile? Or web as well? 

    Also will you be showing any integration of tool example Applitools with Selenium during the demo?

    Thanks 

    • Anand Bagmar
      By Anand Bagmar  ~  1 month ago
      reply Reply

      Hi @Pallavi,

      I will be discussing visual validation as a concept, and why it is a critical aspect in the CI-CD way of working. Once the concept is understood, I will be discussing techniques to bring the concept into reality and different tools that can help in the aspect. One of the tools I will be referring to is Applitools from a visual validation perspective.

      This is NOT an Applitools demo and hence, I will not be showing its integration with Selenium, Appium, etc. However focus on how to extend the Automation strategy to include Visual Test Automation, and the value this brings to the team.

      • Pallavi Sharma
        By Pallavi Sharma  ~  1 month ago
        reply Reply

        Hi Anand, thanks for writing back and explaning the talk intentions. 


  • 20 Mins
    Talk
    Beginner

    In this talk, I will speak about the process and strategies on how can one analyze and select the right tools for test automation. We will look at some tools and processes that help to select the tools, frameworks, libraries, etc.

    I will demonstrate some helpful extensions and tools that will be very handy and useful during the process

    This process can be carried out before every test automation project or engagement. The selection of the right tools will help throughout the process of automation and will help to avoid delays, changes, re-work, incomplete automation, etc. and will save a lot of time, effort and resources for the organization.

    The session is designed for anyone in the Software Testing industry and the knowledge received from this session can be implemented right away in any of the upcoming or existing test automation projects.

  • Liked Naveen Khunteta
    keyboard_arrow_down

    Naveen Khunteta - Best Practices to implement the test automation framework starting from Design - To -> Infrastructure - To -> Execution.

    45 Mins
    Talk
    Intermediate

    Best Practices - How to get the best 'Return ON Investment' (ROI) from your Test Automation.

    This has been observed that, most of the test frameworks wont be able to survive due to lack of expertise, no maintenance, no best practices being followed, and finally your test automation will be dead after few months, and there is no "Return ON Investment" from this. This is the most common problem, most of the companies are struggling and finally back to square to the Manual testing.

    My proposal : HOW to leverage your test automation in terms of best practices, best ROI, and how to adopt best automation culture in your organisation.
    I strongly propose some of the important points/suggestions to achieve this in your Organisation/Team.
    1. Test Automation Practices:
    • Design Patterns (Web/Mobile/API)
    • What to Automate/Not to Automate
    2. Common Automation Frameworks at Org Level:
    • How to design Generic Utilities, Libraries and different Components, which can be suitable for all the teams in the same Org.
    • Best practices to design your Tests (Automation).
      • Common Design Patterns
      • Common application level and Page libraries
      • Best Practices to use Assertions in your Tests (How and What to write for assertions). Most of the people don't write proper assertions and this is making your test unreliable and no defects found during execution.
    3. Inclusion of API/Backend libraries in your UI test automation as an external Maven/Gradle Dependencies to avoid un-necessary tasks, some of the important points to be considered here:
    • User Creation from APIs (No need to automate user creation from web/app for all the test cases)
    • API tests are stabled most of the time
    • API calls takes lesser time as compared to web, hence include API calls in your UI/App framework to save time.
    • Less flaky test

    4. Best Code Review Process (Do not merge your code into Master without proper Code Review)

    • Implement PR (PULL Request) Process
    • Static Code Analysis using SonarQube, Cobertura, JACOCO etc..
    • Get the benefits of Best Test Automation Quality Matrices
    • Sometimes, Manual (Functional Tester) should review your code (Assertion, test steps and use cases) to get the best coverage
    5. Quality is A Team responsibility:
    • Developers, POs, Manual QEs and Automation engineers should be included to get an overview of test automation coverage.
    6. Maintenance of the Frameworks
    • After couple of months, it should not make your life miserable if you don't maintain your libraries and framework properly.
    • Do not use Hard Coded values, make it simple and Generic.
    7. Infrastructure Setup for Test Design and Test Execution:
    • Proper Browser - OS lab setup
    • Proper Mobile Labs setup with different Devices - IOT, iOS, Android, iPad, Tablets
    • Proper CI - CD common configuration using Jenkins, Dev Ops, AWS, Docker and Cloud setup
    • Handling multiple Docker nodes using Kubernates (use of Selenoid, GRID on Cloud)