The Test Automation Pyramid is not a new concept.

The top of the pyramid is our UI / end-2-end functional tests - which simulate end-user behavior and interactions with the product-under test.

While Automation helps validate functionality of your product, aspects of UX validations can only be seen and captured by the human eye and is hence mostly a manual activity. This is an area where AI & ML can truly help.

With everyone wanting to be Agile, make quick releases, the look & feel / UX validation, which is a slow, and error-prone activity, quickly becomes a huge bottleneck. In addition, with any UX related issues propping up cause huge brand-value and revenue loss, may lead to social-trolling and worse - dilute your user-base.

In this session, we will explore why Automated Visual Validation is now essential in your Automation Strategy and also look at some popular tools that can help solve this problem.

 
 

Outline/Structure of the Demonstration

  • Recap of Test Automation Pyramid
  • What is Visual / UX Testing?
  • Challenges of Visual / UX Testing
  • Solutions / Options for Automated Visual Testing with pros & cons

Learning Outcome

  • Importance of Visual Testing
  • Challenges of Automated Visual Testing
  • Techniques to include Visual Testing part of your Testing Strategy
  • Different options with examples and demos, for automating Visual Testing

Target Audience

QA, Dev, Product Owners, Product Managers, Managers, Automation, SDET, DevOps, Manual testers

Prerequisites for Attendees

None

schedule Submitted 1 year ago

Public Feedback


    • Anand Bagmar
      keyboard_arrow_down

      Anand Bagmar / Diego Molina / Jim Evans / Manoj Kumar / Simon Stewart / Sri Harsha - Q&A with the Selenium Committee

    • 20 Mins
      Talk
      Beginner

      In this talk, I will speak about the process and strategies on how can one analyze and select the right tools for test automation. We will look at some tools and processes that help to select the tools, frameworks, libraries, etc.

      I will demonstrate some helpful extensions and tools that will be very handy and useful during the process

      This process can be carried out before every test automation project or engagement. The selection of the right tools will help throughout the process of automation and will help to avoid delays, changes, re-work, incomplete automation, etc. and will save a lot of time, effort and resources for the organization.

      The session is designed for anyone in the Software Testing industry and the knowledge received from this session can be implemented right away in any of the upcoming or existing test automation projects.

    • 45 Mins
      Case Study
      Intermediate

      Challenges in Testing & Automating Games

      Testing real-time, multi-user games built for native apps and / or browser-based on phones / mobile devices / tablets / desktop browsers makes testing of regular products as apps, or websites appear like a piece of cake. Testing such real-time and multi-user games becomes even more challenging when you think about automating the same.

      I got an opportunity to build Functional Test Automation for a suite of games – and what an exciting time it turned out to be!

      These games are built either using Cocos2d-x, or Unity (cross platform game engines for mobile games, apps and other cross platform interactive GUI and are known for their speed, stability, and ease-of-use).

      The key challenges I encountered here were:

      • Millions of users, playing games on a huge variety of devices (Android & iOS native apps, Mobile-Web, and Desktop Web)
      • Limited unit testing
      • API testing & Functional Testing done in isolation (mini-waterfall approach)
      • Usage of Cocos2d-x & Unity for game rendering – which cannot be automated via Appium
      • Limited Functional Automation (for native apps)

      An approach to Functional Automation of Real-Time, Multi-User Game scenarios

      I overcame the above mentioned challenges by doing the following:

      • A better way of working (you can call it ‘Agile’)
      • Break down the walls by fostering a mindset of “build quality-in, as opposed to test for quality”
      • Built a new functional automation framework using java / testing / appium-test-distribution / reportportal / jenkins with focus on – specify test intent once and run on all supported channels (ex: Android, iOS, Mobile-Web, and Web)
      • Built a solution for Cocos2d-x layer automation
      • Created a vision of CI-CD for the organization, and setup code-based CI pipelines to enable end-2-end visibility
      • Made the framework extensible by providing ability to use same framework for multiple games

      The focus of this talk will be to:

      • Share an example of a particular use case
      • Share the solutions, including code snippets, implemented for:
        • Functional Test Automation Framework Architecture & Design
        • Ease of Test Implementation, while maintaining code quality and promoting reuse
        • Test Execution on local Vs CI, on-demand as well as on every new build

    • Naveen Khunteta
      keyboard_arrow_down

      Naveen Khunteta - Best Practices to implement the test automation framework starting from Design - To -> Infrastructure - To -> Execution.

      45 Mins
      Talk
      Intermediate

      Best Practices - How to get the best 'Return ON Investment' (ROI) from your Test Automation.

      This has been observed that, most of the test frameworks wont be able to survive due to lack of expertise, no maintenance, no best practices being followed, and finally your test automation will be dead after few months, and there is no "Return ON Investment" from this. This is the most common problem, most of the companies are struggling and finally back to square to the Manual testing.

      My proposal : HOW to leverage your test automation in terms of best practices, best ROI, and how to adopt best automation culture in your organisation.
      I strongly propose some of the important points/suggestions to achieve this in your Organisation/Team.
      1. Test Automation Practices:
      • Design Patterns (Web/Mobile/API)
      • What to Automate/Not to Automate
      2. Common Automation Frameworks at Org Level:
      • How to design Generic Utilities, Libraries and different Components, which can be suitable for all the teams in the same Org.
      • Best practices to design your Tests (Automation).
        • Common Design Patterns
        • Common application level and Page libraries
        • Best Practices to use Assertions in your Tests (How and What to write for assertions). Most of the people don't write proper assertions and this is making your test unreliable and no defects found during execution.
      3. Inclusion of API/Backend libraries in your UI test automation as an external Maven/Gradle Dependencies to avoid un-necessary tasks, some of the important points to be considered here:
      • User Creation from APIs (No need to automate user creation from web/app for all the test cases)
      • API tests are stabled most of the time
      • API calls takes lesser time as compared to web, hence include API calls in your UI/App framework to save time.
      • Less flaky test

      4. Best Code Review Process (Do not merge your code into Master without proper Code Review)

      • Implement PR (PULL Request) Process
      • Static Code Analysis using SonarQube, Cobertura, JACOCO etc..
      • Get the benefits of Best Test Automation Quality Matrices
      • Sometimes, Manual (Functional Tester) should review your code (Assertion, test steps and use cases) to get the best coverage
      5. Quality is A Team responsibility:
      • Developers, POs, Manual QEs and Automation engineers should be included to get an overview of test automation coverage.
      6. Maintenance of the Frameworks
      • After couple of months, it should not make your life miserable if you don't maintain your libraries and framework properly.
      • Do not use Hard Coded values, make it simple and Generic.
      7. Infrastructure Setup for Test Design and Test Execution:
      • Proper Browser - OS lab setup
      • Proper Mobile Labs setup with different Devices - IOT, iOS, Android, iPad, Tablets
      • Proper CI - CD common configuration using Jenkins, Dev Ops, AWS, Docker and Cloud setup
      • Handling multiple Docker nodes using Kubernates (use of Selenoid, GRID on Cloud)