How to kill test flake in Appium?

schedule Jun 14th 01:45 - 02:30 PM place Jupiter people 65 Interested

Whether you are a beginner learning the ways of writing UI automated tests or an experienced practitioner, you must have faced the below situation:

You write a test and it works perfectly in your local, happily you promote the tests to your CI environment and surely it passes initially, however after sometime the test starts failing sporadically. This phenomenon is described as Test flakiness and it’s a bummer for any engineer worth his salt. This can cause numerable side effects and doubts into the minds of software teams not trusting UI automation.

However all hope is not lost. We all know a Tool is only as good as its master.

In this talk, I will cover the measures or patterns which you can implement to reduce the flakiness in your Appium tests as much as "humanly" possible and increase their speed and reliability.

 
 

Outline/Structure of the Talk

  1. Introduction of test flakiness as a first class test automation problem [~2 mins]
  2. Common patterns for why tests are flaky and ideas to avoid them [~30 mins]
    1. Unsuitable Locator or selector strategy
      1. Avoiding XPATH and look into what are the alternatives? E.g. Accessibility ids, IOS class chain, UI Selector
    2. Unreliable wait strategies
      1. Avoiding Hard coded sleeps and replacing with standard explicit waits (Expected conditions)
    3. Setting up App states (making tests faster)
      1. Will discuss use of android activities to arrive at exact screen or try out kitchen sinks as hooks to different app state
      2. Use of Deep links.
      3. Disable animations
    4. Making tests reliable
      1. Avoiding unstable network API calls and instead use mocking as a technique for predictable test data
      2. Use API to setup app data
  3. Debugging (~7 mins)
    1. Reading exception stack traces and error messages to separate errors in Test code vs actual Appium problems
    2. Analyzing Appium server logs or device logs
  4. Best practices (~ 2 mins)
  5. Q&A (~ 3 mins)

Learning Outcome

By the end of this talk,

You as a test automation engineer would be able to have a broad understand of what are the different factors which cause tests to be flaky and identify approaches which you can try to improve your tests reliability

Key takeaways:

  • Understand waits and locators pitfalls
  • Understand cross platform or specific implementations for android/IOS to improve speed and reliability
  • Basics of debugging and log analysis to arrive at root cause of problems

Target Audience

Automation Testers or developers who want to use Appium effectively for their test automation efforts

Prerequisites for Attendees

  • Should have basic working experience in UI test automation using either Appium/selenium.
  • Code examples would be in Java
schedule Submitted 11 months ago

Public Feedback

comment Suggest improvements to the Speaker