Trust Issues with UI Automation

schedule Jun 15th 04:00 - 04:45 PM place Jupiter people 59 Interested

UI Test Automation is notorious for stability issues. UI automation is almost synonymous with "Flaky Test". We discuss taking a proactive strategy towards test automation flakiness as opposed to a reactive approach where we look at failures and then decide the course of action. How can we in the test automation community use and develop tools that helps eliminate flakiness and also identifies flaky tests before they are not just run but even before they are merged. How do we create these tools in a service model that plugs into our Continuous Integration pipelines ?

 
 

Outline/Structure of the Talk

Introduce the problem.

Work around that are commonly used in the industry

Why the above does not work

Introduce the services and approach we have designed

Go into details of the "Test Stability Validation Service"
Overview of benefits after using this service

Data to support the claims

Q&A

Learning Outcome

The outcome of this is to understand where we are going wrong as an industry to address a core problem with our output. What we can do to change it and see examples of how it has already been done at Sony. Understand the impact it has had on our test stability and release cycles and brain storm ideas on how this can be imported to other Companies.

Target Audience

Test Automation Engineers working on UI Automation using frameworks like Appium

Prerequisites for Attendees

Has basic understanding of UI automation. Understands concepts such as code linting, service module, CI/CD, Stable identifiers, multi-platform support.

schedule Submitted 11 months ago

Public Feedback

comment Suggest improvements to the Speaker
  • Naresh Jain
    By Naresh Jain  ~  9 months ago
    reply Reply

    Hi Bijoya,

    The team is interested in inviting you to the conference. However, the current proposal is very light on its connection to Appium. Also, the slides are pretty much focused on CI.

    Would it be possible for you to focus more on the Appium/mobile testing aspect? Also, would you be able to share some details on how you test Sony PlayStation? Do they use Appium?

    Request you to please update the proposal accordingly.

    Thanks.

    • Anand Bagmar
      By Anand Bagmar  ~  10 months ago
      reply Reply

      Hi Bijoya,

      Can you please share some reference slides and also a short video of your speaking experience?

      Also, how reusable are the services and approach you have designed for the attendees? Is it concepts / ideas you are sharing here?

      Similar questions for the "Test Stability Validation Service.

      Thanks

      • Anand Bagmar
        By Anand Bagmar  ~  10 months ago
        reply Reply

        Hi,

        The sooner you can send reference slides, the better opportunities for you to work with the program team on specific, if any on the content. These slides do not need to be the final version at this time.

        For video - a simple video with you explaining / summarizing this talk will help. You can just share that link with the program team

         


      • Justin Ison
        Justin Ison
        Sr. Software Engineer
        Applitools
        schedule 11 months ago
        Sold Out!
        45 Mins
        Demonstration
        Beginner

        In today’s agile world the time to market is becoming increasingly shorter. There is a constant desire to release ASAP to keep ahead of the competition and to please users with updated/new features. Because of this, we have less time to fully do manual and exploratory testing of our apps. Especially, when you consider all the combinations of OS's, Locales, Accessibility, Orientations & Resolutions apps support. Running anywhere from 1 to 100's of Appium crawler bots (covering all of those combinations) at once we can discover more issues quickly and efficiently without having to write a line of code.

        UI Automation also has its limitations as it only tests for expected results. Crawler bots test the unexpected, by collecting metadata such as logs, app strings, screenshots, memory and reporting back it’s finding for review so we can test all these combinations quickly and more efficiently. In this talk, I will go over the current challenges we face in today's development world, why we need more tools to help us keep pace, and cover how you can build your own Appium crawler.

        I've open sourced this tool and is available here for everyone to use: https://github.com/isonic1/Appium-Native-Crawler

      • Liked Jonah Stiennon
        keyboard_arrow_down

        Jonah Stiennon - Find Elements By Accessibility Id

        Jonah Stiennon
        Jonah Stiennon
        Partner
        Cloud Grey
        schedule 10 months ago
        Sold Out!
        20 Mins
        Talk
        Intermediate

        You'll find many written sources which assert that AccessibilityId is the preferred selector to use for finding elements, but why is this the case? What's wrong with XPath?

        In this talk, we will discuss Accessibility Id's in depth. How Appium uses them to find elements on iOS, Android, and web, and how developers add these Id's to apps. We will specifically focus on the performance impact of using XPath selectors instead of Accessibility Id's.

        We will demonstrate techniques we can use to improve the performance of XPath selectors if accessibility Id's are missing, and discuss strategies for convincing managers and app developers to add accessibility Id's to the apps we automate.

      • Liked Jonah Stiennon
        keyboard_arrow_down

        Jonah Stiennon - All Desired Capabilities

        Jonah Stiennon
        Jonah Stiennon
        Partner
        Cloud Grey
        schedule 10 months ago
        Sold Out!
        90 Mins
        Tutorial
        Intermediate

        Appium has so many desired capabilities! How many exactly? Let's count!

        Join Appium contributor Jonah Stiennon as he iterates through every desired capability supported by Appium.

        Many special behaviors and specific workarounds are made available only through desired capabilities. Often the key to selecting an element reliably, launching an app, or avoiding a timeout, is picking the right set of desired capabilities. There's so many that testers don't get the chance to sit down and become acquainted with them all.

        There's too many to easily memorize and documentation can be sparse. Jonah will introduce novel ways to classify, visualize, and organize all of the desired capabilities, making it easier to find ones which can be useful.

      • Liked Bruno Alassia
        keyboard_arrow_down

        Bruno Alassia - Go beyond the software, automate hardware scenarios on Android Emulators

        Bruno Alassia
        Bruno Alassia
        Sr. Software Engineer
        SauceLabs
        schedule 11 months ago
        Sold Out!
        45 Mins
        Talk
        Intermediate

        Explore the limits Appium can help you achieve to control the Android Emulator behavior. Automate network emulation, receive fake SMS and phone calls, change the battery states and levels and a lot more, all using real-life test cases. What happens to my app if I lost connection? How does my UI react to an incoming phone call? And more.

        https://slides.com/vrunoa/appium-conf-2019#/

        https://github.com/vrunoa/appium-conf-2019

      • Liked Rajdeep
        keyboard_arrow_down

        Rajdeep - There is more in Espresso Driver's Element than you think/ WhiteBox testing using Appium

        Rajdeep
        Rajdeep
        Sr. Test Automation Engineer
        Badoo
        schedule 10 months ago
        Sold Out!
        45 Mins
        Demonstration
        Intermediate

        Have you been in a situation when some cases are impossible to test because Appium doesn't have support? Appium is improving continuously and release of espresso-driver is an example of this.
        Espresso driver opens up possibilities of white-box testing using Appium. One of which is the ability to call platform methods on elements via backdoor without modifying Application under test. What it means is, if the Android platform supports some actions or properties on an element, then Appium will support it out of the box.
        For an automation engineer, that's a very powerful addition. I was fortunate enough to work on this feature and would like to showcase what are the various tricky cases where this feature can be applied. There will be real examples shown with a sample android app. I will also introduce one more small bonus feature at the end, about flashing elements on screen ;)

      • Liked Tomer Cohen
        keyboard_arrow_down

        Tomer Cohen - Low-level android automation with adb

        45 Mins
        Talk
        Beginner

        When we speak about Android automation, the first thing that comes to mind is Appium, however, what if Appium doesn't serve all your Android automation needs, at least not in the most efficient way?

        When I was asked to create a test automation framework for our android service (no GUI), I needed to choose the right tool for the mission. After several weeks of researching many options (both commercial and open source). I realized that ADB is my best call because of the following reasons:
        • ADB provides a faster to run solution for controlling an android device
        • ADB is in a lower level of Android OS therefore it's more flexible
        • ADB is going through less code components (like appium client, appium server etc..) and as a result is more reliable

        In this tutorial I will share my journey towards deciding to use Android Debug Bridge when it comes to non-gui android application and how I have implemented my choice.
        After this tutorial, you will have a better understanding of what is ADB, how it works, and how you can start and apply automation for the Low-Level layer of Android OS with ADB and Python.

      • Liked Eran Kinsbruner
        keyboard_arrow_down

        Eran Kinsbruner / Uzi Eilon - Future-Proofing Testers in the Age of AI, ML, and Bots

        45 Mins
        Talk
        Advanced

        We're all hearing the buzzwords of AI, machine learning, chatbots, and next-generation testing. Does this mean that the days of traditional testing as we know and practice it are over? Eran Kinsbruner doesn't think so. Join him to learn about the clear transformation happening toward smarter testing techniques and tools. These approaches will drive better pipeline efficiency and release velocity with high quality, and Eran thinks this means good things for the testing practice and practitioners. You'll discover the key trends that are happening around AI, machine learning, and bots in the web and mobile landscapes, and get the ability to identify some early adopters who are taking the lead in these domains. Learn some key requirements and skills that are required for a test engineer to shift toward the future of automation, as well as the changes testers will need to undergo in order to become future leaders in this space.