Unlocking New Testing Capabilities with Espresso Driver
Android's Espresso framework was created as a "gray-box" testing framework, "gray-box" meaning that it's an automated UI testing framework ("black-box") but it has access to the internals of the application ("white-box"). Allowing access to the internals of the application-under-test opens up many new testing possibilities that weren't possible with UiAutomator2 (with some risks).
Here are the four new possibilities I would like to discuss (with coding examples)
- Access to internal app code
- Less flakiness thanks to "IdlingResource". No need for 'waits' and 'delays' for UI.
- Navigate WebViews using Espresso WebAtoms (this feature is in progress, will be ready well before June)
- Find elements off-screen using Espresso's DataMatchers (this feature is in progress, will be ready well before June)
Outline/Structure of the Talk
Explain what Espresso is, and what the Espresso driver does that can enhance the testing experience. Discuss 5 topics, give explanations of the topic, provide a live-coding use-case, and then give warnings
1. Access to the internal app code
- Because Espresso has access to the internals, we can execute arbitrary methods that are defined within the Android Application
- Briefly mention the `mobile: backdoor` method (won't go into too much detail, because Rajdeep is already talking about this)
- Use case: `view-tag` selector. React Native testers wanted the ability to select items by `view-tag` because they didn't want to pollute the accessibility tag with data that would confuse people using screen-readers
- Warnings: This violates the "black-box"-ness of Appium so use with caution
2. Less flakiness thanks to "IdlingResource"
- Espresso uses a thing called `IdlingResource` (will provide a brief explanation of what it is)
- What this means is that Espresso blocks on the UI thread until it's idle
- Use-case: Give a code example in UiAutomator2 that shows waits, delays, etc.... then show an example in Espresso that takes those out
- Warnings: There is a risk of the UI thread _never_ idling, and test being stuck. Appium works around this, mostly, by disabling animations
- Explain the Chromedriver problems we've had and the version mismatching, and the problem with finding/setting the context
- Sometimes all you need to do is a few basic operations, like submitting a username/password in an OAuth form
- Espresso has a library called the `webAtoms` library, which allows us to execute WebDriver atoms on WebViews, without the need for Chromedriver (or any native WebDriver)
- Appium has a `mobile: webAtoms` endpoint
- Use-case: A native app that uses OAuth and needs to enter username and password into a webview. Show WebAtoms example of entering username and password and how we don't need to use Chromedriver, or context-switching to pull this off
- Warnings: WebAtoms is not as powerful as ChromeDriver. It should only be used for convenience sake. If you're testing a WebView app (PhoneGap, Cordova), use ChromeDriver.
4. Find elements off-screen
- UiAutomator2 is constrained to selecting elements that are inside of the viewport
- Finding elements off-screen requires scrolling, checking, scrolling-again until the element is in viewport (show code example)
- Using Espresso's `onData` matcher, we can target elements off-screen
- Explain how it uses `Adapter`
- Use-case: Write a data matcher for an element that is an Adapter item and is off-screen
5. You can still use UiAutomator
- The one problem of all of this, is that you can only run Espresso tests on the AUT (app-under-test).
- Espresso can't be run on other apps because the other apps are signed
- No problem, we have the `mobile: uiautomator2` endpoint to take care of those cases
- Use-case: Show an example navigating to another app
Attendees will understand the basic internal difference between Espresso and UiAutomator2 frameworks and ways that they can use the Espresso framework to write better Android tests
People who are familiar with Android testing and who have an interest in learning about Espresso, how it's different from UiAutomator2, and how they can leverage Espresso to write better tests
Prerequisites for Attendees
Be familiar with Appium Android testing or at least have an interest in it.
schedule Submitted 1 year ago
People who liked this proposal, also liked:
Anton Angelov - How to Test the Test Automation Framework?Anton AngelovCTOAutomate The Planet
schedule 1 year agoSold Out!
Nowadays, more and more companies are building test automation frameworks based on WebDriver and Appium for testing their web and mobile projects. A big part of why there are so many flaky tests is that we don't treat our tests as production code. Moreover, we don't treat our framework as a product. In the talk, you will see examples of how you can automate the testing of your test automation framework and be sure that it is highly reliable. You can get lots of ideas for various types of tests such as learning tests verifying that 3rd party dependencies are not breaking the code, compatibility tests checking that the UI components are working for each mobile control on each OS, cross-platform verifications for testing whether everything is working on multiple OS. Sample test environments for storing different distributions of the framework packages will be presented.
Srinivasan Sekar / Sai Krishna - Native mobile commands in AppiumSrinivasan SekarLead ConsultantThoughtWorksSai KrishnaLead ConsultantThoughtWorks
schedule 1 year agoSold Out!
Apple and Google’s test automation framework does not natively support W3C standards for few web driver spec implementations directly for e.g TouchActions interface in XCTest, etc. Although test automation frameworks support a rich set of those functions specific to platforms, Appium does provide ways to directly invoke these functions e.g gestures, biometric handling, etc.
Many special behaviors and workarounds are made available and achieved only through executing platform-specific native commands of Appium. For instance, there are 100+ issues been reported on date picker or handling picker wheel in the appium organization but it can be achieved quite easily by executing native mobile commands.
There are so many that testers might not get chance to go through each one of these and get acquainted with all of those. Native mobile commands help to handle much complex use cases like biometric handling, talking to Siri, performance profiling, etc quite easily.
Kazuaki Matsuo - Uncovering breaking changes behind UI on mobile applicationsKazuaki MatsuoSr. Software Engineer, Device AutomationHeadSpin
schedule 1 year agoSold Out!
It is essential to track user logs correctly to improve and develop your own web/mobile services continuously. For instance, how users flow on your applications to evaluate if they work expectedly or not.
Meanwhile, mobile application trends have been changed quickly such as architectural things or UI related one. Developers continue to add, refactor or rewrite their applications frequently. They also need to release them frequently, 2-week release for instance. Their business also changes quickly. A number of developers working for one application also has been increasing.
As a result, it is quite difficult to catch up with everything. Developers know a part of them. They add, rewrite or refactor codebase they do not know well with exploratory it. Functionalities related to UI are easy to understand. But, it is difficult to uncover what happens in the backend such as what kind of logs the app sends to servers.
In Android case, if one application sends a log to a server on a fragment's onCreate. But the fragment can use in another view. If a developer does not know what the log means, he/she might re-use the fragment in another view if he/she think they can re-use it. It can break activity logs collecting on the server side. How to maintain logs is also an interesting topic though.
In general, we notice the breaking after releasing the app since we can easy to observe the number statistically. But, it means we can not use the data to evaluate our business correctly until we fix it and re-release it.
In this talk, I would like to show an example of how I had been implemented to uncover the above thing following some scenarios based on my experience. It might be an example what we already can automate in the mobile world.
This topic is similar to monitor CPU/Memory/network thing. This story is based on my experience I had been worked for a couple of years.
Jonathan Lipps / Daniel Graham / Kazuaki Matsuo - Fix a bug become a committerJonathan LippsFounderCloud GreyDaniel GrahamSoftware EngineerSauce LabsKazuaki MatsuoSr. Software Engineer, Device AutomationHeadSpin
schedule 11 months agoSold Out!
Have you ever wondered how Appium works under the covers? Do you get frustrated with locators not locating, app screens not loading, or test behaving inconsistently from one run to the next? Appium is an attempt to unify thousands of disparate elements across a wide spectrum of challenges into a single, common interface that works seamlessly across all the major mobile and desktop OSs - and yet only a handful of volunteers work to maintain this gigantic effort. If you would like to enhance your own Appium experience while contributing back to the software that has defined so many of our careers, come to this workshop. We'll dissect the different elements of Appium, dive into its internals, learn how it was built and how to make changes to it, and even write a unit test you can contribute on the same day!
Kazuaki Matsuo - Start contributing to OSS projects on your wayKazuaki MatsuoSr. Software Engineer, Device AutomationHeadSpin
schedule 1 year agoSold Out!
Through a story of my contribution to OSS projects, especially Appium, you can learn how you can start contributing to OSS world. I would also give some examples to start it.