Uncovering breaking changes behind UI on mobile applications

schedule Jun 15th 03:00 - 03:45 PM place Grand Ball Room 2

It is essential to track user logs correctly to improve and develop your own web/mobile services continuously. For instance, how users flow on your applications to evaluate if they work expectedly or not.

Meanwhile, mobile application trends have been changed quickly such as architectural things or UI related one. Developers continue to add, refactor or rewrite their applications frequently. They also need to release them frequently, 2-week release for instance. Their business also changes quickly. A number of developers working for one application also has been increasing.

As a result, it is quite difficult to catch up with everything. Developers know a part of them. They add, rewrite or refactor codebase they do not know well with exploratory it. Functionalities related to UI are easy to understand. But, it is difficult to uncover what happens in the backend such as what kind of logs the app sends to servers.

In Android case, if one application sends a log to a server on a fragment's onCreate. But the fragment can use in another view. If a developer does not know what the log means, he/she might re-use the fragment in another view if he/she think they can re-use it. It can break activity logs collecting on the server side. How to maintain logs is also an interesting topic though.

In general, we notice the breaking after releasing the app since we can easy to observe the number statistically. But, it means we can not use the data to evaluate our business correctly until we fix it and re-release it.

In this talk, I would like to show an example of how I had been implemented to uncover the above thing following some scenarios based on my experience. It might be an example what we already can automate in the mobile world.

This topic is similar to monitor CPU/Memory/network thing. This story is based on my experience I had been worked for a couple of years.

 
3 favorite thumb_down thumb_up 2 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/Structure of the Case Study

  • A story what is a problem developers face when they add, rewrite or refactor codebase
    • Behind UI functionality layer
    • It can impact to evaluate business
  • An example using HTTP proxy server to capture network
    • I had been used for a couple of years
  • Other additional tools (If I have a time)

Learning Outcome

  • Leaning we can automation test cases which is not only interacting UI

Target Audience

Engineers who are interested in what tests we can automate behind UI layer

Prerequisites for Attendees

  • Engineers who know Android/iOS development
    • It will help to imagine how easy to happen this kind of topic
  • Engineers who already have experienced UI automation
  • Engineers who know of network layer
    • HTTP request, for example
schedule Submitted 3 months ago

Public Feedback

comment Suggest improvements to the Speaker
  • Anand Bagmar
    By Anand Bagmar  ~  2 months ago
    reply Reply

    Hi Kazuaki,

    Do you have any past video recording that you can share? Also, will you be doing any demos in this session?

    • Kazuaki Matsuo
      By Kazuaki Matsuo  ~  2 months ago
      reply Reply

      No, I have no video. I just published this topic as blog posts.

      This topic is a combination of Appium(automation script) and https://github.com/KazuCocoa/http_proxy (Previously, I used WwireMock instead of it), I am thinking. I had implemented and conducted this since 2016. So, technically, this is not new.

       

      > will you be doing any demos in this session?

      Currently, I won't do demos, but I can do this only local machine and local network if you request.


  • Justin Ison
    Justin Ison
    Sr. Software Engineer
    Applitools
    schedule 4 months ago
    Sold Out!
    45 Mins
    Demonstration
    Beginner

    In today’s agile world the time to market is becoming increasingly shorter. There is a constant desire to release ASAP to keep ahead of the competition and to please users with updated/new features. Because of this, we have less time to fully do manual and exploratory testing of our apps. Especially, when you consider all the combinations of OS's, Locales, Accessibility, Orientations & Resolutions apps support. Running anywhere from 1 to 100's of Appium crawler bots (covering all of those combinations) at once we can discover more issues quickly and efficiently without having to write a line of code.

    UI Automation also has its limitations as it only tests for expected results. Crawler bots test the unexpected, by collecting metadata such as logs, app strings, screenshots, memory and reporting back it’s finding for review so we can test all these combinations quickly and more efficiently. In this talk, I will go over the current challenges we face in today's development world, why we need more tools to help us keep pace, and cover how you can build your own Appium crawler.

    I've open sourced this tool and is available here for everyone to use: https://github.com/isonic1/Appium-Native-Crawler

  • Liked Srinivasan Sekar
    keyboard_arrow_down

    Srinivasan Sekar / Sai Krishna - Native mobile commands in Appium

    45 Mins
    Tutorial
    Intermediate

    Apple and Google’s test automation framework does not natively support W3C standards for few web driver spec implementations directly for e.g TouchActions interface in XCTest, etc. Although test automation frameworks support a rich set of those functions specific to platforms, Appium does provide ways to directly invoke these functions e.g gestures, biometric handling, etc.

    Many special behaviors and workarounds are made available and achieved only through executing platform-specific native commands of Appium. For instance, there are 100+ issues been reported on date picker or handling picker wheel in the appium organization but it can be achieved quite easily by executing native mobile commands.

    There are so many that testers might not get chance to go through each one of these and get acquainted with all of those. Native mobile commands help to handle much complex use cases like biometric handling, talking to Siri, performance profiling, etc quite easily.

  • 45 Mins
    Case Study
    Intermediate

    Experimentation and quick feedback is the key to success of any product, while of course ensuring a good quality product with new and better features is being shipped out at a decent / regular frequency to the users.

    In this session, we will discuss how to enable experimentation, get quick feedback and reduce risk for the product by using a case study of a media / entertainment domain product, used by millions of users across 10+ countries - i.e. - we will discuss Testing Strategy and the Release process an Android & iOS Native app - that will help enable CI & CD.

    To understand these techniques, we will quickly recap the challenges and quirks of testing Native Apps and how that is different than Web / Mobile Web Apps.

    The majority of the discussion will focus on different techniques / practices related to Testing & Releases that can be established to achieve our goals, some of which are listed below:

    • Functional Automation approach - identify and automate user scenarios, across supported regions
    • Testing approach - what to test, when to test, how to test!
    • Manual Sanity before release - and why it was important!
    • Staged roll-outs via Google’s Play Store and Apple’s App Store
    • Extensive monitoring of the release as users come on board, and comparing the key metrics (ex: consumer engagement) with prior releases
    • Understanding Consumer Sentiments (Google’s Play Store / Apple’s App Store review comments, Social Media scans, Issues reported to / by Support, etc.)

  • Liked Anton Angelov
    keyboard_arrow_down

    Anton Angelov - How to Test the Test Automation Framework?

    Anton Angelov
    Anton Angelov
    CTO
    Automate The Planet
    schedule 3 months ago
    Sold Out!
    45 Mins
    Demonstration
    Intermediate

    Nowadays, more and more companies are building test automation frameworks based on WebDriver and Appium for testing their web and mobile projects. A big part of why there are so many flaky tests is that we don't treat our tests as production code. Moreover, we don't treat our framework as a product. In the talk, you will see examples of how you can automate the testing of your test automation framework and be sure that it is highly reliable. You can get lots of ideas for various types of tests such as learning tests verifying that 3rd party dependencies are not breaking the code, compatibility tests checking that the UI components are working for each mobile control on each OS, cross-platform verifications for testing whether everything is working on multiple OS. Sample test environments for storing different distributions of the framework packages will be presented.

  • Liked Dawid Pacia
    keyboard_arrow_down

    Dawid Pacia - Mobile & IoT integration testing - "Mission Impossible" or "A last hope"?

    Dawid Pacia
    Dawid Pacia
    Test Automation Manager
    Brainly
    schedule 4 months ago
    Sold Out!
    45 Mins
    Case Study
    Intermediate

    It is predicted that till 2025 there will be over 75 billion devices spewing 180 zettabytes of data and generating up to 6 trillion dollars. That enormous increase force companies to introduce a continuous approach to deliver the product as fast as possible and be able to compete on the market.


    The main question is how to test application for end user among so much hardware equipment and ecosystems combining HW, FW, mobile devices and complex backend architecture? Considering all factors and possible obstacles is it for companies a real “A New Hope” for companies or just simply “Mission Impossible”?


    I will take the participants on a journey to the IoT world. It will be a talk about the challenges that any tester will face at some point. I will present the dangers, risks and snares but also good practices and practical approach to mobile E2E test automation for the IoT solutions in CI approach.


    Technical examples will be presented using Python languages and supported by physical devices (mobile phones and IoT equipment).

  • Liked Jonah Stiennon
    keyboard_arrow_down

    Jonah Stiennon - All Desired Capabilities

    Jonah Stiennon
    Jonah Stiennon
    Partner
    Cloud Grey
    schedule 3 months ago
    Sold Out!
    90 Mins
    Tutorial
    Intermediate

    Appium has so many desired capabilities! How many exactly? Let's count!

    Join Appium contributor Jonah Stiennon as he iterates through every desired capability supported by Appium.

    Many special behaviors and specific workarounds are made available only through desired capabilities. Often the key to selecting an element reliably, launching an app, or avoiding a timeout, is picking the right set of desired capabilities. There's so many that testers don't get the chance to sit down and become acquainted with them all.

    There's too many to easily memorize and documentation can be sparse. Jonah will introduce novel ways to classify, visualize, and organize all of the desired capabilities, making it easier to find ones which can be useful.

  • Liked Sravan Kumar
    keyboard_arrow_down

    Sravan Kumar - Bootcamp to understand Appium android internals

    Sravan Kumar
    Sravan Kumar
    Sr. Software Engineer
    DigitalSurgery
    schedule 3 months ago
    Sold Out!
    45 Mins
    Tutorial
    Intermediate

    Appium is a world's most popular open source mobile test automation framework developed using WebDriver protocol and I am fortunate enough to get a chance to contribute to appium-uiautomator2-server and appium-uiautomator2-driver modules.

    The goal of the session is to help the Appium community to have a better understanding of Appium and how it works.

    In this session, I will be discussing Appium architecture specifically towards its Android modules(UiAutomator2 and Espresso) and how the communication happens between Appium modules.

  • Liked Wim Selles
    keyboard_arrow_down

    Wim Selles - Building your own native app visual testing module with Appium

    Wim Selles
    Wim Selles
    Solution Architect
    Sauce Labs
    schedule 3 months ago
    Sold Out!
    45 Mins
    Case Study
    Beginner

    We all know that Appium can do (almost) everything what you tell it to do, but in the end we always ask it to do the same thing. We ask Appium to automate our happy and or our error flow. The result can give us the confirmation that the user can still buy a product, or verify if his bank account still has a positive balance.

    But what if we need to know more than that - what if we also need to verify the layout of our app? The answer would be easy, as there are plenty of open-source or paid solutions that can help you with that. However, almost all of these solutions focus on web/hybrid apps, not on native iOS and or Android apps.

    I faced this challenge when I was automating a React Native app for my customer. There was no image comparison tool that could do what I wanted it to do, so I rolled up my sleeves and started building my own. In this talk we’ll be walking down the path I’ve been taking and I’ll explain:

    • Why I used ResembleJS as my core visual comparison solution
    • The things I’ve learned about Appium in the process (pro tips!)
    • The differences between iOS and Android and how I managed to solve this in one cross-platform solution
    • The pros and cons of my solution
  • Liked Angie Jones
    keyboard_arrow_down

    Angie Jones - Your Tests Lack Vision: Adding Eyes to Your Automation Framework

    Angie Jones
    Angie Jones
    Sr. Developer Advocate
    Applitools
    schedule 2 months ago
    Sold Out!
    45 Mins
    Demonstration
    Intermediate

    Automation has come a long way in assisting with regression testing efforts. Teams worldwide are successfully running hundreds of functional regression tests at every check-in. While this provides a great source of confidence, critical regression bugs are still missed using this approach. That’s because these tests can only assert on what their human programmer asks them to. Additional errors with functionality, UX, and usability often go uncaught using today’s most common test automation techniques.

    For this reason, the top companies in all sectors of the industry are turning to visual validation. Visual validation is a relatively new concept that can be used to enhance existing automated tests and provide an easy way to perform those difficult checks for things like UX, localization, usability, responsive design, and cross-device testing.

    In this talk, you’ll learn how visual validation works, see a live integration into an existing Appium test code base, and discuss the pros and cons of using various visual validation techniques.

  • Liked Jonathan Lipps
    keyboard_arrow_down

    Jonathan Lipps / Daniel Graham / Kazuaki Matsuo - Fix a bug become a committer

    Jonathan Lipps
    Jonathan Lipps
    Founder
    Cloud Grey
    Daniel Graham
    Daniel Graham
    Software Engineer
    Sauce Labs
    Kazuaki Matsuo
    Kazuaki Matsuo
    Sr. Software Engineer, Device Automation
    HeadSpin
    schedule 1 month ago
    Sold Out!
    480 Mins
    Workshop
    Advanced

    Have you ever wondered how Appium works under the covers? Do you get frustrated with locators not locating, app screens not loading, or test behaving inconsistently from one run to the next? Appium is an attempt to unify thousands of disparate elements across a wide spectrum of challenges into a single, common interface that works seamlessly across all the major mobile and desktop OSs - and yet only a handful of volunteers work to maintain this gigantic effort. If you would like to enhance your own Appium experience while contributing back to the software that has defined so many of our careers, come to this workshop. We'll dissect the different elements of Appium, dive into its internals, learn how it was built and how to make changes to it, and even write a unit test you can contribute on the same day!

  • Liked Daniel Graham
    keyboard_arrow_down

    Daniel Graham - Unlocking New Testing Capabilities with Espresso Driver

    Daniel Graham
    Daniel Graham
    Software Engineer
    Sauce Labs
    schedule 1 month ago
    Sold Out!
    45 Mins
    Talk
    Intermediate
    Android's Espresso framework was created as a "gray-box" testing framework, "gray-box" meaning that it's an 
    automated UI testing framework ("black-box") but it has access to the internals of the application
    ("white-box"). Allowing access to the internals of the application-under-test opens up many new testing
    possibilities that weren't possible with UiAutomator2 (with some risks).
    Here are the four new possibilities I would like to discuss (with coding examples)
    
    * Access to internal app code
    * Less flakiness thanks to "IdlingResource". No need for 'waits' and 'delays' for UI.
    * Navigate WebViews using Espresso WebAtoms (this feature is in progress, will be ready well before June)
    * Find elements off-screen using Espresso's DataMatchers (this feature is in progress, will be ready well before June)
  • Liked Kazuaki Matsuo
    keyboard_arrow_down

    Kazuaki Matsuo - Start contributing to OSS projects on your way

    20 Mins
    Talk
    Beginner

    Through a story of my contribution to OSS projects, especially Appium, you can learn how you can start contributing to OSS world. I would also give some examples to start it.

  • Liked Anton Angelov
    keyboard_arrow_down

    Anton Angelov - The 5th Generation of Test Automation Frameworks

    Anton Angelov
    Anton Angelov
    CTO
    Automate The Planet
    schedule 3 months ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    The need for test automation nowadays is undeniable. For choosing the right solution for our context, we need to know what are our options and fully understand them. We will talk about the five generations of test automation frameworks, what they include, who uses/used them and how they are related to the evolution of the QA profession.

    The full-stack test automation frameworks are the 5th generation of tools. You will hear why they are different, what they include and why this is important. Understanding the evolution of the automation tooling entirely will help you define better the requirements for your test solution or give you ideas on what to improve in your existing one. As a bonus, you will see demonstrations of sample features of such a framework and get inspired how to build them yourself.