How to be Maniacal about User Experienece with the help of Performance Test
Hike is an user-centric, mobile-first product, where user experience is the top priority. Hike started as an instant messaging app but during the course of the last four years, it has evolved into a messaging platform that offers a bouquet of features and services on top of messaging.
As a part of quality team, it becomes our primary goal to make sure that app is doing well in terms of performance for a 100+ million user base with an average smartphone cost of $120. We as a product want to achieve and solve lot of user problems but with a constraint of app running with limited device resources.
So the question arise, how do we achieve this goal?
Now, every night, every Sprint (2 weeks) and every public release (monthly), we run our automated perf tests. We measure our app's performance using several app specific use-cases on 4 key areas:
- Network (data consumption)
We also benchmark the following scenarios for app latency:
- App launch time upon Force Stop
- App launch time upon Force Kill
- App's busiest screen opening time
- Scrolling latency in different parts of the app
- Contact loading time in Compose screen
Embracing the fail-fast, fail-early principle, we now provide the necessary feedback to the developers as quickly as possible to avoid downstream delays.
Outline/Structure of the Case Study
- Driving automated tests to measure performance in mobile.
- How hike ensures the performance.
- Tool selection
- How/What to benchmark in performance.
- App specific parameters to benchmark
- Challenges with utilization of mobile resources.
- Next Steps in our Journey (making Perf-tests part of CI builds, etc.) and running the perf in the field with the instrumentation in place
- Advantages and Disadvantages of the current Perf suite and next steps
- Bench-marking mobile app on the parameters such as CPU, Memory utilization, Data consumption and Battery usage.
- Bench-marking app specific tests
- Automating Pref-Tests
- Making perf test more realistic and evolved
Performance Testers, QA Managers, QA Leads, Tool Developers, Data Driven Testers