SDET at Hike Limited.
SDET at Hike Limited.
Hike is used by 100 Million users and many of our users have cheap smart phone (~ $120 USD) that can install no more than 3 mobile apps.
So the questions is: Should testing of app be limited to its functionality? At Hike, we believe "Performance is Queen!" For our users, if we misuse the critical resources such as Battery, CPU, Network and Memory, its a deal-breaker. Hence pref-testing is very important.
During the initial days of Hike, we were very reactive and only did (manual) perf testing, when our users reported issues.
Now, every Sprint (2 weeks) and every public release (monthly), we run our automated perf tests. We measure our app's performance using several app specific use-cases on 4 key areas:
We also benchmark the following scenarios for app latency:
We still have a long way to go in terms of our pref-testing journey at Hike. But we feel, we've some key learnings, which would be worth while to share with the community. Join us, for a fast paced perf-testing session.
Today over 100 million users share over 40 billion messages per month on Hike. Its not just simple 1:1 chat messages. Users can do a VoIP call or share rich multi-media content in 8 different languages in group chats with hundreds of members. User can transfer large (upto 100 MB) file using Wifi-Direct .i.e. device to device file transfer without using Internet. And many more features. How do you ensure that you can roll out a release every month without breaking any of these features?
With such a large user based, which is very sensitive to frequent upgrades due to data consumption cost, rigorously testing the app becomes extremely critical.
When we started our automation journey in 2014, we were looking for a device lab which can simplify our testing effort. However we gave up and ended up building our own setup. The reason being, we require multiple devices that can communicate with each other for a single test. And we have 6000+ such tests, which we want to run in parallel. While many device labs allow you to run tests in parallel, they don't allow the devices to communicate with each other. Also its not possible to run the same test across multiple devices. Imagine testing a group-chat flow with photo sharing or imagine the device to device file transfer using hotspot. How would you test these features?
If this interests you, join us and we'll share our learning trying to achieve this at Hike.