Challenges in Testing & Automating Games

Testing real-time, multi-user games built for native apps and / or browser-based on phones / mobile devices / tablets / desktop browsers makes testing of regular products as apps, or websites appear like a piece of cake. Testing such real-time and multi-user games becomes even more challenging when you think about automating the same.

I got an opportunity to build Functional Test Automation for a suite of games – and what an exciting time it turned out to be!

These games are built either using Cocos2d-x, or Unity (cross platform game engines for mobile games, apps and other cross platform interactive GUI and are known for their speed, stability, and ease-of-use).

The key challenges I encountered here were:

• Millions of users, playing games on a huge variety of devices (Android & iOS native apps, Mobile-Web, and Desktop Web)
• Limited unit testing
• API testing & Functional Testing done in isolation (mini-waterfall approach)
• Usage of Cocos2d-x & Unity for game rendering – which cannot be automated via Appium
• Limited Functional Automation (for native apps)

An approach to Functional Automation of Real-Time, Multi-User Game scenarios

I overcame the above mentioned challenges by doing the following:

• A better way of working (you can call it ‘Agile’)
• Break down the walls by fostering a mindset of “build quality-in, as opposed to test for quality”
• Built a new functional automation framework using java / testing / appium-test-distribution / reportportal / jenkins with focus on – specify test intent once and run on all supported channels (ex: Android, iOS, Mobile-Web, and Web)
• Built a solution for Cocos2d-x layer automation
• Created a vision of CI-CD for the organization, and setup code-based CI pipelines to enable end-2-end visibility
• Made the framework extensible by providing ability to use same framework for multiple games

The focus of this talk will be to:

• Share an example of a particular use case

• Share the solutions, including code snippets, implemented for:
  • Functional Test Automation Framework Architecture & Design
  • Ease of Test Implementation, while maintaining code quality and promoting reuse
  • Test Execution on local Vs CI, on-demand as well as on every new build