Do you want to hear a story about overcoming obstacles and achieving seemingly unattainable goals at a massive scale? Well, we have one to tell - it’s a true story, and like all good stories, teaches us some valuable lessons. We have gone through the ups and downs of this tale and come out better and smarter. We would love to share those experiences and learning with everyone.

The story starts with a mission...automate 5000 hours of manual tests for our enterprise product. Like many other product based companies, we had one big monolithic application to test. The mission was to be accomplished with the resources available - no new magical dream team, we had to work with the resources we had - QA analysts with no technical background, a very small automation team, and a huge offshore manual testing group. Go figure! There was another twist - we had to accomplish our mission without dropping the current level of support for testing our enterprise application, including regression and new feature tests. Doesn't it all sound very familiar?

This presentation will cover all aspects of our journey from the beginning to the end. We went through a lot of ups and down, and every single decision we made taught us a great deal. It is those experiences that we want to share with everyone.

• We created a tool that wrapped the Selenium API in order to make it easy for non developers to write tests. The tests were written in a Domain Specific Language that made Selenium API calls with some application specific logic added in.
• We needed to build our own execution framework to support our growing automated test base. The framework offered many customized features and was able to sustain 60,000 hours of tests running every single day.
• We wrote our own best practices and worked closely with the QA team to make sure everyone wrote high quality tests.
• The results from the tests needed to be displayed in a way that made sense. We created several different dashboards for that purpose and had many different views of the test suite performance, including a heat map to highlight problem areas.
• Elasticsearch and Kibana were instrumental in helping us parse through the massive volume of test results and make sense of them, giving us metrics in different forms.
• Daily environment setup for this execution was also massive - 100 or so slaves and several SUTs for every codeline, with support for 3 codelines meant that we needed a big lab setup.

We successfully completed the mission of automating the manual test behemoth and gained a rich understanding of test automation at scale along the way.