Test and deploy quickly through Agile - A customized approach
“Agile Testing is about focusing on exploration, discovery, investigation, learning and reporting rather than defining process and writing heavy documentation” - Bret Pettichord
Changes are inevitable. It is an eternal truth. Our software world too could not depart from it. In large scale deployments for multichannel business environments, customers are highly focused on time to market. Thus practicing and embracing Agile as a tool is becoming increasingly popular as customers are highly benefited from the quick turnaround time for software implementations.
Working as a cohesive team being the underlying concept of Agile ways of working; facilitates, “continuous integration” between “automated tests” and code, in every code deployment to pre-production environment. This ensures Product Owner gets a “quick feedback” throughout the journey and also re-iterates the quality of deployment at all intervals of “seamless integrated quality deployments”.
This whitepaper illustrates a proposed customized Agile Testing solution Methodology which helped the customer delivering high quality and frequent deployments in a short span of time for a large scale enterprise deployment.
The discussed customer needed to deploy its multichannel solution of various markets within a year. The delivery model was decided to be Agile, Scrum. For the rollout, this customer was looking at seamless incremental deliveries across locales, providing quicker solutions to the end users by matured Agile engineering test methodology, strategy, tools, and best engineering practices from Agile experts.
Upon detailed analysis it was felt the following are the critical success factors that needed to be taken care while building a robust Agile testing framework.
- Behavior driven testing to be adopted by Agile testers within Scrum teams, in releasing a ‘ potentially shippable software’ at the end of each sprint
- Vital to have an common independent end-to-end testing team of all deployments at a high level where multiple Scrum teams are involved
- Discrete support functions of the Agile test methodology for a matured Agile testing centre which build high energy teams with an Agile mindset
- Proposed Solution
In shaping up the complete solution that incrementally got developed and deployed in production on a monthly basis, solution for the Agile Test Model was derived based on the important functions of People, Product, Automation and Test in a Scrum model which has multiple scrum teams working on various systems. These functions provide a customized approach to be adopted keeping in mind all the core Agile principles.
- PEOPLE: “Techno functional “Agile testers up skilling themselves with technology, building an open technical mindset towards developing test fixtures, Acceptance Criteria, models and harness to quickly automate while the product is being built. By pairing up with developers, testers should assimilate thinking to build a safety net which ensures failures are shown upon even the smallest deviation along the Journey. Testers in this context are well equipped to test behaviors of the product and helping product owners to enhance the features by suggesting improvements
- AUTOMATION: Automated Tests: Thorough focus on the Automation: Develop and run automation test fixtures to integrate, test and deploy application software with production-like quality in a production-like environment every single day during the sprints. Trigger and Publish test results in an automated way across key stakeholders of the project and strive continuously improve on the coverage with clear focus on adopting new tools and techniques along the lines of automation.
- CONTINOUS TEST: Continuous integrated Test Strategy: Seamless automation and integrated System testing at multiple scrum team levels focusing all non functional tests( Performance,CrossBrowser,Usablity etc) with a dedicated Scrum team of Testers. Team which is dedicated in building Tool Kits, Frameworks and Reusable repositories. Motive of this team is to provide continuous testing solution in different promotional test environments from UAT to till production and keep adding defects in product backlog to refine the product.
- PRODUCT: Maturing Product :With a deep insight into the product by continuous communication with product owners and architects Agile testers should be able to reach deeply into the product's domain and suggest ‘nice to have’ and ‘must have ‘ features and build a knowledge base for re use.
- Customized Agile test framework :Customized Agile test methodologies, test processes and tools which can facilitate automation and act as a safety net for all build releases across program level keeping all Agile principles in mind.
The generic benefits of Agile are known to everyone. However, the customized approach that was adopted here has following calculated and proven benefits for the customer.
- Continuous test keeps the product health up-to-date always. There is an increased confidence on the periodic production release across locales. Test suite is always up-to-date and ready to be run at will
- With regression automation the test execution time would be considerably cut down up to 60%
- With more interaction during the sprint between all the stakeholders of the project especially between developers, testers and business, gives more clarity around the growth of the Product. It also ensures any project risk and potential wrong implementation caught sooner than later.
- Customized Agile approach which is being proposed brings the benefits of creating a core team with niche skills. For eg. Automation tester, Non-functional tester etc, entire testing team is capable of handling any Agile project at organization level at any given point of time. Hence increase in flexibility.
- Having an independent end-to-end testing team for multiple Scrum teams, acts as the overall product expert. This team knows about the overall product functionality and designs the test cases keeping the whole product in mind, rather than designing tests only for the Scrum team.
All framework and solution has its own limitation. The proposed solution for this customer was by keeping the entire test organization in mind. The limitation was only observed at a micro level than in the macro-level. The proposed solution has more benefit than limitation. Some of the limitations are given below.
- Scrum testers had to engage with independent end-to-end testing team, taking bit more of their time spent in meetings
- Independent end-to-end testing team does not attend any daily Scrum hence less aware of the changes happening in a day to day basis
The proposed solution is unique as opposed to generic Agile Scrum and other Scrum framework followed by different teams. Some of the unique solution offerings are noted below.
- The highlight and the most important uniqueness of this model is to have a core testing team apart from the testers in working in the individual Scrum teams( Eliminates Programmer-centric view)
- Highly-scalable model. It can support onsite-offshore model efficiently as opposed to other Agile flavors
- Knowledge retention and knowledge sharing becomes effective vide the core team and addresses Resource career development
- Tool management and Automation is centralized and one shop stop for all needs
- Scrum testers’ sole focus is on story testing and Core testing team can work as a launch pad before a tester moves to Scrum team, where the tester needs to perform from the very first day
- Implementation Details
Agile Testers at Sprint level
Agile Tester participates in planning activities in the sprint and helps the team in elaborating requirements by White boarding and developing Acceptance Criteria, also ensures to build a text fixture with a component level automated test for the Acceptance test functions along the lines of the developed piece of code. Every piece of code checked in by the developer should have 100% Unit tested and all automated tests of the agile tester to be run along with the daily build process.
Agile testers perform ‘exploratory tests’ for non automatable functionality if any, focusing on exploration, discovery, investigation, learning and reporting rather than defining the process and writing heavy documentations. At this end juncture of sprint, agile tester helps the team to provide customer, a demo of the potentially shippable software thus marking the end of the sprint
Agile Testers at Scrum of Scrum level
An independent team of testers validate the product from the end to end functionality perspective which takes care of all the overlapping tasks like Performance Testing, System Integration Testing and Non Functional Testing( Cross Browser, Usablity, Accesiblity Etc). They also get involved in building and continuously enhance Test Framework with reusable test fixtures, classes and libraries, develop Test Approaches and adopt accelerators for new areas of technology in the testing. This team also set best practices in Agile testing in terms of process, Technology and tools etc. This team ensures that there are end to end test scripts build and executed in various environment as agreed and promoted in stages ( Ex: UAT->Production)
- Summary and Conclusion
Embracing Agile as a delivery model has become ‘essential’ as opposed to ‘optional’. The adoption of a robust agile process framework provides a definite advantage to release the product in time with a better quality and instigates all testers to transform their skill as per the agile demand. Need for time to market in a competitive environment where small releases to production with more business value is certainly a deal breaker compared to a big bang traditional release.
Outline/structure of the Session
Knowing Agile Testing in a much better sense and the person would be able to customize Agile to suit the need