A technique for IT Agile Metrics Benchmarking
Organizations across industry today are on a lookout for a faster-better-cheaper way of project delivery. Thus Agile Development practices have become a widespread phenomenon across all domains. Information Technology Sector, in particular, has been completely enchanted with the ‘Agility Mantra’. Needless to say, effective agile project management for successful execution of sprints is of paramount importance.
In this scenario, the current customer ask is primarily to measure and compare team performance across sprints and teams, self and vendor alike. At the same time any method built to address this should be pragmatic as well as intuitive so that it blends in well with prevalent agile practices. But when IT organizations across the world today want to manage productivity, they are faced with multiple practical challenges.
Though there are some means to measure productivity through velocity or ideal-time, they are not full-proof on the turf. Some techniques may indicate an improvement but the question remains whether it is ‘good enough’. The underlying reason for this is
- Sizing in silos – “Apples to Apples” comparison of story points and velocity across sprints or projects not possible as story point definition is used in a highly narrow project/sprint-specific perspective
- Lack of a common scale of sizing poses a question on the credibility of downstream metrics like productivity, defect density and so on.
- Poor metrics calculation leads to inability to baseline and benchmark performance and identify right improvement levers.
The effect of the above mentioned challenges can be summarized as below
- No Correlation between investments made with Size delivered
- No Quantitative way for clients to evaluate Vendors
- No Objective Resource Utilization
- Customer losing on Time-to-Market
The above, in turn, leads to dissatisfaction of the agile team as they are unsure of their own performance and do not have a clear picture as to where they stand with respect to their peers as well as industry. On the other hand, both clients and service providers like TCS are absorbing significant effort and revenue leakage.
Hence, the need of the hour is to have a standardized sizing framework which is quick and easy to use. At the same time this should aid in planning, tracking & benchmarking performance without being a hindrance to the agile way of executing projects. This presentation would describe TCS patented Agile SPACE estimation model which addresses the above challenges. TCS has invested huge amount of effort in collecting and analyzing more than 2000 agile sprints data across multiple domains and geographies as a precursor to building this model. This information was then processed by applying statistical methods for creating the basic structure. This was then validated by interviewing more than a thousand agile practitioners/experts and studying industry thought leadership/trends for conceptualizing this model.
This model estimates size and effort of agile sprints for software development/enhancement projects. The sprint size can be identified based on a comprehensive set of predefined parameters which evaluates the user stories and additional peripheral factors. The sprint size is further used to perform a capacity validation by arriving at the sprint effort. This is obtained using pre-base-lined “productivity” depending on technology. The “productivity” itself is derived from actuals from several thousand sprint results which has gone through strict and controlled statistical analysis. This self-sustaining model is also powered by “actual data collection” framework and feedback adapter. Project end results in terms of actual size and effort, user feedback, inputs from industry etc. are fed to the closed feedback loop using them.
The model intrinsically brings in a gamut of project management value-adds.
- Standardization: It drives standardization by bringing in a common scale for uniform definition of story points. Hence person dependency is removed without changing the existing agile process of estimating story points.
- Transparency:Story point measurement is transparent owing to segregation of parameterized complexity contributors and peripheral factors. This, in turn, enables quick and unanimous consensus of all participants of planning poker session.
- Comparability:The common scale of story point sizing allows comparability of sprint velocity across sprints within the project, across projects within the organization and benchmarking across industry.
- Repeatable & Predictable Sizing:It also ensures that same definitions of story point is maintained across multiple sprint estimations thereby allowing repeatable and predictable estimates.
- Metrics Calculation:The standard story point definition prepares a solid platform to derive significant downstream delivery metrics which includes say-do ratio, can-do ratio, capacity validation and above all productivity and defect density thereby enabling continuous improvement.
- Disciplined Cost Control: It facilitates the fast-catching trend of output based pricing approach in terms of 'Cost per Story-point'. This, in turn, equips customer to perform quantitative vendor evaluation.
The innovation has been termed as a "breakthrough" and "game-changer" for agile projects by lead assessors of the industry leading capability maturity evaluation body, CMMi. Zero similar inventions were found during a global prior art search in 2014 and a patent was filed for SPACE with US-PTO in the same year. It is with the aid of this innovation that agile practitioners in every corner of the globe can speak in a uniform language of standard story points thereby ensuring that 80 story points is exactly double of 40 story points.
Outline/Structure of the Talk
Key Takeaways, Lessons Learnt, Conclusion
Questions and Answers
Project Managers, Project Leads, Scrum Masters, Product Owners