Biting the Bullet - CMMi Level 5 Assessment for Agile Programs

CMMi Level 5 assessment has been a necessity and a hygiene factor for any service organization. Our organization has been assessed at Level 5 continuously since 1999. However, with spread of Agile methodologies in offshore scenarios we were faced with a question of ensuring CMMi compliance in Agile programs.

This session will delve into details of our CMMi journey for Agile programs and how NIIT Technologies successfully got re-assessed at CMMi Level 5 with Agile in August 2013. The session will talk about various mapping of agile practices with CMMi process areas, identified gaps and how these gaps were addressed to ensure a successful assessment without diluting the Agile promise. 

Some examples of how various process areas have been mapped to various agile practices. The session will be mostly example based and experience sharing. 

Example 1 (Level 2 Maturity)

CMMi Requirements Management PA talks about bidirectional traceability. If you go to sub practice (SP 1.4 of Requirements Management PA) level then there are three requirements: 

  • 1) Maintain requirements traceability to ensure that the source of lower level (i.e., derived) requirements is documented.
  • 2) Maintain requirements traceability from a requirement to its derived requirements and allocation to work products.
  • 3) Generate a requirements traceability matrix.

Looking at the above, we managed this through requirements management being done in Agile projects using tools like Rally, TFS (agile templates), Jira Greenhopper and NIIT's internal agile management tools. These tools provide traceability from Epics --> Stories --> Tasks --> Test Cases --> Defects. This fullfils the intent of the PA. Hence we do not explicitly generate any RTM especially for agile projects (3rd bullet above) as the intent is already met. 

Example 2  (Level 2 Maturity)

CMMi Project Monitoring and Control (PMC) PA talks about a goal "Manage Corrective Actions to Closure" (SG2). This has 3 separate sub processes

  • 1) Analyze Issues
  • 2) Take corrective Action 
  • 3) Manage Corrective Action

All these have been covered through the Agile practice of having "retrospectives", which happen regularily in Scrum. Retrospectives is where the people point out issues amongst other things, discuss what actions can be taken to improve and/or address the problem at hand. These action items coming out of retrospectives are then tracked as they would normally  have been done to closure. We do not mandate any specific format/tool for capturing and tracking this information.  

Example 3 (Level 4 Maturity)

CMMi Organization Process Defintion PA, has a goal for "Establishing Performance Baselines and Models" and a Sub Practice "Analyze Process Performance and Establish Process Performance Baseline".

Here instead of taking traditional  metrices like productivity (hrs/FP), schedule slippage etc. we have taken metrices which are more close to the agile process. Examples of these are: 

  • 1) UAT Defects per Engineering Effort Unit
  • 2) Velocity Improvement over time
  • 3) % of story point change to measure requirements stability   
 
1 favorite thumb_down thumb_up 9 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

  • CMMi Introdution and Process Areas
  • Agile practices mapping with process areas 
  • Gaps in Level 4 Maturity and means to address them 
  • Gaps in Level 5 Maturity and means to address them 
  • Key Learnings 

Learning Outcome

  • CMMi Process Areas and their mapping with Agile Practices
  • High Maturity Areas and their discomfort with Agile 
    • Level 4 Gaps and how to address them in an agile program for a successful assessment
    • Quantitative Project Management 
    • Organization Process Peformance 
  • Level 5 Gaps and how to address them in an agile program for a successful assessment. 
    • CAR
    • Organization Performance Management 

Target Audience

Agile practitioners, process experts, IT professionals

schedule Submitted 4 years ago

Comments Subscribe to Comments

comment Comment on this Proposal
  • Doc Norton
    By Doc Norton  ~  4 years ago
    reply Reply

    Puneet:

    It appears you've given this presentation a couple of times in the past. What are the common questions you receive during the session and how have you made updates to the presentation to address those questions? Along those same lines, what are the common criticisms you receive and how have you made updates to the presentation to address those criticisms?

    - Doc

     

    • Puneet Sachdev
      By Puneet Sachdev  ~  4 years ago
      reply Reply

      Hi Doc,

      The past presentations (especially the one in ALN in Delhi) centered around three aspects: 

      1. Drivers for choosing Agile and outsourcing and a discussion on how to marry the two to create a win-win. 

      2. CMMi assessment and its importance in the service industry. Its mapping with agile practices and implications of getting an agile program assessed for level 5 maturity. 

      3. Different forms of agile contracts which ensure certainty while keeping benefits of Agile intact. 

      This proposal is about topic 2 above, hence we intend to go a lot deeper into this specific subject. Our organization recently got assessed at Level 5 (in Aug 2013) and during the process for the first time multiple large agile programs took part in the assessment successfully. 

      On the point of questions....get lot of quesitons every time any of the above 3 topics are presented. I believe your question is in relation to CMMi assessment for agile programs as that is the topic of the proposal. In this area some of the common questions are as follows: 

      1. Questions around the need to do this in the first place. Why is it important...why did we do it. This is also a criticising point...as especially product companies do not see value in such assessments. On the other hand service companies do...and hence discussions happen between these two camps. Many service organizations typically do not subject Agile programs to this audit...hence they share their reservations and ask questions around those. (see Srinath's comments below)

      2. Questions around certain process areas and how these were handled.  High maturity are the ones where there is maximum confusion. 

      3. Questions around estimates and metrices. E.g: On the topic of bringing in some formalization on estimation technique and usage of FP instead of SP in some of our large agile programs. Same for baselines. This also leads to criticism relating to loss of agility due to some of these measures.  

      Typically such questions are addressed by giving concrete examples of scenarios, why certain practice is introduced and its implications. E.g. One criticism we typically get is that in cases we use FP for estimation we loose agility. Our response: SPs are used for a specific reason in agile...and that is to reduce/eliminate time wasted in estimation management by the team. If same can be achieved by using some other metric like FP, then it is acceptable.

      Here as well, this proposal being a casestudy, we plan to address these questions with specific examples. What challenges we faced and how did we address the reqiurements of CMMi. Hence, plan to include multiple examples in the presentation as we are covering only the CMMi topic in detail this time. 3 examples are given in the proposal above as well.  

  • Ravi Kumar
    By Ravi Kumar  ~  4 years ago
    reply Reply

    Hi Puneet,

    Thanks for submitting the proposal.

    I see this topic is a best fit in Agile Life Cycle theme since CMMI is not necessarily meant only of offshoring companies. Besides; the coverage on the topic is on co-existance/implementation of agile in CMMI context. Strongly recommend to move this over quickly.

     

     

    Thanks and regards,

     

    Ravi

    • Puneet Sachdev
      By Puneet Sachdev  ~  4 years ago
      reply Reply

      Moved as suggested. 

  • Srinath Chandrasekharan
    By Srinath Chandrasekharan  ~  4 years ago
    reply Reply

    Puneet,

    This is a very "controversial" topic if I may say so. I have seen your session in ALN in July. In addition to whatever you ahve on the deck, I would also like to see some more content on the need for assessing Agile projects using CMMI. In my experience and in my organization, we have conciously kept these two separate .

    Another thing of interest would be to see how did your organization meet the needs of "evidence" in CMMI ? Was Agile model made more heavy in this process ? Was there any loss of Agility due to this ?

     

    Regards,

    Srinath

     

    • Puneet Sachdev
      By Puneet Sachdev  ~  4 years ago
      reply Reply

      Srinath, 

      A very simplistic answer is that in NIIT, Agile has spread to proportions where we cannot afford to keep it separate/isolated any more. The guidelines for project/programs being selected for assessment from SEI are very clear and for the complete organization to be assessed, you need representation from every BU, Geography, Location, Verticals etc. Since Agile is so widespread in NIIT, many of these programs cannot escape the assessment. 

      We had realized this fact almost 2 years ago and hence had been preparing for the final assessment. CMMi need evidence to make sure that the activity has actually happened. Hence we looked at artifacts that get generated naturally in the process as evidences rather than generate something specifically for evidence. More challenge comes in the level 4, 5 areas. 

      To the point of losing agility, adaptations to the process have been made definately in some of the programs in the organization. These adaptations are more done keeping the offshore nature and scale of the program in mind rather than CMMi needs. 

       

      • Srinath Chandrasekharan
        By Srinath Chandrasekharan  ~  4 years ago
        reply Reply

        Puneet,

         

        Thanks for the information. Can you add some more examples either in your deck or update the proposal for us do a better review. I personally see this as a topic of interest for many people with services background and your org's experience would definitely be of value to others.

        Regards,

        Srinath

        • Puneet Sachdev
          By Puneet Sachdev  ~  4 years ago
          reply Reply

          Hi Srinath, 

          Am writing a few examples here. Will also update the proposal above. Basically while mapping process areas with agile practices we go deep into sub processes in CMMi model and try to map the intent of the process area instead of mapping the sub process area's requirements verbatim. 

          Example 1 (Level 2 Maturity)

          CMMi Requirements Management PA talks about bidirectional traceability. If you go to sub practice (SP 1.4 of Requirements Management PA) level then there are three requirements: 

          • 1)Maintain requirements traceability to ensure that the source of lower level (i.e., derived) requirements is documented.
          • 2)Maintain requirements traceability from a requirement to its derived requirements and allocation to work products.
          • 3)Generate a requirements traceability matrix.

          Looking at the above, we managed this through requirements management being done in Agile projects using tools like Rally, TFS (agile templates), Jira Greenhopper and NIIT's internal agile management tools. These tools provide traceability from Epics --> Stories --> Tasks --> Test Cases --> Defects. This fullfils the intent of the PA. Hence we do not explicitly generate any RTM especially for agile projects (3rd bullet above) as the intent is already met. 

          Example 2  (Level 2 Maturity)

          CMMi Project Monitoring and Control (PMC) PA talks about a goal "Manage Corrective Actions to Closure" (SG2). This has 3 separate sub processes

          • 1)Analyze Issues
          • 2)Take corrective Action 
          • 3)Manage Corrective Action

          All these have been covered through the Agile practice of having "retrospectives", which happen regularily in Scrum. Retrospectives is where the people point out issues amongst other things, discuss what actions can be taken to improve and/or address the problem at hand. These action items coming out of retrospectives are then tracked as they would normally  have been done to closure. We do not mandate any specific format/tool for capturing and tracking this information.  

          Example 3 (Level 4 Maturity)

          CMMi Organization Process Defintion PA, has a goal for "Establishing Performance Baselines and Models" and a Sub Practice "Analyze Process Performance and Establish Process Performance Baseline".

          Here instead of taking traditional  metrices like productivity (hrs/FP), schedule slippage etc. we have taken metrices which are more close to the agile process. Examples of these are: 

          • 1)UAT Defects per Engineering Effort Unit
          • 2)Velocity Improvement over time
          • 3)% of story point change to measure requirements stability   

          As we get into Level 4 and 5, more and more process areas need to be dealt with innovatively. Let me know in case more examples are needed. 

          • Srinath Chandrasekharan
            By Srinath Chandrasekharan  ~  4 years ago
            reply Reply

            Puneet,

             

            This is useful. Hpwever as Ravi has suggested, this seems appropriate for the Agile LIfe Cycle theme.

             

            Regards,

            Srinath