Mirror, Mirror on the wall, what are the worst Vanity Metrics of them all?
Let’s face it, Metrics are a staple of virtually every organization.
The intent behind Metrics is to improve. A very Agile idea indeed. Unfortunately, rampant misuse tends to create more waste than value. Teams often don't understand or believe in them. Managers often misinterpret them or never really use them. NOT a very Agile idea.
We can change this. It's not just a Fairy Tale.
We can empower teams to take ownership of their data, including defining what data really helps them improve. We can educate management to better understand what to ask for and how to properly interpret the data. Thus, turning Vanity metrics into Valuable metrics, that can be used for good, not for evil.
It starts by having an open conversation with genuine curiosity about what really matters and asking, "Is the data a real reflection of the truth?", "Can you consistently reproduce the same results (good or bad)?" and "What decisions can we make, or actions we can take, based on these metrics?"
This workshop will explore these and other questions as well as provide a way to apply a well-known model to test for 'vanity'.
Outline/Structure of the Talk
Quick Introduction of self and topic
Connection (C1) –
Ask audience to turn to a table mate that they don’t know (ideally), introduce themselves and share an experience they’ve had or observed related to a Vanity Metrics.
Lecture / Concept (C2) portion –
Get everyone grounded on the foundational concepts of Empiricism and Continuous Improvement.
Talk about Metrics, how they have been mis-used and how they can be used for Good (Learning and Improving).
Introduce SMART model and apply to Metrics
Share some real-world examples from coaching experience.
– i.e. Client 'X' Management rolling out a set of mandated metrics for all IT/Dev teams. Well intended, but poorly communicated, it caused a wave of frustration and anxiety with the teams. Including inconsistent definitions of metrics and poor understanding of how they would use the data.
Talk about the importance of Transparency and Visualization (i.e. Information Radiators).
Before going into Exercise, ask for a few people to share what they discussed in the opening connection exercise.
Concrete Practice (C3) Core Exercise –
Provide handouts (next page) for individuals to take away.
Exercise best preformed on a flip chart, so all table participants can effectively interact and contribute.
Ask them to create a flipchart resembling the handout.
The point of the exercise is to have them identify a few common metrics and describe the characteristics as a Vanity Metric versus a Valuable Metric.
Brainstorm on Metrics that they want to explore for the exercise and to share their own experiences and examples with each other.
Check-in to make sure everyone is understanding the exercise and if they have any questions.
Complete the exercise.
Close/Conclusion (C4) –
Wrap-up the exercise with table “Shout-Out” – Ask each table to pick 1 example and share it with the group.
Encourage everyone to ID 1 key take-away that they think they can use immediately with a team that would help turn a Vanity Metric into a Valuable Metric and share that with one of their table mates as we close out the session.
Reconnecting with the concepts of Empiricism and Continuous Improvement.
Applying the SMART Goals model to identify a Vanity Metric, and how to refine it to be "smart".
Learn the characteristics of a Vanity metric.
Learn the characteristics of a Valuable metric.
Learn from others how they define and apply Metrics in their organizations.
Techniques for addressing these challenges in their organizations.
Anyone (Team member, Manager, Scrum Master, Coach ) having to deal w/ metrics in their work place.
Prerequisites for Attendees
no prerequisites. virtually everyone has some personal experience with metrics, both good and bad.