Mirror, Mirror on the wall, what are the worst Vanity Metrics of them all?

Let’s face it, Metrics are a staple of virtually every IT organization.

Unfortunately, they are often poorly understood and horribly misused.

But what are Metrics anyway? It’s just Data, right? Specifically, it’s empirical data. Learning from, and basing our decision-making on, empirical data is a good thing.

The intent behind Metrics is to improve. A very Agile idea indeed. Ironically, rampant misuse tends to create more waste than value. Teams often don't understand or believe in them. Managers often misinterpret them or never really use them. NOT a very Agile idea.

We can change this. It's not just a Fairy Tale.

We can empower teams to take ownership of their data, including defining what data really helps them improve. We can educate management to better understand what to ask for and how to properly interpret the data. Thus, turning Vanity metrics into Valuable metrics, that can be used for good, not for evil.

It starts by having an open conversation with genuine curiosity about what really matters and asking, "Is the data a real reflection of the truth?", "Can you consistently reproduce the same results (good or bad)?" and "What decisions can we make, or actions we can take, based on these metrics?" This workshop will explore these and other questions as well as provide a way to apply a well-known model to test for 'vanity'.


Outline/Structure of the Workshop

Quick Introduction of self and topic

Connection (C1) –

<5-10 minutes>

boxAsk audience to turn to a table mate that they don’t know (ideally), introduce themselves and share an experience they’ve had or observed related to a Vanity Metrics. Ask them to right down what the other person shares with them and see if it resonates with their own experiences.

Lecture / Concept (C2) portion –

<20-25 minutes> . *Not as one single lecture time. Will provide about 20 minutes of initial lecture and add 5 more minutes in the middle of the exercise to amplify a few key points and layer in a little more information.

Get everyone grounded on the foundational concepts of Empiricism and Continuous Improvement.

Talk about Metrics, how they have been mis-used and how they can be used for Good (Learning and Improving).

Share some real-world examples from coaching experience.

– i.e. Client 'X' Management rolling out a set of mandated metrics for all IT/Dev teams. Well intended, but poorly communicated, it caused a wave of frustration and anxiety with the teams. Including inconsistent definitions of metrics and poor understanding of how they would use the data.

Talk about the importance of Transparency and Visualization (i.e. Information Radiators).

Introduce SMART model and apply to Metrics

Before going into Exercise, ask for a few people to share what they discussed in the opening connection exercise.

* In middle of exercise, will add additional information for them to think about and/or amplify a few key points (see below)

Concrete Practice (C3) Core Exercise –

<40-45 minutes> depending size of the audience

The point of the exercise is to have them identify a few common metrics and describe the characteristics as a Vanity Metric versus a Valuable Metric.

Provide handouts for individuals to take away.

Exercise best preformed on a flip chart, so all table participants can effectively interact and contribute.

Ask them to create a flipchart resembling the handout.

Give them 13-15 minutes to brainstorm on Metrics that they want to explore for the exercise and to share their own experiences and examples with each other.

Check-in to make sure everyone is understanding the exercise and if they have any questions.

*Add additional information, talk about the connection between Metrics and 'Individuals and Interactions of Processes and Tools', talk about this as a Cultural issue, including the difficulty in changing people's thinking and behaviors relating to Metrics / Empirical Data. 5 minutes.

Give them 13-15 more minutes to complete the exercise

Wrap-up the exercise with table “Shout-Out” – Ask each table to pick 1 example and share it with the group. 13-15 minutes.

Close/Conclusion (C4) –

<5 - 10 minutes>

Encourage everyone to ID 1 key take-away that they think they can use immediately with a team or manager that would help turn a Vanity Metric into a Valuable Metric and share that with one of their table mates.

Open up for Q&A to close out the session, depending on time.

Learning Outcome

  • Reconnecting with the concepts of Empiricism and Continuous Improvement.
  • Applying the SMART Goals model to identify a Vanity Metric, and how to refine it to be "smart".
  • Learn the characteristics of a Vanity metric.
  • Learn the characteristics of a Valuable metric.
  • Learn from others how they define and apply Metrics in their organizations.
  • Techniques for addressing these challenges in their organizations.

Target Audience

Scrum Masters, Managers, Project Managers, Team Members... anyone who is challenged by, but responsible for Metrics.

Prerequisites for Attendees

Participants should have a basic level of experience with Metrics, whether defining them, tracking them, or using them in the context of a team or project.

Participants should be prepared to bring an example(s) of a Metrics that they have to deal to share with the group and use in the workshop.

schedule Submitted 4 years ago

  • Melissa Boggs

    Melissa Boggs - Exhaustion is Not a Status Symbol

    Melissa Boggs
    Melissa Boggs
    VP, Business Agility
    Sauce Labs
    schedule 4 years ago
    Sold Out!
    60 Mins

    In her book, The Gifts of Imperfection: Let Go of Who You Think You're Supposed to Be and Embrace Who You Are, Brene Brown shares her 10 Guideposts of Wholehearted Living. Number 7 on that list is “Cultivating Play and Rest: Letting Go of Exhaustion as a Status Symbol and Productivity as Self-Worth”. This resonates strongly with the 8th agile principle about sustainable pace.

    In the world of Scrum software development, it is all too easy to get caught up in pumping out user stories and increasing velocity sprint after sprint, but what does that type of hamster wheel mentality do to us physically, mentally, and spiritually? For that matter, what impact does it have on our products? Are we building fast things, or the right things? Are we making time to dream up big, new ideas and/or to build a cohesive team around our mission?

    Work-life balance is a buzzword that we throw around, but how often does the culture of an organization support exactly the opposite? Hero culture is rewarded, and our output viewed as a measure of our worth on performance reviews. We set out to transform the world of work with agile and with Scrum, yet I’ve heard the Scrum sprint cycle described as a “hamster wheel”, an endless conveyor belt of backlog and sprint reviews that the developers cannot escape. This is not congruent with what we read in the agile values and principles.

    I’m interested in inspiring a discussion about the pitfalls of a competitive exhausted culture, and how we in the Scrum community, even with the best of intentions, could be “accidentally responsible” for continuing to spin the hamster wheel. Hero culture has been discussed before, but have we addressed our own potential culpability in creating it? We need to make sure that the principles and practices of Scrum are being used for good, not for evil. It all starts with a conversation.

    In this discussion, we will explore the dangers of exhaustion as a status symbol -- for our organizational culture, our teams, and ourselves. We will discuss the specific risks of inadvertently creating a competitive exhausted culture within an agile transformation, and the ways in which we can leverage the agile values and principles in order to mitigate those risks. Lastly, we will take a look inward to assess our own attitudes and views about work life balance.

  • Sunny Dhillon

    Sunny Dhillon / Hugo Medeiros - Spice up your scrum with improv!!

    60 Mins

    Have you ever been part of a great team? A team where you loved to come to work every day, a team that encouraged to accomplish goals that you felt were nearly impossible?

    Have you ever been on a team from hell? A team with constant conflict, disagreements and fear of speaking the truth?

    In his book, “The Five Dysfunctions of a Team”, Peter Lencioni refers to trust as a foundational and key ingredient to high performing teams. Exercises in this workshop will allow individuals to become more self aware and increase their vulnerability to build a stronger bond of trust.

    Many principles that actors live by are the same principles that a high performing agile team should live by as well. The session is all about focusing and strengthening those key skills needed by agile teams.

    Many of us think we can’t do improv. We get stuck in our head and panic, afraid we might not know what to do or say. This workshop involves highly interactive group activities that are fun and fast paced designed to help communicate effectively and think on your feet. Games are intentionally ordered to focus first on creating safety for the groups before advancing to more complex topics as collaboration and risk taking.

    Improv will make you think about your team, your organisation and yourself differently, in the context of your agile environment.

  • Richard Dolman

    Richard Dolman - STATIK (be)FOR(e) SCRUM: Start your Scrum Teams off with Kanban - It's not just a "blue pill or red pill" choice

    90 Mins

    STATIK (Systems Thinking Approach To Introducing Kanban) can be a great way to help Scrum teams get up and running quickly. “Heresy!” you say.
    But it doesn’t have to be just one or the other, a blue pill or red pill decision.

    This is not the same thing as “Scrum-ban”.
    This is “Start with what you do now” in order to help avoid over-thinking how existing work, structures and/or roles “fit” into Scrum.
    Specifically, applying this approach designed for Kanban can dramatically accelerate the forming and improvements of a Scrum team.

    This session will provide an understanding of STATIK and how to apply it effectively as part of a new Team Chartering workshop for a new Scrum team. We have used this approach to help teams Form and establish Norms for them to work toward, including how they will norm around Scrum.

    … and tomorrow "you will wake up in your bed and believe whatever you want to believe."

  • Sunny Dhillon

    Sunny Dhillon / Daniel M Lynn - Powerful Observational Techniques For Coaching Teams

    90 Mins

    Starting out as a new agile coach is difficult. Where do you go? How do you start? Learn to leverage a coaching approach that focuses on observations and to form insights and goals.

    You will uncover a structured approach to coaching which aims at improving team performance by providing clear guidance and structure to the coaches and scrum masters. Through a structured approach, coaches and scrum masters will be able to better target their efforts and create demonstrable improvement in teams.

    A structured coaching approach can be applied to, but is not limited to:

    • Agile, Lean, Scrum project practices: planning, tracking, improving.
    • Agile technical practices testing
    • Product management, business analysis, and Product Owner role
    • Scrum Master role (team leadership and project management)
    • Management and Leadership principles, skills, and practices
    • Effective agile process mechanics: iteration planning, reviews, retrospectives, daily stand-up
    • Building effective self-organized and empowered product teams
    • Evaluation of tools to assist in Agile Lean practices

    After an introduction to effective observation techniques, groups will be asked to collect observations from various scenarios drawn from real-life experience with teams and organizations.. Groups will then collaborate with others and draw insights from common behaviors and trends. From this, we will leverage the Coaching Card technique to plan possible coaching paths forward and identify ways that progress could be validated and demonstrated in practice.

    Attendees will leave this session with a structured approach to guide their ongoing coaching efforts and share those experiences with others in the organization.