Using Metrics for Good not Evil or: How I learned to Stop Worrying and Love the KPIs

schedule Sep 23rd 02:00 - 02:45 PM place Room 4 people 27 Interested

Using metrics for punitive reasons is a problem as old as time. In software, this is further complicated by the fact that people rarely agree on why we are collecting metrics in the first place. In this session we will explore how we can use metrics for good instead of evil.

By focusing on the goal of system improvement, rather than individual performance, we can begin leveraging data to make a positive difference in how we work while also delving into why we work the way we do.

This session will include real-world examples of problems that organizations create for themselves by using metrics for the wrong intent. We will also discuss examples of good metrics and how they can be used to make our lives better.

 
 

Outline/Structure of the Workshop/Game

Defining the Problem

  • Why capture metrics at all?
  • Defining KPIs, OKRs and other related acronyms
  • Understanding true, underlying, organizational drivers

Good Intentions Driving Bad Behaviors

  • Team-Level (Punitive Measures)
  • Program-Level (Obfuscation Measures)
  • Portfolio-Level (Misguided Measures)

Visible Outcomes Driving Good Behaviors

  • Team-Level (Sanity Measures)
  • Program-Level (Predictability Measures)
  • Portfolio-Level (Investment Measures)

Creating the Environment and Inertia for Change

  • A five-step guide to getting leadership buy-in
  • Creating the environment for change
  • Thinking strategically about how we collect and leverage data for good

Wrap Up & Questions

Learning Outcome

  • Know why we need good metrics
  • Describe why organizations, often unintentionally, capture harmful metrics
  • Explain how good metrics can drive good behaviors and how bad metrics can drive bad behaviors
  • Understand how to identify good metrics at each tier of business
  • Able to analyze an organization’s current metrics and determine whether they are being used for good or for evil
  • Comprehend how to continuously improve the behaviors we see in our organization by continuously improving the metrics we use to measure success

Target Audience

Anyone that wants to learn how to employ metrics and measures across their organization, in a meaningful and responsible manner.

Prerequisites for Attendees

There are no prerequisites for this session.

schedule Submitted 3 months ago

Public Feedback

comment Suggest improvements to the Speaker

  • 45 Mins
    Talk
    Beginner

    Have you tidied up your personal life with Marie Kondo and are now wondering how to achieve the same effect in your work life? Do you have the feeling that the most valuable product backlog items (PBIs) are getting lost under a mountain of old stories, bugs, and tasks? Maybe you know a change is needed, but feel completely overwhelmed about where to start? If so, join us to learn how to make your product better through the life changing magic of tidying up your backlog.

    We’ll start by exploring the costs of a large, cluttered product backlog and share a short quiz you can use to gauge the current state of your own backlog. Next, we’ll cover how we’ve adapted the KonMari method and introduce five easy steps you can take to get started in your tidying process. We'll share real-life examples along the way, calling out potential pitfalls to avoid (don’t become a storage expert!), and illustrating how story mapping may be the magical backlog equivalent to Kondo’s “vertical folding” technique. By the end of the session, you’ll know the specific next steps to take so that you too can realize the many benefits of a tidied-up product backlog: improved visibility, increased self-organization, and easier decision-making.

  • Liked Mark Grove
    keyboard_arrow_down

    Mark Grove / Julie Wyman - What’s REALLY Going On? An Observational Skills Workshop

    45 Mins
    Workshop/Game
    Beginner

    Imagine you are asked to sit in on a team’s sprint review and retrospective. The team has been having difficulty forming and the Scrum Master has asked you to observe the team dynamics during these two sessions. Are you simply going to watch what’s going on or is there more you can do? Perhaps you are seeing interactions and team dynamics at play without truly realizing what you are observing. And when you do observe, are you injecting your own biases into those observations? Observation is a powerful tool, but one which we may not take advantage of to its true potential. After all, what exactly should we be observing, anyway?

    By learning how to expand our observational skills in a non-biased and non-judgmental manner, we can gain a deeper understanding of team dynamics and interactions allowing us to offer more meaningful and impactful support, coaching, and empathy. Because there are many observational aspects that pass us by, the best way to become more observant is through deliberate practice. So, let’s practice together with a group exercise in a fun and safe setting!

    In this highly interactive workshop, we’ll start by sharing tools and tips to make you a better observer. Then we’ll ask for a small group of volunteers (“builders”) to be observed performing a brief task. The remaining attendees will practice applying the observation techniques, and, after the builders finish, will share their observations in small groups. We’ll conclude with a full-group debrief and discussion of the key takeaways and opportunities to improve our effectiveness and observations.

    If you’re looking for new ways to connect with your team, to enhance your agilist toolkit, or simply participate in an informative and interactive workshop, this session is for you!

  • Liked Mathias Eifert
    keyboard_arrow_down

    Mathias Eifert - Complexity is the Enemy! How Agile Practices Allow Us to Operate in a VUCA World

    45 Mins
    Talk
    Intermediate

    One of the key advantages of Agile over plan-driven approaches is that an Agile mindset acknowledges our ever-diminishing ability to usefully predict the future and focuses our efforts on managing change instead of trying to suppress it. This “new reality” has become pervasive enough to drive its own buzz word – VUCA, which stands for Volatility, Uncertainty, Complexity, Ambiguity. But beyond the hype lies a truth that Agile leaders need to understand and embrace – that certain problems really do respond differently to our attempts to manage and solve them. Why does this matter? Because problem contexts that defy straightforward cause-and-effect expectations significantly impact productivity while simultaneously presenting much higher risks to success. Even worse, applying leadership approaches that aren’t matched to the problem context dramatically increases the danger of catastrophic failure.

    In this session, we’ll examine how the Cynefin framework helps us make sense of what kinds of problems we’re dealing with and how we should approach them. We will then look at ten ways in which Agile frameworks, approaches and technical practices help us manage or even reduce complexity and one where they fall short. You will walk away with a deeper understanding of how - and why - the things we do as agilists increase stability and reduce risks for our teams.

  • Liked Mindy Bohannon
    keyboard_arrow_down

    Mindy Bohannon - I’m a BA Girl in an Agile World

    Mindy Bohannon
    Mindy Bohannon
    Agile Business Analyst
    Excella
    schedule 2 months ago
    Sold Out!
    45 Mins
    Experience Report
    Beginner

    Have you ever worked with a Business Analyst (BA)? Is what a BA does on an agile project much different from what is done on a waterfall project? All analysts bring excellent communication, collaboration, and trust to their work on project teams. During this session we’ll review the roles a BA can play, a BA's responsibility on the development team, and the skills a good BA possesses. For fun, lets also talk about why an Analyst is part of the 3 Amigos and the complexity of communication channels. Generally speaking, let’s discuss how BAs participate in an agile project’s success and I’ll share some stories about my experience going from waterfall to agile, how I’ve interacted with the PO, and important things I think an Analyst should be involved in.

  • Liked Derek Huether
    keyboard_arrow_down

    Derek Huether - 10 Steps to Better Outcomes by Using Metrics

    45 Mins
    Talk
    Beginner

    This session is not intended to offer an exhaustible list of metrics or instructions on how to improve all systems. Rather, the intent is to provide a framework on how to ensure the quantitative and qualitative metrics you use are measuring the right things and how to apply them to a system of continuous improvement. Attendees will have a repeatable framework they can apply after leaving the session.

    The session is broken into two main parts.

    • Part 1: Identify the right (quantitative and qualitative) metrics that will help people and teams meet outcomes or goals.
    • Part 2: Create a cycle of learning and improvement that aligns people and teams to the outcomes or goals.
  • Liked Katrina L. C. Tanner
    keyboard_arrow_down

    Katrina L. C. Tanner - Journey from Analyst to Product Owner to Scrum Master in less than a year

    Katrina L. C. Tanner
    Katrina L. C. Tanner
    Scrum Master
    Cvent
    schedule 3 months ago
    Sold Out!
    10 Mins
    Lightning Talk
    Beginner

    Not many people can say they have performed as an Analyst, Product Owner, and Scrum Master, much less in the same year, and sometimes at the same time.That has been my journey throughout 2018 and into 2019 thus far, as I made intentional decisions in pursuing my next best step, exploring where I really fit, and experiencing quite a few surprises along the way.

    As a CSM and CSPO, I navigated each role taking what I learned in the classroom and implementing it as a practitioner. Being very team AND operations oriented, I have always thrived when one of these were included in my job description. 3 companies, 3 roles, and 2 organizational restructures later, I have found a career that involves both. I learned what works and doesn't work for me and my teams, as well as, what I like and dislike compared to my strengths and weaknesses, especially among changes in organizational needs and structure.

    This short and sweet talk will be an overview of my journey and leave attendees with characteristics of each role to help them decide where they might fit in this agile world and tips on how to talk to your boss about your next step.

  • Liked Raj Indugula
    keyboard_arrow_down

    Raj Indugula / George Lively - Being Test-driven: It's Not Really About Testing

    45 Mins
    Talk
    Beginner

    Good news: Test-driven practices have jumped the chasm to general acceptance! The bad news, though, is that while TDD, BDD, and ATDD are prominent buzzwords in the industry today, they are rife with misconceptions, with many people incorrectly assuming that being test-driven is all about testing.

    In this talk, learners will leave with a clearer understanding of Test-Driven Development (TDD), Behavior Driven-Development (BDD), and Acceptance Test-Driven Development (ATDD), and gain practical insights into how these practices can help teams develop better software. We will gain an appreciation for TDD as being primarily a specification and design technique, and how to get the whole team involved earlier in the delivery cycle using a BDD approach.

  • Liked Paul Boos
    keyboard_arrow_down

    Paul Boos - Agile Leadership 201: Enriching Management

    Paul Boos
    Paul Boos
    IT Executive Coach
    Excella
    schedule 4 months ago
    Sold Out!
    45 Mins
    Tutorial
    Executive

    The Agile Manifesto doesn't explicitly talk about what changes in management should happen and neither do the approaches. In fact, sometimes we hear the exact opposite from teams - "What do we need managers for..?" or perhaps "Can't they just get rid of all the impediments we have?"

    As a former manager and now as a coach, I find the words Servant Leadership sometimes doesn't resonate. It actually only paints part of the picture anyway. What we want are ways to enrich management so that they can do more for the organization and its teams. Let's discover what some of this enrichment might be.

  • 45 Mins
    Workshop/Game
    Beginner

    How do you get people to agree on priorities when they may have different objectives? You may be facing an issue now where stakeholders are pushing against each other in order to get their work done first. What would happen if we could create an open dialog among stakeholders and have them understand different perspectives and focus on the goals of the greater good instead of just their own? Let’s face it, proper prioritization is the difference between writing code and developing valuable solutions.

    In this simulation style workshop, you’ll learn practical methods for bringing stakeholders together and openly discuss their different priorities to agree on what’s most important overall. You will see first hand how a combining group discussion with proven prioritization methods such as Weighted-Shortest Job First and (WSJF) and Must Have, Should Have, Could Have and Won’t Have (MoSCoW) work.

  • Liked Nicole Spence-Goon
    keyboard_arrow_down

    Nicole Spence-Goon - Peaks or Valleys? The power of Scaling Agility in the alphabet soup of Government agencies

    Nicole Spence-Goon
    Nicole Spence-Goon
    Agile Coach
    Excella
    schedule 2 months ago
    Sold Out!
    45 Mins
    Experience Report
    Beginner

    Do you feel like Agile Scaling has become a goal rather than the means to an end for your organization? To determine where you stand on the Scaling spectrum, ask yourself a few soul-searching questions: Why do we need to scale? Is this the right time for us to scale? If you’ve checked these boxes, you may wonder “where do I go from here?”

    This talk will focus on 3 areas that emerged as common themes throughout my experience working on government Agile Scaling projects and ultimately influenced the trajectory of each agency's scaling journey:

    • Communicate vision consistently
    • Focus on your people genuinely
    • Create your own path intentionally

    I've seen successes and some struggles with Agile Scaling efforts in government agencies. Regardless of the agency acronym or the frameworks used, these key elements shaped their scaling outcomes.

  • Liked Zack Ayers
    keyboard_arrow_down

    Zack Ayers / Joshua Cohen - Andon Cords in Development Teams: Our Experience of Driving Continuous Learning through a Culture of Experimentation

    45 Mins
    Talk
    Beginner

    Summary

    In this session, you’ll learn about one team’s struggle to improve collaboration and how they sought to shorten cycle time by carefully crafting an experiment with an Andon Cord. The Andon Cord is a Toyota innovation designed to empower front-line employees to recognize issues, initiate a stoppage of work, and work together as a team to quickly identify a path forward. The emergency cable strung above assembly lines became a symbol of the Toyota Way, and has widely been copied throughout the auto industry and beyond.

    You’ll be introduced to metrics that show a surprising correlation between collaboration through Andon Cord pulls and Cycle Time!

  • Liked Itopa Sulé
    keyboard_arrow_down

    Itopa Sulé - What NOT to do when prioritization fails

    Itopa Sulé
    Itopa Sulé
    Lead Agile Coach, Canada
    Capital One
    schedule 3 months ago
    Sold Out!
    45 Mins
    Workshop/Game
    Intermediate

    One common mistake by new Agilists is to think that the product backlog is a linear list with the highest priority item on the top. As it's often the case, the reality is different. Prioritization is neither binary nor linear. Your team will be asked to work on several competing priorities, like meeting the new regulation while working on the next product milestone (the product is the company lifeline) - and your product owner will insist these are both top priority!!

    In this workshop we are going to explore different tools, to ensure that our teams are working on the highest value backlog items: from Eisenhower Method to Cost of Delay, CD3 and modified Weighted Shortest Job First (WSJF).

  • 45 Mins
    Talk
    Beginner

    On average Agile transformations get into trouble at about 14 months from the time they start. Have you ever wondered why?

    Is it because you started from 'the bottom up?" Or is it because you spend time, money, resources and people creating great teams and didn't pay attention to having a professional, empowered Product Owner organization?

    Agile transformations that target department level and not the organization have a very low success rate and you end up with an agile department enveloped by a traditional organization. That leads to dark work, decision delay and dysfunction.

    So, what can you do?

    Come and learn some of what I learned the "hard way" over 7 transformations.

  • Liked Dane Weber
    keyboard_arrow_down

    Dane Weber - Undercover Scrum Master

    Dane Weber
    Dane Weber
    Lead Consultant
    Excella
    schedule 2 months ago
    Sold Out!
    45 Mins
    Experience Report
    Intermediate

    After three years as a Scrum Master and Agile coach, I hit a wall coaching a team that did not want to try popular Agile engineering techniques such as TDD and pair programming. I had become a Scrum Master after four years working on the business analysis and account ownership side of things and could not speak from personal experience about engineering practices. In order to get some first-hand experience and to gain a new perspective, I chose to spend a year or two as a software developer on a Scrum team.

    The experience has been eye-opening. I experienced a tremendous cognitive load working with a wide array of technologies; this pulled my attention away from many of the collaborative and process-oriented activities I cared about as a Scrum Master. I was surprised to feel strong pressure to complete work quickly, cutting corners, even when the Product Owner and Scrum Master were not asking me to. When this pressure was explicit, it usually came from my fellow developers. On the other hand, there is real joy in writing code and seeing a system do something worthwhile that it wasn't doing before. My outlook has changed tremendously and is something I want to share with anyone who works with development teams, especially Scrum Masters and other coaches. I am still enjoying my time as a developer, but I'm looking forward to returning to coaching and incorporating this experience into my approach.

  • Liked Katrina L. C. Tanner
    keyboard_arrow_down

    Katrina L. C. Tanner - Playdoh...play on!

    Katrina L. C. Tanner
    Katrina L. C. Tanner
    Scrum Master
    Cvent
    schedule 3 months ago
    Sold Out!
    10 Mins
    Lightning Talk
    Beginner

    What happens when 25 product owners, developers, and quality engineers reflect on their previous sprint through the eyes of a child?

    Playdoh took my teams, remote and collocated, from showing up to an obligatory meeting on their calendar to intentional interpersonal and intrapersonal dialogue.

    Between the carpet and the sticky fingers, adults are usually cleaning up more Playdoh than playing with it themselves. Even to my own surprise, in this “Play Retro,” our team members connected, not only on a professional, but also on a personal level in ways they had not done before.

    This session will give an overview of our experience using a “Play Retro” and leave you with tips and things to consider planning a retrospective.

  • Liked Mark Grove
    keyboard_arrow_down

    Mark Grove - Authority and Trust: Finding the Scrum Master Balance

    Mark Grove
    Mark Grove
    Excella Consulting
    schedule 3 months ago
    Sold Out!
    45 Mins
    Talk
    Beginner

    Have you ever seen the term “process authority” used to describe the role of a Scrum Master? You don’t have to look too hard to find this description. In my experience, it has caused a lot of confusion and misunderstanding on project teams. So why is it even used? And should it?

    I worked on a new Scrum project where management informed the project teams that the Scrum Master role was given process authority over the team. There was a lot of confusion and unease about this. Questions started popping up such as “I already have a manager – is the Scrum Master now my boss,” “What does process authority mean,” and “Do we really have to do what she says?” As a result, the teams were becoming hostile to the Scrum Master role, untrusting of the Scrum Master’s responsibility, and concerned Scrum Masters had authority to tell the teams what to do. I was the project’s agile coach and needed to diffuse the confusion and better the working environment as quickly as possible. This did not happen overnight, but our journey started with this interactive discussion which we used as a foundation for the project moving forward.

    In this interactive presentation, we explore the concept of “process authority” and consider the various directions it takes us. To do that, this discussion goes far beyond a typical “the role of a Scrum Master” presentation; It explores…

    • What it should (and should not) mean when/if “process authority” is used to describe the Scrum Master role
    • How the responsibility and expectations of a Scrum Master are different than that of team members
    • How different leadership styles play into understanding the role of the Scrum Master
    • The importance of trust in a Scrum Master/team relationship

    The presentation uses real-time audience feedback to further explore these topics. Audience members will provide answers to questions given throughout the presentation, so we can explore members’ thoughts, opinions, and experiences they have had with the Scrum Master role.

  • Liked Dr. Patrick McConnell
    keyboard_arrow_down

    Dr. Patrick McConnell - The Velocity Trap: Learning to Value Enterprise Outcomes over Team Output

    45 Mins
    Tutorial
    Beginner

    Behind the creation of every Agile Framework was the intention to improve the Return on Investment for creative work, primarily through faster and richer feedback. But as team-level frameworks like Scrum are internalized by large organizations, that message gets mistranslated. Instead of, “Get better outcomes, sooner,” the drive instead becomes, “Just do more, faster.”

    A common expression of this problem is the confusion of Velocity for Value, where teams are directly managed based on their Output, but the connection between Output and Outcome is lost or ignored. This plays out in all kinds of ways that distract from achieving tangible results, and often incentivizes internal competition over collaboration. This problem can be especially tenacious in organizations with significant institutional ‘status-ing’ behaviors, where leadership struggles to translate common Agile methods like Relative Estimating into existing artifacts like Executive Dashboards or Earned-Value Management.

    In this tutorial, participants will explore the 'Velocity Trap', and will be shown how to setup a results-driven framework that prioritizes 'Maximizing Outcomes while Minimizing Output.'

  • Liked Toby V. Rao
    keyboard_arrow_down

    Toby V. Rao - Why is my Agile Talent leaving? How can I stop that?

    45 Mins
    Talk
    Intermediate

    More and more organizations are embracing Agility and are transforming. According to 13th State of Agile Report - as many as 97% of organizations state that they practice agile development methods. This rapid change and the strong US economy has resulted in a surge in demand for talented Agile Coaches, Developers, Scrum Masters, DevOps Engineers, Full Stack Developers etc. The problem is that the demand far exceeds the supply of good agile talent. This imbalance has resulted in turbulence in the job market and consequently causing a heavy movement of agile talent.

    If your organization is affected by the movement of agile talent, you should pay attention!

    • How is your organization reacting to these changes?
    • Want to understand why agile talent make the choice to stay or leave organizations?
    • Would you like to learn about 5 pragmatic actions you can take immediately to help reduce attrition?

    Join Toby Rao for an interactive discussion that will not only change your paradigm but also help you and your organization to take simple steps to control attrition.

  • Liked Mindy Bohannon
    keyboard_arrow_down

    Mindy Bohannon - GiveCamp Case Study: Agile and Scrum at the Speed of Light

    Mindy Bohannon
    Mindy Bohannon
    Agile Business Analyst
    Excella
    schedule 2 months ago
    Sold Out!
    45 Mins
    Experience Report
    Beginner

    Cleveland’s GiveCamp is the most magical, exhausting, and worthwhile 48 hours I’ve ever experienced professionally. In 2015, 200 techies volunteered their talents and passion to help 19 non-profits with projects including data collection, websites, GPS-enabled mobile app, an educational online game, and database applications. How does so much work get done in such little amount of time? Agile in hyperdrive – standups every 4 hours, adhoc swarming with heavy hitter devs roaming the projects, and lots of free food and drinks to keep everyone fortified. This case study reviews how so much work was done in such a short amount of time.

  • Liked Tyler Grant
    keyboard_arrow_down

    Tyler Grant - Words Matter: Building Trust and Dropping Jargon to Create Stronger Teams

    Tyler Grant
    Tyler Grant
    Scrum Master
    Capital One
    schedule 4 months ago
    Sold Out!
    45 Mins
    Talk
    Beginner

    Within the Agile and technology communities, it can be easy to assume that the language we use every day is common knowledge and shared throughout our professional spaces. Certainly when we interact with other professionals and hear them using the same terms, this just reinforces our assumptions. Especially when we leave the boundaries of technically specific language and enter the realm of jargon or “buzzwords” it can be easy to find validation through other people’s language preferences.

    However, using this type of language can not only create divisions within our teams, it can obfuscate meaning, delay action, and create misunderstandings that can inhibit a team's ability to deliver. By examining the differences between jargon and technical language, we will look at how this use of language can create in- and out- groups and break down trust within an organization.

    By committing to using broadly understood language and being direct in the way we interact with each other, we can improve our ability to perform at a high level in the fast paced technology space.