There and back again: A Journey through Agile Metrics at Scale.

Any journey is a combination of planning, expectation, experiences, conversations, and knowledge gained.  The metrics journey that this Enterprise Agile Coach and experienced Release Train Engineer have travelled has all of this and more!  And many travelers like to have a copy of their Travel Guide to hand to not only plan their experiences but to help them manage some of the intricacies of their journey.  Consider this talk your opportunity to dip into an expedition through the wilds of agile metrics – with a cheat sheet.

This presentation will cover:

  • Where am I?  Context is king.  Knowing your starting point is key.  Baseline metrics.
  • Who’s on the journey?  Knowing your audience, who else is on the journey with you and what are their expectations. Agile vs Waterfall PMO. Internal vs External metrics
  • What could go wrong?  Being clear about the challenges for metrics at scale
  • Metrics as your compass.  Using metrics to help identify changes in direction

We started our metrics journey with different goals in mind and have uncovered some surprising aspects in what works and in particular what doesnt!  In this presentation they weave their expectations, experience and knowledge to show how these perspectives can meet.

 
3 favorite thumb_down thumb_up 0 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

The RTE will deal with the deep but narrower challenges faced running a very large release trains, how to improve performance of those trains while managing expectations above, The Coach will help give that broader but necessarily more shallow viewpoint as these metrics are scaled out across the Enterprise and how they need to blend with more traditional metrics that hold large enterprises together. Our intent is to swap between us to give the different viewpoints tangible context 
  • Introduction – 5 Minutes
    - Who are we, where did we start from two years ago and what did we think the path ahead looked like.
  • Getting Context – 5 mins
    - It is crucial to understand what the point of the metrics you are gathering is used for, how it sits within the current organisation and how to gauge what the expectations are for your endevours
  • Our Journey - 15 mins 
    The crucial difference between Internal versus External metrics including the service catalog of metrics, the toolkit you are going to need. How do business performance metrics differ from cultural and customer focused metrics and the judgement calls about when to use them
  • Challenges of Metrics Travel at Scale – 5 mins
    As you continue to grow, your metrics are going to have to change as well. How do you automate this and bring it into your standard culture? Can metrics and transparency really be 'evil' and what to look out for!
  • Conclusion – 5 mins
    A summary of our experience and a cheat sheet to go away with. And finally a look ahead on what the next wave of challenge may bring.
  • QA - 5 mins

Learning Outcome

There are a number of key learnings we are looking to impart, which we have learned running a very large scale Agile program encompassing multiple Release Trains in a complex corporate environment.
 
  • The importance of contextualisation for your organisation. Taking off the shelf solutions is only going to get you so far.
  • Understanding the difference between internal and external metrics, what they are used for and the intent behind them
  • Understanding the additional challenges for metrics at scale. Running a couple of teams is easy! As you scale into the hundreds metrics take on additional complexities
  • Gaining clarity around the type of metrics required at different levels of the organisation. What is needed by your PMO is different from your sponsors and is likely very different from what the Board wants to see
  • The need to matrix metrics to balance both cultural and performance responses
  • When metrics can be used for harm; the devil isn't always in the detail. Metrics can be evil!

Target Audience

Anyone interested or involved in large scale agile delivery

schedule Submitted 1 month ago

Comments Subscribe to Comments

comment Comment on this Proposal

    • Liked Ed O'Shaughnessy
      keyboard_arrow_down

      Go forth and self organise! Easier said than done

      Ed O'Shaughnessy
      Ed O'Shaughnessy
      schedule 3 weeks ago
      Sold Out!
      45 mins
      Workshop
      Beginner

      An Agile approach encourages organisations to form self-organising teams that are empowered to figure out how to work together and deliver value. But it's one thing to tell a team they are responsible for themselves and quite another to understand what this even means and to put it into practice in a healthy, sustainable and rewarding way.

      This workshop will explore what it means to self-organise through an understanding of the fundamentals, relaying of experiences helping teams in the field develop their self-organising capability, some simple role playing to illustrate and convey techniques that anyone can easily adopt to help improve their team effectiveness. 

    • Liked Peter Lam
      keyboard_arrow_down

      Using Jurgen Appelo's Management 3.0 to build better teams

      Peter Lam
      Peter Lam
      Craig Brown
      Craig Brown
      schedule 2 months ago
      Sold Out!
      45 mins
      Talk
      Intermediate

      World of work has broken.  According to Gallup in a study conducted worldwide between 2011-2012 - only 13% of employees working for an organisation are involved in, enthusiastic about and committed to their work and workplace. 

      I'll explain why our current standard patterns in management are no longer good enough, and what we can do differently and better.  I will showcase a number of modern modern management techniques - popularised by Jurgen Appelo and Management 3.0 - that are fundamentally aligned with lean and agile principles.  These will help you get better engagement, better teamwork and make management an easier job.

      You will walk away with a number of simple techniques you can use to help initiate teams or projects; change the level of conversation that you will with your peers and team members; and give you a way to build happier teams.

    • Liked Brad Bennett
      keyboard_arrow_down

      How a Large Business Sprints Like an Agile Startup

      Brad Bennett
      Brad Bennett
      schedule 1 month ago
      Sold Out!
      45 mins
      Case Study
      Intermediate
      How a Large Business Sprints Like an Agile Startup
    • Christiane Anderson
      Christiane Anderson
      schedule 1 month ago
      Sold Out!
      90 mins
      Workshop
      Beginner

      Who procrastinates on a regular basis? Who occasionally self sabotages? Are you sure what you really want from life? Do the experiences you are having right now fulfill you?  Do you think you could be more fulfilled and if you had that, is that a goal worth obtaining? Is happiness the same as fulfillment and where does being passionate come into it? How can understanding fulfillment help us be more connected to what we do and who we work with? 

      If there was a structured approach, a formula, to guide you towards fulfillment and help you identify your gaps, would that be worth exploring? 

      Join Christiane in an interactive session to collaboratively explore what fulfillment means to you. We will break down the elements of the fulfillment formula and understand how each element plays a vital part in bringing deep, meaningful experiences to our lives. Through self reflection exercises and shared group insights participants will walk out with a clearer understanding on what fulfillment means to them. 

       

    • 45 mins
      Workshop
      Intermediate

      A diverse set of luminaries to discuss how to enhance agility in the enterprise. Covering all the big scale challenges that large organisations face, such as multiple business lines, disparate technology stacks, geographically dispersed teams, 10s to 100s of products, etc.

    • Liked Michelle Prosser-Roberts
      keyboard_arrow_down

      PI Planning Day: A Waste of Time?

      Michelle Prosser-Roberts
      Michelle Prosser-Roberts
      schedule 1 month ago
      Sold Out!
      20 mins
      Lightning Talk
      Beginner

      One principle in the Agile Manifesto talks about the importance of face-to-face conversation. SAFe takes this to the next level with PI Planning which is a routine face-to-face event where teams create their plans for the upcoming Program Increment (PI). PI planning is facilitated by the Release Train Engineer (RTE) and includes all members of the Agile Release Train (ART).

      To the non-SAFe practitioner, when they hear about the whole PI planning day palaver, they look askance and shake their heads and think to themselves "what a waste of time".  Is PI Planning a waste of time?  As part of an organisation that have had to embrace a framework to enable them to scale, PI planning is considered the heartbeat of the Agile Release Train.  In this lightening talk, I will share my experience with planning in 10-12 week increments.  Will 5 minutes be sufficient to change your mind?

    • Liked Niall McShane
      keyboard_arrow_down

      Agile framework anti-pattern risks and how to avoid them

      Niall McShane
      Niall McShane
      schedule 2 months ago
      Sold Out!
      20 mins
      Talk
      Advanced

      An anti-pattern: “something that looks like a good idea, but which backfires badly when applied”.

      My recent work implementing scaling frameworks showed me the inherent risk of using frameworks as an explicit means to affect change; any framework will always contain anti-pattern risks.

      The infographics associated with frameworks (whether SAFe, LeSS or DaD, Nexus) are always drawn with the best of intent but how they are applied is where things can run contrary to agile principles.

      Using real work examples and telling the story of a recent 18-month, 40-team experiment in scaling; Niall will highlight some of the "pot holes" to avoid when using frameworks.

      Niall will also present his approach to using frameworks including when and how to "let go" of ceremonies/practices when they become redundant.

      Lastly, a coaching approach will be presented to enable coaches to leverage the best of what frameworks offer without getting caught up in dogma.

      NOTE: this is NOT a framework-bashing presentation but a hands-on report on the risks of mis-interpretation of any principles-based scaling framework.

    • Liked Daniel Prager
      keyboard_arrow_down

      Agile in Everyday Life

      Daniel Prager
      Daniel Prager
      schedule 2 months ago
      Sold Out!
      45 mins
      Talk
      Intermediate

      Please don't assume that Agile is only good for Software Development, Startups, DevOps, Change Management, Personal Productivity, Product Development, Marketing, Sales, Time Management, Organisational Change, Educating Children, Operations, Infrastructure, Design, Teamwork and Saving your Organisation just because they are the only things that happened to be on the list.

      Applying Agile principles in everyday life is a great way to not only improve your quality of life, but to experiment with Agile approaches and find out what you've really internalised.

      For coaches and leaders practicing Agile in Everyday Life presents a rich opportunity to not just talk the talk, but also walk the walk.

      In this talk I'll give a couple of examples of how I've drawn on Agile approaches to help address personal challenges, plus discuss with the audience what areas folks are "doin' it for themselves".

      This will be an interactive session, somewhere between a talk and a short workshop.

    • Liked Phil  Gadzinski
      keyboard_arrow_down

      All your Projects are Red until you deliver something working

      Phil  Gadzinski
      Phil Gadzinski
      schedule 2 months ago
      Sold Out!
      90 mins
      Workshop
      Intermediate

      When we decide to move from big batch, plan driven long dated constructs of change, so projects and programs in the traditional sense, to more agile methods, we need to rethink a whole raft of stuff. Assume for a second, if you will, we are using the historical program and project construct to identify and deliver change. Initiatives are bundled into a sequential approach and governed through stages of largely documentation production accordingly. As long as you are on track to the stages, your documents are reviewed and signed off, you are Green.

      Then we disrupt this. We want more agility on how we identify, plan, deploy and release change. We want to deliver outputs to our customers and ultimately value or outcomes earlier and more regularly, which is one of the significant advantages of an agile delivery model. So we get an ROI uplift. Projects can be self funded.

      So lets start a new theory and trial it. EVERY project is red until your first release to a real customer. No more debate, hand wringing, angst or arguments. Recognise and accept that until you have delivered something that tests your theories, is validated and works, and your customers are using it, you have done nothing. Use the RED as it is meant to be - a call to action to pay attention, provide support , facilitate and break through barriers, provide resources and time. Help the team get to done.

      My goal is for the attendees to use this session to actually crowd source and  develop a model that people can take back to their business and propose, from scratch using design thinking and agile techniques. 

       

       

    • Liked Andrew Elms
      keyboard_arrow_down

      The Zen of Agile

      Andrew Elms
      Andrew Elms
      schedule 1 month ago
      Sold Out!
      20 mins
      Talk
      Beginner

      I used to get told to be more flexible implementing Scrum. How can I be flexible when there are all these rules and principals to be followed. It seemed like the more flexible I was the more I watered it down till nothing left. Then I gained agile-enlightenment* when I was de-programmed from the cult of buzzwords and hard rules.

      This is what I learnt…

      * Disclaimer: There is no such thing as agile-enlightenment as you will soon see…

    • Liked Andrew Elms
      keyboard_arrow_down

      5 Signs your MVP isn’t minimal and what to do about it.

      Andrew Elms
      Andrew Elms
      schedule 1 month ago
      Sold Out!
      20 mins
      Talk
      Beginner

      MVP is about learning, testing assumptions and deciding if its wise to continue working on the current product or if its better to drop it or pivot to something new.

      I see teams working on year long projects but saying they are working on an MVP. I hear about projects which once the MVP is delivered the project stops. No matter how you define it MVP is not a destination!

    • Liked Mirco Hering
      keyboard_arrow_down

      Not A Factory Anymore - What got us here won’t get us there

      Mirco Hering
      Mirco Hering
      schedule 2 months ago
      Sold Out!
      45 mins
      Talk
      Advanced

      As they say, nothing is more dangerous than using yesterday’s logic for today’s problems, yet we are still working in our organisations with mental models that were inspired by manufacturing. You can see artefacts of it even in the language we use: people are resources and they work in development factories. If we are surprised why our transformations are not progressing as fast as we hoped when Agile took the stage, then looking to these old mental models provides part of the answer.

       

      In this talk I will explain from practical experience in my work, how the old models still influence us every day and how we can break away from them and learn new models. I will give positive and negative examples from real projects to show that it is normal to experience failures and how to course correct from the lessons such failures teach us.

       

      I will also provide pragmatic steps that everyone can take in their own organisations that don’t rely on buying new tools or following specific methods. Charting your own course starts with understanding where the problem is and understanding where our mental models let us down is part of that journey.

    • Mark Barber
      Mark Barber
      Simon Bristow
      Simon Bristow
      schedule 2 months ago
      Sold Out!
      90 mins
      Case Study
      Advanced

      Lean change management is an approach to organisational improvement and transformation that uses lean startup principles to drive change. Collaboration, feedback and experimentation allow us to make small and incremental changes that we can measure and learn from and counters the big bang approach that so often makes change painful.  Embarking on a journey of discovery and change, we have adopted lean change management and applied it in ways that have created real change at MYOB. We will share our experiences to give you the tools to both assess if lean change is suitable for your organisations, and how to get started if it is.

    • Liked Benji Portwin
      keyboard_arrow_down

      How identity defines Spotify’s culture

      Benji Portwin
      Benji Portwin
      schedule 2 months ago
      Sold Out!
      45 mins
      Talk
      Beginner

      From the outside Spotify is a company who has solved the riddle of scaling agile, but the insiders perspective might surprise you, with our structure more closely resembling a collection of startups sharing a wifi connection.

    • Liked Niall McShane
      keyboard_arrow_down

      Why LeSS is the heart of Scaled Agile

      Niall McShane
      Niall McShane
      schedule 2 months ago
      Sold Out!
      20 mins
      Talk
      Intermediate

      Over the last 2 years Niall has coached teams of agile teams through the launch of two Agile Release Trains (SAFe). Niall helped leaders, managers and teams through the process of adopting SAFe.

      Using his real experience as the basis for the presentation, Niall will make the case for moving past SAFe and going to the heart of scaled Agile using Large-Scale Scrum (LeSS).