Product Usage Metrics - Practical Lessons from the Trenches

location_city Virtual schedule Oct 14th 04:55 - 05:15 PM place Online 2 people 4 Interested
  1. How do I motivate my developers when I cannot show them the financial value of the features they build?

  2. How do I prioritize which production defect to fix first?

  3. How do I choose the right solution for the customer when I have multiple solutions to solve the problem?

  4. How do I decide which feature to enhance first, using real data instead of personal intuition?

If any of these questions keep you awake at night, this session is for you.

In this interactive session, I will share practical lessons from great product development teams. You can hear from me about how simply creating an intentional product usage metric strategy can help in avoiding many conflicts.

 
 

Outline/Structure of the Experience Report

  1. Introduction - 2 min
  2. Case - 1: Quick Story - How product usage metrics solved prioritization problems - 3 min
  3. Case - 2: Quick Story - How product usage metrics helped motivate developers - 3 min
  4. Case - 3: Quick Story - How product usage metrics helped in choosing right solution - 3 min
  5. Summary - 2 min
  6. Q&A - 7 min

Learning Outcome

After the talk, as a participant,

  1. I will be able to understand feature usages which will help me prioritize bugs
  2. I will able to identify unused (or used very less) features and remove those to improve the application performance
  3. I will be able to experiment with different solutions.

Target Audience

Developers/Testers/Product Owners/Managers

Prerequisites for Attendees

Curiosity

schedule Submitted 1 year ago

Public Feedback

comment Suggest improvements to the Author
  • Naresh Jain
    By Naresh Jain  ~  8 months ago
    reply Reply

    Hi Kiran,

    Can you please provide more details around which specific product usage metrics you plan to cover? Do you have any samples you can share?

    Where can I read more about your experience?

    • Kiran Kashyap
      By Kiran Kashyap  ~  8 months ago
      reply Reply

      Hi Naresh,

      Here are my responses, as promised.

      Question:

      Where can I read more about your experience?

      Response:

      Quick Story - 1:

      • The team: PO, developers and Scrum master.
      • Product owner was unsure about how to prioritize when there were multiple user stories having same business value and urgency.
      • Decided a mechanism to log product usage in a table for future so that they can prioritize enhancements on most used features first.
      • Product Usage Metrics logging started.
      • Over a period of time, PO used data from this table to prioritize enhancements in more creative ways than earlier intended. (Like prioritizing highest paying customer used feature first, some other time, prioritizing for customer whose subscription is going to expire in current month) - See Graphs in next section

      Quick Story - 2: (Continued from story 1)

      • Developers expressed happiness when a customer told them how much time he saved because of a new feature.
      • Thought we could aggregate and display this data (time saved x no. of times a feature is used) in a live dashboard to motivate developers.
      • Did it. Developers now look forward to know how they impacted the customers.

      Quick Story - 3: (Continued from Story 2)

      • When the team had multiple solutions to go for an enhancement, they decided to perform A/B testing.
      • Put all the solutions into beta env and tracked usage metrics alongside bug statistics. 
      • Correlated bug data with usage metrics to see which solution was used most.

      Question:

      Can you please provide more details around which specific product usage metrics you plan to cover?

      Response:

      The product usage metrics I am going to cover are:

      1. No. of Hits Per Feature Per Period (and variations of the same)
      2. Time Saved Per Feature per release  (To motivate developers)
      3. Bugs per 100 hits per solution (For A/B testing)

      Question:

      Do you have any samples you can share?

      Response:

      I have the following handmade graphs. I will demo Grafana dashboards in my talk.

      Please let me know in case i need to change/add/delete something.

      Thank you.

      Regards,
      Kiran

    • Kiran Kashyap
      By Kiran Kashyap  ~  8 months ago
      reply Reply

      Hi Naresh,

      Thanks for going through my proposal.  Can you please give me a day's time so that I can come up with a small writeup with points that answer your questions?

      Thank you.

      Regards,
      Kiran

  • Aino Corry
    By Aino Corry  ~  11 months ago
    reply Reply

    Hi Kiran

    I am worried about the ambition for this 20 mins talk. Is there time for an interactive session as you propose? Is there time for 3 learning outcomes?

    • Kiran Kashyap
      By Kiran Kashyap  ~  11 months ago
      reply Reply

      Hi Aino,

      Thanks for taking a look at my proposal and letting me know your feedback.

      I am going to share 3 quick stories, 3 minutes each. 

      I confess the word 'interactive' is a bit of a stretch :)  I can reword it or remove it if you suggest.  There are two places where interaction with the audience will happen:

      1. As a intro to the topic, i plan to ask a question, for which audience can respond and i can take 2 or 3 responses.

      2. Q&A slot at the end

      To answer your question about time for learning outcomes, yes, each story will deliver a learning outcome.

      Please let me know if this looks okay or needs some modification.

      Thank you.

      Regards,
      Kiran


  • Kiran Kashyap
    keyboard_arrow_down

    Kiran Kashyap - Behavior Driven Development in Stealth Mode - An Exploration

    Kiran Kashyap
    Kiran Kashyap
    Agile Change Agent
    Lowe's
    schedule 1 year ago
    Sold Out!
    45 Mins
    Talk
    Beginner

    “This is not what I wanted” - Is this a repeated frustration of your product owner? Do you wonder if Behavior Driven Development can help your team in specifying requirements better? Yet, is budget a constraint for starting BDD as a practice? If yes, this session is for you.

    This interactive talk will take you through practical scenarios which play out in a team, while you role-play as a developer/tester/product owner/scrum master. We will end up deriving guiding principles that anyone can use in order to influence the team towards Behavior Driven Development, even when there is no additional budget.

  • Kiran Kashyap
    keyboard_arrow_down

    Kiran Kashyap - A Tale of Test Pyramid, Double Loop TDD and Mocha

    Kiran Kashyap
    Kiran Kashyap
    Agile Change Agent
    Lowe's
    schedule 1 year ago
    Sold Out!
    45 Mins
    Demonstration
    Beginner
    • Does your unit test suite take an obscenely long time to run?
    • Do you want to optimize the cost of testing, without resorting to traditional cost-cutting stuff?
    • Do you still use debuggers for fixing production defects?
    • Do you crave for an easier way to manage test cases written by developers?
    • Do you understand TDD but don't know where to start writing the first unit test case?

    If you say yes to any of the above questions, this session is for you.

    This demo-driven talk will help you understand

    1. How to start with TDD in a brownfield monolithic application?
    2. How to organize test cases with cost efficiency and time efficiency as goals?
    3. How you can demonstrate the above concepts to your team, in your office, by using an open-source code repo?