Inverting Test Pyramid - A First Hand Experience Report

Test automation is extremely crucial in adoption of an agile delivery. However, it can take one for a ride, if the approach is not correct. In this sensational, heart throbbing, experience report, we'll share our story of how we turned around an inefficient, expensive automation style to lean, efficient style. In addition to sharing a real-world example, we'll also share some of the key challenges we faced and how we solved them. If you are convinced about the Testing Pyramid, but are struggling to invert it, then this session is for you.

Business Impact:

  Earlier Defect Detection - Higher test coverage at Unit/Intermediate layers lead to earlier defect detection. Reduced number of issues found on higher test environments/Production. Reduced cost of defect fixing.

  Reduced maintenance cost - UI tests are fragile and costlier to maintain Vs backend tests. No of changes in services layer are comparatively less.

  Reduced test execution time - Backend tests are much faster. Almost 7-10  times faster than UI Tests - improved build certification time.

  Test feedbacks are naturally distributed across layers of application. Test feedbacks are more pin pointed/ granular.

 
 

Outline/structure of the Session

  • Traditional test automation approach and challenges (5 Min)
  • Existing Vs Inverted test pyramid (10 Min)
  • Challenges in inverting the test pyramid (15 Min)
  • Benefits with example case. (10 Min)
  • Questions ( 5 min )

Learning Outcome

  • Developer – Tester engagement is extremely crucial in bucketing and shaping the test pyramid. How to realistic achieve the nature of tight collaboration in an organisation where developers and testers work in cross-function team, but are more driven by their respective roles and targets.
  • How does bottom up test automation reduces automation backlog
  • How to shift tester's mindset from finding bugs to mistaking proofing
  • How tests can be made lighter, faster and more reliable
  • Divide and Conquer rule applies everywhere!

Target Audience

Developer / Tester

Requirements

Projector, Laptop, Mic, Speakers, Pointing device.

schedule Submitted 2 years ago

Comments Subscribe to Comments

comment Comment on this Proposal
  • Ulhas Deshmukh
    By Ulhas Deshmukh  ~  2 years ago
    reply Reply

    Are you going to discuss on
    How to convince CTO, Architect, Manager that we have Ice cone and not test pyramid
    And you got to change test strategy to focus on unit testing?

    • Sachin Natu
      By Sachin Natu  ~  2 years ago
      reply Reply

      Hello Ulhas,

      Surely we can discuss this point. I know it's challenging to invert the pyramid right away if we are already settled in Ice cone. It demands for different strategy/skills.

      Biggest selling point for this is test execution speed, crisp tests, low maintenance cost, pin pointed feedbacks, improved collaboration in dev/qa. We have certain slides which discuss about improved regression testing periods enabling quicker release cycles.

      -Sachin

  • Pramod Sadalage
    By Pramod Sadalage  ~  2 years ago
    reply Reply

    Sachin,

     

    Do you have slides we can look at?

    Thanks

    • Sachin Natu
      By Sachin Natu  ~  2 years ago
      reply Reply

      Pramod

      Added slides.

      Thanks,

      Sachin

  • Rahul Saraf
    By Rahul Saraf  ~  2 years ago
    reply Reply

    This should be an ideal way to do a Test Automation


  • Naveen Indusekhar
    Naveen Indusekhar
    schedule 2 years ago
    Sold Out!
    45 mins
    Experience Report
    Advanced

    While Agile adoption and practice is a complex model for mid/large sized Product companies to implement, we see a need for some sort of framework that can help drive it.

    Why do you need to be Agile? Where do you start, when do you start, and also how do you start? These are questions that reside in most of us during early stages of adoption. Here is a 9-Step breakdown that will help companies and teams adopt multiple Agile methodologies in different situations and scenarios. Maturing it from a ‘Persona Perspective’ framework helps the ‘decision-maker’ have a case for investment, further evaluate if it works, and improve it all through!!

    The discussion and idea is just my proposal that should help a team or an organization and is need not be the only way to implement Agile in an organization. Further detailing into these phases can form an overall Mindmap structure that is open to adopting from an organization and people standpoint.

  • Liked Naveen Indusekhar
    keyboard_arrow_down

    Research paper on 'What happens to Engineering Manager in Agile world'

    Naveen Indusekhar
    Naveen Indusekhar
    schedule 2 years ago
    Sold Out!
    20 mins
    Case Study
    Intermediate

    This is an extremly simple topic with a very complex answer. I have spent last 3 years experimenting and working with lot of people to understand what happens to the so called Manager role in Agile world. With a self organizing, empowered team, does Manager still have a role? Do you shelve these senior people who drove all your deliverables in the past for your company? Do you just let go the technical expertize these people bring along?

    The research information around the experiment and results will be shared and am very positive that this will help lot of organizations move forward with great benefits. We will see how certain gaps that are created with introduction of Agile can be solved through these senior professionals. At the same time teams don't need to compromize or business doesn't need to compromize on new found agility by adopting any of the Agile practices.

    The very next discussion that I will touch base as part of this presentation is the role of Project Leads. Many Leads resist movement to Agile because they feel their growth will now be stagnated. Is this true? It is again an interesting data set captured by talking to Leads in waterfall world and understanding what they really aspire for.

    Looking forward to sharing my insights with real implementation and resulting data that I'm sure each of you will benefit from.

  • Liked Pooja Wandile
    keyboard_arrow_down

    Scaling Scrum To Large Distributed Teams and its Challenges

    Pooja Wandile
    Pooja Wandile
    schedule 2 years ago
    Sold Out!
    60 mins
    Experience Report
    Intermediate

    As designed for use, Scrum works well for small co-located teams, cross functional teams. As the popularity of Scrum starting growing, more and more companies started adopting Scrum with mixed results. But overall, the trend has been increasing. Off late, big enterprises have also started adopting agile for their business needs and building complex products. In a recent study, the number of enterprise customers adopting scrum has doubled in last 2 years and they have used distributed scrum on large programs. So, the question is, can agile scale beyond small and co-located teams and what challenges could be there?

    This presentation talks about our experience of implementing scrum on a large program, the challenges of scaling scrum and how to overcome these challenges.

  • Archana Joshi
    Archana Joshi
    Senthilkumar Rajamarthandan
    Senthilkumar Rajamarthandan
    schedule 2 years ago
    Sold Out!
    45 mins
    Case Study
    Intermediate

    Background: 

    • One of the world’s largest legal product & services provider.
    • Development and global rollout of online legal research product for law firms to efficiently provide online research and analytical capabilities.
    • Several hundred million dollar product revenue
    • 400 Team size.
    • Product Development and subsequent enhancement and support.
    • Stringent Timelines
    • Most of the team new to agile
    • 4 Timezones

    Key Challenges on "people" front:

    1. Scale up the team to use agile in short time frame
    2. Engage and motivate the team 
    3. Be a true vendor partner with the customer

    Industry analysts predict rapid adoption of gamification. By 2014, more than 70% of Global 2000 organizations will have at least one gamified application, according to Gartner, Inc. In this presentation, we would be focusing on how gamification helped us adddress the above challenges.  Concept of “gamification” was used to drive the team to embrace agile and improve employee engagement and motivation. This helped further boost the quality and increase the overall customer satisfaction across sprints and releases.

  • Prafulla Girgaonkar
    Prafulla Girgaonkar
    Naresh Jain
    Naresh Jain
    schedule 2 years ago
    Sold Out!
    45 mins
    Experience Report
    Intermediate

    Over the last decade, eXtreme Programming practices like User Stories, Evolutionary Design, Test-Driven Development (TDD), Behavior Driven Developer (BDD), Refactoring, Continuous Integration and Automation have fundamentally changed software development processes and inherently how engineers work.

    Having experienced various benefits from XP practices on our J2EE stack, our team started to apply these practices to extract, transform, and load (ETL) and Data Analytics side of our product. Unfortunately, there is very little guidance available in this context, esp. for the SAS Platform. Right from finding the unit testing framework to structuring the code to designing our modules and setting up a Continuous Integration builds, our team had to figure out everything, the hard way.

    Join us to understand the challenges we faced during this process and how we resolved these challenges.

  • Dhaval Dalal
    Dhaval Dalal
    Naresh Jain
    Naresh Jain
    schedule 2 years ago
    Sold Out!
    90 mins
    Workshop
    Beginner

    "To know, is good. To live, is better. To be, that is perfect." - The Mother

    During the Agile adoption, its a common complain that many team in many organizations get caught up in the ceremonies or mechanics of Agile and fail to understand/appreciate the true value and spirit of Agile. And because of this, the original intent of the Agile movement itself is lost. This is a serious issue!

    This workshop will highlight, a well-proven approach to transformation (not adoption) and show the distinct steps in this journey that an individual or a collective goes through when learning anything new. Activities, serving as examples, in the workshop, will focus to show the journey - that is, how to begin with rituals, then gradually move to practices, arriving at principles and eventually internalizing the values. Witnessing this gradual process of transformation will help participants discover for themselves their current progression. We hope this will serve as a guiding light during their Agile journey.

    Finally, we will leave the participants to ponder upon and discover for themselves their ideals in life and work as this is not only applicable to software development, but also to any discipline where humans are involved, including life itself.

  • Prafulla Girgaonkar
    Prafulla Girgaonkar
    Naresh Jain
    Naresh Jain
    schedule 2 years ago
    Sold Out!
    45 mins
    Experience Report
    Intermediate

    "We've tested this feature thoroughly and it worked really well. But for some weird reason, it's really slow in production today...must be a network issue...or may be the server is having a bad day..."

    Do you often hear these kinds of comments in your development team? Let us guess, your application is very data-centric and churns big blocks of data on every user request. And under the hood, your application is most probably heavily dependent on long/complex queries with joins, temp-tables, case-statements, nested queries, etc.

    These SQL queries probably started-out very simple. But as your requirements evolved, iteration after iterations, the queries also grew in complexity. And most often, even if you test-drove your newer stories, the performance of these complex queries is not evident until you run them in production. 

    Given that our requirements will evolve and so will our database, how do you deal with the above problems?

    There are TWO essential parts to evolutionary database design:

    1. The art of refactoring your SQL queries.
    2. Figuring out the right balance of what processing is done in SQL on the DB sides and what is done on your service side in your App/Web Server.

    Join us as we take a tour of how we refactored our complex, non-performant queries and overall DB without hurting our time-to-market.

  • Alexey Ilyichev
    Alexey Ilyichev
    schedule 2 years ago
    Sold Out!
    45 mins
    Case Study
    Intermediate

    In 2011, I worked for Qik, a startup that got aqcuired by Skype. At that time Skype was in the middle of an agile transition. Аfter aquisition, Qik team was told to adopt the Agile process used by Skype. I worked with the team as an agile coach. After adopting Skype's "agile" process, our ability to deliver was brought down to almost zero. In this talk, I'll tell you the story as it happened, analyze the key problems that we faced and describe how we finally solved them. Come to this talk, if you want to avoid similar mistakes. If you are already through with your transition, I would be interested to know if you see any patterns. 

  • Anand Bagmar
    Anand Bagmar
    schedule 2 years ago
    Sold Out!
    90 mins
    Case Study
    Intermediate

    The key objectives of Organizations is to provide / derive value from the products / services they offer. To achieve this, they need to be able to deliver their offerings in the quickest time possible, and of good quality!
    In such a fast moving environment, CI (Continuous Integration) and CD (Continuous Delivery) are now a necessity and not a luxury!

    There are various practices that Organizations and Enterprises need to implement to enable CD. Testing (automation) is one of the important practices that needs to be setup correctly for CD to be successful.

    Testing in Organizations on the CD journey is tricky and requires a lot of discipline, rigor and hard work. In Enterprises, the Testing complexity and challenges increase exponentially.

    In this session, I am sharing my vision of the Test Strategy required to make successful the journey of an Enterprise on the path of implementing CD.

  • Liked Yilmaz Guleryuz
    keyboard_arrow_down

    Fibonacci Pairing: Learning through Collaboration

    Yilmaz Guleryuz
    Yilmaz Guleryuz
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    According to many researches, we learn the best when we experience it personally, and we comprehend when we teach to someone else.

    So if you want to achieve ample understanding of a topic, you’ll need to follow these basic steps:

    "Listen -> Observe -> Do -> Share"

     

    Pair-programming is a well-known practice, and it has been introduced by one of the thought leaders of Agile world, Kent Beck. I respect any technique that encourages collaboration and interaction among teams, and gets teams to communicate better.

    However, if you want to establish understanding of mastery or a skill-set, and increase the shared knowledge across your team members, then you need to re-think & apply new techniques.

    In this talk, I’ll share about What-Why-How of "Fibonacci Pairing" technique, which is yet another discovery from practice.

    The main goal of that technique is to establish understanding and improve collaboration, without overcrowding by artificial ways like lecturing.

    Fibonacci Pairing is simple yet very effective technique that can improve team work. This technique should be regarded as a complementary way to improve team culture effectively.

    We will elaborate with practical examples, real life experiences, and applicable ways onto different contexts.

  • Liked Frans Overbeek
    keyboard_arrow_down

    Spotless Clean Frontend Development with BDD, DSL and Storyboards

    Frans Overbeek
    Frans Overbeek
    schedule 2 years ago
    Sold Out!
    45 mins
    Demonstration
    Beginner

    After 3 years of improving our frontend development process we ended up with mixing BDD, DSL and Storyboards for developing spotless clean frontend code. Each approach is already powerful on his own but combining these will make them unbeatable. The characteristics are collaborate, communicate and visualize, BDD is the collaborating process between disciplines, DSL for enabling the communication and Storyboards for visualization.

    Let me show you how this works by live coding a frontend application using this mixture of approaches. The process is supported by a homemade tool based on CasperJS and PhantomCSS. And all scripts are written in JavaScript. At the end of the session we can discuss if the created code is spotless clean.

  • Liked Ashish Parkhi
    keyboard_arrow_down

    Techniques to Speed Up your Build Pipeline for Faster Feedback.

    Ashish Parkhi
    Ashish Parkhi
    Naresh Jain
    Naresh Jain
    schedule 2 years ago
    Sold Out!
    45 mins
    Experience Report
    Intermediate

    I would like to share my experience and journey on how we brought down our Jenkins build pipeline time down from over 90 minutes to under 12 minutes. In the process, I would share specific techniques which helped and also some, which logically made sense, but actually did not help. If your team is trying to optimize their build times, then this session might give you some ideas on how to approach the problem.

    Development Impact - For one of our build job, below graph shows how the number of builds in a day have increased over a period of time as the build time has reduced. Frequency of code check-in has increased; Wait time has reduced; failed test case faster to isolate and fix.

    Business Impact - More builds leading to quicker feedback and faster story acceptance and less story spill over.

  • Liked Shirish Padalkar
    keyboard_arrow_down

    Application Security - The Agile Way

    Shirish Padalkar
    Shirish Padalkar
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Beginner

    Traditionally application security has involved upfront design and a big bang penetration test after development. This leads to the phenomenon of “bolt-on” security that translates into increased cost and complexity.

    Drawing on our experience on real-world projects, we show how security can be baked-in on an agile project. Using case studies we demonstrate how security concerns are captured during project inceptions, how developers write secure code, security testing is automated and how configuration management can help achieve secure deployments. This talk introduces several new concepts like secure by design, secure design patterns and lightweight code reviews.