What You are Doing Wrong with Automated Testing

We firmly believe that automated testing puts the "A" in "Agile". Without an effective suite of automated tests your ability to be truly agile (that is embrace change) can only be based on the hope that your latest change doesn't have unintended consequences. Additionally, without automated tests, you are missing a vital component in getting feedback into the development team's hands. In our travels, we have encountered many organizations that are struggling with automated testing. These organizations are successfully adopting many Agile techniques but are failing when it comes to automated testing. We frequently hear "Automated testing just doesn't work for us" (eerily reminiscent of the days when we would hear, "Agile just doesn't work for us"). From our experience addressing their challenges, we have identified anti-patterns common across these organizations. These anti-patterns look like they should work, but are in fact doing more harm than good.

This talk is about those anti-patterns. We have given those anti-patterns a name and a face to help organizations understand why they are not getting the benefits from automated testing that others are. We describe several anti-patterns, such as the "Ice Cream Cone", the "Monolith", the "Sunk Cost". We explain why these anti-patterns appear to be good solutions, what makes them attractive, and why they do more harm than good. We talk about the right approach and draw on our experiences helping organizations adopt a robust automated testing strategy that instills confidence and provides fast feedback to the development team. We explain what benefits from automated testing the anti-pattern is preventing. 

 
28 favorite thumb_down thumb_up 0 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

The talk is presented with two speakers. The first presenter (Shawn) introduces the anti-pattern and why it looks like it should be a good solution and then the second presenter (Martin) explains why it is not a good solution, but rather an anti-pattern, what the correct approach should be, and what benefit the organization will get from the correct approach. The back and forth with two presenters on each anti-pattern works very well for this type of talk. The audience clearly identifies the anti-pattern and its false solutions with Shawn's presentation. When Martin talks about the problems the anti-pattern creates and what the correct approach is there is a clear delineation (through the use of the two speakers) to understand which is the anti-pattern (bad) and which is the pattern (good). Throughout the presentation the audience is engaged to share their thoughts and experience with the anti-pattern. 

 

The timing will likely run like this: (essentially 4 minutes per anti-pattern x 8 anti-patterns = 32 minutes plus Intro & Q&A)

  • 0:00 Introduction
  • 0:05 Explanation of what an anti-pattern is
  • 0:10 Anti-pattern1: Ice Cream Cone
  • 0:14 Anti-pattern2: Accepting Failures
  • 0:18 Anti-pattern3: Recorders
  • 0:22 Anti-pattern4: Monolith
  • 0:26 Anti-pattern5: Ownership Displacement
  • 0:30 Anti-pattern6: Coverage = Quality
  • 0:34 Anti-pattern7: Sunk Cost
  • 0:38 Anti-pattern8: Under the Hood
  • 0:40 Final Q&A

Learning Outcome

  • Define what an anti-pattern is
  • Describe common anti-patterns in automated testing
  • Describe best practices in automated testing

Target Audience

Developers, Test Automation Engineers, Testers, Project Managers

schedule Submitted 11 months ago

Comments Subscribe to Comments

comment Comment on this Proposal

  • Liked David Horowitz
    keyboard_arrow_down

    The 7 Secrets of Highly Effective Retrospectives

    David Horowitz
    David Horowitz
    schedule 11 months ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    Retrospectives are the core of agility. And yet they are the scrum ceremony that is most frequently skipped. Many teams like the idea of the retrospective but find them boring, or worse ineffective.

    This talk aims to re-energize retrospective facilitators and participants. Starting with the basics: "what's a retrospective and how do you run one?", this talk reveals 7 secrets that lead to more engaging, more effective retrospectives.

    You'll learn:

    * The best way to ensure your retrospectives lead to real change

    * The "pledge" everyone on your team should take before participating

    * How to know who to include in each retrospective

    * The single most important thing you can do to keep your team engaged during the retro

    * And much, much more!

  • Liked Shawn Faunce
    keyboard_arrow_down

    The Awkward Teenager of Testing: Exploratory Testing

    Shawn Faunce
    Shawn Faunce
    schedule 11 months ago
    Sold Out!
    45 mins
    Talk
    Beginner

    We think we understand that awkward teenager.

    Many experienced testers will claim exploratory testing expertise, but too few have ever written an exploratory testing charter, and even fewer have applied a heuristic in that charter. We think we understand exploratory testing just as we think we understand teenagers, because “we have been there”. However the reality is that many of the words currently used in exploratory testing are foreign to us and we feel awkward about our lack of knowledge. The goal of this talk is to give people experience writing and executing exploratory testing charters, creating mind maps, and applying exploratory testing heuristics.

    The talk is intended to introduce people to the exploratory testing techniques described by Elisabeth Hendrickson in her book Explore It! with some added material from the work of Cem Kaner and James Bach.

     

  • Camille Bell
    Camille Bell
    schedule 10 months ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    You're a Certified Scrum Master. Perhaps you are an Agile Manger, Agile Coach or Facilitator.

    Maybe you are newly minted or maybe you've been doing it a while, but either way you've noticed that not everything seems to work according the way the training or certification class implied it should.

    In this session, Camille Bell will explore what you weren't told in training, but need to know. Such as:

    • What assumptions Scrum makes that may not apply to your company or organization
    • Why some types of teams should not use Scrum and what they should use instead
    • How soon Scrum of Scrum stops scaling and what to use when it doesn't scale
    • Why some teams don't improve despite holding retrospectives
    • How to recognize the hockey stick burn down and what to do about it
    • What's a WIP limit and when it can be helpful
    • When estimation most helpful, when it's a complete waste and what to do instead
    • Why simple prioritization of a Product Backlog won't generate a Minimal Viable Product
    • Why the As a.., I want.. So that.. user story isn't enough and what you need to add
    • What are the critical missing practices your development team needs

     

  • Liked Craeg K Strong
    keyboard_arrow_down

    Behavior Driven Development Workshop

    Craeg K Strong
    Craeg K Strong
    schedule 10 months ago
    Sold Out!
    45 mins
    Workshop
    Beginner

    Behavior Driven Development / Acceptance Test Driven Development (BDD/ATDD) is a new, exciting approach to developing software that has been shown to reduce rework and increase customer satisfaction. While other testing tools focus primarily on “are we building the thing right?”, BDD tools such as Cucumber and SpecFlow attack the problem of software directly at its source: “are we building the right thing?” By retaining all the benefits of automated unit testing, while extending them upstream to cover requirements, we cut the Gordian knot of risk and complexity to unleash hyper-productivity. 

    Why is BDD so effective?

    • As a form of Test driven design, BDD helps produce frugal, effective and testable software.
    • As a development tool, BDD frameworks like SpecFlow provide many convenience functions and are pre-integrated with powerful libraries like Nunit and selenium to make writing tests a snap.
    • As a collaboration tool, BDD helps ensure the “three amigos” (tester, analyst and developer) sync up – ahead of time.
    • As a facilitation technique, BDD enables product owners to efficiently provide the team with concrete examples that clarify the true intent of a user story and define the boundaries.
    • As a reporting tool, BDD captures functional coverage, mapping features to their acceptance criteria to their test results, in an attractive hierarchical presentation.

    Want functional documentation? How about documentation that is guaranteed to be correct, because every feature maps to its test results? Witness the holy grail of traceability – executable specifications.

    We will spend a few minutes talking about the context and pre-requisites, so attendees have an idea of where BDD fits in, and what type of investment they are signing their teams up for. We will see that in return for a modest amount of investment in tools and training, very significant benefits can be realized, and the benefits compound over time.

    This workshop then dives right in to Gherkin, the structured English language technique used to capture BDD specifications. We will spend the better part of the session learning the tricks and techniques that make for robust and maintainable gherkin specifications. We will review and critique lots of examples, both good and bad.

    We will review several examples of reports generated from BDD tools, to provide context and to immediately highlight the bottom line business value that makes an investment in BDD so worthwhile.

     

     

    Come and learn why Behavior driven design is taking the software world by storm!

  • Liked David Kane
    keyboard_arrow_down

    "Hitting the Target" - Business Value in Mission-Focused Organizations

    David Kane
    David Kane
    Deepak Srinivasan
    Deepak Srinivasan
    schedule 10 months ago
    Sold Out!
    45 mins
    Workshop
    Beginner

    In the simplest of terms, software development decisions for commercial organizations can be reduced to a calculation of whether the cost of developing the software will be outweighed by the estimated revenue generated or costs saved by the software.  However, as Mark Schwartz points out in his book, “The Art of Business Value Paperback” this simple explanation is insufficient for commercial organizations, and not applicable for government and other non-commercial organizations for whom the impact of software isn’t primarily measured in terms of revenue.  

    In this session participants will experience a simulation that has been created to explore these question of how to make decisions about investments to deliver mission and business value by examining the impact of these decisions on the performance of organizations in changing environments.

  • Brian Sjoberg
    Brian Sjoberg
    schedule 10 months ago
    Sold Out!
    45 mins
    Workshop
    Beginner

    Do you want to write great User Stories that provide the vehicle for conversation and confirmation that we build the right thing? Do you struggle with splitting stories so that they still provide business value but can be accomplished within a fraction of your iteration and be potentially shippable to production? We will do a quick refresher on User Story formatting to include Acceptance Criteria. Then we will dive into learning techniques for splitting stories in this interactive workshop. 

  • Andy Bacon
    Andy Bacon
    schedule 11 months ago
    Sold Out!
    45 mins
    Experience Report
    Intermediate

     

    Are you a Scrum Master that has a strong affinity to sticky notes, dry erase markers, and other tools of Agile facilitation? Are you tired of seeing the same old blue painters tape on every kanban board? Want to learn some new facilitation techniques that you can use in retrospectives or to build awesome information radiators with your team?

    If so, join me for my session on the Super Agile Satchel.

    During the session, I’ll briefly talk about how the satchel came to be and its awesome contents.  Then we’ll jump right in to actual examples of information radiators that I created with the tools the satchel contains.  I'll discuss why choosing the right materials, colors, and sizes are critical to facilitation and creating information radiators that draw attention, are maintainable, and focus the team on what's truly important.

  • Liked Mathias Eifert
    keyboard_arrow_down

    Don’t assume you’re creating value – prove it!

    Mathias Eifert
    Mathias Eifert
    schedule 11 months ago
    Sold Out!
    45 mins
    Talk
    Beginner

    Does your organization find it hard to determine “the right thing” to build? You are not alone – studies show that even in very high performing organizations only 10-35% of initial ideas actually generate business value. Agile development should make it easier to obtain early customer feedback, but in most organizations Agile approaches are limited to software development teams with little connection to the rest of the business. In addition, Agile methods by themselves offer few guidelines on how to translate organizational goals and customer needs into the backlog’s content and relative priorities in the first place. As a result, there is a significant, but often underappreciated risk that Agile teams end up very efficiently building “the wrong thing right.”

    In this session, we explore how Lean Discovery and experimentation can expand the scope of Agile’s “inspect and adapt” feedback loops to systematically identify and validate critical assumptions about our product’s value proposition. Based on the Lean Startup and Lean UX approach to product development as a series of hypotheses about customers’ behaviors and value perceptions, we discuss ways to derive testable assumptions from organizational goals to enable validated learning. Finally, we explore the implications of this approach on project planning and budgeting to support increased business agility.

  • Liked Paul Boos
    keyboard_arrow_down

    Mind Meld: Why Pair Programming Works

    Paul Boos
    Paul Boos
    schedule 11 months ago
    Sold Out!
    45 mins
    Workshop
    Beginner

    So why does pair programming (or any form of pairing really) work? Well rather than tell you why, let's experience it! 

    This is a simple 3 round exercise that you can do with your teams and managers to demonstrate the benefits of pairing. It will show the linkage between having a shared mental model through collaboration and ease of integrating the resulting work.

  • Liked Trent Hone
    keyboard_arrow_down

    DevOps Darwinism: Advancing our Art through Safe-to-Fail Experimentation

    Trent Hone
    Trent Hone
    schedule 11 months ago
    Sold Out!
    45 mins
    Talk
    Beginner

    Static designs and monolithic structures are brittle and prone to failure. How can we progress beyond them? By understanding the nature of our challenges and applying the right tools at the right time. This talk will describe how to use evolutionary principles to foster changes in your architecture and infrastructure so that you can create antifragile systems.

    Nassim Nicholas Taleb spoke to these ideas in “Antifragile: Things That Gain from Disorder.” Join us as we describe how we can:

    • Move beyond planning for known failure modes and instead develop antifragile systems that are safe-to-fail in unanticipated ways.
    • Leverage the Cynefin framework and its 5 sense-making domains to better frame problems and drive action.
    • Employ microservice architectures to make the variability of our environments work for us, not against us.
    • Gain knowledge more rapidly through multiple parallel experiments.
  • Liked Thomas Cagley
    keyboard_arrow_down

    Storytelling: Developing the Big Picture for Agile Efforts

    Thomas Cagley
    Thomas Cagley
    schedule 11 months ago
    Sold Out!
    45 mins
    Tutorial
    Intermediate

    Agile reminds us that the focus of any set of requirements needs to be on an outcome rather than a collection of whats and whos.  Storytelling is a powerful tool to elevate even the most diehard requirements analyst from a discussion of individual requirements to a discussion of outcomes. Outcomes are the big picture that acts as an anchor for whole efforts and which is continuously broken down into more and more detailed backlogs. The onion metaphor that is popularly used in agile planning (Cohn’s Planning Onion) can be used to describe the evolution of backlogs. Building an initial backlog is much like peeling through the layers of an onion to get to the core. There are many mechanisms for developing and maintaining the detailed backlogs, including asking, observing, showing and all sorts of hybrids. Using the onion metaphor, techniques for developing and splitting user stories are the second layer of the onion. However, before getting to the center of the backlog evolution onion, composed of features, epics, and user stories, we need to understand the big picture. 

     Presentation:

    Provide an overview of storytelling in a business context and a lean change canvas framework.

    Exercise

    The room will be broken into teams (aisles will be used if auditorium seating).  Each team will be seeded with a common product change scenario. Based on the scenario the teams will be asked to tell the story of the change and capture the story on a small change canvas.  The exercise and session will culminate in by sharing ideas and lessons learned.

    (Note the longer workshop would break the changing canvas into epics and stories)

     

  • Liked Marie Dingess
    keyboard_arrow_down

    Using Visuals to Master the ART of SAFe/Release Planning

    Marie Dingess
    Marie Dingess
    schedule 11 months ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    A well-facilitated large-scale Planning event (such as a SAFe PI Planning) can leave you feeling energized and more confident in your team and your direction. Poorly facilitated events have the adverse effect and can leave you feeling like you’ve just been run over by a Mack Truck.

    There is not a lot of guidance from SAFe on how to facilitate this critical event from their website, other than an example agenda and program board. This session will focus on how information radiators and other visual tools can make a tremendous difference in increasing engagement and collaboration amongst teams in order to orchestrate a thoughtful plan with high confidence.

    We will show battle-tested ways on how visuals are critical to...

    • increase collaboration across teams
    • reinforce development practices & standards
    • track team progress through the day
    • highlight dependencies
    • increase the confidence vote
    • motivate teams to strive for success

    Participants will leave this session with some visual tools and tips they can immediately put to use in their designing their next PI Planning or ANY large-scale planning event.
    “There is no magic in SAFe… except for maybe PI Planning” - SAFe authors

  • Liked Tim Gifford
    keyboard_arrow_down

    "DevOps" on Day 1 with Operations First Delivery

    Tim Gifford
    Tim Gifford
    schedule 10 months ago
    Sold Out!
    45 mins
    Talk
    Beginner

    DevOps lore tell legendary tales of “Unicorn” companies. We’re told these mythical companies continuously deliver software to production with nary a blemished aura or mussed mane. Can it be so? Is this but a fairytale?

    In this talk, we will dispel the fantasy and show you how to get similar results. Spawning the first unicorn is the most difficult, so I will show you the specific steps, tools, techniques and architectural patterns of Operations First Delivery to create your first “Unicorn” project.

  • Liked M. Scott Ford
    keyboard_arrow_down

    Embracing the Red Bar: A Technique for Safely Refactoring Your Test Code

    M. Scott Ford
    M. Scott Ford
    schedule 11 months ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    Does your team treat test code differently than production code? Do you let your test code accumulate duplication and complexity that you'd normally attempt to squash in your production code? Have your tests become brittle? Are you worried that they aren't providing you the same value they used to? Have you strongly considered dumping your test suite and starting over? Are you afraid that if you refactor your test code, you'll introduce false positives?

    If you said yes to any of those questions, then this talk is for you.

    We'll explore the technique of "refactoring against the red bar" (http://butunclebob.com/ArticleS.MichaelFeathers.RefactoringAgainstTheRedBar), and how you can employ this technique to confidently refactor your test code. No longer do you need to let your test code have a lower standard of quality than your production code.

  • Liked Ken Furlong
    keyboard_arrow_down

    Upgrade Your Metrics – Cumulative Flow Diagrams and Beyond

    Ken Furlong
    Ken Furlong
    schedule 11 months ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    When dealing with metrics, most Agile or Lean teams begin with a Burndown Chart.  Unfortunately, that is also where most of them stop.  While a Burndown Chart is a great first step, it only provides a small sliver of the information the team has access to.

     

    In this talk, we’ll be starting at the beginning with what a Cumulative Flow Diagram is, how it relates to a Burndown Chart, its advantages, and where it too ultimately stops.  We’ll then look at additional information radiators that the team can easily use based on existing data to provide transparency to stakeholders and the raw material for continuous improvement.

  • Liked Atif Salam
    keyboard_arrow_down

    Cracking the Code... Implementing SCRUM at Scale within Enterprise Data at Fannie Mae

    Atif Salam
    Atif Salam
    PHILLIP MANKETO
    PHILLIP MANKETO
    Dave McMunn
    Dave McMunn
    schedule 11 months ago
    Sold Out!
    45 mins
    Case Study
    Intermediate

    Fannie Mae, a leading source of residential mortgage credit in the U.S. secondary market, provides reliable, large­scale access to affordable mortgage credit across the country so people can buy, refinance, or rent homes. In November 2014, Fannie Mae’s Enterprise Data Warehouse completed a multi­year project to implement several thousand data attributes across numerous interfaces which were then integrated with various development organizations working in parallel; the results were sub­optimal. In January 2015, Fannie Mae undertook a brownfield initiative to transform Fannie Mae’s Enterprise Data Warehouse, responsible for sourcing, vending and provisioning data attributes, which informed numerous interfaces, as part of a larger Federal compliance mandate, to SCRUM. Initially, starting with two teams using SCRUM, the initiative progressed to six-teams implementing SCRUM­OF­SCRUMS, within 6 months. Thereafter, the SAFe framework was adopted encompassing twelve­teams (130+ team members) on a single Agile Release Train. As a result of transforming from Waterfall to Agile, Fannie Mae experienced a significant boost in productivity and reduction in delivery risks through the relentless focus on innovation and automation to ship "production ready" code with high and higher frequency. The transformation to Agile has revolutionized how the firm plans for the delivery of large scale ($100 million plus investment/year) programs, has significantly mitigated risk inherent in complex in integration between legacy and new architectures/applications, as well as between internal and external systems. Within the larger context of the transformation to Enterprise Agility, this Experience Report will focus on the changes to the organization, architecture, and technical practices required to implement data attributes every two­-weeks and the corresponding benefits realized.

  • Liked Ben Morris
    keyboard_arrow_down

    DevOps for the Rest of Us, Reprise

    Ben Morris
    Ben Morris
    Chris Cassatt
    Chris Cassatt
    schedule 11 months ago
    Sold Out!
    45 mins
    Demonstration
    Beginner

    (This talk was well-received in 2015, so we can do an updated version for 2016)

    DevOps as a buzzword is gaining traction, but what does it really mean? Managers, non-techies, and developers-new-to-devops will get a guided demo of development automation. See all the cool tools in action - continuous integration, automated testing, cloud deployment, etc. More importantly, we'll walk through what they do, and why that adds value to a project. 

    This talk will...

    • Break down the buzzwords and define some key technical practices in plain english.
    • Uncover the pain that leads teams to seek greater automation.
    • Demonstrate a continuous integration pipeline working in practice via live demo.
    • Diminish the knowledge gap between technical practitioners and managers/analysts/coaches.
    • Level-up the vocabulary of non-technical attendees.
    • Introduce practices to developers who don't yet work in an automated environment.
    • Spark "ah-ha" moments to convert skeptics into DevOps believers!

    By the way, all of the tools in the demo are some combination of free and/or open source. DevOps doesn't have to cost a lot.

  • Liked Tony Timbol
    keyboard_arrow_down

    Agile Governance for Agile Adults – Grow up and give the Enterprise Real not Relative Data

    Tony Timbol
    Tony Timbol
    schedule 11 months ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    Agile cost money just like waterfall does. Your software labor costs will be all internal (no vendors), all external (only vendors) or a mix. Paying for software labor by the billable hour means no connection to delivered business value and if vendors are using fixed pricing methods, hidden premiums may unnecessarily inflate your costs.

     This 45 minute interactive discussion includes how two organizations focus on quantitative sizing analytics and estimation best practices and how their Enterprise Managers today manage & reduce costs, improve predictable delivery and quantify business value. These Enterprises are investing tens of millions of dollars in Agile implementations at scale and need non-relative (normalized) data that is meaningful instead of relative aggregation of team level measures. Learn how they are adopting program and portfolio level measures and metrics to have real conversation about business value delivered.

     

  • Liked kelly snavely
    keyboard_arrow_down

    Women in Agile and the Confidence Code

    kelly snavely
    kelly snavely
    schedule 11 months ago
    Sold Out!
    45 mins
    Talk
    Beginner

    This talk is inspired by the book ‘The Confidence Code’ by Katty Kay and Claire Shipman.

    What is confidence and how do you know you have it?   While confidence is partly influenced by genetics, it is not a fixed psychological state.  However, you won’t discover it thinking positive thoughts or by simply squaring your shoulders and faking it.  It requires work and choices: less focus on people pleasing and perfectionism and more action, risk taking and fast failures.  This is why it can seem harder for women because these behaviors aren’t typically the ‘norm’ for women but generally come naturally for men.

    In this talk we will explore the roots of confidence and the gender gap between men and women.  To ground the learnings, we will also hear interview summaries from four great and diverse women in agile: 

     Lyssa Adkins, Esther Derby, Ellen Grove, and Kat Conner

  • Liked Ryan Jenkins
    keyboard_arrow_down

    Going Agile, But Are You Leaving Your Teams Behind?

    Ryan Jenkins
    Ryan Jenkins
    schedule 11 months ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    Building an Agile organization starts with great teams and a positive culture, but what really happens when your organization starts to transition its product delivery to Agile?  How much focus are you putting on building the culture, teams and growing communities inside the organization to drive real, lasting change?  Do you have a strategy to change the organization and build better teams & communities or do you prescriptively follow a framework and hope the culture change follows?

    In transforming the way you work, focus on things that will enable the long term, sustainable growth of Agile throughout the organization, and the larger the organization the stronger the underlying community needs to be.  This is a key difference between the organization "being Agile" vs. "doing Agile" and one piece of the cultural transition.

    In this talk we'll explore team building and community inside an organization.  We'll look at what helps to enable a high performing team, how to grow your organizational communities around the team to enable them for success, how to prime the organization for the team's value delivery and finally how to spread the knowledge & culture to drive organizational change.