Selecting & Implementing an Automated Software Testing Tool at EPA - Lessons Learned

This presentation delves into the details of selecting and implementing an automated software testing tool at a Federal Agency. We learnt a lot of lessons during this process and we hope that others considering software automation can learn from our successes and failures.
 
4 favorite thumb_down thumb_up 0 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

1.  Introductions

2.  Our project Context & Goals

3.  The process we adopted for tool selection

4.  Our decision

5.  Lessons learned

Learning Outcome

Learn how to select and implement the right software automation testing tool for regression and other use cases

 

Target Audience

Software Development Teams considering adopting automated software testing tool for regression and other purposes

schedule Submitted 11 months ago

Comments Subscribe to Comments

comment Comment on this Proposal

  • Liked Manjit Singh
    keyboard_arrow_down

    The Power of Mindsets and Questions in Agile Coaching

    45 mins
    Talk
    Intermediate

    “The important thing is not to stop questioning.” –Albert Einstein, because “every question missed is a crisis waiting to happen.” Discover the technique to ask the questions to make breakthrough differences in decision- making, problem-solving, innovation & culture. Typically, questions open thinking, while answers often close down thinking. Question Thinking is a tool for moving beyond limitations in perception and thinking and advancing to novel and extraordinary solutions and answers.

  • Liked David Horowitz
    keyboard_arrow_down

    The 7 Secrets of Highly Effective Retrospectives

    David Horowitz
    David Horowitz
    Cofounder and CEO
    Retrium, LLC
    schedule 11 months ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    Retrospectives are the core of agility. And yet they are the scrum ceremony that is most frequently skipped. Many teams like the idea of the retrospective but find them boring, or worse ineffective.

    This talk aims to re-energize retrospective facilitators and participants. Starting with the basics: "what's a retrospective and how do you run one?", this talk reveals 7 secrets that lead to more engaging, more effective retrospectives.

    You'll learn:

    * The best way to ensure your retrospectives lead to real change

    * The "pledge" everyone on your team should take before participating

    * How to know who to include in each retrospective

    * The single most important thing you can do to keep your team engaged during the retro

    * And much, much more!

  • Liked Craeg K Strong
    keyboard_arrow_down

    Behavior Driven Development Workshop

    45 mins
    Workshop
    Beginner

    Behavior Driven Development / Acceptance Test Driven Development (BDD/ATDD) is a new, exciting approach to developing software that has been shown to reduce rework and increase customer satisfaction. While other testing tools focus primarily on “are we building the thing right?”, BDD tools such as Cucumber and SpecFlow attack the problem of software directly at its source: “are we building the right thing?” By retaining all the benefits of automated unit testing, while extending them upstream to cover requirements, we cut the Gordian knot of risk and complexity to unleash hyper-productivity. 

    Why is BDD so effective?

    • As a form of Test driven design, BDD helps produce frugal, effective and testable software.
    • As a development tool, BDD frameworks like SpecFlow provide many convenience functions and are pre-integrated with powerful libraries like Nunit and selenium to make writing tests a snap.
    • As a collaboration tool, BDD helps ensure the “three amigos” (tester, analyst and developer) sync up – ahead of time.
    • As a facilitation technique, BDD enables product owners to efficiently provide the team with concrete examples that clarify the true intent of a user story and define the boundaries.
    • As a reporting tool, BDD captures functional coverage, mapping features to their acceptance criteria to their test results, in an attractive hierarchical presentation.

    Want functional documentation? How about documentation that is guaranteed to be correct, because every feature maps to its test results? Witness the holy grail of traceability – executable specifications.

    We will spend a few minutes talking about the context and pre-requisites, so attendees have an idea of where BDD fits in, and what type of investment they are signing their teams up for. We will see that in return for a modest amount of investment in tools and training, very significant benefits can be realized, and the benefits compound over time.

    This workshop then dives right in to Gherkin, the structured English language technique used to capture BDD specifications. We will spend the better part of the session learning the tricks and techniques that make for robust and maintainable gherkin specifications. We will review and critique lots of examples, both good and bad.

    We will review several examples of reports generated from BDD tools, to provide context and to immediately highlight the bottom line business value that makes an investment in BDD so worthwhile.

     

     

    Come and learn why Behavior driven design is taking the software world by storm!

  • Liked Mathias Eifert
    keyboard_arrow_down

    Don’t assume you’re creating value – prove it!

    45 mins
    Talk
    Beginner

    Does your organization find it hard to determine “the right thing” to build? You are not alone – studies show that even in very high performing organizations only 10-35% of initial ideas actually generate business value. Agile development should make it easier to obtain early customer feedback, but in most organizations Agile approaches are limited to software development teams with little connection to the rest of the business. In addition, Agile methods by themselves offer few guidelines on how to translate organizational goals and customer needs into the backlog’s content and relative priorities in the first place. As a result, there is a significant, but often underappreciated risk that Agile teams end up very efficiently building “the wrong thing right.”

    In this session, we explore how Lean Discovery and experimentation can expand the scope of Agile’s “inspect and adapt” feedback loops to systematically identify and validate critical assumptions about our product’s value proposition. Based on the Lean Startup and Lean UX approach to product development as a series of hypotheses about customers’ behaviors and value perceptions, we discuss ways to derive testable assumptions from organizational goals to enable validated learning. Finally, we explore the implications of this approach on project planning and budgeting to support increased business agility.

  • Liked Paul Boos
    keyboard_arrow_down

    Mind Meld: Why Pair Programming Works

    Paul Boos
    Paul Boos
    IT Executive Coach
    Excella
    schedule 11 months ago
    Sold Out!
    45 mins
    Workshop
    Beginner

    So why does pair programming (or any form of pairing really) work? Well rather than tell you why, let's experience it! 

    This is a simple 3 round exercise that you can do with your teams and managers to demonstrate the benefits of pairing. It will show the linkage between having a shared mental model through collaboration and ease of integrating the resulting work.

  • Liked M. Scott Ford
    keyboard_arrow_down

    Embracing the Red Bar: A Technique for Safely Refactoring Your Test Code

    45 mins
    Talk
    Intermediate

    Does your team treat test code differently than production code? Do you let your test code accumulate duplication and complexity that you'd normally attempt to squash in your production code? Have your tests become brittle? Are you worried that they aren't providing you the same value they used to? Have you strongly considered dumping your test suite and starting over? Are you afraid that if you refactor your test code, you'll introduce false positives?

    If you said yes to any of those questions, then this talk is for you.

    We'll explore the technique of "refactoring against the red bar" (http://butunclebob.com/ArticleS.MichaelFeathers.RefactoringAgainstTheRedBar), and how you can employ this technique to confidently refactor your test code. No longer do you need to let your test code have a lower standard of quality than your production code.

  • Liked Ken Furlong
    keyboard_arrow_down

    Upgrade Your Metrics – Cumulative Flow Diagrams and Beyond

    45 mins
    Talk
    Intermediate

    When dealing with metrics, most Agile or Lean teams begin with a Burndown Chart.  Unfortunately, that is also where most of them stop.  While a Burndown Chart is a great first step, it only provides a small sliver of the information the team has access to.

     

    In this talk, we’ll be starting at the beginning with what a Cumulative Flow Diagram is, how it relates to a Burndown Chart, its advantages, and where it too ultimately stops.  We’ll then look at additional information radiators that the team can easily use based on existing data to provide transparency to stakeholders and the raw material for continuous improvement.

  • Liked Joel Bancroft-Connors
    keyboard_arrow_down

    Restarting your agile transformation- An Agile Playbook

    45 mins
    Talk
    Intermediate

    Warning, road closed use alternate route.

    Has your agile transformation stalled? Are you treading water? Does it feel like your standups are just going through the motion?  Sounds like you need a Transformation bypass.

    Any agile transformation is a challenge and can stall if it doesn't get the right amount of attention. Maybe you had a consulting firm come in to lay out a roadmap. Only after they left the anti-bodies rushed back in. Or your tried to kick it off internally, to find yourself never getting off the ground despite executive support?

    This session provides a playbook for re-engaging (or engaging from the get go) an enterprise agile transformation. A week by week, play by play guide for enabling the change engine to get enough momentum to keep going despite the resistance.

    This playbook is based on what we are doing right now in the AOL Platform's division. From day one hitting the ground to when and how you roll out common training, to what you do next, and next and next.

  • Liked Atif Salam
    keyboard_arrow_down

    Cracking the Code... Implementing SCRUM at Scale within Enterprise Data at Fannie Mae

    45 mins
    Case Study
    Intermediate

    Fannie Mae, a leading source of residential mortgage credit in the U.S. secondary market, provides reliable, large­scale access to affordable mortgage credit across the country so people can buy, refinance, or rent homes. In November 2014, Fannie Mae’s Enterprise Data Warehouse completed a multi­year project to implement several thousand data attributes across numerous interfaces which were then integrated with various development organizations working in parallel; the results were sub­optimal. In January 2015, Fannie Mae undertook a brownfield initiative to transform Fannie Mae’s Enterprise Data Warehouse, responsible for sourcing, vending and provisioning data attributes, which informed numerous interfaces, as part of a larger Federal compliance mandate, to SCRUM. Initially, starting with two teams using SCRUM, the initiative progressed to six-teams implementing SCRUM­OF­SCRUMS, within 6 months. Thereafter, the SAFe framework was adopted encompassing twelve­teams (130+ team members) on a single Agile Release Train. As a result of transforming from Waterfall to Agile, Fannie Mae experienced a significant boost in productivity and reduction in delivery risks through the relentless focus on innovation and automation to ship "production ready" code with high and higher frequency. The transformation to Agile has revolutionized how the firm plans for the delivery of large scale ($100 million plus investment/year) programs, has significantly mitigated risk inherent in complex in integration between legacy and new architectures/applications, as well as between internal and external systems. Within the larger context of the transformation to Enterprise Agility, this Experience Report will focus on the changes to the organization, architecture, and technical practices required to implement data attributes every two­-weeks and the corresponding benefits realized.

  • Liked Ben Morris
    keyboard_arrow_down

    DevOps for the Rest of Us, Reprise

    45 mins
    Demonstration
    Beginner

    (This talk was well-received in 2015, so we can do an updated version for 2016)

    DevOps as a buzzword is gaining traction, but what does it really mean? Managers, non-techies, and developers-new-to-devops will get a guided demo of development automation. See all the cool tools in action - continuous integration, automated testing, cloud deployment, etc. More importantly, we'll walk through what they do, and why that adds value to a project. 

    This talk will...

    • Break down the buzzwords and define some key technical practices in plain english.
    • Uncover the pain that leads teams to seek greater automation.
    • Demonstrate a continuous integration pipeline working in practice via live demo.
    • Diminish the knowledge gap between technical practitioners and managers/analysts/coaches.
    • Level-up the vocabulary of non-technical attendees.
    • Introduce practices to developers who don't yet work in an automated environment.
    • Spark "ah-ha" moments to convert skeptics into DevOps believers!

    By the way, all of the tools in the demo are some combination of free and/or open source. DevOps doesn't have to cost a lot.

  • Liked kelly snavely
    keyboard_arrow_down

    Women in Agile and the Confidence Code

    45 mins
    Talk
    Beginner

    This talk is inspired by the book ‘The Confidence Code’ by Katty Kay and Claire Shipman.

    What is confidence and how do you know you have it?   While confidence is partly influenced by genetics, it is not a fixed psychological state.  However, you won’t discover it thinking positive thoughts or by simply squaring your shoulders and faking it.  It requires work and choices: less focus on people pleasing and perfectionism and more action, risk taking and fast failures.  This is why it can seem harder for women because these behaviors aren’t typically the ‘norm’ for women but generally come naturally for men.

    In this talk we will explore the roots of confidence and the gender gap between men and women.  To ground the learnings, we will also hear interview summaries from four great and diverse women in agile: 

     Lyssa Adkins, Esther Derby, Ellen Grove, and Kat Conner

  • Dave Nicolette
    Dave Nicolette
    Consultant
    Neo Pragma LLC
    schedule 1 year ago
    Sold Out!
    45 mins
    Others
    Beginner

    Ideas similar to test-infected development or test-driven development have been around quite a while - at least since Alan Perlis wrote about interleaving small amounts of design with small amounts of testing in the 1968 Proceedings of the NATO Software Engineering Conference. Yet, even today, there are endless debates about whether such an approach is useful. Some consider it a baseline practice for any professional developer. Others consider it extra work that adds no value. 

    There's certainly more than one way to achieve a goal. What are the goals, when we write and deliver software professionally? Let's identify the various stakeholders of a software system and enumerate the needs of each. Then, let's walk through several popular ways of building software - TDD and others - and see how we can meet those needs using each approach. 

  • Liked Jim Damato
    keyboard_arrow_down

    PiMP My Scrum - Looking at Scrum Through the Lens of Waterfall

    Jim Damato
    Jim Damato
    Transformation Manager
    Scrum Inc
    schedule 11 months ago
    Sold Out!
    45 mins
    Talk
    Beginner

    Scrum is hard to understand, or so they say. But when we describe new topics in easy to understand language, suddenly it can all make sense.

    We'll look at Scrum through the lens of Waterfall, to discuss how similar they really are if you think about the core concepts that make Scrum work so well and apply them to Waterfall in the right way.

    We'll talk about how the first big hurdle isn't in changing the process, but rather in how you think about the process. This is particularly important because often we cannot change the process. However, we do have flexibility to operate *within* and *in addition to* the process, where we can inject tangible benefits, so we can create trust around the approaches, so we can get the support to move closer to pure Scrum.

    This presentation will open your eyes to understanding how close you really are to Scrum if you just use this conceptual stepping stone.

  • Liked Charlotte Chang
    keyboard_arrow_down

    Software Development Lessons Learned from the Failures of General Motors

    45 mins
    Talk
    Beginner

    In the mid-late 20th century, industry giants asked themselves; “How do we continuously improve?”, “How do we build a quality product?” and “How do we design for end users?” These are the same challenges that we face as software developers and designers today. Issues with technical innovation, resource constraints, and organizational support, are experienced not only in zeros and ones but were experienced on the road. In the early 20th century, the U.S. was considered a global leader in economic and scientific achievement. After those major innovations in transportation, space exploration, and computer science, American industries focused on manufacturing advantages, such as mass production and repeatable use. Progress was now measured by process improvements, adapting to market changes and ability to pivot. Why has the tech industry continued to boom, while other industries struggle? Using examples from General Motors, this talk examines their failures in reliability, collaboration, and product design. Take a step back in history to relive the experiences of these pioneers, learn from their past failures, and how they apply to crafting software products today.

  • Paul Boos
    Paul Boos
    IT Executive Coach
    Excella
    schedule 11 months ago
    Sold Out!
    45 mins
    Demonstration
    Intermediate

    Government missions are complicated undertakings; wouldn't it be great to be able to understand all of your key components at a glance? This session will review the use of a Business Canvas to capture what your organization does, regardless of size or mission, on a single page.  It will also cover  analyses that can be done using such a canvas, both logically and via experiments, and how this can fit into transformation. The demonstration will also show how decisions you make for improvements  on your canvas can fit into your project or program portfolio.

    This interactive demonstration will walk you through the concepts using a real Government organization.

  • Liked Shawn Faunce
    keyboard_arrow_down

    What You are Doing Wrong with Automated Testing

    45 mins
    Talk
    Beginner

    We firmly believe that automated testing puts the "A" in "Agile". Without an effective suite of automated tests your ability to be truly agile (that is embrace change) can only be based on the hope that your latest change doesn't have unintended consequences. Additionally, without automated tests, you are missing a vital component in getting feedback into the development team's hands. In our travels, we have encountered many organizations that are struggling with automated testing. These organizations are successfully adopting many Agile techniques but are failing when it comes to automated testing. We frequently hear "Automated testing just doesn't work for us" (eerily reminiscent of the days when we would hear, "Agile just doesn't work for us"). From our experience addressing their challenges, we have identified anti-patterns common across these organizations. These anti-patterns look like they should work, but are in fact doing more harm than good.

    This talk is about those anti-patterns. We have given those anti-patterns a name and a face to help organizations understand why they are not getting the benefits from automated testing that others are. We describe several anti-patterns, such as the "Ice Cream Cone", the "Monolith", the "Sunk Cost". We explain why these anti-patterns appear to be good solutions, what makes them attractive, and why they do more harm than good. We talk about the right approach and draw on our experiences helping organizations adopt a robust automated testing strategy that instills confidence and provides fast feedback to the development team. We explain what benefits from automated testing the anti-pattern is preventing. 

  • Chris Li
    Chris Li
    Founder
    SparkPlug Agility
    schedule 10 months ago
    Sold Out!
    45 mins
    Workshop
    Intermediate

    The energy teams that spend performing some sort of estimation exercise among themselves can often burn them out, put them at odds, and not foster any sort of collaboration.  Even worse, when a team is looking to take on some work and someone not a part of the team gives them the estimates rather than creating them as a team.  These sorts of challenges are present on many teams who wish to work in an incremental fashion and can have a seriously negative impact on productivity and team morale.

    In this workshop participants will visit some of the typical scenarios encountered with respect to estimating backlog items, a challenge that many teams encounter on a regular basis. Attendees will revisit what a backlog item represents as well as basic concepts and reasoning behind estimation.  They will then look at four specific pitfalls that make up a tumultuous cycle around how estimates can be misunderstood, miscalculated, misinterpreted and as a result misused. 

    Participants will perform exercises where they identify and share their challenges around estimation and learn a lightweight, fun and interactive approach to backlog item - level estimation that simplifies the approach and focuses on the conversation.  For those looking for an alternative way to visualize these conversations, stop by this workshop to pick up a lightweight technique to bring back to your teams. 

  • Awais Sheikh
    Awais Sheikh
    Business Proces Engineer
    MITRE
    schedule 11 months ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    Empowerment. All of the agile literature focuses on it being a key characteristic of a successful Product Owner. Necessary to ensure decisions can be made quickly and representative of business value. Yet in most environments, particularly in the public sector, the notion of a single Product Owner empowered to represent the multitude of stakeholders isn't feasible. In this session, if you have a situation where you are a Product Owner, or know a Product Owner, who is not in that ideal textbook situation (and even those who are), learn how we can harness the power of classic and emerging innovation methods to put you in a position of success.

  • Liked Dave Nicolette
    keyboard_arrow_down

    The value of diversity in an agile environment

    Dave Nicolette
    Dave Nicolette
    Consultant
    Neo Pragma LLC
    schedule 1 year ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    There is much talk about diversity in the software development field and in the tech industry in general, and yet most teams and organizations comprise mostly a single cultural group. The dominant group may be 20- and 30-something white males, as is common in Silicon Valley, or some other culturally homogenous group, such as H1B holders from the same country, as is common in large corporations. 

    When nearly everyone in an organization has the same general worldview, problem-solving approach, educational background, life experience, and so forth, the organization tends to suffer from groupthink - they can only conceive of a single approach to achieving a goal or solving a problem. When faced with a unique challenge or an unexpected change in circumstances, such an organization often has great difficulty.

    In contrast, a diverse organization can bring to bear a variety of perspectives, experiences, collaboration styles, and problem-solving approaches. The rich blend of differences enables the organization to adapt to change and to overcome unexpected challenges creatively and flexibly. How can we build a more diverse workforce in the software development field?

  • Liked Jason Anders
    keyboard_arrow_down

    Delivering Data to Agile Initiatives - a Fannie Mae Case Study

    Jason Anders
    Jason Anders
    Director Technology
    Fannie Mae
    schedule 10 months ago
    Sold Out!
    45 mins
    Case Study
    Intermediate

    Even for waterfall initiatives where requirements are (allegedly) known, delivering the right data to developers and testers is slow, painful, complicated, and often rife with errors.  For Agile initiatives, this process is even more difficult.  How do you get the right data to your team members during 1 or 2-week sprints when it often takes 3 or more weeks to generate?  Come learn how Fannie Mae has solved this problem for several of its largest Agile efforts, and how Fannie Mae has formally incorporated Data Delivery into its DevOps journey.  Fannie Mae's approach is one that can be reproduced in most organizations, and should prove to be very helpful and eye opening.