Test Automation Strategies and Frameworks: What Should Your Team Do?

Agile practices have done a magnificent job of speeding up the software development process. Unfortunately, simply applying agile practices to testing isn't enough to keep testers at the same pace. Test automation is necessary to support agile delivery. Max Saperstone explores popular test automation frameworks and shares the benefits of applying these frameworks, their implementation strategies, and best usage practices. Focusing on the pros and cons of each framework, Max discusses data-driven, keyword-driven, and action-driven approaches. Find out which framework and automation strategy are most beneficial for specific situations. Although this presentation is tool agnostic, Max demonstrates automation with examples from current tooling options. If you are new to test automation or trying to optimize your current automation strategy, this session is for you.

 

 
5 favorite thumb_down thumb_up 3 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

  • Introduction
  • Test Automation Planning
    • Test Automation Objectives
    • Test Automation Scope
    • Testing Types
  • Test Automation Framework Selection
    • What is a Framework
    • Automation Framework Types
    • Test Automation Tool Selection
  • Test Automation Strategy
    • Selecting Tests for Automation
    • Determining what to Automate
    • Data Management
    • Traceability
    • Best Practices
    • Parameterized Testing
  • What Testing Framework is Right For Me?
  • Questions

Learning Outcome

At the end of this session, students will understand all of the intricacies of choosing a framework, what to look for, what capabilities exist, and be better prepared to make a selection on a test automation framework

Target Audience

Testers and QA. Automation Engineers responsible for setting up tooling

schedule Submitted 2 years ago

Comments Subscribe to Comments

comment Comment on this Proposal
  • George Dinwiddie
    By George Dinwiddie  ~  2 years ago
    reply Reply

    Hi, Max. The title intrigues me, but the outline looks pretty generic. It could be great or it could be anemic. Can you give us a bit more information to help the review process?

     

    • Max Saperstone
      By Max Saperstone  ~  2 years ago
      reply Reply

      The Test Strategy is (in my opinion) the most interesting part of this topic. One of the big things discussed, is not just what to test, but how to structure your tests to get the most out of them. I discuss multi-tiered testing frameworks, how to make them reusable and managable, to truly simplify your tests (re-use over multiple tiers). I will run through an example of a custom developed hybrid framework that is based on cucumber, showing how one test can test the back end, middle tier (API), and front end (web application). 

  • Donald Patti
    By Donald Patti  ~  2 years ago
    reply Reply

    This would be a great addition to the conference and touches on an underserved topic. Hope it makes it.


  • Liked Thad Scheer
    keyboard_arrow_down

    Thad Scheer - Un-beach the whale and turbocharge productivity in your post-Agile organization

    45 mins
    Talk
    Intermediate

    When you transitioned to Agile you solicited the best advice, updated your development tools, hired coaches, installed furniture, and embraced painful cultural changes. Now, a few years later, you wonder whether Agile is working. Are you realizing the productivity you are paying for, or did Agile flop? With so many people developing software every day you expect more to get done. Your advisors tell you not to worry about productivity, this is how it’s supposed to be. Questioning themselves, many executives are awakening to these gut feelings of disillusionment in their post-transition organizations. How much productivity should they expect from Agile teams? How do you know if Agile is working? Can Agile organizations be slow and unwieldy despite their Agileness? This session offers a strategic business management perspective about the honest reality of Agile in a post-transition organization. Attendees will learn to recognize the signals of poor Agile productivity and how to fix development so it stays fixed.

  • Liked Theresa Smith
    keyboard_arrow_down

    Theresa Smith - Product Design with Intent: How to Drive Product Design in an Agile Project

    45 mins
    Talk
    Beginner

    When design is based on random choices, the end product is an assembly of random elements that have little or nothing in common. But when design forces all elements to work together then it makes a single, powerful, and meaningful impression to the user. While agile can get the job done faster, it doesn’t help guide design choices for a software product.    

    This session presents a design driven approach called Strong Center Design that incorporates design into an agile workflow.

    If you have an interest in improving design of your software products, then this is the session for you.

  • Liked Donald Patti
    keyboard_arrow_down

    Donald Patti - Leaning Up: Eliminating the Seven Wastes in your Agile Shop

    45 mins
    Workshop
    Beginner
    When many of us hear "Lean" we think of Kanban, but it's clearly more than that.  In this session, I'll go beyond the Kanban and explore Lean's seven wastes, defining each one and providing concrete examples.  Then, we'll conduct a "Lean Up" activity to help you ferret out wastes that you can take back and apply in your own Agile shop.
     
  • 45 mins
    Talk
    Beginner

    The Zombie Retrospective - presented by Tommie Adams 

    So they say the retrospective is one of the strongest and most powerful tools in the agile scrum methodology tool kit, and is often overlooked or skipped. So how does a scrum master find ways to creatively explain and express the importance of this agile scrum ceremony, or even the basics of agile scrum in general. How does the scrum master explain the importance of banding together as a team in this brave new agile scrum world.  In many organizations, nowadays, the teams are even made up of outside vendors as well as in house associates. So how do you even start to pique the interest and the importance of team collaboration to a bunch of folks who are strangers to one another on a agile scrum team?  Even more specifically, how do you explain how the retrospective ceremony will help improve the way they work with one another over time?

    My answer: ZOMBIES!!!  Everyone loves zombies, right?  So come, take a bite!

    Tommie works for Marriott International in Bethesda MD. His background is in theater and communication which he studied at Grinnell College in Iowa. He has worked for Marriott International for 26 years with jobs ranging from reservation sales associate, to group sales manager, to functional IT tester to his current position as scrum master for the Marriott Rewards Agile Scrum Team. A native of Omaha, Nebraska, his hobbies include photography, cello and learning the ukulele, (you know, in case you were curious.) 

  • 45 mins
    Workshop
    Intermediate

    Most scrum teams create effort estimates, often using story points. As a product owner, you also want to have an estimate of the business value of each user story. Business value estimates help you create a more rational backlog and maximize the value the team delivers.

    This workshop explores the art and science of estimating the business value of user stories. Participants will gain an understanding of the essence of business value,
and why it is more complex than just revenue or profit. Then we will then learn a surprisingly simple technique to estimate and quantify business value.

     

  • Liked Christy Hermansen
    keyboard_arrow_down

    Christy Hermansen - Inside the GSA – a Case Study of user-centered Agile in a high-profile government agency

    45 mins
    Case Study
    Intermediate

    This unique journey will transport you deep inside the world of the General Services Administration (GSA) Integrated Award Environment (IAE).  You will see how user-centered Agile is transforming the way software applications are engineered, how users' voices have been integrated with large-scale Agile development, and what issues we encountered along the way.

    When Eric Schmidt, former CEO of Google, predicted, "Everything in the future online is going to look like a multiplayer game," perhaps he was envisioning a user community such as ours.  The IAE family of software applications have more than a million users representing federal, state, local, and tribal government organizations; congressional staff; large and small businesses; universities, schools, and hospitals; non-profit organizations; foreign entities; private citizens and others.  Our greatest challenge is the diversity of our user base, resembling a massive multiplayer game in many ways. 

    This case study looks inside a major reengineering effort to migrate 10 legacy applications into an integrated environment while at the same time transitioning from Waterfall to Agile development.  It tells the story of how IAE users have shaped our transformation thus far.  

     

     

  • Liked Scott Pringle
    keyboard_arrow_down

    Scott Pringle - Horseshoes, Hand Grenades, and Agile

    45 mins
    Talk
    Intermediate

     

    "My end date is fixed."
    "My customer gave me 1000 traditional requirements."
    “I have a lot of algorithms.”
    “The user interface is limited – most of the work is on the back end.”
    “My customer has a mission to execute, they cannot be here every day.”

    Like in Horseshoes and Hand Grenades, being close in Agile has great value.  We need not abandon our brethren in the trenches just because some of the most recognizable practices are out of play.  The great principles of Agile help in even the most difficult environments.  Shipping great software while under some traditional constraints is a true test of the movement – and we have a track record of doing just that.  We have employed Stealth Agile, Green Box Agile, Cafeteria Agile, Agile Pathfinding, and even Agile Treason in order to deliver in less than ideal circumstances.

    A series of actual scenarios will reveal the adaptations to Agile practices that kept us close to principles.  A small set of practices are present in so many of our projects that we consider them to be part of our Agile core.  Sometimes we rename them, sometimes we feather them, sometimes we disguise them, but they are always present and are pivotal to success.  Chief among these are the various practices that enable and encourage rapid feedback at multiple levels including Customer, Product, System, and Development.

  • Liked Thomas Stiehm
    keyboard_arrow_down

    Thomas Stiehm - How to add Application Security to your Agile Practices

    Thomas Stiehm
    Thomas Stiehm
    CTO
    Coveros, Inc.
    schedule 2 years ago
    Sold Out!
    45 mins
    Talk
    Intermediate

    The Internet is full of insecure applications that cost organizations time and money, while damaging their reputations when their systems are compromised. We need to build secure applications as never before. While at the same time Agile Software Development practices are moving into the mainstream because they offer companies a faster path to getting their software in the hands of their customers. While security and agility may appear to be in natural opposites that don’t mix well, they don’t need to be. Learn how to integrate application security practices into your Agile practices in a way that doesn’t compromise either. Join Tom to explore real-world examples of secure application development practices integrated into the regular cycle of iterative development used in Agile projects. Learn to marry Agile development with application security practices in a way that best leverages the strengths of both.

  • Liked Erik E. Stein
    keyboard_arrow_down

    Erik E. Stein - Size Does Matter: A New Metric to Keep Your Architecture Agile.

    45 mins
    Talk
    Intermediate

    Large codebases have structural problems that do not occur in smaller codebases. These problems are emergent in nature and typically not noticed until some indirect symptoms rear their ugly heads. (“What do you mean it will take six weeks to change that!?”) Even then, the root causes are often not identified and, regardless, are costly to correct. Left untreated, these structural problems eventually make changes to code so costly that the software becomes brittle and unchangeable within the cost and time constraints of the business. This is the software equivalent of hardened arteries, and, like the medical condition, it is a silent killer.

    The traditional metrics of architecture complexity are insensitive to, or worse, favor structural changes that increase codebase complexity. Our recent research has identified and characterized this phenomenon and we offer a new metric, the Layer Respect Metric, which sheds light on this problem and helps us balance our uses of the traditional measures of architectural complexity to ensure our codebases are as agile as our teams.

  • Liked Donald Patti
    keyboard_arrow_down

    Donald Patti - Surviving Vendorfall: Strategies for Dealing with a Vendor who's Waterfall

    45 mins
    Workshop
    Intermediate

    As more of us go the route of Agile, it's becoming increasingly common to deal with vendors and external teams who still follow phase-based approaches to delivering work, like Waterfall. Or, they follow a bastardized version of Agile that looks a lot like the old SDLC.  In this session, I'll describe some common and not-so-common approaches that have been applied to dealing with Vendorfall, explain which are more successful; and then hand control over to attendees as they grapple with their own Vendorfall scenario.

  • Liked Max Saperstone
    keyboard_arrow_down

    Max Saperstone - Testing with a Rooted Mobile Device

    45 mins
    Talk
    Advanced

    Traditional applications are tested through the GUI and through all exposed APIs. However, typical mobile app testing is only done through the front-end GUI. In addition, performance and security details are not readily available from the mobile device. Max Saperstone demonstrates some benefits of testing a native mobile application on a rooted device—one with privileged access control. Although Max does not describe how to root a device, he shares how to access back-end processes and test at this detailed level. He discusses the technical controls made available through a rooted device—together with its auditing, logging, and monitoring—and describes the gathering of additional metrics. Max demonstrates tools for penetration testing, sniffing, and network hacking; shares how to access application data directly; and shows how data security is implemented for the application. Learn how to use the admin rights associated with a rooted device to examine device performance and to simulate interrupts and system faults.

     

  • Liked Dave Rooney
    keyboard_arrow_down

    Dave Rooney - Emergent Design with Test-Driven Development

    90 mins
    Workshop
    Beginner

    This workshop shows how Test-Driven Development (TDD) is used to enable emergent design. Using a simple but representative example in Java, the presenter will demonstrate how a low-level design naturally emerges when using the TDD cycle of test/code/refactor. The audience will be involved by suggesting the next steps and also by pairing with the presenter.

    Note that the goal of the session isn't necessarily to have a complete working example at the end, but to illustrate the process of low-level design through TDD.

  • Liked Ben Pick
    keyboard_arrow_down

    Ben Pick - Testing for Security: The oft forgotten aspect of DevOps

    45 mins
    Talk
    Intermediate

    Agile development and DevOps churn through testing at a rate that is impossible for a human to keep up with. Security tools are often designed to have someone at the helm, targeting the systems and applications or performing time intensive penetration tests.

    What if there were a way to layer in security as applications are being developed? It is unreasonable to believe that automation can completely replace a knowledgeable security tester, but much of the groundwork and preliminary analysis can be incorporated into the software lifecycle.

    If nothing else, these tools and methods will help prevent completing an application, only to discover security findings that cannot be resolved before being released.