Test your tests: developer practices aren't just for developers.

The growth of automation testing in today’s software development organizations is changing the the way we test applications. Software development practices have matured over the last 30 years, to include all forms of testing to verify software quality. In the last ten years, there has been a huge spike in the adoption of automated tests, effectively replacing some of these manual testing practices, and supplementing many traditional testing activities. Many parts of the software development industry, however, are wary of replacing manual testingwith automated testing. Not only is there often a lack of confidence in the automation tests, many see automated testing as fragile, unmaintainable, and ultimately, something delivering a low return on investment. Max believes that by employing mature software development techniques, we can achieve robust, maintainable, tests, that deliver confidence of the application under test. In addition to discussing how to structure automated tests that are cleaner, more maintainable and efficient, developer testing, and deployment techniques can be used to programmatically verify test correctness. Drawing on his experiences building test automation, test frameworks and advising organizations to adopt test automation, Max will walk us through how to mature your test automation practices.
 
 

Outline/Structure of the Talk

  • Testing Framework vs Testing Tools
  • Testing your Framework
  • Keeping your Framework Updated
  • Continuous Integration for Testing Frameworks
  • Testing your Tools
  • Validation of Tests
  • Testing Pitfalls

Learning Outcome

At the end of the session, attendees will have a better of understanding of development practices, and how to apply them to functional testing frameworks and tests. A good understanding of what needs to be testing in the framework, and what can be overlooked, along with metrics to gather to prove out frameworks. Additionally, attendees will receive tips to help keep tests from registering as false positives or negatives, and get more accurate measurements.

Target Audience

QAs, SDETs, and Test Architects

Prerequisites for Attendees

A working knowledge of best QA and Dev practices will be helpful, but isn't necessary. Some coding experience will be helpful, so keep up with the terminology used, such as unit testing, integration, and static analysis, but again, isn't needed as all terms will be reviewed to ensure the discussion is on a common ground

schedule Submitted 4 years ago

  • Cherie Silas
    keyboard_arrow_down

    Cherie Silas - Power Coaching – Pushing the Boundaries to build better teams

    45 Mins
    Talk
    Intermediate

    Elevator Pitch

    Sometimes teams need more than just questions. They need scrum masters and coaches who are courageous enough to have the hard conversations, challenge their decisions, push them to the next level. During this session we will introduce participants to some anti-patterns that have arisen in the scrum master and agile coaching communities and discuss ways to break free!

    Description:

    Coaching Agile Teams is all about asking questions and allowing them to self organize, right? Well, that's just part of the mission. During this session we will introduce participants to some anti-patterns that have arisen in the scrum master and agile coaching communities and discuss ways to break free!

    Sometimes teams need more than just questions. They need scrum masters and coaches who are courageous enough to have the hard conversations, challenge their decisions, push them to the next level. However, sometimes we push our teams a bit too hard and create negative conflict. It's times like this when we need to demonstrate how to reach out and make the first move to repair the relationship. We will introduce the concept of repair bids to help in this area.

    Lastly, we learn a model to put into practice to create a coaching alliance with teams so you can be in agreement on how you will work together for their best interest and improvement over a period of time.

    The reason we chose to create this session is that over the past few years we have noticed that as people are learning more about coaching they are getting out of balance and believing that the only thing that coaches are allowed to do is ask questions. We've noticed that scrum masters lean so far in the direction of self organization that they no longer believe they can challenge teams to grow or to move beyond where the team decides to be. We believe that the root of the problem rests in the fact that people are learning a bit about coaching but not actually learning how to be a coach. We would like to introduce to the attendees the more direct coaching methods that are available for use such as 1) direct communication, 2) challenging, 3) courageous questions that push the edge of the comfort zone, etc.

    Session is collaborative and includes interaction with the participants throughout. Also has collaborative exercises.

  • Gene Gotimer
    keyboard_arrow_down

    Gene Gotimer - Building the Pipeline of My Dreams

    Gene Gotimer
    Gene Gotimer
    DevSecOps Engineer
    Steampunk
    schedule 4 years ago
    Sold Out!
    45 Mins
    Case Study
    Beginner

    I often suggest to teams that they should be using all sorts of tools in their pipelines- from simple static analysis checks and automated builds to security scans and performance testing. I've done presentations and talks at conferences. I've lobbied to clients. I've commiserated with my colleagues. But I've never put together my dream pipeline in one of my own projects.

    There are always reasons that some tests and tools get left out- our policies won't allow them, they will take too long to get approved, we don't have time, we have bigger problems to deal with, it just isn't what the client is looking for right now. And I usually think, if only I were in charge, I'd make sure we were using those...

    In late 2017 I took over maintenance on an open-source project. Now I have no restrictions. The sky's the limit. No one is around to tell me what I can't do. So why don't I have my dream pipeline in place yet?

    I'll talk about the trade-offs and compromises I made when building out the pipeline. Why I decided to focus on some tools and tests but skipped others, and what I need to do or change to make this delivery process the pipeline I've always dreamed about, now that I have no one else to blame.

  • Richard Mills
    keyboard_arrow_down

    Richard Mills - DevOpsing Your Greenfield: Cultivating New Growth

    45 Mins
    Talk
    Intermediate

    You have a golden gem of an activity. There's a brand new project and your project sponsor says "I want to do some DevOps on our new Agile project!" Sigh. You respond with "Well, how about this? Let's BE Agile and adopt a DevOps approach to structuring our teams, designing our architecture, and leveraging automation to rapidly deliver value to our customers." There. At least we've set the mood.


    Regardless, greenfield projects provide a unique opportunity for us as DevOps professionals. You don't have the established baggage of a legacy project. The project is probably open to modern tools and architectures. The project is trying to set up team structure that will have the right skill sets.


    The problem is: where you do you actually start with greenfield projects? When we introduce DevOps to an existing project (brownfield) we have a unique set of challenges and we can prioritize where to start based on our biggest problems. What do you do when you have a blank page? "Do everything!" Well, what actually makes up "everything" and where do we start?


    Putting a solid DevOps solution in place involves some key things. You can follow the religion of the "Three Ways of DevOps" (fast delivery, fast feedback, constant learning) made popular by Gene Kim, but you still have to start somewhere. In this talk, I'll provide a pragmatic formula to setting up well-integrated teams, establishing a DevOps platform, organically growing an initial DevOps pipeline with continuous integration and continuous delivery, establishing some (useful) standards, and guiding the system architecture to support rapid build, deployment, and testing.

  • Thomas Stiehm
    keyboard_arrow_down

    Thomas Stiehm - Failure is Inevitable But it Isn’t Permanent

    Thomas Stiehm
    Thomas Stiehm
    CTO
    Coveros, Inc.
    schedule 4 years ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    Elevator Pitch:

    Agile Transformation is harder than it needs to be because we often find ways to consciously or subconsciously sabotage our efforts if we can recognize this behavior it is possible to intervene and make a change for the positive.

    Abstract:

    Have you ever been on a project where it seems like team members are preventing the team from getting better? Why do they do that? I don’t know either- a psychologist might have to answer that. What I can tell you about is my experiences in seeing teams become their own worst enemies and unwittingly sabotaging the projects they are trying to make successful. My goal is to help you realize when you or those around you are behaving in a way that is going to lead the team plateauing or even failing. I have often found that many teams can get stuck, or plateau, at a certain point along the continuum of agile maturity. These teams can meander around without getting better or even changing anything for long stretches of time. I have also worked with teams that put so many hurdles in their own way that they had no option but to fail. They often fell back into old patterns and gave up hope that things can get better. As an Agile Coach, I have often felt that one of the most valuable things I can share with the people I coach are my failures. I have worked on Agile projects for a long time, and I have failed in many different ways. Having been through failure, I have learned that to keep getting better you have to recognize the things that you do that lead to plateaus and failures to overcome them. This talk is for coaches and team leads who want to make sure their team isn't getting stuck in a rut, or who are trying to get out of a rut with their health and sanity intact.

    Failure signs and examples

    No process is defined and followed

    • ex. Projects that claim to be agile without any experience or training, or doesn’t have basic agile practices such as retrospectives, I.e. we are agile because we have hour long daily standup meetings.

    Process practices are ignored or removed with no compensating practices

    • ex. Agile practices hold each other together, supporting each other by the value they bring to the project, some teams decide to not do some practices without doing something else to get that value, for instance pair programming provides code review and knowledge transfer, many teams don’t pair program and don’t do code reviews and or knowledge transfer.

    Automation is not valued or planned into work

    • ex. We will automate tests later. Often that later never comes and the team is left with a code base that is hard to maintain and change because you don’t know what your changes break.

    No stakeholder expectations management

    • ex. The only way a project can negotiate scope and or schedule is to actively manage stakeholder expectations. An example of unmanaged expectations is the PO that never says no to a feature request or the executive that decides what must to delivered and when it must be delivered.

    Quality and testing practices are an after thought or short changed on schedule

    • ex. Teams that don’t complete sprint commitments because the testers get coded stories too late in a sprint to do all the required testing and the rest of the team isn’t held responsible to help test.

    No negotiation allowed in deliverables and or schedule

    • ex. Executives that dictate all of the terms of a project before a team is even selected.

    The team doing the work didn’t estimate the work but are held to an estimate

    • Many government projects have such a long procurement cycle that no one from the proposal team is put on the project.

    Part time team members are in the critical path

    • ex. Sometimes people with special skills are needed for a part of a project. If the person is part time but their work is in the critical path the project is in trouble.

    Heavy team turn over

    • ex. Heavy turn over is a sign of a project that isn’t on track, even if it hits its deadlines the quality and output will suffer.

    Political motivations more important than team’s ability to do work

    • ex. If the team is setup to fail for reasons outside the team, they will most likely fail.

    Distraction from issues outside the work that needs to be done

    • ex. Scrum Masters that don’t shield the team from issues outside the work that needs to be done during a sprint will end up with a team that doesn’t hit the mark.

    Examples of what can be done to avoid failed projects:

    Focus on shielding the team from outside influence

    • Have the team focus on the things they can control and prevent outside issues from distracting the team.

    Negotiate delivery with the team

    • The team can develop an understanding of what it can deliver. Trying to make the team do more is going to lower quality and potentially make the project take longer.

    Management of stakeholder expectations

    • Stakeholders always want more, that is their job. Let them ask for anything but set their expectations on what is really going to happen.

    Focus on technical excellence, quality, and automation

    • If you want your teams to get better, have them focus internally on things they can control like technical aspects of the project including quality and automation.

    Hire motivated team members and make it possible for them to work

    • People who care about what they are doing will always be better than the cheapest people that don’t care. Hire people who care.

    Maintain a progressive planning pace for getting requirements ready

    • Agile requires planning at different levels, skipping a level for any reason means there are going to be disconnects between your stakeholders and the people doing the work. Disconnects means the project will not product the results you want.
  • Julie Wyman
    keyboard_arrow_down

    Julie Wyman - Responding to Change over Following a Plan: Agile Lessons from Antarctica

    Julie Wyman
    Julie Wyman
    Agile Coach
    Excella
    schedule 4 years ago
    Sold Out!
    10 Mins
    Lightning Talk
    Beginner

    I spent January in Antarctica hanging out with penguins, whales, and seals. It was about as different from my day-to-day work as can be. And yet, on my long flight home, I couldn’t help but reflect on how well my trip aligned with one specific value of the Agile Manifesto: “Responding to change over following a plan.”

    Antarctica is a place that truly drives home why we need both planning AND, even more importantly, the ability to respond to change. This trip helped me fully appreciate how true this value is - and not just in software development. And after being stuck in Antarctica six days longer than planned, it also built up my empathy for team members struggling with dynamic situations!

  • Jolly Rajan
    keyboard_arrow_down

    Jolly Rajan - Measure It! - How You Can Drive Continuous Improvement With Simple Metrics

    10 Mins
    Lightning Talk
    Intermediate

    You don't need complex metrics that are onerous to track and analyze to drive continuous improvement. Lets look at 6 metrics that are easy for any team to use to accelerate their progress to being a high performing team.

  • Joshua Seckel
    keyboard_arrow_down

    Joshua Seckel - Modern Agile 101 for Government

    Joshua Seckel
    Joshua Seckel
    Specialist Leader
    Deloitte
    schedule 4 years ago
    Sold Out!
    45 Mins
    Talk
    Beginner

    In 2001, a group of software developers got together in Snowbird, UT, and created the Agile Manifesto. The Manifesto was a statement of core value and principles. The core values are:

    • Individuals and interactions over processes and tools
    • Working software over comprehensive documentation
    • Customer collaboration over contract negotiation
    • Responding to change over following a plan

    These four values are supplemented by 12 principles of agile software. The original 17 signatories were joined by thousands of additional people with the ability to sign cut off in 2016.

    These principles are the foundation of much of the work in agile that has occurred in agile development, but have been mostly frozen as practices and agile has evolved.

    Modern Agile has been created recently to update the underlying foundational values and to provide a focus beyond software delivery. Those four values are:

    • Make People Awesome
    • Deliver Value Continuously
    • Make Safety a Prerequisite
    • Experience and Learn Rapidly

    This talk will walk through this reimagining of the agile values and what they mean for delivery within a government context. We will take each value and look at government cultural and technical challenges and opportunities to advance modern development practices.

  • 45 Mins
    Talk
    Intermediate

    The Triple Track Method

    This talk is for Agile teams delivering using the Scrum framework

    Who desire to put the customer as the central force in maximizing product value but struggle to do this in the midst of an intense focus solely on delivery of feature after feature

    Unlike maximizing how many features are shipped by a certain date in a certain budget with limited customer engagement or having separate teams gather customer insights for the Agile delivery teams

    The discussed approach focuses Agile teams on maximizing a positive customer behavior outcome and the resulting business impact through direct customer interaction throughout the delivery cycle

    This results in moving the team towards a collective product ownership mindset, bringing the team and the customer together to achieve optimal outcomes.

  • Bob Duffy
    keyboard_arrow_down

    Bob Duffy - Fannie Mae's SDLC Journey from Waterfall to Agile

    Bob Duffy
    Bob Duffy
    Internal Controls Tech
    Fannie Mae
    schedule 4 years ago
    Sold Out!
    10 Mins
    Lightning Talk
    Intermediate

    A well-defined Software Development Life Cycle (SDLC) is a requirement for many government institutions. However, the typical SDLC process is very "Waterfallish" by nature of it's phase gates and documentation requirements. This talk will explain how the SDLC at Fannie Mae has evolved as the company has transformed from a Waterfall to a lean Agile organization in alignment with Agile best practices.

  • Gene Gotimer
    keyboard_arrow_down

    Gene Gotimer - Build a Better, Faster Pipeline for Software Delivery

    Gene Gotimer
    Gene Gotimer
    DevSecOps Engineer
    Steampunk
    schedule 4 years ago
    Sold Out!
    45 Mins
    Workshop
    Beginner

    The software delivery pipeline is the process of taking features from developers and getting them delivered to customers. The earliest tests should be the quickest and easiest to run, giving developers the fastest feedback. Successive rounds of testing should increase confidence that the code is a viable candidate for production and that more expensive tests—be it time, effort, cost—are justified. Manual testing should be performed toward the end of the pipeline, leaving computers to do as much work as possible before people get involved. Although it is tempting to arrange the delivery pipeline in phases (e.g., functional tests, then acceptance tests, then load and performance tests, then security tests), this can lead to problems progressing down the pipeline.

    In this interactive workshop, we will discuss how to arrange your pipeline, automated or not, and so each round of tests provides just enough testing to give you confidence that the next set of tests is worth the investment. We'll explore how to get the right types of testing into your pipeline at the right points so that you can determine which builds are viable candidates for production.

  • Gene Gotimer
    keyboard_arrow_down

    Gene Gotimer - A Definition of Done for DevSecOps

    Gene Gotimer
    Gene Gotimer
    DevSecOps Engineer
    Steampunk
    schedule 4 years ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    DevOps needs to consider many different aspects of software quality, including security. The term DevSecOps was developed to highlight that security is a focus of the pipeline, not a second-class citizen.

    Fortunately, we can define done for our pipeline so that it includes security. Continuous integration can invoke static analysis tools to test for security errors and check if we are using components with known vulnerabilities. Automated deployments and virtualization make dynamic environments available for testing in a production-like setting. Regression tests can drive traffic through proxies for security analysis. From the code to the systems where we deploy the software, the process can be designed to make sure that we follow security best practices, and not produce insecure software.

    Participants will learn how to construct a definition of done that focuses on security in a DevOps pipeline. They will see how to define security practices that build confidence that they are doing DevSecOps, and how those practices and criteria might mature over time.

  • Glenn Buckholz
    keyboard_arrow_down

    Glenn Buckholz - Failing Faster with Kubernetes and then Recovering

    Glenn Buckholz
    Glenn Buckholz
    Technical Manager
    Coveros
    schedule 4 years ago
    Sold Out!
    45 Mins
    Case Study
    Intermediate

    Kubernetes is quickly gaining popularity, but does the technology deserve so many accolades. In this talk we learn what exactly kubernetes has to offer in the DevOps world. We see that as a service it can provide resources for your DevOps pipeline, automated tests, and the application you are trying to build. We visit an example implementation of such a pipeline and discuss some lessons learned about how to balance these three needs. Additionally, we look at what design considerations your application needs to make, you should consider for your automated testing infrastructure, and how all of this can be leveraged to keep consistent CM while deploying rapidly. Lastly, we explore how threading it all together can help you fail fast and recover quickly as well.

  • Matthew Kleiman
    keyboard_arrow_down

    Matthew Kleiman - Pair Programming: Better Than Adderall

    45 Mins
    Talk
    Beginner

    Not all programmers are comfortable with coding alone for 10 hours a day. Matt Kleiman spent years programming inefficiently in a cubicle. His Attention Deficit Disorder (ADD) and other learning disabilities deterred his success on traditional development teams. Seeking out an environment that would nourish his strengths, he found Pivotal Labs. Pivotal's Extreme Programming practices empowered Matt. Attend his talk to learn how pair programming and test-driven development transforms ADD into an asset in the workplace.

  • Victoria Guido
    keyboard_arrow_down

    Victoria Guido - Avoiding Pitfalls of Non-Technical Managers

    10 Mins
    Lightning Talk
    Beginner

    This talk is intended to help folks who are managing technical projects avoid common pitfalls, and help technical teams better prepare managers for overall project success.

  • Leland Newsom
    keyboard_arrow_down

    Leland Newsom - Sprint with Agile, Deliver with DevOps

    45 Mins
    Talk
    Beginner

    Enterprises want to deliver more value with higher quality at a faster pace. Many development teams have adopted agile frameworks to improve their ability to deliver software. This has led to a local optimization for the development teams and they have become good at delivering potentially shippable increments of their products, but from there, they typically see organizational constraints in moving it to the customer. The development organization is quickly adding features to the queue waiting to be released, but the operations teams are struggling to support fires in production, maintain stability, and provide the environments and infrastructure needed so development teams can move their new functionality forward. The operation team’s focus on stability usually minimizes the number of changes in production thus creating infrequent, large batches being deployed at a planned date. Can Agile and DevOps bring the development and operations teams together to remove the organizational constraints in moving the software to the customer?

    In this session, we’ll talk about the relationship of Agile and DevOps, not as an intersection, but as a progression of capability with development and operation teams working together to remove those constraints. We’ll discuss how using Agile and DevOps practices together, teams can release value faster, with higher quality, and in more stable environments making it safer to deploy.

  • Rupesh Kumar
    keyboard_arrow_down

    Rupesh Kumar - DevSecOps: Building a resilient pipeline

    45 Mins
    Talk
    Beginner

    To all technical members involved with building DevSecOps pipeline this talk is for you.

    In today's competitive world, fixing problems in the pipeline is still a human task and how many times have we seen that when pipeline stops, notifications are sent out and then we wait........ for someone to identify, diagnose and resolve the issue. I am sure we have all been there and done that.

    Is this good enough?

    In my talk I will share some smart ways of infusing self healing into your existing pipeline. We all know it’s good to set thresholds and quality gates in the pipeline, so you can stop the pipeline when the threshold or the gate fails and notify appropriate stakeholders. But in this approach the problem is brought to the attention of the humans to take corrective action what if this issue gets self diagnosed and self healed by the pipeline itself!

    Imagine that....

    I will shows ways on how self-healing can be built into the pipeline by leveraging the data that gets generated from within the pipeline by converting it into information, and the information into knowledge, and the knowledge into insight to make intelligent data driven decisions and remediation.

  • Donald Patti
    keyboard_arrow_down

    Donald Patti / Lisa Brown / Meghana Ekbote / Yogita dhond - Scrum in a Snap: Using Snap Circuits to Excite & Educate Scrum Newcomers

    45 Mins
    Workshop
    Intermediate

    The best way to learn Scrum is by doing, but it can be difficult to simulate Scrum and see how well Scrum helps team overcome technical hurdles without actually building something technically challenging. Lego's have always been a fun option for introducing Scrum, but it's difficult to recreate technical impediments, the need for spikes and managing technical debt with our tried and true friend, the box of Lego's.

    Arguably, a better alternative might be Snap Circuits, a toy designed to introduce children to electronics in a fun and easy-to-understand format. Like Lego's, adults gravitate toward Snap Circuits because they are colorful, quickly understood and snap together with ease.

    But, Snap Circuits have the added advantage of requiring a small amount of technical learning during the simulation that make it a closer match to the technical obstacles faced by a typical Scrum team.

    In this workshop, you'll learn one "Scrum in a Snap" simulation exercise. In addition, we'll provide you with a few other "Scrum in a Snap" ideas and encourage you to experiment on your own. Four lucky attendees will also win their own Snap Circuits kit so they can develop their own Scrum games.

    Past participants in "Scrum in a Snap" have said "The best Scrum exercise I've ever done", "I can't believe how much it's like coding - without actually coding", "What a blast - I'll never forget this activity!" and "Where can I buy one?"

    Attend this workshop to see why.

  • Hunter Willett
    keyboard_arrow_down

    Hunter Willett - "Frameworks are Like New Golf Clubs, They Won't Fix a Terrible Swing" How Understanding the Principles of Agile is the First Step

    Hunter Willett
    Hunter Willett
    Agile Coach
    CapTech
    schedule 4 years ago
    Sold Out!
    45 Mins
    Case Study
    Beginner

    We have all been there, a shiny new and improved framework is released and we must implement it, but is this always the answer to improve your Agile organization? Frameworks are needed and provide guidelines for teams but if the teams/companies do not follow the principles and fundamentals of the Agile Manifesto it makes it very difficult for the framework to be successful. The belief is that switching up the specific framework is the answer but they soon realize that the framework is not the main issue that is the driving force. This can leave the teams/companies in a tough situation after committing to a framework that they are not ready for.

    I will be sharing my experiences across multiple different companies on how this assumption has let them down and how we had to return the teams back to the fundamentals to solve the issues they are experiencing.

  • David Bujard
    keyboard_arrow_down

    David Bujard / Chris Meaker / David Fogel - Impossible deadlines? Fail safely, learn rapidly with Spaceteam

    45 Mins
    Workshop
    Beginner

    Communication chaos under looming deadlines - sound familiar? We'll level up our teamwork, practice rapid learning, and identify ways to calm the chaos and focus on getting to done, all using Spaceteam, a chaotic and collaborative card game.

    You'll work with your teammates to repair a failing spaceship before it falls into a black hole. in order to escape, you'll communicate problems, request help, assist colleagues and respond to constant change -- all in five minutes!

    You'll learn from your failures, improve as a team, and gain insights into what helps organizations and teams collaborate effectively and achieve flow.

  • Jonathan Kauffman
    keyboard_arrow_down

    Jonathan Kauffman - Accept Kanban, or Keep Living a Scrum Lie?

    Jonathan Kauffman
    Jonathan Kauffman
    Consultant
    Coveros, Inc.
    schedule 4 years ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    Are you lying to yourself? Do you keep using Scrum even though you should have switched to Kanban months ago? Scrum is a popular Agile development methodology, but if it doesn’t make sense for your team, then you might be better off switching to Kanban. I will describe my experience working with a team transitioning to Agile and attempting to use Scrum. We initially did a good job at planning, but external dependencies and unplanned work disrupted our commitments. We stopped planning and fell into the pattern of moving most of the stories to the next sprint at the end of every sprint. I will discuss some telltale signs to help you identify when you shouldn’t be using Scrum and might benefit from a Lean methodology like Kanban. I will also provide you with some practical suggestions that you can use to help your team make the transition from Scrum to Kanban.

help