For a decade of product development relying heavily on test automation, for status from test automation we used information radiators. Jobs run in Jenkins could be visually monitored for staying blue, and team rooms were equipped with large screens covering large sets of jobs. Organization of jobs to feedback on our components, on our dependencies and on end to end experience helped us in making sense into the snapshot as it was changing color to yellow or red indicating issues.

Test Automation is a fairly complex system for us. Just to give you some keywords of it: Python3, Nosetest, Selenium, DVMPS/KVM, Jenkins and Radiators. It serves multiple products and teams, and for my team, runs 200 000 tests a day.

Working to figure out the scale and source of possible problems better, we added self-reporting (telemetry) to our test automation system. We now find it an invaluable source of information that the more traditional test status radiators could never offer us.

In this talk, we look at how test automation telemetry transformed our ways of working on testing as a team, providing visibility to numbers, the scale of problems, trends, timing and naming in case of failures. We also look at the implementation of sending data to a data lake, and discuss the exploratory nature of being able to ask questions about our tests long after they’ve been run.

I know something you don’t know. You know something I don’t know. And I don’t know what you know that I would need to know. This is where individual contributor approach to software development and testing breaks down. Why aren’t we working together, contributing together and learning together? Why do we, often at best, collaborate on the requirements and understanding of what to build, and then step away for implementation, only to come back to test it after?

This talk looks into my experiences in pairing (two people one computer) and mobbing (more than two people one computer), and the wonders of being a non-programming tester whose ideas get translated into code as equal. The journey to get to pairing and mobbing has been a rocky road, with loads of practical tips to offer on how to approach it.

In software development, those who learn the fastest do best. Could pairing and mobbing take us teamwork to the next level by enabling learning between everyone? Lessons specific to skillsets rub in both ways, leaving everyone better off after the experience.

For a decade of software product agile, we had worked in a structure where business responsibility of what to build was allocated to a product owner and the responsibility of how to build it was allocated to a development team. Product owner would maintain a backlog, act as voice of the customers. Until one day we realized that the choice of what to build or fix is hard, and critical to everyone’s success. If we wanted to do it poorly, we delegated it to a single product owner.

We started a no product owner experiment. For three months, we experienced the development team delivering multitudes of value to what we had grown to expect, and innovate customer-oriented solutions in direct collaboration with customers. Team satisfaction and happiness bloomed. The experiment turned into a continuous way of working.

Customer-focused team directly in touch with their customers performs better without a proxy. Join me to learn how the decision power shared for everyone in the team transformed the ability to deliver, and how collaboration is organized with product experts and business representatives.

Delivering continuously - easier said than done. As I joined my team delivering Windows Desktop products two years ago with the aspiration of implementing continuous delivery, I was told it cannot be done. It started from the fact that making a single release was five-day effort, and continued with “no user would allow us to deliver frequently”.

Two years later, making a release takes us two hours and the way we work together looks very different. With the principle of fixing maintenance issues in the latest release has resulted in improved flow forward and savings of effort in supporting small number of releases.

This talk goes through our lessons learned on the journey of shortening release cycles to a continuous daily flow.

We’ve experienced three essentially different sets of practices, with the core difference coming from release frequency. The good place is when we deliver continuously. The soulsucking place is when we deliver on two week cadence. And the insane asylum is when we deliver just once at the end of the project.

Think of it like this: a pool is not just a bigger bathtub. The things you can do in a pool are different to those you can do in a bathtub. While both are containers of water, they serve different purposes. It is the same with release frequency. While it is all still releases, the continuous ones enable different practices.

I’m using exploratory testing to design which tests I leave behind as automated. Creating automation forces me to explore details in a natural way. When an automated test fails, it is an invitation to explore. The two sides of testing, automation, and exploration, complement each other. These intertwine the considerations of the best for today and for the future.

For great testing bringing value now as well as when we are not around, we need to be great at testing - uncovering relevant information - and programming - building maintainable test systems. At the core of all this is learning. With our industry doubling in size every five years, half of us have less than five years of experience. We all start somewhere on our learning journey.

In this talk, we look at the skills-focused path to better testing in the intersection of automation and exploratory testing. We can arrive at the intersection by enhancing our individual skills, or our collaboration skills. What could you do to become one of those testers who companies seek after that work well in the intersection, giving up the false dichotomy?