Is Agile Data Science a thing now?
How come there’s no standard text on how to operate a Data Science team? At its current scale this is a young practice without a widely accepted mode of operation. Because so many practitioners are housed in technology shops, we tend to align our delivery cycles with developers… and hence with the Agile framework.
I will argue that if a data team fits within Agile it is probably not performing data science but operational analytics—a separate and venerable practice, and a requisite for data science. To ‘do’ science we need a fair bit of leeway, although not a complete lack of boundaries. It’s a tricky balance.
In this talk I will share my experience as a data scientist in a variety of circumstances: in foundational, service, and advisory roles. I will also bring some parallels from my past life in scientific research to discuss how I think data science should be performed at scale. And I will share my current Agile-ish process at Atlassian.
Outline/Structure of the Talk
I will start by establishing that most mature technical crafts have one or more accepted, mainstream methodologies. Agile is the main framework used for developing software, and it is being increasingly applied to analytics.
With that in mind I will question whether this is an appropriate framework for data. I will bring a few snippets from colleagues to convey a sentiment more general than my own, but one that exists in the community. Along with that I will go through what I have learned from a set of roles I've had in the field, and focus on what worked and what did not.
At the culmination of the talk I will focus on my current method, a bit of an 'Scrumban' approach to data science operation at the Enterprise level. This has worked better than any method I've tried before, both for me and my stakeholders. It combines quantifiable delivery, which is the business objective (and fundamental limitation) with the wiggle room that practitioners of this creative craft often crave.
How to work better with your stakeholders. It takes a little bit of a shared context and education, and empathy to establish a great set of patterns in Data Science.
Managers and practitioners of data science.