Not so Weird Science: applying the scientific method to software delivery

Hang on, what does the scientific method have to do with agile?  Agile delivery includes fast feedback loops and experimentation.  Many LAST attendees will have some exposure to A/B testing.  My experience of this has been long debates and hand wringing once the results are in, factions using the results to justify their starting position and  frequent use of the phrase "statistical significance" .  At one point I asked a group "who would be comfortable providing a definition of statistical significance" and received no takers.  Something is not right!

This talk will discuss one teams efforts to get a bit more scientific with their delivery experiments and why it's been worthwhile so far.


Outline/Structure of the Talk

 talk plus questions

intro, team current state and frustration
brief overview of the scientific method
teams first attempt at applying to an experiment and the pitfalls
teams second attempt and the benefits of increase rigor 
the result, not of the experiment but how the team now want to operate

Learning Outcome

 An overview of the scientific method and basic statistical concepts. The benefits of applying these to software delivery.

Target Audience

People looking to use experimentation to inform their decision making or perhaps already dabbling with A/B testing and feel something isn't quite right.

schedule Submitted 3 years ago