location_city Sydney schedule May 7th 10:50 - 11:20 AM place Green Room people 64 Interested

We've all made mistakes. With databases, mistakes are particularly costly because they lead to performance bottlenecks, deployment disasters, lost data and intractable technical debt. Join us and learn from my mistakes. You'll hear harrowing tales of schema design blunders that were never rectified, and where recursive SQL is a path to a dark place. You'll learn why databases make lousy queues, and what to use instead. You'll learn the perils of table locking and botched migrations that can cause downtime and data loss. You'll laugh at my futile attempt to tune queries after choosing the wrong database, and why certain workloads work well on some databases, but not on others. Whether you're new to database engineering, or have made all the same errors, hearing about my missteps will help you avoid mistakes in your own data engineering challenges.


Outline/Structure of the Case Study

  • A story about a time I had a chance to fix a schema design mistake, but didn't, and it led to a build-up of technical debt that slowed feature development
  • A story about a time I chose a suboptimal schema for representing a tree, which led to bugs, poor query performance and a dependence on difficult to maintain recursive queries
  • A story about a time I tried to use a database as a job queue, leading to table bloat and contention caused by frequent vacuuming and explicit locking
  • A story about a time I tried to alter a table as part of a production deploy, and it caused downtime because it locked a highly contended table and blocked incoming web requests
  • A story about a time I tried to run an huge analytics query on a transactional database, and couldn't get it to run fast enough necessitating a move to a column-store analytics database

Learning Outcome

The audience will learn about database pitfalls and receive valuable, cogent and rarely-heard advice on database engineering that they can apply to their own jobs. They'll leave with a deeper understanding of how databases work, and how they should and shouldn't be used, that can help them succeed in their own database development challenges.

Target Audience

Anyone who uses a relational database

Prerequisites for Attendees

Relational database fundamentals, including one-to-many relationships, many-to-many relationships, indices, joins and constraints

schedule Submitted 2 years ago