Systems That Don't Forget
Software systems form an intrinsic feedback loop, the more we use and rely on a system, the more we demand of that system. This loop drives a cost and complexity that if left unchecked eventually inhibits growth and improvements to the software or in the worst case brings it crashing down.
The crux of this complexity in many (most?) systems is the management of state and data over time, and importantly how different systems co-ordinate around that state. If we look to modern, and even not so modern, software we can see a recurring pattern to dealing with this through the use of immutable, persistent data. We see this pattern from databases and distributed systems through to user interface frameworks. Building complex systems that work correctly, reliably and efficiently often comes down to building systems that don't forget.
In this talk we will look at the design, benefits and challenges of building a system around the ideas of immutable and persistent data using a specific example of a system that I work on to support urban planning & climate policy modelling.