How much does your software cost to operate?

Software design and implementation has an uneasy relationship with cost. As software developers, we often dwell on the cost of building software, but far less so on the cost to operate. The current zeitgeist of pay-as-you-go infrastructure makes this particularly apparent where rigid, hard to deploy and adapt software directly equates to larger bills. If we want to take control of operational cost, it is critical that we start to make operational cost a first-class constraint in the design and architecture of modern software.

In this talk we will work through the idea of price-aware architectures, answering: what are the properties of a price-aware architecture? what happens when we design our systems in a way that can leverage variable price and availability of infrastructure? and finally, how do we implement such systems?

Data is a powerful tool. Data-driven systems leveraging modern analytical and predictive techniques can offer significant improvements over static or heuristic driven systems. The question is: how much can you trust your data?

Data collection, processing and aggregation is a challenging task. How do we build confidence in our data? Where did the data come from? How was it generated? What checks have or should be applied? What is affected when it all goes wrong?

This talk looks at the mechanics of maintaining data-quality at scale. Firstly looking at bad-data, what it is and where it comes from. Then diving into the techniques required to detect, avoid and ultimately deal with bad-data. At the end of this talk the audience should come away with an idea of how to design quality data-driven systems that ultimately build confidence and trust rather than inflate expectations.

Web development in Haskell is efficient, contorting, rewarding, frustrating and elegant all at the same time. 12 Months building production quality Haskell has taught many lessons, and in this talk we will discuss the amazing, with the downright painful, and hope to provide a glimpse at what it is like building real software in a purely functional world.

We will look at the Haskell eco-system and tools; Yesod and the other technologies used to build ApiEngine; as well as the social side of Haskell development by looking at what it takes to spin up new developers.

The term zipper is a colloquial used to describe n-hole (most often, 1-hole) contexts. That is, a data structure that has a hole or pointer focused on a specific element with the ability to efficiently traverse to its neighbouring elements, providing an elegant solution for the need to efficiently traverse and modify immutable data structures. Comonads, the categorical dual of Monads, provide an abstraction for dealing with types that allow extraction. That is, the ability to take a value out of a context; for example, the fact that it is a total operation to extract the head of a non-empty list.

In this talk, we will dive into the implementation of purely-functional data structures in Scala. Exploring zippers, Comonads, and their relationship.

Developers are often very good at spotting repetition in their programs at the value level, but for some reason struggle, or are reluctant to apply that same careful analysis at the type level. This talk aims to build up an intuition for spotting when you can factor your types, examining what this gains you and looking at the practicalities for doing so in Scala. For appropriate buzz-word compliance during this talk you will see Scalaz, Monads, Monad transformers, data types and type classes.

Argonaut is a purely-functional library for dealing with JSON in Scala. Argonaut utilises advanced functional programming techniques that emphasise correctness and performance without sacrificing convenience. Argonaut provides zippers and lenses for efficient navigation and modification of immutable data structures; type-class based codecs for robust, composable mappings between Scala data structures and JSON; and data-types for accurately capturing the essence of JSON and the errors that can occur.

In this talk we will take a look at general principles for designing robust, purely functional libraries in Scala. We will then be taking a deeper technical dive into the specifics of Argonaut, examining the abstractions it uses, the specifics of how they are applied and why these techniques are invaluable to quality code.

Purely functional, elegant, correct, incremental and composable stream processing that is CPU and memory efficient. This is our (worthy) goal, but where do we start?

This problem space is being extensively explored across a variety of languages and libraries, each with subtly different trade-offs and not-so subtly different APIs and terminology. However, these libraries share common goals, and most share common ancestry from Oleg Kiselyov’s original Iteratee work or its Free Monad based derivatives.

This talk aims to build up an intuition for stream processing in general by first building up the core concepts and language of stream processing, and then grounding those by carefully examining the trade-offs and internals of several productionised implementations. Of particular interest are the pipes and conduits libraries from the Haskell community, and scalaz-stream from the Scala community.

Functional programming provides a fundamental basis for reasoning about our programs and building out principled abstractions. However, it is not enough on its own and we also rely on other programming tools to aid in constructing programs correctly and efficiently. An advanced example of such a tool is dependent types; that is, types that depend on values. Dependent types can be used to provide strong guarantees about a program’s behaviour that are difficult or impossible to express with traditional type-systems.

Dependent types are gaining traction in mainstream FP languages such as Haskell (via language extensions) and Scala (via path-dependent sub-typing), but disappointingly, they are still less accessible to everyday programming than they can and should be. To help address this problem, Idris is a new programming language built from the ground up with the explicit goal of having better support for dependent types (as well as a number of other useful tools such as totality checking and tactic based theorem proving).

This talk will step through an end-to-end example in Idris, from fundamentals like building type-safe APIs, and performing IO through to building user interfaces. By the end of the talk we will have identified the strengths of this programming model, the practicalities of using them in Idris (in contrast to Scala and Haskell) and produced a battle-hardened, multi-player, tic-tac-toe game ready for production.

Even correct software fails.

So what happens if we shift the focus of functional programming to reliable systems? Let’s attack the hard and ugly set of programming problems, the solutions that don’t naturally fall out from building a neat library. Let’s apply our functional programming toolkit to delivering systems end-to-end.

This is a talk on reliable systems. What it takes to build them, how functional programming can and is being leveraged, and perhaps of more interest, where current approaches are letting us down.

Functional programming is a useful technique. We spend a lot of time discussing it in the context of hard, challenging or interesting problems, but no where nearly enough in the context of mundane problems.

The UNIX shell, or command line interpreter. A simple, but useful program too often mistaken as complex, or mysterious. Almost every programmer is exposed to shells from a users perspective, but far fewer have ever implemented one, or even know where to start.

This talk aims to be a fun look at using functional programming in and around a traditional, some would say mundane, system utility. We will work through the concepts involved with implementing your own shell, and live code our way to a basic functional shell implemented in Haskell.

From this talk, attendees will walk a way with a better understanding of a program they use every day, as well as ideas and inspiration around using functional programming to solve mundane programming problems.

WORKSHOP/CODE JAM – 120mins

Time to get hands-on. In the talk ‘Mundane Utility: A Functional Shell’ we learned the basic responsibilities and structure of a shell, now is the opportunity to build a shell optimized for you.

In this jam, you will be asked to build up your own shell. There will be a starting structure in Haskell, but the problem can be approached in almost any language with decent Unix and process management APIs.

To solve the problem, you will need to tackle: command parsing and validation; concurrent process management; and input/output handling. Throughout the jam we will work through neat functional solutions for each of these, as well as looking at the overall approach to designing a robust functional program.

At the conclusion of the jam we will compare the solutions and the potentially unique features that appear from people tackling this problem in a novel, functional, manner.

Data-driven systems and machine learning continue to be a significant trend across our industry. However, most attempts at these systems face serious difficulties due the tension between the clean, controlled, lab environments where statisticians apply their skills, and the messy unpredictable, production environments where we want to apply their results at scale.

In this talk, we will provide an overview of the machine learning landscape, with an emphasis on the distinction between machine learning as a scientific practice and the larger concept of machine learning systems. Using this base, we will walk through the challenges of taking machine learning out of the lab and applying it successfully in an industrial setting.

By the conclusion of this talk, the audience should take away a better understanding of machine learning as a practice, together with an idea of what it takes to build and deploy machine-learning systems in an environment that deals with real customers and data at scale.

KEYWORDS

Machine Learning, Data, Scale, Deployment

Data-driven systems and machine learning continue to be a significant trend across our industry. However, most attempts at these systems face serious difficulties due the tension between the clean, controlled, lab environments where statisticians apply their skills, and the messy unpredictable, production environments where we want to apply their results at scale.

In this talk, we will provide an overview of the machine learning landscape, with an emphasis on the distinction between machine learning as a scientific practice and the larger concept of machine learning systems. Using this base, we will walk through the challenges of taking machine learning out of the lab and applying it successfully in an industrial setting.

By the conclusion of this talk, the audience should take away a better understanding of machine learning as a practice, together with an idea of what it takes to build and deploy machine-learning systems in an environment that deals with real customers and data at scale.

KEYWORDS

Machine Learning, Data, Scale, Deployment

Data-driven systems and machine learning continue to be a significant trend across our industry. However, most attempts at these systems face serious difficulties due the tension between the clean, controlled, lab environments where statisticians apply their skills, and the messy unpredictable, production environments where we want to apply their results at scale.

In this talk, we will provide an overview of the machine learning landscape, with an emphasis on the distinction between machine learning as a scientific practice and the larger concept of machine learning systems. Using this base, we will walk through the challenges of taking machine learning out of the lab and applying it successfully in an industrial setting.

By the conclusion of this talk, the audience should take away a better understanding of machine learning as a practice, together with an idea of what it takes to build and deploy machine-learning systems in an environment that deals with real customers and data at scale.

KEYWORDS

Machine Learning, Data, Scale, Deployment

Building robust, quality systems is hard. We trade off organizational issues against technical decisions; the ability to deliver quickly against our ability to change; and the ability to build systems easily against the ability to run those systems in production. However, good architectural decisions can free us to choose the right tools and techniques, allowing us to manage these challenges and concentrate on solving real problems rather than our made up ones.

In this talk, we will run through some stereotypical projects, come to terms with legacy systems, and look at the properties of robust architectures. In particular we are interested in how architectures lend themselves to experimentation and change in terms of both function and technology.

We will attempt to ground the discussion with examples from my past projects. Looking at where things have worked well and probably of more interest, where they really have not.

Building robust, quality systems is hard. We trade off organizational issues against technical decisions; the ability to deliver quickly against our ability to change; and the ability to build systems easily against the ability to run those systems in production. However, good architectural decisions can free us to choose the right tools and techniques, allowing us to manage these challenges and concentrate on solving real problems rather than our made up ones.

In this talk, we will run through some stereotypical projects, come to terms with legacy systems, and look at the properties of robust architectures. In particular we are interested in how architectures lend themselves to experimentation and change in terms of both function and technology.

We will attempt to ground the discussion with examples from my past projects. Looking at where things have worked well and probably of more interest, where they really have not.

Building robust, quality systems is hard. We trade off organizational issues against technical decisions; the ability to deliver quickly against our ability to change; and the ability to build systems easily against the ability to run those systems in production. However, good architectural decisions can free us to choose the right tools and techniques, allowing us to manage these challenges and concentrate on solving real problems rather than our made up ones.

In this talk, we will run through some stereotypical projects, come to terms with legacy systems, and look at the properties of robust architectures. In particular we are interested in how architectures lend themselves to experimentation and change in terms of both function and technology.

We will attempt to ground the discussion with examples from my past projects. Looking at where things have worked well and probably of more interest, where they really have not.

As we construct larger or more complex systems, failure and change are ever-present. We need to accept and even embrace these tensions to build software that works and keeps working.

This is a talk on building and operating reliable systems. We will look at how systems fail, particularly in the face of complexity or scale, and build up a set of principles and practices that will help us implement, understand and verify reliable systems.

As we construct larger or more complex systems, failure and change are ever-present. We need to accept and even embrace these tensions to build software that works and keeps working.

This is a talk on building and operating reliable systems. We will look at how systems fail, particularly in the face of complexity or scale, and build up a set of principles and practices that will help us implement, understand and verify reliable systems.

As we construct larger or more complex systems, failure and change are ever-present. We need to accept and even embrace these tensions to build software that works and keeps working.

This is a talk on building and operating reliable systems. We will look at how systems fail, particularly in the face of complexity or scale, and build up a set of principles and practices that will help us implement, understand and verify reliable systems.

Time has a profound impact on the complexity of the systems we build.

A significant amount of this software complexity comes from either an inability to recall previous states or the inability to understand how a state was arrived at.

From the foundations of AI, LISP and functional programming [1], to causality in distributed systems [2], to the more grungy practices of immutable infrastructure, or the unreasonable effectiveness of fact-based approaches to large scale data systems; the ability to adequately cope with time, and the change and conflict it inevitably creates, is a common thread to being able to build and reason about these systems.

This talk looks at the impact of time on system design. We will walk through examples of large-scale systems and their battles with complexity. At the end of the talk, the audience should start to see the common spectre of time and have an appreciation of how understanding time is fundamental to maintaining clarity, correctness and reliability in systems.

[1] Situations, Actions, and Causal Laws
John McCarthy
http://www.dtic.mil/dtic/tr/fulltext/u2/785031.pdf

[2] Times, Clocks and the Ordering of Events in a Distributed System
Leslie Lamport
https://amturing.acm.org/p558-lamport.pdf