location_city Sydney schedule May 22nd 11:05 - 11:35 AM AEST place Red Room people 150 Interested

Transducers -- composable algorithmic transformation decoupled from input or output sources -- are Clojure’s take on data transformation. In this talk we will look at what makes a transducer; push their composability to the limit chasing the panacea of building complex single-pass transformations out of reusable components (eg. calculating a bunch of descriptive statistics like sum, sum of squares, mean, variance, ... in a single pass without resorting to a spaghetti ball fold); explore how the fact they are decoupled from input and output traversal opens up some interesting possibilities as they can be made to work in both online and batch settings; all drawing from practical examples of using Clojure to analize “awkward-size” data.

 
 

Outline/Structure of the Talk

* Clojure at a glance

* Motivation/The problem

* Introducing transducers

* Anatomy of a transducer

** Transducers as wrappers for state

* Composing transducers

** Encoding computation with data and the lisp way

** The importance of single-pass algorithms and performance tradeoffs

* Using transducers in a streaming context

* How transducers change how I write code

** data science vs. engineering

Learning Outcome

* a peak into the dynamic side of the FP family

* a new way to think about datfa transformation and transformation composition

* food for thought about using FP in a data science context

Target Audience

Programers working with data and transformations thereof (especially numerical); programers curious about Clojure

Prerequisites for Attendees

Knowledge of Clojure is not required, but it helps if one is not afraid of parentheses.

Many of the examples will draw from statistics and ETL tasks, so a basic familiarity with the two helps to get a better idea of the context, but it is not mandatory.

Slides


Video


schedule Submitted 5 years ago

  • Eugenia Cheng
    keyboard_arrow_down

    Eugenia Cheng - Category Theory and Life

    60 Mins
    Keynote
    Beginner

    Category theory can be thought of as being "very abstract algebra". It is thought of as "too abstract" by some people, and as "abstract nonsense" by some others. In this talk I will show that while it is abstract, it is far from being nonsense. I will argue that the abstraction has a purpose and that broad applicability is one of the powerful consequences. To demonstrate this, I will show how I apply concepts of category theory to important questions of life such as prejudice, privilege, blame and responsibility. I will introduce the category theory concepts from scratch so no prior knowledge is needed. These concepts will include objects and morphisms, isomorphisms and universal properties.

  • Yaron Minsky
    keyboard_arrow_down

    Yaron Minsky - Reactive Programming with Diff and Patch

    60 Mins
    Keynote
    Intermediate
    Writing reactive programs can be challenging. Such programs need to be able to react efficiently to changes as they stream in from the outside world, which often leads people to make their program logic incremental in an ad-hoc way, directly expressing their logic in terms of how to integrate a change, rather than as an all-at-once
    computation.
    But such ad-hoc incrementalization is complicated, and the resulting programs can be hard to reason about. In this talk, I'll discuss an approach that leverages the fact that functional data structures can often be diffed efficiently. I'll show how the ability to diff and patch can be integrated into a more general reactive programming
    framework to let you write programs which read like simple, all-at-once implementations of your program logic, but which perform like hand-optimized change-oriented programs.
  • Edward Kmett
    keyboard_arrow_down

    Edward Kmett - Combinators Revisited

    45 Mins
    Invited Talk
    Intermediate

    Back in the 80's, one approach to compiling functional programming languages was to compile down to combinators such as SKI. John Hughes' initial work on supercombinators changed the way folks thought about compiling functional languages and caused folks to turn away from this approach by customizing the combinator set to your particular program. Then Lennart Augustsson's work on implementing supercombinators more efficiently sealed the deal. GHC's compilation technique is a descendant of this school of thought.

    But what did we give up to get to where we are? Let's explore a bit of alternate history.

  • Manuel Chakravarty
    keyboard_arrow_down

    Manuel Chakravarty - Welcome to FP Introductory Workshop

    480 Mins
    Introductory Workshop
    Beginner

    Functional programming has become inevitable. New programming languages draw inspiration from the functional paradigm; old programming languages retrofit support for functional programming; and development teams change their coding style to adopt the best functional programming idioms. We are clearly experiencing a paradigm shift in our industry.

    Due to its academic roots, functional programming sometimes seems unapproachable, with unfamiliar jargon, obscure concepts, and bewildering theories. It doesn’t have to be like that.

    In this one-day series of lectures and hands-on workshops, we will translate the jargon, demystify the concepts, and put the theories into practice. There is nothing inherently difficult about functional programming. In fact, its main aim is to simplify programming and to make it more widely accessible. Functional programming is about being able to understand one function without the million lines of code it is a part of. It is about code reuse. It is about modularity and keeping code easy to change and refactor. These are all goals of good program design that every developer appreciates. Based on this common ground, we will explore functional programming together and see how it can help us to achieve these design goals. In fact, by learning the fundamentals of functional programming in Haskell, we can improve program design in mainstream languages, such as Javascript and C++, and even more so, in hybrid languages, such as Scala and Swift.

    Throughout the day, we will explain the most commonly used functional programming terminology. You will learn the fundamentals of Haskell, one of the most popular functional programming languages. In the process, we will look at a lot of concrete code to understand what functional programming is all about and how to use it in your own programs. In the workshops, you will have plenty of opportunity to write code yourself, experiment, and ask questions. It’ll be fun!

    Bring your laptop and your curiosity and by the end of the day, functional programming will be another tool in your toolbox, and you will be ready to enjoy the main YOW! Lambda Jam conference.

  • Yaron Minsky
    keyboard_arrow_down

    Yaron Minsky - Introduction to OCaml

    480 Mins
    Introductory Workshop
    Intermediate
    The goal of this workshop is to get you a basic familiarity with OCaml and the tools you'll need to be effective working in the language. It will be aimed at experienced programmers who don't know OCaml, and don't necessarily know any functional programming.
    The workshop is organized around a set of exercises that should take you through the basics of the language, up through building a simple client/server application using Async RPC.
    In addition to that, we hope to teach people how to use the latest tools for OCaml, including installing dependencies with opam , building your code with Dune, using Merlin for IDE-style functionality like type throwback and go-to-definition, and writing expect tests to visualize what your code is doing. We'll also show you how to write a simple web-game using js_of_ocaml.
  • Colin Fleming
    keyboard_arrow_down

    Colin Fleming - Developing an IDE for Clojure code

    Colin Fleming
    Colin Fleming
    Owner/Developer
    Ideogram Limited
    schedule 5 years ago
    Sold Out!
    30 Mins
    Talk
    Intermediate

    Cursive is an IDE for Clojure, based on the IntelliJ framework. In contrast to the majority of Clojure development environments, Cursive uses static analysis of the source code to work its magic rather than using runtime inspection of a live system via the REPL. IntelliJ provides a sophisticated indexing infrastructure, and this in combination with static analysis allows many interesting features which are difficult or impossible to achieve with a traditional REPL-based environment. Essential code navigation tools such as Find Usages and refactorings such as Rename become possible, and using IntelliJ's code inspections can provide error marking and static analysis of code, right in the editor. This ability to see problems immediately provides an even shorter feedback loop than working in the REPL.

    I'll discuss some of the challenges of developing a traditional IDE for a language as flexible as Clojure, including the implementation details of how to deal with syntax extensions from macros. I'll also talk about the various ups and downs of developing much of it in Clojure, and why some of it is also developed in Kotlin. I'll also talk about why I believe our editors should be syntax aware, not just text based.

  • Mark Hibberd
    keyboard_arrow_down

    Mark Hibberd - Hanging on in Quiet Desperation: Time & Programming

    Mark Hibberd
    Mark Hibberd
    CTO
    Kinesis
    schedule 5 years ago
    Sold Out!
    30 Mins
    Talk
    Intermediate

    Time has a profound impact on the complexity of the systems we build.

    A significant amount of this software complexity comes from either an
    inability to recall previous states or the inability to understand
    how a state was arrived at.

    From the foundations of AI, LISP and functional programming [1], to
    causality in distributed systems [2], to the more grungy practices of
    immutable infrastructure, or the unreasonable effectiveness of
    fact-based approaches to large scale data systems; the ability to
    adequately cope with time, and the change and conflict it inevitable
    creates, is a common thread to being able to build and reason about
    these systems.

    This talk looks at the impact of time on system design. We will walk
    through examples of large-scale systems and their battles with
    complexity. At the end of the talk, the audience should start to see
    the common spectre of time and have an appreciation of how
    understanding time is fundamental to maintaining clarity, correctness
    and reliability in systems.

    [1] Situations, Actions, and Causal Laws
    John McCarthy
    http://www.dtic.mil/dtic/tr/fulltext/u2/785031.pdf
    [2] Times, Clocks and the Ordering of Events in a Distributed System
    Leslie Lamport
    https://amturing.acm.org/p558-lamport.pdf

  • Brian McKenna
    keyboard_arrow_down

    Brian McKenna - Teaching functional programming at Atlassian

    Brian McKenna
    Brian McKenna
    Functional Programmer
    Atlassian
    schedule 5 years ago
    Sold Out!
    30 Mins
    Talk
    Beginner

    My team works on Atlassian Marketplace, a project which started 10 years ago and uses Scala, Haskell and Nix. The Marketplace code uses concepts such as applicative functors, monad transformers and lenses. People joining the team mostly learned concepts as they needed to, through experimentation and asking questions.

    Last year we started hosting functional programming classes for more fundamental and broad understanding. We've been using the Data61 (formerly NICTA) Functional Programming Course with a lot of success. This talk will describe how we teach FP, what we've learned about teaching FP and the challenges we face.

  • Simon Belak
    keyboard_arrow_down

    Simon Belak - clojure.spec: a lisp-flavoured type system

    Simon Belak
    Simon Belak
    Mad Scientist
    Metabase
    schedule 5 years ago
    Sold Out!
    30 Mins
    Talk
    Intermediate

    With the introduction of spec, Clojure got its own distinct spin on a type system. Just as macros add another -time (runtime and compile time) where the full power of the language can be used, spec does to describing data. Ostensibly and implementation of regular expressions for data, it weaves the full power of the language into the task and at the same time -- in typical lisp fashion -- opens the internals to the end user. The result is an entire additional type system that is a first class citizen and accessible at runtime that facilitates validation, generative testing (a la QuickCheck), destructuring (pattern matching into deeply nested data), data macros (recursive transformations of data) and a pluginable error system. And then you can start building on top of it ...

help