The Road to a Masterless Multi-node Distributed System in Elixir

location_city Bengaluru schedule Dec 13th 02:45 - 03:30 PM IST place Senate people 12 Interested

In this talk, we will cover our journey from the base to the summit of a masterless multi-node distributed system in Elixir. We’ll talk about the challenges faced, decisions made and how they shaped/evolved the architecture.


Outline/Structure of the Talk

  • Basis of the talk
  • The starting line
  • Challenges and how we dealt with them
  • How the architecture evolved in light of the lessons learnt
  • Where we are at the moment
  • Where we’re headed
  • Q & A

Learning Outcome

This session will teach the audience:

  • What to and what not to do when building a distributed, concurrent and highly available system in Elixir
  • How the Erlang ecosystem makes it easier to build distributed systems

Target Audience

Developers who are interested in building Distributed Systems using the Erlang ecosystem

Prerequisites for Attendees

Developers should be familiar with:

  • Actor Model of Concurrency
  • Erlang/Elixir
  • OTP


schedule Submitted 4 years ago

  • Abhinav Sarkar
    Abhinav Sarkar
    Senior Software Engineer
    schedule 4 years ago
    Sold Out!
    45 Mins

    Easy concurrency is one of the main prophesied benefits of the modern functional programming (FP) languages. But the implementation of concurrency differs widely between different FP languages. In this talk, we shall explore the methods and primitives of concurrency across three FP languages: Haskell, Erlang, and Clojure (with core.async).

    We shall learn about and compare the trade-offs between

    • the green threads and STM channels oriented concurrency of Haskell
    • everything-is-a-process and message-passing actor pattern of Erlang
    • macro-based CSP transformation of Clojure/core.async
  • Nitin Misra

    Nitin Misra - Background Processing in Elixir with GenStage

    Nitin Misra
    Nitin Misra
    Product Engineer
    schedule 4 years ago
    Sold Out!
    45 Mins

    We recently started working on a new Elixir project and had the need for a background processing system.

    We decided to build our own background processing system in Elixir.

    This talk describes why and how we built our own background processing system in Elixir.

    How we use Elixir's GenStage package for adding back-pressure between the producer and consumer stages and how we add support for custom rate-limiting between several pipelines.

  • Govind Krishna Joshi

    Govind Krishna Joshi - DSLs for Free

    Govind Krishna Joshi
    Govind Krishna Joshi
    Partner, Engineer
    schedule 4 years ago
    Sold Out!
    45 Mins
    Case Study

    DSLs allow us to express our code in verbs and nouns that describe our domain. This helps keeping business logic clean of implementation specific details, and allows for more readable code. Few languages have support for easily defining DSLs, for example macros in clojure/lisp, metaprogramming in ruby, but these techniques are not portable to other languages. Free monads is a way to convert a functor into monads, and allow us to create a structure for our desired computation, which can be interpreted later. This also allows us to decouple the definition of a DSL from its interpretation. In this talk we explore the use of Free monads to easily define DSLs and their interpreters.