Service Discovery using CRDTs
Service Discovery is an important aspect of microservices, particularly when using containers on elastic infrastructure. Most popular approaches for service discovery require a standalone fault tolerant cluster like Consul, Zookeeper or etcd.
If you are already using Akka as a clustering platform for your services, there is an interesting alternative available for Service Discovery. CRDTs, or Conflict-free Replicated Data Types are a relatively new primitive in distributed systems. CRDTs allows concurrent updates without coordination and has provisions for tuning consistency levels. 'Distributed Data' extension natively supports CRDTs on top of Akka-Cluster.
We used CRDTs in Akka to build the service registration and discovery mechanism for TMT's (Thirty Meter Telescope) software platform. TMT will be world's largest optical telescope once operational. The ongoing software work is open source and available on github. We would like to share the details of our implementation and also compare it existing approaches.
Outline/structure of the Session
- What and Why of Service Discovery
- Popular approaches (Consul, Zookeeper, etcd)
- Introduction to CRDTs
- Introduction to TMT project.
- TMT project's design for CRDT based service discovery
- Comparisons and pros/cons
- Demo only if time permits!
- Why are CRDTs so cool
- Why are distributed systems so tricky
- Why is service discovery so important
- What is TMT project
Erlang/Akka enthusiasts, Distributed systems geeks, Curious about CRDTs.
schedule Submitted 1 month ago
People who liked this proposal, also liked:
The path to generic endpoints using ShapelessMaria Livia Chiorean
schedule 3 months agoSold Out!
Programming in Scala is sometimes very time consuming. Finding the most efficient way to solve a problem can end up in days of frustration. This talk is a story of trail and error with a twist at the end. It's a story of API endpoints, generic programming, Shapeless and what happens when they all come together. Everything with examples and plenty of code.
From Haskell 98 to Modern Haskell - Making sense of the Haskell type systemRyan Lemmer
schedule 1 month agoSold Out!
There are several great books and courses that serve as an introduction to functional programming in Haskell. Typically, these introductions focus on the original core of Haskell ("Haskell 98").
However, going from there into real world Haskell systems can be a bewildering experience. This is because Haskell has evolved significantly since Haskell 98. In this talk we will investigate the key advances in the type-system that make up "modern Haskell".
- type-system extensions: Rank-n types, GADTs, functional dependencies, ...
- kind-system extensions: type functions, kind polymorphism, type promotion, ...
By taking a historical perspective we will look at why new language extensions were introduced, and the context in which they first appeared. This approach gives us a layered view of seemingly disparate language features.
We will see that the evolution of the type system has been a steady march from "programming with types" to type-level programming, and ultimately dependently-typed programming in Haskell.
(This talk will draw from the book "Haskell Design Patterns")
More is different: Mistaking trees for ForestShripad Agashe
schedule 1 month agoSold Out!
Generally scientific approach is kept limited to only reductionist approach i.e. trying to understand the system by understanding its parts. Thats how we have been taught to think. However this construct fails in non trivial systems. Often the property of the system can not be reduced to any property of its constituent parts. To take an example “pressure” is not a property individual atoms/molecules but that of a gas. In any complex system it is important to identify relationship between the property of the system and property of its parts. This relationship is known as emergence in complex systems.
In this talk I would cover variety of techniques I’ve used to identify emergent properties. It helped me unravel knotty designs and truly understand “deep insight” aspect of domain driven design. I feel this topic is important in the context of functional programming as functions are touted “pure” abstractions: missing the point of interaction.