Overview: Streamly is a Haskell library for writing programs in a high level, declarative data flow programming paradigm. It provides a simple API, very close to standard Haskell lists. A program is expressed as a composition of data processing pipes, generally known as streams. Streams can be generated, merged, chained, mapped, zipped, and consumed concurrently – enabling a high level, declarative yet concurrent composition of programs. Programs can be concurrent or non-concurrent without any significant change. Concurrency is auto scaled based on consumption rate. Programmers do not have to be aware of threads, locking or synchronization to write scalable concurrent programs. Streamly provides C like performance, orders of magnitude better compared to existing streaming libraries.

Motivation: Haskell has a beautiful abstraction capability. Haskell lists, several streaming and concurrency libraries are some examples of good programming abstractions. The problem is that concurrency is not handled elegantly and declaratively by these libraries. Moreover, the lack of good performance gives rise to special purpose performance centric libraries like text and bytestring creating more confusion. Streamly unifies, lists, streaming, concurrency, logic programming and reactive programming with unparalleled performance. We can finally have a unified string handling.

Audience: Streamly is designed to express the full spectrum of programs. Do not think that if you are writing a small and simple program it may not be for you. It expresses a small program with the same efficiency, simplicity and elegance as a large scale concurrent application. It unifies many different aspects of special purpose libraries into a single yet simple framework. It is especially suitable for high performance and scalable backend data processing systems. If you use or plan to use any real time streaming or data flow programming frameworks including Apache Flink, Apache Spark or asynchronous/reactive programming frameworks like ReactiveX/RxJs then you may be interested in Streamly.

Streamly is a Haskell library that marries monadic streaming and concurrency
providing an elegant way of doing declarative concurrent dataflow programming.
The Haskell ecosystem has some fine streaming libraries like pipes and conduit.
However, these libraries are inherently serial and provide no support for
concurrency within the streaming framework. Streamly is the first ever haskell
streaming library that defines and implements concurrency semantics within
a monadic streaming framework.

The streaming API of Streamly is surprisingly simple, it is almost the same as
standard Haskell lists, therefore, programmers do not need to learn a new DSL.
By using specific concurrency-style combinators, streams can be generated,
merged, chained, mapped, zipped, and consumed concurrently. Imagine your
program as a pipeline of queues composed in a combination of serial and
parallel configurations, and you can easily control which of these queues to
run concurrently and at what rate. The degree of concurrency is auto scaled
based on the feedback from the stream consumer, or based on a programmer
specified rate limit. Moreover, Streamly does not trade performance with high
level declarative concurrency, it provides excellent non-concurrent and
concurrent performance.

Haskell can and does perform as well as C, sometimes even better. However,
writing high performance software in Haskell is often challenging especially
because performance is sensitive to strictness, inlining and specialization.
This talk focuses on how to write high performance code using Haskell. It is
derived from practical experience writing high performance Haskell libraries. We
will go over some of the experiences from optimizing the "unicode-transforms"
library whose performance rivals the best C library for unicode normalization.
From more recent past, we will go over some learnings from optimizing and
benchmarking "streamly", a high performance concurrent streaming library. We
will discuss systematic approach towards performances improvement, pitfalls and
the tools of the trade.