Streamly is a Haskell library that marries monadic streaming and concurrency
providing an elegant way of doing declarative concurrent dataflow programming.
The Haskell ecosystem has some fine streaming libraries like pipes and conduit.
However, these libraries are inherently serial and provide no support for
concurrency within the streaming framework. Streamly is the first ever haskell
streaming library that defines and implements concurrency semantics within
a monadic streaming framework.

The streaming API of Streamly is surprisingly simple, it is almost the same as
standard Haskell lists, therefore, programmers do not need to learn a new DSL.
By using specific concurrency-style combinators, streams can be generated,
merged, chained, mapped, zipped, and consumed concurrently. Imagine your
program as a pipeline of queues composed in a combination of serial and
parallel configurations, and you can easily control which of these queues to
run concurrently and at what rate. The degree of concurrency is auto scaled
based on the feedback from the stream consumer, or based on a programmer
specified rate limit. Moreover, Streamly does not trade performance with high
level declarative concurrency, it provides excellent non-concurrent and
concurrent performance.


Outline/Structure of the Talk

In this talk we will introduce Streamly, its streaming API and how concurrency
fits in the streaming framework. We will go through some examples and see how
Streamly can express complex real world programs in a simple and concise
manner. We will also compare Streamly with some existing popular streaming
libraries and concurrency mechanisms.

* Overview of streaming and concurrency
* Motivation for Streamly
* Streaming API overview
* Concurrency overview
* Non-determinism and logic programming
* Functional reactive programming
* Expressing a real world program
* Functionality covered by streamly
* Comparison and interworking with other packages
* Performance
* Future Work

Learning Outcome

After this talk one should be able to understand why and where to use Streamly, and how to write streaming and concurrent programs using Streamly. One should be able to understand what advantages Streamly provides over existing concurrency and streaming facilities available in the Haskell ecosystem.

Target Audience

Streamly is a general programming framework, therefore, this session is for everyone whether they are beginners or advanced programmers.

Prerequisites for Attendees

A basic knowledge of Haskell and how lists work in Haskell should be enough to appreciate this session.

schedule Submitted 1 year ago

Public Feedback

comment Suggest improvements to the Speaker
  • Saurabh Nanda
    By Saurabh Nanda  ~  1 year ago
    reply Reply

    This is highly personal opinions on the content/outline of the talk. Please consider them as such, and not formal requests to alter your talk.

    In the first functional-conf that I attended, there were two talks on streaming libraries IIRC. At that time I was unable to appreciate the _need_ for streaming libraries in the first place. IMO, a good way to explain the need for streaming libraries is to write a simple version of some very real-world code, and show how it blows up when input-size increases. Something that a beginner would end-up writing without realising what s/he is doing.

    Next, would be to similarly explain the problem with forkIO and aysnc -- I believe that they are the most commonly used threading primitives that any newbie would reach-out for, right?

    A logical next step would be to introduce streamly as the solution for both the problems. Show code written in streamly for the problem already discussed earlier along with CPU/memory benchmarks.

    IMO, motivating the problem is of more beneficial than detailing the library features itelf. If the talk can drive adoption of this library, then users will discover relevant features as-and-when required by their problem at hand.

    • Harendra Kumar
      By Harendra Kumar  ~  1 year ago
      reply Reply

      Points taken, I agree with the intention.  We will focus on the motivation but with emphasis on the concurrency aspects, we can have a brief motivation for the streaming problem itself, but if we go into more details, due to lack of time, we won't be able to cover the new stuff. In fact the concurrency aspect makes streaming even more relevant and a big motivation for programmers to use the streaming paradigm as it simplifies concurrent programs and make them look more like non-concurrent programs. One reason for lack of appreciation earlier could be due to an entirely new way of programming, streamly is very much like the familiar standard list API and not much new to learn as an API. It can be considered as an extension of lists which are pure to monadic effects.

      I am planning to start with a real world example right from the motivation section. Most of the talk focuses more on how to express concurrent programs concisely with examples rather than what the API looks like or what it does, there is ample documentation for that.

  • Liked Michael Snoyman

    Michael Snoyman - Functional Programming for the Long Haul

    Michael Snoyman
    Michael Snoyman
    VP, Engineering
    FP Complete
    schedule 1 year ago
    Sold Out!
    45 Mins

    How do you decide whether a programming language is worth using or not? By necessity, such decisions are usually based on assessments that can be made relatively quickly: the ease of using the language, how productive you feel in the first week, and so on. Unfortunately, this tells us very little about the costs involved in continuing to maintain a project past that initial phase. And in reality, the vast majority of time spent on most projects is spent in those later phases.

    I'm going to claim, based on my own experience and analysis of language features, that functional programming in general, and Haskell in particular, are well suited for improving this long tail of projects. We need languages and programming techniques that allow broad codebase refactorings, significant requirements changes, improving performance in hotspots of the code, and reduced debug time. I believe Haskell checks these boxes.

  • Liked Harendra Kumar

    Harendra Kumar - High Performance Haskell

    45 Mins

    Haskell can and does perform as well as C, sometimes even better. However,
    writing high performance software in Haskell is often challenging especially
    because performance is sensitive to strictness, inlining and specialization.
    This talk focuses on how to write high performance code using Haskell. It is
    derived from practical experience writing high performance Haskell libraries. We
    will go over some of the experiences from optimizing the "unicode-transforms"
    library whose performance rivals the best C library for unicode normalization.
    From more recent past, we will go over some learnings from optimizing and
    benchmarking "streamly", a high performance concurrent streaming library. We
    will discuss systematic approach towards performances improvement, pitfalls and
    the tools of the trade.

  • Liked Michael Snoyman

    Michael Snoyman - Applied Haskell Workshop

    Michael Snoyman
    Michael Snoyman
    VP, Engineering
    FP Complete
    schedule 1 year ago
    Sold Out!
    480 Mins

    This full day workshop will focus on applying Haskell to normal, everyday programming. We'll be focusing on getting comfortable with common tasks, libraries, and paradigms, including:

    • Understanding strictness, laziness, and evaluation
    • Data structures
    • Structuring applications
    • Concurrency and mutability
    • Library recommendations

    By the end of the workshop, you should feel confident in working on production Haskell codebases. While we obviously cannot cover all topics in Haskell in one day, the goal is to empower attendees with sufficient knowledge to continue developing their Haskell skillset through writing real applications.

  • Liked Zubin Duggal

    Zubin Duggal - The Future of Haskell Tooling

    20 Mins

    Haskell tooling, particularly editor and IDE integration has been sorely lacking for years. This talk will be a broad overview of the state of Haskell tooling, and the challenges faced by it. It will be derived from the experience of working on two summer of code projects trying to improve Haskell tooling.

    For tooling users, we will discuss different Haskell tools available today(intero, haskell-ide-engine, ghcid etc.), their differences and trade-offs with respect to build tool support(stack, cabal, new-build, nix, make etc.), editor integration, features, limitations and speed.

    We will go over the approach taken by haskell-ide-engine and the Language Server Protocol in order to solve these problems. Then we will discuss the new .hie file format designed to make information about Haskell source collected by GHC more accessible to tooling and users.

    Finally, for budding tooling writers, we will discuss the various approaches to writing programs that extract information from and manipulate Haskell source, and their pros and cons:

    • The GHC API
      • Via ghc-mod and haskell-ide-engine
    • ghc-exactprint
    • haskell-src-extras
    • GHC source plugins
    • .hie files
  • Liked Ankit Rokde

    Ankit Rokde - My Haskell Program does not fail. Proof?

    Ankit Rokde
    Ankit Rokde
    IIT Bombay
    schedule 1 year ago
    Sold Out!
    20 Mins

    Proving correctness of programs and ensuring they are bug-free has always been a challenging problem.

    Mostly we have relied on manual testing to check the correctness of programs.

    Strong static type systems help us to write bug free programs from the start but many interesting cases can miss out.

    Many tools such as QuickCheck, Liquid Haskell have been developed to address this issue.

    In this talk, we will presenting a different approach, Bounded Model Checking (BMC), which has been very successful in proving correctness of imperative programs by means of tools such as CBMC.

    We will explain how BMC works at high level, how we have adopted it for Haskell and our success with the same.

    We will also present how you can use it to prove correctness of your Haskell programs.