Harendra will be presenting the following session
filter_list help_outline
  • Harendra Kumar

    Harendra Kumar - Streamly: Declarative Concurrency and Dataflow Programming in Haskell

    45 Mins

    Overview: Streamly is a Haskell library for writing programs in a high level, declarative data flow programming paradigm. It provides a simple API, very close to standard Haskell lists. A program is expressed as a composition of data processing pipes, generally known as streams. Streams can be generated, merged, chained, mapped, zipped, and consumed concurrently – enabling a high level, declarative yet concurrent composition of programs. Programs can be concurrent or non-concurrent without any significant change. Concurrency is auto scaled based on consumption rate. Programmers do not have to be aware of threads, locking or synchronization to write scalable concurrent programs. Streamly provides C like performance, orders of magnitude better compared to existing streaming libraries.

    Motivation: Haskell has a beautiful abstraction capability. Haskell lists, several streaming and concurrency libraries are some examples of good programming abstractions. The problem is that concurrency is not handled elegantly and declaratively by these libraries. Moreover, the lack of good performance gives rise to special purpose performance centric libraries like text and bytestring creating more confusion. Streamly unifies, lists, streaming, concurrency, logic programming and reactive programming with unparalleled performance. We can finally have a unified string handling.

    Audience: Streamly is designed to express the full spectrum of programs. Do not think that if you are writing a small and simple program it may not be for you. It expresses a small program with the same efficiency, simplicity and elegance as a large scale concurrent application. It unifies many different aspects of special purpose libraries into a single yet simple framework. It is especially suitable for high performance and scalable backend data processing systems. If you use or plan to use any real time streaming or data flow programming frameworks including Apache Flink, Apache Spark or asynchronous/reactive programming frameworks like ReactiveX/RxJs then you may be interested in Streamly.

1. What got you into Functional Programming (FP)?

I have a background in hard core systems programming using C for more than a decade. I have worked on large scale (millions of line of code), complex and high performance systems, heavily deployed in data centers across the world. I even worked on modularizing these large and complex systems to make them more maintainable and less bug prone. After knowing these problems so well and struggling to solve them I was looking for a solution that I could use if I were building these systems from scratch. I zeroed in on Haskell, but then found that even though the language provides tremendous power, currently available libraries do not fully utilize that power to provide a convenient and powerful interface to the programmer.  Streamly (https://github.com/composewell/streamly) is an effort to unify some powerful concepts in Haskell, provide excellent expressive power to the programmer, with inherent and composable concurrency capabilities as well as performance that rivals a low level language like C.

2. What has been your best moment or highlight working with FP?

Coming from the imperative programming background I was initially overwhelmed by the amount of new knowledge that I had to absorb when learning Haskell.  But once you get comfortable with the language you derive satisfaction by the tremendous composability and expressive power of the language as well as the ability to build programs that are correct by construction. Building Streamly, a concurrent data flow programming framework, has been the most satisfying experience for me, it harnesses these properties of the language and provides C like performance characteristics to alleviate the complaints that people generally have for using Haskell in production. The conciseness and power with which you can build high performance concurrent applications is very satisfying.

3. What are some of the greatest challenges of working with FP?

The greatest challenge is learning it in the first place. Most students are taught the imperative programming paradigm in college. They get trained in thinking in that way and that makes it harder for them to transition to a functional paradigm. To transition they have to unlearn what they learnt and start thinking in a new way. That is the greatest challenge facing the adoption of functional programming paradigm. I believe functional and imperative both are necessary to be able to build maintainable software systems. Haskell provides excellent tools to use both the paradigms. Haskell would be a great choice of language to teach functional programming in college.

After a major refactor of a Haskell program correctness is largely unaffected, however, there is no guarantee that performance would remain unaffected. That's opposite to imperative languages like C. With Streamly we are trying to address that challenge to a large extent.

4. All the mainstream programming languages are adding functional programming features. Most new languages and frameworks are strongly influenced by FP. What is your advice to object-oriented programmers?

There is tremendous value in functional programming, it helps you organize your program better, cut the clutter, reduce programming bugs, which in turn helps reduce the maintenance cost and time to market. Popular imperative languages have recognized this value and are adding more and more of functional features which is a very good thing. I would encourage object oriented programmers to use more and more of these features and benefit from them. I would also
encourage them to explore and consider using pure functional programming languages like Haskell, especially in new projects. It may take some time to untrain yourself from imperative thinking and train in pure functional thinking but it would be worth the effort in the end.

5. What will be some of the key takeaways from your sessions at the conference?

We all know that Haskell has strong static typing and immutability which helps us avoid almost all classes of bugs other than business logic bugs.  However, there is a perception that performance and memory consumption in Haskell is unpredictable. In this talk we will demonstrate that, using Streamly, applications that are traditionally written in C can be written in Haskell with very high composability, inherent concurrency and with the same performance and memory consumption characteristics as C. Overall, dramatically reducing the development and maintenance cost of software.

Our goal at Composewell Technologies is to reduce the cost of development and maintenance of large, complex and scalable systems by an order of magnitude. We want to enable Haskell to be widely used wherever programmers would use C/C++, Python or Java but with C like performance, predictable resource consumption and higher level composition capabilities, so that you can get the best of both worlds. Streamly is a step in that direction and our customers have witnessed dramatic benefits due to the high level composability and C level performance of Streamly.

6. The conference has more than 50 sessions. Which ones are you most looking forward to attending and why?

All sessions look good but I am especially looking forward to the ones that overlap with my areas of interest, "Logic Programming à la Carte" by Edward Kmett is one such session. Logic programming domain has an overlap with Streamly.  I would also like to attend "Thinking with Arrays" by Marshall Lochbaum. I have found that Streamly has some similarity with array programming, Marshall would be talking about the outer product combinator in APL, incidentally, we have introduced an outer product combinator in Streamly recently.  "Learnings from using haskell in production" by Sreenidhi Nair also looks exciting, I am especially interested in problems that can be solved in a better way.