Parallel Programming in Dyalog using Futures and Isolates

schedule Sep 12th 05:00 PM - 05:45 PM place Grand Ballroom

Dyalog is an array first multi-paradigm language based on an APL language kernel. In APL, it is common for the application of primitive functions to arrays of conforming shape to imply a map; the function is applied item-wise to corresponding elements of the arguments, producing a result of the same shape. APL also provides a variety of explicit operators that provide different mappings of user-defined functions, or more generally functional compositions, to multi-dimensional and nested arrays.

One might expect that such language features would provide a complete framework for parallel programming. In practice, writing programs that can be efficiently executed in parallel on current hardware is extremely difficult, if implicit or explicit maps are the only parallel language feature available and an interpreter or compiler needs to deduce where to introduce the asynchronicity that is required in order to take advantage of the hardware.

This talk will present Futures and Isolates, and related parallel operators constructed upon them, which were added to Dyalog version 14.0 in 2014. The goal has been to provide explicit forms of deterministic asynchronous execution, designed to be very familiar to APL users, and continue to allow them to view the language as an executable mathematical notation, while taking advantage of parallel hardware that is now readily available.

 
6 favorite thumb_down thumb_up 0 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/structure of the Session

Following an introduction to the new language features (more formal/theoretical and aimed at users with no knowledge of APL than the one that can be seen in my presentation to the Dyalog User Meeting last year), a live demonstration of the language features in action.

Learning Outcome

Insight into a rather different way to express concurrency than is common in other programming languages.

Target Audience

Anyone interested in concurrent, functional programming

schedule Submitted 2 years ago

Comments Subscribe to Comments

comment Comment on this Proposal

  • Liked Viral B. Shah
    keyboard_arrow_down

    Viral B. Shah - Why Julia?

    60 mins
    Keynote
    Beginner

    In this talk, I will talk about our motivation for creating Julia. Julia is a high-level, high-performance dynamic programming language. It provides a sophisticated compiler, distributed parallel execution, numerical accuracy, and an extensive mathematical function library. Julia’s Base library, largely written in Julia itself, also integrates mature, best-of-breed open source C and Fortran libraries for linear algebra, random number generation, signal processing, and string processing. In addition, the Julia developer community is contributing a number of external packages through Julia’s built-in package manager at a rapid pace. This is why Julia is seeing rapid adoption in universities for teaching and research, as well as in businesses.

    I will discuss what makes Julia fast. Julia's ability to combine these levels of performance and productivity in a single language stems from the choice of a number of features that work well with each other:

    1. An expressive parametric type system, allowing optional type annotations;
    2. Multiple dispatch using those types to select implementations;
    3. A dynamic dataflow type inference algorithm allowing types of most expressions to be inferred;
    4. Aggressive code specialization against run-time types;
    5. Metaprogramming;
    6. Just-In-Time compilation using the LLVM compiler framework; and
    7. Careful design of the language and standard library to be amenable to type analysis;

    I will also touch upon how the language design has made it possible to explore many parallel programming models in Julia, completely within the language.

    See: http://www.julialang.org/

  • Liked Morten Kromberg
    keyboard_arrow_down

    Morten Kromberg / Jay Foad - Array Oriented Functional Programming with Dyalog

    Morten Kromberg
    Morten Kromberg
    CXO
    Dyalog
    Jay Foad
    Jay Foad
    Developer
    Dyalog Ltd
    schedule 2 years ago
    Sold Out!
    480 mins
    Workshop
    Beginner

    Dyalog is a modern, array-first, multi-paradigm programming language, which supports functional, object-oriented and imperative programming based on an APL language kernel. During the morning and early afternoon, Dyalog's functional language guru John Scholes will introduce the fundamentals of functional programming in APL. At the end of the day, Dyalog CTO Morten Kromberg will round off with an application development session, showing how a simple web application with a HTML5/JS front-end and a RESTful web service can be used to deliver the power of APL to end users, or as embeddable components for other application tools. The "hands on" sections of the workshop can be followed under Mac OS X, Linux, or Microsoft Windows.

  • Liked Shantanu Kumar
    keyboard_arrow_down

    Shantanu Kumar - Production Clojure: An Experience Report

    Shantanu Kumar
    Shantanu Kumar
    Principal Engineer
    Concur
    schedule 2 years ago
    Sold Out!
    45 mins
    Experience Report
    Intermediate

    This talk is about our experience of using Clojure in production at Concur for building a SaaS platform.

    To set the perspective, Concur is a 22 years old, USD 8.3 billion enterprise, now a part of SAP. More than 70% of Fortune 500 companies use Concur for travel and expense solutions. Concur's SaaS platform serves 25,000 customers with 25 million users. The Concur SaaS platform was reaching its limits and needed "re-architecting". This talk is about why we chose Clojure at Concur and how we are building one of the world's largest B2B SaaS platforms in Clojure.

    We will discuss the practices we established, experiments we did and the lessons learnt. We have now built a platform that follows open standards, is rigorously tested, tolerates failure and performs well. We will go over how we reached there, how we deployed the code to production and how we monitor those today.

    If you are considering using Clojure in production or even simply interested in Clojure, you may get useful insights to take away from this talk.

  • Liked Tanmai Gopal
    keyboard_arrow_down

    Tanmai Gopal - Haskell In Production

    Tanmai Gopal
    Tanmai Gopal
    Engineering Head, Founder
    34 Cross
    schedule 2 years ago
    Sold Out!
    45 mins
    Experience Report
    Intermediate

    From the perspective of one of the few companies in the country using Haskell in production (hasura.io), we will talk about the journey of using Haskell as a programming language to build and maintain real world software.
    From the burden of choosing a language that probably has a single digit number of active developers in the country, to
    the joys of a small team churning out code that just keeps on working.

    We'll go over how we created a team, organized work ("software development practices"), architecting code, tooling up, testing, performance tweaking and finally deploying a Haskell codebase. We hope that Haskell developers with some prior experience will be able to take away
    something new about the Haskell ecosystem, and those interested in just functional programming will see tangible value in how certain features unique to Haskell have an impact.

    Expect a case study, a review of some cutting-edge tools and how to use them, and some Haskell code samples that will make you grunt in geeky approval.

  • Liked Mayank Jain
    keyboard_arrow_down

    Mayank Jain - Testing business logic using DSLs in Clojure

    Mayank Jain
    Mayank Jain
    Software Artisan
    Helpshift
    schedule 2 years ago
    Sold Out!
    45 mins
    Experience Report
    Intermediate

    Testing business logic quickly becomes tricky, as applications grow and scale. Example based unit and integration tests, and exploratory tests become poor choices to check and verify a large state space. Also such methods are not  well-suited to clearly describe the state space / transitions we want to test. 

    Since we use Clojure to write test suites and tools at Helpshift, we tried experimenting with DSLs to express some of our testing problems. We've found that mini DSLs can indeed become useful to describe and test fairly complicated business logic.

    In this talk I will cover some of the mini-DSL approaches we've tried, demonstrate one of them by example, and discuss the benefits and drawbacks based on current experience.

  • Liked Shashi Gowda
    keyboard_arrow_down

    Shashi Gowda / Viral B. Shah - Julia - A Lisp for Fast Number Crunching

    240 mins
    Workshop
    Beginner

    Julia is a programming language for data science and numerical computing. Julia is a high-level, high-performance dynamic language, it uses just-in-time compilation to provide fast machine code - dynamic code runs at about half the speed of C, and orders of magnitude faster than traditional numerical computing tools.

    Julia borrows two main ideas from Lisp lore:

    1. Multiple-dispatch: a method is identified by a name and the types of all of its arguments (traditional OOP only uses the type of a single argument) multiple-dispatch allows for a natural programming model suitable for mathematics and general purpose programming. Interestingly, the same feature is responsible for Julia's ability to generate really fast machine code.
    2. Macros: Julia has syntax that is familiar to users of other technical computing environments, however, Julia expressions are homoiconic -- Julia code can be represented as Julia data structures, and transformed from one form to another via hygienic macros. There are some very interesting uses of macros to create domain-specific languages for effective programming. 

    (I will be presenting this workshop along with Viral B Shah - co-creator of Julia)

  • Liked John Thomas
    keyboard_arrow_down

    John Thomas / Shyam Sundar - A Introduction to Functional Design Patterns in scala

    90 mins
    Workshop
    Beginner

    First notorious word, every functional programming encounter (without mathematical background ) during initial days  is "Category theory " . Our aim is to introduce the very minimum concept a functional developer should know in category theorem .

    The goal is to introduce basics of category theory and develop a small prototype library to validate and process data  , out a few principles from category theory.

    We will also be introducing and implementing representations of Functors, SemiGroup, Applicative, Monoid and Monad. We also make use of some interesting Scala features like ADT - Algebraic data type , Implicit conversion, Types . 

     

     

     

  • Liked Daniel Steinberg
    keyboard_arrow_down

    Daniel Steinberg - Introduction to Swift Workshop

    Daniel Steinberg
    Daniel Steinberg
    Author
    dimsumthinking.com
    schedule 2 years ago
    Sold Out!
    480 mins
    Tutorial
    Beginner

    If you are an experienced developer who hasn't had a chance to look at the Swift Programming Language, this workshop is for you.

    We'll look at language fundamentals: functions, variables and constants, collections, control-flow, optionals, generics, and closures. We'll look at Swift classes, objects, protocols, structs, modules, and enumerations.

    We'll use Swift Playgrounds to explore some of these aspects of the language including the latest additions to Swift 2.0.

    Swift is scheduled to be open sourced this fall. You will be able to use it for development on Linux and OS X.

    There will be several hands-on portions of this workshop. To fully participate in those, you must have the Xcode 7 beta installed on your Mac.

  • Liked Ravi Mohan
    keyboard_arrow_down

    Ravi Mohan - Building a General Game Playing Engine with OCaml and Erlang

    Ravi Mohan
    Ravi Mohan
    CEO
    AxiomChoice
    schedule 2 years ago
    Sold Out!
    90 mins
    Experience Report
    Beginner

    "General Game Playing  is the design of artificial intelligence programs that play more than one game"  Wikipedia [1]

    In other words, one program has to be able to play (and win!) multiple games ( Chess, checkers, Go, Othello etc).

    Summary: This experience report is about my  "Forever Project" [2,3,4] to build such a system in OCaml, and the problems (theoretical and practical) and will include a demo of the program, warts and all.

    Detail: GGP is an active CS research area [5]Annual competitions are held every year where  ggp programs compete against each other[6], playing games which they have never seen before (the rules are supplied as an input file for the system to consume five minutes before the match begins).

    Over the last year, in my non-existent spare time, I've been building a GGP engine to compete in a GGP competition.

    Most such engines are written in Java, C++  etc.. I'm using OCaml for the game playing agent, and Manoj is building the backend in Erlang.

    As mentioned above this is our "Forever Project"  We've made some decent progress.

    Come, see, laugh, jeer!

  • Liked Nivedita Priyadarshini
    keyboard_arrow_down

    Nivedita Priyadarshini / Srihari Sriraman - Building an Experimentation Platform in Clojure

    45 mins
    Experience Report
    Intermediate

    Here's a markdown version of this proposal: https://gist.github.com/ssrihari/1cad915e7ef22ce61b54

    Over the last year and half at Staples SparX, we built a multivariate testing platform as a service. It satisfies an SLA of 10ms at 99.9th percentile, services all of Staples' experimentation from a single machine, is simulation tested, and is written in Clojure.

    We'll give an introduction to the Experimentation domain, design of experiments and our battle in attaining statistical significance with constrained traffic. We will share our experiences in loading and reporting over months of data in Datomic, using Clojure to grow a resilient postgres cluster, using a homegrown jdbc driver, interesting anecdotes, and OLAP solutions with ETL built in Clojure using core.async. Expect to see references to google white papers, latency and network graphs, histograms, comparison tables and an eyeful of clojure code.

  • Liked Srihari Sriraman
    keyboard_arrow_down

    Srihari Sriraman - Carnatic music synthesis in Clojure

    Srihari Sriraman
    Srihari Sriraman
    Partner, Engineer
    nilenso
    schedule 2 years ago
    Sold Out!
    45 mins
    Demonstration
    Beginner

    Here's a markdown version of this proposal: https://gist.github.com/ssrihari/b25ddda331ec220663de

    I'm building a carnatic music (south indian classical music) synthesizer using Clojure and Overtone. Here's the library I'm building.

    Carnatic music is different fundamentally in the movements of pitch ('Gamakams'), and needs to be modelled differently than most MIDI music out there today. The library now, can understand prescriptive carnatic music notation, build finer abstractions from that, and play back a not-so-terrible synth version with a plethora of configurable inputs.

    The concept of 'Gamakam' (the transitions between notes) is central, so I'll go into it's guts and show how the library piggybacks on recent work on PASR (pitch-attack-sustain-release) and Dual-PASR ('stage' and 'dance' PASR components) transcriptions. I'll also put forth a path I see for fully synthetic carnatic music based on machine learning 'Ragams' (melodic modes).

    I'll give a detailed demo of the synthetic music on the repl through the talk, and do a live voice vs synth comparison in the end.

  • Liked Bhasker Kode
    keyboard_arrow_down

    Bhasker Kode - Workshop: Erlang by example

    Bhasker Kode
    Bhasker Kode
    Building stuff
    Helpshift
    schedule 2 years ago
    Sold Out!
    480 mins
    Workshop
    Beginner

    From fundamentals to deploying Erlang micro-service.