Aditya will be presenting the following session
filter_list help_outline
  • Aditya Athalye
    keyboard_arrow_down

    Aditya Athalye - Dr. StrangePipes or: How I learned to stop worrying && "function" in Shell

    90 Mins
    Workshop
    Intermediate

    What scares the Shell out of people?

    Artefacts of the long and idiosyncratic history of Unix.

    We have so many powerful, stable, widely-available Unix tools and standards, but also plenty of inconsistencies, variants, and arcane caveats; enough to help us... surprise... ourselves in creative new ways. See that cliff you're currently falling off? Yeah, someone went down that road when you were still eating chalk in kindergarten. (Or perhaps you fell off it once before, when I was eating chalk in kindergarten, but it was so long ago that you forgot you've been there done that :-))

    Worse, there's no adult supervision, only user manuals. Within Shell's neon-lit environs, one must learn to tread with care, creativity, and cool calculation.

    One must learn to accept the silent echo as reward for success.

    Why go to Shell at all?

    Several of us already get by just fine with a few memorised spells from Shell. That's not a bad way to live, but we miss out on tremendous everyday value that we can unlock with the Unix way.

    For there exists a vast no-man's land between memorised tricks and large scale "designed" solutions; wherein no single software tool can really solve all the problems we encounter.

    Yet, any modern *nix PC has all one needs to adequately and speedily tailor-make solutions for almost any problem we may encounter in the underserved no man's land.

    Solutions that we can use effectively until we truly hit the kind of scale problems or domain complexity that merit use of specialised tools.

    Unix nature is deeply Functional

    The Unix tools philosophy naturally leads to functional architectures that can scale almost effortlessly. And functional programmers could do worse than learn to exploit such power.

    I'll just leave you with Douglas McIlroy's answer to "Read a file of text, determine the n most frequently used words, and print out a sorted list of those words along with their frequencies.", as seen in More Shell, Less Egg:

    tr -cs A-Za-z '\n' |
    tr A-Z a-z |
    sort |
    uniq -c |
    sort -rn |
    sed ${1}q

    A purely functional, automatically buffered data processing pipeline, built with standard reusable parts, written in 1986, directly usable on today's computers.

    Imaginative Functional Programmers (aren't we all?!) will find striking parallels between the facilities, concepts, and design principles available in their $Langs, and their *nix environments.

    No surprise, because in the dreamy mists of time, the wise hackers of Unix lore, and SmallTalk lore, and APL lore, and Lisp lore, and Forth lore etc... were all different, but they were all together, too.

1. What got you into Functional Programming (FP)?

At my first software engineering job (with Helpshift.com), I discovered that some problem domains are object hierarchies, some are pipelines, some are procedural, and in every case the State is the Frenemy. So, to the extent that I try to reduce statefulness of my code (i.e. stave off insanity and remain functional), I might agree to be labeled a "Functional" programmer.

2. What has been your best moment or highlight working with FP?

In the foggy mists of time, 70-odd pages into solving SICP, I realized I'd already implemented a whole lot of logic without once setting a variable. Then it took me a few years to actually understand the implications of that. And years more to develop some intuition for when to keep The State at bay, and when to gingerly embrace it.

While at it, I discovered "FP" things in "OO" lands, and "OO" things in "FP" lands. Apparently one can do excellent "object oriented" programming using Clojure without knowing a single classical OO design pattern, process hundreds of GBs of logs with purely functional bash pipelines, learn to safely orchestrate complex infrastructure with Ansible (procedural, but idempotent), and discover new and clever ways to shoot oneself in the foot with all of these (after the foot is shot, of course).

Well, well, well.

I suppose if I zoom out enough, the whole thing will start looking like a best moment?

3. What are some of the greatest challenges of working with FP?

For me, "Why are we doing X?" needs to be at top of mind at all times. So my challenge is how to not lapse into thinking in terms of language-provided constructs rather than FP fundamentals. One of the side-effects of lapsing into a language is loss of objectivity; it becomes hard to drop "FP" when imperative or OO or array-oriented is actually the better model. Another side effect is missing the system for the code; like missing the woods for the trees. The code I'm writing is just one bit of something bigger. How should the constraints of the large be handled by the details I'm writing? Conversely; how are the choices I make in the small due to my $lang constraints/defaults leaking into the large?

Getting too clever with functional abstractions for my own good and/or for the the good of others. Like Russian dolls, it can take a lot of unpacking to see the entirety of sufficiently dense functional abstractions/stacks. This is especially important when one wants to adopt strict type discipline (i.e. not cowboy type astronautics). Groups cannot maintain what groups cannot be understand. So, how, in a team setting, should I restrain myself to abstracting at the level of understanding of the group? This requires discipline, taste, and a teacher's mindset --- abstract the code at the rate at which one is able to abstract the people, but no faster.

4. All the mainstream programming languages are adding functional programming features. Most new languages and frameworks are strongly influenced by FP. What is your advice to object-oriented programmers?

All of the following, IMHO, and YMMV:

Whatever one's persuasion re: language, State is the Frenemy. The core challenge of all programming is how to cause _only_ the intended effects, _only_ in predictable ways. And here's the kicker... If one really stares it in the face, _all_ code is inherently stateful, because of the many unwritten intentions and unsaid assumptions that _don't_ go with it (but which are so essential to its correct working today, and to its correct transmutation over time). I.e. we're already attempting the impossible.

So, I don't think there's any one absolute reality called "object oriented" programming; nor one called "functional" programming.

I do think, however, that there's purpose-appropriate problem-solving. And that the trickiest problems that manifest in our code do not fundamentally originate in language capability, but through misunderstanding the actual problem at hand and/or our inability to solve it correctly (under natural constraints of money, mindspace, and markets). Also, I believe, these problems are due to our recently-acquired penchant for seeking technological solutions to _whatever_ may be the problem (real or imagined). Said more tongue-in-cheek; whether mainstream or niche, "object oriented" or "functional"; if deleted code is good code, then _averted_ code is great code.

Concurrent to these comments; one can't beat the laws of Physics. Entropy will always surprise us in ways we cannot imagine or plan for. I feel that the humility and watchfulness that come through acquiring this perspective are far better bedfellows than the paradigms and preconceptions that come bundled with Language Oriented Problem Solving* (or LOPSided programming).

* This is different from what Racket people mean when they say the same words.

5. What will be some of the key takeaways from your sessions at the conference?

The Unix tools philosophy naturally leads to functional architectures that can scale almost effortlessly. And functional programmers could do worse than learn to exploit such power.

There exists a vast no-man's land between memorised command-line tricks and large scale "designed" solutions; wherein no single software tool can really solve all the problems we encounter. Yet, any modern *nix PC has all one needs to adequately and speedily tailor-make solutions for almost any problem we may encounter in the underserved no man's land.

We will see how to apply FP techniques to build our own "swiss army toolkits" of custom functions and utilities...

  • that work seamlessly with the standard Unix model, and so
  • are useful not just within scripts, but also interactively at the command line
  • can interoperate with any other tool that understands the Unix way
  • allowing us to get surprisingly little code to do surprisingly powerful things
6. The conference has more than 50 sessions. Which ones are you most looking forward to attending and why?

Mainly the "hallway track" at lunch and coffee, where I get to gab with old friends, make new ones, and serendipitously find myself part of interesting conversations.

help