Rafal is an engineer focused on data infrastructure at Spotify. He has operated Hadoop clusters (of size from 1 to more than 2000 nodes). He is a contributor to snakebite - pure python HDFS client and Scio - Scala DSL for Apache Beam.
Apache Beam (based on Google’s Dataflow Model) provides a simple, unified programming model for both batch and streaming data processing. If only it wasn’t so unfamiliar and verbose for our Scala engineers. Learn how Scio leverages Scala’s type system, macros and functional paradigm to provide more engineer-friendly and type safe API.