I will talk about a much simplified version of functional programming in Scala, in building an abstraction for Feature Generation in Data Engineering space.
The program made using this abstraction will get interpreted to the popular data source languages of our choice - such as Spark or Flink. However, before it gets interpreted to any of these engines, we will explain how these programs could be optimised by introspecting its nodes, and help run these interpretations faster. The core idea is similar to that of Free Applicative, however, implementing it in Scala hasn't been straight forward. Here, we provide a similar capability but without mentioning much about FreeAp, and without the usual Scala boilerplates of implicits, macros and a proliferated usage of type classes.

The purpose of the talk is not just to demonstrate a set of code, but to showcase the fact that sticking on to fundamentals of Functional Programming, and finding the right abstraction enables writing solutions quickly and relatively easily.

It proves we don't need to learn a bulk of libraries to apply these concepts in real world applications. The learning curve and a massive set of libraries was often termed as the functional programming in Scala industry, resulting in lesser adoption and developers moving away from it. With this talk my intention is to motivate developers to come back and start writing FP even if they are in the world of JVM.