Living in Big Data
And Now For Something Completely Different?
In this talk how one can interactively explore and visualize massive data sets using a vector functional data language. Vector languages are heavily used in finance and insurance for analysis of large time series. Vector/Array dynamic languages originated with APL family and include NIAL, J and k and databases such as Kdb+, MonetDB and Vectorwise. The have inspired array capabilities in other languages, such as R, MatLab. Python NumPy and Pandas.
Vector languages enable rapid interactive development of concise and efficient applications. Vectors are a natural low impedance abstraction for dealing with high performance column stores. We introduce the key concepts of vector programming, applicative style and idioms of FP. We then illustrate their use on typical data science problems exploring large data sets.