Tick Tock: What the heck is time-series data?
The rise of IoT and smart infrastructure has led to the generation of massive amounts of complex data. In this session, we will talk about time-series data, the challenges of working with time series data, ingestion of this data using data from NYC cabs and running real time queries to gather insights. By the end of the session, we will have an understanding of what time-series data is, how to build streaming data pipelines for massive time series data using Flink, Kafka and CrateDB, and visualising all this data with the help of a dashboard.
Outline/Structure of the Talk
High level outline of topics that will be covered in this presentation:
1. Growth of IoT and Sensor Data
2. Time-series data
3. Challenges that are posed by large volumes of time-series data
4. Showcasing and overcoming the problem: A case-study
5. Demo time: Creating a highly available data pipeline with Kafka, Flink and CrateDB to visualise with Grafana. We will be ingesting ~4 million records of the NYC cab data
By the end of this session, we will be able to set up a highly scalable data pipeline for complex time series data with real time query performance.
Developers, Managers, IoT Specialists
Prerequisites for Attendees
Some knowledge of databases, data pipelines and containers will help the audiences to follow along and make the most of this talk.
schedule Submitted 1 month ago
People who liked this proposal, also liked:
Tanay Pant - Machine data: how to handle it better?Tanay PantDeveloper AdvocateCrate.io
schedule 1 month agoSold Out!
The rise of IoT and smart infrastructure has led to the generation of massive amounts of complex data. Traditional solutions struggle to cope with this shift, leading to a decrease in performance and an increase in cost. In this talk, we will take a look at this kind of data using a simulated Curiosity rover. Participants will learn how to create a data pipeline for ingestion and visualisation. By the end of this session, we will be able to set up a highly scalable data pipeline for complex time series data with real time query performance.