Tick Tock: What the heck is time-series data?

The rise of IoT and smart infrastructure has led to the generation of massive amounts of complex data. In this session, we will talk about time-series data, the challenges of working with time series data, ingestion of this data using data from NYC cabs and running real time queries to gather insights. By the end of the session, we will have an understanding of what time-series data is, how to build streaming data pipelines for massive time series data using Flink, Kafka and CrateDB, and visualising all this data with the help of a dashboard.

 
14 favorite thumb_down thumb_up 3 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist
 

Outline/Structure of the Talk

High level outline of topics that will be covered in this presentation:

1. Growth of IoT and Sensor Data

2. Time-series data

3. Challenges that are posed by large volumes of time-series data

4. Showcasing and overcoming the problem: A case-study

5. Demo time: Creating a highly available data pipeline with Kafka, Flink and CrateDB to visualise with Grafana. We will be ingesting ~4 million records of the NYC cab data

Learning Outcome

By the end of this session, we will be able to set up a highly scalable data pipeline for complex time series data with real time query performance.

Target Audience

Developers, Managers, IoT Specialists

Prerequisites for Attendees

Some knowledge of databases, data pipelines and containers will help the audiences to follow along and make the most of this talk.

schedule Submitted 1 month ago

Public Feedback

comment Suggest improvements to the Speaker
  • Ashay Tamhane
    By Ashay Tamhane  ~  1 week ago
    reply Reply

    Hello Tanay, thanks for the submission. Do you also plan to compare and contrast different architectures and explain why the recommended combination of Kafka, Flink and CrateDB works as opposed to other options that could be possible?

    • Tanay Pant
      By Tanay Pant  ~  1 week ago
      reply Reply

      Hi Akshay, yes I plan to compare various different databases and architectures and present a case study on benchmarks and use-cases related to it in the talk.

      • Ashay Tamhane
        By Ashay Tamhane  ~  4 days ago
        reply Reply

        Great, thanks for the clarification.


  • Liked Tanay Pant
    keyboard_arrow_down

    Tanay Pant - Machine data: how to handle it better?

    Tanay Pant
    Tanay Pant
    Developer Advocate
    Crate.io
    schedule 1 month ago
    Sold Out!
    45 Mins
    Talk
    Intermediate

    The rise of IoT and smart infrastructure has led to the generation of massive amounts of complex data. Traditional solutions struggle to cope with this shift, leading to a decrease in performance and an increase in cost. In this talk, we will take a look at this kind of data using a simulated Curiosity rover. Participants will learn how to create a data pipeline for ingestion and visualisation. By the end of this session, we will be able to set up a highly scalable data pipeline for complex time series data with real time query performance.