Tick Tock: What the heck is time-series data?
The rise of IoT and smart infrastructure has led to the generation of massive amounts of complex data. In this session, we will talk about time-series data, the challenges of working with time series data, ingestion of this data using data from NYC cabs and running real time queries to gather insights. By the end of the session, we will have an understanding of what time-series data is, how to build streaming data pipelines for massive time series data using Flink, Kafka and CrateDB, and visualising all this data with the help of a dashboard.
Outline/Structure of the Talk
High level outline of topics that will be covered in this presentation:
1. Growth of IoT and Sensor Data
2. Time-series data
3. Challenges that are posed by large volumes of time-series data
4. Showcasing and overcoming the problem: A case-study
5. Demo time: Creating a highly available data pipeline with Kafka, Flink and CrateDB to visualise with Grafana. We will be ingesting ~4 million records of the NYC cab data
Learning Outcome
By the end of this session, we will be able to set up a highly scalable data pipeline for complex time series data with real time query performance.
Target Audience
Developers, Managers, IoT Specialists
Prerequisites for Attendees
Some knowledge of databases, data pipelines and containers will help the audiences to follow along and make the most of this talk.
Links
Here's some highlights of my previous work:
- Talk on 'Multiplayer Games with WebXR': https://www.youtube.com/watch?v=-UCfNmB8618
- Talk on 'Learning WebVR': https://www.youtube.com/watch?v=vTntqtIh3mM
- My interview at DevFestNN: https://www.linkedin.com/feed/update/urn:li:activity:6491638347432230913/
- My books: https://www.amazon.com/Tanay-Pant/e/B015R1B9RE
- Some other misc stuff: https://devdiner.com/tag/tanay-pant
schedule Submitted 1 month ago
People who liked this proposal, also liked:
-
keyboard_arrow_downTanay Pant - Machine data: how to handle it better?
45 Mins
Talk
Intermediate
The rise of IoT and smart infrastructure has led to the generation of massive amounts of complex data. Traditional solutions struggle to cope with this shift, leading to a decrease in performance and an increase in cost. In this talk, we will take a look at this kind of data using a simulated Curiosity rover. Participants will learn how to create a data pipeline for ingestion and visualisation. By the end of this session, we will be able to set up a highly scalable data pipeline for complex time series data with real time query performance.
-
Recent Activities
- reply Ashay Tamhane replied "Great, thanks for the cla..." on a comment on proposal Tick Tock: What the heck...schedule 4 days ago
- reply Tanay Pant replied "Hi Akshay, yes I plan to..." on the comment Hello Tanay, thanks for t...schedule 1 week ago
- comment Ashay Tamhane commented "Hello Tanay, thanks for t..." on proposal Tick Tock: What the heck...schedule 1 week ago
Call for Proposals 20 days left
Submit before Apr 15 '19 11:30 PM IST
We are looking for deep technical topics related to Data Science. We are committed to diversity and believe in transparency. Hence, all proposals will be public.
Registered user will be able to comment on your proposal. You are required to reply to those comments to provide clarifications, explain revisions and respond to questions. The program team will look at how well you've responded to comments and updated your proposal to incorporate public suggestions. Ultimately the decision to accept a session resides with the program team.
Your proposal stands the best chance to be selected, if it's unique, fully flushed, ready-to-go. Please ensure you've read:
To encourage early submissions and iterative improvement of proposals, we start accepting proposals as soon as we find them a good fit. Since competition gets tougher with progressing time, it is advisable to avoid waiting until the last minute for proposal submission.
Public Feedback
Hello Tanay, thanks for the submission. Do you also plan to compare and contrast different architectures and explain why the recommended combination of Kafka, Flink and CrateDB works as opposed to other options that could be possible?
Hi Akshay, yes I plan to compare various different databases and architectures and present a case study on benchmarks and use-cases related to it in the talk.
Great, thanks for the clarification.