Operating at scale with Elastic Search in production
Having problems in your application taking hours of effort to debug because of needle in a haystack of errors? Or unable to find anomalies quickly before it booms up to an issue? Want to do behaviorial analysis to learn about your customers? Want to find customers near you? Elasticsearch would be an one stop solution. You can convert data into valuable insights by uncovering hidden patterns and correlations, with its efficient full text search and analytics.
Setting up a cluster in smaller scale is simple. But, complexity grows with scale for any distributed system. Elasticsearch is not far from this fact. This presentation will cover the below:
- Challenges met during the journey and the learnings that built us more stronger
- Designing & Building a highly scalable and resilient reactive application to process/ingest multi billions of records/day into elasticsearch
Outline/Structure of the Talk
- Elasticsearch - Basics.
- Challenges while scaling application for high throughput.
- Tuning Elasticsearch for production use.
- Resilient by design (Application architecture and reactive systems).
The key take aways will be
- Mistakes one can avoid when scaling elasticsearch in production
- How to handle resiliency when working with petabyte scale systems
Anyone ranging from beginner to experienced (in elasticsearch)