1 billion+ logs per day with ELK stack, hardware and scalable architecture

I remember hitting my first 100 logs per second still very distinctly. After a few weeks of tweaking Elasticsearch to get an understanding of it, and fighting with Logstash and grok to make it actually do what I wanted….all without crashing Elasticsearch due to the very anemic resources I had given it and having Logstash actually keep up with the ingestion rate without falling behind and dropping events. (more…)

Continue Reading
Close Menu