ELK vs Graylog: Backend index management

Both ELK and Graylog use Elasticsearch for log storage and indexing.  Most of what makes both Kibana and Graylog work so well for searching through logs is mostly provided by Elasticsearch with Graylog and Kibana acting as a GUI to interface with it.  Data is stored in indexes in Elasticsearch and those indexes need to be maintained and managed.  How each product does it though is different. (more…)

Continue Reading

1 billion+ logs per day with ELK stack, hardware and scalable architecture

I remember hitting my first 100 logs per second still very distinctly. After a few weeks of tweaking Elasticsearch to get an understanding of it, and fighting with Logstash and grok to make it actually do what I wanted….all without crashing Elasticsearch due to the very anemic resources I had given it and having Logstash actually keep up with the ingestion rate without falling behind and dropping events. (more…)

Continue Reading
Close Menu