ELK Stack
TLDR: The ELK Stack is a collection of open-source tools—Elasticsearch, Logstash, and Kibana—designed for centralized logging, search, and data visualization. Introduced in the early 2010s by Elastic, the stack enables users to aggregate, analyze, and visualize log data from diverse sources in real time. It has become a popular choice for monitoring system performance, diagnosing issues, and ensuring security compliance.
https://en.wikipedia.org/wiki/ELK_Stack
Elasticsearch serves as the core of the ELK Stack, providing a scalable, distributed search and analytics engine. It stores and indexes data ingested by Logstash, enabling fast querying and analysis. Data is structured into documents, making it easy to perform complex aggregations or searches using the Elasticsearch Query DSL. This foundation supports high-velocity data handling, critical for modern observability systems.
https://www.elastic.co/elasticsearch/
Logstash acts as the data processing pipeline, ingesting logs and metrics from various sources such as servers, applications, and cloud services. It applies filters to parse and enrich the data, transforming it into formats suitable for indexing by Elasticsearch. Logstash's plugin-based architecture ensures flexibility, accommodating a wide range of input sources and processing needs.
https://www.elastic.co/logstash/
Kibana provides the visualization layer, allowing users to create interactive dashboards and reports based on data stored in Elasticsearch. Its real-time monitoring capabilities enable tracking system metrics, error trends, and user activity. Combined, the ELK Stack offers a powerful, scalable solution for logging and observability, widely used in DevOps, security, and business intelligence.