Observability with Prometheus and Grafana

What is Prometheus?

Prometheus is an open-source system used for monitoring and alerting, which records near real-time metrics in a time series database. It was originally developed at SoundCloud in 2012 to address the limitations of existing monitoring tools like StatsD and Graphite, which were found inadequate for SoundCloud's scaling requirements. It provides a multi-dimensional data model, operational simplicity, scalable data collection, and a powerful query language, all within a single tool. The project, written in Go and licensed under the Apache 2 License, is hosted on GitHub.

Prometheus is today one of the most popular monitoring tools in the Cloud Native ecosystem, if not the most popular. It is widely used for monitoring Kubernetes, microservices, containerized applications, and other Cloud Native workloads, but it can also integrate with traditional systems. It operates on an HTTP pull model, with flexible queries and fast data processing.

ℹ️ The pull model is a method of data collection where the Prometheus server fetches metrics from the target systems in contrast to the push model where the target systems push metrics to the monitoring server. Designing Prometheus around the pull model allows for better scalability and more control over server-side (Prometheus) data collection and less complexity on the client-side (the monitored systems).

As a graduated project of the Cloud Native Computing Foundation (CNCF), alongside Kubernetes, Envoy, Istio, Jaeger, etcd, CoreDNS, and other popular tools, Prometheus is widely adopted by companies such as Google, DigitalOcean, Red Hat, Slack, Uber, Medium, Trivago, and many others.

ℹ️ The CNCF defines itself as a vendor-neutral foundation that hosts open-source projects in the Cloud Native ecosystem. It was founded in 2015 by the Linux Foundation and has since become the home of many popular projects such as Kubernetes, Prometheus, Envoy, and others.

The tool was inspired by Borgmon, a monitoring system used at Google. By 2013, Prometheus was introduced for production monitoring at SoundCloud, and the official public announcement was made in January 2015.

"Prometheus is Borgmon. Which is a bit funny as it's used by k8s now. It's also funny as the author left Google, wrote it in 2 years for SoundCloud, then went back to Google." ~ kyrra, a user on Hacker News

If you're curious about Google's Borgmon, it is a monitoring system that was developed at Google to monitor Borg, Google's internal cluster management system and Kubernetes' predecessor. The SRE Book