Setup Prometheus and Grafana on Ubuntu

1. Introduction

In this guide, we will demonstrate how to set up Prometheus and Grafana on Ubuntu 22.04.

Written in the Go language, Prometheus is a free and open-source monitoring and alerting solution that collects metrics data from services and stores them in a time-series database. It scrapes metric data from HTTP endpoints, either directly or via an intermediary push gateway for systems not compatible with push-gateway scraping.

It’s a reliable tool for collecting numeric time-series data and provides support for multi-dimensional data collection and querying. Once data has been collected, it can be visualized using Grafana or other API tools.

Key features provided by Prometheus include:

Prometheus Server: Scrapes and stores time-series data.

PromQL (Prometheus Query Language): This is a powerful and flexible query language that aggregates time series data.

Alert Manager: Triggers alerts when they meet certain rules.

Client libraries: Used for instrumenting application code.

Push Gateway: Gathers services and metrics from external systems that are not compatible with pull-based scraping.

Special Purpose exporters: This allows Prometheus to scrape metrics from third-party systems. E.g node_exporter.

Grafana is an open-source data analysis and visualization platform that visualizes data in interactive dashboards in the form of charts, and graphs from supported data sources. This allows for easier interpretation and understanding of data. It enables monitoring of application and service performance and pinpoints errors or any issues that impact performance.

2. Install Prometheus 

To get started, we will first install and configure Prometheus, then focus on installing Grafana and creating beautiful dashboards for visualizing data collected by Prometheus.

2.1. Setup Prometheus System

First, we are going to create dedicated non-login system account users for Prometheus and Node exporters. Having individual users for each service acts as a security measure to reduce the attack surface in case of an incident. It also allows you to track what resources are associated with which service.

To create Prometheus and node_exporter users, run the following commands

sudo useradd --no-create-home --shell /bin/false prometheus
sudo useradd --no-create-home --shell /bin/false node_exporter

Let’s break down the command options.

• --system: Creates system accounts.
• --no-create-home: Omits the creation of a home directory for Prometheus and node_exporter system accounts. 
• --shell /bin/false: Denies login access to the system account users.

Next, create Prometheus directories for storing Prometheus settings and libraries as shown.

sudo mkdir /etc/prometheus
sudo mkdir /var/lib/prometheus

Now we will have to update the group and user ownership on the newly created directories to Prometheus.