Creating a monitoring dashboard using Prometheus and Grafana

Prometheus is a monitoring platform that collects metrics from monitored targets by scraping metrics HTTP endpoints on these targets.

Grafana is a graphical tool that connects to Prometheus and helps build a visualization dashboard.

In this blog

  1. We shall spinup Prometheus as a docker container on a Ubuntu based server in Azure cloud.
  2. Install Node exporter , a tool that scrapes metrics from the host linux machine
  3. Spin Up Grafana as another docker container on the same Ubuntu server
  4. Create a datasource from Grafana to Prometheus
  5. Build a dashboard to view the Node exporter metrics
  • Spin Up an Ubuntu server in Azure cloud
  • copy the following Prometheus configuration file : prometheus.yml to /home/ubuntu/config directory
scrape_configs:
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.
    static_configs:
    - targets: ['xx.yy.ww.zz:9090']

  - job_name: 'node'
    static_configs:
    - targets: ['xx.yy.ww.zz:9100']

where xx.yy.ww.zz is the public ip address of the Azure server.

The job name ‘node’ tells the Prometheus instance to scrape from the Node Exporter via xx.yy.ww.zz:9100

  • download , install and startup node exporter
sudo wget https://github.com/prometheus/node_exporter/releases/download/v1.0.1/node_exporter-1.0.1.linux-amd64.tar.gz

tar zxvf node_exporter-1.0.1.linux-amd64.tar.gz

cd node_exporter-1.0.1.linux-amd64

./node_exporter
  • Fireup Prometheus docker container
sudo apt-get install docker.io

sudo docker run --name prometheus -d -p 0.0.0.0:9090:9090 --volume="$PWD/config":/etc/config prom/prometheus --config.file=/etc/config/prometheus.yml

The ip address is entered 0.0.0.0 above in order to facilitate remote access of the server. Its ironic that all content on the web that provides information on setting up Prometheus or for that matter any other tool, is setup to run on localhost or 127.0.0.1. Who would ever run a server on localhost ?

  • Fireup Grafana docker container
sudo docker run -d --name=grafana -p 0.0.0.0:3000:3000 grafana/grafana
  • Ensure that ports 9090,3000 and 9100 are opened up for inbound traffic in Azure and also in Ubuntu using
sudo ufw allow <port>
  • Incase of any issues stop and restart the docker containers using the commands
docker container stop 65dccb7ea5eb970da62f639457bbc28962ac90f7bc84f2fcb10b1fa3a9691aa1

docker container rm 65dccb7ea5eb970da62f639457bbc28962ac90f7bc84f2fcb10b1fa3a9691aa1
  • Create a new datasource for Prometheus in Grafana.
  • Enter node exporter metrics id in the field and click load.
  • View the real time metrics generated.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: