Expose Self-Hosted Grafana behind VPN with Cloudflare Tunnel

Introduction

Hosting the Grafana on the home server behind the VPN protection is safe. But it’s inconvenient when we need to access it and connect to other APN server (e.g.,)

Are there any solutions to safely expose a self-hosted Grafana instance beyond the VPN boundary to the public internet?

In this article, we will go through the steps to set up the monitor system for the home server and expose it.

Agenda

• Prerequisites
• Requirements
• System Design Overview
• Monitor Home Servers with Grafana, Prometheus, Exporters
• Expose System to Internet with Cloudflare Tunnel, Cloudflare Access
• Architecture Decision Records

Prerequisites

• Have an account and a domain on [Cloudflare](https://www.cloudflare.com/)

Requirements

Functional Requirements

• Monitor multiple servers and metrics
• Monitor resources of macOS server (e.g., CPU/RAM usage, storage, etc)
• Monitor resources and SNMP devices of the Linux VM server
• Expose the monitoring system to the public internet

Non-functional Requirements

• This system won’t be a distributed system (it’s just for the home server) => No consistency, partitioning concerns
• Resource efficiency
• Maintainability
• Security: must authenticate the user after exposing

Constraints

• The target server is behind the VPN
• The client is behind VPN
• Those two network can’t reach out to each other

System Design Overview

Component Diagram

The following diagram is the architecture for this monitoring system.

• Monitor Home Servers with Grafana, Prometheus, Exporters
• Expose System to Internet with Cloudflare Tunnel, Cloudflare Acess

Monitor Home Servers with Grafana, Prometheus, Exporters

The Prometheus is one of the datasource for Grafana. It provides the flexibility of queries and metrics.

Exporter is responsible to collect the data from our target machines, we have to install different exporters according to different purposes.

The following diagram shows the architecture of the Grafana, Prometheus, exporters. The Prometheus can poll the data from multiple exporter.

Create Exporters for Resources and SNMP Devices of the Linux VM Server

• Create the following folder structure
• node-exporter: store docker compose file for the Node exporter
• smnp-exporter: store docker compose file for the SNMP exporter

.
├── node-exporter
└── snmp-exporter

• Create node-exporter/docker-compose.yml to launch Node exporter as docker container

[embed]

• Launch the Node exporter

sudo docker-compose up -d

• Create snmp-exporter/docker-compose.yml to launch SNMP exporter as docker container

[embed]

• Launch the SNMP exporter

sudo docker-compose up -d

Monitor Resources of the macOS Server (CPU, RAM, etc)

• Install Node exporter

brew install node_exporter
brew services start node_exporter

• Create the following folder structure
• grafana/dashboard: store the Grafana dashboard files
• grafana/provisioning/dashboards: store dashboard configs for Grafana
• grafana/provisioning/datasources: store Prometheus configs for Grafana
• prometheus: store configs for Prometheus and the IPs of SNMP devices

.
├── grafana
│   ├── dashboards
│   └── provisioning
│       ├── dashboards
│       └── datasources
└── prometheus

• Create docker-compose.yml to launch Grafana and Promethues as docker containers

[embed]

• Create .env to store environment variables for Grafana

[embed]

• Create prometheus/prometheus.yml to manage monitored targets as jobs
• Prometheus itself
• Node exporter on the host machine
• Node exporter on the Linux server
• SNMP exporter on the Linux server

[embed]

• Create prometheus/snmp_targets.json for the SNMP exporter to trace SNMP devices

[embed]

• Create grafana/provisioning/datasources/prometheus.yml to set Prometheus as Grafana datasource

[embed]

• Create grafana/provisioning/dashboard/default.yml to set dashboard configurations

[embed]

• Launch the Grafana and Prometheus

sudo docker-compose up -d

Import Dashboard for Node Exporter

• Open the Grafana on localhost:3000

• Download the dashbard template for the Node exporter
• Check the node exporter dashboard

Expose System to Internet with Cloudflare Tunnel, Cloudflare Access

To expose the local port to the internet without exposing IP address, we can use Cloudflare Tunnel secured by the Claudflare Access.

Even though the Grafana has been exposed to the internet, Cloudflare Access will only allow the specific email to login.

Establish the Cloudflare Tunnel between Server and Cloudflare

• Install Cloudflared

# 1. Install cloudflared
brew install cloudflared

# 2. Login to Cloudflare
cloudflared tunnel login

# 3. Establish Tunnel
cloudflared tunnel create home-grafana

• Copy the id and the file path from the response

Tunnel credentials written to 
/Users/hsiehsean/.cloudflared/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.json. 
cloudflared chose this file based on where your origin certificate was found. 
Keep this file secret. To revoke these credentials, delete the tunnel.

Created tunnel home-grafana with id xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

• Create the configuration file with the following format by using the nano ~/.cloudflared/config.yml

tunnel: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
credentials-file: /Users/hsiehsean/.cloudflared/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.json

ingress:
  - hostname: grafana.alayman.io
    service: http://localhost:3000 # Forward to the local service
  - service: http_status:404 # Other traffic

• Create a DNS record

cloudflared tunnel route dns home-grafana grafana.{your domain}

• Check the DNS records on Cloudflare

Set up the Cloudflare Access

• Check the Cloudflare Access

• Create a new policy to only for the personal email to login

• Create a new application

• Open grafana.{your domain} on the browser to access the login page

Architecture Decision Records

ADR 1. Use Cloudflare Tunnel to Enable Access to Home Grafana from Corporate Network

Consider we can’t install software on the company’s notebook, we decide to use the Cloudflare Tunnel to expose Grafana to the company laptop via browser.

The following table is our trade-off.

[embed]

ADR 2. Add Zero Trust Identity Verification Layer in Front of Grafana Using Cloudflare Access

Grafana Native Auth and Grafana + OAuth protect Grafana after the client reaches out to the server. Cloudflare Access can reject the request at the Cloudflare level.

The following table is our trade-off.

[embed]

Summary

Thanks for your patient. I am Sean. I work as a software engineer.

This article is my note. Please feel free to give me advice if any mistakes. I am looking forward to your feedback.

• Subscribe me

[embed]Join Medium with my referral link — Jen-Hsuan Hsieh (Sean) As a Medium member, a portion of your membership fee goes to writers you read, and you get full access to every story…medium.com

• Recommended places for digital nomads

[embed]A Layman Emoji cafe is founded by the Taiwan Digital Nomad Association (TDNA) on 2023. Sometimes I like to worked for my side…seanhs.blogspot.com

• The Daily Learning website

[embed]Search Articles for Web Development | ALayman Daily Learning ALayman Daily learning provides articles, challenges, or videos to people who are also self-learner for programming.www.alayman.io