Set up comprehensive NGINX monitoring using Prometheus metrics collection and custom Grafana dashboards to track web server performance, request rates, and response times in real-time.
Prerequisites
- Root or sudo access
- Basic knowledge of NGINX configuration
- Familiarity with systemd services
What this solves
NGINX monitoring with Prometheus and Grafana provides real-time visibility into your web server's performance metrics. This setup tracks request rates, response times, active connections, and server health status through automated metrics collection and customizable dashboards.
Step-by-step configuration
Update system packages
Start by updating your package manager to ensure you get the latest versions.
sudo apt update && sudo apt upgrade -y
Install NGINX with stub_status module
Install NGINX and verify the stub_status module is available for metrics collection.
sudo apt install -y nginx
sudo nginx -V 2>&1 | grep -o with-http_stub_status_module
Configure NGINX stub_status endpoint
Enable the stub_status module to expose basic NGINX metrics at a dedicated endpoint.
server {
listen 80;
server_name example.com;
location / {
root /var/www/html;
index index.html index.htm;
}
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
allow ::1;
deny all;
}
}
Test NGINX configuration
Validate the configuration syntax and reload NGINX to apply changes.
sudo nginx -t
sudo systemctl reload nginx
sudo systemctl enable nginx
Verify stub_status endpoint
Test that the stub_status endpoint is working and returning metrics.
curl http://localhost/nginx_status
You should see output like:
Active connections: 1
server accepts handled requests
1 1 1
Reading: 0 Writing: 1 Waiting: 0
Install Prometheus NGINX Exporter
Download and install the official NGINX Prometheus exporter to convert stub_status metrics to Prometheus format.
wget https://github.com/nginxinc/nginx-prometheus-exporter/releases/download/v1.1.0/nginx-prometheus-exporter_1.1.0_linux_amd64.tar.gz
tar -xzf nginx-prometheus-exporter_1.1.0_linux_amd64.tar.gz
sudo mv nginx-prometheus-exporter /usr/local/bin/
sudo chmod +x /usr/local/bin/nginx-prometheus-exporter
Create systemd service for NGINX exporter
Set up the NGINX exporter as a systemd service for automatic startup and management.
[Unit]
Description=NGINX Prometheus Exporter
After=network.target
[Service]
Type=simple
User=nginx
Group=nginx
ExecStart=/usr/local/bin/nginx-prometheus-exporter -nginx.scrape-uri=http://localhost/nginx_status
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
Create nginx user for exporter
Create a dedicated system user for running the NGINX exporter service.
sudo useradd --system --no-create-home --shell /bin/false nginx-exporter
sudo sed -i 's/User=nginx/User=nginx-exporter/' /etc/systemd/system/nginx-prometheus-exporter.service
sudo sed -i 's/Group=nginx/Group=nginx-exporter/' /etc/systemd/system/nginx-prometheus-exporter.service
Start NGINX exporter service
Enable and start the NGINX Prometheus exporter service.
sudo systemctl daemon-reload
sudo systemctl enable --now nginx-prometheus-exporter
sudo systemctl status nginx-prometheus-exporter
Install Prometheus server
Install Prometheus to collect and store metrics from the NGINX exporter.
wget https://github.com/prometheus/prometheus/releases/download/v2.45.0/prometheus-2.45.0.linux-amd64.tar.gz
tar -xzf prometheus-2.45.0.linux-amd64.tar.gz
sudo mv prometheus-2.45.0.linux-amd64/prometheus /usr/local/bin/
sudo mv prometheus-2.45.0.linux-amd64/promtool /usr/local/bin/
sudo mkdir -p /etc/prometheus /var/lib/prometheus
sudo useradd --system --no-create-home --shell /bin/false prometheus
Configure Prometheus to scrape NGINX metrics
Create the main Prometheus configuration file to scrape metrics from the NGINX exporter.
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
- job_name: 'prometheus'
static_configs:
- targets: ['localhost:9090']
- job_name: 'nginx'
static_configs:
- targets: ['localhost:9113']
scrape_interval: 5s
metrics_path: /metrics
Set Prometheus permissions
Configure correct ownership and permissions for Prometheus directories and files.
sudo chown -R prometheus:prometheus /etc/prometheus /var/lib/prometheus
sudo chmod -R 755 /etc/prometheus /var/lib/prometheus
Create Prometheus systemd service
Set up Prometheus as a systemd service for automatic startup and management.
[Unit]
Description=Prometheus
After=network-online.target
[Service]
Type=simple
User=prometheus
Group=prometheus
ExecStart=/usr/local/bin/prometheus \
--config.file=/etc/prometheus/prometheus.yml \
--storage.tsdb.path=/var/lib/prometheus \
--web.console.templates=/etc/prometheus/consoles \
--web.console.libraries=/etc/prometheus/console_libraries \
--web.listen-address=0.0.0.0:9090 \
--web.enable-lifecycle
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
Start Prometheus service
Enable and start the Prometheus service.
sudo systemctl daemon-reload
sudo systemctl enable --now prometheus
sudo systemctl status prometheus
Install Grafana
Install Grafana for creating dashboards and visualizing NGINX metrics.
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee /etc/apt/sources.list.d/grafana.list
sudo apt update
sudo apt install -y grafana
Start Grafana service
Enable and start the Grafana service.
sudo systemctl daemon-reload
sudo systemctl enable --now grafana-server
sudo systemctl status grafana-server
Configure firewall rules
Open necessary ports for Prometheus and Grafana web interfaces.
sudo ufw allow 3000/tcp comment 'Grafana'
sudo ufw allow 9090/tcp comment 'Prometheus'
sudo ufw allow 9113/tcp comment 'NGINX Exporter'
sudo ufw reload
Add Prometheus data source in Grafana
Access Grafana at http://your-server-ip:3000 with default credentials admin/admin. Add Prometheus as a data source with URL http://localhost:9090.
Name: Prometheus
Type: Prometheus
URL: http://localhost:9090
Access: Server (default)
Scrape interval: 15s
Import NGINX dashboard
Create a comprehensive NGINX monitoring dashboard with key metrics and visualizations.
{
"dashboard": {
"title": "NGINX Monitoring",
"panels": [
{
"title": "Active Connections",
"type": "stat",
"targets": [{
"expr": "nginx_connections_active"
}]
},
{
"title": "Request Rate",
"type": "graph",
"targets": [{
"expr": "rate(nginx_connections_handled[5m])"
}]
},
{
"title": "Connection States",
"type": "graph",
"targets": [
{
"expr": "nginx_connections_reading",
"legendFormat": "Reading"
},
{
"expr": "nginx_connections_writing",
"legendFormat": "Writing"
},
{
"expr": "nginx_connections_waiting",
"legendFormat": "Waiting"
}
]
}
]
}
}
Verify your setup
Test that all components are working correctly and collecting metrics.
# Check NGINX exporter metrics
curl http://localhost:9113/metrics | grep nginx_
Check Prometheus targets
curl http://localhost:9090/api/v1/targets | grep nginx
Check service status
sudo systemctl status nginx nginx-prometheus-exporter prometheus grafana-server
Generate test traffic
for i in {1..10}; do curl http://localhost/; sleep 1; done
nginx_connections_active, nginx_connections_handled, nginx_connections_reading, nginx_connections_writing, and nginx_connections_waiting.Set up alerting rules
Configure Prometheus alerting rules for critical NGINX conditions and integrate with your existing monitoring setup such as Prometheus Alertmanager.
groups:
- name: nginx_alerts
rules:
- alert: NginxDown
expr: up{job="nginx"} == 0
for: 5m
labels:
severity: critical
annotations:
summary: "NGINX exporter is down"
description: "NGINX exporter has been down for more than 5 minutes."
- alert: NginxHighConnections
expr: nginx_connections_active > 1000
for: 10m
labels:
severity: warning
annotations:
summary: "High number of active NGINX connections"
description: "NGINX has {{ $value }} active connections for more than 10 minutes."
- alert: NginxHighRequestRate
expr: rate(nginx_connections_handled[5m]) > 100
for: 5m
labels:
severity: warning
annotations:
summary: "High NGINX request rate"
description: "NGINX request rate is {{ $value }} requests per second."
Advanced dashboard configuration
Enhance your NGINX monitoring with additional panels and metrics for comprehensive observability similar to production monitoring stacks.
Add request duration metrics
Configure NGINX to log request processing times for latency monitoring.
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" '
'rt=$request_time uct="$upstream_connect_time" '
'uht="$upstream_header_time" urt="$upstream_response_time"';
access_log /var/log/nginx/access.log main;
server {
location /metrics {
access_log off;
return 200 "# NGINX metrics exported to Prometheus\n";
add_header Content-Type text/plain;
}
}
}
Common issues
| Symptom | Cause | Fix |
|---|---|---|
| NGINX exporter not starting | Permission denied on stub_status | Check allow 127.0.0.1 in NGINX config |
| No metrics in Prometheus | Incorrect scrape configuration | Verify targets: ['localhost:9113'] in prometheus.yml |
| Grafana shows no data | Data source not configured | Add Prometheus data source with URL http://localhost:9090 |
| Connection refused on port 9113 | NGINX exporter service down | sudo systemctl restart nginx-prometheus-exporter |
| Permission denied errors | Incorrect file ownership | sudo chown prometheus:prometheus /etc/prometheus -R |
allow and deny directives to prevent unauthorized access.Next steps
- Configure NGINX log analysis with Loki and Grafana for centralized log monitoring
- Optimize NGINX performance for high-traffic websites with advanced tuning
- Configure NGINX rate limiting and advanced security for DDoS protection
- Set up NGINX SSL certificate monitoring with Prometheus
- Configure NGINX upstream health checks with Grafana alerts
Running this in production?
Automated install script
Run this to automate the entire setup
#!/usr/bin/env bash
set -euo pipefail
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
# Configuration
NGINX_EXPORTER_VERSION="1.1.0"
PROMETHEUS_VERSION="2.45.0"
GRAFANA_VERSION="10.2.0"
# Usage function
usage() {
echo "Usage: $0 [domain_name] [server_ip]"
echo "Example: $0 example.com 10.0.1.100"
echo "Default: Uses localhost and 127.0.0.1"
exit 1
}
# Cleanup function
cleanup() {
echo -e "${RED}[ERROR] Installation failed. Cleaning up...${NC}"
systemctl stop nginx-prometheus-exporter 2>/dev/null || true
systemctl stop prometheus 2>/dev/null || true
systemctl stop grafana-server 2>/dev/null || true
rm -f /etc/systemd/system/nginx-prometheus-exporter.service
rm -f /etc/systemd/system/prometheus.service
systemctl daemon-reload
}
trap cleanup ERR
# Check if running as root or with sudo
if [[ $EUID -ne 0 ]]; then
echo -e "${RED}This script must be run as root or with sudo${NC}"
exit 1
fi
# Parse arguments
DOMAIN_NAME="${1:-localhost}"
SERVER_IP="${2:-127.0.0.1}"
# Auto-detect distribution
if [ -f /etc/os-release ]; then
. /etc/os-release
case "$ID" in
ubuntu|debian)
PKG_MGR="apt"
PKG_INSTALL="apt install -y"
PKG_UPDATE="apt update && apt upgrade -y"
NGINX_CONFIG_PATH="/etc/nginx/sites-available"
NGINX_ENABLED_PATH="/etc/nginx/sites-enabled"
FIREWALL_CMD="ufw"
;;
almalinux|rocky|centos|rhel|ol)
PKG_MGR="dnf"
PKG_INSTALL="dnf install -y"
PKG_UPDATE="dnf update -y"
NGINX_CONFIG_PATH="/etc/nginx/conf.d"
NGINX_ENABLED_PATH="/etc/nginx/conf.d"
FIREWALL_CMD="firewall-cmd"
;;
fedora)
PKG_MGR="dnf"
PKG_INSTALL="dnf install -y"
PKG_UPDATE="dnf update -y"
NGINX_CONFIG_PATH="/etc/nginx/conf.d"
NGINX_ENABLED_PATH="/etc/nginx/conf.d"
FIREWALL_CMD="firewall-cmd"
;;
amzn)
PKG_MGR="yum"
PKG_INSTALL="yum install -y"
PKG_UPDATE="yum update -y"
NGINX_CONFIG_PATH="/etc/nginx/conf.d"
NGINX_ENABLED_PATH="/etc/nginx/conf.d"
FIREWALL_CMD="firewall-cmd"
;;
*)
echo -e "${RED}Unsupported distribution: $ID${NC}"
exit 1
;;
esac
else
echo -e "${RED}Cannot detect distribution. /etc/os-release not found.${NC}"
exit 1
fi
echo -e "${GREEN}[1/12] Updating system packages...${NC}"
$PKG_UPDATE
echo -e "${GREEN}[2/12] Installing NGINX...${NC}"
$PKG_INSTALL nginx curl wget tar
echo -e "${GREEN}[3/12] Verifying NGINX stub_status module...${NC}"
if ! nginx -V 2>&1 | grep -q "with-http_stub_status_module"; then
echo -e "${RED}NGINX stub_status module not available${NC}"
exit 1
fi
echo -e "${GREEN}[4/12] Configuring NGINX with stub_status...${NC}"
if [[ "$PKG_MGR" == "apt" ]]; then
# Debian/Ubuntu use sites-available/sites-enabled
cat > "$NGINX_CONFIG_PATH/monitoring" << EOF
server {
listen 80;
server_name $DOMAIN_NAME;
location / {
root /var/www/html;
index index.html index.htm;
}
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
allow ::1;
allow $SERVER_IP;
deny all;
}
}
EOF
ln -sf "$NGINX_CONFIG_PATH/monitoring" "$NGINX_ENABLED_PATH/"
rm -f /etc/nginx/sites-enabled/default
else
# RHEL-based use conf.d
cat > "$NGINX_CONFIG_PATH/monitoring.conf" << EOF
server {
listen 80;
server_name $DOMAIN_NAME;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
allow ::1;
allow $SERVER_IP;
deny all;
}
}
EOF
fi
echo -e "${GREEN}[5/12] Testing and starting NGINX...${NC}"
nginx -t
systemctl enable nginx
systemctl restart nginx
echo -e "${GREEN}[6/12] Installing NGINX Prometheus Exporter...${NC}"
cd /tmp
wget "https://github.com/nginxinc/nginx-prometheus-exporter/releases/download/v${NGINX_EXPORTER_VERSION}/nginx-prometheus-exporter_${NGINX_EXPORTER_VERSION}_linux_amd64.tar.gz"
tar -xzf "nginx-prometheus-exporter_${NGINX_EXPORTER_VERSION}_linux_amd64.tar.gz"
mv nginx-prometheus-exporter /usr/local/bin/
chmod 755 /usr/local/bin/nginx-prometheus-exporter
useradd --system --no-create-home --shell /bin/false nginx-exporter 2>/dev/null || true
echo -e "${GREEN}[7/12] Creating NGINX Exporter systemd service...${NC}"
cat > /etc/systemd/system/nginx-prometheus-exporter.service << EOF
[Unit]
Description=NGINX Prometheus Exporter
After=network.target nginx.service
Requires=nginx.service
[Service]
Type=simple
User=nginx-exporter
Group=nginx-exporter
ExecStart=/usr/local/bin/nginx-prometheus-exporter -nginx.scrape-uri=http://localhost/nginx_status
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable nginx-prometheus-exporter
systemctl start nginx-prometheus-exporter
echo -e "${GREEN}[8/12] Installing Prometheus...${NC}"
wget "https://github.com/prometheus/prometheus/releases/download/v${PROMETHEUS_VERSION}/prometheus-${PROMETHEUS_VERSION}.linux-amd64.tar.gz"
tar -xzf "prometheus-${PROMETHEUS_VERSION}.linux-amd64.tar.gz"
mv "prometheus-${PROMETHEUS_VERSION}.linux-amd64/prometheus" /usr/local/bin/
mv "prometheus-${PROMETHEUS_VERSION}.linux-amd64/promtool" /usr/local/bin/
chmod 755 /usr/local/bin/prometheus /usr/local/bin/promtool
useradd --system --no-create-home --shell /bin/false prometheus 2>/dev/null || true
mkdir -p /etc/prometheus /var/lib/prometheus
chown prometheus:prometheus /var/lib/prometheus
echo -e "${GREEN}[9/12] Configuring Prometheus...${NC}"
cat > /etc/prometheus/prometheus.yml << EOF
global:
scrape_interval: 15s
scrape_configs:
- job_name: 'prometheus'
static_configs:
- targets: ['localhost:9090']
- job_name: 'nginx'
static_configs:
- targets: ['localhost:9113']
EOF
chown prometheus:prometheus /etc/prometheus/prometheus.yml
chmod 644 /etc/prometheus/prometheus.yml
cat > /etc/systemd/system/prometheus.service << EOF
[Unit]
Description=Prometheus Server
After=network.target
[Service]
Type=simple
User=prometheus
Group=prometheus
ExecStart=/usr/local/bin/prometheus --config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/var/lib/prometheus --web.console.templates=/etc/prometheus/consoles --web.console.libraries=/etc/prometheus/console_libraries --web.listen-address=0.0.0.0:9090
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable prometheus
systemctl start prometheus
echo -e "${GREEN}[10/12] Installing Grafana...${NC}"
if [[ "$PKG_MGR" == "apt" ]]; then
wget -q -O - https://packages.grafana.com/gpg.key | apt-key add -
echo "deb https://packages.grafana.com/oss/deb stable main" > /etc/apt/sources.list.d/grafana.list
apt update
$PKG_INSTALL grafana
else
cat > /etc/yum.repos.d/grafana.repo << EOF
[grafana]
name=grafana
baseurl=https://packages.grafana.com/oss/rpm
repo_gpgcheck=1
enabled=1
gpgcheck=1
gpgkey=https://packages.grafana.com/gpg.key
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
EOF
$PKG_INSTALL grafana
fi
systemctl enable grafana-server
systemctl start grafana-server
echo -e "${GREEN}[11/12] Configuring firewall...${NC}"
if command -v ufw >/dev/null 2>&1; then
ufw --force enable
ufw allow 80/tcp
ufw allow 3000/tcp
ufw allow 9090/tcp
elif command -v firewall-cmd >/dev/null 2>&1; then
systemctl enable firewalld
systemctl start firewalld
firewall-cmd --permanent --add-port=80/tcp
firewall-cmd --permanent --add-port=3000/tcp
firewall-cmd --permanent --add-port=9090/tcp
firewall-cmd --reload
fi
echo -e "${GREEN}[12/12] Verifying installation...${NC}"
sleep 5
# Verify NGINX status endpoint
if curl -s http://localhost/nginx_status | grep -q "Active connections"; then
echo -e "${GREEN}✓ NGINX stub_status working${NC}"
else
echo -e "${RED}✗ NGINX stub_status failed${NC}"
exit 1
fi
# Verify NGINX exporter
if curl -s http://localhost:9113/metrics | grep -q "nginx_"; then
echo -e "${GREEN}✓ NGINX Exporter working${NC}"
else
echo -e "${RED}✗ NGINX Exporter failed${NC}"
exit 1
fi
# Verify Prometheus
if curl -s http://localhost:9090/-/ready | grep -q "Prometheus"; then
echo -e "${GREEN}✓ Prometheus working${NC}"
else
echo -e "${RED}✗ Prometheus failed${NC}"
exit 1
fi
# Verify Grafana
if systemctl is-active grafana-server >/dev/null 2>&1; then
echo -e "${GREEN}✓ Grafana working${NC}"
else
echo -e "${RED}✗ Grafana failed${NC}"
exit 1
fi
echo -e "${GREEN}Installation completed successfully!${NC}"
echo -e "${YELLOW}Access URLs:${NC}"
echo "- Grafana: http://$SERVER_IP:3000 (admin/admin)"
echo "- Prometheus: http://$SERVER_IP:9090"
echo "- NGINX Status: http://$SERVER_IP/nginx_status"
echo "- NGINX Metrics: http://$SERVER_IP:9113/metrics"
Review the script before running. Execute with: bash install.sh