Configure HAProxy with Consul for dynamic service discovery

Advanced 50 min Aug 03, 2026 197 views
Ubuntu 24.04 Debian 12 AlmaLinux 9 Rocky Linux 9

Learn how to combine HAProxy, Consul and consul-template to build a self-updating load balancer that discovers backends automatically, reacts to health checks, and reloads without downtime.

Prerequisites

  • A running Consul server cluster (or single dev-mode server) reachable from the HAProxy node
  • Root or sudo access on all nodes
  • Basic familiarity with HAProxy configuration syntax
  • Application services exposing an HTTP health check endpoint

What this solves

Static HAProxy backend lists break down once services scale up, scale down or move between hosts. This tutorial wires HAProxy to Consul's service catalog through consul-template, so backend servers are generated dynamically from live health check data and configuration reloads happen automatically when the topology changes.

By the end you will have a Consul agent registering services with health checks, consul-template rendering HAProxy configuration from those checks, and a reload pipeline that survives scaling events and failures without manual intervention.

Step-by-step installation

Install HAProxy

HAProxy will act as the load balancer that receives dynamically generated backend definitions.

sudo apt update
sudo apt install -y haproxy
sudo dnf install -y haproxy

Install the Consul agent

Each HAProxy node runs a local Consul agent in client mode, joined to your Consul cluster, to query service health data with low latency.

curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update
sudo apt install -y consul
sudo dnf install -y dnf-plugins-core
sudo dnf config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
sudo dnf install -y consul

Configure the Consul client agent

This configuration joins the node to an existing Consul server cluster. Replace 203.0.113.10 with your actual Consul server addresses.

datacenter = "dc1"
data_dir = "/opt/consul"
bind_addr = "{{ GetInterfaceIP \"eth0\" }}"
retry_join = ["203.0.113.10", "203.0.113.11", "203.0.113.12"]
client_addr = "127.0.0.1"
enable_local_script_checks = true
ports {
  grpc = 8502
}
sudo mkdir -p /opt/consul
sudo chown -R consul:consul /opt/consul
sudo systemctl enable --now consul
consul members
Note: If you are building a fresh Consul cluster rather than joining an existing one, see install and configure Consul for service discovery with clustering and security first.

Register a service with health checks

Services register themselves in Consul's catalog along with a health check. HAProxy will only route to instances that pass this check.

{
  "service": {
    "name": "web-app",
    "port": 8080,
    "tags": ["http"],
    "check": {
      "http": "http://localhost:8080/healthz",
      "interval": "10s",
      "timeout": "3s",
      "deregister_critical_service_after": "90s"
    }
  }
}
sudo systemctl reload consul
consul catalog services
consul health checks web-app

Repeat this registration file on every application host, adjusting the port for each instance. Consul tracks each instance independently by node and check status.

Install consul-template

consul-template watches the Consul catalog for changes and renders templates, in this case an HAProxy configuration file, whenever backend state changes.

sudo apt install -y consul-template
sudo dnf install -y consul-template
Note: If your distro's repo does not ship consul-template, download the binary release from HashiCorp and place it in /usr/local/bin with chmod 755.

Create the HAProxy template

This template queries Consul for healthy instances of web-app and generates an HAProxy frontend and backend block. It also defines the stats page and load balancing algorithm.

global
    log /dev/log local0
    maxconn 20000
    user haproxy
    group haproxy

defaults
    log global
    mode http
    timeout connect 5s
    timeout client 30s
    timeout server 30s
    option httplog

frontend stats
    bind *:8404
    stats enable
    stats uri /stats
    stats refresh 10s
    stats auth admin:{{ key "haproxy/stats_password" }}

frontend web_front
    bind *:80
    default_backend web_back

backend web_back
    balance leastconn
    option httpchk GET /healthz
{{ range service "web-app" }}
    server {{ .Node }}-{{ .Port }} {{ .Address }}:{{ .Port }} check
{{ end }}
Warning: Do not hardcode the stats password in the template. Store it in Consul's KV store with consul kv put haproxy/stats_password 'S3cur3-Stats!Pass' so it never appears in version control.

Configure consul-template to render and reload

This configuration tells consul-template where to write the rendered file and what command to run afterward to validate and reload HAProxy safely.

consul {
  address = "127.0.0.1:8500"
}

template {
  source      = "/etc/consul-template/haproxy.ctmpl"
  destination = "/etc/haproxy/haproxy.cfg"
  command     = "haproxy -c -f /etc/haproxy/haproxy.cfg && systemctl reload haproxy"
  command_timeout = "30s"
  wait {
    min = "2s"
    max = "10s"
  }
}

The wait block debounces rapid-fire changes during scaling events, so a burst of instance registrations does not trigger dozens of reloads in a few seconds.

Set correct ownership and permissions

consul-template needs to write to the HAProxy config directory and trigger a reload. Rather than opening permissions widely, grant a dedicated service account exactly what it needs.

sudo useradd -r -s /usr/sbin/nologin consul-template
sudo chown -R consul-template:haproxy /etc/haproxy
sudo chmod 750 /etc/haproxy
sudo chmod 640 /etc/haproxy/haproxy.cfg
Never use chmod 777. It gives every user on the system full read, write and execute access to your HAProxy configuration, including the stats credentials. Use group ownership between consul-template and haproxy instead, with 750/640 permissions so only the two accounts that need access have it.

Grant the reload command via sudoers instead of running consul-template as root:

consul-template ALL=(root) NOPASSWD: /bin/systemctl reload haproxy, /usr/sbin/haproxy -c -f /etc/haproxy/haproxy.cfg

Run consul-template as a systemd service

Running consul-template under systemd ensures it restarts on failure and starts on boot alongside Consul and HAProxy.

[Unit]
Description=Consul Template for HAProxy
After=network-online.target consul.service
Wants=network-online.target

[Service]
User=consul-template
Group=haproxy
ExecStart=/usr/bin/consul-template -config=/etc/consul-template/config.hcl
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload
sudo systemctl enable --now consul-template
sudo systemctl status consul-template

Enable and start HAProxy

Start HAProxy once the first rendered config is in place.

sudo haproxy -c -f /etc/haproxy/haproxy.cfg
sudo systemctl enable --now haproxy
sudo systemctl status haproxy

Verify your setup

Confirm Consul sees the service as healthy, the template rendered correctly, and HAProxy is routing traffic.

consul catalog services
consul health checks web-app
cat /etc/haproxy/haproxy.cfg | grep -A5 backend
curl -I http://127.0.0.1/
curl -u admin:S3cur3-Stats!Pass http://127.0.0.1:8404/stats

Register a second instance of web-app on another port and watch the backend update without restarting HAProxy manually.

sudo journalctl -u consul-template -f

You should see a render event followed by a successful reload log line within a few seconds of the new registration.

Testing failover and scaling scenarios

Simulate a backend failure

Stop the application process on one instance and confirm Consul marks the check critical, and that consul-template removes it from the backend within one interval cycle.

sudo systemctl stop web-app
consul health checks web-app
curl -u admin:S3cur3-Stats!Pass http://127.0.0.1:8404/stats

The stats page should show the server entry as down, and the HAProxy config should no longer list it after the next consul-template render.

Simulate horizontal scale-out

Register three additional instances in quick succession to confirm the debounce window in the wait block coalesces the reload into a single event instead of five separate reloads.

for port in 8081 8082 8083; do
  sudo tee /etc/consul.d/web-app-$port.json > /dev/null <<EOF
{"service":{"name":"web-app","port":$port,"check":{"http":"http://localhost:$port/healthz","interval":"10s"}}}
EOF
done
sudo systemctl reload consul

Watch journalctl -u consul-template -f during this test. You should see one render cycle cover all three new instances rather than three separate reload commands.

Common issues

SymptomCauseFix
HAProxy config never updatesconsul-template cannot reach the Consul HTTP APICheck consul_addr in config.hcl and confirm curl 127.0.0.1:8500/v1/status/leader returns a leader
Reload command fails silentlyconsul-template user lacks sudo rights for systemctl reloadVerify the sudoers entry with sudo -l -U consul-template
Backend flaps rapidly under loadHealth check interval too aggressive for app startup timeIncrease interval and add a startup grace period with deregister_critical_service_after
Stats page returns 403Wrong credentials or password stored incorrectly in Consul KVRe-run consul kv get haproxy/stats_password and confirm it matches your curl command
New instances never appear in backendHealth check failing due to firewall blocking the check portAllow the specific port with sudo ufw allow from 203.0.113.0/24 to any port 8080 instead of disabling the firewall

Next steps

Running this in production?

Want this handled for you? Running this at scale adds a second layer of work: capacity planning for the Consul cluster, failover drills for HAProxy nodes, TLS and ACL rotation, and being on-call when a datacenter link flaps at 3am. See how we run infrastructure like this for European teams.

Automated install script

Run this to automate the entire setup

#haproxy #consul #service discovery #consul-template #load balancing #high availability

Verwandte Tutorials

networking Intermediate

Set up HAProxy SSL termination with Let's Encrypt certificates

Learn how to terminate SSL/TLS at HAProxy using Let's Encrypt certificates, redirect HTTP to HTTPS, automate renewal with deploy hooks, and harden your cipher suites for production load balancing.

35 min 4 distros 168 views
networking Advanced

Implement enterprise network QoS with Cisco integration using FRRouting and traffic shaping

Configure enterprise-grade Quality of Service policies with DSCP marking, traffic shaping using tc and HTB, and seamless integration with Cisco equipment through FRRouting BGP and OSPF routing protocols for comprehensive network bandwidth management.

45 min 4 distros 609 views
networking Advanced

Configure advanced iptables QoS with fwmark and multiple interfaces

Set up traffic shaping with iptables packet marking, HTB queueing discipline, and multi-interface QoS policies for bandwidth management and network performance optimization.

45 min 4 distros 698 views
Browse all tutorials in our complete directory →

Sie möchten das nicht selbst verwalten?

Wir betreiben Infrastruktur für Unternehmen, die auf Verfügbarkeit angewiesen sind. Vollständig verwaltet, mit einem festen Ansprechpartner, der Ihre Umgebung kennt.

Sie erhalten einen festen Ansprechpartner, der Ihr Setup kennt

Rotterdam 04:04 · erreichbar per Nachricht, kein Ticketformular

+31 10 477 5362 WhatsApp