Configure Consul's Access Control List (ACL) system with bootstrap tokens, implement TLS encryption for client-server communication, and enable gossip encryption to secure your Consul cluster for production environments with comprehensive authentication and authorization.

Prerequisites

Consul installed and basic cluster setup
Root or sudo access
Basic understanding of PKI and certificates
Network connectivity between Consul nodes

What this solves

Consul's default configuration lacks security controls, allowing any client to access the cluster without authentication. This tutorial implements Consul's ACL system to provide fine-grained access control, enables TLS encryption for secure client-server communication, and configures gossip encryption to protect cluster communication. You'll create ACL policies and tokens for service authentication while establishing proper certificate management for production deployments.

Prerequisites and Consul installation verification

Verify Consul installation

Confirm that Consul is installed and check the current version before proceeding with security configuration.

consul version
sudo systemctl status consul

Stop Consul service for configuration

Stop the Consul service to modify configuration files safely without conflicts.

sudo systemctl stop consul

Create backup of existing configuration

Create a backup of your current Consul configuration before making security changes.

sudo cp -r /etc/consul.d /etc/consul.d.backup
sudo cp /opt/consul/consul.hcl /opt/consul/consul.hcl.backup

Configure Consul ACL system with bootstrap token

Enable ACL system in Consul configuration

Modify the Consul configuration to enable the ACL system with default deny policy for maximum security.

datacenter = "dc1"
data_dir = "/opt/consul"
log_level = "INFO"
server = true
bind_addr = "0.0.0.0"
client_addr = "0.0.0.0"
bootstrap_expect = 3
ui_config {
  enabled = true
}

ACL Configuration
acl = {
  enabled = true
  default_policy = "deny"
  enable_token_persistence = true
}

Performance and Connect
performance {
  raft_multiplier = 1
}
connect {
  enabled = true
}

Set proper permissions for Consul configuration

Ensure the Consul configuration directory has correct ownership and permissions for security.

Never use chmod 777. It gives every user on the system full access to your files. Instead, fix ownership with chown and use minimal permissions.

sudo chown -R consul:consul /etc/consul.d
sudo chmod 750 /etc/consul.d
sudo chmod 640 /etc/consul.d/*.hcl

Start Consul with ACL enabled

Start Consul service with the new ACL-enabled configuration to prepare for bootstrap token creation.

sudo systemctl start consul
sudo systemctl status consul

Bootstrap the ACL system

Create the initial bootstrap token that provides full administrative access to the Consul cluster.

consul acl bootstrap

Note: Save the bootstrap token SecretID output. This token provides full administrative access and cannot be recovered if lost.

Configure environment variable for ACL token

Set the bootstrap token as an environment variable for subsequent Consul CLI operations.

export CONSUL_HTTP_TOKEN="your-bootstrap-token-secret-id"
echo 'export CONSUL_HTTP_TOKEN="your-bootstrap-token-secret-id"' >> ~/.bashrc

Set up TLS encryption for client-server communication

Create Consul CA and certificates directory

Create a dedicated directory for Consul certificates and generate the Certificate Authority for TLS encryption.

sudo mkdir -p /opt/consul/tls
sudo chown consul:consul /opt/consul/tls
sudo chmod 750 /opt/consul/tls

Generate Consul CA certificate

Create the Certificate Authority that will sign all Consul server and client certificates.

cd /opt/consul/tls
sudo -u consul consul tls ca create

Generate server certificates

Create server certificates for each Consul server node in your cluster.

sudo -u consul consul tls cert create -server -dc dc1
sudo -u consul consul tls cert create -server -dc dc1
sudo -u consul consul tls cert create -server -dc dc1

Generate client certificate

Create client certificates for Consul agents and applications that need to communicate with the cluster.

sudo -u consul consul tls cert create -client -dc dc1

Configure TLS in Consul configuration

Update the Consul configuration to enable TLS encryption for all communications.

datacenter = "dc1"
data_dir = "/opt/consul"
log_level = "INFO"
server = true
bind_addr = "0.0.0.0"
client_addr = "0.0.0.0"
bootstrap_expect = 3
ui_config {
  enabled = true
}

ACL Configuration
acl = {
  enabled = true
  default_policy = "deny"
  enable_token_persistence = true
}

TLS Configuration
tls {
  defaults {
    ca_file = "/opt/consul/tls/consul-agent-ca.pem"
    cert_file = "/opt/consul/tls/dc1-server-consul-0.pem"
    key_file = "/opt/consul/tls/dc1-server-consul-0-key.pem"
    verify_incoming = true
    verify_outgoing = true
  }
  internal_rpc {
    verify_server_hostname = true
  }
}

ports {
  grpc_tls = 8503
  https = 8501
}

Performance and Connect
performance {
  raft_multiplier = 1
}
connect {
  enabled = true
}

Enable gossip encryption and secure cluster communication

Generate gossip encryption key

Create a symmetric encryption key for securing gossip protocol communications between Consul agents.

consul keygen

Note: Save this key securely. All Consul agents in the cluster must use the same gossip encryption key.

Add gossip encryption to configuration

Update the Consul configuration to include gossip encryption and disable script checks for security.

datacenter = "dc1"
data_dir = "/opt/consul"
log_level = "INFO"
server = true
bind_addr = "0.0.0.0"
client_addr = "0.0.0.0"
bootstrap_expect = 3
ui_config {
  enabled = true
}

Gossip Encryption
encrypt = "your-generated-gossip-key"
encrypt_verify_incoming = true
encrypt_verify_outgoing = true

Security Hardening
enable_script_checks = false
enable_local_script_checks = false

ACL Configuration
acl = {
  enabled = true
  default_policy = "deny"
  enable_token_persistence = true
}

TLS Configuration
tls {
  defaults {
    ca_file = "/opt/consul/tls/consul-agent-ca.pem"
    cert_file = "/opt/consul/tls/dc1-server-consul-0.pem"
    key_file = "/opt/consul/tls/dc1-server-consul-0-key.pem"
    verify_incoming = true
    verify_outgoing = true
  }
  internal_rpc {
    verify_server_hostname = true
  }
}

ports {
  grpc_tls = 8503
  https = 8501
}

Performance and Connect
performance {
  raft_multiplier = 1
}
connect {
  enabled = true
}

Create ACL policies and tokens for service authentication

Create a service registration policy

Define an ACL policy that allows services to register with Consul and perform health checks.

service_prefix "" {
  policy = "write"
}

node_prefix "" {
  policy = "read"
}

key_prefix "" {
  policy = "read"
}

session_prefix "" {
  policy = "read"
}

Create the service policy in Consul

consul acl policy create -name "service-policy" -description "Service registration and health checks" -rules @/tmp/service-policy.hcl

Create a read-only policy for monitoring

Define a restricted policy for monitoring tools that only need read access to Consul data.

service_prefix "" {
  policy = "read"
}

node_prefix "" {
  policy = "read"
}

key_prefix "" {
  policy = "read"
}

session_prefix "" {
  policy = "read"
}

operator = "read"

query_prefix "" {
  policy = "read"
}

Create the read-only policy

consul acl policy create -name "readonly-policy" -description "Read-only access for monitoring" -rules @/tmp/readonly-policy.hcl

Create service token

Generate an ACL token with service registration permissions for applications to use.

consul acl token create -description "Service registration token" -policy-name service-policy

Create monitoring token

Generate a read-only token for monitoring tools like Prometheus or Grafana to access Consul metrics.

consul acl token create -description "Monitoring read-only token" -policy-name readonly-policy

Implement certificate management and rotation

Create certificate rotation script

Prepare an automated script to handle certificate rotation before expiration.

#!/bin/bash

Consul Certificate Rotation Script
CONSUL_TLS_DIR="/opt/consul/tls"
BACKUP_DIR="/opt/consul/tls-backup-$(date +%Y%m%d)"

Create backup
sudo mkdir -p "$BACKUP_DIR"
sudo cp -r "$CONSUL_TLS_DIR"/*.pem "$BACKUP_DIR"/

Generate new certificates
cd "$CONSUL_TLS_DIR"
sudo -u consul consul tls cert create -server -dc dc1
sudo -u consul consul tls cert create -client -dc dc1

Update configuration with new certificate paths
sudo systemctl reload consul

echo "Certificate rotation completed. Backup stored in: $BACKUP_DIR"

Make rotation script executable

Set proper permissions for the certificate rotation script and create directory structure.

sudo mkdir -p /opt/consul/scripts
sudo chown consul:consul /opt/consul/scripts
sudo chmod 750 /opt/consul/scripts
sudo chmod 750 /opt/consul/scripts/rotate-certs.sh

Configure systemd environment for TLS

Create systemd environment file to ensure Consul uses TLS for all operations.

CONSUL_HTTP_SSL=true
CONSUL_HTTP_SSL_VERIFY=true
CONSUL_CACERT=/opt/consul/tls/consul-agent-ca.pem
CONSUL_CLIENT_CERT=/opt/consul/tls/dc1-client-consul-0.pem
CONSUL_CLIENT_KEY=/opt/consul/tls/dc1-client-consul-0-key.pem

Restart Consul with full security configuration

Restart Consul service to apply all security configurations including ACL, TLS, and gossip encryption.

sudo systemctl restart consul
sudo systemctl status consul

Verify your setup

# Verify Consul is running with TLS
curl -k --cert /opt/consul/tls/dc1-client-consul-0.pem \
     --key /opt/consul/tls/dc1-client-consul-0-key.pem \
     --cacert /opt/consul/tls/consul-agent-ca.pem \
     "https://localhost:8501/v1/status/leader"

Check ACL system status
consul acl auth-method list

Verify gossip encryption
consul members

Test token authentication
consul catalog services

Common issues

Symptom	Cause	Fix
"Permission denied" ACL errors	Missing or invalid ACL token	Set `CONSUL_HTTP_TOKEN` environment variable with valid token
TLS certificate verification fails	Certificate path or permissions issue	Verify certificate files exist and `consul` user can read them
Consul UI shows "ACL not found"	Bootstrap token not properly set	Re-run bootstrap process or verify token configuration
Gossip encryption errors in logs	Mismatched encryption keys across nodes	Ensure all nodes use identical gossip encryption key
Service registration fails	Service lacks proper ACL token	Create service token with appropriate policy and configure application
Certificate expired errors	TLS certificates have expired	Run certificate rotation script and restart Consul

Next steps

Automated install script

Run this to automate the entire setup

install.sh

#!/usr/bin/env bash
set -euo pipefail

# Consul ACL and TLS Security Setup Script
# Implements ACL system, TLS encryption, and gossip encryption for production

# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color

# Default values
DATACENTER="${1:-dc1}"
BOOTSTRAP_EXPECT="${2:-1}"

usage() {
    echo "Usage: $0 [datacenter] [bootstrap_expect]"
    echo "  datacenter: Consul datacenter name (default: dc1)"
    echo "  bootstrap_expect: Number of servers to expect (default: 1)"
    exit 1
}

log_info() { echo -e "${GREEN}$1${NC}"; }
log_warn() { echo -e "${YELLOW}$1${NC}"; }
log_error() { echo -e "${RED}$1${NC}"; }

# Error cleanup
cleanup() {
    log_error "Script failed. Restoring backup if available..."
    if [ -f /opt/consul/consul.hcl.backup ]; then
        sudo cp /opt/consul/consul.hcl.backup /opt/consul/consul.hcl 2>/dev/null || true
    fi
    if [ -d /etc/consul.d.backup ]; then
        sudo rm -rf /etc/consul.d && sudo mv /etc/consul.d.backup /etc/consul.d 2>/dev/null || true
    fi
}

trap cleanup ERR

# Detect OS
if [ -f /etc/os-release ]; then
    . /etc/os-release
    case "$ID" in
        ubuntu|debian) PKG_MGR="apt"; PKG_INSTALL="apt install -y" ;;
        almalinux|rocky|centos|rhel|ol|fedora) PKG_MGR="dnf"; PKG_INSTALL="dnf install -y" ;;
        amzn) PKG_MGR="yum"; PKG_INSTALL="yum install -y" ;;
        *) log_error "Unsupported distro: $ID"; exit 1 ;;
    esac
else
    log_error "Cannot detect OS distribution"
    exit 1
fi

# Check if running as root or with sudo
if [ $EUID -ne 0 ]; then
    log_error "This script must be run as root or with sudo"
    exit 1
fi

log_info "[1/12] Verifying prerequisites..."
if ! command -v consul >/dev/null 2>&1; then
    log_error "Consul is not installed. Please install Consul first."
    exit 1
fi

consul version
systemctl status consul --no-pager || true

log_info "[2/12] Stopping Consul service..."
systemctl stop consul

log_info "[3/12] Creating backup of existing configuration..."
if [ -f /opt/consul/consul.hcl ]; then
    cp /opt/consul/consul.hcl /opt/consul/consul.hcl.backup
fi
if [ -d /etc/consul.d ]; then
    cp -r /etc/consul.d /etc/consul.d.backup
fi

log_info "[4/12] Generating gossip encryption key..."
GOSSIP_KEY=$(consul keygen)

log_info "[5/12] Creating ACL-enabled configuration..."
mkdir -p /opt/consul /etc/consul.d

cat > /opt/consul/consul.hcl <<EOF
datacenter = "$DATACENTER"
data_dir = "/opt/consul"
log_level = "INFO"
server = true
bind_addr = "0.0.0.0"
client_addr = "0.0.0.0"
bootstrap_expect = $BOOTSTRAP_EXPECT
ui_config {
  enabled = true
}

# ACL Configuration
acl = {
  enabled = true
  default_policy = "deny"
  enable_token_persistence = true
}

# Gossip Encryption
encrypt = "$GOSSIP_KEY"

# Performance and Connect
performance {
  raft_multiplier = 1
}
connect {
  enabled = true
}
EOF

log_info "[6/12] Setting proper permissions..."
chown -R consul:consul /opt/consul
chmod 750 /opt/consul
chmod 640 /opt/consul/consul.hcl

if [ -d /etc/consul.d ]; then
    chown -R consul:consul /etc/consul.d
    chmod 750 /etc/consul.d
    find /etc/consul.d -name "*.hcl" -exec chmod 640 {} \;
fi

log_info "[7/12] Starting Consul with ACL enabled..."
systemctl start consul
sleep 5

# Wait for Consul to be ready
RETRIES=30
while [ $RETRIES -gt 0 ]; do
    if consul members >/dev/null 2>&1; then
        break
    fi
    sleep 2
    ((RETRIES--))
done

if [ $RETRIES -eq 0 ]; then
    log_error "Consul failed to start properly"
    exit 1
fi

log_info "[8/12] Bootstrapping ACL system..."
BOOTSTRAP_OUTPUT=$(consul acl bootstrap 2>/dev/null || echo "Already bootstrapped")
if [[ "$BOOTSTRAP_OUTPUT" == *"SecretID"* ]]; then
    BOOTSTRAP_TOKEN=$(echo "$BOOTSTRAP_OUTPUT" | grep "SecretID" | awk '{print $2}')
    log_info "Bootstrap token created: $BOOTSTRAP_TOKEN"
    echo "export CONSUL_HTTP_TOKEN=\"$BOOTSTRAP_TOKEN\"" >> /root/.bashrc
    export CONSUL_HTTP_TOKEN="$BOOTSTRAP_TOKEN"
else
    log_warn "ACL system already bootstrapped. Using existing token from environment."
fi

log_info "[9/12] Creating TLS certificates directory..."
mkdir -p /opt/consul/tls
chown consul:consul /opt/consul/tls
chmod 750 /opt/consul/tls

log_info "[10/12] Generating CA and certificates..."
cd /opt/consul/tls
sudo -u consul consul tls ca create
sudo -u consul consul tls cert create -server -dc $DATACENTER
sudo -u consul consul tls cert create -client -dc $DATACENTER

log_info "[11/12] Updating configuration with TLS..."
cat > /opt/consul/consul.hcl <<EOF
datacenter = "$DATACENTER"
data_dir = "/opt/consul"
log_level = "INFO"
server = true
bind_addr = "0.0.0.0"
client_addr = "0.0.0.0"
bootstrap_expect = $BOOTSTRAP_EXPECT
ui_config {
  enabled = true
}

# ACL Configuration
acl = {
  enabled = true
  default_policy = "deny"
  enable_token_persistence = true
}

# Gossip Encryption
encrypt = "$GOSSIP_KEY"

# TLS Configuration
ca_file = "/opt/consul/tls/consul-agent-ca.pem"
cert_file = "/opt/consul/tls/$DATACENTER-server-consul-0.pem"
key_file = "/opt/consul/tls/$DATACENTER-server-consul-0-key.pem"
verify_incoming = true
verify_outgoing = true
verify_server_hostname = true
ports {
  grpc = 8502
  https = 8501
  http = -1
}

# Performance and Connect
performance {
  raft_multiplier = 1
}
connect {
  enabled = true
}
EOF

chown consul:consul /opt/consul/consul.hcl
chmod 640 /opt/consul/consul.hcl

log_info "[12/12] Restarting Consul with TLS..."
systemctl restart consul
sleep 10

# Verification
log_info "Verifying Consul security configuration..."
export CONSUL_HTTP_SSL=true
export CONSUL_HTTP_ADDR="https://127.0.0.1:8501"
export CONSUL_CACERT="/opt/consul/tls/consul-agent-ca.pem"
export CONSUL_CLIENT_CERT="/opt/consul/tls/$DATACENTER-client-consul-0.pem"
export CONSUL_CLIENT_KEY="/opt/consul/tls/$DATACENTER-client-consul-0-key.pem"

# Add environment variables to bashrc
cat >> /root/.bashrc <<EOF
export CONSUL_HTTP_SSL=true
export CONSUL_HTTP_ADDR="https://127.0.0.1:8501"
export CONSUL_CACERT="/opt/consul/tls/consul-agent-ca.pem"
export CONSUL_CLIENT_CERT="/opt/consul/tls/$DATACENTER-client-consul-0.pem"
export CONSUL_CLIENT_KEY="/opt/consul/tls/$DATACENTER-client-consul-0-key.pem"
EOF

if consul members >/dev/null 2>&1; then
    log_info "SUCCESS: Consul is running with ACL and TLS security enabled"
    log_info "Web UI: https://$(hostname -I | awk '{print $1}'):8501"
    log_info "Bootstrap token saved in /root/.bashrc"
    log_info "Certificate files located in /opt/consul/tls/"
else
    log_error "Verification failed. Check Consul logs: journalctl -u consul"
    exit 1
fi

Review the script before running. Execute with: bash install.sh

#consul #acl #security #tls #encryption

Implement Consul ACL security and encryption for production deployments

Prerequisites

What this solves

Prerequisites and Consul installation verification

Verify Consul installation

Stop Consul service for configuration

Create backup of existing configuration

Configure Consul ACL system with bootstrap token

Enable ACL system in Consul configuration

ACL Configuration

Performance and Connect

Set proper permissions for Consul configuration

Start Consul with ACL enabled

Bootstrap the ACL system

Configure environment variable for ACL token

Set up TLS encryption for client-server communication

Create Consul CA and certificates directory

Generate Consul CA certificate

Generate server certificates

Generate client certificate

Configure TLS in Consul configuration

ACL Configuration

TLS Configuration

Performance and Connect

Enable gossip encryption and secure cluster communication

Generate gossip encryption key

Add gossip encryption to configuration

Gossip Encryption

Security Hardening

ACL Configuration

TLS Configuration

Performance and Connect

Create ACL policies and tokens for service authentication

Create a service registration policy

Create the service policy in Consul

Create a read-only policy for monitoring

Create the read-only policy

Create service token

Create monitoring token

Implement certificate management and rotation

Create certificate rotation script

Consul Certificate Rotation Script

Create backup

Generate new certificates

Update configuration with new certificate paths

Make rotation script executable

Configure systemd environment for TLS

Restart Consul with full security configuration

Verify your setup

Check ACL system status

Verify gossip encryption

Test token authentication

Common issues

Next steps

Related tutorials

Setup remote backup storage with S3-compatible encryption and automated retention policies

Configure advanced iptables rules for bridge network security and container isolation

Implement Istio security policies with authorization and authentication for Kubernetes service mesh

Don't want to manage this yourself?