Set up Consul Connect service mesh with Envoy sidecar proxies to enable secure, encrypted service-to-service communication with mutual TLS authentication, traffic policies, and observability for your microservices architecture.
Prerequisites
- Root or sudo access
- At least 2GB RAM
- Basic understanding of microservices architecture
- Familiarity with systemd services
What this solves
Consul Connect provides a service mesh solution that enables secure service-to-service communication through mutual TLS (mTLS) authentication and authorization. When combined with Envoy proxy as a sidecar, it creates a robust infrastructure for microservices that need encrypted communication, traffic management, and observability without modifying application code.
This tutorial shows you how to configure Consul Connect with Envoy proxy to secure microservices communication, implement traffic policies, and enable distributed tracing for production environments.
Step-by-step configuration
Update system packages
Start by updating your package manager to ensure you have the latest security patches and package definitions.
sudo apt update && sudo apt upgrade -y
Install required dependencies
Install curl, wget, and other tools needed for downloading and configuring Consul and Envoy.
sudo apt install -y curl wget unzip jq net-tools
Install Consul
Download and install the latest Consul binary from HashiCorp's releases. This tutorial uses Consul 1.17.0.
cd /tmp
wget https://releases.hashicorp.com/consul/1.17.0/consul_1.17.0_linux_amd64.zip
unzip consul_1.17.0_linux_amd64.zip
sudo mv consul /usr/local/bin/
sudo chmod +x /usr/local/bin/consul
consul version
Install Envoy proxy
Install Envoy proxy which will serve as the sidecar proxy for service mesh communication.
curl -sL 'https://deb.dl.getenvoy.io/public/gpg.8115BA8E629CC074.key' | sudo gpg --dearmor -o /usr/share/keyrings/getenvoy-keyring.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/getenvoy-keyring.gpg] https://deb.dl.getenvoy.io/public/deb/ubuntu $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/getenvoy.list
sudo apt update
sudo apt install -y getenvoy-envoy
Create Consul user and directories
Create a dedicated user for Consul and set up the necessary directory structure with proper permissions.
sudo useradd --system --home /etc/consul --shell /bin/false consul
sudo mkdir -p /opt/consul /etc/consul /var/log/consul
sudo chown -R consul:consul /opt/consul /etc/consul /var/log/consul
sudo chmod 755 /opt/consul /etc/consul /var/log/consul
Configure Consul server
Create the main Consul configuration file with Connect enabled and security features configured.
datacenter = "dc1"
data_dir = "/opt/consul"
log_level = "INFO"
log_file = "/var/log/consul/consul.log"
server = true
bootstrap_expect = 1
bind_addr = "0.0.0.0"
client_addr = "0.0.0.0"
retry_join = ["127.0.0.1"]
ui_config {
enabled = true
}
connect {
enabled = true
}
ports {
grpc = 8502
}
acl = {
enabled = true
default_policy = "allow"
enable_token_persistence = true
}
encrypt = "$(consul keygen)"
verify_incoming = false
verify_outgoing = false
verify_server_hostname = false
ca_file = "/etc/consul/consul-agent-ca.pem"
cert_file = "/etc/consul/consul-agent.pem"
key_file = "/etc/consul/consul-agent-key.pem"
Generate Consul encryption key
Generate a base64 encryption key for Consul gossip protocol and update the configuration.
ENCRYPT_KEY=$(consul keygen)
echo "Generated encryption key: $ENCRYPT_KEY"
sudo sed -i "s/\$(consul keygen)/$ENCRYPT_KEY/g" /etc/consul/consul.hcl
Generate TLS certificates
Create TLS certificates for Consul using the built-in CA functionality.
cd /etc/consul
sudo consul tls ca create
sudo consul tls cert create -server
sudo chown consul:consul /etc/consul/*.pem
sudo chmod 644 /etc/consul/*.pem
Create Consul systemd service
Create a systemd service file to manage the Consul process with proper security restrictions.
[Unit]
Description=Consul
Documentation=https://www.consul.io/
Requires=network-online.target
After=network-online.target
ConditionFileNotEmpty=/etc/consul/consul.hcl
[Service]
Type=notify
User=consul
Group=consul
ExecStart=/usr/local/bin/consul agent -config-dir=/etc/consul
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
LimitNOFILE=65536
TimeoutStopSec=10
RestartSec=5
StartLimitBurst=3
StartLimitIntervalSec=10
[Install]
WantedBy=multi-user.target
Start and enable Consul
Enable and start the Consul service, then verify it's running correctly.
sudo systemctl daemon-reload
sudo systemctl enable consul
sudo systemctl start consul
sudo systemctl status consul
Configure sample web service
Create a simple web service configuration to demonstrate Connect functionality.
{
"service": {
"name": "web",
"tags": ["frontend"],
"port": 8080,
"connect": {
"sidecar_service": {
"proxy": {
"upstreams": [
{
"destination_name": "api",
"local_bind_port": 8081
}
]
}
}
},
"check": {
"http": "http://localhost:8080/health",
"interval": "10s"
}
}
}
Configure sample API service
Create an API service configuration that the web service will communicate with through Connect.
{
"service": {
"name": "api",
"tags": ["backend"],
"port": 8090,
"connect": {
"sidecar_service": {}
},
"check": {
"http": "http://localhost:8090/health",
"interval": "10s"
}
}
}
Register services with Consul
Register both services with Consul to make them available for Connect communication.
consul services register /etc/consul/web.json
consul services register /etc/consul/api.json
consul catalog services
Start Envoy sidecar for web service
Start the Envoy sidecar proxy for the web service using Consul Connect.
consul connect envoy -sidecar-for web-sidecar-proxy -admin-bind localhost:19001 > /var/log/consul/web-envoy.log 2>&1 &
echo $! | sudo tee /var/run/web-envoy.pid
Start Envoy sidecar for API service
Start the Envoy sidecar proxy for the API service to handle incoming Connect traffic.
consul connect envoy -sidecar-for api-sidecar-proxy -admin-bind localhost:19002 > /var/log/consul/api-envoy.log 2>&1 &
echo $! | sudo tee /var/run/api-envoy.pid
Configure Connect intentions
Set up service intentions to control which services can communicate with each other.
consul intention create web api
consul intention list
Create traffic policy configuration
Configure advanced traffic management policies for load balancing and circuit breaking.
Kind = "service-resolver"
Name = "api"
LoadBalancer = {
Policy = "round_robin"
HashPolicies = [
{
Field = "header"
FieldValue = "x-user-id"
}
]
}
Failover = {
"*" = {
Datacenters = ["dc2"]
}
}
Apply traffic policies
Apply the traffic management policies to Consul's configuration.
consul config write /etc/consul/api-resolver.hcl
consul config list -kind service-resolver
Configure observability with Jaeger integration
Enable distributed tracing by configuring Envoy to send traces to Jaeger. This requires a running Jaeger instance as described in our Jaeger distributed tracing tutorial.
Kind = "proxy-defaults"
Name = "global"
Config = {
"envoy_tracing_json" = <
Apply observability configuration
Apply the tracing configuration to enable distributed observability across your service mesh.
consul config write /etc/consul/tracing.hcl
consul config list -kind proxy-defaults
Configure firewall rules
Open the necessary ports for Consul Connect and Envoy communication.
sudo ufw allow 8300/tcp comment 'Consul server RPC'
sudo ufw allow 8301/tcp comment 'Consul Serf LAN'
sudo ufw allow 8301/udp comment 'Consul Serf LAN'
sudo ufw allow 8500/tcp comment 'Consul HTTP API'
sudo ufw allow 8502/tcp comment 'Consul gRPC API'
sudo ufw allow 21000:21255/tcp comment 'Consul Connect sidecar proxies'
Verify your setup
Test that Consul Connect and Envoy are working correctly with these verification commands.
# Check Consul cluster status
consul members
consul catalog services
Verify Connect is enabled
consul connect ca get-config
Check Envoy proxy status
curl -s localhost:19001/stats | grep connect
curl -s localhost:19002/stats | grep connect
Test service-to-service communication
curl -s localhost:8081/api/health
Check intentions
consul intention check web api
Common issues
| Symptom | Cause | Fix |
|---|---|---|
| Envoy won't start | Missing gRPC port configuration | Ensure ports { grpc = 8502 } in consul.hcl |
| Services can't communicate | Missing Connect intentions | Run consul intention create source destination |
| Certificate errors | TLS verification enabled without proper certs | Set verify_incoming = false for development |
| Permission denied on logs | Incorrect directory ownership | Run sudo chown consul:consul /var/log/consul |
| Connect proxy registration fails | Service not registered first | Register service before starting sidecar proxy |
| Metrics not appearing | Envoy admin interface not accessible | Check admin bind addresses don't conflict |
default_policy = "allow" for ACLs and disables TLS verification for simplicity. In production, enable strict ACL policies and proper TLS certificate verification.Next steps
- Install and configure Consul for service discovery for the foundational setup
- Set up Prometheus and Grafana monitoring to monitor your service mesh metrics
- Configure Consul Connect with Kubernetes integration for container orchestration
- Implement Consul Connect mTLS with Vault PKI backend for enhanced certificate management
- Configure Envoy rate limiting and circuit breakers for advanced traffic management
Automated install script
Run this to automate the entire setup
#!/usr/bin/env bash
set -euo pipefail
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
# Script configuration
CONSUL_VERSION="1.17.0"
ENVOY_VERSION="1.28.0"
# Usage function
usage() {
echo "Usage: $0 [OPTIONS]"
echo "Options:"
echo " -h, --help Show this help message"
echo " -v, --version Consul version (default: $CONSUL_VERSION)"
exit 1
}
# Logging functions
log_info() { echo -e "${GREEN}[INFO]${NC} $1"; }
log_warn() { echo -e "${YELLOW}[WARN]${NC} $1"; }
log_error() { echo -e "${RED}[ERROR]${NC} $1"; }
# Parse arguments
while [[ $# -gt 0 ]]; do
case $1 in
-h|--help) usage ;;
-v|--version) CONSUL_VERSION="$2"; shift 2 ;;
*) log_error "Unknown option: $1"; usage ;;
esac
done
# Cleanup function for rollback
cleanup() {
log_warn "Installation failed. Cleaning up..."
systemctl stop consul 2>/dev/null || true
systemctl disable consul 2>/dev/null || true
rm -f /etc/systemd/system/consul.service
rm -rf /opt/consul /etc/consul /var/log/consul
userdel -r consul 2>/dev/null || true
rm -f /usr/local/bin/consul /usr/local/bin/envoy
systemctl daemon-reload
}
trap cleanup ERR
# Check if running as root
if [[ $EUID -ne 0 ]]; then
log_error "This script must be run as root or with sudo"
exit 1
fi
# Detect distribution
if [ -f /etc/os-release ]; then
. /etc/os-release
case "$ID" in
ubuntu|debian)
PKG_MGR="apt"
PKG_UPDATE="apt update && apt upgrade -y"
PKG_INSTALL="apt install -y"
;;
almalinux|rocky|centos|rhel|ol|fedora)
PKG_MGR="dnf"
PKG_UPDATE="dnf update -y"
PKG_INSTALL="dnf install -y"
;;
amzn)
PKG_MGR="yum"
PKG_UPDATE="yum update -y"
PKG_INSTALL="yum install -y"
;;
*)
log_error "Unsupported distribution: $ID"
exit 1
;;
esac
else
log_error "Cannot detect distribution"
exit 1
fi
log_info "Detected distribution: $PRETTY_NAME"
# Step 1: Update system packages
echo "[1/10] Updating system packages..."
$PKG_UPDATE
# Step 2: Install dependencies
echo "[2/10] Installing dependencies..."
$PKG_INSTALL curl wget unzip jq net-tools
# Step 3: Install Consul
echo "[3/10] Installing Consul $CONSUL_VERSION..."
cd /tmp
wget -q "https://releases.hashicorp.com/consul/${CONSUL_VERSION}/consul_${CONSUL_VERSION}_linux_amd64.zip"
unzip -q "consul_${CONSUL_VERSION}_linux_amd64.zip"
mv consul /usr/local/bin/
chmod 755 /usr/local/bin/consul
rm -f "consul_${CONSUL_VERSION}_linux_amd64.zip"
# Verify Consul installation
if ! consul version >/dev/null 2>&1; then
log_error "Consul installation failed"
exit 1
fi
# Step 4: Install Envoy
echo "[4/10] Installing Envoy proxy..."
if [[ "$PKG_MGR" == "apt" ]]; then
curl -sL 'https://deb.dl.getenvoy.io/public/gpg.8115BA8E629CC074.key' | gpg --dearmor -o /usr/share/keyrings/getenvoy-keyring.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/getenvoy-keyring.gpg] https://deb.dl.getenvoy.io/public/deb/ubuntu $(lsb_release -cs) main" > /etc/apt/sources.list.d/getenvoy.list
apt update
apt install -y getenvoy-envoy
else
curl -sL https://func-e.io/install.sh | bash -s -- -b /usr/local/bin
/usr/local/bin/func-e use $ENVOY_VERSION
cp ~/.func-e/versions/$ENVOY_VERSION/bin/envoy /usr/local/bin/
chmod 755 /usr/local/bin/envoy
fi
# Step 5: Create Consul user and directories
echo "[5/10] Creating Consul user and directories..."
if ! id consul >/dev/null 2>&1; then
useradd --system --home /etc/consul --shell /bin/false consul
fi
mkdir -p /opt/consul /etc/consul /var/log/consul
chown -R consul:consul /opt/consul /etc/consul /var/log/consul
chmod 755 /opt/consul /etc/consul /var/log/consul
# Step 6: Generate encryption key
echo "[6/10] Generating Consul encryption key..."
ENCRYPT_KEY=$(consul keygen)
log_info "Generated encryption key: $ENCRYPT_KEY"
# Step 7: Create Consul configuration
echo "[7/10] Creating Consul configuration..."
cat > /etc/consul/consul.hcl << EOF
datacenter = "dc1"
data_dir = "/opt/consul"
log_level = "INFO"
log_file = "/var/log/consul/consul.log"
server = true
bootstrap_expect = 1
bind_addr = "0.0.0.0"
client_addr = "0.0.0.0"
retry_join = ["127.0.0.1"]
ui_config {
enabled = true
}
connect {
enabled = true
}
ports {
grpc = 8502
}
acl = {
enabled = true
default_policy = "allow"
enable_token_persistence = true
}
encrypt = "$ENCRYPT_KEY"
verify_incoming = false
verify_outgoing = false
verify_server_hostname = false
ca_file = "/etc/consul/consul-agent-ca.pem"
cert_file = "/etc/consul/consul-agent.pem"
key_file = "/etc/consul/consul-agent-key.pem"
EOF
chown consul:consul /etc/consul/consul.hcl
chmod 640 /etc/consul/consul.hcl
# Step 8: Generate TLS certificates
echo "[8/10] Generating TLS certificates..."
cd /etc/consul
consul tls ca create
consul tls cert create -server
chown consul:consul /etc/consul/*.pem
chmod 644 /etc/consul/*.pem
# Step 9: Create systemd service
echo "[9/10] Creating systemd service..."
cat > /etc/systemd/system/consul.service << 'EOF'
[Unit]
Description=Consul
Documentation=https://www.consul.io/
Requires=network-online.target
After=network-online.target
ConditionFileNotEmpty=/etc/consul/consul.hcl
[Service]
Type=notify
User=consul
Group=consul
ExecStart=/usr/local/bin/consul agent -config-dir=/etc/consul/
ExecReload=/bin/kill -HUP $MAINPID
ExecStopPost=/bin/sleep 5
KillMode=process
Restart=on-failure
LimitNOFILE=65536
TimeoutStopSec=10s
SendSIGKILL=no
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable consul
systemctl start consul
# Step 10: Verification
echo "[10/10] Verifying installation..."
sleep 5
if ! systemctl is-active --quiet consul; then
log_error "Consul service is not running"
exit 1
fi
if ! consul members >/dev/null 2>&1; then
log_error "Consul cluster is not healthy"
exit 1
fi
log_info "Installation completed successfully!"
log_info "Consul UI available at: http://$(hostname -I | awk '{print $1}'):8500"
log_info "Consul is running with Connect enabled"
log_info "Service status: $(systemctl is-active consul)"
Review the script before running. Execute with: bash install.sh