Configure Consul Connect service mesh with Envoy proxy for secure microservices communication

Advanced 45 min Apr 01, 2026 419 views
Ubuntu 24.04 Debian 12 AlmaLinux 9 Rocky Linux 9

Set up Consul Connect service mesh with Envoy sidecar proxies to enable secure, encrypted service-to-service communication with mutual TLS authentication, traffic policies, and observability for your microservices architecture.

Prerequisites

  • Root or sudo access
  • At least 2GB RAM
  • Basic understanding of microservices architecture
  • Familiarity with systemd services

What this solves

Consul Connect provides a service mesh solution that enables secure service-to-service communication through mutual TLS (mTLS) authentication and authorization. When combined with Envoy proxy as a sidecar, it creates a robust infrastructure for microservices that need encrypted communication, traffic management, and observability without modifying application code.

This tutorial shows you how to configure Consul Connect with Envoy proxy to secure microservices communication, implement traffic policies, and enable distributed tracing for production environments.

Step-by-step configuration

Update system packages

Start by updating your package manager to ensure you have the latest security patches and package definitions.

sudo apt update && sudo apt upgrade -y
sudo dnf update -y

Install required dependencies

Install curl, wget, and other tools needed for downloading and configuring Consul and Envoy.

sudo apt install -y curl wget unzip jq net-tools
sudo dnf install -y curl wget unzip jq net-tools

Install Consul

Download and install the latest Consul binary from HashiCorp's releases. This tutorial uses Consul 1.17.0.

cd /tmp
wget https://releases.hashicorp.com/consul/1.17.0/consul_1.17.0_linux_amd64.zip
unzip consul_1.17.0_linux_amd64.zip
sudo mv consul /usr/local/bin/
sudo chmod +x /usr/local/bin/consul
consul version

Install Envoy proxy

Install Envoy proxy which will serve as the sidecar proxy for service mesh communication.

curl -sL 'https://deb.dl.getenvoy.io/public/gpg.8115BA8E629CC074.key' | sudo gpg --dearmor -o /usr/share/keyrings/getenvoy-keyring.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/getenvoy-keyring.gpg] https://deb.dl.getenvoy.io/public/deb/ubuntu $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/getenvoy.list
sudo apt update
sudo apt install -y getenvoy-envoy
curl -sL https://func-e.io/install.sh | sudo bash -s -- -b /usr/local/bin
func-e use 1.28.0
sudo ln -s ~/.func-e/versions/1.28.0/bin/envoy /usr/local/bin/envoy

Create Consul user and directories

Create a dedicated user for Consul and set up the necessary directory structure with proper permissions.

sudo useradd --system --home /etc/consul --shell /bin/false consul
sudo mkdir -p /opt/consul /etc/consul /var/log/consul
sudo chown -R consul:consul /opt/consul /etc/consul /var/log/consul
sudo chmod 755 /opt/consul /etc/consul /var/log/consul

Configure Consul server

Create the main Consul configuration file with Connect enabled and security features configured.

datacenter = "dc1"
data_dir = "/opt/consul"
log_level = "INFO"
log_file = "/var/log/consul/consul.log"
server = true
bootstrap_expect = 1
bind_addr = "0.0.0.0"
client_addr = "0.0.0.0"
retry_join = ["127.0.0.1"]
ui_config {
  enabled = true
}
connect {
  enabled = true
}
ports {
  grpc = 8502
}
acl = {
  enabled = true
  default_policy = "allow"
  enable_token_persistence = true
}
encrypt = "$(consul keygen)"
verify_incoming = false
verify_outgoing = false
verify_server_hostname = false
ca_file = "/etc/consul/consul-agent-ca.pem"
cert_file = "/etc/consul/consul-agent.pem"
key_file = "/etc/consul/consul-agent-key.pem"

Generate Consul encryption key

Generate a base64 encryption key for Consul gossip protocol and update the configuration.

ENCRYPT_KEY=$(consul keygen)
echo "Generated encryption key: $ENCRYPT_KEY"
sudo sed -i "s/\$(consul keygen)/$ENCRYPT_KEY/g" /etc/consul/consul.hcl

Generate TLS certificates

Create TLS certificates for Consul using the built-in CA functionality.

cd /etc/consul
sudo consul tls ca create
sudo consul tls cert create -server
sudo chown consul:consul /etc/consul/*.pem
sudo chmod 644 /etc/consul/*.pem

Create Consul systemd service

Create a systemd service file to manage the Consul process with proper security restrictions.

[Unit]
Description=Consul
Documentation=https://www.consul.io/
Requires=network-online.target
After=network-online.target
ConditionFileNotEmpty=/etc/consul/consul.hcl

[Service]
Type=notify
User=consul
Group=consul
ExecStart=/usr/local/bin/consul agent -config-dir=/etc/consul
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
Restart=on-failure
LimitNOFILE=65536
TimeoutStopSec=10
RestartSec=5
StartLimitBurst=3
StartLimitIntervalSec=10

[Install]
WantedBy=multi-user.target

Start and enable Consul

Enable and start the Consul service, then verify it's running correctly.

sudo systemctl daemon-reload
sudo systemctl enable consul
sudo systemctl start consul
sudo systemctl status consul

Configure sample web service

Create a simple web service configuration to demonstrate Connect functionality.

{
  "service": {
    "name": "web",
    "tags": ["frontend"],
    "port": 8080,
    "connect": {
      "sidecar_service": {
        "proxy": {
          "upstreams": [
            {
              "destination_name": "api",
              "local_bind_port": 8081
            }
          ]
        }
      }
    },
    "check": {
      "http": "http://localhost:8080/health",
      "interval": "10s"
    }
  }
}

Configure sample API service

Create an API service configuration that the web service will communicate with through Connect.

{
  "service": {
    "name": "api",
    "tags": ["backend"],
    "port": 8090,
    "connect": {
      "sidecar_service": {}
    },
    "check": {
      "http": "http://localhost:8090/health",
      "interval": "10s"
    }
  }
}

Register services with Consul

Register both services with Consul to make them available for Connect communication.

consul services register /etc/consul/web.json
consul services register /etc/consul/api.json
consul catalog services

Start Envoy sidecar for web service

Start the Envoy sidecar proxy for the web service using Consul Connect.

consul connect envoy -sidecar-for web-sidecar-proxy -admin-bind localhost:19001 > /var/log/consul/web-envoy.log 2>&1 &
echo $! | sudo tee /var/run/web-envoy.pid

Start Envoy sidecar for API service

Start the Envoy sidecar proxy for the API service to handle incoming Connect traffic.

consul connect envoy -sidecar-for api-sidecar-proxy -admin-bind localhost:19002 > /var/log/consul/api-envoy.log 2>&1 &
echo $! | sudo tee /var/run/api-envoy.pid

Configure Connect intentions

Set up service intentions to control which services can communicate with each other.

consul intention create web api
consul intention list

Create traffic policy configuration

Configure advanced traffic management policies for load balancing and circuit breaking.

Kind = "service-resolver"
Name = "api"
LoadBalancer = {
  Policy = "round_robin"
  HashPolicies = [
    {
      Field = "header"
      FieldValue = "x-user-id"
    }
  ]
}
Failover = {
  "*" = {
    Datacenters = ["dc2"]
  }
}

Apply traffic policies

Apply the traffic management policies to Consul's configuration.

consul config write /etc/consul/api-resolver.hcl
consul config list -kind service-resolver

Configure observability with Jaeger integration

Enable distributed tracing by configuring Envoy to send traces to Jaeger. This requires a running Jaeger instance as described in our Jaeger distributed tracing tutorial.

Kind = "proxy-defaults"
Name = "global"
Config = {
  "envoy_tracing_json" = <

Apply observability configuration

Apply the tracing configuration to enable distributed observability across your service mesh.

consul config write /etc/consul/tracing.hcl
consul config list -kind proxy-defaults

Configure firewall rules

Open the necessary ports for Consul Connect and Envoy communication.

sudo ufw allow 8300/tcp comment 'Consul server RPC'
sudo ufw allow 8301/tcp comment 'Consul Serf LAN'
sudo ufw allow 8301/udp comment 'Consul Serf LAN'
sudo ufw allow 8500/tcp comment 'Consul HTTP API'
sudo ufw allow 8502/tcp comment 'Consul gRPC API'
sudo ufw allow 21000:21255/tcp comment 'Consul Connect sidecar proxies'
sudo firewall-cmd --permanent --add-port=8300/tcp --add-port=8301/tcp --add-port=8301/udp
sudo firewall-cmd --permanent --add-port=8500/tcp --add-port=8502/tcp
sudo firewall-cmd --permanent --add-port=21000-21255/tcp
sudo firewall-cmd --reload

Verify your setup

Test that Consul Connect and Envoy are working correctly with these verification commands.

# Check Consul cluster status
consul members
consul catalog services

Verify Connect is enabled

consul connect ca get-config

Check Envoy proxy status

curl -s localhost:19001/stats | grep connect curl -s localhost:19002/stats | grep connect

Test service-to-service communication

curl -s localhost:8081/api/health

Check intentions

consul intention check web api
Note: The service mesh communication will only work once you have actual web and API applications running on ports 8080 and 8090. The proxies route traffic between these services securely.

Common issues

SymptomCauseFix
Envoy won't startMissing gRPC port configurationEnsure ports { grpc = 8502 } in consul.hcl
Services can't communicateMissing Connect intentionsRun consul intention create source destination
Certificate errorsTLS verification enabled without proper certsSet verify_incoming = false for development
Permission denied on logsIncorrect directory ownershipRun sudo chown consul:consul /var/log/consul
Connect proxy registration failsService not registered firstRegister service before starting sidecar proxy
Metrics not appearingEnvoy admin interface not accessibleCheck admin bind addresses don't conflict
Security note: This configuration uses default_policy = "allow" for ACLs and disables TLS verification for simplicity. In production, enable strict ACL policies and proper TLS certificate verification.

Next steps

Automated install script

Run this to automate the entire setup

Need help?

Don't want to manage this yourself?

We handle managed devops services for businesses that depend on uptime. From initial setup to ongoing operations.