Configure network load balancing with keepalived and VRRP for high availability failover

Update system packages

Start by updating your package manager to ensure you get the latest versions of keepalived and dependencies.

sudo apt update && sudo apt upgrade -y

sudo dnf update -y

Install keepalived

Install keepalived package which includes VRRP support and health checking capabilities.

sudo apt install -y keepalived ipvsadm

sudo dnf install -y keepalived ipvsadm

Enable IP forwarding

Enable IP forwarding in the kernel to allow the system to route traffic between interfaces and handle virtual IP addresses.

echo 'net.ipv4.ip_forward = 1' | sudo tee -a /etc/sysctl.conf
echo 'net.ipv4.ip_nonlocal_bind = 1' | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

Configure primary server keepalived

Create the keepalived configuration for the primary server. This server will have higher priority and own the virtual IP by default.

global_defs {
    router_id LB_PRIMARY
    enable_script_security
    script_user root
}

vrrp_script chk_nginx {
    script "/bin/curl -f http://localhost:80 || exit 1"
    interval 2
    weight -2
    fall 3
    rise 2
}

vrrp_instance VI_1 {
    state MASTER
    interface ens3
    virtual_router_id 51
    priority 110
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass changeme123
    }
    virtual_ipaddress {
        203.0.113.10/24
    }
    track_script {
        chk_nginx
    }
    notify_master "/etc/keepalived/master.sh"
    notify_backup "/etc/keepalived/backup.sh"
    notify_fault "/etc/keepalived/fault.sh"
}

Configure backup server keepalived

Create the keepalived configuration for the backup server with lower priority. Replace the interface name and adjust IP addresses for your network.

global_defs {
    router_id LB_BACKUP
    enable_script_security
    script_user root
}

vrrp_script chk_nginx {
    script "/bin/curl -f http://localhost:80 || exit 1"
    interval 2
    weight -2
    fall 3
    rise 2
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens3
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass changeme123
    }
    virtual_ipaddress {
        203.0.113.10/24
    }
    track_script {
        chk_nginx
    }
    notify_master "/etc/keepalived/master.sh"
    notify_backup "/etc/keepalived/backup.sh"
    notify_fault "/etc/keepalived/fault.sh"
}

Create notification scripts

Create scripts that run when VRRP state changes occur. These scripts can update DNS, send alerts, or perform custom actions during failover.

#!/bin/bash
echo "$(date): Becoming MASTER" >> /var/log/keepalived-state.log
# Add custom actions when becoming master
# Example: systemctl start nginx
</code><h2>`Example: curl -X POST https://monitoring.example.com/webhook`</h2></pre><pre class="terminal"><code>sudo chmod 755 /etc/keepalived/master.sh

Create backup and fault scripts

Create additional notification scripts for backup and fault states to provide complete state tracking.

#!/bin/bash
echo "$(date): Becoming BACKUP" >> /var/log/keepalived-state.log
# Add custom actions when becoming backup
</code><h2>`Example: systemctl stop nginx`</h2></pre><pre class="terminal">`#!/bin/bash
echo "$(date): Entering FAULT state" >> /var/log/keepalived-state.log
# Add custom actions when entering fault state
`<h2>`Example: send alert email`</h2></pre><pre class="terminal"><code>sudo chmod 755 /etc/keepalived/backup.sh
sudo chmod 755 /etc/keepalived/fault.sh

Configure firewall rules

Open the necessary firewall ports for VRRP communication. VRRP uses IP protocol 112 for heartbeat messages between nodes.

sudo ufw allow in on ens3 to 224.0.0.18
sudo ufw allow in on ens3 proto vrrp
sudo ufw reload

sudo firewall-cmd --permanent --add-protocol=vrrp
sudo firewall-cmd --permanent --add-rich-rule="rule family='ipv4' destination address='224.0.0.18' accept"
sudo firewall-cmd --reload

Install and configure NGINX for testing

Install a web server on both nodes to test the health checks and load balancing functionality.

sudo apt install -y nginx
echo "<h1>$(hostname) - Primary Server</h1>" | sudo tee /var/www/html/index.html
sudo systemctl enable --now nginx

sudo dnf install -y nginx
echo "<h1>$(hostname) - Primary Server</h1>" | sudo tee /usr/share/nginx/html/index.html
sudo systemctl enable --now nginx

Start keepalived services

Enable and start keepalived on both servers. The primary server should claim the virtual IP address within a few seconds.

sudo systemctl enable --now keepalived
sudo systemctl status keepalived

Configure advanced health checks

Create a more comprehensive health check script that monitors multiple services and system resources.

#!/bin/bash

# Check HTTP response
if ! curl -f -s http://localhost:80 > /dev/null; then
    echo "HTTP check failed"
    exit 1
fi

# Check system load
load=$(uptime | awk -F'load average:' '{print $2}' | awk '{print $1}' | tr -d ',')
if (( $(echo "$load > 10.0" | bc -l) )); then
    echo "System load too high: $load"
    exit 1
fi

# Check memory usage
mem_usage=$(free | grep Mem | awk '{printf("%.1f", $3/$2 * 100.0)}')
if (( $(echo "$mem_usage > 90.0" | bc -l) )); then
    echo "Memory usage too high: $mem_usage%"
    exit 1
fi

echo "All health checks passed"
exit 0

sudo chmod 755 /etc/keepalived/check_services.sh

Update configuration with advanced checks

Modify the keepalived configuration to use the comprehensive health check script instead of the simple curl command.

vrrp_script chk_services {
    script "/etc/keepalived/check_services.sh"
    interval 5
    weight -10
    fall 2
    rise 2
    timeout 3
}

sudo systemctl restart keepalived

Add secondary VRRP instance

Configure a second VRRP instance with different virtual IP and router ID for additional services or load distribution.

vrrp_instance VI_2 {
    state MASTER
    interface ens3
    virtual_router_id 52
    priority 110
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass changeme456
    }
    virtual_ipaddress {
        203.0.113.11/24
    }
    track_script {
        chk_services
    }
}

Add virtual server configuration

Configure keepalived to act as a load balancer with real servers for distributing traffic beyond simple IP failover.

virtual_server 203.0.113.10 80 {
    delay_loop 6
    lb_algo rr
    lb_kind NAT
    persistence_timeout 50
    protocol TCP

    real_server 192.168.1.10 80 {
        weight 1
        HTTP_GET {
            url {
                path /health
                status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.1.11 80 {
        weight 1
        HTTP_GET {
            url {
                path /health
                status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

Simulate primary server failure

Test the failover mechanism by stopping services or disconnecting the primary server to verify automatic VIP migration.

# Stop keepalived on primary to trigger failover
sudo systemctl stop keepalived

# Or stop the monitored service
sudo systemctl stop nginx

# Watch failover in logs on backup server
sudo journalctl -u keepalived -f

Monitor failover timing

Use these commands to monitor how quickly failover occurs and verify the backup server assumes the virtual IP.

# Monitor VIP changes
watch -n 1 'ip addr show | grep 203.0.113.10'

# Test connectivity during failover
while true; do curl -m 2 http://203.0.113.10 && sleep 1; done

# Check VRRP advertisements
sudo tcpdump -i ens3 vrrp