Implement Lighttpd 1.4 load balancing with multiple backend servers for high availability

Intermediate 45 min Apr 27, 2026 559 views
Ubuntu 24.04 Debian 12 AlmaLinux 9 Rocky Linux 9

Configure Lighttpd 1.4 with mod_proxy for load balancing across multiple backend web servers. Set up health checks, SSL termination, and failover mechanisms for production environments.

Prerequisites

  • Root or sudo access
  • Multiple backend servers
  • Basic understanding of HTTP load balancing
  • SSL certificate (optional)

What this solves

Lighttpd 1.4 can distribute incoming HTTP requests across multiple backend servers using mod_proxy and mod_magnet. This setup provides high availability by automatically routing traffic away from failed backends and enables horizontal scaling by adding more servers as needed.

Step-by-step configuration

Install Lighttpd 1.4 with required modules

Start by installing Lighttpd and the proxy modules needed for load balancing functionality.

sudo apt update
sudo apt install -y lighttpd lighttpd-mod-magnet
sudo dnf update -y
sudo dnf install -y lighttpd lighttpd-mod-magnet

Enable proxy and required modules

Enable the modules needed for load balancing, health checks, and request routing.

sudo lighttpd-enable-mod proxy
sudo lighttpd-enable-mod magnet
sudo lighttpd-enable-mod rewrite

Create backend server configuration

Configure the backend servers that will handle the actual requests. Create a dedicated configuration file for the load balancer setup.

server.modules += (
    "mod_proxy",
    "mod_magnet",
    "mod_rewrite"
)

# Backend server definitions
proxy.server = (
    "" => (
        "backend1" => (
            "host" => "192.168.1.10",
            "port" => 8080,
            "check-local" => "disable"
        ),
        "backend2" => (
            "host" => "192.168.1.11",
            "port" => 8080,
            "check-local" => "disable"
        ),
        "backend3" => (
            "host" => "192.168.1.12",
            "port" => 8080,
            "check-local" => "disable"
        )
    )
)

# Load balancing method
proxy.balance = "fair"

# Connection settings
proxy.forwarded = (
    "for" => 1,
    "host" => 1,
    "proto" => 1,
    "remote_user" => 1
)

# Timeout settings
proxy.server.timeout = 30

Configure health checks with Lua script

Create a Lua script that implements health checking for backend servers using mod_magnet.

-- Backend health check configuration
local backends = {
    {host = "192.168.1.10", port = 8080, name = "backend1"},
    {host = "192.168.1.11", port = 8080, name = "backend2"},
    {host = "192.168.1.12", port = 8080, name = "backend3"}
}

-- Health check endpoint
local health_path = "/health"

-- Check if backend is healthy
function check_backend_health(backend)
    local handle = io.popen("curl -f -s -m 5 http://" .. backend.host .. ":" .. backend.port .. health_path .. " >/dev/null 2>&1; echo $?")
    local result = handle:read("*a")
    handle:close()
    return tonumber(result) == 0
end

-- Main health check logic
function health_check_handler()
    local healthy_backends = {}
    
    for i, backend in ipairs(backends) do
        if check_backend_health(backend) then
            table.insert(healthy_backends, backend)
        else
            lighty.r.log(3, "Backend " .. backend.name .. " is unhealthy")
        end
    end
    
    if #healthy_backends == 0 then
        lighty.r.status = 503
        lighty.r.header["Content-Type"] = "text/plain"
        lighty.r.content = "All backend servers are unavailable"
        return lighty.RESPONSE_FINISHED
    end
    
    return lighty.RESPONSE_GO_ON
end

-- Register the health check handler
lighty.request.env["health_check"] = health_check_handler

Update main configuration for load balancing

Modify the main Lighttpd configuration to include health checking and proper request handling.

server.modules = (
    "mod_indexfile",
    "mod_access",
    "mod_alias",
    "mod_redirect",
    "mod_rewrite",
    "mod_proxy",
    "mod_magnet"
)

server.document-root        = "/var/www/html"
server.upload-dirs          = ( "/var/cache/lighttpd/uploads" )
server.errorlog             = "/var/log/lighttpd/error.log"
server.pid-file             = "/run/lighttpd.pid"
server.username             = "www-data"
server.groupname            = "www-data"
server.port                 = 80

# Enable health checking
magnet.attract-physical-path-to = ( "/etc/lighttpd/healthcheck.lua" )

# Logging for debugging
accesslog.filename = "/var/log/lighttpd/access.log"
accesslog.format = "%h %V %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\" %Tf"

# Performance tuning
server.max-connections = 1024
server.max-fds = 2048
server.max-worker = 8

index-file.names = ( "index.php", "index.html", "index.lighttpd.html" )
url.access-deny = ( "~", ".inc" )
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )

compress.cache-dir = "/var/cache/lighttpd/compress/"
compress.filetype = ( "application/javascript", "text/css", "text/html", "text/plain" )

# Include additional configurations
include_shell "/usr/share/lighttpd/create-mime.conf.pl"
include_shell "/usr/share/lighttpd/include-conf-enabled.pl"

Set up SSL termination

Configure SSL termination at the load balancer level to handle HTTPS traffic securely.

sudo apt install -y certbot
sudo certbot certonly --standalone -d example.com
sudo dnf install -y certbot
sudo certbot certonly --standalone -d example.com

Configure SSL in Lighttpd

Enable SSL support and configure certificate paths for secure connections.

sudo lighttpd-enable-mod ssl
server.modules += ( "mod_openssl" )

$HTTP["scheme"] == "https" {
    ssl.engine = "enable"
    ssl.privkey = "/etc/letsencrypt/live/example.com/privkey.pem"
    ssl.ca-file = "/etc/letsencrypt/live/example.com/fullchain.pem"
    ssl.cipher-list = "ECDHE+AESGCM:ECDHE+CHACHA20:DHE+AESGCM:DHE+CHACHA20:!aNULL:!SHA1:!AESCCM"
    ssl.honor-cipher-order = "enable"
    ssl.disable-client-renegotiation = "enable"
}

# Redirect HTTP to HTTPS
$HTTP["scheme"] == "http" {
    url.redirect = ("^/(.*)" => "https://example.com/$1")
}

Configure advanced load balancing options

Set up session persistence, request routing, and failover behavior for production use.

# Session persistence based on client IP
proxy.balance = "hash"
proxy.hash-key = "remote_ip"

# Request routing rules
$HTTP["url"] =~ "^/api/" {
    proxy.server = (
        "" => (
            "api-backend1" => (
                "host" => "192.168.1.20",
                "port" => 3000,
                "check-local" => "disable"
            ),
            "api-backend2" => (
                "host" => "192.168.1.21",
                "port" => 3000,
                "check-local" => "disable"
            )
        )
    )
}

# Static content routing
$HTTP["url"] =~ "\.(css|js|png|jpg|gif|ico)$" {
    proxy.server = (
        "" => (
            "static-backend1" => (
                "host" => "192.168.1.30",
                "port" => 80,
                "check-local" => "disable"
            )
        )
    )
}

# Custom headers for backend identification
setenv.add-response-header = (
    "X-Load-Balancer" => "lighttpd-1.4",
    "X-Frame-Options" => "DENY",
    "X-Content-Type-Options" => "nosniff"
)

Create log rotation configuration

Set up log rotation to prevent disk space issues from accumulating logs.

/var/log/lighttpd/*.log {
    daily
    missingok
    rotate 52
    compress
    delaycompress
    notifempty
    sharedscripts
    postrotate
        systemctl reload lighttpd
    endscript
}

Set proper file permissions

Ensure correct ownership and permissions for configuration files and logs.

sudo chown -R www-data:www-data /var/log/lighttpd/
sudo chmod 755 /var/log/lighttpd/
sudo chmod 644 /etc/lighttpd/conf-enabled/*.conf
sudo chmod 644 /etc/lighttpd/healthcheck.lua

Test configuration and start services

Validate the configuration syntax and start Lighttpd with the new load balancing setup.

sudo lighttpd -t -f /etc/lighttpd/lighttpd.conf
sudo systemctl restart lighttpd
sudo systemctl enable lighttpd

Configure firewall rules

Open the necessary ports for HTTP, HTTPS, and backend communication.

sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw allow from 192.168.1.0/24 to any port 8080
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --permanent --add-rich-rule="rule family='ipv4' source address='192.168.1.0/24' port protocol='tcp' port='8080' accept"
sudo firewall-cmd --reload

Verify your setup

Test the load balancer functionality and health checking mechanisms.

# Check Lighttpd status
sudo systemctl status lighttpd

# Test load balancing
curl -H "Host: example.com" http://localhost/
curl -H "Host: example.com" https://localhost/

# Check backend distribution
for i in {1..10}; do
    curl -s -H "Host: example.com" http://localhost/ | grep -o "Backend: [0-9]*" || echo "Request $i completed"
done

# Monitor access logs
sudo tail -f /var/log/lighttpd/access.log

# Check error logs
sudo tail -f /var/log/lighttpd/error.log

Configure monitoring and alerting

Set up basic monitoring to track backend server health and load balancer performance.

#!/bin/bash

# Backend monitoring script
BACKENDS=("192.168.1.10:8080" "192.168.1.11:8080" "192.168.1.12:8080")
LOG_FILE="/var/log/lighttpd/backend-health.log"
EMAIL_ALERT="admin@example.com"

for backend in "${BACKENDS[@]}"; do
    if ! curl -f -s -m 5 "http://$backend/health" >/dev/null 2>&1; then
        echo "$(date): Backend $backend is DOWN" >> "$LOG_FILE"
        echo "Backend $backend is unreachable" | mail -s "Load Balancer Alert" "$EMAIL_ALERT"
    else
        echo "$(date): Backend $backend is UP" >> "$LOG_FILE"
    fi
done

Schedule monitoring script

Set up automated monitoring using cron to check backend health every minute.

sudo chmod +x /usr/local/bin/monitor-backends.sh
echo "* * * * * /usr/local/bin/monitor-backends.sh" | sudo crontab -

Common issues

SymptomCauseFix
503 Service UnavailableAll backend servers downCheck backend health with curl http://backend-ip:port/health
Configuration test failsSyntax error in configRun sudo lighttpd -t -f /etc/lighttpd/lighttpd.conf for details
SSL certificate errorsCertificate path or permissionsVerify paths in SSL config and check sudo ls -la /etc/letsencrypt/live/
Health checks failingLua script permissionsEnsure sudo chmod 644 /etc/lighttpd/healthcheck.lua
Uneven load distributionSession persistence enabledChange proxy.balance = "fair" for round-robin
Backend connection timeoutsNetwork or backend issuesCheck proxy.server.timeout and backend response times

Performance optimization

Fine-tune the load balancer for production workloads and high traffic scenarios.

Note: The health check configuration provides basic functionality. For production use, consider implementing more sophisticated monitoring with Prometheus and Grafana monitoring or automated health alerts.
# Connection tuning
server.max-connections = 4096
server.max-fds = 8192
server.max-worker = 16
server.max-request-size = 65536

# Keepalive settings
server.max-keep-alive-requests = 100
server.max-keep-alive-idle = 30

# Buffer sizes
server.network-backend = "linux-sendfile"
server.upload-dirs = ( "/var/cache/lighttpd/uploads" )

# Proxy optimizations
proxy.server.connect-timeout = 10
proxy.server.timeout = 60
proxy.server.disable-time = 60

# Enable compression for proxied content
compress.cache-dir = "/var/cache/lighttpd/compress/"
compress.filetype += ( "application/json", "application/xml" )

Next steps

Running this in production?

Want this handled for you? Setting this up once is straightforward. Keeping it patched, monitored, backed up and performant across environments is the harder part. See how we run infrastructure like this for European teams.

Automated install script

Run this to automate the entire setup

Don't want to manage this yourself?

We handle infrastructure for businesses that depend on uptime. Fully managed, with one fixed contact who knows your setup.

You get one fixed contact who knows your setup

Rotterdam 01:26 · reachable in a message, no ticket form