Setup Gunicorn blue-green deployment with NGINX for zero downtime Python applications

Update system packages

Start by updating your package manager to ensure you have the latest versions of all required components.

sudo apt update && sudo apt upgrade -y

sudo dnf update -y

Install Python, NGINX, and required packages

Install the base components needed for running Gunicorn applications with NGINX reverse proxy.

sudo apt install -y python3 python3-pip python3-venv nginx curl jq

sudo dnf install -y python3 python3-pip nginx curl jq

Create application user and directory structure

Set up a dedicated user for running your Python applications and create the directory structure for blue-green deployments.

sudo useradd --system --shell /bin/bash --home-dir /opt/app app
sudo mkdir -p /opt/app/{blue,green,shared/{logs,uploads}}
sudo chown -R app:app /opt/app
sudo chmod 755 /opt/app
sudo chmod 775 /opt/app/shared/{logs,uploads}

Create a sample Python application

Create a simple Flask application to demonstrate the blue-green deployment process. This will serve as your application template.

from flask import Flask, jsonify
import os
import socket

app = Flask(__name__)

@app.route('/health')
def health():
    return jsonify({
        'status': 'healthy',
        'version': os.environ.get('APP_VERSION', 'blue'),
        'hostname': socket.gethostname()
    })

@app.route('/')
def home():
    return jsonify({
        'message': 'Hello from Blue Environment',
        'version': os.environ.get('APP_VERSION', 'blue')
    })

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

Set up Python virtual environments

Create separate virtual environments for blue and green deployments to isolate dependencies.

sudo -u app python3 -m venv /opt/app/blue/venv
sudo -u app python3 -m venv /opt/app/green/venv

# Install dependencies for blue environment
sudo -u app /opt/app/blue/venv/bin/pip install flask gunicorn

# Copy application to green environment
sudo -u app cp /opt/app/blue/app.py /opt/app/green/
sudo -u app /opt/app/green/venv/bin/pip install flask gunicorn

Create Gunicorn configuration files

Configure Gunicorn with separate configuration files for blue and green environments, each using different ports.

bind = "127.0.0.1:8000"
workers = 2
worker_class = "sync"
worker_connections = 1000
timeout = 30
keepalive = 2
preload_app = True
max_requests = 1000
max_requests_jitter = 100
pidfile = "/opt/app/blue/gunicorn.pid"
accesslog = "/opt/app/shared/logs/blue-access.log"
errorlog = "/opt/app/shared/logs/blue-error.log"
loglevel = "info"
user = "app"
group = "app"

bind = "127.0.0.1:8001"
workers = 2
worker_class = "sync"
worker_connections = 1000
timeout = 30
keepalive = 2
preload_app = True
max_requests = 1000
max_requests_jitter = 100
pidfile = "/opt/app/green/gunicorn.pid"
accesslog = "/opt/app/shared/logs/green-access.log"
errorlog = "/opt/app/shared/logs/green-error.log"
loglevel = "info"
user = "app"
group = "app"

Create systemd service files

Set up systemd services for both blue and green Gunicorn instances to manage them as system services.

[Unit]
Description=Gunicorn Blue Environment
After=network.target

[Service]
Type=forking
User=app
Group=app
WorkingDirectory=/opt/app/blue
Environment=APP_VERSION=blue
ExecStart=/opt/app/blue/venv/bin/gunicorn --config gunicorn.conf.py app:app
ExecReload=/bin/kill -s HUP $MAINPID
PIDFile=/opt/app/blue/gunicorn.pid
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

[Unit]
Description=Gunicorn Green Environment
After=network.target

[Service]
Type=forking
User=app
Group=app
WorkingDirectory=/opt/app/green
Environment=APP_VERSION=green
ExecStart=/opt/app/green/venv/bin/gunicorn --config gunicorn.conf.py app:app
ExecReload=/bin/kill -s HUP $MAINPID
PIDFile=/opt/app/green/gunicorn.pid
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

Configure NGINX upstream blocks

Set up NGINX with upstream configurations that can switch between blue and green environments. This configuration allows dynamic switching without reloading NGINX.

upstream app_blue {
    server 127.0.0.1:8000 max_fails=3 fail_timeout=30s;
}

upstream app_green {
    server 127.0.0.1:8001 max_fails=3 fail_timeout=30s;
}

upstream app_active {
    server 127.0.0.1:8000 max_fails=3 fail_timeout=30s;
}

Configure NGINX virtual host

Create the main NGINX configuration that uses the upstream blocks and includes health check endpoints.

server {
    listen 80;
    server_name example.com;
    
    # Health check endpoints
    location /health/blue {
        proxy_pass http://app_blue/health;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        access_log off;
    }
    
    location /health/green {
        proxy_pass http://app_green/health;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        access_log off;
    }
    
    # Main application
    location / {
        proxy_pass http://app_active;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # Connection and timeout settings
        proxy_connect_timeout 5s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
        
        # Buffer settings
        proxy_buffering on;
        proxy_buffer_size 4k;
        proxy_buffers 8 4k;
        proxy_busy_buffers_size 8k;
    }
    
    # Status endpoint for deployment script
    location /nginx_status {
        stub_status on;
        access_log off;
        allow 127.0.0.1;
        deny all;
    }
}

Enable the NGINX configuration

Enable the site configuration and ensure NGINX can start successfully.

sudo ln -sf /etc/nginx/sites-available/app /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx

Start the blue environment

Enable and start the blue environment as the initial active deployment.

sudo systemctl daemon-reload
sudo systemctl enable app-blue
sudo systemctl start app-blue
sudo systemctl status app-blue

Create the deployment script

Build an automated deployment script that handles the blue-green switching process with health checks and rollback capability.

#!/bin/bash

set -euo pipefail

# Configuration
BLUE_PORT=8000
GREEN_PORT=8001
HEALTH_TIMEOUT=30
HEALTH_RETRIES=6
NGINX_UPSTREAM_CONF="/etc/nginx/conf.d/upstream.conf"
LOG_FILE="/opt/app/shared/logs/deployment.log"

# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color

# Logging function
log() {
    echo "$(date +'%Y-%m-%d %H:%M:%S') $1" | tee -a "$LOG_FILE"
}

error() {
    log "ERROR: $1" >&2
}

info() {
    log "INFO: $1"
}

# Get current active environment
get_current_env() {
    if grep -q "server 127.0.0.1:$BLUE_PORT" "$NGINX_UPSTREAM_CONF" | grep -q "app_active" -A1; then
        if grep -A1 "upstream app_active" "$NGINX_UPSTREAM_CONF" | grep -q "127.0.0.1:$BLUE_PORT"; then
            echo "blue"
        else
            echo "green"
        fi
    else
        echo "blue"  # Default to blue if unclear
    fi
}

# Get inactive environment
get_inactive_env() {
    current=$(get_current_env)
    if [ "$current" = "blue" ]; then
        echo "green"
    else
        echo "blue"
    fi
}

# Health check function
health_check() {
    local env=$1
    local port
    
    if [ "$env" = "blue" ]; then
        port=$BLUE_PORT
    else
        port=$GREEN_PORT
    fi
    
    info "Performing health check for $env environment (port $port)"
    
    for i in $(seq 1 $HEALTH_RETRIES); do
        if curl -sf "http://127.0.0.1:$port/health" >/dev/null; then
            info "Health check passed for $env environment (attempt $i/$HEALTH_RETRIES)"
            return 0
        fi
        info "Health check failed for $env environment (attempt $i/$HEALTH_RETRIES)"
        sleep 5
    done
    
    error "Health check failed for $env environment after $HEALTH_RETRIES attempts"
    return 1
}

# Switch NGINX upstream
switch_upstream() {
    local target_env=$1
    local target_port
    
    if [ "$target_env" = "blue" ]; then
        target_port=$BLUE_PORT
    else
        target_port=$GREEN_PORT
    fi
    
    info "Switching NGINX upstream to $target_env environment (port $target_port)"
    
    # Create temporary config with new upstream
    cat > "${NGINX_UPSTREAM_CONF}.tmp" << EOF
upstream app_blue {
    server 127.0.0.1:$BLUE_PORT max_fails=3 fail_timeout=30s;
}

upstream app_green {
    server 127.0.0.1:$GREEN_PORT max_fails=3 fail_timeout=30s;
}

upstream app_active {
    server 127.0.0.1:$target_port max_fails=3 fail_timeout=30s;
}
EOF
    
    # Test NGINX configuration
    if nginx -t 2>/dev/null; then
        mv "${NGINX_UPSTREAM_CONF}.tmp" "$NGINX_UPSTREAM_CONF"
        systemctl reload nginx
        info "Successfully switched to $target_env environment"
        return 0
    else
        error "NGINX configuration test failed"
        rm -f "${NGINX_UPSTREAM_CONF}.tmp"
        return 1
    fi
}

# Deploy function
deploy() {
    local current_env
    local inactive_env
    
    current_env=$(get_current_env)
    inactive_env=$(get_inactive_env)
    
    info "Starting deployment process"
    info "Current active environment: $current_env"
    info "Deploying to: $inactive_env"
    
    # Stop inactive environment
    info "Stopping $inactive_env environment"
    systemctl stop "app-$inactive_env" || true
    
    # Update green application for demo (normally you'd deploy your code here)
    if [ "$inactive_env" = "green" ]; then
        cat > "/opt/app/green/app.py" << 'EOF'
from flask import Flask, jsonify
import os
import socket

app = Flask(__name__)

@app.route('/health')
def health():
    return jsonify({
        'status': 'healthy',
        'version': os.environ.get('APP_VERSION', 'green'),
        'hostname': socket.gethostname()
    })

@app.route('/')
def home():
    return jsonify({
        'message': 'Hello from Green Environment',
        'version': os.environ.get('APP_VERSION', 'green')
    })

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)
EOF
    fi
    
    # Start inactive environment
    info "Starting $inactive_env environment"
    systemctl start "app-$inactive_env"
    
    # Wait for service to be ready
    sleep 5
    
    # Perform health checks
    if health_check "$inactive_env"; then
        info "$inactive_env environment is healthy, proceeding with switch"
        
        # Switch traffic
        if switch_upstream "$inactive_env"; then
            info "Deployment successful! Traffic switched to $inactive_env"
            
            # Wait a bit then stop the old environment
            sleep 10
            info "Stopping old $current_env environment"
            systemctl stop "app-$current_env"
            
            info "Deployment completed successfully"
        else
            error "Failed to switch upstream, rolling back"
            systemctl stop "app-$inactive_env"
            exit 1
        fi
    else
        error "Health check failed for $inactive_env environment"
        systemctl stop "app-$inactive_env"
        exit 1
    fi
}

# Rollback function
rollback() {
    local current_env
    local previous_env
    
    current_env=$(get_current_env)
    previous_env=$(get_inactive_env)
    
    info "Rolling back from $current_env to $previous_env"
    
    # Start previous environment
    systemctl start "app-$previous_env"
    sleep 5
    
    # Health check
    if health_check "$previous_env"; then
        if switch_upstream "$previous_env"; then
            info "Rollback successful! Traffic switched back to $previous_env"
            systemctl stop "app-$current_env"
        else
            error "Failed to switch upstream during rollback"
            exit 1
        fi
    else
        error "Health check failed for $previous_env during rollback"
        exit 1
    fi
}

# Status function
status() {
    local current_env
    current_env=$(get_current_env)
    
    echo -e "${BLUE}=== Deployment Status ===${NC}"
    echo -e "Current active environment: ${GREEN}$current_env${NC}"
    echo
    
    echo -e "${BLUE}=== Service Status ===${NC}"
    echo -e "Blue environment: $(systemctl is-active app-blue)"
    echo -e "Green environment: $(systemctl is-active app-green)"
    echo
    
    echo -e "${BLUE}=== Health Checks ===${NC}"
    if curl -sf http://127.0.0.1:$BLUE_PORT/health >/dev/null 2>&1; then
        echo -e "Blue health: ${GREEN}OK${NC}"
    else
        echo -e "Blue health: ${RED}FAIL${NC}"
    fi
    
    if curl -sf http://127.0.0.1:$GREEN_PORT/health >/dev/null 2>&1; then
        echo -e "Green health: ${GREEN}OK${NC}"
    else
        echo -e "Green health: ${RED}FAIL${NC}"
    fi
}

# Main script logic
case "${1:-}" in
    deploy)
        deploy
        ;;
    rollback)
        rollback
        ;;
    status)
        status
        ;;
    *)
        echo "Usage: $0 {deploy|rollback|status}"
        echo "  deploy   - Deploy to inactive environment and switch traffic"
        echo "  rollback - Switch back to previous environment"
        echo "  status   - Show current deployment status"
        exit 1
        ;;
esac

Make deployment script executable and set permissions

Set proper permissions for the deployment script and create required log directories.

sudo chmod +x /opt/app/deploy.sh
sudo chown app:app /opt/app/deploy.sh
sudo touch /opt/app/shared/logs/deployment.log
sudo chown app:app /opt/app/shared/logs/deployment.log

Configure log rotation

Set up log rotation for application and deployment logs to prevent disk space issues.

/opt/app/shared/logs/*.log {
    daily
    missingok
    rotate 30
    compress
    delaycompress
    notifempty
    copytruncate
    su app app
}

Create monitoring script

Build a monitoring script that continuously checks the health of your blue-green deployment and can send alerts.

#!/bin/bash

set -euo pipefail

# Configuration
BLUE_PORT=8000
GREEN_PORT=8001
MONITOR_LOG="/opt/app/shared/logs/monitor.log"
ALERT_EMAIL="admin@example.com"

# Logging function
log() {
    echo "$(date +'%Y-%m-%d %H:%M:%S') $1" | tee -a "$MONITOR_LOG"
}

# Check if environment is responding
check_environment() {
    local env=$1
    local port=$2
    
    if curl -sf --max-time 5 "http://127.0.0.1:$port/health" >/dev/null; then
        return 0
    else
        return 1
    fi
}

# Get active environment from NGINX config
get_active_env() {
    if grep -A1 "upstream app_active" /etc/nginx/conf.d/upstream.conf | grep -q "127.0.0.1:$BLUE_PORT"; then
        echo "blue"
    else
        echo "green"
    fi
}

# Send alert (placeholder - integrate with your alerting system)
send_alert() {
    local message=$1
    log "ALERT: $message"
    # echo "$message" | mail -s "App Deployment Alert" "$ALERT_EMAIL"
}

# Main monitoring loop
monitor() {
    local active_env
    active_env=$(get_active_env)
    
    # Check active environment
    if [ "$active_env" = "blue" ]; then
        if ! check_environment "blue" $BLUE_PORT; then
            send_alert "Active blue environment is not responding"
            return 1
        fi
    else
        if ! check_environment "green" $GREEN_PORT; then
            send_alert "Active green environment is not responding"
            return 1
        fi
    fi
    
    log "Active environment ($active_env) is healthy"
    return 0
}

# Run monitoring check
monitor

sudo chmod +x /opt/app/monitor.sh
sudo chown app:app /opt/app/monitor.sh

Set up monitoring cron job

Configure a cron job to run the monitoring script every minute and log the results.

sudo -u app crontab -l 2>/dev/nu