Configure Linux Log Rotation with Logrotate

Master log rotation with logrotate to manage disk space efficiently. Configure custom rotation policies, compression settings, and automated cleanup for applications and system logs.

Prerequisites

Root or sudo access
Basic understanding of Linux file permissions
Email server configured (optional for notifications)

What this solves

Log files can consume enormous amounts of disk space without proper rotation, potentially causing system outages when storage fills up. This tutorial shows you how to configure logrotate to automatically compress, rotate, and clean up logs based on size, age, and custom policies, ensuring your servers maintain optimal performance and storage availability.

Step-by-step configuration

Install logrotate package

Most Linux distributions include logrotate by default, but verify it's installed and up to date.

sudo apt update
sudo apt install -y logrotate

sudo dnf update -y
sudo dnf install -y logrotate

Verify logrotate installation and status

Check that logrotate is properly installed and review the main configuration file structure.

logrotate --version
ls -la /etc/logrotate.conf
ls -la /etc/logrotate.d/

Configure global logrotate settings

Edit the main configuration file to set default rotation policies that apply to all logs unless overridden.

# Global settings for log rotation
weekly
rotate 4
create
dateext
compress
delaycompress

# Include application-specific configurations
include /etc/logrotate.d

# System logs configuration
/var/log/wtmp {
    monthly
    create 0664 root utmp
    minsize 1M
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0600 root utmp
    rotate 1
}

Create custom application log rotation

Configure rotation for web server logs with compression and size-based rotation to prevent disk space issues.

/var/log/webapp/*.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    create 0644 webapp webapp
    size 100M
    maxage 90
    postrotate
        systemctl reload webapp 2>/dev/null || true
    endscript
}

Configure system service log rotation

Set up rotation for system services like nginx with proper permissions and reload commands.

/var/log/nginx/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 0640 www-data adm
    size 50M
    maxage 30
    prerotate
        if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
            run-parts /etc/logrotate.d/httpd-prerotate; \
        fi
    endscript
    postrotate
        if [ -f /var/run/nginx.pid ]; then \
            kill -USR1 `cat /var/run/nginx.pid`; \
        fi
    endscript
}

Configure database log rotation with compression

Set up MySQL/MariaDB log rotation with proper ownership and compression settings.

/var/log/mysql/*.log {
    weekly
    rotate 8
    compress
    delaycompress
    missingok
    notifempty
    create 0640 mysql mysql
    copytruncate
    maxage 56
    postrotate
        systemctl reload mysql 2>/dev/null || true
    endscript
}

/var/log/mysql/error.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 0640 mysql mysql
    size 10M
    maxage 14
}

Set up email notifications for rotation failures

Configure logrotate to send email alerts when rotation fails, helping you catch issues before they cause outages.

sudo apt install -y mailutils postfix

sudo dnf install -y mailx postfix

# Add to top of /etc/logrotate.conf
mail admin@example.com
mailfirst

Create disk space monitoring script

Set up automated disk usage monitoring to complement log rotation and prevent storage issues.

#!/bin/bash

# Disk space monitoring script for log directories
THRESHOLD=80
EMAIL="admin@example.com"

check_disk_usage() {
    local path="$1"
    local usage=$(df "$path" | tail -1 | awk '{print $5}' | sed 's/%//')
    
    if [ "$usage" -gt "$THRESHOLD" ]; then
        echo "WARNING: Disk usage for $path is ${usage}%" | \
        mail -s "High Disk Usage Alert - $(hostname)" "$EMAIL"
    fi
}

# Monitor critical log directories
check_disk_usage "/var/log"
check_disk_usage "/var/log/nginx"
check_disk_usage "/var/log/mysql"

# Force logrotate if disk usage is critical (>90%)
for dir in /var/log /var/log/nginx /var/log/mysql; do
    usage=$(df "$dir" | tail -1 | awk '{print $5}' | sed 's/%//')
    if [ "$usage" -gt 90 ]; then
        /usr/sbin/logrotate -f /etc/logrotate.conf
        echo "Emergency logrotate executed for high disk usage" | \
        mail -s "Emergency Log Rotation - $(hostname)" "$EMAIL"
    fi
done

sudo chmod 755 /usr/local/bin/check-disk-space.sh

Schedule automated monitoring and rotation

Set up cron jobs to run logrotate and disk monitoring automatically.

sudo crontab -e

# Run logrotate daily at 2 AM
0 2 * * * /usr/sbin/logrotate /etc/logrotate.conf

# Check disk space every 4 hours
0 */4 * * * /usr/local/bin/check-disk-space.sh

# Weekly cleanup of old compressed logs
0 3 * * 0 find /var/log -name "*.gz" -mtime +90 -delete

Configure logrotate status tracking

Set up proper status file permissions and monitoring for logrotate execution history.

sudo touch /var/lib/logrotate/status
sudo chmod 644 /var/lib/logrotate/status
sudo chown root:root /var/lib/logrotate/status

Test rotation configuration

Perform a dry run to verify your configuration works correctly before relying on automated rotation.

# Test configuration syntax
sudo logrotate -d /etc/logrotate.conf

# Force rotation for testing (creates actual rotated files)
sudo logrotate -f /etc/logrotate.conf

# Test specific configuration
sudo logrotate -d /etc/logrotate.d/nginx

Advanced compression and retention settings

Configure different compression methods

Optimize compression settings for different log types to balance CPU usage and space savings.

# High compression for infrequently accessed logs
/var/log/archive/*.log {
    monthly
    rotate 12
    compress
    compresscmd /usr/bin/xz
    uncompresscmd /usr/bin/unxz
    compressext .xz
    compressoptions -9
    delaycompress
    create 0644 root root
}

# Fast compression for frequently rotated logs
/var/log/highvolume/*.log {
    hourly
    rotate 168
    compress
    compresscmd /usr/bin/gzip
    compressoptions -1
    delaycompress
    size 1G
    create 0644 app app
}

Set up conditional rotation based on system load

Create intelligent rotation that considers system resources to avoid impacting performance during peak times.

/var/log/conditional/*.log {
    daily
    rotate 30
    compress
    missingok
    notifempty
    create 0644 app app
    prerotate
        # Skip rotation if system load is too high
        load=$(uptime | awk -F'load average:' '{print $2}' | awk '{print $1}' | sed 's/,//')
        if (( $(echo "$load > 5.0" | bc -l) )); then
            echo "Skipping rotation due to high system load: $load"
            exit 1
        fi
    endscript
    postrotate
        systemctl reload app 2>/dev/null || true
    endscript
}

Verify your setup

# Check logrotate service status
sudo systemctl status logrotate.timer

# Verify configuration files
sudo logrotate -d /etc/logrotate.conf

# Check rotation status
sudo cat /var/lib/logrotate/status

# Monitor disk usage
df -h /var/log

# List rotated and compressed logs
find /var/log -name "*.gz" -o -name "*.xz" | head -10

# Check cron job status
sudo grep logrotate /var/log/cron

Monitor rotation effectiveness

# Create monitoring script for rotation metrics
cat > /usr/local/bin/logrotate-stats.sh << 'EOF'
#!/bin/bash
echo "=== Log Rotation Statistics ==="
echo "Total log files: $(find /var/log -name "*.log" | wc -l)"
echo "Compressed files: $(find /var/log -name "*.gz" -o -name "*.xz" | wc -l)"
echo "Disk usage: $(du -sh /var/log | awk '{print $1}')"
echo "Last rotation: $(stat -c %y /var/lib/logrotate/status)"
echo "Rotation errors: $(grep -c ERROR /var/log/logrotate.log 2>/dev/null || echo 0)"
EOF

sudo chmod +x /usr/local/bin/logrotate-stats.sh
/usr/local/bin/logrotate-stats.sh

Common issues

Symptom	Cause	Fix
Files not rotating	Incorrect permissions on log files	`sudo chown root:root /etc/logrotate.d/*` and check log file ownership
Compression fails	Missing compression utilities	Install gzip/xz: `sudo apt install gzip xz-utils` (Ubuntu) or `sudo dnf install gzip xz` (RHEL)
Service reload fails	Invalid postrotate command	Test commands manually and use `\|\| true` to prevent failures
Email notifications not working	Mail system not configured	Configure postfix: `sudo dpkg-reconfigure postfix`
Logs still consuming space	Application writing to rotated files	Use `copytruncate` option or ensure proper service reload
Permission denied errors	Logrotate running as wrong user	Check cron runs as root and log directories have correct permissions

Never use chmod 777. It gives every user on the system full access to your log files. Instead, set specific ownership with chown and use minimal permissions like 644 for log files and 755 for log directories.

Next steps

Automated install script

Run this to automate the entire setup

install.sh

#!/usr/bin/env bash

set -euo pipefail

# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color

# Global variables
EMAIL_ADMIN="${1:-admin@localhost}"
DISK_THRESHOLD="${2:-80}"

# Usage message
usage() {
    echo "Usage: $0 [admin_email] [disk_threshold]"
    echo "  admin_email: Email for alerts (default: admin@localhost)"
    echo "  disk_threshold: Disk usage alert threshold % (default: 80)"
    exit 1
}

# Output functions
success() { echo -e "${GREEN}[SUCCESS]${NC} $1"; }
error() { echo -e "${RED}[ERROR]${NC} $1" >&2; }
warning() { echo -e "${YELLOW}[WARNING]${NC} $1"; }
info() { echo -e "[INFO] $1"; }

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        error "This script must be run as root or with sudo"
        exit 1
    fi
}

# Cleanup function for rollback
cleanup() {
    local exit_code=$?
    if [[ $exit_code -ne 0 ]]; then
        warning "Script failed. Cleaning up..."
        # Restore original logrotate.conf if backup exists
        [[ -f /etc/logrotate.conf.backup ]] && mv /etc/logrotate.conf.backup /etc/logrotate.conf
        # Remove created custom configs
        rm -f /etc/logrotate.d/webapp /etc/logrotate.d/nginx-custom /etc/logrotate.d/mysql-custom
        rm -f /usr/local/bin/check-disk-space.sh
        error "Installation failed and changes have been rolled back"
    fi
}

# Detect distribution
detect_distro() {
    if [[ ! -f /etc/os-release ]]; then
        error "Cannot detect distribution - /etc/os-release not found"
        exit 1
    fi
    
    . /etc/os-release
    case "$ID" in
        ubuntu|debian)
            PKG_MGR="apt"
            PKG_INSTALL="apt install -y"
            PKG_UPDATE="apt update"
            MAIL_PKG="mailutils"
            MTA_PKG="postfix"
            ;;
        almalinux|rocky|centos|rhel|ol|fedora)
            PKG_MGR="dnf"
            PKG_INSTALL="dnf install -y"
            PKG_UPDATE="dnf update -y"
            MAIL_PKG="mailx"
            MTA_PKG="postfix"
            ;;
        amzn)
            PKG_MGR="yum"
            PKG_INSTALL="yum install -y"
            PKG_UPDATE="yum update -y"
            MAIL_PKG="mailx"
            MTA_PKG="postfix"
            ;;
        *)
            error "Unsupported distribution: $ID"
            exit 1
            ;;
    esac
    
    success "Detected distribution: $PRETTY_NAME"
}

# Validate email format
validate_email() {
    if [[ ! "$EMAIL_ADMIN" =~ ^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$ ]]; then
        error "Invalid email format: $EMAIL_ADMIN"
        usage
    fi
}

# Validate threshold
validate_threshold() {
    if [[ ! "$DISK_THRESHOLD" =~ ^[0-9]+$ ]] || [[ "$DISK_THRESHOLD" -lt 1 ]] || [[ "$DISK_THRESHOLD" -gt 99 ]]; then
        error "Invalid disk threshold: $DISK_THRESHOLD (must be 1-99)"
        usage
    fi
}

# Main installation function
main() {
    trap cleanup ERR
    
    check_root
    validate_email
    validate_threshold
    
    info "Starting logrotate configuration with compression..."
    info "Admin email: $EMAIL_ADMIN"
    info "Disk threshold: $DISK_THRESHOLD%"
    
    echo "[1/8] Detecting distribution..."
    detect_distro
    
    echo "[2/8] Installing logrotate package..."
    $PKG_INSTALL logrotate
    success "Logrotate installed successfully"
    
    echo "[3/8] Backing up original configuration..."
    cp /etc/logrotate.conf /etc/logrotate.conf.backup
    success "Configuration backed up"
    
    echo "[4/8] Configuring global logrotate settings..."
    cat > /etc/logrotate.conf << 'EOF'
# Global logrotate configuration
weekly
rotate 4
create
dateext
compress
delaycompress
mail admin@example.com
mailfirst

# Include application-specific configurations
include /etc/logrotate.d

# System logs configuration
/var/log/wtmp {
    monthly
    create 0664 root utmp
    minsize 1M
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0600 root utmp
    rotate 1
}
EOF
    
    # Update email in config
    sed -i "s/admin@example.com/$EMAIL_ADMIN/g" /etc/logrotate.conf
    success "Global configuration updated"
    
    echo "[5/8] Creating custom application log rotation configs..."
    
    # Web application logs
    cat > /etc/logrotate.d/webapp << 'EOF'
/var/log/webapp/*.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    create 0644 www-data www-data
    size 100M
    maxage 90
    postrotate
        systemctl reload webapp 2>/dev/null || true
    endscript
}
EOF
    
    # Nginx logs (if nginx directory exists)
    if [[ -d /var/log/nginx ]] || [[ -d /etc/nginx ]]; then
        cat > /etc/logrotate.d/nginx-custom << 'EOF'
/var/log/nginx/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 0640 www-data adm
    size 50M
    maxage 30
    prerotate
        if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
            run-parts /etc/logrotate.d/httpd-prerotate; \
        fi
    endscript
    postrotate
        if [ -f /var/run/nginx.pid ]; then \
            kill -USR1 $(cat /var/run/nginx.pid) 2>/dev/null || true; \
        fi
    endscript
}
EOF
        # Fix ownership for RHEL-based systems
        if [[ "$PKG_MGR" != "apt" ]]; then
            sed -i 's/www-data adm/nginx nginx/g' /etc/logrotate.d/nginx-custom
        fi
        success "Nginx log rotation configured"
    fi
    
    # MySQL/MariaDB logs
    if [[ -d /var/log/mysql ]] || [[ -d /var/log/mariadb ]]; then
        cat > /etc/logrotate.d/mysql-custom << 'EOF'
/var/log/mysql/*.log /var/log/mariadb/*.log {
    weekly
    rotate 8
    compress
    delaycompress
    missingok
    notifempty
    create 0640 mysql mysql
    copytruncate
    maxage 56
    postrotate
        systemctl reload mysql 2>/dev/null || systemctl reload mariadb 2>/dev/null || true
    endscript
}

/var/log/mysql/error.log /var/log/mariadb/error.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    create 0640 mysql mysql
    size 10M
    maxage 14
}
EOF
        success "MySQL/MariaDB log rotation configured"
    fi
    
    echo "[6/8] Installing mail utilities for notifications..."
    $PKG_INSTALL $MAIL_PKG $MTA_PKG
    systemctl enable postfix
    systemctl start postfix
    success "Mail utilities configured"
    
    echo "[7/8] Creating disk space monitoring script..."
    cat > /usr/local/bin/check-disk-space.sh << EOF
#!/bin/bash
# Disk space monitoring script for log directories
THRESHOLD=$DISK_THRESHOLD
EMAIL="$EMAIL_ADMIN"

check_disk_usage() {
    local path="\$1"
    if [[ ! -d "\$path" ]]; then
        return 0
    fi
    
    local usage=\$(df "\$path" | tail -1 | awk '{print \$5}' | sed 's/%//')
    
    if [[ "\$usage" -gt "\$THRESHOLD" ]]; then
        echo "WARNING: Disk usage for \$path is \${usage}%" | \\
        mail -s "High Disk Usage Alert - \$(hostname)" "\$EMAIL"
    fi
    
    # Force logrotate if usage is critical (>90%)
    if [[ "\$usage" -gt 90 ]]; then
        /usr/sbin/logrotate -f /etc/logrotate.conf
    fi
}

# Monitor critical log directories
check_disk_usage "/var/log"
[[ -d "/var/log/nginx" ]] && check_disk_usage "/var/log/nginx"
[[ -d "/var/log/mysql" ]] && check_disk_usage "/var/log/mysql"
[[ -d "/var/log/mariadb" ]] && check_disk_usage "/var/log/mariadb"
[[ -d "/var/log/webapp" ]] && check_disk_usage "/var/log/webapp"
EOF
    
    chmod 755 /usr/local/bin/check-disk-space.sh
    chown root:root /usr/local/bin/check-disk-space.sh
    
    # Add to cron
    (crontab -l 2>/dev/null; echo "0 */6 * * * /usr/local/bin/check-disk-space.sh") | crontab -
    success "Disk monitoring script created and scheduled"
    
    echo "[8/8] Performing verification checks..."
    
    # Test logrotate configuration
    if logrotate -d /etc/logrotate.conf > /dev/null 2>&1; then
        success "Logrotate configuration syntax is valid"
    else
        error "Logrotate configuration has syntax errors"
        exit 1
    fi
    
    # Check logrotate service
    logrotate --version > /dev/null
    success "Logrotate is working properly"
    
    # Test disk monitoring script
    if /usr/local/bin/check-disk-space.sh > /dev/null 2>&1; then
        success "Disk monitoring script is functional"
    else
        warning "Disk monitoring script may have issues"
    fi
    
    success "Logrotate configuration completed successfully!"
    info "Configuration files created:"
    info "  - /etc/logrotate.conf (main configuration)"
    info "  - /etc/logrotate.d/webapp (web application logs)"
    [[ -f /etc/logrotate.d/nginx-custom ]] && info "  - /etc/logrotate.d/nginx-custom (nginx logs)"
    [[ -f /etc/logrotate.d/mysql-custom ]] && info "  - /etc/logrotate.d/mysql-custom (database logs)"
    info "  - /usr/local/bin/check-disk-space.sh (monitoring script)"
    info ""
    info "Email notifications will be sent to: $EMAIL_ADMIN"
    info "Disk usage threshold set to: $DISK_THRESHOLD%"
    info "Monitoring runs every 6 hours via cron"
}

# Run main function
main "$@"

Review the script before running. Execute with: bash install.sh

#logrotate #log rotation #compression #disk management #system maintenance

Configure Linux log rotation with logrotate and compression for system maintenance