Linux Performance Tuning Guide

Learn production-grade Linux performance optimization through systematic kernel parameter tuning, memory management, I/O optimization, and network stack configuration to maximize system throughput and responsiveness.

Prerequisites

Root or sudo access
Basic Linux command line knowledge
Understanding of system administration concepts

What this solves

This tutorial teaches you how to optimize Linux system performance through systematic kernel parameter tuning, memory management optimization, and resource configuration. You'll learn how to identify performance bottlenecks, configure sysctl parameters for maximum throughput, optimize file system I/O, tune memory management, and configure network stack settings for high-performance applications like databases, web servers, and containerized workloads.

System assessment and baseline performance metrics

Install performance monitoring tools

Install essential tools for system performance analysis and monitoring before making any optimizations.

sudo apt update
sudo apt install -y htop iotop iftop sysstat dstat procps-ng

sudo dnf update -y
sudo dnf install -y htop iotop iftop sysstat dstat procps-ng

Collect baseline performance metrics

Gather baseline system performance data to compare against after optimization. This creates reference points for measuring improvements.

# CPU and memory usage
vmstat 1 10

# I/O statistics
iostat -x 1 10

# Network statistics
netstat -i
ss -tuln

# Memory information
free -h
cat /proc/meminfo

# Current kernel parameters
sysctl -a | head -20

Identify performance bottlenecks

Use monitoring tools to identify specific areas where performance can be improved.

# Check CPU usage patterns
top -bn1 | head -15

# Monitor I/O wait times
sar -u 1 5

# Check memory pressure
cat /proc/pressure/memory
cat /proc/pressure/cpu
cat /proc/pressure/io

Kernel parameter optimization with sysctl

Create sysctl configuration backup

Backup existing sysctl configuration before making changes to ensure you can restore original settings if needed.

sudo cp /etc/sysctl.conf /etc/sysctl.conf.backup
sudo sysctl -a > /tmp/sysctl-original.txt

Configure kernel parameters for performance

Create optimized sysctl configuration for improved system performance, focusing on memory management, network throughput, and process handling.

# Memory management optimization
vm.dirty_background_ratio = 5
vm.dirty_ratio = 10
vm.dirty_expire_centisecs = 12000
vm.dirty_writeback_centisecs = 1500
vm.overcommit_memory = 1
vm.overcommit_ratio = 50
vm.swappiness = 10
vm.vfs_cache_pressure = 50
vm.min_free_kbytes = 131072

# Network stack optimization
net.core.rmem_default = 262144
net.core.rmem_max = 16777216
net.core.wmem_default = 262144
net.core.wmem_max = 16777216
net.core.netdev_max_backlog = 5000
net.core.netdev_budget = 600
net.ipv4.tcp_congestion_control = bbr
net.ipv4.tcp_rmem = 8192 262144 16777216
net.ipv4.tcp_wmem = 8192 262144 16777216
net.ipv4.tcp_max_syn_backlog = 8192
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 1024 65535

# File system optimization
fs.file-max = 2097152
fs.nr_open = 1048576

# Kernel scheduler optimization
kernel.sched_migration_cost_ns = 5000000
kernel.sched_autogroup_enabled = 0

# Security and stability
kernel.panic = 10
kernel.panic_on_oops = 1

Apply sysctl configuration

Load the new kernel parameters and verify they are applied correctly.

sudo sysctl --system
sudo sysctl -p /etc/sysctl.d/99-performance.conf

# Verify key parameters
sysctl vm.swappiness
sysctl net.ipv4.tcp_congestion_control
sysctl vm.dirty_ratio

File system and I/O performance tuning

Configure I/O scheduler optimization

Set appropriate I/O schedulers for different storage types to optimize disk performance based on your hardware.

# Check current I/O scheduler
cat /sys/block/sda/queue/scheduler

# For SSDs, use mq-deadline or none
echo mq-deadline | sudo tee /sys/block/sda/queue/scheduler

# For HDDs, use bfq for better responsiveness
echo bfq | sudo tee /sys/block/sdb/queue/scheduler

Make I/O scheduler changes persistent

Create udev rules to automatically set I/O schedulers on boot based on device type.

# Set mq-deadline for SSD devices
ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="0", ATTR{queue/scheduler}="mq-deadline"
# Set bfq for HDD devices
ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="1", ATTR{queue/scheduler}="bfq"
# Set none for NVMe devices
ACTION=="add|change", KERNEL=="nvme[0-9]n[0-9]", ATTR{queue/scheduler}="none"

Optimize mount options for performance

Configure filesystem mount options for better I/O performance. These options reduce overhead and improve throughput.

# Check current mount options
mount | grep -E '(ext4|xfs)'

# Example optimized fstab entry for ext4
sudo cp /etc/fstab /etc/fstab.backup

# Example optimized ext4 mount options
/dev/sda1 / ext4 defaults,noatime,nodiratime,commit=60 0 1
# For data partitions, add discard for SSD TRIM support
/dev/sda2 /var/lib/data ext4 defaults,noatime,nodiratime,discard,commit=60 0 2

Memory management and swap optimization

Configure swap settings

Optimize swap usage to reduce memory pressure while maintaining system stability during high memory usage scenarios.

# Check current swap usage
swapon --show
free -h

# Configure transparent huge pages
echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/defrag

Make transparent huge pages setting persistent

Create systemd service to apply transparent huge pages settings on boot for consistent memory performance.

[Unit]
Description=Configure Transparent Huge Pages
After=sysinit.target local-fs.target
Before=mongod.service

[Service]
Type=oneshot
ExecStart=/bin/sh -c 'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled'
ExecStart=/bin/sh -c 'echo madvise > /sys/kernel/mm/transparent_hugepage/defrag'
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

sudo systemctl enable thp-settings.service
sudo systemctl start thp-settings.service

Optimize memory allocation

Configure kernel memory allocation parameters for better performance under memory pressure.

# Add to /etc/sysctl.d/99-performance.conf
sudo tee -a /etc/sysctl.d/99-performance.conf << 'EOF'

# Additional memory optimizations
vm.zone_reclaim_mode = 0
vm.page-cluster = 3
vm.drop_caches = 0
kernel.numa_balancing = 1
EOF

sudo sysctl -p /etc/sysctl.d/99-performance.conf

Network stack tuning for high throughput

Configure advanced TCP settings

Optimize TCP stack parameters for high-throughput applications and low-latency networking requirements.

# Add advanced network tuning to sysctl configuration
sudo tee -a /etc/sysctl.d/99-performance.conf << 'EOF'

# Advanced TCP optimization
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_sack = 1
net.ipv4.tcp_fack = 1
net.ipv4.tcp_low_latency = 1
net.ipv4.tcp_adv_win_scale = 2
net.ipv4.tcp_moderate_rcvbuf = 1
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_frto = 2
net.ipv4.tcp_mtu_probing = 1

# Connection tracking optimization
net.netfilter.nf_conntrack_max = 1048576
net.netfilter.nf_conntrack_tcp_timeout_established = 7200

# UDP buffer optimization
net.ipv4.udp_mem = 102400 873800 16777216
net.ipv4.udp_rmem_min = 8192
net.ipv4.udp_wmem_min = 8192
EOF

Apply network optimizations

Load the network optimizations and verify the settings are applied correctly.

sudo sysctl -p /etc/sysctl.d/99-performance.conf

# Verify network settings
sysctl net.ipv4.tcp_congestion_control
sysctl net.core.rmem_max
sysctl net.core.wmem_max

Process limits and resource management

Configure system-wide limits

Increase system limits for file descriptors, processes, and memory locks to support high-performance applications.

# Append to existing limits.conf
* soft nofile 1048576
* hard nofile 1048576
* soft nproc 1048576
* hard nproc 1048576
* soft memlock unlimited
* hard memlock unlimited
root soft nofile 1048576
root hard nofile 1048576
root soft nproc 1048576
root hard nproc 1048576

Configure systemd service limits

Set appropriate limits for systemd services to ensure they can utilize system resources effectively.

# Uncomment and modify these lines in /etc/systemd/system.conf
DefaultLimitNOFILE=1048576
DefaultLimitNPROC=1048576
DefaultLimitMEMLOCK=infinity
DefaultTasksMax=1048576

sudo systemctl daemon-reexec

Verify limits configuration

Check that the new limits are applied correctly for current and new sessions.

# Check current limits
ulimit -n
ulimit -u

# Check systemd limits
systemctl show --property DefaultLimitNOFILE
systemctl show --property DefaultLimitNPROC

Performance monitoring and validation

Create performance monitoring script

Create a script to continuously monitor key performance metrics and validate optimization effectiveness.

#!/bin/bash

# Performance monitoring script
echo "=== System Performance Report ==="
echo "Date: $(date)"
echo

echo "=== CPU Information ==="
lscpu | grep -E '(CPU\(s\)|Model name|Thread|Core)'
echo

echo "=== Memory Usage ==="
free -h
echo

echo "=== Load Average ==="
uptime
echo

echo "=== I/O Statistics ==="
iostat -x 1 1 | tail -n +4
echo

echo "=== Network Statistics ==="
ss -tuln | wc -l
echo "Active connections: $(ss -tuln | wc -l)"
echo

echo "=== Key Kernel Parameters ==="
echo "vm.swappiness: $(sysctl -n vm.swappiness)"
echo "net.ipv4.tcp_congestion_control: $(sysctl -n net.ipv4.tcp_congestion_control)"
echo "vm.dirty_ratio: $(sysctl -n vm.dirty_ratio)"
echo "net.core.rmem_max: $(sysctl -n net.core.rmem_max)"
echo

sudo chmod +x /usr/local/bin/perf-monitor.sh

Set up automated performance monitoring

Configure cron job to run performance monitoring at regular intervals and log results for trend analysis.

# Create log directory
sudo mkdir -p /var/log/performance

# Add cron job for performance monitoring
(crontab -l 2>/dev/null; echo "*/15 * * * * /usr/local/bin/perf-monitor.sh >> /var/log/performance/perf-$(date +\%Y\%m\%d).log 2>&1") | crontab -

Verify your setup

Test the performance optimizations and ensure all settings are applied correctly.

# Check all sysctl parameters are loaded
sudo sysctl --system

# Verify key performance settings
sysctl vm.swappiness vm.dirty_ratio net.ipv4.tcp_congestion_control

# Check I/O schedulers
for disk in /sys/block/sd*; do echo "$disk: $(cat $disk/queue/scheduler)"; done

# Verify system limits
ulimit -a | grep -E '(open files|max user processes)'

# Test network performance (if applicable)
ss -tuln | head -10

# Run performance monitoring script
/usr/local/bin/perf-monitor.sh

# Check system stability
dmesg | tail -20
systemctl status

Common issues

Symptom	Cause	Fix
High I/O wait times	Wrong I/O scheduler for storage type	Set mq-deadline for SSD, bfq for HDD using udev rules
Network connection timeouts	TCP buffer sizes too small	Verify net.core.rmem_max and net.core.wmem_max settings
Memory allocation failures	vm.overcommit_memory too restrictive	Set vm.overcommit_memory=1 and adjust vm.overcommit_ratio
High swap usage	vm.swappiness too high	Reduce vm.swappiness to 10 or lower for server workloads
File descriptor limits exceeded	Default limits too low	Increase nofile limits in /etc/security/limits.conf
Sysctl changes not persistent	Settings not in /etc/sysctl.d/	Create configuration files in /etc/sysctl.d/ directory

Next steps

Automated install script

Run this to automate the entire setup

install.sh

#!/usr/bin/env bash

set -euo pipefail

# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color

# Configuration
SYSCTL_BACKUP="/etc/sysctl.conf.backup"
PERFORMANCE_CONF="/etc/sysctl.d/99-performance.conf"
UDEV_RULES="/etc/udev/rules.d/60-io-scheduler.rules"
LIMITS_CONF="/etc/security/limits.d/99-performance.conf"

# Cleanup function
cleanup() {
    if [[ $? -ne 0 ]]; then
        echo -e "${RED}[ERROR] Installation failed. Rolling back changes...${NC}"
        if [[ -f "$SYSCTL_BACKUP" ]]; then
            sudo cp "$SYSCTL_BACKUP" /etc/sysctl.conf
            sudo sysctl -p
        fi
        if [[ -f "$PERFORMANCE_CONF" ]]; then
            sudo rm -f "$PERFORMANCE_CONF"
        fi
        if [[ -f "$UDEV_RULES" ]]; then
            sudo rm -f "$UDEV_RULES"
        fi
        if [[ -f "$LIMITS_CONF" ]]; then
            sudo rm -f "$LIMITS_CONF"
        fi
    fi
}

trap cleanup ERR

print_usage() {
    echo "Usage: $0 [OPTIONS]"
    echo "Options:"
    echo "  --skip-backup    Skip creating backup of existing sysctl.conf"
    echo "  --help           Show this help message"
    echo ""
    echo "This script optimizes Linux system performance through kernel parameter tuning."
}

# Parse arguments
SKIP_BACKUP=false
while [[ $# -gt 0 ]]; do
    case $1 in
        --skip-backup)
            SKIP_BACKUP=true
            shift
            ;;
        --help)
            print_usage
            exit 0
            ;;
        *)
            echo -e "${RED}Unknown option: $1${NC}"
            print_usage
            exit 1
            ;;
    esac
done

# Check if running as root or with sudo
if [[ $EUID -eq 0 ]]; then
    SUDO_CMD=""
else
    if ! command -v sudo &> /dev/null; then
        echo -e "${RED}[ERROR] This script requires sudo or root privileges${NC}"
        exit 1
    fi
    SUDO_CMD="sudo"
fi

# Detect distribution
if [[ -f /etc/os-release ]]; then
    . /etc/os-release
    case "$ID" in
        ubuntu|debian)
            PKG_MGR="apt"
            PKG_UPDATE="apt update"
            PKG_INSTALL="apt install -y"
            ;;
        almalinux|rocky|centos|rhel|ol|fedora)
            PKG_MGR="dnf"
            PKG_UPDATE="dnf update -y"
            PKG_INSTALL="dnf install -y"
            if ! command -v dnf &> /dev/null; then
                PKG_MGR="yum"
                PKG_UPDATE="yum update -y"
                PKG_INSTALL="yum install -y"
            fi
            ;;
        amzn)
            PKG_MGR="yum"
            PKG_UPDATE="yum update -y"
            PKG_INSTALL="yum install -y"
            ;;
        *)
            echo -e "${RED}[ERROR] Unsupported distribution: $ID${NC}"
            exit 1
            ;;
    esac
else
    echo -e "${RED}[ERROR] Cannot detect distribution${NC}"
    exit 1
fi

echo -e "${GREEN}[1/8] Installing performance monitoring tools...${NC}"
$SUDO_CMD $PKG_UPDATE
$SUDO_CMD $PKG_INSTALL htop iotop iftop sysstat dstat procps-ng 2>/dev/null || $SUDO_CMD $PKG_INSTALL htop iotop iftop sysstat dstat procps

echo -e "${GREEN}[2/8] Collecting baseline performance metrics...${NC}"
echo -e "${YELLOW}Current system state:${NC}"
free -h
echo "Load average: $(uptime | awk -F'load average:' '{print $2}')"

echo -e "${GREEN}[3/8] Creating configuration backups...${NC}"
if [[ "$SKIP_BACKUP" == false ]]; then
    if [[ -f /etc/sysctl.conf ]]; then
        $SUDO_CMD cp /etc/sysctl.conf "$SYSCTL_BACKUP"
    fi
    $SUDO_CMD sysctl -a > /tmp/sysctl-original.txt 2>/dev/null || true
fi

echo -e "${GREEN}[4/8] Configuring kernel parameters for performance...${NC}"
$SUDO_CMD tee "$PERFORMANCE_CONF" > /dev/null <<EOF
# Performance optimization configuration
# Memory management optimization
vm.dirty_background_ratio = 5
vm.dirty_ratio = 10
vm.dirty_expire_centisecs = 12000
vm.dirty_writeback_centisecs = 1500
vm.overcommit_memory = 1
vm.overcommit_ratio = 50
vm.swappiness = 10
vm.vfs_cache_pressure = 50
vm.min_free_kbytes = 131072

# Network stack optimization
net.core.rmem_default = 262144
net.core.rmem_max = 16777216
net.core.wmem_default = 262144
net.core.wmem_max = 16777216
net.core.netdev_max_backlog = 5000
net.core.netdev_budget = 600
net.ipv4.tcp_rmem = 8192 262144 16777216
net.ipv4.tcp_wmem = 8192 262144 16777216
net.ipv4.tcp_max_syn_backlog = 8192
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 1024 65535

# File system optimization
fs.file-max = 2097152
fs.nr_open = 1048576

# Kernel scheduler optimization
kernel.sched_migration_cost_ns = 5000000
kernel.sched_autogroup_enabled = 0

# Security and stability
kernel.panic = 10
kernel.panic_on_oops = 1
EOF

$SUDO_CMD chmod 644 "$PERFORMANCE_CONF"

echo -e "${GREEN}[5/8] Applying sysctl configuration...${NC}"
$SUDO_CMD sysctl --system
if modinfo tcp_bbr &>/dev/null; then
    echo 'net.ipv4.tcp_congestion_control = bbr' | $SUDO_CMD tee -a "$PERFORMANCE_CONF" > /dev/null
    $SUDO_CMD sysctl -w net.ipv4.tcp_congestion_control=bbr 2>/dev/null || true
fi

echo -e "${GREEN}[6/8] Configuring I/O scheduler optimization...${NC}"
$SUDO_CMD tee "$UDEV_RULES" > /dev/null <<EOF
# I/O Scheduler optimization rules
# Set mq-deadline for SSD devices
ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="0", ATTR{queue/scheduler}="mq-deadline"
# Set bfq for HDD devices  
ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="1", ATTR{queue/scheduler}="bfq"
# Set none for NVMe devices
ACTION=="add|change", KERNEL=="nvme[0-9]n[0-9]", ATTR{queue/scheduler}="none"
EOF

$SUDO_CMD chmod 644 "$UDEV_RULES"
$SUDO_CMD udevadm control --reload-rules

echo -e "${GREEN}[7/8] Configuring system limits...${NC}"
$SUDO_CMD tee "$LIMITS_CONF" > /dev/null <<EOF
# Performance limits configuration
* soft nofile 1048576
* hard nofile 1048576
* soft nproc 1048576
* hard nproc 1048576
root soft nofile 1048576
root hard nofile 1048576
EOF

$SUDO_CMD chmod 644 "$LIMITS_CONF"

echo -e "${GREEN}[8/8] Verifying configuration...${NC}"
echo -e "${YELLOW}Key parameters verification:${NC}"
echo "vm.swappiness = $(sysctl -n vm.swappiness)"
echo "vm.dirty_ratio = $(sysctl -n vm.dirty_ratio)"
echo "net.core.rmem_max = $(sysctl -n net.core.rmem_max)"
echo "fs.file-max = $(sysctl -n fs.file-max)"

if command -v systemctl &> /dev/null; then
    echo -e "${YELLOW}Restarting system services...${NC}"
    $SUDO_CMD systemctl restart systemd-sysctl 2>/dev/null || true
fi

echo -e "${GREEN}Performance optimization completed successfully!${NC}"
echo -e "${YELLOW}Note: Some optimizations require a system reboot to take full effect.${NC}"
echo -e "${YELLOW}Backup files created:${NC}"
[[ -f "$SYSCTL_BACKUP" ]] && echo "  - $SYSCTL_BACKUP"
echo "  - /tmp/sysctl-original.txt"

Review the script before running. Execute with: bash install.sh

#linux performance tuning #sysctl optimization #kernel parameters #system tuning #linux optimization

Optimize Linux system performance with kernel parameters and system tuning