Configure Linux system resource limits with systemd and ulimit for application performance

View current limits for your session

Check the resource limits currently applied to your shell session.

ulimit -a

Check limits for a running process

View the resource limits for any running process by examining its proc filesystem entry.

cat /proc/$(pgrep nginx | head -1)/limits

View systemd service limits

Check the resource limits configured for a systemd service.

systemctl show nginx --property=LimitNOFILE --property=LimitNPROC --property=LimitMEMLOCK

Configure system-wide limits with systemd

Create a systemd configuration file to set system-wide default limits for all services.

sudo mkdir -p /etc/systemd/system.conf.d

[Manager]
DefaultLimitNOFILE=65536
DefaultLimitNPROC=32768
DefaultLimitMEMLOCK=infinity
DefaultLimitCORE=infinity

Configure per-service limits with systemd

Create a systemd drop-in directory to override limits for specific services like nginx.

sudo mkdir -p /etc/systemd/system/nginx.service.d

[Service]
LimitNOFILE=100000
LimitNPROC=65536
LimitMEMLOCK=infinity

Configure user session limits

Edit the limits.conf file to set resource limits for user sessions and login processes.

sudo cp /etc/security/limits.conf /etc/security/limits.conf.backup

# Add these lines to the end of the file
* soft nofile 65536
* hard nofile 65536
* soft nproc 32768
* hard nproc 32768
www-data soft nofile 100000
www-data hard nofile 100000
root soft nofile 65536
root hard nofile 65536

Configure kernel limits

Set kernel-level limits that apply system-wide using sysctl parameters.

# Maximum number of open files system-wide
fs.file-max = 2097152
Maximum number of processes system-wide
kernel.pid_max = 4194304
Maximum number of memory map areas
vm.max_map_count = 262144

Apply the configuration changes

Reload systemd configuration and apply kernel parameter changes.

sudo systemctl daemon-reload
sudo sysctl --system

Restart services to apply limits

Restart the services you modified to apply the new resource limits.

sudo systemctl restart nginx
sudo systemctl restart systemd-logind

File descriptor limits (NOFILE)

Configure limits for the number of open files, crucial for web servers and databases handling many connections.

[Service]
LimitNOFILE=100000
LimitNOFILESoft=100000

Process limits (NPROC)

Set limits on the number of processes a user or service can create.

[Service]
LimitNPROC=65536
LimitNPROCSoft=32768

Memory limits (AS, RSS, MEMLOCK)

Configure memory-related limits including virtual memory and locked memory.

[Service]
LimitMEMLOCK=infinity
LimitAS=infinity
LimitRSS=8G

Set temporary limits with ulimit

Adjust limits temporarily for the current shell session or script execution.

# Increase file descriptor limit for current session
ulimit -n 100000

Set memory limit to 2GB
ulimit -v 2097152

Set maximum processes to 16384
ulimit -u 16384

Apply limits to specific commands

Use the prlimit command to set limits for specific processes or commands.

# Run command with specific limits
prlimit --nofile=50000:100000 --nproc=10000:20000 your-application

Set limits for running process
sudo prlimit --pid 1234 --nofile=100000

Debug "too many open files" errors

Identify which processes are consuming file descriptors and adjust limits accordingly.

# Find processes using the most file descriptors
sudo lsof | awk '{print $2}' | sort | uniq -c | sort -nr | head -10

Check file descriptor usage for specific process
sudo ls -la /proc/$(pgrep nginx | head -1)/fd | wc -l

Monitor resource usage

Set up monitoring to track resource consumption and prevent limit-related issues.

# Create monitoring script
sudo tee /usr/local/bin/check-limits.sh << 'EOF'
#!/bin/bash
echo "=== System-wide limits ==="
echo "Open files: $(cat /proc/sys/fs/file-nr)"
echo "Max files: $(cat /proc/sys/fs/file-max)"
echo "Processes: $(ps aux | wc -l)"
echo "Max PID: $(cat /proc/sys/kernel/pid_max)"

echo -e "\n=== Top processes by open files ==="
for pid in $(ps -eo pid --no-headers | head -10); do
    if [ -d /proc/$pid/fd ]; then
        count=$(ls /proc/$pid/fd 2>/dev/null | wc -l)
        comm=$(cat /proc/$pid/comm 2>/dev/null || echo "unknown")
        echo "$pid ($comm): $count files"
    fi
done | sort -k3 -nr
EOF

sudo chmod +x /usr/local/bin/check-limits.sh