Configure Linux user session limits with systemd and pam_limits for resource management

Install required packages

Install systemd development tools and PAM configuration utilities for managing user limits.

sudo apt update
sudo apt install -y systemd-container libpam-systemd

sudo dnf install -y systemd-container systemd-pam

Check current user limits

Examine existing resource limits for your user session to understand current restrictions.

ulimit -a
systemctl --user show --property=DefaultMemoryAccounting,DefaultTasksAccounting

Configure PAM limits

Set up traditional resource limits that apply when users log in through PAM authentication.

sudo cp /etc/security/limits.conf /etc/security/limits.conf.backup

# User limits configuration
# Format: 

# Limit open files (fixes "too many open files" error)
* soft nofile 65536
* hard nofile 65536

# Limit maximum processes per user
* soft nproc 32768
* hard nproc 32768

# Limit virtual memory (in KB)
* soft as unlimited
* hard as unlimited

# Limit core file size
* soft core 0
* hard core 0

# Specific limits for web server users
www-data soft nofile 100000
www-data hard nofile 100000
www-data soft nproc 16384
www-data hard nproc 16384

# Limits for database users
mysql soft nofile 65536
mysql hard nofile 65536
postgres soft nofile 65536
postgres hard nofile 65536

# Group-based limits for developers
@developers soft nofile 32768
@developers hard nofile 32768
@developers soft nproc 16384
@developers hard nproc 16384

Configure systemd user session limits

Create systemd configuration to control resources for entire user sessions using cgroups.

sudo mkdir -p /etc/systemd/user.conf.d
sudo mkdir -p /etc/systemd/system.conf.d

[Manager]
# Enable resource accounting for user sessions
DefaultMemoryAccounting=yes
DefaultTasksAccounting=yes
DefaultIOAccounting=yes
DefaultCPUAccounting=yes

# Default limits for user sessions
DefaultTasksMax=16384
DefaultMemoryHigh=4G
DefaultMemoryMax=8G
DefaultTimeoutStopSec=30s

Create user-specific systemd limits

Set up per-user resource limits using systemd user slice configuration files.

sudo mkdir -p /etc/systemd/system/user-.slice.d

[Slice]
# Memory limits for user sessions
MemoryAccounting=yes
MemoryHigh=2G
MemoryMax=4G

# Process limits
TasksAccounting=yes
TasksMax=8192

# CPU limits (percentage of one CPU core)
CPUAccounting=yes
CPUQuota=200%

# I/O limits
IOAccounting=yes
IODeviceLatencyTargetSec=/dev/sda 100ms

Configure application-specific limits

Create targeted limits for high-resource applications like web servers and databases.

sudo mkdir -p /etc/systemd/system/nginx.service.d
sudo mkdir -p /etc/systemd/system/mysql.service.d

[Service]
# Nginx resource limits
LimitNOFILE=100000
LimitNPROC=32768
MemoryHigh=1G
MemoryMax=2G
TasksMax=4096

[Service]
# MySQL resource limits
LimitNOFILE=65536
LimitNPROC=16384
MemoryHigh=2G
MemoryMax=4G
TasksMax=8192
OOMScoreAdjust=-600

Enable systemd resource accounting

Enable comprehensive resource tracking and accounting in systemd for better limit enforcement.

# Add these lines to the [Manager] section
DefaultMemoryAccounting=yes
DefaultTasksAccounting=yes
DefaultIOAccounting=yes
DefaultCPUAccounting=yes
DefaultBlockIOAccounting=yes

Configure session limits per user

Set up individual user resource limits using systemd user configuration overrides.

sudo mkdir -p /etc/systemd/system/user@1000.service.d

[Service]
# Limits for user ID 1000
MemoryHigh=1G
MemoryMax=2G
TasksMax=4096
CPUQuota=150%

# File descriptor limits
LimitNOFILE=32768
LimitNPROC=8192

Apply configuration changes

Reload systemd configuration and restart user sessions to apply the new resource limits.

sudo systemctl daemon-reload
sudo systemctl daemon-reexec

# Restart user manager for current user
systemctl --user daemon-reload
sudo systemctl restart systemd-logind

Configure PAM session limits

Ensure PAM properly applies limits during user login sessions.

sudo cp /etc/pam.d/common-session /etc/pam.d/common-session.backup

# Add this line to enable PAM limits
session required pam_limits.so

# Existing PAM session configuration
session [default=1] pam_permit.so
session requisite pam_deny.so
session required pam_permit.so
session optional pam_systemd.so

Monitor user resource usage

Use systemd tools to monitor current resource consumption and limit enforcement.

# View user session resource usage
systemctl status user@$(id -u).service
systemd-cgtop

# Check specific user slice resources
systemctl status user-$(id -u).slice

# Monitor file descriptor usage
lsof | wc -l
cat /proc/sys/fs/file-nr

Debug limit violations

Identify and troubleshoot processes that exceed configured resource limits.

# Check system resource limits
cat /proc/sys/kernel/pid_max
cat /proc/sys/fs/file-max

# View per-process limits
cat /proc/self/limits

# Check systemd limit violations in logs
sudo journalctl -u user@$(id -u).service | grep -i limit
sudo journalctl | grep -i "memory limit\|task limit\|file limit"

Create monitoring script

Set up automated monitoring for resource limit violations and usage patterns.

#!/bin/bash
# Resource limit monitoring script

LOG_FILE="/var/log/resource-limits.log"
DATE=$(date '+%Y-%m-%d %H:%M:%S')

# Check open files
OPEN_FILES=$(lsof | wc -l)
FILE_LIMIT=$(ulimit -n)
FILE_USAGE=$(echo "scale=2; $OPEN_FILES * 100 / $FILE_LIMIT" | bc -l)

# Check processes
PROC_COUNT=$(ps aux | wc -l)
PROC_LIMIT=$(ulimit -u)
PROC_USAGE=$(echo "scale=2; $PROC_COUNT * 100 / $PROC_LIMIT" | bc -l)

# Log usage statistics
echo "$DATE - Files: ${OPEN_FILES}/${FILE_LIMIT} (${FILE_USAGE}%) Processes: ${PROC_COUNT}/${PROC_LIMIT} (${PROC_USAGE}%)" >> $LOG_FILE

# Alert if usage is high
if (( $(echo "$FILE_USAGE > 80" | bc -l) )); then
    echo "$DATE - WARNING: High file descriptor usage: $FILE_USAGE%" >> $LOG_FILE
fi

if (( $(echo "$PROC_USAGE > 80" | bc -l) )); then
    echo "$DATE - WARNING: High process usage: $PROC_USAGE%" >> $LOG_FILE
fi

sudo chmod +x /usr/local/bin/monitor-limits.sh

# Add to crontab for regular monitoring
echo "*/5 * * * * /usr/local/bin/monitor-limits.sh" | sudo crontab -