Configure Linux cgroups v2 for container resource management and isolation

Verify cgroups v2 support

Check if your system already has cgroups v2 enabled and mounted.

mount | grep cgroup
cat /proc/filesystems | grep cgroup

Look for cgroup2 in the output. If you see /sys/fs/cgroup mounted as cgroup2, you're ready to proceed.

Enable cgroups v2 at boot

If cgroups v2 isn't enabled, add the kernel parameter to force it on next boot.

sudo sed -i 's/GRUB_CMDLINE_LINUX_DEFAULT="/GRUB_CMDLINE_LINUX_DEFAULT="systemd.unified_cgroup_hierarchy=1 /' /etc/default/grub
sudo update-grub
sudo reboot

sudo grubby --update-kernel=ALL --args="systemd.unified_cgroup_hierarchy=1"
sudo reboot

Configure systemd for cgroups v2

Create a systemd drop-in directory and enable cgroup accounting for all services.

sudo mkdir -p /etc/systemd/system.conf.d
sudo tee /etc/systemd/system.conf.d/99-cgroups.conf > /dev/null << 'EOF'
[Manager]
DefaultCPUAccounting=yes
DefaultMemoryAccounting=yes
DefaultIOAccounting=yes
DefaultBlockIOAccounting=yes
DefaultTasksAccounting=yes
EOF

Install cgroup management tools

Install utilities for managing and monitoring cgroups.

sudo apt update
sudo apt install -y cgroup-tools libcgroup-dev systemd-cgroup-utils

sudo dnf install -y libcgroup-tools systemd

Create container resource limits

Set up a systemd slice for container workloads with specific resource constraints.

[Unit]
Description=Container Workloads Slice
Before=slices.target

[Slice]
CPUQuota=200%
MemoryMax=4G
MemorySwapMax=0
IOWeight=100
TasksMax=1024

Configure memory limits for services

Create a service configuration that uses the container slice and applies memory limits.

[Unit]
Description=Example Container Service
After=network.target

[Service]
Type=simple
Slice=container-workloads.slice
ExecStart=/usr/bin/podman run --name example nginx:alpine
Restart=always
RestartSec=10
MemoryMax=512M
CPUQuota=50%

[Install]
WantedBy=multi-user.target

Set up CPU limits with systemd

Configure CPU quotas and weights for different service categories.

[Unit]
Description=High Priority Services
Before=slices.target

[Slice]
CPUWeight=200
CPUQuota=150%
MemoryMax=2G

[Unit]
Description=Background Services
Before=slices.target

[Slice]
CPUWeight=50
CPUQuota=25%
MemoryMax=1G

Enable and reload systemd configuration

Apply the new slice configurations and reload systemd.

sudo systemctl daemon-reload
sudo systemctl enable container-workloads.slice
sudo systemctl start container-workloads.slice

Configure I/O limits

Set up block I/O limits to prevent storage bottlenecks.

[Unit]
Description=I/O Limited Services
Before=slices.target

[Slice]
IOWeight=50
IOReadBandwidthMax=/dev/sda 50M
IOWriteBandwidthMax=/dev/sda 25M

Set up cgroup monitoring script

Create a monitoring script to track resource usage across cgroups.

#!/bin/bash

echo "=== Cgroup Resource Usage ==="
echo "Date: $(date)"
echo

Memory usage by slice
echo "Memory Usage by Slice:"
for slice in $(systemctl list-units --type=slice --state=active --no-legend | awk '{print $1}'); do
    if [[ -f "/sys/fs/cgroup/system.slice/${slice}/memory.current" ]]; then
        current=$(cat "/sys/fs/cgroup/system.slice/${slice}/memory.current")
        max=$(cat "/sys/fs/cgroup/system.slice/${slice}/memory.max" 2>/dev/null || echo "unlimited")
        echo "  ${slice}: $(numfmt --to=iec ${current}) / ${max}"
    fi
done

echo
echo "CPU Usage by Slice:"
for slice in $(systemctl list-units --type=slice --state=active --no-legend | awk '{print $1}'); do
    if [[ -f "/sys/fs/cgroup/system.slice/${slice}/cpu.stat" ]]; then
        usage=$(grep usage_usec "/sys/fs/cgroup/system.slice/${slice}/cpu.stat" | awk '{print $2}')
        echo "  ${slice}: ${usage} microseconds"
    fi
done

sudo chmod 755 /usr/local/bin/cgroup-monitor.sh

Create systemd timer for monitoring

Set up automated monitoring with systemd timers.

[Unit]
Description=Cgroup Resource Monitor

[Service]
Type=oneshot
ExecStart=/usr/local/bin/cgroup-monitor.sh
User=root

[Unit]
Description=Run Cgroup Monitor every 5 minutes
Requires=cgroup-monitor.service

[Timer]
OnCalendar=*:0/5
Persistent=true

[Install]
WantedBy=timers.target

sudo systemctl enable --now cgroup-monitor.timer

Configure container runtime integration

For Docker or Podman integration, ensure they use systemd cgroups driver.

{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "cgroup-parent": "system.slice"
}

Apply limits to running containers

Use systemctl to apply resource limits to container services.

# Set memory limit for existing service
sudo systemctl set-property example-container.service MemoryMax=256M

Set CPU quota (50% of one core)
sudo systemctl set-property example-container.service CPUQuota=50%

Apply I/O weight
sudo systemctl set-property example-container.service IOWeight=100

Create custom cgroup hierarchy

Set up a custom cgroup for manual resource management.

# Create custom cgroup
sudo mkdir -p /sys/fs/cgroup/custom-workload

Set memory limit (1GB)
echo "1073741824" | sudo tee /sys/fs/cgroup/custom-workload/memory.max

Set CPU weight
echo "200" | sudo tee /sys/fs/cgroup/custom-workload/cpu.weight

Move process to cgroup
echo $$ | sudo tee /sys/fs/cgroup/custom-workload/cgroup.procs

Set up performance alerts

Create a script to alert when resource limits are approached.

#!/bin/bash

THRESHOLD=80  # Alert at 80% usage

for slice in $(systemctl list-units --type=slice --state=active --no-legend | awk '{print $1}'); do
    cgroup_path="/sys/fs/cgroup/system.slice/${slice}"
    
    if [[ -f "${cgroup_path}/memory.current" ]] && [[ -f "${cgroup_path}/memory.max" ]]; then
        current=$(cat "${cgroup_path}/memory.current")
        max=$(cat "${cgroup_path}/memory.max")
        
        if [[ "$max" != "max" ]] && [[ $current -gt 0 ]] && [[ $max -gt 0 ]]; then
            usage_percent=$((current * 100 / max))
            
            if [[ $usage_percent -gt $THRESHOLD ]]; then
                echo "WARNING: ${slice} memory usage at ${usage_percent}%"
                logger "cgroup-alert: ${slice} memory usage at ${usage_percent}%"
            fi
        fi
    fi
done

sudo chmod 755 /usr/local/bin/cgroup-alerts.sh

Create pressure monitoring

Monitor memory and CPU pressure using PSI (Pressure Stall Information).

# Check memory pressure
cat /proc/pressure/memory

Check CPU pressure
cat /proc/pressure/cpu

Check I/O pressure
cat /proc/pressure/io