Configure advanced iptables QoS with fwmark and multiple interfaces

Advanced 45 min Jun 12, 2026
Ubuntu 24.04 Debian 12 AlmaLinux 9 Rocky Linux 9

Set up traffic shaping with iptables packet marking, HTB queueing discipline, and multi-interface QoS policies for bandwidth management and network performance optimization.

Prerequisites

  • Root access to Linux server
  • Multiple network interfaces
  • Basic understanding of iptables and networking

What this solves

This tutorial shows you how to configure Quality of Service (QoS) with iptables packet marking and traffic control (tc) to prioritize network traffic across multiple interfaces. You'll use fwmark to tag packets in iptables, then apply Hierarchical Token Bucket (HTB) queueing to shape bandwidth based on those marks. This is essential when you need to guarantee bandwidth for critical services while limiting less important traffic.

Step-by-step configuration

Install traffic control tools

Install the necessary packages for advanced traffic shaping and QoS management.

sudo apt update
sudo apt install -y iproute2 iptables-persistent netfilter-persistent
sudo dnf install -y iproute-tc iptables-services
sudo systemctl enable iptables

Configure network interfaces

Set up your network interfaces with proper IP addresses. We'll use eth0 for internal traffic and eth1 for external connections.

sudo ip addr add 192.168.1.10/24 dev eth0
sudo ip addr add 203.0.113.10/24 dev eth1
sudo ip link set eth0 up
sudo ip link set eth1 up

Create iptables mangle rules with fwmark

Set up packet marking rules to classify traffic by service type, source, and destination. These marks will be used by tc for traffic shaping.

# Clear existing mangle rules
sudo iptables -t mangle -F
sudo iptables -t mangle -X

Mark SSH traffic as high priority (mark 1)

sudo iptables -t mangle -A OUTPUT -p tcp --dport 22 -j MARK --set-mark 1 sudo iptables -t mangle -A INPUT -p tcp --sport 22 -j MARK --set-mark 1

Mark HTTP/HTTPS traffic as medium priority (mark 2)

sudo iptables -t mangle -A OUTPUT -p tcp --dport 80 -j MARK --set-mark 2 sudo iptables -t mangle -A OUTPUT -p tcp --dport 443 -j MARK --set-mark 2 sudo iptables -t mangle -A INPUT -p tcp --sport 80 -j MARK --set-mark 2 sudo iptables -t mangle -A INPUT -p tcp --sport 443 -j MARK --set-mark 2

Mark database traffic as high priority (mark 3)

sudo iptables -t mangle -A OUTPUT -p tcp --dport 3306 -j MARK --set-mark 3 sudo iptables -t mangle -A OUTPUT -p tcp --dport 5432 -j MARK --set-mark 3 sudo iptables -t mangle -A INPUT -p tcp --sport 3306 -j MARK --set-mark 3 sudo iptables -t mangle -A INPUT -p tcp --sport 5432 -j MARK --set-mark 3

Mark bulk transfer traffic as low priority (mark 4)

sudo iptables -t mangle -A OUTPUT -p tcp --dport 21 -j MARK --set-mark 4 sudo iptables -t mangle -A OUTPUT -p tcp --dport 20 -j MARK --set-mark 4 sudo iptables -t mangle -A INPUT -p tcp --sport 21 -j MARK --set-mark 4 sudo iptables -t mangle -A INPUT -p tcp --sport 20 -j MARK --set-mark 4

Set up HTB root qdisc on primary interface

Configure the Hierarchical Token Bucket queueing discipline on your primary interface (eth0) with a total bandwidth limit.

# Remove existing qdisc
sudo tc qdisc del dev eth0 root 2>/dev/null || true

Create HTB root qdisc with 100Mbit total bandwidth

sudo tc qdisc add dev eth0 root handle 1: htb default 40

Create root class with total bandwidth

sudo tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit

Configure HTB classes for different priorities

Create HTB classes that correspond to your iptables marks, with guaranteed and maximum bandwidth allocations.

# High priority class for SSH and database (mark 1 and 3) - 40% guaranteed, can use up to 80%
sudo tc class add dev eth0 parent 1:1 classid 1:10 htb rate 40mbit ceil 80mbit prio 1

Medium priority class for HTTP/HTTPS (mark 2) - 30% guaranteed, can use up to 60%

sudo tc class add dev eth0 parent 1:1 classid 1:20 htb rate 30mbit ceil 60mbit prio 2

Low priority class for bulk transfers (mark 4) - 10% guaranteed, can use up to 40%

sudo tc class add dev eth0 parent 1:1 classid 1:30 htb rate 10mbit ceil 40mbit prio 3

Default class for unmarked traffic - 20% guaranteed, can use up to 50%

sudo tc class add dev eth0 parent 1:1 classid 1:40 htb rate 20mbit ceil 50mbit prio 4

Add queueing disciplines to leaf classes

Attach Stochastic Fair Queueing (SFQ) to each HTB class to ensure fair distribution among flows within each priority level.

# Add SFQ to high priority class
sudo tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10

Add SFQ to medium priority class

sudo tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10

Add SFQ to low priority class

sudo tc qdisc add dev eth0 parent 1:30 handle 30: sfq perturb 10

Add SFQ to default class

sudo tc qdisc add dev eth0 parent 1:40 handle 40: sfq perturb 10

Create tc filters based on fwmark

Set up traffic control filters that direct packets to appropriate HTB classes based on their iptables fwmark values.

# Filter for SSH traffic (mark 1) -> high priority class
sudo tc filter add dev eth0 protocol ip parent 1:0 prio 1 handle 1 fw flowid 1:10

Filter for HTTP/HTTPS traffic (mark 2) -> medium priority class

sudo tc filter add dev eth0 protocol ip parent 1:0 prio 2 handle 2 fw flowid 1:20

Filter for database traffic (mark 3) -> high priority class

sudo tc filter add dev eth0 protocol ip parent 1:0 prio 1 handle 3 fw flowid 1:10

Filter for bulk transfer traffic (mark 4) -> low priority class

sudo tc filter add dev eth0 protocol ip parent 1:0 prio 3 handle 4 fw flowid 1:30

Configure QoS on secondary interface

Set up similar traffic shaping on your secondary interface (eth1) with different bandwidth allocations.

# Remove existing qdisc from eth1
sudo tc qdisc del dev eth1 root 2>/dev/null || true

Create HTB root qdisc with 50Mbit total bandwidth for external interface

sudo tc qdisc add dev eth1 root handle 2: htb default 240

Create root class

sudo tc class add dev eth1 parent 2: classid 2:1 htb rate 50mbit

High priority class - 20Mbit guaranteed, 40Mbit max

sudo tc class add dev eth1 parent 2:1 classid 2:10 htb rate 20mbit ceil 40mbit prio 1

Medium priority class - 15Mbit guaranteed, 30Mbit max

sudo tc class add dev eth1 parent 2:1 classid 2:20 htb rate 15mbit ceil 30mbit prio 2

Low priority class - 5Mbit guaranteed, 20Mbit max

sudo tc class add dev eth1 parent 2:1 classid 2:30 htb rate 5mbit ceil 20mbit prio 3

Default class - 10Mbit guaranteed, 25Mbit max

sudo tc class add dev eth1 parent 2:1 classid 2:40 htb rate 10mbit ceil 25mbit prio 4

Add SFQ and filters to secondary interface

Complete the QoS setup on eth1 with fair queueing and fwmark-based filters.

# Add SFQ to all classes on eth1
sudo tc qdisc add dev eth1 parent 2:10 handle 210: sfq perturb 10
sudo tc qdisc add dev eth1 parent 2:20 handle 220: sfq perturb 10
sudo tc qdisc add dev eth1 parent 2:30 handle 230: sfq perturb 10
sudo tc qdisc add dev eth1 parent 2:40 handle 240: sfq perturb 10

Add filters based on fwmark

sudo tc filter add dev eth1 protocol ip parent 2:0 prio 1 handle 1 fw flowid 2:10 sudo tc filter add dev eth1 protocol ip parent 2:0 prio 2 handle 2 fw flowid 2:20 sudo tc filter add dev eth1 protocol ip parent 2:0 prio 1 handle 3 fw flowid 2:10 sudo tc filter add dev eth1 protocol ip parent 2:0 prio 3 handle 4 fw flowid 2:30

Create advanced marking rules for subnet-based QoS

Add more sophisticated iptables rules that classify traffic based on source and destination subnets for better traffic management.

# Mark internal management traffic as high priority
sudo iptables -t mangle -A OUTPUT -s 192.168.1.0/24 -d 192.168.1.0/24 -j MARK --set-mark 1
sudo iptables -t mangle -A INPUT -s 192.168.1.0/24 -d 192.168.1.0/24 -j MARK --set-mark 1

Mark traffic to/from DMZ as medium priority

sudo iptables -t mangle -A OUTPUT -s 192.168.2.0/24 -j MARK --set-mark 2 sudo iptables -t mangle -A OUTPUT -d 192.168.2.0/24 -j MARK --set-mark 2 sudo iptables -t mangle -A INPUT -s 192.168.2.0/24 -j MARK --set-mark 2

Mark guest network traffic as low priority

sudo iptables -t mangle -A OUTPUT -s 192.168.3.0/24 -j MARK --set-mark 4 sudo iptables -t mangle -A OUTPUT -d 192.168.3.0/24 -j MARK --set-mark 4 sudo iptables -t mangle -A INPUT -s 192.168.3.0/24 -j MARK --set-mark 4

Mark traffic based on packet size for bulk transfers

sudo iptables -t mangle -A OUTPUT -m length --length 1000:65535 -j MARK --set-mark 4 sudo iptables -t mangle -A INPUT -m length --length 1000:65535 -j MARK --set-mark 4

Save configuration for persistence

Save your iptables rules and create systemd service for tc rules to ensure they persist after reboot.

sudo netfilter-persistent save
sudo systemctl enable netfilter-persistent
sudo iptables-save > /etc/sysconfig/iptables
sudo systemctl enable iptables
[Unit]
Description=QoS Traffic Control Setup
After=network.target

[Service]
Type=oneshot
ExecStart=/usr/local/bin/setup-qos.sh
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Create QoS setup script

Create a script to automatically configure tc rules on system startup.

#!/bin/bash

Setup QoS on eth0

tc qdisc del dev eth0 root 2>/dev/null || true tc qdisc add dev eth0 root handle 1: htb default 40 tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit tc class add dev eth0 parent 1:1 classid 1:10 htb rate 40mbit ceil 80mbit prio 1 tc class add dev eth0 parent 1:1 classid 1:20 htb rate 30mbit ceil 60mbit prio 2 tc class add dev eth0 parent 1:1 classid 1:30 htb rate 10mbit ceil 40mbit prio 3 tc class add dev eth0 parent 1:1 classid 1:40 htb rate 20mbit ceil 50mbit prio 4

Add SFQ

tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10 tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10 tc qdisc add dev eth0 parent 1:30 handle 30: sfq perturb 10 tc qdisc add dev eth0 parent 1:40 handle 40: sfq perturb 10

Add filters

tc filter add dev eth0 protocol ip parent 1:0 prio 1 handle 1 fw flowid 1:10 tc filter add dev eth0 protocol ip parent 1:0 prio 2 handle 2 fw flowid 1:20 tc filter add dev eth0 protocol ip parent 1:0 prio 1 handle 3 fw flowid 1:10 tc filter add dev eth0 protocol ip parent 1:0 prio 3 handle 4 fw flowid 1:30

Setup QoS on eth1

tc qdisc del dev eth1 root 2>/dev/null || true tc qdisc add dev eth1 root handle 2: htb default 240 tc class add dev eth1 parent 2: classid 2:1 htb rate 50mbit tc class add dev eth1 parent 2:1 classid 2:10 htb rate 20mbit ceil 40mbit prio 1 tc class add dev eth1 parent 2:1 classid 2:20 htb rate 15mbit ceil 30mbit prio 2 tc class add dev eth1 parent 2:1 classid 2:30 htb rate 5mbit ceil 20mbit prio 3 tc class add dev eth1 parent 2:1 classid 2:40 htb rate 10mbit ceil 25mbit prio 4

Add SFQ

tc qdisc add dev eth1 parent 2:10 handle 210: sfq perturb 10 tc qdisc add dev eth1 parent 2:20 handle 220: sfq perturb 10 tc qdisc add dev eth1 parent 2:30 handle 230: sfq perturb 10 tc qdisc add dev eth1 parent 2:40 handle 240: sfq perturb 10

Add filters

tc filter add dev eth1 protocol ip parent 2:0 prio 1 handle 1 fw flowid 2:10 tc filter add dev eth1 protocol ip parent 2:0 prio 2 handle 2 fw flowid 2:20 tc filter add dev eth1 protocol ip parent 2:0 prio 1 handle 3 fw flowid 2:10 tc filter add dev eth1 protocol ip parent 2:0 prio 3 handle 4 fw flowid 2:30
sudo chmod +x /usr/local/bin/setup-qos.sh
sudo systemctl enable qos-setup.service
sudo systemctl start qos-setup.service

Monitor QoS performance

Set up monitoring to track QoS effectiveness and bandwidth utilization across your interfaces.

#!/bin/bash

echo "=== QoS Statistics for eth0 ==="
tc -s class show dev eth0

echo -e "\n=== QoS Statistics for eth1 ==="
tc -s class show dev eth1

echo -e "\n=== iptables Packet Counters ==="
iptables -t mangle -L -v -n

echo -e "\n=== Interface Statistics ==="
ip -s link show eth0
ip -s link show eth1
sudo chmod +x /usr/local/bin/qos-monitor.sh

Verify your setup

Check that your QoS configuration is working correctly and traffic is being classified properly.

# View current tc configuration
sudo tc qdisc show
sudo tc class show dev eth0
sudo tc filter show dev eth0

Check iptables mangle rules

sudo iptables -t mangle -L -v -n

Monitor real-time traffic classification

sudo /usr/local/bin/qos-monitor.sh

Test bandwidth limits with iperf3 (if available)

iperf3 -c example.com -p 80 -t 10

You can also monitor QoS effectiveness by examining the packet and byte counters:

# Watch real-time statistics
watch -n 2 'tc -s class show dev eth0'

Check specific class statistics

tc -s class show dev eth0 classid 1:10

Common issues

Symptom Cause Fix
Traffic not being shaped fwmark not set or filters missing Check iptables rules with iptables -t mangle -L -v
HTB classes not working Incorrect parent-child relationships Verify class hierarchy with tc class show dev ethX
No bandwidth limiting Default class not configured Ensure HTB has default parameter set
Filters not matching Wrong handle values in filters Match filter handles to iptables marks exactly
Configuration lost after reboot Rules not persisted Enable systemd service: systemctl enable qos-setup
High latency on priority traffic SFQ not attached to classes Add SFQ qdisc to each HTB leaf class

Next steps

Running this in production?

Need this managed? Running QoS at scale adds complexity: capacity planning, performance tuning, and monitoring across multiple interfaces. See how we run infrastructure like this for European teams who need guaranteed network performance.

Automated install script

Run this to automate the entire setup

Need help?

Don't want to manage this yourself?

We handle private cloud infrastructure for businesses that depend on uptime. From initial setup to ongoing operations.