Configure enterprise-grade Quality of Service policies with DSCP marking, traffic shaping using tc and HTB, and seamless integration with Cisco equipment through FRRouting BGP and OSPF routing protocols for comprehensive network bandwidth management.
Prerequisites
- Root access to Linux servers
- Basic networking knowledge
- Existing Cisco infrastructure
- Multiple network interfaces
What this solves
Enterprise networks require sophisticated Quality of Service (QoS) policies to prioritize critical traffic and maintain performance across diverse applications. This tutorial configures advanced QoS with DSCP marking, hierarchical traffic shaping, and seamless integration with Cisco infrastructure through FRRouting's BGP and OSPF implementations.
Prerequisites
You need root access to your Linux servers, basic understanding of networking concepts, and existing network infrastructure with Cisco equipment. Your servers should have multiple network interfaces for testing traffic separation.
Step-by-step configuration
Install FRRouting and traffic control tools
Install FRRouting for Cisco protocol compatibility and Linux traffic control utilities for QoS implementation.
curl -s https://deb.frrouting.org/frr/keys.asc | sudo apt-key add -
echo 'deb https://deb.frrouting.org/frr jammy frr-stable' | sudo tee /etc/apt/sources.list.d/frr.list
sudo apt update
sudo apt install -y frr frr-pythontools iproute2 tc iptables-persistent
Enable FRRouting daemons
Configure FRRouting to enable BGP and OSPF daemons for Cisco integration.
bgpd=yes
ospfd=yes
zebra=yes
vtysh_enable=yes
zebra_options=" -A 127.0.0.1 -s 90000000"
bgpd_options=" -A 127.0.0.1"
ospfd_options=" -A 127.0.0.1"
Configure FRRouting BGP for Cisco integration
Set up BGP peering with Cisco equipment and implement route policies for QoS integration.
frr version 8.4
frr defaults traditional
hostname frrouting-qos
log syslog informational
ipv6 forwarding
!
interface eth0
description WAN-Interface
ip address 203.0.113.10/24
!
interface eth1
description LAN-Interface
ip address 192.168.1.1/24
!
router bgp 65001
bgp router-id 203.0.113.10
neighbor 203.0.113.1 remote-as 65000
neighbor 203.0.113.1 description Cisco-BGP-Peer
!
address-family ipv4 unicast
neighbor 203.0.113.1 activate
neighbor 203.0.113.1 route-map QOS-OUT out
neighbor 203.0.113.1 route-map QOS-IN in
exit-address-family
!
router ospf
ospf router-id 192.168.1.1
network 192.168.1.0/24 area 0
network 203.0.113.0/24 area 0
!
route-map QOS-OUT permit 10
set community 65001:100
set ip next-hop unchanged
!
route-map QOS-IN permit 10
match community PRIORITY-TRAFFIC
set local-preference 200
!
ip community-list standard PRIORITY-TRAFFIC permit 65000:100
!
line vty
!
Start and enable FRRouting services
Enable FRRouting services and verify BGP neighbor establishment.
sudo systemctl enable --now frr
sudo systemctl status frr
sudo vtysh -c "show ip bgp summary"
sudo vtysh -c "show ip ospf neighbor"
Configure DSCP marking with iptables
Implement DSCP marking for traffic classification and QoS policy enforcement.
# DSCP marking for enterprise QoS
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
Voice traffic - EF (Expedited Forwarding) DSCP 46
-A POSTROUTING -p udp --dport 5060:5090 -j DSCP --set-dscp 46
-A POSTROUTING -p tcp --dport 5060:5090 -j DSCP --set-dscp 46
-A POSTROUTING -p udp --dport 16384:32767 -j DSCP --set-dscp 46
Video conferencing - AF41 DSCP 34
-A POSTROUTING -p tcp --dport 443 -m string --string "zoom" --algo bm -j DSCP --set-dscp 34
-A POSTROUTING -p udp --dport 8801:8810 -j DSCP --set-dscp 34
-A POSTROUTING -p tcp --dport 1935 -j DSCP --set-dscp 34
Database traffic - AF31 DSCP 26
-A POSTROUTING -p tcp --dport 3306 -j DSCP --set-dscp 26
-A POSTROUTING -p tcp --dport 5432 -j DSCP --set-dscp 26
-A POSTROUTING -p tcp --dport 1433 -j DSCP --set-dscp 26
Management traffic - CS6 DSCP 48
-A POSTROUTING -p tcp --dport 22 -j DSCP --set-dscp 48
-A POSTROUTING -p tcp --dport 161:162 -j DSCP --set-dscp 48
-A POSTROUTING -p tcp --dport 179 -j DSCP --set-dscp 48
Web traffic - AF21 DSCP 18
-A POSTROUTING -p tcp --dport 80 -j DSCP --set-dscp 18
-A POSTROUTING -p tcp --dport 443 -j DSCP --set-dscp 18
Best effort - default DSCP 0
-A POSTROUTING -j DSCP --set-dscp 0
COMMIT
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
COMMIT
Apply iptables rules
Load the DSCP marking rules and make them persistent across reboots.
sudo iptables-restore < /etc/iptables/rules.v4
sudo systemctl enable netfilter-persistent
sudo netfilter-persistent save
Configure HTB traffic shaping
Implement Hierarchical Token Bucket (HTB) for bandwidth management and traffic prioritization.
#!/bin/bash
Interface configuration
INTERFACE="eth0"
BANDWIDTH="100mbit"
Clear existing qdisc
tc qdisc del dev $INTERFACE root 2>/dev/null
Create root HTB qdisc
tc qdisc add dev $INTERFACE root handle 1: htb default 60
Create root class with total bandwidth
tc class add dev $INTERFACE parent 1: classid 1:1 htb rate $BANDWIDTH
Voice traffic - 20% guaranteed, 50% max
tc class add dev $INTERFACE parent 1:1 classid 1:10 htb rate 20mbit ceil 50mbit prio 1
tc filter add dev $INTERFACE parent 1: protocol ip prio 1 u32 match ip tos 0xb8 0xfc flowid 1:10
Video traffic - 30% guaranteed, 60% max
tc class add dev $INTERFACE parent 1:1 classid 1:20 htb rate 30mbit ceil 60mbit prio 2
tc filter add dev $INTERFACE parent 1: protocol ip prio 2 u32 match ip tos 0x88 0xfc flowid 1:20
Database traffic - 15% guaranteed, 40% max
tc class add dev $INTERFACE parent 1:1 classid 1:30 htb rate 15mbit ceil 40mbit prio 3
tc filter add dev $INTERFACE parent 1: protocol ip prio 3 u32 match ip tos 0x68 0xfc flowid 1:30
Management traffic - 10% guaranteed, 30% max
tc class add dev $INTERFACE parent 1:1 classid 1:40 htb rate 10mbit ceil 30mbit prio 4
tc filter add dev $INTERFACE parent 1: protocol ip prio 4 u32 match ip tos 0xc0 0xfc flowid 1:40
Web traffic - 15% guaranteed, 50% max
tc class add dev $INTERFACE parent 1:1 classid 1:50 htb rate 15mbit ceil 50mbit prio 5
tc filter add dev $INTERFACE parent 1: protocol ip prio 5 u32 match ip tos 0x48 0xfc flowid 1:50
Best effort - 10% guaranteed, remaining available
tc class add dev $INTERFACE parent 1:1 classid 1:60 htb rate 10mbit ceil $BANDWIDTH prio 6
Add fair queuing to each class for better distribution
tc qdisc add dev $INTERFACE parent 1:10 handle 10: sfq perturb 10
tc qdisc add dev $INTERFACE parent 1:20 handle 20: sfq perturb 10
tc qdisc add dev $INTERFACE parent 1:30 handle 30: sfq perturb 10
tc qdisc add dev $INTERFACE parent 1:40 handle 40: sfq perturb 10
tc qdisc add dev $INTERFACE parent 1:50 handle 50: sfq perturb 10
tc qdisc add dev $INTERFACE parent 1:60 handle 60: sfq perturb 10
echo "QoS configuration applied successfully"
tc class show dev $INTERFACE
Make QoS script executable and apply configuration
Set proper permissions and apply the traffic shaping configuration.
sudo chmod +x /usr/local/bin/setup-qos.sh
sudo /usr/local/bin/setup-qos.sh
Create systemd service for persistent QoS
Ensure QoS configuration persists across system reboots with a systemd service.
[Unit]
Description=Network QoS Configuration
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/local/bin/setup-qos.sh
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target
Enable QoS service and configure bandwidth monitoring
Enable the QoS service and set up monitoring for traffic analysis.
sudo systemctl daemon-reload
sudo systemctl enable --now network-qos
sudo systemctl status network-qos
Configure advanced OSPF for Cisco integration
Set up OSPF areas and route redistribution for enterprise network integration. For complex QoS routing, you might also want to explore OSPF multi-area design with FRRouting for larger deployments.
sudo vtysh
configure terminal
router ospf
area 0 authentication message-digest
area 1 stub
network 10.0.0.0/8 area 1
redistribute connected metric 20 metric-type 2
redistribute static metric 10 metric-type 1
passive-interface default
no passive-interface eth0
no passive-interface eth1
timers throttle spf 200 1000 5000
max-metric router-lsa on-startup 60
!
interface eth0
ip ospf message-digest-key 1 md5 cisco123
ip ospf priority 100
ip ospf cost 10
!
interface eth1
ip ospf message-digest-key 1 md5 cisco123
ip ospf priority 200
ip ospf cost 5
!
write memory
exit
Configure traffic monitoring and QoS statistics
Set up monitoring scripts to track QoS effectiveness and bandwidth utilization.
#!/bin/bash
INTERFACE="eth0"
LOGFILE="/var/log/qos-stats.log"
echo "$(date): QoS Statistics for $INTERFACE" >> $LOGFILE
echo "===========================================" >> $LOGFILE
Display class statistics
tc -s class show dev $INTERFACE >> $LOGFILE
Display current DSCP markings
echo "\nDSCP Statistics:" >> $LOGFILE
iptables -t mangle -L POSTROUTING -v -n | grep DSCP >> $LOGFILE
BGP neighbor status
echo "\nBGP Neighbor Status:" >> $LOGFILE
vtysh -c "show ip bgp summary" >> $LOGFILE
OSPF neighbor status
echo "\nOSPF Neighbor Status:" >> $LOGFILE
vtysh -c "show ip ospf neighbor" >> $LOGFILE
echo "" >> $LOGFILE
Set up automated QoS monitoring
Create a cron job for regular QoS monitoring and performance tracking.
sudo chmod +x /usr/local/bin/qos-monitor.sh
(crontab -l 2>/dev/null; echo "/5 * /usr/local/bin/qos-monitor.sh") | sudo crontab -
Configure Cisco integration verification
Verify BGP and OSPF peering with Cisco equipment and QoS policy synchronization.
# Verify BGP neighbor establishment
sudo vtysh -c "show ip bgp neighbor 203.0.113.1"
Check OSPF database
sudo vtysh -c "show ip ospf database"
Verify route redistribution
sudo vtysh -c "show ip route ospf"
sudo vtysh -c "show ip route bgp"
Verify your setup
Test the QoS configuration and verify Cisco integration with comprehensive checks.
# Check QoS class configuration
tc class show dev eth0
Verify DSCP marking rules
iptables -t mangle -L POSTROUTING -v -n
Test BGP connectivity
sudo vtysh -c "show ip bgp summary"
sudo vtysh -c "show ip bgp neighbors"
Check OSPF neighbors
sudo vtysh -c "show ip ospf neighbor"
sudo vtysh -c "show ip ospf interface"
Monitor traffic classification
watch -n 2 'tc -s class show dev eth0'
Test DSCP marking with ping
ping -Q 184 203.0.113.1 # Test AF21 marking
ping -Q 136 203.0.113.1 # Test AF31 marking
Check routing table
ip route show
sudo vtysh -c "show ip route"
Performance tuning
Optimize QoS performance for high-throughput enterprise environments.
| Parameter | Default | Optimized | Purpose |
|---|---|---|---|
| HTB quantum | 1500 | 8000 | Better packet scheduling |
| SFQ perturb | 10 | 5 | More frequent hash regeneration |
| OSPF SPF throttle | 5000 | 1000 | Faster convergence |
| BGP keepalive | 60 | 30 | Faster failure detection |
Apply performance optimizations:
# Optimize network buffers
echo 'net.core.rmem_max = 16777216' | sudo tee -a /etc/sysctl.conf
echo 'net.core.wmem_max = 16777216' | sudo tee -a /etc/sysctl.conf
echo 'net.ipv4.tcp_rmem = 4096 65536 16777216' | sudo tee -a /etc/sysctl.conf
echo 'net.ipv4.tcp_wmem = 4096 65536 16777216' | sudo tee -a /etc/sysctl.conf
Apply changes
sudo sysctl -p
Optimize BGP timers
sudo vtysh -c "configure terminal"
sudo vtysh -c "router bgp 65001"
sudo vtysh -c "neighbor 203.0.113.1 timers 30 90"
sudo vtysh -c "write memory"
Integration with advanced iptables QoS
For more sophisticated packet marking and filtering rules, consider implementing advanced iptables QoS with fwmark to complement your FRRouting setup.
Common issues
| Symptom | Cause | Fix |
|---|---|---|
| BGP neighbor won't establish | Firewall blocking port 179 | sudo iptables -A INPUT -p tcp --dport 179 -j ACCEPT |
| OSPF adjacency fails | Area ID mismatch | Verify area configuration with show ip ospf interface |
| Traffic not shaped correctly | Wrong interface specified | Check with ip link show and update script |
| DSCP marking not working | iptables rules not applied | sudo iptables-save | grep DSCP to verify |
| QoS classes empty | Filter rules not matching | Use tcpdump -i eth0 -v to check DSCP values |
| Route redistribution fails | Metric conflicts | Adjust redistribution metrics in OSPF config |
Security considerations
Implement security hardening for QoS infrastructure:
# Protect against QoS abuse
iptables -A INPUT -p tcp --dport 179 -m limit --limit 10/minute -j ACCEPT
iptables -A INPUT -p tcp --dport 179 -j DROP
Monitor for DSCP abuse
echo '/10 * /usr/bin/iptables -t mangle -Z' | sudo crontab -
BGP authentication
sudo vtysh -c "configure terminal"
sudo vtysh -c "router bgp 65001"
sudo vtysh -c "neighbor 203.0.113.1 password CiscoQoSAuth2024"
sudo vtysh -c "write memory"
Next steps
- Implement OSPF multi-area design with FRRouting and advanced routing policies
- Configure advanced iptables QoS with fwmark and multiple interfaces
- Configure enterprise BGP route filtering with FRRouting communities
- Implement network automation with FRRouting SNMP monitoring
- Setup enterprise network redundancy with VRRP failover
Running this in production?
Automated install script
Run this to automate the entire setup
#!/usr/bin/env bash
set -euo pipefail
# Enterprise QoS with FRRouting and Cisco Integration Install Script
# Production-ready installation for Linux systems
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
# Configuration variables
WAN_IP="${1:-203.0.113.10/24}"
LAN_IP="${2:-192.168.1.1/24}"
BGP_AS="${3:-65001}"
CISCO_PEER="${4:-203.0.113.1}"
CISCO_AS="${5:-65000}"
usage() {
echo "Usage: $0 [WAN_IP] [LAN_IP] [BGP_AS] [CISCO_PEER] [CISCO_AS]"
echo "Example: $0 203.0.113.10/24 192.168.1.1/24 65001 203.0.113.1 65000"
exit 1
}
log() {
echo -e "${GREEN}[$(date +'%H:%M:%S')]${NC} $1"
}
warn() {
echo -e "${YELLOW}[$(date +'%H:%M:%S')] WARNING:${NC} $1"
}
error() {
echo -e "${RED}[$(date +'%H:%M:%S')] ERROR:${NC} $1" >&2
exit 1
}
cleanup() {
warn "Installation failed. Cleaning up..."
systemctl stop frr 2>/dev/null || true
systemctl disable frr 2>/dev/null || true
}
trap cleanup ERR
# Check if running as root
if [[ $EUID -ne 0 ]]; then
error "This script must be run as root"
fi
# Detect distribution
if [ ! -f /etc/os-release ]; then
error "/etc/os-release not found. Cannot detect distribution."
fi
. /etc/os-release
case "$ID" in
ubuntu|debian)
PKG_MGR="apt"
PKG_INSTALL="apt install -y"
PKG_UPDATE="apt update"
IPTABLES_SERVICE="netfilter-persistent"
;;
almalinux|rocky|centos|rhel|ol|fedora)
PKG_MGR="dnf"
PKG_INSTALL="dnf install -y"
PKG_UPDATE="dnf update -y"
IPTABLES_SERVICE="iptables"
if command -v yum >/dev/null 2>&1 && ! command -v dnf >/dev/null 2>&1; then
PKG_MGR="yum"
PKG_INSTALL="yum install -y"
PKG_UPDATE="yum update -y"
fi
;;
amzn)
PKG_MGR="yum"
PKG_INSTALL="yum install -y"
PKG_UPDATE="yum update -y"
IPTABLES_SERVICE="iptables"
;;
*)
error "Unsupported distribution: $ID"
;;
esac
log "[1/8] Installing FRRouting and traffic control tools..."
# Install based on distribution
if [[ "$ID" == "ubuntu" || "$ID" == "debian" ]]; then
curl -fsSL https://deb.frrouting.org/frr/keys.asc | gpg --dearmor -o /usr/share/keyrings/frr-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/frr-archive-keyring.gpg] https://deb.frrouting.org/frr $(lsb_release -cs) frr-stable" > /etc/apt/sources.list.d/frr.list
$PKG_UPDATE
$PKG_INSTALL frr frr-pythontools iproute2 tc iptables-persistent
else
$PKG_INSTALL epel-release
$PKG_INSTALL https://rpm.frrouting.org/repo/rpm-repo-0-*.noarch.rpm || true
$PKG_INSTALL frr iproute tc iptables-services
fi
log "[2/8] Configuring FRRouting daemons..."
# Configure FRRouting daemons
cat > /etc/frr/daemons << 'EOF'
bgpd=yes
ospfd=yes
zebra=yes
vtysh_enable=yes
zebra_options=" -A 127.0.0.1 -s 90000000"
bgpd_options=" -A 127.0.0.1"
ospfd_options=" -A 127.0.0.1"
EOF
chmod 644 /etc/frr/daemons
chown frr:frr /etc/frr/daemons
log "[3/8] Creating FRRouting configuration..."
# Create FRRouting configuration
WAN_ADDR=$(echo $WAN_IP | cut -d'/' -f1)
LAN_ADDR=$(echo $LAN_IP | cut -d'/' -f1)
cat > /etc/frr/frr.conf << EOF
frr version 8.4
frr defaults traditional
hostname frrouting-qos
log syslog informational
ipv6 forwarding
!
interface eth0
description WAN-Interface
ip address $WAN_IP
!
interface eth1
description LAN-Interface
ip address $LAN_IP
!
router bgp $BGP_AS
bgp router-id $WAN_ADDR
neighbor $CISCO_PEER remote-as $CISCO_AS
neighbor $CISCO_PEER description Cisco-BGP-Peer
!
address-family ipv4 unicast
neighbor $CISCO_PEER activate
neighbor $CISCO_PEER route-map QOS-OUT out
neighbor $CISCO_PEER route-map QOS-IN in
exit-address-family
!
router ospf
ospf router-id $LAN_ADDR
network $(echo $LAN_IP | sed 's/[0-9]*\//.0\//')24 area 0
network $(echo $WAN_IP | sed 's/[0-9]*\//.0\//')24 area 0
!
route-map QOS-OUT permit 10
set community $BGP_AS:100
set ip next-hop unchanged
!
route-map QOS-IN permit 10
match community PRIORITY-TRAFFIC
set local-preference 200
!
ip community-list standard PRIORITY-TRAFFIC permit $CISCO_AS:100
!
line vty
!
EOF
chmod 644 /etc/frr/frr.conf
chown frr:frr /etc/frr/frr.conf
log "[4/8] Starting FRRouting services..."
systemctl enable frr
systemctl start frr
sleep 5
log "[5/8] Configuring DSCP marking with iptables..."
# Create iptables rules for DSCP marking
if [[ "$ID" == "ubuntu" || "$ID" == "debian" ]]; then
IPTABLES_DIR="/etc/iptables"
else
IPTABLES_DIR="/etc/sysconfig"
fi
mkdir -p $IPTABLES_DIR
cat > /tmp/dscp-rules << 'EOF'
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
# Voice traffic - EF DSCP 46
-A POSTROUTING -p udp --dport 5060:5090 -j DSCP --set-dscp 46
-A POSTROUTING -p tcp --dport 5060:5090 -j DSCP --set-dscp 46
-A POSTROUTING -p udp --dport 16384:32767 -j DSCP --set-dscp 46
# Video conferencing - AF41 DSCP 34
-A POSTROUTING -p tcp --dport 443 -m string --string "zoom" --algo bm -j DSCP --set-dscp 34
-A POSTROUTING -p udp --dport 8801:8810 -j DSCP --set-dscp 34
-A POSTROUTING -p tcp --dport 1935 -j DSCP --set-dscp 34
# Database traffic - AF31 DSCP 26
-A POSTROUTING -p tcp --dport 3306 -j DSCP --set-dscp 26
-A POSTROUTING -p tcp --dport 5432 -j DSCP --set-dscp 26
-A POSTROUTING -p tcp --dport 1433 -j DSCP --set-dscp 26
# Management traffic - CS6 DSCP 48
-A POSTROUTING -p tcp --dport 22 -j DSCP --set-dscp 48
-A POSTROUTING -p tcp --dport 161:162 -j DSCP --set-dscp 48
-A POSTROUTING -p tcp --dport 179 -j DSCP --set-dscp 48
COMMIT
EOF
# Apply iptables rules
iptables-restore < /tmp/dscp-rules
rm /tmp/dscp-rules
log "[6/8] Saving iptables configuration..."
# Save iptables configuration
if [[ "$ID" == "ubuntu" || "$ID" == "debian" ]]; then
iptables-save > /etc/iptables/rules.v4
systemctl enable netfilter-persistent
else
iptables-save > /etc/sysconfig/iptables
systemctl enable iptables
systemctl start iptables
fi
log "[7/8] Enabling services at boot..."
# Enable services
systemctl enable frr
systemctl enable $IPTABLES_SERVICE
log "[8/8] Verifying installation..."
# Verification checks
sleep 10
# Check FRRouting status
if ! systemctl is-active --quiet frr; then
error "FRRouting service is not running"
fi
# Check BGP status
BGP_STATUS=$(vtysh -c "show ip bgp summary" 2>/dev/null || echo "BGP check failed")
if [[ "$BGP_STATUS" == *"BGP check failed"* ]]; then
warn "BGP status check failed - this is expected if Cisco peer is not configured yet"
else
log "BGP service is operational"
fi
# Check OSPF status
OSPF_STATUS=$(vtysh -c "show ip ospf neighbor" 2>/dev/null || echo "OSPF check failed")
if [[ "$OSPF_STATUS" == *"OSPF check failed"* ]]; then
warn "OSPF status check failed - this is expected without OSPF neighbors"
else
log "OSPF service is operational"
fi
# Check iptables rules
if iptables -t mangle -L | grep -q "DSCP"; then
log "DSCP marking rules applied successfully"
else
error "DSCP marking rules not found"
fi
log "Enterprise QoS with FRRouting installation completed successfully!"
log "BGP AS: $BGP_AS, Cisco Peer: $CISCO_PEER"
log "Configure your Cisco equipment to peer with this BGP router"
log "Monitor with: vtysh -c 'show ip bgp summary'"
Review the script before running. Execute with: bash install.sh