Configure advanced nftables logging and monitoring for network security

Advanced 45 min May 26, 2026 463 views
Ubuntu 24.04 Debian 12 AlmaLinux 9 Rocky Linux 9

Set up comprehensive nftables logging with structured syslog integration, Prometheus metrics collection, and ELK stack analysis. Configure Grafana dashboards for real-time firewall monitoring and automated alerting for security events.

Prerequisites

  • Root access to the server
  • Basic knowledge of nftables and iptables
  • Prometheus and Grafana installed
  • Elasticsearch cluster available

What this solves

nftables provides powerful packet filtering capabilities, but without proper logging and monitoring, security events go unnoticed. This tutorial configures advanced nftables logging with structured output, integrates with Prometheus for metrics collection, and sets up ELK stack analysis with Grafana dashboards for comprehensive firewall monitoring and automated security alerting.

Step-by-step configuration

Install nftables and logging components

Install nftables, rsyslog for structured logging, and monitoring tools for metrics collection.

sudo apt update
sudo apt install -y nftables rsyslog rsyslog-gnutls python3-prometheus-client filebeat
sudo dnf install -y nftables rsyslog rsyslog-gnutls python3-prometheus-client filebeat

Configure nftables with structured logging

Create a comprehensive nftables configuration with detailed logging for different traffic types and security events.

#!/usr/sbin/nft -f

flush ruleset

table inet filter {
    chain input {
        type filter hook input priority filter; policy drop;
        
        # Allow loopback traffic
        iif "lo" accept
        
        # Allow established and related connections
        ct state established,related accept
        
        # Log and allow SSH with rate limiting
        tcp dport 22 limit rate 10/minute burst 5 packets log prefix "NFTABLES-SSH-ALLOW: " level info accept
        tcp dport 22 log prefix "NFTABLES-SSH-DROP: " level warn drop
        
        # Log and allow HTTP/HTTPS
        tcp dport { 80, 443 } log prefix "NFTABLES-WEB-ALLOW: " level info accept
        
        # Log suspicious port scans
        tcp flags & (fin|syn|rst|psh|ack|urg) == syn limit rate 1/second burst 5 packets log prefix "NFTABLES-SCAN-DETECT: " level warn
        
        # Log and drop invalid packets
        ct state invalid log prefix "NFTABLES-INVALID: " level warn drop
        
        # Log ICMP for network troubleshooting
        icmp type { echo-request, destination-unreachable, time-exceeded } limit rate 5/second log prefix "NFTABLES-ICMP: " level info accept
        icmpv6 type { echo-request, destination-unreachable, time-exceeded, nd-neighbor-solicit, nd-neighbor-advert } limit rate 5/second log prefix "NFTABLES-ICMP6: " level info accept
        
        # Log everything else before dropping
        log prefix "NFTABLES-INPUT-DROP: " level notice
    }
    
    chain forward {
        type filter hook forward priority filter; policy drop;
        
        # Log forwarded traffic
        ct state established,related accept
        log prefix "NFTABLES-FORWARD-DROP: " level notice
    }
    
    chain output {
        type filter hook output priority filter; policy accept;
        
        # Log outgoing connections to suspicious ports
        tcp dport { 1433, 3389, 5432, 6379 } log prefix "NFTABLES-OUT-SUSPICIOUS: " level warn
    }
    
    # Custom chain for intrusion detection
    chain intrusion_detection {
        # Log multiple failed SSH attempts from same IP
        tcp dport 22 ct state new @ssh_attempts { ip saddr limit rate over 5/minute } log prefix "NFTABLES-SSH-BRUTE: " level err drop
        
        # Log port scan attempts
        tcp flags & (fin|syn) == (fin|syn) log prefix "NFTABLES-XMAS-SCAN: " level err drop
        tcp flags & (fin|syn|rst|psh|ack|urg) == 0 log prefix "NFTABLES-NULL-SCAN: " level err drop
    }
}

# Create named sets for tracking
table inet filter {
    set ssh_attempts {
        type ipv4_addr
        size 1000
        timeout 1h
        flags dynamic
    }
}

Configure rsyslog for structured nftables logging

Set up rsyslog to capture nftables logs with structured formatting and separate log files for different event types.

# nftables logging configuration
# Create separate log files for different nftables events

# Define templates for structured logging
$template NFTablesFormat,"%timestamp:::date-rfc3339% %hostname% %syslogtag% [%structured-data%] %msg%\n"

$template NFTablesJSONFormat,"{\"timestamp\":\"%timestamp:::date-rfc3339%\",\"hostname\":\"%hostname%\",\"facility\":\"%syslogfacility-text%\",\"severity\":\"%syslogseverity-text%\",\"tag\":\"%syslogtag%\",\"message\":\"%msg%\"}\n"

# SSH-related logs
:msg, contains, "NFTABLES-SSH" /var/log/nftables/ssh.log;NFTablesFormat
:msg, contains, "NFTABLES-SSH" /var/log/nftables/ssh.json;NFTablesJSONFormat

# Web traffic logs
:msg, contains, "NFTABLES-WEB" /var/log/nftables/web.log;NFTablesFormat
:msg, contains, "NFTABLES-WEB" /var/log/nftables/web.json;NFTablesJSONFormat

# Security incident logs
:msg, contains, "NFTABLES-SCAN\|NFTABLES-BRUTE\|NFTABLES-XMAS\|NFTABLES-NULL" /var/log/nftables/security.log;NFTablesFormat
:msg, contains, "NFTABLES-SCAN\|NFTABLES-BRUTE\|NFTABLES-XMAS\|NFTABLES-NULL" /var/log/nftables/security.json;NFTablesJSONFormat

# General nftables logs
:msg, contains, "NFTABLES" /var/log/nftables/general.log;NFTablesFormat
:msg, contains, "NFTABLES" /var/log/nftables/general.json;NFTablesJSONFormat

# Stop processing nftables messages after logging
:msg, contains, "NFTABLES" stop

Create log directories and set permissions

Create the nftables log directory structure with proper permissions for security and log rotation.

sudo mkdir -p /var/log/nftables
sudo chown syslog:adm /var/log/nftables
sudo chmod 755 /var/log/nftables
sudo systemctl restart rsyslog

Configure log rotation for nftables logs

Set up logrotate to manage nftables log files with compression and retention policies.

/var/log/nftables/*.log /var/log/nftables/*.json {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    create 644 syslog adm
    sharedscripts
    postrotate
        /usr/lib/rsyslog/rsyslog-rotate
    endscript
    # Send HUP signal to rsyslog
    postrotate
        systemctl reload rsyslog > /dev/null 2>&1 || true
    endscript
}

Create Prometheus exporter for nftables metrics

Develop a Python script that parses nftables logs and exposes metrics for Prometheus scraping.

#!/usr/bin/env python3
import re
import time
import json
from collections import defaultdict, Counter
from prometheus_client import start_http_server, Counter as PrometheusCounter, Gauge, Histogram
import threading
import subprocess

# Prometheus metrics
nftables_packets_total = PrometheusCounter(
    'nftables_packets_total',
    'Total packets processed by nftables',
    ['chain', 'action', 'protocol']
)

nftables_bytes_total = PrometheusCounter(
    'nftables_bytes_total', 
    'Total bytes processed by nftables',
    ['chain', 'action', 'protocol']
)

nftables_connections_active = Gauge(
    'nftables_connections_active',
    'Active connections tracked by nftables'
)

nftables_security_events = PrometheusCounter(
    'nftables_security_events_total',
    'Security events detected by nftables',
    ['event_type', 'source_ip']
)

nftables_rule_hits = PrometheusCounter(
    'nftables_rule_hits_total',
    'Number of times each rule was triggered',
    ['rule_name']
)

class NFTablesMetricsCollector:
    def __init__(self):
        self.ssh_attempts = defaultdict(int)
        self.blocked_ips = set()
        
    def parse_nftables_log(self, log_line):
        """Parse nftables log entries and extract metrics"""
        try:
            # Parse JSON logs for structured data
            if log_line.startswith('{'):
                log_data = json.loads(log_line)
                message = log_data.get('message', '')
            else:
                message = log_line
                
            # Extract rule information
            if 'NFTABLES-SSH-ALLOW' in message:
                nftables_packets_total.labels(chain='input', action='allow', protocol='tcp').inc()
                nftables_rule_hits.labels(rule_name='ssh_allow').inc()
                
            elif 'NFTABLES-SSH-DROP' in message:
                nftables_packets_total.labels(chain='input', action='drop', protocol='tcp').inc()
                nftables_rule_hits.labels(rule_name='ssh_drop').inc()
                
            elif 'NFTABLES-WEB-ALLOW' in message:
                nftables_packets_total.labels(chain='input', action='allow', protocol='tcp').inc()
                nftables_rule_hits.labels(rule_name='web_allow').inc()
                
            elif 'NFTABLES-SSH-BRUTE' in message:
                src_ip = self.extract_ip(message)
                if src_ip:
                    nftables_security_events.labels(event_type='ssh_brute_force', source_ip=src_ip).inc()
                    self.ssh_attempts[src_ip] += 1
                    
            elif 'NFTABLES-SCAN-DETECT' in message:
                src_ip = self.extract_ip(message)
                if src_ip:
                    nftables_security_events.labels(event_type='port_scan', source_ip=src_ip).inc()
                    
            elif 'NFTABLES-INVALID' in message:
                nftables_security_events.labels(event_type='invalid_packet', source_ip='unknown').inc()
                
        except Exception as e:
            print(f"Error parsing log line: {e}")
            
    def extract_ip(self, message):
        """Extract IP address from log message"""
        ip_pattern = r'SRC=(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})'
        match = re.search(ip_pattern, message)
        return match.group(1) if match else None
        
    def update_connection_stats(self):
        """Update active connection statistics"""
        try:
            result = subprocess.run(['nft', 'list', 'table', 'inet', 'filter'], 
                                  capture_output=True, text=True)
            # Parse nftables output for connection tracking info
            # This is a simplified example - real implementation would be more complex
            active_conns = len(re.findall(r'ct state established', result.stdout))
            nftables_connections_active.set(active_conns)
        except Exception as e:
            print(f"Error updating connection stats: {e}")
            
    def monitor_logs(self):
        """Monitor nftables log files for new entries"""
        import subprocess
        
        # Monitor JSON log file for new entries
        cmd = ['tail', '-F', '/var/log/nftables/general.json']
        process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
        
        for line in iter(process.stdout.readline, ''):
            if line.strip():
                self.parse_nftables_log(line.strip())
                
def main():
    collector = NFTablesMetricsCollector()
    
    # Start Prometheus metrics server
    start_http_server(9090)
    print("nftables exporter started on port 9090")
    
    # Start log monitoring in background thread
    log_thread = threading.Thread(target=collector.monitor_logs)
    log_thread.daemon = True
    log_thread.start()
    
    # Periodic connection stats update
    while True:
        collector.update_connection_stats()
        time.sleep(30)

if __name__ == '__main__':
    main()

Create systemd service for nftables exporter

Configure the nftables Prometheus exporter as a systemd service for automatic startup and monitoring.

[Unit]
Description=NFTables Prometheus Exporter
After=network.target nftables.service
Requires=nftables.service

[Service]
Type=simple
User=nftables-exporter
Group=nftables-exporter
ExecStart=/usr/local/bin/nftables_exporter.py
Restart=always
RestartSec=5
Environment=PYTHONUNBUFFERED=1

# Security settings
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/log/nftables

[Install]
WantedBy=multi-user.target

Create dedicated user for nftables exporter

Create a dedicated system user for running the nftables exporter with minimal privileges.

sudo useradd --system --no-create-home --shell /bin/false nftables-exporter
sudo usermod -a -G adm nftables-exporter
sudo chmod +x /usr/local/bin/nftables_exporter.py

Configure Filebeat for ELK Stack integration

Set up Filebeat to ship nftables logs to Elasticsearch for analysis and long-term storage.

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/nftables/security.json
  fields:
    log_type: nftables_security
  fields_under_root: true
  json.keys_under_root: true
  json.add_error_key: true
  multiline.pattern: '^{'
  multiline.negate: true
  multiline.match: after

- type: log
  enabled: true
  paths:
    - /var/log/nftables/ssh.json
  fields:
    log_type: nftables_ssh
  fields_under_root: true
  json.keys_under_root: true
  json.add_error_key: true
  multiline.pattern: '^{'
  multiline.negate: true
  multiline.match: after

- type: log
  enabled: true
  paths:
    - /var/log/nftables/web.json
  fields:
    log_type: nftables_web
  fields_under_root: true
  json.keys_under_root: true
  json.add_error_key: true
  multiline.pattern: '^{'
  multiline.negate: true
  multiline.match: after

processors:
- add_host_metadata:
    when.not.contains.tags: forwarded
- add_docker_metadata: ~
- add_kubernetes_metadata: ~

# Elasticsearch output
output.elasticsearch:
  hosts: ["localhost:9200"]
  template.settings:
    index.number_of_shards: 1
    index.number_of_replicas: 0
  index: "nftables-logs-%{+yyyy.MM.dd}"

# Kibana dashboard loading
setup.kibana:
  host: "localhost:5601"

# Enable index lifecycle management
setup.ilm:
  enabled: true
  rollover_alias: "nftables-logs"
  pattern: "%{now/d}-000001"
  policy: "nftables-policy"

logging.level: info
logging.to_files: true
logging.files:
  path: /var/log/filebeat
  name: filebeat
  keepfiles: 7
  permissions: 0644

Enable and start all services

Enable nftables, start the exporter service, and configure Filebeat to begin log shipping.

sudo systemctl enable --now nftables
sudo systemctl enable --now nftables-exporter
sudo systemctl enable --now filebeat

# Load nftables rules
sudo nft -f /etc/nftables.conf

# Verify services are running
sudo systemctl status nftables nftables-exporter filebeat

Configure Prometheus to scrape nftables metrics

Add the nftables exporter as a scrape target in your Prometheus configuration.

# Add to scrape_configs section
scrape_configs:
  - job_name: 'nftables'
    static_configs:
      - targets: ['localhost:9090']
    scrape_interval: 15s
    scrape_timeout: 10s
    metrics_path: /metrics
    honor_labels: true
    params:
      format: ['prometheus']

  # Additional job for detailed nftables statistics
  - job_name: 'nftables-detailed'
    static_configs:
      - targets: ['localhost:9090']
    scrape_interval: 5s
    metrics_path: /metrics
    params:
      collect[]: ['nftables.security']
      
# Global config
global:
  scrape_interval: 15s
  evaluation_interval: 15s
  external_labels:
    monitor: 'nftables-monitor'

# Rule files for alerting
rule_files:
  - "nftables_alerts.yml"

# Alertmanager configuration  
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          - alertmanager:9093

Create Prometheus alerting rules

Define alerting rules for nftables security events and anomalies.

groups:
- name: nftables_security
  rules:
  - alert: NFTablesHighDropRate
    expr: rate(nftables_packets_total{action="drop"}[5m]) > 100
    for: 2m
    labels:
      severity: warning
      service: nftables
    annotations:
      summary: "High packet drop rate detected"
      description: "nftables is dropping {{ $value }} packets per second on {{ $labels.instance }}"

  - alert: NFTablesSSHBruteForce
    expr: rate(nftables_security_events_total{event_type="ssh_brute_force"}[1m]) > 0
    for: 0m
    labels:
      severity: critical
      service: nftables
    annotations:
      summary: "SSH brute force attack detected"
      description: "SSH brute force attack from {{ $labels.source_ip }} detected on {{ $labels.instance }}"

  - alert: NFTablesPortScanDetected
    expr: rate(nftables_security_events_total{event_type="port_scan"}[5m]) > 10
    for: 1m
    labels:
      severity: warning
      service: nftables
    annotations:
      summary: "Port scan activity detected"
      description: "Port scan from {{ $labels.source_ip }} detected on {{ $labels.instance }}"

  - alert: NFTablesExporterDown
    expr: up{job="nftables"} == 0
    for: 1m
    labels:
      severity: critical
      service: nftables
    annotations:
      summary: "nftables exporter is down"
      description: "nftables Prometheus exporter has been down for more than 1 minute on {{ $labels.instance }}"

  - alert: NFTablesInvalidPackets
    expr: rate(nftables_security_events_total{event_type="invalid_packet"}[5m]) > 50
    for: 2m
    labels:
      severity: warning
      service: nftables
    annotations:
      summary: "High rate of invalid packets"
      description: "{{ $value }} invalid packets per second detected on {{ $labels.instance }}"

  - alert: NFTablesConnectionTableFull
    expr: nftables_connections_active > 50000
    for: 5m
    labels:
      severity: warning
      service: nftables
    annotations:
      summary: "Connection tracking table nearly full"
      description: "{{ $value }} active connections on {{ $labels.

Automated install script

Run this to automate the entire setup

#nftables #firewall #logging #prometheus #monitoring

संबंधित tutorials

security Advanced

Implement Kubernetes security scanning with Falco and OPA Gatekeeper for runtime protection

Set up comprehensive Kubernetes security with Falco for runtime threat detection and OPA Gatekeeper for admission control policy enforcement. This tutorial covers installation, configuration, and custom security policies.

45 min 4 distros 364 views
security Advanced

Implement OpenResty rate limiting and DDoS protection with advanced Lua rules

Set up comprehensive rate limiting and DDoS protection for OpenResty using nginx directives, Redis-backed Lua middleware, and advanced security rules with monitoring and alerting.

45 min 4 distros 539 views
security Advanced

Configure Redis 7 cluster SSL encryption and authentication for production security

Secure your Redis 7 cluster with TLS encryption, client authentication, and inter-node SSL communication for production environments. Includes certificate generation, authentication setup, and security validation.

45 min 4 distros 526 views
Browse all tutorials in our complete directory →

इसे खुद मैनेज नहीं करना चाहते?

हम उन businesses के लिए infrastructure संभालते हैं जो uptime पर निर्भर हैं। Fully managed, एक fixed contact के साथ जो आपके setup को जानता है।

आपको एक निश्चित contact मिलता है जो आपके setup को जानता है

Rotterdam 22:46 · एक message में पहुंचें, कोई ticket form नहीं

+31 10 477 5362 WhatsApp