Implement SNMP trap monitoring and alerting system for proactive network management

Intermediate 45 min Apr 15, 2026 246 views
Ubuntu 24.04 Debian 12 AlmaLinux 9 Rocky Linux 9

Set up comprehensive SNMP trap monitoring with snmptrapd, automated alerting via email and webhooks, and integration with monitoring platforms like Nagios and Zabbix for proactive network management and real-time issue detection.

Prerequisites

  • Root access
  • Basic SNMP knowledge
  • Network devices that support SNMP traps
  • Email server (postfix) for notifications

What this solves

SNMP trap monitoring provides proactive network management by receiving and processing unsolicited notifications from network devices when events occur. This tutorial sets up a complete trap monitoring system with Net-SNMP 5.9, automated alerting through multiple channels, and integration with existing monitoring platforms for comprehensive network visibility.

Step-by-step installation

Update system packages

Start by updating your package manager to ensure you get the latest versions of SNMP packages.

sudo apt update && sudo apt upgrade -y
sudo dnf update -y

Install SNMP packages and MIB tools

Install Net-SNMP daemon, utilities, and MIB management tools for comprehensive trap handling.

sudo apt install -y snmp snmp-mibs-downloader snmptrapd postfix mailutils curl jq
sudo dnf install -y net-snmp net-snmp-utils postfix mailx curl jq

Create trap handler directory structure

Set up directories for trap handlers, logs, and configuration files with proper permissions.

sudo mkdir -p /etc/snmp/traphandlers
sudo mkdir -p /var/log/snmp/traps
sudo mkdir -p /var/lib/snmp/trapdb
sudo chown -R snmp:snmp /var/log/snmp
sudo chown -R snmp:snmp /var/lib/snmp
sudo chmod 755 /etc/snmp/traphandlers

Configure snmptrapd daemon

Create the main snmptrapd configuration file with trap receiving settings and handler definitions.

# SNMP Trap Daemon Configuration

Listen on all interfaces, port 162 (standard SNMP trap port)

snmpTrapdAddr udp:162,udp6:162

Authentication settings

disableAuthorization yes

Logging configuration

logOption s 6 logTimestamp yes

Output format for traps

format1 %.4y-%.2m-%.2l %.2h:%.2j:%.2k [%B] %N: %W\n\t%V\n format2 %.4y-%.2m-%.2l %.2h:%.2j:%.2k [%B] %N: Enterprise Specific Trap (%w) Uptime: %#T\n\t%v\n

Trap handlers for different trap types

traphandle default /etc/snmp/traphandlers/default_handler.sh traphandle SNMPv2-MIB::coldStart /etc/snmp/traphandlers/system_handler.sh traphandle SNMPv2-MIB::warmStart /etc/snmp/traphandlers/system_handler.sh traphandle SNMPv2-MIB::linkDown /etc/snmp/traphandlers/interface_handler.sh traphandle SNMPv2-MIB::linkUp /etc/snmp/traphandlers/interface_handler.sh traphandle SNMPv2-MIB::authenticationFailure /etc/snmp/traphandlers/security_handler.sh

Create PID file

pidFile /var/run/snmptrapd.pid

Run as snmp user

agentuser snmp

Create default trap handler script

Build the main trap processing script that logs all traps and triggers alert mechanisms.

#!/bin/bash

Default SNMP Trap Handler

Processes all incoming SNMP traps and triggers alerts

TRAP_LOG="/var/log/snmp/traps/default.log" ALERT_CONFIG="/etc/snmp/alert_config.conf" TRAP_DB="/var/lib/snmp/trapdb/traps.db"

Source alerting configuration

if [ -f "$ALERT_CONFIG" ]; then source "$ALERT_CONFIG" fi

Get trap information from environment variables

TRAP_TIME=$(date '+%Y-%m-%d %H:%M:%S') SOURCE_IP="${SNMP_TRANSPORT_ADDRESS}" OID="${SNMP_COMMAND}" UPTIME="${SNMP_ARG2:-Unknown}" TRAP_OID="${SNMP_ARG3:-Unknown}"

Read all trap variables

TRAP_DATA="" while read line; do TRAP_DATA+="$line\n" done

Log trap to file

echo "[$TRAP_TIME] Source: $SOURCE_IP, OID: $TRAP_OID" >> "$TRAP_LOG" echo -e "$TRAP_DATA" >> "$TRAP_LOG" echo "---" >> "$TRAP_LOG"

Store in simple database for tracking

echo "$TRAP_TIME|$SOURCE_IP|$TRAP_OID|$(echo -e "$TRAP_DATA" | tr '\n' ' ')" >> "$TRAP_DB"

Determine trap severity

SEVERITY="INFO" case "$TRAP_OID" in coldStart|warmStart) SEVERITY="WARNING" ;; linkDown|authenticationFailure) SEVERITY="CRITICAL" ;; linkUp) SEVERITY="INFO" ;; esac

Send alerts based on severity

if [ "$SEVERITY" = "CRITICAL" ] || [ "$SEVERITY" = "WARNING" ]; then /etc/snmp/traphandlers/send_alert.sh "$SEVERITY" "$SOURCE_IP" "$TRAP_OID" "$TRAP_DATA" fi exit 0
sudo chmod 755 /etc/snmp/traphandlers/default_handler.sh

Create specialized trap handlers

Create specific handlers for different types of network events with targeted processing logic.

#!/bin/bash

System Event Trap Handler (coldStart, warmStart)

TRAP_LOG="/var/log/snmp/traps/system.log" SOURCE_IP="${SNMP_TRANSPORT_ADDRESS}" TRAP_TIME=$(date '+%Y-%m-%d %H:%M:%S')

Read trap data

TRAP_DATA="" while read line; do TRAP_DATA+="$line\n" done

Determine event type

EVENT_TYPE="System Event" if [[ "$SNMP_ARG3" == "coldStart" ]]; then EVENT_TYPE="System Cold Start" elif [[ "$SNMP_ARG3" == "warmStart" ]]; then EVENT_TYPE="System Warm Start" fi

Log system event

echo "[$TRAP_TIME] $EVENT_TYPE from $SOURCE_IP" >> "$TRAP_LOG" echo -e "$TRAP_DATA" >> "$TRAP_LOG" echo "---" >> "$TRAP_LOG"

Trigger system event alert

/etc/snmp/traphandlers/send_alert.sh "WARNING" "$SOURCE_IP" "$EVENT_TYPE" "Device restarted: $TRAP_DATA" exit 0
#!/bin/bash

Interface Event Trap Handler (linkUp, linkDown)

TRAP_LOG="/var/log/snmp/traps/interface.log" SOURCE_IP="${SNMP_TRANSPORT_ADDRESS}" TRAP_TIME=$(date '+%Y-%m-%d %H:%M:%S')

Read trap data

TRAP_DATA="" INTERFACE_ID="Unknown" while read line; do TRAP_DATA+="$line\n" # Extract interface ID if present if [[ "$line" == "ifIndex" ]]; then INTERFACE_ID=$(echo "$line" | grep -o '[0-9]\+' | head -1) fi done

Determine link status

LINK_STATUS="Unknown" SEVERITY="INFO" if [[ "$SNMP_ARG3" == "linkDown" ]]; then LINK_STATUS="Link Down" SEVERITY="CRITICAL" elif [[ "$SNMP_ARG3" == "linkUp" ]]; then LINK_STATUS="Link Up" SEVERITY="INFO" fi

Log interface event

echo "[$TRAP_TIME] $LINK_STATUS on interface $INTERFACE_ID from $SOURCE_IP" >> "$TRAP_LOG" echo -e "$TRAP_DATA" >> "$TRAP_LOG" echo "---" >> "$TRAP_LOG"

Send alert for link down events

if [ "$SEVERITY" = "CRITICAL" ]; then /etc/snmp/traphandlers/send_alert.sh "$SEVERITY" "$SOURCE_IP" "Interface Down" "Interface $INTERFACE_ID is down on device $SOURCE_IP" fi exit 0
#!/bin/bash

Security Event Trap Handler (authenticationFailure)

TRAP_LOG="/var/log/snmp/traps/security.log" SOURCE_IP="${SNMP_TRANSPORT_ADDRESS}" TRAP_TIME=$(date '+%Y-%m-%d %H:%M:%S')

Read trap data

TRAP_DATA="" while read line; do TRAP_DATA+="$line\n" done

Log security event

echo "[$TRAP_TIME] SECURITY ALERT: Authentication failure from $SOURCE_IP" >> "$TRAP_LOG" echo -e "$TRAP_DATA" >> "$TRAP_LOG" echo "---" >> "$TRAP_LOG"

Always send critical alert for security events

/etc/snmp/traphandlers/send_alert.sh "CRITICAL" "$SOURCE_IP" "Authentication Failure" "SNMP authentication failure detected from $SOURCE_IP" exit 0
sudo chmod 755 /etc/snmp/traphandlers/*.sh

Create alert configuration and sender script

Configure email and webhook alert settings with the alert dispatch script.

# SNMP Trap Alert Configuration

Email settings

EMAIL_ENABLED=true EMAIL_TO="admin@example.com" EMAIL_FROM="snmp-monitor@example.com" EMAIL_SUBJECT_PREFIX="[SNMP Alert]"

Webhook settings

WEBHOOK_ENABLED=true WEBHOOK_URL="https://hooks.example.com/snmp-alerts" WEBHOOK_SECRET="your-webhook-secret-here"

Slack webhook (optional)

SLACK_ENABLED=false SLACK_WEBHOOK_URL="https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK"

Alert throttling (seconds between duplicate alerts)

ALERT_THROTTLE=300

Log settings

ALERT_LOG="/var/log/snmp/traps/alerts.log"
#!/bin/bash

SNMP Trap Alert Sender

Sends alerts via email, webhook, and other channels

ALERT_CONFIG="/etc/snmp/alert_config.conf" THROTTLE_FILE="/var/lib/snmp/trapdb/alert_throttle"

Source configuration

if [ -f "$ALERT_CONFIG" ]; then source "$ALERT_CONFIG" else echo "Alert configuration not found: $ALERT_CONFIG" exit 1 fi SEVERITY="$1" SOURCE_IP="$2" EVENT_TYPE="$3" MESSAGE="$4" TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')

Throttle duplicate alerts

ALERT_KEY="${SOURCE_IP}_${EVENT_TYPE}" if [ -f "$THROTTLE_FILE" ]; then LAST_ALERT=$(grep "^$ALERT_KEY" "$THROTTLE_FILE" | cut -d'|' -f2) if [ -n "$LAST_ALERT" ]; then TIME_DIFF=$(($(date +%s) - $LAST_ALERT)) if [ $TIME_DIFF -lt $ALERT_THROTTLE ]; then echo "[$TIMESTAMP] Alert throttled: $ALERT_KEY" >> "$ALERT_LOG" exit 0 fi fi fi

Update throttle file

grep -v "^$ALERT_KEY" "$THROTTLE_FILE" 2>/dev/null > "${THROTTLE_FILE}.tmp" || touch "${THROTTLE_FILE}.tmp" echo "$ALERT_KEY|$(date +%s)" >> "${THROTTLE_FILE}.tmp" mv "${THROTTLE_FILE}.tmp" "$THROTTLE_FILE"

Log alert

echo "[$TIMESTAMP] Sending $SEVERITY alert for $SOURCE_IP: $EVENT_TYPE" >> "$ALERT_LOG"

Send email alert

if [ "$EMAIL_ENABLED" = "true" ]; then EMAIL_BODY="SNMP Trap Alert\n\nSeverity: $SEVERITY\nSource: $SOURCE_IP\nEvent: $EVENT_TYPE\nTime: $TIMESTAMP\n\nDetails:\n$MESSAGE" echo -e "$EMAIL_BODY" | mail -s "$EMAIL_SUBJECT_PREFIX $SEVERITY - $EVENT_TYPE from $SOURCE_IP" "$EMAIL_TO" fi

Send webhook alert

if [ "$WEBHOOK_ENABLED" = "true" ] && [ -n "$WEBHOOK_URL" ]; then WEBHOOK_PAYLOAD=$(cat <Send Slack alert if [ "$SLACK_ENABLED" = "true" ] && [ -n "$SLACK_WEBHOOK_URL" ]; then SLACK_COLOR="good" case "$SEVERITY" in "CRITICAL") SLACK_COLOR="danger" ;; "WARNING") SLACK_COLOR="warning" ;; esac SLACK_PAYLOAD=$(cat <
sudo chmod 755 /etc/snmp/traphandlers/send_alert.sh

Configure Nagios integration

Create Nagios passive check integration for SNMP trap events.

#!/bin/bash

Nagios SNMP Trap Integration

Sends passive check results to Nagios based on trap events

NAGIOS_COMMAND_FILE="/var/lib/nagios3/rw/nagios.cmd" NAGIOS_CONFIG="/etc/snmp/nagios_trap_config.conf"

Source Nagios configuration

if [ -f "$NAGIOS_CONFIG" ]; then source "$NAGIOS_CONFIG" else # Default settings NAGIOS_ENABLED=false NAGIOS_SERVICE_PREFIX="SNMP_" fi if [ "$NAGIOS_ENABLED" != "true" ]; then exit 0 fi SOURCE_IP="$1" EVENT_TYPE="$2" SEVERITY="$3" MESSAGE="$4"

Map severity to Nagios states

NAGIOS_STATE=0 case "$SEVERITY" in "CRITICAL") NAGIOS_STATE=2 ;; "WARNING") NAGIOS_STATE=1 ;; "INFO") NAGIOS_STATE=0 ;; esac

Create service name

SERVICE_NAME="${NAGIOS_SERVICE_PREFIX}${EVENT_TYPE}"

Map IP to hostname if configured

HOSTNAME="$SOURCE_IP" if [ -f "/etc/snmp/ip_hostname_map.conf" ]; then MAPPED_HOST=$(grep "^$SOURCE_IP" /etc/snmp/ip_hostname_map.conf | cut -d' ' -f2) if [ -n "$MAPPED_HOST" ]; then HOSTNAME="$MAPPED_HOST" fi fi

Submit passive check result

TIMESTAMP=$(date +%s) echo "[$TIMESTAMP] PROCESS_SERVICE_CHECK_RESULT;$HOSTNAME;$SERVICE_NAME;$NAGIOS_STATE;$MESSAGE" >> "$NAGIOS_COMMAND_FILE" exit 0
# Nagios SNMP Trap Integration Configuration

NAGIOS_ENABLED=true
NAGIOS_SERVICE_PREFIX="SNMP_"

Command file location (check your Nagios installation)

NAGIOS_COMMAND_FILE="/var/lib/nagios3/rw/nagios.cmd"
# IP to Hostname mapping for Nagios integration

Format: IP_ADDRESS HOSTNAME

203.0.113.10 router-01 203.0.113.11 switch-01 203.0.113.12 server-01
sudo chmod 755 /etc/snmp/traphandlers/nagios_integration.sh

Create Zabbix integration script

Build Zabbix integration for trap data using zabbix_sender utility.

#!/bin/bash

Zabbix SNMP Trap Integration

Sends trap data to Zabbix server using zabbix_sender

ZABBIX_CONFIG="/etc/snmp/zabbix_trap_config.conf"

Source Zabbix configuration

if [ -f "$ZABBIX_CONFIG" ]; then source "$ZABBIX_CONFIG" else # Default settings ZABBIX_ENABLED=false ZABBIX_SERVER="127.0.0.1" ZABBIX_PORT=10051 fi if [ "$ZABBIX_ENABLED" != "true" ]; then exit 0 fi SOURCE_IP="$1" EVENT_TYPE="$2" SEVERITY="$3" MESSAGE="$4"

Map IP to Zabbix hostname if configured

ZABBIX_HOST="$SOURCE_IP" if [ -f "/etc/snmp/zabbix_host_map.conf" ]; then MAPPED_HOST=$(grep "^$SOURCE_IP" /etc/snmp/zabbix_host_map.conf | cut -d' ' -f2) if [ -n "$MAPPED_HOST" ]; then ZABBIX_HOST="$MAPPED_HOST" fi fi

Create trap item key

ITEM_KEY="snmp.trap[${EVENT_TYPE}]"

Send data to Zabbix

echo "$ZABBIX_HOST $ITEM_KEY $MESSAGE" | zabbix_sender -z "$ZABBIX_SERVER" -p "$ZABBIX_PORT" -T -i - >/dev/null 2>&1

Send trap count metric

COUNT_KEY="snmp.trap.count[${EVENT_TYPE}]" echo "$ZABBIX_HOST $COUNT_KEY 1" | zabbix_sender -z "$ZABBIX_SERVER" -p "$ZABBIX_PORT" -T -i - >/dev/null 2>&1 exit 0
# Zabbix SNMP Trap Integration Configuration

ZABBIX_ENABLED=true
ZABBIX_SERVER="203.0.113.100"
ZABBIX_PORT=10051
# IP to Zabbix Hostname mapping

Format: IP_ADDRESS ZABBIX_HOSTNAME

203.0.113.10 Router-Office-01 203.0.113.11 Switch-Floor-01 203.0.113.12 Server-Web-01

Install Zabbix sender utility

Install zabbix_sender for sending trap data to Zabbix server.

sudo apt install -y zabbix-sender
sudo dnf install -y zabbix-sender

Update trap handlers with integrations

Modify the main trap handlers to call monitoring system integrations.

sudo cp /etc/snmp/traphandlers/default_handler.sh /etc/snmp/traphandlers/default_handler.sh.backup
#!/bin/bash

Enhanced Default SNMP Trap Handler with Monitoring Integration

TRAP_LOG="/var/log/snmp/traps/default.log" ALERT_CONFIG="/etc/snmp/alert_config.conf" TRAP_DB="/var/lib/snmp/trapdb/traps.db"

Source alerting configuration

if [ -f "$ALERT_CONFIG" ]; then source "$ALERT_CONFIG" fi

Get trap information from environment variables

TRAP_TIME=$(date '+%Y-%m-%d %H:%M:%S') SOURCE_IP="${SNMP_TRANSPORT_ADDRESS}" OID="${SNMP_COMMAND}" UPTIME="${SNMP_ARG2:-Unknown}" TRAP_OID="${SNMP_ARG3:-Unknown}"

Read all trap variables

TRAP_DATA="" while read line; do TRAP_DATA+="$line\n" done

Log trap to file

echo "[$TRAP_TIME] Source: $SOURCE_IP, OID: $TRAP_OID" >> "$TRAP_LOG" echo -e "$TRAP_DATA" >> "$TRAP_LOG" echo "---" >> "$TRAP_LOG"

Store in simple database for tracking

echo "$TRAP_TIME|$SOURCE_IP|$TRAP_OID|$(echo -e "$TRAP_DATA" | tr '\n' ' ')" >> "$TRAP_DB"

Determine trap severity and event type

SEVERITY="INFO" EVENT_TYPE="Generic" case "$TRAP_OID" in coldStart) SEVERITY="WARNING" EVENT_TYPE="ColdStart" ;; warmStart) SEVERITY="WARNING" EVENT_TYPE="WarmStart" ;; linkDown) SEVERITY="CRITICAL" EVENT_TYPE="LinkDown" ;; linkUp) SEVERITY="INFO" EVENT_TYPE="LinkUp" ;; authenticationFailure) SEVERITY="CRITICAL" EVENT_TYPE="AuthFailure" ;; esac

Send alerts based on severity

if [ "$SEVERITY" = "CRITICAL" ] || [ "$SEVERITY" = "WARNING" ]; then /etc/snmp/traphandlers/send_alert.sh "$SEVERITY" "$SOURCE_IP" "$EVENT_TYPE" "$TRAP_DATA" fi

Send to monitoring systems

/etc/snmp/traphandlers/nagios_integration.sh "$SOURCE_IP" "$EVENT_TYPE" "$SEVERITY" "$(echo -e "$TRAP_DATA" | head -1)" & /etc/snmp/traphandlers/zabbix_integration.sh "$SOURCE_IP" "$EVENT_TYPE" "$SEVERITY" "$(echo -e "$TRAP_DATA" | head -1)" & exit 0

Configure systemd service

Create and configure the systemd service for snmptrapd with proper security settings.

[Unit]
Description=Net-SNMP Trap Daemon
After=network.target
Wants=network.target

[Service]
Type=forking
PIDFile=/var/run/snmptrapd.pid
ExecStart=/usr/sbin/snmptrapd -Lsd -p /var/run/snmptrapd.pid
ExecReload=/bin/kill -HUP $MAINPID
User=snmp
Group=snmp
Restart=on-failure
RestartSec=5

Security settings

NoNewPrivileges=yes PrivateTmp=yes ProtectSystem=strict ProtectHome=yes ReadWritePaths=/var/log/snmp /var/lib/snmp /var/run [Install] WantedBy=multi-user.target

Configure firewall for SNMP traps

Open UDP port 162 for incoming SNMP trap traffic.

sudo ufw allow 162/udp comment 'SNMP Traps'
sudo ufw reload
sudo firewall-cmd --permanent --add-port=162/udp
sudo firewall-cmd --reload

Create log rotation configuration

Set up log rotation to prevent trap logs from consuming excessive disk space.

/var/log/snmp/traps/*.log {
    daily
    missingok
    rotate 30
    compress
    delaycompress
    notifempty
    create 644 snmp snmp
    postrotate
        if [ -f /var/run/snmptrapd.pid ]; then
            /bin/kill -HUP cat /var/run/snmptrapd.pid
        fi
    endscript
}

Enable and start the service

Enable snmptrapd to start automatically and start the service.

sudo systemctl daemon-reload
sudo systemctl enable snmptrapd
sudo systemctl start snmptrapd
sudo systemctl status snmptrapd

Configure alert settings

Update alert configuration

Customize the alert configuration file with your specific email and webhook settings.

sudo nano /etc/snmp/alert_config.conf

Update the EMAIL_TO, WEBHOOK_URL, and other settings according to your environment. The system needs a working mail server for email alerts. You can learn more about comprehensive monitoring setups in our Nagios SNMP monitoring guide.

Test email configuration

Verify that email alerts are working properly with the mail system.

echo "Test email from SNMP trap system" | mail -s "SNMP Test Alert" admin@example.com

Verify your setup

sudo systemctl status snmptrapd
sudo netstat -ulnp | grep :162
sudo tail -f /var/log/snmp/traps/default.log

Test trap reception by sending a test trap from another system:

snmptrap -v2c -c public localhost '' 1.3.6.1.4.1.8072.2.3.0.1 1.3.6.1.4.1.8072.2.3.2.1 i 42

Check trap handler permissions and logs:

ls -la /etc/snmp/traphandlers/
sudo tail -20 /var/log/snmp/traps/alerts.log
wc -l /var/lib/snmp/trapdb/traps.db

Integration with monitoring systems

For Nagios integration, ensure your Nagios configuration includes passive service definitions for SNMP trap events. For comprehensive Zabbix monitoring, check our Zabbix network automation guide. Configure your network devices to send traps to your monitoring server's IP address on port 162.

Common issues

SymptomCauseFix
snmptrapd won't startConfiguration syntax errorsudo snmptrapd -f -Lo -c /etc/snmp/snmptrapd.conf to check config
No traps receivedFirewall blocking port 162Check firewall rules and open UDP port 162
Permission denied errorsIncorrect file permissionssudo chown -R snmp:snmp /var/log/snmp /var/lib/snmp
Handler scripts not executingScripts not executable or wrong pathsudo chmod 755 /etc/snmp/traphandlers/*.sh
Email alerts not workingPostfix not configuredConfigure postfix: sudo dpkg-reconfigure postfix
Webhook alerts failingNetwork connectivity or wrong URLTest with curl: curl -X POST $WEBHOOK_URL

Next steps

Automated install script

Run this to automate the entire setup

Need help?

Don't want to manage this yourself?

We handle managed devops services for businesses that depend on uptime. From initial setup to ongoing operations.