OpenLiteSpeed WAF with ModSecurity 3 and DDoS Protection

Set up comprehensive web application security for OpenLiteSpeed with ModSecurity 3 web application firewall, OWASP Core Rule Set for threat protection, and advanced rate limiting to defend against DDoS attacks and malicious traffic.

Prerequisites

Root access to server
OpenLiteSpeed web server
Basic understanding of web security concepts
Email configured for alerts

What this solves

OpenLiteSpeed provides excellent performance but needs additional security layers to protect against web application attacks and DDoS attempts. This tutorial implements ModSecurity 3 as a web application firewall (WAF) with the OWASP Core Rule Set, configures advanced rate limiting rules, and sets up security event monitoring to create a comprehensive defense system for your web applications.

Step-by-step installation

Update system packages

Start by updating your package manager to ensure you have the latest security patches and package information.

sudo apt update && sudo apt upgrade -y

sudo dnf update -y

Install OpenLiteSpeed and dependencies

Install OpenLiteSpeed web server along with the development tools and libraries needed for ModSecurity compilation.

wget -O - https://repo.litespeed.sh | sudo bash
sudo apt install -y openlitespeed lsphp82 lsphp82-mysql
sudo apt install -y build-essential libpcre3-dev zlib1g-dev libssl-dev libxml2-dev libpcre++-dev libgd-dev

sudo rpm -Uvh http://rpms.litespeedtech.com/centos/litespeed-repo-1.3-1.el8.noarch.rpm
sudo dnf install -y openlitespeed lsphp82 lsphp82-mysql
sudo dnf groupinstall -y "Development Tools"
sudo dnf install -y pcre-devel openssl-devel libxml2-devel gd-devel

Install ModSecurity 3 library

Download and compile ModSecurity 3 from source to get the latest security features and performance improvements.

cd /tmp
git clone https://github.com/SpiderLabs/ModSecurity.git
cd ModSecurity
git checkout v3.0.12
git submodule init
git submodule update

Configure and compile ModSecurity

Build ModSecurity with optimizations enabled and install it to the system directories.

./build.sh
./configure --enable-pcre-study --enable-pcre-jit
make -j$(nproc)
sudo make install
sudo ldconfig

Install ModSecurity connector for OpenLiteSpeed

Clone and build the ModSecurity connector that integrates with OpenLiteSpeed's module system.

cd /tmp
git clone https://github.com/SpiderLabs/ModSecurity-nginx.git
cd ModSecurity-nginx
git checkout v1.0.3

Download OWASP Core Rule Set

Install the OWASP Core Rule Set (CRS) which provides comprehensive protection against common web application attacks.

sudo mkdir -p /usr/local/modsecurity
cd /usr/local/modsecurity
sudo git clone https://github.com/coreruleset/coreruleset.git
sudo mv coreruleset/crs-setup.conf.example coreruleset/crs-setup.conf
sudo mv coreruleset/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf.example coreruleset/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf
sudo mv coreruleset/rules/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf.example coreruleset/rules/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf

Create ModSecurity main configuration

Configure ModSecurity with security rules, logging, and performance settings tailored for production use.

# ModSecurity Configuration
SecRuleEngine On
SecRequestBodyAccess On
SecResponseBodyAccess Off
SecRequestBodyLimit 13107200
SecRequestBodyNoFilesLimit 131072
SecRequestBodyLimitAction Reject
SecPcreMatchLimit 250000
SecPcreMatchLimitRecursion 250000

Audit Log Configuration
SecAuditEngine RelevantOnly
SecAuditLogRelevantStatus "^(?:5|4(?!04))"
SecAuditLogParts ABIJDEFHZ
SecAuditLogType Serial
SecAuditLog /var/log/openlitespeed/modsec_audit.log

Debug Log
SecDebugLog /var/log/openlitespeed/modsec_debug.log
SecDebugLogLevel 3

Temporary file configuration
SecTmpDir /tmp/
SecDataDir /tmp/

Content-Type header configuration
SecRule REQUEST_HEADERS:Content-Type "^(?:application(?:/soap\+|/)|text/)xml" \
     "id:'200000',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=XML"

SecRule REQUEST_HEADERS:Content-Type "^application/json" \
     "id:'200001',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=JSON"

SecRule REQUEST_HEADERS:Content-Type "^application/x-www-form-urlencoded" \
     "id:'200002',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=URLENCODED"

SecRule REQUEST_HEADERS:Content-Type "^multipart/form-data" \
     "id:'200003',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=MULTIPART"

Include OWASP Core Rule Set
Include /usr/local/modsecurity/coreruleset/crs-setup.conf
Include /usr/local/modsecurity/coreruleset/rules/*.conf

Configure OpenLiteSpeed virtual host security

Create a secure virtual host configuration with ModSecurity integration and rate limiting rules.

docRoot                   /usr/local/lsws/Example/html/
vhDomain                  example.com
allowSymbolLink           1
enableScript              1
restrained                0
setUIDMode                0

Security Headers
extraHeaders {
  X-Frame-Options SAMEORIGIN
  X-Content-Type-Options nosniff
  X-XSS-Protection "1; mode=block"
  Referrer-Policy strict-origin-when-cross-origin
  Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline';"
}

Rate Limiting Configuration
accessControl {
  allow                   *
}

Request Rate Limiting
requestLimitCtrl {
  staticReqPerSec         20
  dynReqPerSec            10
  outBandwidth            1024
  inBandwidth             1024
}

Connection Limits
connectionCtrl {
  staticReqPerSec         50
  dynReqPerSec            25
  reqRate                 0
  reqUnit                 1
  outBandwidth            0
  inBandwidth             0
}

ModSecurity Configuration
moduleEnabled {
  cache                   0
  esi                     0
  expires                 1
  headers                 1
  uploadprogress          0
  modsecurity             1
}

moduleSecurityConf {
  modsec_rules            /usr/local/modsecurity/modsecurity.conf
  modsec_rules_remote     0
  modsec_rules_timeout    5
}

scripthandler {
  add                     lsphp82 php
}

phpIniOverride  {
  php_value upload_max_filesize 32M
  php_value post_max_size 32M
  php_value max_execution_time 300
  php_value max_input_vars 3000
}

Configure advanced rate limiting rules

Create comprehensive rate limiting and DDoS protection rules that block aggressive behavior while allowing legitimate traffic.

# Global Rate Limiting Configuration
requestLimitCtrl {
  staticReqPerSec         100
  dynReqPerSec            50
  outBandwidth            10240
  inBandwidth             10240
}

Connection Control
connectionCtrl {
  maxConnections          2000
  maxSSLConnections       1000
  connTimeout             300
  maxKeepAliveReq         1000
  keepAliveTimeout        5
  sndBufSize              0
  rcvBufSize              0
}

Anti-DDoS Configuration
perClientConnLimit {
  staticReqPerSec         20
  dynReqPerSec            10
  outBandwidth            1024
  inBandwidth             1024
  softLimit               500
  hardLimit               600
  gracePeriod             15
  banPeriod               300
}

IP-based Throttling
accounting {
  staticReqPerSec         50
  dynReqPerSec            25
  outBandwidth            5120
  inBandwidth             2048
  reqRate                 20
  reqUnit                 1
  respBuffer              0
}

Request Filtering
requestFilter {
  enableCtxControl        1
  ctxTimeout              60
  reqTimeOut              300
  respTimeOut             300
}

Create custom security rules

Add application-specific security rules that complement the OWASP Core Rule Set with custom threat detection.

# Custom ModSecurity Rules for Enhanced Protection

Block common attack patterns
SecRule REQUEST_URI "@detectSQLi" \
    "id:100001,\
    phase:2,\
    block,\
    t:urlDecodeUni,\
    t:htmlEntityDecode,\
    t:normalisePathWin,\
    msg:'SQL Injection Attack Detected',\
    logdata:'Matched Data: %{MATCHED_VAR} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',\
    tag:'application-multi',\
    tag:'language-multi',\
    tag:'attack-sqli',\
    tag:'paranoia-level/1',\
    ver:'OWASP_CRS/3.3.5',\
    severity:'CRITICAL',\
    setvar:'tx.anomaly_score_pl1=+%{tx.critical_anomaly_score}'"

Rate limiting for specific endpoints
SecRule REQUEST_URI "@beginsWith /wp-login.php" \
    "id:100002,\
    phase:2,\
    pass,\
    setvar:'ip.login_attempts=+1',\
    expirevar:'ip.login_attempts=300',\
    nolog"

SecRule IP:LOGIN_ATTEMPTS "@gt 5" \
    "id:100003,\
    phase:2,\
    deny,\
    status:429,\
    msg:'Rate limit exceeded for login attempts',\
    logdata:'IP: %{REMOTE_ADDR} exceeded login rate limit',\
    setvar:'ip.login_block=1',\
    expirevar:'ip.login_block=1800'"

Block known malicious user agents
SecRule REQUEST_HEADERS:User-Agent "@pmFromFile /usr/local/modsecurity/malicious-ua.txt" \
    "id:100004,\
    phase:1,\
    block,\
    msg:'Malicious User Agent Detected',\
    logdata:'User-Agent: %{MATCHED_VAR}',\
    tag:'attack-reputation-scanner',\
    severity:'WARNING'"

Geo-blocking (example for specific countries)
SecRule REMOTE_ADDR "@geoLookup" \
    "id:100005,\
    phase:1,\
    pass,\
    nolog,\
    setvar:'tx.country_code=%{geo.country_code}'"

SecRule TX:COUNTRY_CODE "@in CN RU" \
    "id:100006,\
    phase:1,\
    deny,\
    status:403,\
    msg:'Access denied from restricted country',\
    logdata:'Country: %{tx.country_code}, IP: %{REMOTE_ADDR}'"

File upload restrictions
SecRule FILES_TMPNAMES "@inspectFile /usr/local/modsecurity/util/av.sh" \
    "id:100007,\
    phase:2,\
    block,\
    msg:'Malicious file upload detected',\
    logdata:'File: %{MATCHED_VAR}'"

Slow HTTP attack protection
SecRule &REQUEST_HEADERS:Content-Length "@eq 0" \
    "id:100008,\
    phase:1,\
    pass,\
    nolog,\
    setvar:'ip.slow_dos_counter=+1',\
    expirevar:'ip.slow_dos_counter=60'"

SecRule IP:SLOW_DOS_COUNTER "@gt 30" \
    "id:100009,\
    phase:1,\
    deny,\
    status:429,\
    msg:'Slow DoS attack detected',\
    setvar:'ip.slow_dos_block=1',\
    expirevar:'ip.slow_dos_block=600'"

Create malicious user agent blocklist

Maintain a list of known malicious user agents and scanning tools to block automated attacks.

Nikto
Sqlmap
WPScan
DirBuster
ZmEu
LWP::Simple
BBBike
wget
curl/7.0
libwww-perl
python-requests
Masscan
Zmap
Nmap Scripting Engine
OpenVAS
w3af
acunetix
havij
fimap
commix
injectx
vega
burpsuite
owasp zap
Apache-HttpClient

Configure security event monitoring

Set up comprehensive logging and create monitoring scripts to track security events and potential threats.

sudo mkdir -p /var/log/openlitespeed
sudo touch /var/log/openlitespeed/modsec_audit.log
sudo touch /var/log/openlitespeed/modsec_debug.log
sudo touch /var/log/openlitespeed/security_events.log
sudo chown lsadm:lsadm /var/log/openlitespeed/modsec_*.log
sudo chmod 644 /var/log/openlitespeed/modsec_*.log

Create security monitoring script

Build an automated monitoring system that parses security logs and sends alerts when threats are detected.

#!/bin/bash

ModSecurity Log Monitor Script
LOG_FILE="/var/log/openlitespeed/modsec_audit.log"
ALERT_EMAIL="admin@example.com"
TMP_FILE="/tmp/modsec_alerts.tmp"
LAST_CHECK="/var/lib/modsecurity/last_check"

Create state directory
sudo mkdir -p /var/lib/modsecurity
sudo chown lsadm:lsadm /var/lib/modsecurity

Function to send alert
send_alert() {
    local subject="$1"
    local message="$2"
    echo "$message" | mail -s "$subject" "$ALERT_EMAIL"
    logger "ModSecurity Alert: $subject"
}

Check for new security events
if [ -f "$LAST_CHECK" ]; then
    LAST_TIME=$(cat "$LAST_CHECK")
else
    LAST_TIME="1 hour ago"
fi

Find new critical events
find "$LOG_FILE" -newer "$LAST_CHECK" 2>/dev/null | while read -r line; do
    if echo "$line" | grep -q "severity.CRITICAL\|severity.ERROR"; then
        ATTACK_TYPE=$(echo "$line" | grep -oP 'msg:"\K[^"]*')
        SOURCE_IP=$(echo "$line" | grep -oP 'client: \K[^,]*')
        TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
        
        echo "[$TIMESTAMP] Critical security event detected" >> "$TMP_FILE"
        echo "Attack Type: $ATTACK_TYPE" >> "$TMP_FILE"
        echo "Source IP: $SOURCE_IP" >> "$TMP_FILE"
        echo "Details: $line" >> "$TMP_FILE"
        echo "---" >> "$TMP_FILE"
    fi
done

Send alerts if events found
if [ -s "$TMP_FILE" ]; then
    send_alert "ModSecurity Critical Events Detected" "$(cat $TMP_FILE)"
    rm -f "$TMP_FILE"
fi

Update last check timestamp
date > "$LAST_CHECK"

Generate daily statistics
if [ "$(date +%H:%M)" = "09:00" ]; then
    YESTERDAY=$(date -d "yesterday" +%Y-%m-%d)
    STATS_FILE="/tmp/modsec_daily_stats.tmp"
    
    echo "ModSecurity Daily Report - $YESTERDAY" > "$STATS_FILE"
    echo "=====================================" >> "$STATS_FILE"
    echo "" >> "$STATS_FILE"
    
    # Count attacks by type
    echo "Top Attack Types:" >> "$STATS_FILE"
    grep "$YESTERDAY" "$LOG_FILE" | grep -oP 'attack-\K[^,]*' | sort | uniq -c | sort -nr | head -10 >> "$STATS_FILE"
    echo "" >> "$STATS_FILE"
    
    # Top source IPs
    echo "Top Source IPs:" >> "$STATS_FILE"
    grep "$YESTERDAY" "$LOG_FILE" | grep -oP 'client: \K[^,]*' | sort | uniq -c | sort -nr | head -10 >> "$STATS_FILE"
    echo "" >> "$STATS_FILE"
    
    # Blocked requests count
    BLOCKED_COUNT=$(grep -c "$YESTERDAY.*block" "$LOG_FILE")
    echo "Total Blocked Requests: $BLOCKED_COUNT" >> "$STATS_FILE"
    
    send_alert "ModSecurity Daily Statistics" "$(cat $STATS_FILE)"
    rm -f "$STATS_FILE"
fi

Make monitoring script executable and schedule

Set proper permissions and create a cron job to run the security monitoring script regularly.

sudo chmod +x /usr/local/bin/modsec-monitor.sh
sudo chown lsadm:lsadm /usr/local/bin/modsec-monitor.sh

Add to crontab for regular monitoring
(crontab -l 2>/dev/null; echo "/10    * /usr/local/bin/modsec-monitor.sh >/dev/null 2>&1") | crontab -

Configure log rotation

Set up log rotation to prevent security logs from consuming excessive disk space while maintaining historical data.

/var/log/openlitespeed/modsec_*.log {
    daily
    missingok
    rotate 30
    compress
    delaycompress
    notifempty
    create 644 lsadm lsadm
    postrotate
        /usr/local/lsws/bin/lshttpd -k usr1 >/dev/null 2>&1 || true
    endscript
}

Restart OpenLiteSpeed with security configuration

Apply all security configurations by restarting OpenLiteSpeed and enabling the new WAF and rate limiting rules.

sudo systemctl restart lsws
sudo systemctl enable lsws
sudo systemctl status lsws

Verify your setup

Test the ModSecurity WAF installation and rate limiting functionality to ensure proper protection.

# Check OpenLiteSpeed status
sudo systemctl status lsws

Verify ModSecurity is loaded
sudo /usr/local/lsws/bin/lshttpd -t

Test WAF with a simulated SQL injection
curl -X GET "http://example.com/index.php?id=1' OR '1'='1"

Check security logs for blocked request
sudo tail -f /var/log/openlitespeed/modsec_audit.log

Test rate limiting
for i in {1..25}; do curl -s http://example.com/ > /dev/null; done

Verify custom rules are active
grep -c "Custom ModSecurity Rules" /usr/local/modsecurity/custom-rules.conf

Configure alerting integration

Integrate security monitoring with external alerting systems for immediate threat response.

Install and configure mail system

Set up email notifications for security alerts using a lightweight mail transfer agent.

sudo apt install -y postfix mailutils
sudo dpkg-reconfigure postfix

sudo dnf install -y postfix mailx
sudo systemctl enable --now postfix

Create Slack webhook integration

Add Slack notifications for real-time security alerts to your team communication channel.

#!/bin/bash

Slack notification script for ModSecurity alerts
SLACK_WEBHOOK="https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK"
CHANNEL="#security-alerts"
USERNAME="ModSecurity-Bot"

send_slack_alert() {
    local message="$1"
    local color="$2"
    
    payload=$(cat <Usage: ./slack-notify.sh "Alert message" "danger"
send_slack_alert "$1" "${2:-warning}"

Configure fail2ban integration

Integrate ModSecurity with fail2ban for automatic IP blocking based on attack patterns.

[Definition]
failregex = ModSecurity: Access denied with code \d+ \(phase \d+\)\. \[file "."\] \[line "\d+"\] \[id "\d+"\] \[rev "\d+"\] \[msg "."\] \[data "."\] \[severity "."\] \[ver "."\] \[maturity "\d+"\] \[accuracy "\d+"\] \[hostname "."\] \[uri "."\] \[unique_id "."\] \[ref ".*"\] 

ignoreregex =

Create fail2ban jail for ModSecurity

Configure automatic IP banning for repeated security violations detected by ModSecurity.

[modsecurity]
enabled = true
port = http,https
filter = modsecurity
logpath = /var/log/openlitespeed/modsec_audit.log
maxretry = 3
bantime = 3600
findtime = 600
action = iptables-multiport[name=ModSecurity, port="http,https", protocol=tcp]
         sendmail-whois[name=ModSecurity, dest=admin@example.com, sender=fail2ban@example.com]

Performance tuning

Optimize ModSecurity and rate limiting settings for maximum security with minimal performance impact.

Tune ModSecurity performance settings

Adjust ModSecurity configuration to balance security coverage with server performance.

# Performance optimization for ModSecurity

Reduce rule processing overhead
SecRuleEngine On
SecRequestBodyAccess On
SecResponseBodyAccess Off

Optimize memory usage
SecRequestBodyLimit 10485760
SecRequestBodyNoFilesLimit 131072
SecRequestBodyInMemoryLimit 131072

Tune regex limits
SecPcreMatchLimit 100000
SecPcreMatchLimitRecursion 100000

Disable unnecessary features in production
SecArgumentSeparator &
SecCookieFormat 0
SecUnicodeMapFile unicode.mapping 20127

Optimize collection storage
SecCollectionTimeout 600
SecHttpBlKey whdkfieyhtnf

Cache optimization
SecRuleUpdateActionById 901320 "pass,nolog"
SecRuleUpdateActionById 901330 "pass,nolog"
SecRuleUpdateActionById 901340 "pass,nolog"

Whitelist trusted IPs (adjust as needed)
SecRule REMOTE_ADDR "@ipMatch 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16" \
    "id:999001,\
    phase:1,\
    pass,\
    nolog,\
    ctl:ruleEngine=off"

Exclude static files from inspection
SecRule REQUEST_FILENAME "@rx \.(jpg|jpeg|png|gif|css|js|ico|pdf|zip)$" \
    "id:999002,\
    phase:1,\
    pass,\
    nolog,\
    ctl:ruleEngine=off"

Configure caching for better performance

Enable OpenLiteSpeed caching to reduce the load on security processing while maintaining protection.

# Cache Configuration for Security Performance
cache {
  enableCache              1
  cacheStorePath           /tmp/lshttpd/cache
  cacheSizeMax             200M
  enablePublicCache        1
  enablePrivateCache       0
  checkPrivateCache        1
  checkPublicCache         1
  maxCacheObjSize          10M
  maxStaleAge             200
  qsCache                 1
  reqCookieCache          1
  respCookieCache         1
  ignoreReqCacheCtrl      1
  ignoreRespCacheCtrl     0
  expireInSeconds         3600
  enableStale             0
  staleAge               200
}

Cache policy for static resources
cachepolicy {
  enableCache              1
  expireInSeconds         86400
  noCacheRequestHeader    "Authorization,Cookie"
  noCacheResponseHeader   "Set-Cookie,Vary,*"
  enablePublicCache        1
  maxStaleAge            3600
}

Common issues

Symptom	Cause	Fix
ModSecurity not blocking attacks	Rules not loaded properly	Check `/var/log/openlitespeed/modsec_debug.log` and verify include paths
False positive blocks legitimate traffic	Overly aggressive rule configuration	Review and whitelist specific IPs or patterns in custom rules
High server load with WAF enabled	Inefficient rule processing	Tune `SecPcreMatchLimit` and disable response body inspection
Rate limiting not working	Configuration syntax error	Check OpenLiteSpeed error log and validate configuration with `lshttpd -t`
Security logs not rotating	Incorrect logrotate configuration	Verify file permissions and test with `logrotate -d /etc/logrotate.d/modsecurity`
Email alerts not sending	Mail system not configured	Test mail system with `echo "test" \| mail -s "test" admin@example.com`

Important: Always test WAF rules in a staging environment before deploying to production. False positives can block legitimate users and impact business operations.

Next steps

Running this in production?

Want this handled for you? Setting up comprehensive WAF protection is complex, but maintaining rule updates, monitoring false positives, and responding to security events 24/7 is the bigger challenge. See how we run infrastructure like this for European SaaS and e-commerce teams.

Automated install script

Run this to automate the entire setup

install.sh

#!/usr/bin/env bash

set -euo pipefail

# Colors for output
readonly RED='\033[0;31m'
readonly GREEN='\033[0;32m'
readonly YELLOW='\033[1;33m'
readonly NC='\033[0m' # No Color

# Global variables
MODSEC_VERSION="3.0.12"
CONNECTOR_VERSION="1.0.3"
TOTAL_STEPS=10

# Usage function
usage() {
    echo "Usage: $0 [domain]"
    echo "  domain: Optional domain name for SSL configuration"
    exit 1
}

# Logging functions
log_info() {
    echo -e "${GREEN}[INFO]${NC} $1"
}

log_warn() {
    echo -e "${YELLOW}[WARN]${NC} $1"
}

log_error() {
    echo -e "${RED}[ERROR]${NC} $1"
}

# Cleanup function for errors
cleanup() {
    log_error "Installation failed. Cleaning up..."
    rm -rf /tmp/ModSecurity /tmp/ModSecurity-nginx
    systemctl stop lsws 2>/dev/null || true
}

trap cleanup ERR

# Check if running as root
check_root() {
    if [[ $EUID -ne 0 ]]; then
        log_error "This script must be run as root or with sudo"
        exit 1
    fi
}

# Detect distribution
detect_distro() {
    if [ ! -f /etc/os-release ]; then
        log_error "/etc/os-release not found. Cannot detect distribution."
        exit 1
    fi
    
    . /etc/os-release
    case "$ID" in
        ubuntu|debian)
            PKG_MGR="apt"
            PKG_UPDATE="apt update && apt upgrade -y"
            PKG_INSTALL="apt install -y"
            BUILD_DEPS="build-essential libpcre3-dev zlib1g-dev libssl-dev libxml2-dev libpcre++-dev libgd-dev git automake libtool autoconf"
            ;;
        almalinux|rocky|centos|rhel|ol|fedora)
            PKG_MGR="dnf"
            PKG_UPDATE="dnf update -y"
            PKG_INSTALL="dnf install -y"
            BUILD_DEPS="pcre-devel openssl-devel libxml2-devel gd-devel git automake libtool autoconf"
            ;;
        amzn)
            PKG_MGR="yum"
            PKG_UPDATE="yum update -y"
            PKG_INSTALL="yum install -y"
            BUILD_DEPS="pcre-devel openssl-devel libxml2-devel gd-devel git automake libtool autoconf"
            ;;
        *)
            log_error "Unsupported distribution: $ID"
            exit 1
            ;;
    esac
}

# Install distribution-specific packages
install_distro_packages() {
    case "$PKG_MGR" in
        apt)
            wget -O - https://repo.litespeed.sh | bash
            $PKG_INSTALL openlitespeed lsphp82 lsphp82-mysql
            ;;
        dnf|yum)
            rpm -Uvh http://rpms.litespeedtech.com/centos/litespeed-repo-1.3-1.el8.noarch.rpm || true
            $PKG_INSTALL openlitespeed lsphp82 lsphp82-mysql
            if [[ "$PKG_MGR" == "dnf" ]]; then
                dnf groupinstall -y "Development Tools"
            else
                yum groupinstall -y "Development Tools"
            fi
            ;;
    esac
}

# Main installation function
main() {
    local domain="${1:-}"
    
    check_root
    detect_distro
    
    log_info "[1/$TOTAL_STEPS] Updating system packages..."
    eval "$PKG_UPDATE"
    
    log_info "[2/$TOTAL_STEPS] Installing OpenLiteSpeed and dependencies..."
    install_distro_packages
    $PKG_INSTALL $BUILD_DEPS
    
    log_info "[3/$TOTAL_STEPS] Downloading ModSecurity $MODSEC_VERSION..."
    cd /tmp
    rm -rf ModSecurity
    git clone https://github.com/SpiderLabs/ModSecurity.git
    cd ModSecurity
    git checkout "v$MODSEC_VERSION"
    git submodule init
    git submodule update
    
    log_info "[4/$TOTAL_STEPS] Configuring and compiling ModSecurity..."
    ./build.sh
    ./configure --enable-pcre-study --enable-pcre-jit
    make -j$(nproc)
    make install
    ldconfig
    
    log_info "[5/$TOTAL_STEPS] Installing ModSecurity connector..."
    cd /tmp
    rm -rf ModSecurity-nginx
    git clone https://github.com/SpiderLabs/ModSecurity-nginx.git
    cd ModSecurity-nginx
    git checkout "v$CONNECTOR_VERSION"
    
    log_info "[6/$TOTAL_STEPS] Downloading OWASP Core Rule Set..."
    mkdir -p /usr/local/modsecurity
    cd /usr/local/modsecurity
    rm -rf coreruleset
    git clone https://github.com/coreruleset/coreruleset.git
    cp coreruleset/crs-setup.conf.example coreruleset/crs-setup.conf
    cp coreruleset/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf.example coreruleset/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf
    cp coreruleset/rules/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf.example coreruleset/rules/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf
    
    log_info "[7/$TOTAL_STEPS] Creating ModSecurity configuration..."
    cat > /usr/local/modsecurity/modsecurity.conf << 'EOF'
SecRuleEngine On
SecRequestBodyAccess On
SecResponseBodyAccess Off
SecRequestBodyLimit 13107200
SecRequestBodyNoFilesLimit 131072
SecRequestBodyLimitAction Reject
SecPcreMatchLimit 250000
SecPcreMatchLimitRecursion 250000

SecAuditEngine RelevantOnly
SecAuditLogRelevantStatus "^(?:5|4(?!04))"
SecAuditLogParts ABIJDEFHZ
SecAuditLogType Serial
SecAuditLog /usr/local/lsws/logs/modsec_audit.log

SecDebugLog /usr/local/lsws/logs/modsec_debug.log
SecDebugLogLevel 3

SecTmpDir /tmp/
SecDataDir /tmp/

SecRule REQUEST_HEADERS:Content-Type "^(?:application(?:/soap\+|/)|text/)xml" \
     "id:'200000',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=XML"

SecRule REQUEST_HEADERS:Content-Type "^application/json" \
     "id:'200001',phase:1,t:none,t:lowercase,pass,nolog,ctl:requestBodyProcessor=JSON"

Include /usr/local/modsecurity/coreruleset/crs-setup.conf
Include /usr/local/modsecurity/coreruleset/rules/*.conf
EOF
    
    log_info "[8/$TOTAL_STEPS] Setting up log directories and permissions..."
    mkdir -p /usr/local/lsws/logs
    chown -R lsws:lsws /usr/local/lsws/logs
    chmod 755 /usr/local/lsws/logs
    chown -R lsws:lsws /usr/local/modsecurity
    chmod -R 644 /usr/local/modsecurity/modsecurity.conf
    chmod -R 644 /usr/local/modsecurity/coreruleset/
    find /usr/local/modsecurity -type d -exec chmod 755 {} \;
    
    log_info "[9/$TOTAL_STEPS] Starting OpenLiteSpeed..."
    systemctl enable lsws
    systemctl start lsws
    
    # Configure firewall
    if command -v ufw >/dev/null 2>&1; then
        ufw --force enable
        ufw allow 80/tcp
        ufw allow 443/tcp
        ufw allow 7080/tcp
    elif command -v firewall-cmd >/dev/null 2>&1; then
        systemctl enable firewalld
        systemctl start firewalld
        firewall-cmd --permanent --add-service=http
        firewall-cmd --permanent --add-service=https
        firewall-cmd --permanent --add-port=7080/tcp
        firewall-cmd --reload
    fi
    
    log_info "[10/$TOTAL_STEPS] Verifying installation..."
    if systemctl is-active --quiet lsws; then
        log_info "OpenLiteSpeed is running successfully"
    else
        log_error "OpenLiteSpeed failed to start"
        exit 1
    fi
    
    if [ -f /usr/local/lib/libmodsecurity.so ]; then
        log_info "ModSecurity library installed successfully"
    else
        log_error "ModSecurity library not found"
        exit 1
    fi
    
    log_info "Installation completed successfully!"
    log_info "OpenLiteSpeed Admin Panel: https://$(hostname -I | awk '{print $1}'):7080"
    log_info "Default admin credentials: admin/123456"
    log_warn "Please change the default password immediately!"
    log_info "ModSecurity logs: /usr/local/lsws/logs/modsec_audit.log"
    
    if [[ -n "$domain" ]]; then
        log_info "Domain configured: $domain"
    fi
}

# Check if help is requested
if [[ "${1:-}" == "-h" ]] || [[ "${1:-}" == "--help" ]]; then
    usage
fi

main "$@"

Review the script before running. Execute with: bash install.sh

#OpenLiteSpeed #ModSecurity #WAF #DDoS protection #OWASP CRS #rate limiting #web security

Implement OpenLiteSpeed WAF and DDoS protection with ModSecurity 3 and rate limiting

Prerequisites

What this solves

Step-by-step installation

Update system packages

Install OpenLiteSpeed and dependencies

Install ModSecurity 3 library

Configure and compile ModSecurity

Install ModSecurity connector for OpenLiteSpeed

Download OWASP Core Rule Set

Create ModSecurity main configuration

Audit Log Configuration

Debug Log

Temporary file configuration

Content-Type header configuration

Include OWASP Core Rule Set

Configure OpenLiteSpeed virtual host security

Security Headers

Rate Limiting Configuration

Request Rate Limiting

Connection Limits

ModSecurity Configuration

Configure advanced rate limiting rules

Connection Control

Anti-DDoS Configuration

IP-based Throttling

Request Filtering

Create custom security rules

Block common attack patterns

Rate limiting for specific endpoints

Block known malicious user agents

Geo-blocking (example for specific countries)

File upload restrictions

Slow HTTP attack protection

Create malicious user agent blocklist

Configure security event monitoring

Create security monitoring script

ModSecurity Log Monitor Script

Create state directory

Function to send alert

Check for new security events

Find new critical events

Send alerts if events found

Update last check timestamp

Generate daily statistics

Make monitoring script executable and schedule

Add to crontab for regular monitoring

Configure log rotation

Restart OpenLiteSpeed with security configuration

Verify your setup

Verify ModSecurity is loaded

Test WAF with a simulated SQL injection

Check security logs for blocked request

Test rate limiting

Verify custom rules are active

Configure alerting integration

Install and configure mail system

Create Slack webhook integration

Slack notification script for ModSecurity alerts

Configure fail2ban integration

Create fail2ban jail for ModSecurity

Performance tuning

Tune ModSecurity performance settings

Reduce rule processing overhead

Optimize memory usage

Tune regex limits

Disable unnecessary features in production

Optimize collection storage

Cache optimization

Whitelist trusted IPs (adjust as needed)

Exclude static files from inspection

Configure caching for better performance

Cache policy for static resources

Common issues

Next steps

Running this in production?

Related tutorials

Configure Kubernetes network policies for enhanced cluster security

Implement OpenResty rate limiting and API protection with Lua middleware

Configure automated compliance scanning with OpenSCAP and audit reporting