Configure auditd with Elasticsearch and Kibana for compliance reporting

Update system packages

Start by updating your package manager to ensure you get the latest versions of all components.

sudo apt update && sudo apt upgrade -y

sudo dnf update -y

Install auditd and audit utilities

Install the Linux audit framework and utilities for managing audit rules and logs.

sudo apt install -y auditd audispd-plugins

sudo dnf install -y audit audit-libs

Configure comprehensive audit rules

Create audit rules to monitor file access, system calls, network connections, and authentication events for compliance reporting.

# File access monitoring for PCI DSS
-w /etc/passwd -p wa -k identity
-w /etc/group -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/sudoers -p wa -k privilege_escalation
-w /var/log/auth.log -p wa -k authentication
-w /var/log/secure -p wa -k authentication

System call monitoring
-a always,exit -F arch=b64 -S adjtimex -S settimeofday -k time-change
-a always,exit -F arch=b32 -S adjtimex -S settimeofday -S stime -k time-change
-a always,exit -F arch=b64 -S clock_settime -k time-change
-a always,exit -F arch=b32 -S clock_settime -k time-change
-w /etc/localtime -p wa -k time-change

Network configuration changes
-a always,exit -F arch=b64 -S sethostname -S setdomainname -k system-locale
-a always,exit -F arch=b32 -S sethostname -S setdomainname -k system-locale
-w /etc/issue -p wa -k system-locale
-w /etc/issue.net -p wa -k system-locale
-w /etc/hosts -p wa -k system-locale
-w /etc/sysconfig/network -p wa -k system-locale
-w /etc/sysconfig/network-scripts/ -p wa -k system-locale

Discretionary access control permission modification
-a always,exit -F arch=b64 -S chmod -S fchmod -S fchmodat -F auid>=500 -F auid!=4294967295 -k perm_mod
-a always,exit -F arch=b32 -S chmod -S fchmod -S fchmodat -F auid>=500 -F auid!=4294967295 -k perm_mod
-a always,exit -F arch=b64 -S chown -S fchown -S fchownat -S lchown -F auid>=500 -F auid!=4294967295 -k perm_mod
-a always,exit -F arch=b32 -S chown -S fchown -S fchownat -S lchown -F auid>=500 -F auid!=4294967295 -k perm_mod
-a always,exit -F arch=b64 -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid>=500 -F auid!=4294967295 -k perm_mod
-a always,exit -F arch=b32 -S setxattr -S lsetxattr -S fsetxattr -S removexattr -S lremovexattr -S fremovexattr -F auid>=500 -F auid!=4294967295 -k perm_mod

Unauthorized file access attempts
-a always,exit -F arch=b64 -S creat -S open -S openat -S truncate -S ftruncate -F exit=-EACCES -F auid>=500 -F auid!=4294967295 -k access
-a always,exit -F arch=b32 -S creat -S open -S openat -S truncate -S ftruncate -F exit=-EACCES -F auid>=500 -F auid!=4294967295 -k access
-a always,exit -F arch=b64 -S creat -S open -S openat -S truncate -S ftruncate -F exit=-EPERM -F auid>=500 -F auid!=4294967295 -k access
-a always,exit -F arch=b32 -S creat -S open -S openat -S truncate -S ftruncate -F exit=-EPERM -F auid>=500 -F auid!=4294967295 -k access

Privileged commands monitoring
-a always,exit -F path=/usr/bin/sudo -F perm=x -F auid>=500 -F auid!=4294967295 -k privileged
-a always,exit -F path=/usr/bin/su -F perm=x -F auid>=500 -F auid!=4294967295 -k privileged
-a always,exit -F path=/bin/mount -F perm=x -F auid>=500 -F auid!=4294967295 -k privileged
-a always,exit -F path=/bin/umount -F perm=x -F auid>=500 -F auid!=4294967295 -k privileged

Make the configuration immutable
-e 2

Configure auditd service settings

Configure audit daemon settings for proper log rotation, remote logging capabilities, and performance optimization.

# Log file configuration
log_file = /var/log/audit/audit.log
log_format = RAW
log_group = adm
priority_boost = 4
flush = INCREMENTAL_ASYNC
freq = 50

Disk space management
max_log_file = 50
num_logs = 10
max_log_file_action = ROTATE
space_left = 75
space_left_action = SYSLOG
admin_space_left = 50
admin_space_left_action = SUSPEND
disk_full_action = SUSPEND
disk_error_action = SUSPEND

Network settings for centralized logging
tcp_listen_port = 60
tcp_listen_queue = 5
tcp_max_per_addr = 1
tcp_client_ports = 1024-65535
tcp_client_max_idle = 0

Performance settings
write_logs = yes
name_format = HOSTNAME
local_events = yes

Install Java for Elasticsearch

Elasticsearch requires Java 11 or later. Install OpenJDK which provides the necessary runtime environment.

sudo apt install -y openjdk-11-jdk

sudo dnf install -y java-11-openjdk-devel

Install Elasticsearch 8

Add the official Elastic repository and install Elasticsearch with security features enabled by default.

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list

sudo apt update
sudo apt install -y elasticsearch

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
cat > /etc/yum.repos.d/elasticsearch.repo << 'EOF'
[elasticsearch]
name=Elasticsearch repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=0
autorefresh=1
type=rpm-md
EOF
sudo dnf install -y --enablerepo=elasticsearch elasticsearch

Configure Elasticsearch for audit logging

Configure Elasticsearch with appropriate memory settings, cluster configuration, and security settings for audit log storage.

# Cluster configuration
cluster.name: audit-cluster
node.name: audit-node-1
network.host: 127.0.0.1
http.port: 9200
discovery.type: single-node

Security settings
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: certs/http.p12
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: certs/transport.p12
xpack.security.transport.ssl.truststore.path: certs/transport.p12

Index lifecycle management
action.destructive_requires_name: true
cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low: 85%
cluster.routing.allocation.disk.watermark.high: 90%
cluster.routing.allocation.disk.watermark.flood_stage: 95%

Set Elasticsearch memory limits

Configure JVM heap size to half of available RAM for optimal performance. This prevents OutOfMemory errors while leaving memory for the OS and other processes.

-Xms2g
-Xmx2g

Start and configure Elasticsearch

Enable and start Elasticsearch service, then configure the built-in users and create audit-specific indices.

sudo systemctl daemon-reload
sudo systemctl enable --now elasticsearch

Wait for service to start
sleep 30

Reset the elastic user password
sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic --batch

Install Kibana 8

Install Kibana for visualization and dashboard creation. Kibana will connect to Elasticsearch for audit log analysis.

sudo apt install -y kibana

sudo dnf install -y --enablerepo=elasticsearch kibana

Configure Kibana connection

Configure Kibana to connect to Elasticsearch with proper security settings and generate enrollment token.

# Generate enrollment token for Kibana
sudo /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana

Configure Kibana with the enrollment token (replace TOKEN with actual token)
sudo /usr/share/kibana/bin/kibana-setup --enrollment-token TOKEN

Configure Kibana settings

Set up Kibana configuration for audit log visualization with appropriate security and network settings.

server.port: 5601
server.host: "0.0.0.0"
server.name: "audit-kibana"
server.publicBaseUrl: "http://example.com:5601"

elasticsearch.hosts: ["https://127.0.0.1:9200"]
elasticsearch.serviceAccountToken: "AAEAAWVsYXN0aWMva2liYW5hL2tpYmFuYS1zZXJ2aWNlLWFjY291bnQ6..."

logging.appenders.file.type: file
logging.appenders.file.fileName: /var/log/kibana/kibana.log
logging.appenders.file.layout.type: json

i18n.locale: "en"

Install Filebeat for log shipping

Install Filebeat to ship audit logs from auditd to Elasticsearch with proper parsing and field mapping.

sudo apt install -y filebeat

sudo dnf install -y --enablerepo=elasticsearch filebeat

Configure Filebeat for audit logs

Configure Filebeat to parse audit logs, extract compliance-relevant fields, and ship them to Elasticsearch with proper indexing.

filebeat.inputs:
type: log
  enabled: true
  paths:
    - /var/log/audit/audit.log
  fields:
    logtype: audit
    compliance: true
  fields_under_root: true
  multiline.pattern: '^type='
  multiline.negate: true
  multiline.match: after
  processors:
    - decode_json_fields:
        fields: ["message"]
        target: ""
        overwrite_keys: true
    - timestamp:
        field: "@timestamp"
        layouts:
          - '2006-01-02T15:04:05.000Z'
          - '2006-01-02T15:04:05Z'
        test:
          - '2023-01-15T14:30:00.123Z'

output.elasticsearch:
  hosts: ["https://127.0.0.1:9200"]
  protocol: "https"
  username: "elastic"
  password: "YOUR_ELASTIC_PASSWORD"
  ssl.certificate_authorities: ["/etc/elasticsearch/certs/http_ca.crt"]
  index: "audit-logs-%{+yyyy.MM.dd}"
  template.name: "audit-logs"
  template.pattern: "audit-logs-*"
  template.settings:
    index.number_of_shards: 1
    index.number_of_replicas: 0
    index.mapping.total_fields.limit: 10000

setup.kibana:
  host: "http://127.0.0.1:5601"

logging.level: info
logging.to_files: true
logging.files:
  path: /var/log/filebeat
  name: filebeat
  keepfiles: 7
  permissions: 0644

processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~
  - dissect:
      tokenizer: 'type=%{audit_type} msg=audit(%{timestamp}:%{sequence}): %{audit_data}'
      field: "message"
      target_prefix: "audit"

Create Elasticsearch index template for audit logs

Create a proper index template with field mappings optimized for audit log analysis and compliance reporting.

curl -X PUT "https://127.0.0.1:9200/_index_template/audit-logs" \
  -u elastic:YOUR_ELASTIC_PASSWORD \
  -k \
  -H "Content-Type: application/json" \
  -d '{
    "index_patterns": ["audit-logs-*"],
    "template": {
      "settings": {
        "number_of_shards": 1,
        "number_of_replicas": 0,
        "index.lifecycle.name": "audit-policy",
        "index.lifecycle.rollover_alias": "audit-logs"
      },
      "mappings": {
        "properties": {
          "@timestamp": {"type": "date"},
          "audit.type": {"type": "keyword"},
          "audit.timestamp": {"type": "date"},
          "audit.sequence": {"type": "long"},
          "audit.uid": {"type": "keyword"},
          "audit.gid": {"type": "keyword"},
          "audit.auid": {"type": "keyword"},
          "audit.ses": {"type": "keyword"},
          "audit.comm": {"type": "keyword"},
          "audit.exe": {"type": "keyword"},
          "audit.key": {"type": "keyword"},
          "audit.success": {"type": "keyword"},
          "audit.exit": {"type": "keyword"},
          "audit.ppid": {"type": "long"},
          "audit.pid": {"type": "long"},
          "host.hostname": {"type": "keyword"},
          "logtype": {"type": "keyword"},
          "compliance": {"type": "boolean"}
        }
      }
    },
    "priority": 100,
    "_meta": {
      "description": "Template for audit logs with compliance fields"
    }
  }'

Create index lifecycle policy

Set up an index lifecycle management policy to automatically manage audit log indices based on retention requirements.

curl -X PUT "https://127.0.0.1:9200/_ilm/policy/audit-policy" \
  -u elastic:YOUR_ELASTIC_PASSWORD \
  -k \
  -H "Content-Type: application/json" \
  -d '{
    "policy": {
      "phases": {
        "hot": {
          "actions": {
            "rollover": {
              "max_size": "5GB",
              "max_age": "7d"
            },
            "set_priority": {
              "priority": 100
            }
          }
        },
        "warm": {
          "min_age": "7d",
          "actions": {
            "set_priority": {
              "priority": 50
            },
            "allocate": {
              "number_of_replicas": 0
            }
          }
        },
        "cold": {
          "min_age": "30d",
          "actions": {
            "set_priority": {
              "priority": 0
            },
            "allocate": {
              "number_of_replicas": 0
            }
          }
        },
        "delete": {
          "min_age": "2555d"
        }
      }
    }
  }'

Start all services

Enable and start auditd, Kibana, and Filebeat services to begin collecting and processing audit logs.

sudo systemctl enable --now auditd
sudo systemctl enable --now kibana
sudo systemctl enable --now filebeat

Load audit rules
sudo augenrules --load

Restart auditd to apply rules
sudo systemctl restart auditd

Create Kibana compliance dashboards

Import pre-built compliance dashboard templates and create custom visualizations for PCI DSS, HIPAA, and SOX reporting.

# Import Filebeat audit module dashboards
sudo filebeat setup --dashboards

Create custom compliance index patterns
curl -X POST "http://127.0.0.1:5601/api/saved_objects/index-pattern/audit-logs-*" \
  -H "kbn-xsrf: true" \
  -H "Content-Type: application/json" \
  -d '{
    "attributes": {
      "title": "audit-logs-*",
      "timeFieldName": "@timestamp",
      "fields": "[{\"name\":\"@timestamp\",\"type\":\"date\",\"searchable\":true,\"aggregatable\":true},{\"name\":\"audit.type\",\"type\":\"string\",\"searchable\":true,\"aggregatable\":true},{\"name\":\"audit.key\",\"type\":\"string\",\"searchable\":true,\"aggregatable\":true}]"
    }
  }'

Configure Kibana alerting for compliance violations

Set up Watcher alerts for compliance violations, failed authentication attempts, and unauthorized access patterns.

curl -X PUT "https://127.0.0.1:9200/_watcher/watch/failed_logins" \
  -u elastic:YOUR_ELASTIC_PASSWORD \
  -k \
  -H "Content-Type: application/json" \
  -d '{
    "trigger": {
      "schedule": {
        "interval": "5m"
      }
    },
    "input": {
      "search": {
        "request": {
          "search_type": "query_then_fetch",
          "indices": ["audit-logs-*"],
          "body": {
            "query": {
              "bool": {
                "must": [
                  {
                    "match": {
                      "audit.type": "USER_AUTH"
                    }
                  },
                  {
                    "match": {
                      "audit.success": "no"
                    }
                  },
                  {
                    "range": {
                      "@timestamp": {
                        "gte": "now-5m"
                      }
                    }
                  }
                ]
              }
            },
            "aggs": {
              "failed_attempts": {
                "cardinality": {
                  "field": "audit.uid"
                }
              }
            }
          }
        }
      }
    },
    "condition": {
      "compare": {
        "ctx.payload.aggregations.failed_attempts.value": {
          "gt": 5
        }
      }
    },
    "actions": {
      "send_email": {
        "email": {
          "profile": "standard",
          "to": ["admin@example.com"],
          "subject": "Security Alert: Multiple Failed Login Attempts",
          "body": "Alert: {{ctx.payload.aggregations.failed_attempts.value}} failed login attempts detected in the last 5 minutes. Please investigate immediately."
        }
      }
    }
  }'

Access Kibana web interface

Access Kibana through your web browser to create compliance dashboards. Use the elastic user credentials.

# Get Kibana enrollment token if needed
sudo /usr/share/kibana/bin/kibana-verification-code

Access Kibana at: http://your-server-ip:5601
Login with user: elastic
Password: (the password you reset earlier)

Create PCI DSS compliance dashboard

Create visualizations specifically for PCI DSS compliance requirements including access monitoring, authentication tracking, and system change detection.

Create HIPAA compliance dashboard

Build HIPAA-specific visualizations focusing on data access auditing, user activity monitoring, and breach detection patterns.

Create SOX compliance dashboard

Develop SOX compliance visualizations emphasizing financial system access, administrative changes, and segregation of duties monitoring.

Set up privilege escalation alerts

Configure alerts for unauthorized privilege escalation attempts and suspicious sudo usage patterns.

curl -X PUT "https://127.0.0.1:9200/_watcher/watch/privilege_escalation" \
  -u elastic:YOUR_ELASTIC_PASSWORD \
  -k \
  -H "Content-Type: application/json" \
  -d '{
    "trigger": {
      "schedule": {
        "interval": "2m"
      }
    },
    "input": {
      "search": {
        "request": {
          "indices": ["audit-logs-*"],
          "body": {
            "query": {
              "bool": {
                "must": [
                  {
                    "match": {
                      "audit.key": "privileged"
                    }
                  },
                  {
                    "range": {
                      "@timestamp": {
                        "gte": "now-2m"
                      }
                    }
                  }
                ]
              }
            },
            "aggs": {
              "user_attempts": {
                "terms": {
                  "field": "audit.uid",
                  "size": 10
                }
              }
            }
          }
        }
      }
    },
    "condition": {
      "compare": {
        "ctx.payload.hits.total.value": {
          "gt": 10
        }
      }
    },
    "actions": {
      "log_alert": {
        "logging": {
          "level": "warn",
          "text": "High privilege escalation activity detected: {{ctx.payload.hits.total.value}} events in 2 minutes"
        }
      }
    }
  }'

Configure file access violation alerts

Set up monitoring for unauthorized access attempts to sensitive files and directories.

curl -X PUT "https://127.0.0.1:9200/_watcher/watch/file_access_violations" \
  -u elastic:YOUR_ELASTIC_PASSWORD \
  -k \
  -H "Content-Type: application/json" \
  -d '{
    "trigger": {
      "schedule": {
        "interval": "1m"
      }
    },
    "input": {
      "search": {
        "request": {
          "indices": ["audit-logs-*"],
          "body": {
            "query": {
              "bool": {
                "must": [
                  {
                    "match": {
                      "audit.key": "access"
                    }
                  },
                  {
                    "match": {
                      "audit.success": "no"
                    }
                  },
                  {
                    "range": {
                      "@timestamp": {
                        "gte": "now-1m"
                      }
                    }
                  }
                ]
              }
            }
          }
        }
      }
    },
    "condition": {
      "compare": {
        "ctx.payload.hits.total.value": {
          "gte": 1
        }
      }
    },
    "actions": {
      "log_alert": {
        "logging": {
          "level": "error",
          "text": "Unauthorized file access attempt detected: {{ctx.payload.hits.total.value}} violations"
        }
      }
    }
  }'

Optimize audit rule performance

Fine-tune audit rules to reduce performance impact while maintaining compliance coverage.

# Exclude high-volume, low-value events
-a never,exit -F arch=b64 -S adjtimex -F auid=unset
-a never,exit -F arch=b32 -S adjtimex -F auid=unset

Reduce noise from system processes
-a never,exit -F auid=-1
-a never,exit -F auid=4294967295

Exclude non-compliance relevant directories
-a never,exit -F dir=/proc
-a never,exit -F dir=/sys
-a never,exit -F dir=/dev/pts

Rate limit high-frequency events
-a exit,always -F arch=b64 -S open -F success=0 -F auid>=500 -F rate=100

Configure log compression and archival

Set up automatic compression and archival of older audit logs to manage disk usage while maintaining compliance retention periods.

/var/log/audit/audit.log {
    missingok
    rotate 99
    daily
    notifempty
    create 0600 root root
    compress
    delaycompress
    postrotate
        /bin/kill -HUP cat /var/run/auditd.pid 2> /dev/null 2> /dev/null || true
    endscript
}