Implement container security monitoring with Falco runtime detection

Update system packages

Start by updating your package manager to ensure you have the latest security updates.

sudo apt update && sudo apt upgrade -y
sudo apt install -y curl gnupg lsb-release

sudo dnf update -y
sudo dnf install -y curl gnupg2

Add Falco repository and install

Add the official Falco repository and install the Falco runtime security engine with kernel module support.

curl -fsSL https://falco.org/repo/falcosecurity-packages.asc | sudo gpg --dearmor -o /usr/share/keyrings/falco-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/falco-archive-keyring.gpg] https://download.falco.org/packages/deb stable main" | sudo tee -a /etc/apt/sources.list.d/falcosecurity.list
sudo apt update
sudo apt install -y falco

curl -fsSL https://falco.org/repo/falcosecurity-packages.asc | sudo gpg --import
cat > /etc/yum.repos.d/falcosecurity.repo << EOF
[falcosecurity]
name=Falco repository
baseurl=https://download.falco.org/packages/rpm
gpgcheck=1
gpgkey=https://falco.org/repo/falcosecurity-packages.asc
EOF
sudo dnf install -y falco

Configure Falco main settings

Configure Falco's main configuration file with optimized settings for container monitoring and JSON output for integration with monitoring systems.

rules_file:
  - /etc/falco/falco_rules.yaml
  - /etc/falco/falco_rules.local.yaml
  - /etc/falco/k8s_audit_rules.yaml
  - /etc/falco/rules.d

time_format_iso_8601: true
json_output: true
json_include_output_property: true
json_include_tags_property: true

log_stderr: true
log_syslog: false
log_level: info

priority: debug

buffered_outputs: true

outputs:
  rate: 1
  max_burst: 1000

syslog_output:
  enabled: true

file_output:
  enabled: true
  keep_alive: false
  filename: /var/log/falco/events.log

stdout_output:
  enabled: true

webserver:
  enabled: true
  listen_port: 8765
  k8s_healthz_endpoint: /healthz
  ssl_enabled: false
  ssl_certificate: /etc/falco/falco.pem

grpc:
  enabled: true
  bind_address: "0.0.0.0:5060"
  threadiness: 8

grpc_output:
  enabled: true

Create custom security rules

Define custom Falco rules for container-specific threats, suspicious activities, and compliance violations.

- rule: Unauthorized Container Privilege Escalation
  desc: Detect attempts to escalate privileges in containers
  condition: >
    spawned_process and container and
    ((proc.name in (sudo, su, setuid, setgid)) or
     (proc.args contains "--privileged") or
     (proc.args contains "--cap-add"))
  output: >
    Privilege escalation attempt in container (user=%user.name command=%proc.cmdline
    container_id=%container.id container_name=%container.name image=%container.image.repository)
  priority: CRITICAL
  tags: [container, privilege_escalation, security]

rule: Suspicious Network Activity in Container
  desc: Detect unusual network connections from containers
  condition: >
    inbound_outbound and container and
    ((fd.sport in (22, 23, 3389, 5900)) or
     (fd.dport in (22, 23, 3389, 5900)) or
     (fd.net contains "0.0.0.0"))
  output: >
    Suspicious network activity (user=%user.name command=%proc.cmdline connection=%fd.name
    container_id=%container.id container_name=%container.name image=%container.image.repository)
  priority: WARNING
  tags: [container, network, security]

rule: Container File System Tampering
  desc: Detect unauthorized file system modifications in containers
  condition: >
    open_write and container and
    (fd.name startswith "/etc/" or
     fd.name startswith "/bin/" or
     fd.name startswith "/sbin/" or
     fd.name startswith "/usr/bin/" or
     fd.name startswith "/usr/sbin/")
  output: >
    File system tampering detected (user=%user.name file=%fd.name command=%proc.cmdline
    container_id=%container.id container_name=%container.name image=%container.image.repository)
  priority: ERROR
  tags: [container, filesystem, security]

rule: Cryptocurrency Mining Activity
  desc: Detect potential cryptocurrency mining in containers
  condition: >
    spawned_process and container and
    (proc.name in (xmrig, cpuminer, cgminer, bfgminer, ethminer) or
     proc.cmdline contains "stratum+tcp" or
     proc.cmdline contains "cryptonight" or
     proc.cmdline contains "scrypt")
  output: >
    Cryptocurrency mining detected (user=%user.name command=%proc.cmdline
    container_id=%container.id container_name=%container.name image=%container.image.repository)
  priority: CRITICAL
  tags: [container, mining, security]

rule: Container Shell Access
  desc: Monitor shell access to containers for security auditing
  condition: >
    spawned_process and container and
    (proc.name in (bash, sh, zsh, fish, csh, tcsh) and
     proc.pname in (kubectl, docker, podman, crictl))
  output: >
    Shell access to container (user=%user.name shell=%proc.name command=%proc.cmdline
    container_id=%container.id container_name=%container.name image=%container.image.repository)
  priority: INFO
  tags: [container, shell, audit]

Configure log rotation

Set up log rotation for Falco events to prevent disk space issues and maintain historical security data.

sudo mkdir -p /var/log/falco
sudo touch /var/log/falco/events.log
sudo chown syslog:adm /var/log/falco/events.log
sudo chmod 640 /var/log/falco/events.log

/var/log/falco/*.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    create 640 syslog adm
    postrotate
        systemctl reload falco
    endscript
}

Enable and start Falco service

Enable Falco to start automatically on boot and start the service for immediate protection.

sudo systemctl enable falco
sudo systemctl start falco
sudo systemctl status falco

Install Falco Kubernetes integration

Deploy Falco as a DaemonSet in Kubernetes for cluster-wide container security monitoring.

kubectl create namespace falco
kubectl apply -f https://raw.githubusercontent.com/falcosecurity/charts/master/falco/values.yaml

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: falco
  namespace: falco
  labels:
    app.kubernetes.io/name: falco
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: falco
  template:
    metadata:
      labels:
        app.kubernetes.io/name: falco
    spec:
      serviceAccountName: falco
      hostNetwork: true
      hostPID: true
      containers:
      - name: falco
        image: falcosecurity/falco-no-driver:latest
        args:
          - /usr/bin/falco
          - --cri=/run/containerd/containerd.sock
          - --cri=/var/run/docker.sock
          - -K=/var/run/secrets/kubernetes.io/serviceaccount/token
          - -k=https://kubernetes.default
          - --k8s-node-name=$(FALCO_K8S_NODE_NAME)
        env:
        - name: FALCO_K8S_NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        securityContext:
          privileged: true
        volumeMounts:
        - mountPath: /host/var/run/docker.sock
          name: docker-socket
        - mountPath: /host/run/containerd/containerd.sock
          name: containerd-socket
        - mountPath: /host/dev
          name: dev-fs
        - mountPath: /host/proc
          name: proc-fs
          readOnly: true
        - mountPath: /host/boot
          name: boot-fs
          readOnly: true
        - mountPath: /host/lib/modules
          name: lib-modules
          readOnly: true
        - mountPath: /host/usr
          name: usr-fs
          readOnly: true
        - mountPath: /host/etc/os-release
          name: etc-os-release
          readOnly: true
        - mountPath: /etc/falco
          name: falco-config
      volumes:
      - name: docker-socket
        hostPath:
          path: /var/run/docker.sock
      - name: containerd-socket
        hostPath:
          path: /run/containerd/containerd.sock
      - name: dev-fs
        hostPath:
          path: /dev
      - name: proc-fs
        hostPath:
          path: /proc
      - name: boot-fs
        hostPath:
          path: /boot
      - name: lib-modules
        hostPath:
          path: /lib/modules
      - name: usr-fs
        hostPath:
          path: /usr
      - name: etc-os-release
        hostPath:
          path: /etc/os-release
      - name: falco-config
        configMap:
          name: falco-config
      tolerations:
      - operator: Exists
        effect: NoSchedule

kubectl apply -f falco-daemonset.yaml

Configure Falco exporter for Prometheus

Install and configure Falco exporter to send security metrics to Prometheus for monitoring and alerting.

wget https://github.com/falcosecurity/falco-exporter/releases/download/v0.8.7/falco-exporter_0.8.7_linux_amd64.tar.gz
tar -xzf falco-exporter_0.8.7_linux_amd64.tar.gz
sudo mv falco-exporter /usr/local/bin/
sudo chmod +x /usr/local/bin/falco-exporter

[Unit]
Description=Falco Prometheus Exporter
After=network.target falco.service
Requires=falco.service

[Service]
Type=simple
User=nobody
Group=nogroup
ExecStart=/usr/local/bin/falco-exporter \
  --client-socket=unix:///var/run/falco/falco.sock \
  --listen-address=0.0.0.0:9376 \
  --metrics-interval=15s
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable falco-exporter
sudo systemctl start falco-exporter

Configure Grafana dashboard

Import and configure a Grafana dashboard for visualizing Falco security events and metrics.

{
  "dashboard": {
    "id": null,
    "title": "Falco Security Monitoring",
    "description": "Container runtime security monitoring with Falco",
    "tags": ["falco", "security", "containers"],
    "timezone": "",
    "panels": [
      {
        "id": 1,
        "title": "Security Events by Priority",
        "type": "stat",
        "targets": [
          {
            "expr": "increase(falco_events_total[5m])",
            "legendFormat": "{{priority}}"
          }
        ],
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "thresholds"
            },
            "thresholds": {
              "steps": [
                {"color": "green", "value": null},
                {"color": "yellow", "value": 5},
                {"color": "red", "value": 10}
              ]
            }
          }
        },
        "gridPos": {"h": 8, "w": 12, "x": 0, "y": 0}
      },
      {
        "id": 2,
        "title": "Critical Security Events",
        "type": "table",
        "targets": [
          {
            "expr": "falco_events_total{priority=\"Critical\"}",
            "format": "table",
            "instant": true
          }
        ],
        "gridPos": {"h": 8, "w": 12, "x": 12, "y": 0}
      },
      {
        "id": 3,
        "title": "Events Timeline",
        "type": "timeseries",
        "targets": [
          {
            "expr": "rate(falco_events_total[5m])",
            "legendFormat": "Events per second"
          }
        ],
        "gridPos": {"h": 8, "w": 24, "x": 0, "y": 8}
      }
    ],
    "time": {
      "from": "now-1h",
      "to": "now"
    },
    "refresh": "30s"
  }
}

Set up alerting rules

Configure Prometheus alerting rules for critical Falco security events that require immediate attention.

groups:
name: falco-security
  rules:
  - alert: FalcoCriticalSecurityEvent
    expr: increase(falco_events_total{priority="Critical"}[5m]) > 0
    for: 0m
    labels:
      severity: critical
    annotations:
      summary: "Critical security event detected by Falco"
      description: "Falco detected {{ $value }} critical security events in the last 5 minutes"

  - alert: FalcoPrivilegeEscalation
    expr: increase(falco_events_total{rule="Unauthorized Container Privilege Escalation"}[10m]) > 0
    for: 0m
    labels:
      severity: critical
    annotations:
      summary: "Container privilege escalation detected"
      description: "Privilege escalation attempt detected in container environment"

  - alert: FalcoSuspiciousNetworkActivity
    expr: increase(falco_events_total{rule="Suspicious Network Activity in Container"}[15m]) > 5
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: "Multiple suspicious network connections detected"
      description: "{{ $value }} suspicious network activities detected in containers"

  - alert: FalcoCryptocurrencyMining
    expr: increase(falco_events_total{rule="Cryptocurrency Mining Activity"}[5m]) > 0
    for: 0m
    labels:
      severity: critical
    annotations:
      summary: "Cryptocurrency mining activity detected"
      description: "Potential cryptocurrency mining detected in container environment"

  - alert: FalcoFileSystemTampering
    expr: increase(falco_events_total{rule="Container File System Tampering"}[10m]) > 3
    for: 2m
    labels:
      severity: warning
    annotations:
      summary: "File system tampering detected"
      description: "{{ $value }} file system modifications detected in containers"