Install Graylog 5 with MongoDB and Elasticsearch

Set up Graylog 5 with MongoDB 8.0 and Elasticsearch 8 for centralized log collection, analysis, and alerting. Learn to configure secure inputs, data streams, and monitoring for production environments.

Prerequisites

At least 8GB RAM
50GB available disk space
Root or sudo access
Open ports 9000, 1514, 12201

What this solves

Centralized logging is critical for monitoring distributed systems, troubleshooting issues, and maintaining security compliance. This tutorial guides you through installing and configuring Graylog 5 with MongoDB 8.0 and Elasticsearch 8 to create a production-ready log management platform that can handle thousands of logs per second.

Step-by-step installation

Update system packages and install dependencies

Start by updating your system packages and installing the required dependencies for the Graylog stack.

sudo apt update && sudo apt upgrade -y
sudo apt install -y curl wget gnupg2 software-properties-common apt-transport-https ca-certificates lsb-release pwgen

sudo dnf update -y
sudo dnf install -y curl wget gnupg2 yum-utils ca-certificates pwgen

Install OpenJDK 17

Graylog requires Java 17 or later. Install OpenJDK 17 which provides the best compatibility.

sudo apt install -y openjdk-17-jre-headless
java -version

sudo dnf install -y java-17-openjdk-headless
java -version

Install MongoDB 8.0

MongoDB serves as Graylog's metadata storage. Add the official MongoDB repository and install version 8.0.

curl -fsSL https://www.mongodb.org/static/pgp/server-8.0.asc | sudo gpg --dearmor -o /usr/share/keyrings/mongodb-server-8.0.gpg
echo "deb [arch=amd64,arm64 signed-by=/usr/share/keyrings/mongodb-server-8.0.gpg] https://repo.mongodb.org/apt/ubuntu $(lsb_release -cs)/mongodb-org/8.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-8.0.list
sudo apt update
sudo apt install -y mongodb-org

sudo tee /etc/yum.repos.d/mongodb-org-8.0.repo << 'EOF'
[mongodb-org-8.0]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/9/mongodb-org/8.0/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-8.0.asc
EOF
sudo dnf install -y mongodb-org

Configure and start MongoDB

Enable and start MongoDB service, then configure it for optimal performance with Graylog.

sudo systemctl enable --now mongod
sudo systemctl status mongod

Configure MongoDB for Graylog usage by editing the configuration file:

storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true
  wiredTiger:
    engineConfig:
      cacheSizeGB: 1

net:
  port: 27017
  bindIp: 127.0.0.1

processManagement:
  timeZoneInfo: /usr/share/zoneinfo

sudo systemctl restart mongod

Install Elasticsearch 8

Elasticsearch stores and indexes the actual log messages. Install Elasticsearch 8 from the official repository.

curl -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
sudo apt update
sudo apt install -y elasticsearch

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
sudo tee /etc/yum.repos.d/elasticsearch.repo << 'EOF'
[elasticsearch]
name=Elasticsearch repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF
sudo dnf install -y elasticsearch

Configure Elasticsearch for Graylog

Configure Elasticsearch to work optimally with Graylog by disabling security features and setting proper cluster configuration.

cluster.name: graylog
node.name: graylog-node-1
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 127.0.0.1
http.port: 9200
discovery.type: single-node
xpack.security.enabled: false
xpack.security.enrollment.enabled: false
xpack.security.http.ssl.enabled: false
xpack.security.transport.ssl.enabled: false
action.auto_create_index: false
indices.query.bool.max_clause_count: 32768

Configure JVM heap size based on available memory (recommended: 50% of RAM, max 32GB):

-Xms2g
-Xmx2g

sudo systemctl enable --now elasticsearch
sudo systemctl status elasticsearch

Install Graylog 5

Add the Graylog repository and install the latest version of Graylog 5.

wget https://packages.graylog2.org/repo/packages/graylog-5.2-repository_latest.deb
sudo dpkg -i graylog-5.2-repository_latest.deb
sudo apt update
sudo apt install -y graylog-server

sudo rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-5.2-repository_latest.rpm
sudo dnf install -y graylog-server

Generate password hash and configure Graylog

Generate a secure password hash for the admin user and create the main Graylog configuration.

ADMIN_PASSWORD=$(pwgen -s 16 1)
echo "Admin password: $ADMIN_PASSWORD"
PASSWORD_SECRET=$(pwgen -s 96 1)
PASSWORD_HASH=$(echo -n $ADMIN_PASSWORD | sha256sum | cut -d" " -f1)
echo "Password hash: $PASSWORD_HASH"

Note: Save the generated admin password securely. You'll need it to log into the Graylog web interface.

is_leader = true
node_id_file = /etc/graylog/server/node-id
password_secret = REPLACE_WITH_PASSWORD_SECRET
root_password_sha2 = REPLACE_WITH_PASSWORD_HASH
root_username = admin
root_email = admin@example.com
root_timezone = UTC
bin_dir = /usr/share/graylog-server/bin
data_dir = /var/lib/graylog-server
plugin_dir = /usr/share/graylog-server/plugin
http_bind_address = 0.0.0.0:9000
http_publish_uri = http://203.0.113.10:9000/
http_enable_cors = true
elasticsearch_hosts = http://127.0.0.1:9200
rotation_strategy = count
elasticsearch_max_docs_per_index = 20000000
elasticsearch_max_number_of_indices = 20
retention_strategy = delete
elasticsearch_shards = 4
elasticsearch_replicas = 0
elasticsearch_index_prefix = graylog
allow_leading_wildcard_searches = false
allow_highlighting = false
elasticsearch_analyzer = standard
output_batch_size = 500
output_flush_interval = 1
output_fault_count_threshold = 5
output_fault_penalty_seconds = 30
processbuffer_processors = 5
outputbuffer_processors = 3
processor_wait_strategy = blocking
ring_size = 65536
inputbuffer_ring_size = 65536
inputbuffer_processors = 2
inputbuffer_wait_strategy = blocking
message_journal_enabled = true
message_journal_dir = /var/lib/graylog-server/journal
message_journal_max_age = 12h
message_journal_max_size = 5gb
message_journal_flush_age = 1m
message_journal_flush_interval = 1000000
message_journal_segment_age = 1h
message_journal_segment_size = 100mb
mongodb_uri = mongodb://localhost/graylog
mongodb_max_connections = 1000
mongodb_threads_allowed_to_block_multiplier = 5

Replace the placeholder values with your generated secrets:

sudo sed -i "s/REPLACE_WITH_PASSWORD_SECRET/$PASSWORD_SECRET/g" /etc/graylog/server/server.conf
sudo sed -i "s/REPLACE_WITH_PASSWORD_HASH/$PASSWORD_HASH/g" /etc/graylog/server/server.conf

Configure system limits and start Graylog

Set proper system limits for Graylog and start the service.

graylog soft nofile 64000
graylog hard nofile 64000

sudo systemctl enable --now graylog-server
sudo systemctl status graylog-server

Configure firewall rules

Open the necessary ports for Graylog web interface and log inputs.

sudo ufw allow 9000/tcp comment "Graylog Web Interface"
sudo ufw allow 1514/tcp comment "Graylog Syslog TCP"
sudo ufw allow 1514/udp comment "Graylog Syslog UDP"
sudo ufw allow 12201/tcp comment "Graylog GELF TCP"
sudo ufw allow 12201/udp comment "Graylog GELF UDP"

sudo firewall-cmd --permanent --add-port=9000/tcp --add-port=1514/tcp --add-port=1514/udp --add-port=12201/tcp --add-port=12201/udp
sudo firewall-cmd --reload

Create Graylog system inputs

Configure common log inputs through the web interface. First, access Graylog at http://your-server-ip:9000 and log in with username 'admin' and the generated password.

Create a Syslog UDP input:

Go to System → Inputs
Select "Syslog UDP" from dropdown
Click "Launch new input"
Set bind address to 0.0.0.0 and port to 1514
Name it "Syslog UDP Input"
Click "Launch Input"

Create a GELF UDP input for application logs:

Select "GELF UDP" from dropdown
Click "Launch new input"
Set bind address to 0.0.0.0 and port to 12201
Name it "GELF UDP Input"
Click "Launch Input"

Configure log streams and rules

Create streams to organize logs by source or application type. In the web interface:

Go to Streams → Create Stream
Name: "System Logs"
Description: "Operating system and service logs"
Add rule: Field "source" must match exactly "rsyslog"
Save and start stream

Create an application stream:

Name: "Application Logs"
Description: "Custom application logs"
Add rule: Field "facility" must match exactly "local0"
Save and start stream

Set up alerting and notifications

Configure email notifications for critical alerts. Go to System → Notifications and create an email notification:

Type: Email Alarm Callback
Title: "Email Notifications"
Configuration:
- Sender: graylog@example.com
- Recipients: admin@example.com
- Subject: "Graylog Alert: ${alert_definition.title}"
- SMTP Server: smtp.example.com
- SMTP Port: 587
- Use TLS/SSL: Yes

Create an alert condition:

Go to Alerts → Alert Conditions
Select your stream and create "Message Count Condition"
Set threshold: More than 100 messages in 5 minutes
Add the email notification callback

Implement SSL/TLS encryption

Secure Graylog web interface with SSL certificates. Generate a self-signed certificate for testing or use Let's Encrypt for production.

sudo mkdir -p /etc/graylog/server/ssl
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/graylog/server/ssl/graylog.key -out /etc/graylog/server/ssl/graylog.crt -subj "/C=US/ST=State/L=City/O=Organization/CN=example.com"
sudo chown -R graylog:graylog /etc/graylog/server/ssl
sudo chmod 600 /etc/graylog/server/ssl/graylog.key

Never use chmod 777. It gives every user on the system full access to your SSL keys. Use chmod 600 for private keys to restrict access to the owner only.

Update Graylog configuration for HTTPS:

http_bind_address = 0.0.0.0:9000
http_publish_uri = https://203.0.113.10:9000/
http_enable_tls = true
http_tls_cert_file = /etc/graylog/server/ssl/graylog.crt
http_tls_key_file = /etc/graylog/server/ssl/graylog.key

sudo systemctl restart graylog-server

Configure log rotation and retention

Set up index rotation and retention policies to manage disk space effectively. Update the server configuration:

rotation_strategy = time
elasticsearch_max_time_per_index = 1d
retention_strategy = delete
elasticsearch_max_number_of_indices = 30

This configuration keeps 30 days of logs with daily rotation. Adjust based on your requirements and available disk space.

sudo systemctl restart graylog-server

Configure rsyslog to send logs

Configure rsyslog forwarding

Configure the local rsyslog service to forward system logs to Graylog for centralized collection.

# Forward all logs to Graylog
. @@127.0.0.1:1514;RSYSLOG_SyslogProtocol23Format

sudo systemctl restart rsyslog
sudo systemctl status rsyslog

Performance monitoring and tuning

Monitor Graylog metrics

Enable Graylog's built-in metrics endpoint for monitoring performance and integrating with external monitoring systems like Prometheus and Grafana.

metrics_enable_console_reporter = false
metrics_enable_csv_reporter = false
metrics_enable_jmx_reporter = true
metrics_enable_graphite_reporter = false

Monitor key metrics through the web interface at System → Metrics or via JMX for external monitoring systems.

Optimize for high throughput

Tune Graylog for high-volume log processing by adjusting buffer sizes and processor counts based on your hardware.

processbuffer_processors = 8
outputbuffer_processors = 6
ring_size = 262144
inputbuffer_ring_size = 262144
inputbuffer_processors = 4
output_batch_size = 1000
output_flush_interval = 1

Adjust these values based on your CPU cores and expected log volume. Monitor system resources and Graylog throughput metrics to find optimal settings.

Verify your setup

sudo systemctl status mongod elasticsearch graylog-server
curl -X GET "http://127.0.0.1:9200/_cluster/health?pretty"
sudo tail -f /var/log/graylog-server/server.log
sudo netstat -tlnp | grep -E '(9000|1514|12201|27017|9200)'
logger -p local0.info "Test message from $(hostname) at $(date)"

Check the Graylog web interface for the test message. You should see logs appearing in your configured streams within a few seconds.

Common issues

Symptom	Cause	Fix
Graylog web interface not accessible	Service not running or firewall blocking	`sudo systemctl status graylog-server` and check firewall rules
"Elasticsearch cluster unavailable" error	Elasticsearch not running or misconfigured	`sudo systemctl restart elasticsearch` and verify cluster health
Logs not appearing in streams	Input not configured or stream rules incorrect	Check System → Inputs status and verify stream rules match log format
High memory usage	JVM heap sizes too large or small	Adjust heap sizes in `/etc/elasticsearch/jvm.options.d/heap.options`
MongoDB connection failed	MongoDB service down or config error	`sudo systemctl restart mongod` and check MongoDB logs
SSL certificate errors	Incorrect certificate permissions or paths	Verify certificate paths and use `chmod 600` for private keys

Next steps

Set up centralized logging with rsyslog and logrotate for more advanced log forwarding
Monitor Elasticsearch cluster with Prometheus and Grafana for comprehensive metrics
Configure Graylog high availability clustering for production resilience
Implement Graylog LDAP authentication for enterprise user management
Set up Graylog Sidecar for automated log collection from multiple servers

Automated install script

Run this to automate the entire setup

install.sh

#!/usr/bin/env bash

set -euo pipefail

# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'

# Configuration
GRAYLOG_PASSWORD="${1:-$(pwgen -s 96 1)}"
GRAYLOG_ROOT_PASSWORD="${2:-admin}"
GRAYLOG_VERSION="5.2"

# Helper functions
log_info() { echo -e "${GREEN}[INFO]${NC} $1"; }
log_warn() { echo -e "${YELLOW}[WARN]${NC} $1"; }
log_error() { echo -e "${RED}[ERROR]${NC} $1"; }

cleanup() {
    log_error "Installation failed. Check the logs above for details."
    exit 1
}

usage() {
    echo "Usage: $0 [password_secret] [root_password]"
    echo "  password_secret: Graylog password secret (auto-generated if not provided)"
    echo "  root_password: Graylog admin password (default: admin)"
    exit 1
}

# Trap errors
trap cleanup ERR

# Check if running as root or with sudo
if [[ $EUID -ne 0 ]]; then
    log_error "This script must be run as root or with sudo"
    exit 1
fi

# Auto-detect distribution
if [ -f /etc/os-release ]; then
    . /etc/os-release
    case "$ID" in
        ubuntu|debian)
            PKG_MGR="apt"
            PKG_UPDATE="apt update"
            PKG_INSTALL="apt install -y"
            PKG_UPGRADE="apt upgrade -y"
            ;;
        almalinux|rocky|centos|rhel|ol|fedora)
            PKG_MGR="dnf"
            PKG_UPDATE="dnf check-update || true"
            PKG_INSTALL="dnf install -y"
            PKG_UPGRADE="dnf update -y"
            ;;
        amzn)
            PKG_MGR="yum"
            PKG_UPDATE="yum check-update || true"
            PKG_INSTALL="yum install -y"
            PKG_UPGRADE="yum update -y"
            ;;
        *)
            log_error "Unsupported distribution: $ID"
            exit 1
            ;;
    esac
else
    log_error "Cannot detect distribution"
    exit 1
fi

log_info "Detected distribution: $ID"
log_info "Starting Graylog 5 installation..."

# Step 1: Update system and install dependencies
log_info "[1/8] Updating system packages..."
$PKG_UPDATE
$PKG_UPGRADE

log_info "Installing base dependencies..."
if [[ "$PKG_MGR" == "apt" ]]; then
    $PKG_INSTALL curl wget gnupg2 software-properties-common apt-transport-https ca-certificates lsb-release pwgen uuid-runtime
else
    $PKG_INSTALL curl wget gnupg2 yum-utils ca-certificates pwgen util-linux
fi

# Step 2: Install OpenJDK 17
log_info "[2/8] Installing OpenJDK 17..."
$PKG_INSTALL java-17-openjdk-headless

java -version

# Step 3: Install MongoDB 8.0
log_info "[3/8] Installing MongoDB 8.0..."
if [[ "$PKG_MGR" == "apt" ]]; then
    curl -fsSL https://www.mongodb.org/static/pgp/server-8.0.asc | gpg --dearmor -o /usr/share/keyrings/mongodb-server-8.0.gpg
    echo "deb [arch=amd64,arm64 signed-by=/usr/share/keyrings/mongodb-server-8.0.gpg] https://repo.mongodb.org/apt/ubuntu $(lsb_release -cs)/mongodb-org/8.0 multiverse" > /etc/apt/sources.list.d/mongodb-org-8.0.list
    $PKG_UPDATE
    $PKG_INSTALL mongodb-org
else
    cat > /etc/yum.repos.d/mongodb-org-8.0.repo << 'EOF'
[mongodb-org-8.0]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/8.0/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-8.0.asc
EOF
    $PKG_INSTALL mongodb-org
fi

systemctl enable mongod
systemctl start mongod

# Step 4: Install Elasticsearch 8
log_info "[4/8] Installing Elasticsearch 8..."
if [[ "$PKG_MGR" == "apt" ]]; then
    curl -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | gpg --dearmor -o /usr/share/keyrings/elastic-keyring.gpg
    echo "deb [signed-by=/usr/share/keyrings/elastic-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" > /etc/apt/sources.list.d/elastic-8.x.list
    $PKG_UPDATE
    $PKG_INSTALL elasticsearch
else
    rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
    cat > /etc/yum.repos.d/elasticsearch.repo << 'EOF'
[elasticsearch]
name=Elasticsearch repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=0
autorefresh=1
type=rpm-md
EOF
    $PKG_INSTALL --enablerepo=elasticsearch elasticsearch
fi

# Step 5: Configure Elasticsearch
log_info "[5/8] Configuring Elasticsearch..."
cat > /etc/elasticsearch/elasticsearch.yml << 'EOF'
cluster.name: graylog
action.auto_create_index: false
xpack.security.enabled: false
xpack.security.enrollment.enabled: false
xpack.security.http.ssl.enabled: false
xpack.security.transport.ssl.enabled: false
discovery.type: single-node
EOF

systemctl enable elasticsearch
systemctl start elasticsearch

# Wait for Elasticsearch to start
sleep 30

# Step 6: Install Graylog
log_info "[6/8] Installing Graylog..."
if [[ "$PKG_MGR" == "apt" ]]; then
    wget -q -O - https://packages.graylog2.org/repo/debian/keyring.gpg | apt-key add -
    echo "deb https://packages.graylog2.org/repo/debian/ stable ${GRAYLOG_VERSION}" > /etc/apt/sources.list.d/graylog.list
    $PKG_UPDATE
    $PKG_INSTALL graylog-server
else
    rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-${GRAYLOG_VERSION}-repository_latest.rpm
    $PKG_INSTALL graylog-server
fi

# Step 7: Configure Graylog
log_info "[7/8] Configuring Graylog..."
GRAYLOG_PASSWORD_SECRET=$(echo -n "$GRAYLOG_PASSWORD" | sha256sum | cut -d" " -f1)
GRAYLOG_ROOT_PASSWORD_SHA2=$(echo -n "$GRAYLOG_ROOT_PASSWORD" | sha256sum | cut -d" " -f1)
GRAYLOG_NODE_ID=$(uuidgen)

# Backup original config
cp /etc/graylog/server/server.conf /etc/graylog/server/server.conf.bak

# Update Graylog configuration
sed -i "s/^#*password_secret =.*/password_secret = $GRAYLOG_PASSWORD_SECRET/" /etc/graylog/server/server.conf
sed -i "s/^#*root_password_sha2 =.*/root_password_sha2 = $GRAYLOG_ROOT_PASSWORD_SHA2/" /etc/graylog/server/server.conf
sed -i "s/^#*node_id_file =.*/node_id_file = \/etc\/graylog\/server\/node-id/" /etc/graylog/server/server.conf
sed -i "s/^#*http_bind_address =.*/http_bind_address = 0.0.0.0:9000/" /etc/graylog/server/server.conf
sed -i "s/^#*elasticsearch_hosts =.*/elasticsearch_hosts = http:\/\/127.0.0.1:9200/" /etc/graylog/server/server.conf
sed -i "s/^#*mongodb_uri =.*/mongodb_uri = mongodb:\/\/localhost:27017\/graylog/" /etc/graylog/server/server.conf

# Create node-id file
echo "$GRAYLOG_NODE_ID" > /etc/graylog/server/node-id
chown graylog:graylog /etc/graylog/server/node-id
chmod 644 /etc/graylog/server/node-id

# Set proper ownership
chown -R graylog:graylog /etc/graylog/

# Enable and start Graylog
systemctl enable graylog-server
systemctl start graylog-server

# Step 8: Configure firewall
log_info "[8/8] Configuring firewall..."
if command -v ufw >/dev/null; then
    ufw allow 9000/tcp
    ufw allow 514/udp
    ufw allow 1514/tcp
elif command -v firewall-cmd >/dev/null; then
    firewall-cmd --permanent --add-port=9000/tcp
    firewall-cmd --permanent --add-port=514/udp
    firewall-cmd --permanent --add-port=1514/tcp
    firewall-cmd --reload
fi

# Verification
log_info "Verifying installation..."
sleep 10

if systemctl is-active --quiet mongod; then
    log_info "MongoDB is running"
else
    log_error "MongoDB is not running"
fi

if systemctl is-active --quiet elasticsearch; then
    log_info "Elasticsearch is running"
else
    log_error "Elasticsearch is not running"
fi

if systemctl is-active --quiet graylog-server; then
    log_info "Graylog is running"
else
    log_error "Graylog is not running"
fi

log_info "Installation completed successfully!"
log_info "Graylog web interface will be available at: http://$(hostname -I | awk '{print $1}'):9000"
log_info "Default login: admin / $GRAYLOG_ROOT_PASSWORD"
log_info "Password secret: $GRAYLOG_PASSWORD"
log_warn "Please change the default password after first login"
log_warn "Allow a few minutes for all services to fully initialize"

Review the script before running. Execute with: bash install.sh

#graylog #elasticsearch #mongodb #centralized-logging #log-management

Install and configure Graylog 5 for centralized log management with MongoDB and Elasticsearch