Setup Elasticsearch 8 SSL/TLS encryption and advanced security hardening with authentication and access control

Generate SSL/TLS certificates for cluster communication

Create the certificate authority and node certificates for secure transport layer communication between Elasticsearch nodes.

sudo /usr/share/elasticsearch/bin/elasticsearch-certutil ca --out /etc/elasticsearch/elastic-stack-ca.p12 --pass ""
sudo /usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca /etc/elasticsearch/elastic-stack-ca.p12 --out /etc/elasticsearch/elastic-certificates.p12 --pass ""
sudo chown elasticsearch:elasticsearch /etc/elasticsearch/elastic-*.p12
sudo chmod 660 /etc/elasticsearch/elastic-*.p12

Generate HTTP SSL certificates for client connections

Create certificates for HTTPS communication between clients and Elasticsearch, including proper hostname verification.

sudo /usr/share/elasticsearch/bin/elasticsearch-certutil http
Follow prompts:
Generate CSR? n
Use existing CA? y
CA Path: /etc/elasticsearch/elastic-stack-ca.p12
Password: (leave empty)
Certificate validity: 5y
Generate certificate per node? y
Node name: elasticsearch-node1
Hostnames: localhost,127.0.0.1,your-server-ip
IP addresses: 127.0.0.1,your-server-ip
More nodes? n
Provide password for private key? n
Output file: /tmp/elasticsearch-ssl-http.zip

cd /tmp
sudo unzip elasticsearch-ssl-http.zip
sudo cp elasticsearch/http.p12 /etc/elasticsearch/
sudo chown elasticsearch:elasticsearch /etc/elasticsearch/http.p12
sudo chmod 660 /etc/elasticsearch/http.p12

Configure Elasticsearch security settings

Enable security features, configure SSL/TLS for transport and HTTP layers, and set network binding options for production security.

# Cluster and node configuration
cluster.name: production-cluster
node.name: elasticsearch-node1
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch

Network and discovery settings
network.host: 0.0.0.0
http.port: 9200
discovery.type: single-node

Security configuration
xpack.security.enabled: true
xpack.security.enrollment.enabled: true

Transport layer SSL/TLS
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.client_authentication: required
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

HTTP layer SSL/TLS
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: http.p12

Security audit logging
xpack.security.audit.enabled: true
xpack.security.audit.logfile.events.include: access_denied, access_granted, anonymous_access_denied, authentication_failed, connection_denied, tampered_request, run_as_denied, run_as_granted

Set JVM heap size and memory settings

Configure JVM memory allocation for optimal performance and security. Set heap size to half of available RAM, maximum 32GB.

# Set heap size to 50% of available RAM (adjust based on your system)
-Xms2g
-Xmx2g

Start and enable Elasticsearch service

Start Elasticsearch with the new security configuration and enable automatic startup on system boot.

sudo systemctl daemon-reload
sudo systemctl enable elasticsearch
sudo systemctl start elasticsearch
sudo systemctl status elasticsearch

Set the built-in superuser password

Configure the elastic superuser password for initial cluster access and user management.

sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic -i
Enter a strong password when prompted
Save this password securely - you'll need it for administration

Create custom roles for application access

Define granular roles following the principle of least privilege for different application access patterns.

# Create a read-only role for monitoring applications
curl -k -u elastic:your-password -X POST "https://localhost:9200/_security/role/monitoring_user" -H 'Content-Type: application/json' -d'
{
  "cluster": ["monitor"],
  "indices": [
    {
      "names": ["logs-", "metrics-"],
      "privileges": ["read", "view_index_metadata"]
    }
  ]
}'

Create an application role with write access to specific indices
curl -k -u elastic:your-password -X POST "https://localhost:9200/_security/role/app_writer" -H 'Content-Type: application/json' -d'
{
  "cluster": [],
  "indices": [
    {
      "names": ["app-logs-", "app-data-"],
      "privileges": ["create", "create_index", "index", "write"]
    }
  ]
}'

Create a data analyst role with broader read access
curl -k -u elastic:your-password -X POST "https://localhost:9200/_security/role/data_analyst" -H 'Content-Type: application/json' -d'
{
  "cluster": ["monitor"],
  "indices": [
    {
      "names": ["*"],
      "privileges": ["read", "view_index_metadata"]
    }
  ]
}'

Create application users with role assignments

Create dedicated users for different applications and assign them appropriate roles based on their access requirements.

# Create monitoring user
curl -k -u elastic:your-password -X POST "https://localhost:9200/_security/user/monitoring_service" -H 'Content-Type: application/json' -d'
{
  "password": "MonitoringPass123!",
  "roles": ["monitoring_user"],
  "full_name": "Monitoring Service Account",
  "email": "monitoring@example.com"
}'

Create application writer user
curl -k -u elastic:your-password -X POST "https://localhost:9200/_security/user/app_service" -H 'Content-Type: application/json' -d'
{
  "password": "AppServicePass123!",
  "roles": ["app_writer"],
  "full_name": "Application Service Account",
  "email": "app@example.com"
}'

Create data analyst user
curl -k -u elastic:your-password -X POST "https://localhost:9200/_security/user/analyst" -H 'Content-Type: application/json' -d'
{
  "password": "AnalystPass123!",
  "roles": ["data_analyst"],
  "full_name": "Data Analyst",
  "email": "analyst@example.com"
}'

Generate API keys for programmatic access

Create API keys for applications that need programmatic access without storing user passwords in configuration files.

# Create API key for monitoring application
curl -k -u monitoring_service:MonitoringPass123! -X POST "https://localhost:9200/_security/api_key" -H 'Content-Type: application/json' -d'
{
  "name": "monitoring-api-key",
  "expiration": "365d",
  "role_descriptors": {
    "monitoring_role": {
      "cluster": ["monitor"],
      "indices": [
        {
          "names": ["logs-", "metrics-"],
          "privileges": ["read"]
        }
      ]
    }
  }
}'

Create API key for application with write access
curl -k -u app_service:AppServicePass123! -X POST "https://localhost:9200/_security/api_key" -H 'Content-Type: application/json' -d'
{
  "name": "app-write-api-key",
  "expiration": "90d",
  "role_descriptors": {
    "app_write_role": {
      "cluster": [],
      "indices": [
        {
          "names": ["app-logs-*"],
          "privileges": ["create_index", "index", "write"]
        }
      ]
    }
  }
}'

Configure firewall rules for network security

Implement network-level access controls to restrict Elasticsearch access to authorized systems only.

# Install and enable UFW firewall
sudo apt install -y ufw
sudo ufw --force enable

Allow SSH (adjust port if needed)
sudo ufw allow 22/tcp

Allow Elasticsearch HTTPS from specific networks
Replace with your actual network ranges
sudo ufw allow from 10.0.0.0/8 to any port 9200 proto tcp
sudo ufw allow from 172.16.0.0/12 to any port 9200 proto tcp
sudo ufw allow from 192.168.0.0/16 to any port 9200 proto tcp

Allow transport layer communication (node-to-node)
sudo ufw allow from 10.0.0.0/8 to any port 9300 proto tcp
sudo ufw allow from 172.16.0.0/12 to any port 9300 proto tcp
sudo ufw allow from 192.168.0.0/16 to any port 9300 proto tcp

Set default policies
sudo ufw default deny incoming
sudo ufw default allow outgoing

Check firewall status
sudo ufw status verbose

# Configure firewalld
sudo systemctl enable --now firewalld

Create custom service for Elasticsearch HTTPS
sudo tee /etc/firewalld/services/elasticsearch-https.xml <

  Elasticsearch HTTPS
  Elasticsearch HTTPS API access
  

EOF

Create custom service for Elasticsearch transport
sudo tee /etc/firewalld/services/elasticsearch-transport.xml <

  Elasticsearch Transport
  Elasticsearch node transport layer
  

EOF

Reload firewalld and add services to trusted networks
sudo firewall-cmd --reload
sudo firewall-cmd --permanent --new-zone=elasticsearch
sudo firewall-cmd --permanent --zone=elasticsearch --add-source=10.0.0.0/8
sudo firewall-cmd --permanent --zone=elasticsearch --add-source=172.16.0.0/12
sudo firewall-cmd --permanent --zone=elasticsearch --add-source=192.168.0.0/16
sudo firewall-cmd --permanent --zone=elasticsearch --add-service=elasticsearch-https
sudo firewall-cmd --permanent --zone=elasticsearch --add-service=elasticsearch-transport
sudo firewall-cmd --reload
sudo firewall-cmd --list-all-zones

Configure system security limits

Set system resource limits and security parameters optimized for Elasticsearch production deployment.

elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
elasticsearch soft nofile 65536
elasticsearch hard nofile 65536
elasticsearch soft nproc 4096
elasticsearch hard nproc 4096

# Configure system sysctl parameters
sudo tee -a /etc/sysctl.d/99-elasticsearch.conf <Virtual memory settings for Elasticsearch
vm.max_map_count=262144

Network security hardening
net.ipv4.tcp_syncookies=1
net.ipv4.ip_forward=0
net.ipv4.conf.all.send_redirects=0
net.ipv4.conf.default.send_redirects=0
net.ipv4.conf.all.accept_redirects=0
net.ipv4.conf.default.accept_redirects=0
net.ipv4.conf.all.accept_source_route=0
net.ipv4.conf.default.accept_source_route=0
EOF

Apply sysctl settings
sudo sysctl -p /etc/sysctl.d/99-elasticsearch.conf

Configure log rotation and audit settings

Set up proper log rotation and audit trail configuration for security monitoring and compliance.

/var/log/elasticsearch/*.log {
    daily
    missingok
    rotate 30
    compress
    delaycompress
    notifempty
    copytruncate
    su elasticsearch elasticsearch
}

# Set proper permissions on Elasticsearch directories
sudo chown -R elasticsearch:elasticsearch /var/lib/elasticsearch
sudo chown -R elasticsearch:elasticsearch /var/log/elasticsearch
sudo chown -R root:elasticsearch /etc/elasticsearch
sudo chmod 750 /etc/elasticsearch
sudo chmod 660 /etc/elasticsearch/*.yml

Restart Elasticsearch to apply all changes
sudo systemctl restart elasticsearch