Configure Redis 7 cluster sharding for horizontal scaling with automated failover and monitoring

Advanced 45 min Apr 29, 2026 654 views
Ubuntu 24.04 Debian 12 AlmaLinux 9 Rocky Linux 9

Set up Redis 7 cluster with automatic sharding across multiple nodes for horizontal scaling. Configure hash slots, master-slave replication, and automated failover with comprehensive monitoring.

Prerequisites

  • At least 4GB RAM
  • Multiple CPU cores
  • Root or sudo access
  • Network connectivity between nodes

What this solves

Redis cluster sharding distributes your data across multiple Redis nodes, allowing you to scale beyond the memory limits of a single server. This tutorial sets up a production-ready Redis 7 cluster with automated failover, hash slot management, and monitoring to handle high-traffic applications that need horizontal scaling.

Step-by-step configuration

Update system packages

Start by updating your package manager to ensure you get the latest versions.

sudo apt update && sudo apt upgrade -y
sudo dnf update -y

Install Redis 7

Install Redis 7 from the official repositories. This provides the cluster functionality we need.

sudo apt install -y redis-server redis-tools
sudo dnf install -y redis redis-tools

Configure Redis cluster nodes

Create Redis configuration files for each cluster node. We'll set up 6 nodes (3 masters, 3 slaves) across different ports.

sudo mkdir -p /etc/redis/cluster
sudo mkdir -p /var/lib/redis/cluster
sudo mkdir -p /var/log/redis

Create configuration for first master node

Configure the first master node on port 7000. This enables cluster mode and sets up data persistence.

port 7000
cluster-enabled yes
cluster-config-file nodes-7000.conf
cluster-node-timeout 5000
appendonly yes
appendfilename "appendonly-7000.aof"
dir /var/lib/redis/cluster/7000
bind 0.0.0.0
protected-mode no
logfile /var/log/redis/redis-7000.log
loglevel notice
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump-7000.rdb
maxmemory 2gb
maxmemory-policy allkeys-lru
tcp-keepalive 60
timeout 0

Create configuration for second master node

Configure the second master node on port 7001 with similar settings but different paths.

port 7001
cluster-enabled yes
cluster-config-file nodes-7001.conf
cluster-node-timeout 5000
appendonly yes
appendfilename "appendonly-7001.aof"
dir /var/lib/redis/cluster/7001
bind 0.0.0.0
protected-mode no
logfile /var/log/redis/redis-7001.log
loglevel notice
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump-7001.rdb
maxmemory 2gb
maxmemory-policy allkeys-lru
tcp-keepalive 60
timeout 0

Create configuration for third master node

Configure the third master node on port 7002 to complete our master node setup.

port 7002
cluster-enabled yes
cluster-config-file nodes-7002.conf
cluster-node-timeout 5000
appendonly yes
appendfilename "appendonly-7002.aof"
dir /var/lib/redis/cluster/7002
bind 0.0.0.0
protected-mode no
logfile /var/log/redis/redis-7002.log
loglevel notice
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump-7002.rdb
maxmemory 2gb
maxmemory-policy allkeys-lru
tcp-keepalive 60
timeout 0

Create configuration for slave nodes

Configure three slave nodes on ports 7003, 7004, and 7005. These will provide automatic failover capability.

port 7003
cluster-enabled yes
cluster-config-file nodes-7003.conf
cluster-node-timeout 5000
appendonly yes
appendfilename "appendonly-7003.aof"
dir /var/lib/redis/cluster/7003
bind 0.0.0.0
protected-mode no
logfile /var/log/redis/redis-7003.log
loglevel notice
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump-7003.rdb
maxmemory 2gb
maxmemory-policy allkeys-lru
tcp-keepalive 60
timeout 0
port 7004
cluster-enabled yes
cluster-config-file nodes-7004.conf
cluster-node-timeout 5000
appendonly yes
appendfilename "appendonly-7004.aof"
dir /var/lib/redis/cluster/7004
bind 0.0.0.0
protected-mode no
logfile /var/log/redis/redis-7004.log
loglevel notice
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump-7004.rdb
maxmemory 2gb
maxmemory-policy allkeys-lru
tcp-keepalive 60
timeout 0
port 7005
cluster-enabled yes
cluster-config-file nodes-7005.conf
cluster-node-timeout 5000
appendonly yes
appendfilename "appendonly-7005.aof"
dir /var/lib/redis/cluster/7005
bind 0.0.0.0
protected-mode no
logfile /var/log/redis/redis-7005.log
loglevel notice
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump-7005.rdb
maxmemory 2gb
maxmemory-policy allkeys-lru
tcp-keepalive 60
timeout 0

Create data directories

Create the data directories for each Redis node and set proper ownership.

sudo mkdir -p /var/lib/redis/cluster/{7000,7001,7002,7003,7004,7005}
sudo chown -R redis:redis /var/lib/redis/cluster
sudo chown -R redis:redis /var/log/redis
sudo chmod 755 /var/lib/redis/cluster/*

Create systemd service files

Create systemd service files for each Redis cluster node to manage them as system services.

[Unit]
Description=Redis Cluster Node 7000
After=network.target

[Service]
Type=notify
ExecStart=/usr/bin/redis-server /etc/redis/cluster/redis-7000.conf
ExecReload=/bin/kill -USR2 $MAINPID
TimeoutStopSec=0
Restart=always
User=redis
Group=redis
RuntimeDirectory=redis-7000
RuntimeDirectoryMode=0755

[Install]
WantedBy=multi-user.target
sudo cp /etc/systemd/system/redis-cluster-7000.service /etc/systemd/system/redis-cluster-7001.service
sudo cp /etc/systemd/system/redis-cluster-7000.service /etc/systemd/system/redis-cluster-7002.service
sudo cp /etc/systemd/system/redis-cluster-7000.service /etc/systemd/system/redis-cluster-7003.service
sudo cp /etc/systemd/system/redis-cluster-7000.service /etc/systemd/system/redis-cluster-7004.service
sudo cp /etc/systemd/system/redis-cluster-7000.service /etc/systemd/system/redis-cluster-7005.service

Update service files for different ports

Modify each service file to reference the correct configuration file and port.

sudo sed -i 's/7000/7001/g' /etc/systemd/system/redis-cluster-7001.service
sudo sed -i 's/7000/7002/g' /etc/systemd/system/redis-cluster-7002.service
sudo sed -i 's/7000/7003/g' /etc/systemd/system/redis-cluster-7003.service
sudo sed -i 's/7000/7004/g' /etc/systemd/system/redis-cluster-7004.service
sudo sed -i 's/7000/7005/g' /etc/systemd/system/redis-cluster-7005.service

Start all Redis cluster nodes

Enable and start all six Redis cluster nodes.

sudo systemctl daemon-reload
sudo systemctl enable redis-cluster-7000 redis-cluster-7001 redis-cluster-7002 redis-cluster-7003 redis-cluster-7004 redis-cluster-7005
sudo systemctl start redis-cluster-7000 redis-cluster-7001 redis-cluster-7002 redis-cluster-7003 redis-cluster-7004 redis-cluster-7005

Initialize the Redis cluster

Create the cluster and assign hash slots automatically. This command distributes the 16384 hash slots across the three master nodes.

redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 --cluster-replicas 1

Type yes when prompted to accept the cluster configuration.

Warning: The bind 0.0.0.0 and protected-mode no settings allow connections from any IP address. In production, use specific IP addresses and enable authentication with the requirepass directive.

Configure firewall rules

Open the necessary ports for Redis cluster communication. Each node needs its data port and cluster bus port (data port + 10000).

sudo ufw allow 7000:7005/tcp
sudo ufw allow 17000:17005/tcp
sudo ufw reload
sudo firewall-cmd --permanent --add-port=7000-7005/tcp
sudo firewall-cmd --permanent --add-port=17000-17005/tcp
sudo firewall-cmd --reload

Set up Redis cluster monitoring script

Create a monitoring script to check cluster health and node status.

#!/bin/bash

# Redis Cluster Health Monitor
echo "=== Redis Cluster Status ==="
echo "Timestamp: $(date)"
echo

# Check cluster info
echo "--- Cluster Info ---"
redis-cli -p 7000 cluster info
echo

# Check cluster nodes
echo "--- Cluster Nodes ---"
redis-cli -p 7000 cluster nodes
echo

# Check individual node status
echo "--- Individual Node Status ---"
for port in 7000 7001 7002 7003 7004 7005; do
    echo "Node $port:"
    timeout 3 redis-cli -p $port ping 2>/dev/null || echo "  FAILED"
done
echo

# Check memory usage
echo "--- Memory Usage ---"
for port in 7000 7001 7002 7003 7004 7005; do
    memory=$(redis-cli -p $port info memory | grep used_memory_human | cut -d: -f2 | tr -d '\r')
    role=$(redis-cli -p $port info replication | grep role | cut -d: -f2 | tr -d '\r')
    echo "Node $port ($role): $memory"
done
echo

# Check hash slot distribution
echo "--- Hash Slot Distribution ---"
redis-cli -p 7000 cluster slots
sudo chmod +x /usr/local/bin/redis-cluster-monitor.sh

Configure automated monitoring with systemd timer

Set up a systemd timer to run cluster health checks every 5 minutes.

[Unit]
Description=Redis Cluster Health Monitor
After=network.target

[Service]
Type=oneshot
ExecStart=/usr/local/bin/redis-cluster-monitor.sh
User=redis
StandardOutput=journal
StandardError=journal
[Unit]
Description=Run Redis Cluster Monitor every 5 minutes
Requires=redis-cluster-monitor.service

[Timer]
OnCalendar=*:0/5
Persistent=true

[Install]
WantedBy=timers.target
sudo systemctl daemon-reload
sudo systemctl enable redis-cluster-monitor.timer
sudo systemctl start redis-cluster-monitor.timer

Configure Prometheus metrics exporter

Install and configure redis_exporter for Prometheus monitoring integration.

wget https://github.com/oliver006/redis_exporter/releases/download/v1.55.0/redis_exporter-v1.55.0.linux-amd64.tar.gz
tar xzf redis_exporter-v1.55.0.linux-amd64.tar.gz
sudo mv redis_exporter-v1.55.0.linux-amd64/redis_exporter /usr/local/bin/
sudo chmod +x /usr/local/bin/redis_exporter
wget https://github.com/oliver006/redis_exporter/releases/download/v1.55.0/redis_exporter-v1.55.0.linux-amd64.tar.gz
tar xzf redis_exporter-v1.55.0.linux-amd64.tar.gz
sudo mv redis_exporter-v1.55.0.linux-amd64/redis_exporter /usr/local/bin/
sudo chmod +x /usr/local/bin/redis_exporter

Create Redis exporter service

Configure the Redis exporter as a systemd service to expose cluster metrics on port 9121.

[Unit]
Description=Redis Cluster Metrics Exporter
After=network.target

[Service]
Type=simple
User=redis
Group=redis
ExecStart=/usr/local/bin/redis_exporter \
  -redis.addr=redis://127.0.0.1:7000 \
  -redis.addr=redis://127.0.0.1:7001 \
  -redis.addr=redis://127.0.0.1:7002 \
  -redis.addr=redis://127.0.0.1:7003 \
  -redis.addr=redis://127.0.0.1:7004 \
  -redis.addr=redis://127.0.0.1:7005 \
  -web.listen-address=:9121
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload
sudo systemctl enable redis-exporter
sudo systemctl start redis-exporter

Set up automatic failover testing

Create a script to test automatic failover by simulating node failures.

#!/bin/bash

# Redis Cluster Failover Test
echo "=== Redis Cluster Failover Test ==="
echo "Starting failover test at $(date)"
echo

# Get current master nodes
echo "--- Current Masters ---"
masters=$(redis-cli -p 7000 cluster nodes | grep master | awk '{print $1 ":" $2}')
echo "$masters"
echo

# Pick first master for failover test
first_master_port=$(echo "$masters" | head -n1 | cut -d: -f2)
echo "Testing failover on port $first_master_port"
echo

# Simulate failure by stopping the node
echo "Stopping node $first_master_port..."
sudo systemctl stop redis-cluster-$first_master_port
echo

# Wait for failover
echo "Waiting 10 seconds for automatic failover..."
sleep 10
echo

# Check cluster status
echo "--- Post-Failover Status ---"
redis-cli -p 7001 cluster nodes | grep -E "master|fail"
echo

# Restart the failed node
echo "Restarting node $first_master_port..."
sudo systemctl start redis-cluster-$first_master_port
sleep 5
echo

# Final status
echo "--- Final Status ---"
redis-cli -p 7000 cluster info | grep cluster_state
echo "Failover test completed at $(date)"
sudo chmod +x /usr/local/bin/redis-failover-test.sh

Verify your setup

Check that all Redis cluster nodes are running and the cluster is healthy.

# Check all services are running
sudo systemctl status redis-cluster-7000 redis-cluster-7001 redis-cluster-7002

# Check cluster status
redis-cli -p 7000 cluster info

# Check cluster nodes
redis-cli -p 7000 cluster nodes

# Test cluster operations
redis-cli -c -p 7000 set test-key "Hello Redis Cluster"
redis-cli -c -p 7001 get test-key

# Check metrics endpoint
curl localhost:9121/metrics | grep redis

# Run monitoring script
/usr/local/bin/redis-cluster-monitor.sh

For additional monitoring integration, you can set up Redis monitoring with Prometheus and Grafana to create comprehensive dashboards for your cluster.

Hash slot management

Redis cluster automatically distributes data across nodes using hash slots. Each key is hashed and mapped to one of 16384 slots, which are distributed among master nodes.

View hash slot distribution

Check how hash slots are distributed across your cluster nodes.

# View detailed slot assignment
redis-cli -p 7000 cluster slots

# Check which node serves a specific key
redis-cli -p 7000 cluster keyslot mykey

# Find which node handles a slot
redis-cli -p 7000 cluster nodes | grep 0-5460

Manual hash slot rebalancing

If needed, you can manually rebalance hash slots across nodes.

# Rebalance cluster (use with caution)
redis-cli --cluster rebalance 127.0.0.1:7000 --cluster-use-empty-masters

# Check cluster after rebalancing
redis-cli --cluster check 127.0.0.1:7000
Note: Hash slot rebalancing can impact performance during the operation. Always test in a staging environment first and perform during maintenance windows.

Adding and removing nodes

Redis cluster supports dynamic scaling by adding or removing nodes without downtime.

Add a new master node

Add a new master node to increase cluster capacity.

# Create new node configuration (port 7006)
sudo cp /etc/redis/cluster/redis-7000.conf /etc/redis/cluster/redis-7006.conf
sudo sed -i 's/7000/7006/g' /etc/redis/cluster/redis-7006.conf

# Create data directory
sudo mkdir -p /var/lib/redis/cluster/7006
sudo chown redis:redis /var/lib/redis/cluster/7006

# Create and start service
sudo cp /etc/systemd/system/redis-cluster-7000.service /etc/systemd/system/redis-cluster-7006.service
sudo sed -i 's/7000/7006/g' /etc/systemd/system/redis-cluster-7006.service
sudo systemctl daemon-reload
sudo systemctl enable redis-cluster-7006
sudo systemctl start redis-cluster-7006

# Add node to cluster
redis-cli --cluster add-node 127.0.0.1:7006 127.0.0.1:7000

# Rebalance slots to new node
redis-cli --cluster rebalance 127.0.0.1:7000

Remove a node from cluster

Safely remove a node from the cluster by first moving its slots to other nodes.

# Get node ID to remove
node_id=$(redis-cli -p 7006 cluster myid)

# Remove all keys from node (move slots)
redis-cli --cluster reshard 127.0.0.1:7000 --cluster-from $node_id --cluster-to-all --cluster-slots 1365

# Remove empty node from cluster
redis-cli --cluster del-node 127.0.0.1:7000 $node_id

# Stop and disable service
sudo systemctl stop redis-cluster-7006
sudo systemctl disable redis-cluster

Automated install script

Run this to automate the entire setup

Nie chcesz zarządzać tym samodzielnie?

Zarządzamy infrastrukturą firm, które zależą od dostępności. W pełni zarządzana, z jednym stałym kontaktem, który zna Twoje środowisko.

Macie jednego stałego opiekuna, który zna Waszą konfigurację

Rotterdam 04:08 · dostępny w wiadomości, bez formularza zgłoszeń