Configure a production-ready TimescaleDB cluster with streaming replication, automatic failover using Patroni, and etcd for distributed consensus to ensure zero-downtime operation of your time-series database.
Prerequisites
- Three servers with at least 4GB RAM each
- Network connectivity between nodes
- Root or sudo access on all nodes
What this solves
TimescaleDB clustering provides high availability for time-series workloads by eliminating single points of failure. This tutorial sets up a three-node TimescaleDB cluster with automatic failover using Patroni and etcd, ensuring your time-series database remains available even if the primary node fails.
Step-by-step installation
Update system packages
Start by updating your package manager on all three nodes to ensure you get the latest versions.
sudo apt update && sudo apt upgrade -y
Install PostgreSQL and TimescaleDB
Install PostgreSQL 15 with the TimescaleDB extension on all three nodes.
sudo sh -c "echo 'deb https://packagecloud.io/timescale/timescaledb/ubuntu/ $(lsb_release -c -s) main' > /etc/apt/sources.list.d/timescaledb.list"
wget --quiet -O - https://packagecloud.io/timescale/timescaledb/gpgkey | sudo apt-key add -
sudo apt update
sudo apt install -y postgresql-15 timescaledb-2-postgresql-15
Install etcd cluster
Install and configure etcd on all three nodes for distributed consensus. Replace the IP addresses with your actual node IPs.
sudo apt install -y etcd
Configure etcd on first node
Configure etcd on the first node (203.0.113.10). This node will bootstrap the cluster.
ETCD_NAME="etcd1"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="http://203.0.113.10:2380"
ETCD_LISTEN_CLIENT_URLS="http://203.0.113.10:2379,http://127.0.0.1:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://203.0.113.10:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://203.0.113.10:2379"
ETCD_INITIAL_CLUSTER="etcd1=http://203.0.113.10:2380,etcd2=http://203.0.113.11:2380,etcd3=http://203.0.113.12:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="timescale-cluster"
Configure etcd on second node
Configure etcd on the second node (203.0.113.11) with the same cluster settings.
ETCD_NAME="etcd2"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="http://203.0.113.11:2380"
ETCD_LISTEN_CLIENT_URLS="http://203.0.113.11:2379,http://127.0.0.1:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://203.0.113.11:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://203.0.113.11:2379"
ETCD_INITIAL_CLUSTER="etcd1=http://203.0.113.10:2380,etcd2=http://203.0.113.11:2380,etcd3=http://203.0.113.12:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="timescale-cluster"
Configure etcd on third node
Configure etcd on the third node (203.0.113.12) to complete the cluster.
ETCD_NAME="etcd3"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="http://203.0.113.12:2380"
ETCD_LISTEN_CLIENT_URLS="http://203.0.113.12:2379,http://127.0.0.1:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://203.0.113.12:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://203.0.113.12:2379"
ETCD_INITIAL_CLUSTER="etcd1=http://203.0.113.10:2380,etcd2=http://203.0.113.11:2380,etcd3=http://203.0.113.12:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="timescale-cluster"
Start etcd cluster
Start etcd on all three nodes simultaneously to form the cluster.
sudo systemctl enable --now etcd
sudo systemctl status etcd
Install Patroni
Install Patroni on all three nodes to manage PostgreSQL with automatic failover.
sudo apt install -y python3-pip python3-psycopg2
sudo pip3 install patroni python-etcd
Create PostgreSQL user and directories
Create the postgres user and required directories on all nodes. This user will manage the database processes.
sudo useradd -m postgres
sudo mkdir -p /data/postgresql
sudo chown postgres:postgres /data/postgresql
sudo chmod 700 /data/postgresql
Configure Patroni on first node
Create the Patroni configuration for the first node. This will become the initial primary.
scope: timescale-cluster
namespace: /db/
name: node1
restapi:
listen: 203.0.113.10:8008
connect_address: 203.0.113.10:8008
etcd:
hosts: 203.0.113.10:2379,203.0.113.11:2379,203.0.113.12:2379
bootstrap:
dcs:
ttl: 30
loop_wait: 10
retry_timeout: 30
maximum_lag_on_failover: 1048576
postgresql:
use_pg_rewind: true
use_slots: true
parameters:
wal_level: replica
hot_standby: "on"
max_connections: 100
max_worker_processes: 8
wal_keep_segments: 8
max_wal_senders: 10
max_replication_slots: 10
max_prepared_transactions: 0
max_locks_per_transaction: 64
wal_log_hints: "on"
track_commit_timestamp: "off"
archive_mode: "on"
archive_timeout: 1800s
archive_command: mkdir -p ../wal_archive && test ! -f ../wal_archive/%f && cp %p ../wal_archive/%f
shared_preload_libraries: 'timescaledb'
recovery_conf:
restore_command: cp ../wal_archive/%f %p
initdb:
- encoding: UTF8
- data-checksums
pg_hba:
- host replication replicator 127.0.0.1/32 md5
- host replication replicator 203.0.113.10/32 md5
- host replication replicator 203.0.113.11/32 md5
- host replication replicator 203.0.113.12/32 md5
- host all all 0.0.0.0/0 md5
users:
admin:
password: StrongAdminPassword123!
options:
- createrole
- createdb
postgresql:
listen: 203.0.113.10:5432
connect_address: 203.0.113.10:5432
data_dir: /data/postgresql
bin_dir: /usr/lib/postgresql/15/bin
pgpass: /tmp/pgpass
authentication:
replication:
username: replicator
password: ReplicatorPassword123!
superuser:
username: postgres
password: PostgresPassword123!
rewind:
username: rewind_user
password: RewindPassword123!
parameters:
unix_socket_directories: '.'
tags:
nofailover: false
noloadbalance: false
clonefrom: false
nosync: false
Configure Patroni on second node
Create the Patroni configuration for the second node with its specific IP address.
scope: timescale-cluster
namespace: /db/
name: node2
restapi:
listen: 203.0.113.11:8008
connect_address: 203.0.113.11:8008
etcd:
hosts: 203.0.113.10:2379,203.0.113.11:2379,203.0.113.12:2379
bootstrap:
dcs:
ttl: 30
loop_wait: 10
retry_timeout: 30
maximum_lag_on_failover: 1048576
postgresql:
use_pg_rewind: true
use_slots: true
parameters:
wal_level: replica
hot_standby: "on"
max_connections: 100
max_worker_processes: 8
wal_keep_segments: 8
max_wal_senders: 10
max_replication_slots: 10
max_prepared_transactions: 0
max_locks_per_transaction: 64
wal_log_hints: "on"
track_commit_timestamp: "off"
archive_mode: "on"
archive_timeout: 1800s
archive_command: mkdir -p ../wal_archive && test ! -f ../wal_archive/%f && cp %p ../wal_archive/%f
shared_preload_libraries: 'timescaledb'
recovery_conf:
restore_command: cp ../wal_archive/%f %p
postgresql:
listen: 203.0.113.11:5432
connect_address: 203.0.113.11:5432
data_dir: /data/postgresql
bin_dir: /usr/lib/postgresql/15/bin
pgpass: /tmp/pgpass
authentication:
replication:
username: replicator
password: ReplicatorPassword123!
superuser:
username: postgres
password: PostgresPassword123!
rewind:
username: rewind_user
password: RewindPassword123!
parameters:
unix_socket_directories: '.'
tags:
nofailover: false
noloadbalance: false
clonefrom: false
nosync: false
Configure Patroni on third node
Create the Patroni configuration for the third node to complete the cluster setup.
scope: timescale-cluster
namespace: /db/
name: node3
restapi:
listen: 203.0.113.12:8008
connect_address: 203.0.113.12:8008
etcd:
hosts: 203.0.113.10:2379,203.0.113.11:2379,203.0.113.12:2379
bootstrap:
dcs:
ttl: 30
loop_wait: 10
retry_timeout: 30
maximum_lag_on_failover: 1048576
postgresql:
use_pg_rewind: true
use_slots: true
parameters:
wal_level: replica
hot_standby: "on"
max_connections: 100
max_worker_processes: 8
wal_keep_segments: 8
max_wal_senders: 10
max_replication_slots: 10
max_prepared_transactions: 0
max_locks_per_transaction: 64
wal_log_hints: "on"
track_commit_timestamp: "off"
archive_mode: "on"
archive_timeout: 1800s
archive_command: mkdir -p ../wal_archive && test ! -f ../wal_archive/%f && cp %p ../wal_archive/%f
shared_preload_libraries: 'timescaledb'
recovery_conf:
restore_command: cp ../wal_archive/%f %p
postgresql:
listen: 203.0.113.12:5432
connect_address: 203.0.113.12:5432
data_dir: /data/postgresql
bin_dir: /usr/lib/postgresql/15/bin
pgpass: /tmp/pgpass
authentication:
replication:
username: replicator
password: ReplicatorPassword123!
superuser:
username: postgres
password: PostgresPassword123!
rewind:
username: rewind_user
password: RewindPassword123!
parameters:
unix_socket_directories: '.'
tags:
nofailover: false
noloadbalance: false
clonefrom: false
nosync: false
Create Patroni systemd service
Create a systemd service file for Patroni on all three nodes to manage the service lifecycle.
[Unit]
Description=Runners to orchestrate a high-availability PostgreSQL
After=syslog.target network.target
[Service]
Type=simple
User=postgres
Group=postgres
ExecStart=/usr/local/bin/patroni /etc/patroni/patroni.yml
KillMode=process
TimeoutSec=30
Restart=no
[Install]
WantedBy=multi-user.target
Set correct permissions
Set the correct ownership and permissions for Patroni configuration files. The postgres user needs read access to the configuration.
sudo mkdir -p /etc/patroni
sudo chown postgres:postgres /etc/patroni/patroni.yml
sudo chmod 600 /etc/patroni/patroni.yml
sudo systemctl daemon-reload
Start Patroni cluster
Start Patroni on the first node, wait for it to initialize, then start the other nodes.
sudo systemctl enable --now patroni
sudo systemctl status patroni
Enable TimescaleDB extension
Connect to the primary node and enable the TimescaleDB extension in your database.
sudo -u postgres psql -h 203.0.113.10 -p 5432
CREATE DATABASE timeseries;
\c timeseries
CREATE EXTENSION IF NOT EXISTS timescaledb;
Configure firewall rules
Open the required ports for PostgreSQL, etcd, and Patroni communication.
sudo ufw allow 5432/tcp
sudo ufw allow 8008/tcp
sudo ufw allow 2379/tcp
sudo ufw allow 2380/tcp
Verify your setup
Check that your TimescaleDB cluster is running correctly and replication is working.
patronictl -c /etc/patroni/patroni.yml list
etcdctl cluster-health
sudo -u postgres psql -h 203.0.113.10 -c "SELECT * FROM pg_stat_replication;"
sudo -u postgres psql -h 203.0.113.10 -c "SELECT version();"
sudo -u postgres psql -h 203.0.113.10 -d timeseries -c "SELECT default_version, installed_version FROM pg_available_extensions WHERE name = 'timescaledb';"
Test high availability scenarios
Test automatic failover
Simulate a primary node failure to verify automatic failover works correctly.
sudo systemctl stop patroni
patronictl -c /etc/patroni/patroni.yml list
patronictl -c /etc/patroni/patroni.yml failover --master node1 --candidate node2
Monitor cluster status
Use Patroni's built-in monitoring to check cluster health and replication lag.
patronictl -c /etc/patroni/patroni.yml list
patronictl -c /etc/patroni/patroni.yml show-config
sudo -u postgres psql -h 203.0.113.11 -c "SELECT client_addr, state, sync_state FROM pg_stat_replication;"
Common issues
| Symptom | Cause | Fix |
|---|---|---|
| Patroni won't start | etcd cluster not running | sudo systemctl restart etcd on all nodes |
| Replication lag high | Network latency or disk I/O | Check pg_stat_replication and optimize disk performance |
| Split-brain condition | Network partition | Check etcd quorum with etcdctl cluster-health |
| TimescaleDB extension missing | Not loaded in shared_preload_libraries | Restart PostgreSQL after adding to config |
| Connection refused | Firewall blocking ports | Open ports 5432, 8008, 2379, 2380 |
Next steps
Automated install script
Run this to automate the entire setup
#!/usr/bin/env bash
set -euo pipefail
# TimescaleDB Clustering Setup Script
# Sets up TimescaleDB with Patroni and etcd for high availability
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
# Configuration
NODE_IP=${1:-}
CLUSTER_NAME=${2:-timescale-cluster}
ETCD_CLUSTER=${3:-}
usage() {
echo "Usage: $0 <node_ip> [cluster_name] [etcd_cluster_endpoints]"
echo "Example: $0 192.168.1.10 my-cluster node1=192.168.1.10:2380,node2=192.168.1.11:2380,node3=192.168.1.12:2380"
exit 1
}
log() {
echo -e "${GREEN}$1${NC}"
}
warn() {
echo -e "${YELLOW}$1${NC}"
}
error() {
echo -e "${RED}$1${NC}" >&2
exit 1
}
cleanup() {
warn "Installation failed. Cleaning up..."
systemctl stop postgresql || true
systemctl stop etcd || true
systemctl stop patroni || true
}
trap cleanup ERR
# Validate arguments
if [[ -z "$NODE_IP" ]]; then
usage
fi
# Check if running as root
if [[ $EUID -ne 0 ]]; then
error "This script must be run as root"
fi
# Detect distribution
if [ -f /etc/os-release ]; then
. /etc/os-release
case "$ID" in
ubuntu|debian)
PKG_MGR="apt"
PKG_UPDATE="apt update && apt upgrade -y"
PKG_INSTALL="apt install -y"
POSTGRES_SERVICE="postgresql"
;;
almalinux|rocky|centos|rhel|ol|fedora)
PKG_MGR="dnf"
PKG_UPDATE="dnf update -y"
PKG_INSTALL="dnf install -y"
POSTGRES_SERVICE="postgresql"
;;
amzn)
PKG_MGR="yum"
PKG_UPDATE="yum update -y"
PKG_INSTALL="yum install -y"
POSTGRES_SERVICE="postgresql"
;;
*)
error "Unsupported distro: $ID"
;;
esac
else
error "Cannot detect distribution"
fi
log "[1/8] Updating system packages..."
$PKG_UPDATE
log "[2/8] Installing prerequisites..."
case "$PKG_MGR" in
apt)
$PKG_INSTALL wget gnupg lsb-release curl software-properties-common
;;
dnf|yum)
$PKG_INSTALL wget curl epel-release
;;
esac
log "[3/8] Installing PostgreSQL 15 and TimescaleDB..."
case "$PKG_MGR" in
apt)
# Add TimescaleDB repository
echo "deb https://packagecloud.io/timescale/timescaledb/ubuntu/ $(lsb_release -c -s) main" > /etc/apt/sources.list.d/timescaledb.list
wget --quiet -O - https://packagecloud.io/timescale/timescaledb/gpgkey | apt-key add -
# Add PostgreSQL repository
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
echo "deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list
apt update
$PKG_INSTALL postgresql-15 postgresql-server-dev-15 timescaledb-2-postgresql-15
;;
dnf|yum)
# Add TimescaleDB repository
cat > /etc/yum.repos.d/timescale_timescaledb.repo <<EOF
[timescale_timescaledb]
name=timescale_timescaledb
baseurl=https://packagecloud.io/timescale/timescaledb/el/\$releasever/\$basearch
repo_gpgcheck=1
gpgcheck=0
enabled=1
gpgkey=https://packagecloud.io/timescale/timescaledb/gpgkey
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
metadata_expire=300
EOF
# Add PostgreSQL repository
$PKG_INSTALL https://download.postgresql.org/pub/repos/yum/reporpms/EL-$(rpm -E %rhel)-x86_64/pgdg-redhat-repo-latest.noarch.rpm
$PKG_INSTALL postgresql15-server postgresql15-devel timescaledb-2-postgresql-15
;;
esac
log "[4/8] Configuring PostgreSQL..."
# Stop default PostgreSQL service
systemctl stop $POSTGRES_SERVICE || true
systemctl disable $POSTGRES_SERVICE || true
# Initialize PostgreSQL if needed (RHEL-based systems)
if [[ "$PKG_MGR" =~ ^(dnf|yum)$ ]]; then
/usr/pgsql-15/bin/postgresql-15-setup initdb || true
fi
log "[5/8] Installing etcd..."
ETCD_VER=v3.5.9
case "$PKG_MGR" in
apt)
$PKG_INSTALL etcd-server
;;
dnf|yum)
# Install etcd from binary for RHEL-based systems
curl -L https://github.com/etcd-io/etcd/releases/download/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz -o /tmp/etcd.tar.gz
tar xzf /tmp/etcd.tar.gz -C /tmp
mv /tmp/etcd-${ETCD_VER}-linux-amd64/etcd* /usr/local/bin/
useradd -r -s /bin/false etcd || true
mkdir -p /var/lib/etcd
chown etcd:etcd /var/lib/etcd
chmod 755 /var/lib/etcd
;;
esac
log "[6/8] Installing Patroni..."
$PKG_INSTALL python3 python3-pip
pip3 install patroni[etcd] psycopg2-binary
log "[7/8] Configuring etcd..."
mkdir -p /etc/etcd
chown etcd:etcd /etc/etcd
chmod 755 /etc/etcd
# Create etcd configuration
cat > /etc/etcd/etcd.conf.yml <<EOF
name: 'node-$(hostname)'
data-dir: /var/lib/etcd
listen-client-urls: http://${NODE_IP}:2379,http://127.0.0.1:2379
advertise-client-urls: http://${NODE_IP}:2379
listen-peer-urls: http://${NODE_IP}:2380
initial-advertise-peer-urls: http://${NODE_IP}:2380
initial-cluster: ${ETCD_CLUSTER:-node-$(hostname)=http://${NODE_IP}:2380}
initial-cluster-token: 'etcd-cluster'
initial-cluster-state: 'new'
EOF
chown etcd:etcd /etc/etcd/etcd.conf.yml
chmod 644 /etc/etcd/etcd.conf.yml
# Create etcd systemd service for RHEL-based systems
if [[ "$PKG_MGR" =~ ^(dnf|yum)$ ]]; then
cat > /etc/systemd/system/etcd.service <<EOF
[Unit]
Description=etcd
After=network.target
[Service]
Type=notify
User=etcd
ExecStart=/usr/local/bin/etcd --config-file /etc/etcd/etcd.conf.yml
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF
fi
log "[8/8] Configuring Patroni..."
mkdir -p /etc/patroni
useradd -r -s /bin/bash postgres || true
cat > /etc/patroni/patroni.yml <<EOF
scope: ${CLUSTER_NAME}
namespace: /db/
name: node-$(hostname)
restapi:
listen: ${NODE_IP}:8008
connect_address: ${NODE_IP}:8008
etcd:
hosts: ${NODE_IP}:2379
bootstrap:
dcs:
ttl: 30
loop_wait: 10
retry_timeout: 30
postgresql:
use_pg_rewind: true
use_slots: true
parameters:
wal_level: replica
hot_standby: "on"
max_connections: 100
max_worker_processes: 8
wal_keep_segments: 8
max_wal_senders: 10
max_replication_slots: 10
shared_preload_libraries: 'timescaledb'
initdb:
- encoding: UTF8
- data-checksums
postgresql:
listen: ${NODE_IP}:5432
connect_address: ${NODE_IP}:5432
data_dir: /var/lib/postgresql/15/main
bin_dir: /usr/lib/postgresql/15/bin
pgpass: /tmp/pgpass
authentication:
replication:
username: replicator
password: replicator_password
superuser:
username: postgres
password: postgres_password
tags:
nofailover: false
noloadbalance: false
clonefrom: false
nosync: false
EOF
# Adjust paths for RHEL-based systems
if [[ "$PKG_MGR" =~ ^(dnf|yum)$ ]]; then
sed -i 's|/var/lib/postgresql/15/main|/var/lib/pgsql/15/data|g' /etc/patroni/patroni.yml
sed -i 's|/usr/lib/postgresql/15/bin|/usr/pgsql-15/bin|g' /etc/patroni/patroni.yml
fi
chown postgres:postgres /etc/patroni/patroni.yml
chmod 600 /etc/patroni/patroni.yml
# Create Patroni systemd service
cat > /etc/systemd/system/patroni.service <<EOF
[Unit]
Description=Patroni PostgreSQL Cluster
After=syslog.target network.target
[Service]
Type=simple
User=postgres
ExecStart=/usr/local/bin/patroni /etc/patroni/patroni.yml
ExecReload=/bin/kill -HUP \$MAINPID
KillMode=process
TimeoutSec=30
Restart=no
[Install]
WantedBy=multi-user.target
EOF
# Enable and start services
systemctl daemon-reload
systemctl enable etcd patroni
systemctl start etcd
sleep 5
systemctl start patroni
log "TimescaleDB cluster node installation completed!"
warn "Remember to:"
warn "1. Configure firewall to allow ports 2379, 2380 (etcd) and 5432, 8008 (patroni)"
warn "2. Update ETCD_CLUSTER parameter and re-run on other nodes"
warn "3. Enable TimescaleDB extension: CREATE EXTENSION IF NOT EXISTS timescaledb;"
Review the script before running. Execute with: bash install.sh