Learn how to configure and optimize Linux RAID arrays with mdadm for maximum disk performance. This comprehensive guide covers RAID level selection, advanced mdadm parameters, kernel I/O scheduler optimization, and file system tuning for high-throughput workloads.
Prerequisites
- Multiple disks (minimum 4 for RAID 10)
- Root access
- Basic understanding of storage concepts
What this solves
Linux RAID configuration with mdadm provides redundancy and performance benefits, but default settings often leave significant performance gains on the table. This tutorial shows you how to optimize RAID arrays for maximum disk throughput by selecting the right RAID level, configuring advanced mdadm parameters, tuning kernel I/O schedulers, and optimizing file system mount options for high-performance workloads.
Understanding RAID levels and performance characteristics
Different RAID levels provide varying levels of performance, redundancy, and storage efficiency. Understanding these trade-offs is crucial for optimal configuration.
| RAID Level | Read Performance | Write Performance | Redundancy | Storage Efficiency |
|---|---|---|---|---|
| RAID 0 | Excellent | Excellent | None | 100% |
| RAID 1 | Good | Moderate | 1 disk failure | 50% |
| RAID 5 | Good | Poor | 1 disk failure | 75% (4 disks) |
| RAID 6 | Good | Poor | 2 disk failures | 67% (4 disks) |
| RAID 10 | Excellent | Good | Multiple disk failures | 50% |
Step-by-step RAID configuration and optimization
Install mdadm and disk utilities
Install the necessary tools for RAID management and disk optimization.
sudo apt update
sudo apt install -y mdadm gdisk parted smartmontools hdparm
Identify and prepare disks
List available disks and verify they are suitable for RAID configuration.
sudo fdisk -l
lsblk
sudo smartctl -a /dev/sdb
sudo smartctl -a /dev/sdc
sudo smartctl -a /dev/sdd
sudo smartctl -a /dev/sde
Clear any existing partition tables and create GPT partition tables for optimal alignment.
sudo sgdisk --zap-all /dev/sdb
sudo sgdisk --zap-all /dev/sdc
sudo sgdisk --zap-all /dev/sdd
sudo sgdisk --zap-all /dev/sde
Create optimized RAID 10 array
Create a RAID 10 array with optimized parameters for maximum performance. The chunk size and layout significantly impact performance.
sudo mdadm --create /dev/md0 \
--level=10 \
--raid-devices=4 \
--layout=f2 \
--chunk=512 \
--bitmap=internal \
--assume-clean \
/dev/sdb /dev/sdc /dev/sdd /dev/sde
Save the RAID configuration to ensure it persists across reboots.
sudo mdadm --detail --scan | sudo tee -a /etc/mdadm/mdadm.conf
sudo update-initramfs -u
Optimize RAID array parameters
Configure advanced RAID parameters for maximum performance.
# Set stripe cache size for better write performance
echo 8192 | sudo tee /sys/block/md0/md/stripe_cache_size
Optimize read-ahead settings
sudo blockdev --setra 65536 /dev/md0
Set appropriate queue depth
echo mq-deadline | sudo tee /sys/block/md0/queue/scheduler
echo 128 | sudo tee /sys/block/md0/queue/nr_requests
Make these optimizations persistent by creating a systemd service.
[Unit]
Description=Optimize RAID array performance
After=local-fs.target
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/bash -c 'echo 8192 > /sys/block/md0/md/stripe_cache_size'
ExecStart=/bin/bash -c 'blockdev --setra 65536 /dev/md0'
ExecStart=/bin/bash -c 'echo mq-deadline > /sys/block/md0/queue/scheduler'
ExecStart=/bin/bash -c 'echo 128 > /sys/block/md0/queue/nr_requests'
[Install]
WantedBy=multi-user.target
sudo systemctl enable raid-optimize.service
sudo systemctl start raid-optimize.service
Configure optimal I/O scheduler for individual disks
Set the most appropriate I/O scheduler for the underlying disks based on their type.
# For SSDs, use none or mq-deadline
echo none | sudo tee /sys/block/sdb/queue/scheduler
echo none | sudo tee /sys/block/sdc/queue/scheduler
echo none | sudo tee /sys/block/sdd/queue/scheduler
echo none | sudo tee /sys/block/sde/queue/scheduler
For HDDs, use mq-deadline or bfq
echo mq-deadline | sudo tee /sys/block/sdb/queue/scheduler
Make I/O scheduler settings persistent across reboots.
# Set I/O scheduler for SSDs
ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="0", ATTR{queue/scheduler}="none"
Set I/O scheduler for HDDs
ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="1", ATTR{queue/scheduler}="mq-deadline"
Optimize kernel parameters for RAID performance
Configure kernel parameters to maximize RAID array performance and reduce latency.
# Increase dirty page cache limits for better write performance
vm.dirty_ratio = 15
vm.dirty_background_ratio = 5
vm.dirty_expire_centisecs = 1000
vm.dirty_writeback_centisecs = 100
Optimize memory allocation for RAID
vm.vfs_cache_pressure = 50
vm.swappiness = 10
Increase maximum read-ahead size
vm.max_readahead_kb = 512
Optimize network and I/O performance
net.core.rmem_default = 262144
net.core.rmem_max = 16777216
net.core.wmem_default = 262144
net.core.wmem_max = 16777216
sudo sysctl -p /etc/sysctl.d/99-raid-performance.conf
Create optimized file system
Create an ext4 file system with parameters optimized for RAID performance.
sudo mkfs.ext4 -F \
-b 4096 \
-E stride=128,stripe-width=256 \
-O ^has_journal \
-m 1 \
-L raid_storage \
/dev/md0
For XFS file system (better for large files and parallel I/O):
# sudo mkfs.xfs -f \
-d sunit=1024,swidth=2048 \
-l logdev=/dev/md0,size=128m \
-L raid_storage \
/dev/md0
Configure optimized mount options
Create the mount point and configure optimal mount options for maximum performance.
sudo mkdir -p /mnt/raid_storage
Add the optimized mount configuration to fstab.
/dev/md0 /mnt/raid_storage ext4 defaults,noatime,nodiratime,data=writeback,barrier=0,delalloc,max_batch_time=0 0 2
For XFS file systems, use these mount options instead:
# /dev/md0 /mnt/raid_storage xfs defaults,noatime,nodiratime,logbufs=8,logbsize=32k,largeio,inode64 0 2
Mount the file system and verify the configuration.
sudo mount /mnt/raid_storage
sudo chown $USER:$USER /mnt/raid_storage
Monitor and optimize RAID performance
Set up monitoring and performance tracking for your RAID array.
# Monitor RAID status
sudo mdadm --detail /dev/md0
Check real-time I/O statistics
sudo iostat -x 1
Monitor RAID rebuild progress (if needed)
watch -n 1 'cat /proc/mdstat'
Create a monitoring script for ongoing RAID health checks.
#!/bin/bash
Check RAID array health
RAID_STATUS=$(sudo mdadm --detail /dev/md0 | grep "State :" | awk '{print $3}')
if [ "$RAID_STATUS" != "clean" ]; then
echo "RAID array /dev/md0 is not clean: $RAID_STATUS"
# Send alert (configure mail or notification service)
logger "RAID Alert: Array /dev/md0 status is $RAID_STATUS"
else
echo "RAID array /dev/md0 is healthy"
fi
Check disk temperatures and SMART status
for disk in sdb sdc sdd sde; do
TEMP=$(sudo smartctl -A /dev/$disk | grep Temperature_Celsius | awk '{print $10}')
if [ "$TEMP" -gt 50 ]; then
echo "Warning: /dev/$disk temperature is ${TEMP}°C"
fi
done
sudo chmod +x /usr/local/bin/raid-monitor.sh
Add to crontab for regular monitoring
echo "/5 * /usr/local/bin/raid-monitor.sh" | sudo crontab -
Advanced performance tuning techniques
Enable write-back caching for maximum write performance
Configure write-back caching on the individual disks for improved write performance.
# Enable write caching (only with UPS/battery backup)
sudo hdparm -W1 /dev/sdb
sudo hdparm -W1 /dev/sdc
sudo hdparm -W1 /dev/sdd
sudo hdparm -W1 /dev/sde
Verify write cache status
sudo hdparm -W /dev/sdb
Optimize for specific workload patterns
Configure RAID parameters based on your specific use case.
# For database workloads (small random I/O)
echo 256 | sudo tee /sys/block/md0/md/stripe_cache_size
sudo blockdev --setra 2048 /dev/md0
For large file storage (sequential I/O)
echo 16384 | sudo tee /sys/block/md0/md/stripe_cache_size
sudo blockdev --setra 131072 /dev/md0
For mixed workloads (balanced)
echo 8192 | sudo tee /sys/block/md0/md/stripe_cache_size
sudo blockdev --setra 65536 /dev/md0
Set up performance benchmarking
Install and configure tools to measure RAID performance improvements.
sudo apt install -y fio bonnie++ iozone3
Run comprehensive performance tests to validate your optimizations.
# Test random read/write performance
sudo fio --name=raid-test \
--directory=/mnt/raid_storage \
--rw=randrw \
--rwmixread=70 \
--bs=4k \
--size=1G \
--numjobs=4 \
--runtime=60 \
--group_reporting
Test sequential read/write performance
sudo fio --name=seq-test \
--directory=/mnt/raid_storage \
--rw=rw \
--bs=1M \
--size=2G \
--numjobs=1 \
--runtime=30 \
--group_reporting
Verify your setup
# Check RAID array status
sudo mdadm --detail /dev/md0
Verify mount and file system
df -h /mnt/raid_storage
mount | grep md0
Check current I/O scheduler and parameters
cat /sys/block/md0/queue/scheduler
cat /sys/block/md0/md/stripe_cache_size
Verify individual disk settings
for disk in sdb sdc sdd sde; do
echo "=== /dev/$disk ==="
cat /sys/block/$disk/queue/scheduler
sudo hdparm -W /dev/$disk
done
Test basic I/O performance
time dd if=/dev/zero of=/mnt/raid_storage/test_file bs=1M count=1024 oflag=direct
time dd if=/mnt/raid_storage/test_file of=/dev/null bs=1M iflag=direct
rm /mnt/raid_storage/test_file
Common issues
| Symptom | Cause | Fix |
|---|---|---|
| Poor write performance | Write cache disabled or small stripe cache | Enable write cache with hdparm -W1, increase stripe cache size |
| High CPU usage during I/O | Inappropriate I/O scheduler | Use 'none' scheduler for SSDs, 'mq-deadline' for HDDs |
| Array won't start on boot | Missing mdadm.conf entries | Run mdadm --detail --scan >> /etc/mdadm/mdadm.conf |
| Inconsistent performance | Misaligned partitions or wrong stride values | Recreate filesystem with correct stride/stripe-width values |
| Disk overheating | Insufficient cooling or too aggressive caching | Improve case ventilation, reduce stripe cache size |
Next steps
- Benchmark and optimize Linux disk I/O performance with fio testing for detailed performance analysis
- Set up Linux performance monitoring with perf tools and flame graphs for ongoing performance tracking
- Configure RAID monitoring with Prometheus and Grafana for comprehensive array health monitoring
- Implement RAID backup strategies with LVM snapshots for data protection
Automated install script
Run this to automate the entire setup
#!/usr/bin/env bash
set -euo pipefail
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m'
# Default values
RAID_LEVEL="10"
CHUNK_SIZE="512"
DISKS=()
FORCE=false
# Cleanup function
cleanup() {
if [ $? -ne 0 ]; then
echo -e "${RED}[ERROR] Script failed. Check logs above.${NC}" >&2
if [ ${#DISKS[@]} -gt 0 ] && [ -e /dev/md0 ]; then
echo -e "${YELLOW}[CLEANUP] Stopping RAID array...${NC}"
mdadm --stop /dev/md0 2>/dev/null || true
fi
fi
}
trap cleanup ERR
usage() {
cat << EOF
Usage: $0 [OPTIONS] <disk1> <disk2> [disk3] [disk4]
Optimize Linux RAID configuration for maximum disk performance.
Options:
-l, --level LEVEL RAID level (0, 1, 5, 6, 10) [default: 10]
-c, --chunk SIZE Chunk size in KB [default: 512]
-f, --force Skip confirmation prompts
-h, --help Show this help
Examples:
$0 /dev/sdb /dev/sdc /dev/sdd /dev/sde
$0 -l 1 -c 256 /dev/sdb /dev/sdc
$0 --force --level 0 /dev/nvme0n1 /dev/nvme1n1
Note: This will DESTROY all data on specified disks!
EOF
}
# Parse arguments
while [[ $# -gt 0 ]]; do
case $1 in
-l|--level)
RAID_LEVEL="$2"
shift 2
;;
-c|--chunk)
CHUNK_SIZE="$2"
shift 2
;;
-f|--force)
FORCE=true
shift
;;
-h|--help)
usage
exit 0
;;
/dev/*)
DISKS+=("$1")
shift
;;
*)
echo -e "${RED}Unknown option: $1${NC}" >&2
usage
exit 1
;;
esac
done
# Validation
if [ ${#DISKS[@]} -lt 2 ]; then
echo -e "${RED}Error: At least 2 disks required${NC}" >&2
usage
exit 1
fi
if [ "$EUID" -ne 0 ]; then
echo -e "${RED}Error: This script must be run as root${NC}" >&2
exit 1
fi
# Detect distribution
if [ -f /etc/os-release ]; then
. /etc/os-release
case "$ID" in
ubuntu|debian)
PKG_MGR="apt"
PKG_INSTALL="apt install -y"
MDADM_CONF="/etc/mdadm/mdadm.conf"
UPDATE_INITRD="update-initramfs -u"
;;
almalinux|rocky|centos|rhel|ol|fedora)
PKG_MGR="dnf"
PKG_INSTALL="dnf install -y"
MDADM_CONF="/etc/mdadm.conf"
UPDATE_INITRD="dracut -f"
;;
amzn)
PKG_MGR="yum"
PKG_INSTALL="yum install -y"
MDADM_CONF="/etc/mdadm.conf"
UPDATE_INITRD="dracut -f"
;;
*)
echo -e "${RED}Unsupported distribution: $ID${NC}" >&2
exit 1
;;
esac
else
echo -e "${RED}Cannot detect distribution${NC}" >&2
exit 1
fi
echo -e "${BLUE}Linux RAID Performance Optimization Setup${NC}"
echo "=========================================="
# Confirmation
if [ "$FORCE" = false ]; then
echo -e "${YELLOW}This will DESTROY all data on: ${DISKS[*]}${NC}"
echo -e "${YELLOW}RAID Level: $RAID_LEVEL, Chunk Size: ${CHUNK_SIZE}KB${NC}"
read -p "Continue? (yes/NO): " confirm
if [ "$confirm" != "yes" ]; then
echo "Aborted."
exit 0
fi
fi
echo -e "${GREEN}[1/8] Installing required packages...${NC}"
if [ "$PKG_MGR" = "apt" ]; then
apt update
fi
$PKG_INSTALL mdadm gdisk parted smartmontools hdparm
echo -e "${GREEN}[2/8] Validating disks...${NC}"
for disk in "${DISKS[@]}"; do
if [ ! -b "$disk" ]; then
echo -e "${RED}Error: $disk is not a valid block device${NC}" >&2
exit 1
fi
echo "Checking $disk..."
smartctl -H "$disk" || echo -e "${YELLOW}Warning: SMART check failed for $disk${NC}"
done
echo -e "${GREEN}[3/8] Preparing disks...${NC}"
for disk in "${DISKS[@]}"; do
echo "Clearing partition table on $disk..."
sgdisk --zap-all "$disk"
# Set I/O scheduler based on disk type
disk_name=$(basename "$disk")
if [[ "$disk" == *"nvme"* ]] || [ "$(cat /sys/block/$disk_name/queue/rotational 2>/dev/null)" = "0" ]; then
echo "none" > "/sys/block/$disk_name/queue/scheduler" 2>/dev/null || echo "mq-deadline" > "/sys/block/$disk_name/queue/scheduler"
echo "Configured SSD scheduler for $disk"
else
echo "mq-deadline" > "/sys/block/$disk_name/queue/scheduler"
echo "Configured HDD scheduler for $disk"
fi
done
echo -e "${GREEN}[4/8] Creating RAID array...${NC}"
mdadm_args=(
--create /dev/md0
--level="$RAID_LEVEL"
--raid-devices="${#DISKS[@]}"
--chunk="$CHUNK_SIZE"
--bitmap=internal
--assume-clean
)
# Add layout for RAID 10
if [ "$RAID_LEVEL" = "10" ]; then
mdadm_args+=(--layout=f2)
fi
mdadm "${mdadm_args[@]}" "${DISKS[@]}"
# Wait for array to be ready
echo "Waiting for RAID array to initialize..."
sleep 5
echo -e "${GREEN}[5/8] Saving RAID configuration...${NC}"
mkdir -p "$(dirname "$MDADM_CONF")"
mdadm --detail --scan | tee -a "$MDADM_CONF"
$UPDATE_INITRD
echo -e "${GREEN}[6/8] Optimizing RAID parameters...${NC}"
# Stripe cache size (for RAID 4/5/6)
if [[ "$RAID_LEVEL" =~ ^(4|5|6)$ ]]; then
echo 8192 > /sys/block/md0/md/stripe_cache_size
fi
# Read-ahead optimization
blockdev --setra 65536 /dev/md0
# Queue settings
echo "mq-deadline" > /sys/block/md0/queue/scheduler
echo 128 > /sys/block/md0/queue/nr_requests
# Disable NCQ for better performance with RAID
echo 1 > /sys/block/md0/queue/nomerges
echo -e "${GREEN}[7/8] Creating optimization service...${NC}"
cat > /etc/systemd/system/raid-optimize.service << 'EOF'
[Unit]
Description=Optimize RAID array performance
After=local-fs.target
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/bash -c '[ -f /sys/block/md0/md/stripe_cache_size ] && echo 8192 > /sys/block/md0/md/stripe_cache_size || true'
ExecStart=/bin/bash -c 'blockdev --setra 65536 /dev/md0'
ExecStart=/bin/bash -c 'echo mq-deadline > /sys/block/md0/queue/scheduler'
ExecStart=/bin/bash -c 'echo 128 > /sys/block/md0/queue/nr_requests'
ExecStart=/bin/bash -c 'echo 1 > /sys/block/md0/queue/nomerges'
[Install]
WantedBy=multi-user.target
EOF
chmod 644 /etc/systemd/system/raid-optimize.service
systemctl daemon-reload
systemctl enable raid-optimize.service
# Create sysctl optimizations
cat > /etc/sysctl.d/99-raid-performance.conf << EOF
# RAID Performance Optimizations
vm.dirty_ratio = 15
vm.dirty_background_ratio = 5
vm.dirty_expire_centisecs = 12000
vm.dirty_writeback_centisecs = 1500
vm.vfs_cache_pressure = 50
kernel.sched_min_granularity_ns = 10000000
kernel.sched_wakeup_granularity_ns = 15000000
EOF
sysctl -p /etc/sysctl.d/99-raid-performance.conf
echo -e "${GREEN}[8/8] Verifying RAID array...${NC}"
if [ ! -b /dev/md0 ]; then
echo -e "${RED}Error: RAID array /dev/md0 not created${NC}" >&2
exit 1
fi
echo -e "${BLUE}RAID Array Details:${NC}"
mdadm --detail /dev/md0
echo -e "${BLUE}Performance Settings:${NC}"
echo "Scheduler: $(cat /sys/block/md0/queue/scheduler | grep -o '\[.*\]' | tr -d '[]')"
echo "Read-ahead: $(blockdev --getra /dev/md0) sectors"
echo "Queue depth: $(cat /sys/block/md0/queue/nr_requests)"
echo
echo -e "${GREEN}✓ RAID optimization complete!${NC}"
echo
echo -e "${YELLOW}Next steps:${NC}"
echo "1. Create filesystem: mkfs.ext4 -F /dev/md0"
echo "2. Mount with optimizations: mount -o noatime,nodiratime,barrier=0 /dev/md0 /mnt"
echo "3. Add to /etc/fstab for persistence"
echo
echo -e "${YELLOW}Performance monitoring:${NC}"
echo "• Monitor: watch -n1 'cat /proc/mdstat'"
echo "• Test: dd if=/dev/zero of=/mnt/testfile bs=1M count=1000 oflag=direct"
Review the script before running. Execute with: bash install.sh