Redis Cluster setup for horizontal scaling

Configuring Redis Cluster for Scaling

A single Redis is limited by RAM of one server and throughput of one CPU core (Redis is single-threaded for data operations). Redis Cluster solves both: data is sharded across nodes, each node handles its own key range. A 6-node cluster (3 masters + 3 replicas) is the minimum configuration.

How Sharding Works

Redis Cluster divides the key space into 16,384 hash slots. Each master node handles a slot range. When operating on a key, Redis calculates CRC16(key) % 16384 and routes the request to the right node.

When adding a new node—slots migrate between nodes without stopping the cluster. The client gets a MOVED error when accessing a slot on the wrong node and auto-switches.

Setting Up a 6-Node Cluster

Configuration redis-cluster.conf for each node (only port changes):

port 7001
cluster-enabled yes
cluster-config-file nodes-7001.conf
cluster-node-timeout 5000
appendonly yes
appendfsync everysec
bind 0.0.0.0
requirepass ClusterPassword123
masterauth ClusterPassword123

# Replication
replica-lazy-flush no
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no

# Performance
tcp-backlog 511
hz 10
aof-rewrite-incremental-fsync yes

Starting 6 instances (ports 7001–7006):

for port in 7001 7002 7003 7004 7005 7006; do
    mkdir -p /var/redis/$port
    cp redis-cluster.conf /var/redis/$port/redis.conf
    sed -i "s/port 7001/port $port/" /var/redis/$port/redis.conf
    sed -i "s/nodes-7001/nodes-$port/" /var/redis/$port/redis.conf
    redis-server /var/redis/$port/redis.conf --daemonize yes
done

Creating the cluster:

redis-cli --cluster create \
  127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 \
  127.0.0.1:7004 127.0.0.1:7005 127.0.0.1:7006 \
  --cluster-replicas 1 \
  -a ClusterPassword123

--cluster-replicas 1—one replica per master. Redis auto-distributes: 3 masters (7001, 7002, 7003) and 3 replicas (7004, 7005, 7006).

Docker Compose for Development

version: '3'
services:
  redis-cluster:
    image: grokzen/redis-cluster:7.0.10
    ports:
      - "7001-7006:7001-7006"
    environment:
      IP: 127.0.0.1
      INITIAL_PORT: 7001
      MASTERS: 3
      SLAVES_PER_MASTER: 1
      MAX_MEMORY: 256mb
      MAX_MEMORY_POLICY: allkeys-lru

Cluster Limitations: Hash Tags

In a cluster, multi-key commands (MGET, MSET, pipeline) don't work if keys are on different slots. To group keys on one slot—use hash tags: part of the key in {} is used to calculate the slot.

# These keys land on different slots—MGET doesn't work
MGET user:1:profile user:2:profile

# With hash tag {user:1}—both on one slot
SET {user:1}:profile "..."
SET {user:1}:settings "..."
MGET {user:1}:profile {user:1}:settings  # OK

In Laravel—hash tag setup for transactions:

// Ensure keys with {user:1} segment land on one slot
Cache::tags(["user:{$userId}"])->put("profile", $data, 3600);
Cache::tags(["user:{$userId}"])->put("settings", $data, 3600);
Cache::tags(["user:{$userId}"])->flush();

Connecting Clients to Cluster

PHP with phpredis:

$redis = new RedisCluster(null, [
    '127.0.0.1:7001',
    '127.0.0.1:7002',
    '127.0.0.1:7003',
], 1.5, 1.5, true, 'ClusterPassword123');

// phpredis auto-routes by slots
$redis->set('key', 'value');
$redis->get('key');

Predis:

use Predis\Client;

$client = new Client(
    [
        'tcp://127.0.0.1:7001?password=ClusterPassword123',
        'tcp://127.0.0.1:7002?password=ClusterPassword123',
        'tcp://127.0.0.1:7003?password=ClusterPassword123',
    ],
    ['cluster' => 'redis']
);

Laravel config/database.php:

'redis' => [
    'client' => 'phpredis',
    'options' => [
        'cluster' => 'redis',
        'prefix' => '',
        'parameters' => ['password' => env('REDIS_PASSWORD')],
    ],
    'clusters' => [
        'default' => [
            ['host' => '127.0.0.1', 'port' => 7001, 'password' => env('REDIS_PASSWORD')],
            ['host' => '127.0.0.1', 'port' => 7002, 'password' => env('REDIS_PASSWORD')],
            ['host' => '127.0.0.1', 'port' => 7003, 'password' => env('REDIS_PASSWORD')],
        ],
    ],
],

Cluster Management

# Cluster status
redis-cli -c -p 7001 -a ClusterPassword123 cluster info

# Nodes and roles
redis-cli -c -p 7001 -a ClusterPassword123 cluster nodes

# Slot distribution check
redis-cli --cluster check 127.0.0.1:7001 -a ClusterPassword123

# Add new master node
redis-cli --cluster add-node 127.0.0.1:7007 127.0.0.1:7001 -a ClusterPassword123

# Reshard slots to new node
redis-cli --cluster reshard 127.0.0.1:7001 -a ClusterPassword123

# Add replica to new node
redis-cli --cluster add-node 127.0.0.1:7008 127.0.0.1:7001 \
  --cluster-slave --cluster-master-id <master-node-id> \
  -a ClusterPassword123

Failover

When a master node fails, a replica is automatically promoted to master (within cluster-node-timeout, default 5 seconds). The app gets a CLUSTERDOWN error during this time—retry logic should be implemented.

// Retry on CLUSTERDOWN
$attempts = 0;
while ($attempts < 3) {
    try {
        $result = $redis->get($key);
        break;
    } catch (\RedisClusterException $e) {
        if (++$attempts >= 3) throw $e;
        usleep(500000); // 500ms
    }
}

Monitoring

# Metrics per node
for port in 7001 7002 7003 7004 7005 7006; do
    echo "=== Node $port ==="
    redis-cli -p $port -a ClusterPassword123 INFO replication | grep -E "role|master_host|connected_slaves"
done

Prometheus redis_exporter can collect cluster metrics—specify just one node, it will traverse the rest via cluster nodes.

Timeline

Deploying Redis Cluster from 6 nodes with setup, failover testing, and monitoring—2–3 business days. Adapting an existing application for cluster (hash tags, cluster client, handling MOVED/ASK errors)—1–2 days depending on Redis usage complexity in code.