Redis Cluster with Sentinel

I wanted to setup a redis cluster with the default setup suggested by redis but after I was ready to test it I found it doesn’t allow you to create multiple databases when in cluster mode.

Since my application is rather sensitive and I need redundancy for my redis instances, I had to opt for deploying Redis with Sentinel which allows you to have multiple instances for redis in a master-replica architecture, and also keep the ability to create multiple databases.

To keep things short, this is what my design looks like:

• 3 VMS
• 3 instances for Redis, 1 Master and 2 replicas
• 3 instances of redis sentinel, each deployed on a different VM with the redis replica or master instances

Here’s a graph of what it looks like:

I’ve used docker to deploy each instance. Here’s what my docker configuration looks like.

version: '3.8'
services:
  redis-master:
    image: reg.zcore.local/redis/redis:7.2.4-bookworm
    container_name: redis-master
    command: redis-server /usr/local/etc/redis/redis_master.conf
    environment:
      - TZ=Asia/Tehran
    ports:
      - "6379:6379"
    network_mode: "host"
    volumes:
      - ./conf:/usr/local/etc/redis

  redis-replica-1:
    image: reg.zcore.local/redis/redis:7.2.4-bookworm
    container_name: redis-replica-1
    command: redis-server /usr/local/etc/redis/redis_replica-1.conf
    environment:
      - TZ=Asia/Tehran
    depends_on:
      - redis-master
    ports:
      - "6380:6380"
    network_mode: "host"
    volumes:
      - ./conf:/usr/local/etc/redis

  redis-replica-2:
    image: reg.zcore.local/redis/redis:7.2.4-bookworm
    container_name: redis-replica-2 
    command: redis-server /usr/local/etc/redis/redis_replica-2.conf
    environment:
      - TZ=Asia/Tehran
    depends_on:
      - redis-master
      - redis-replica-1
    ports:
      - "6381:6381"
    network_mode: "host"
    volumes:
      - ./conf:/usr/local/etc/redis

  sentinel-1:
    image: reg.zcore.local/redis/redis:7.2.4-bookworm
    container_name: srv21-redis-sentinel-1
    ports:
      - 26379:26379
    depends_on:
      - redis-master
      - redis-replica-1
      - redis-replica-2
    command: redis-sentinel /usr/local/etc/redis/sentinel-1.conf
    volumes:
      - ./conf:/usr/local/etc/redis
    network_mode: "host"

  sentinel-2:
    image: reg.zcore.local/redis/redis:7.2.4-bookworm
    container_name: redis-sentinel-2
    ports:
      - 26380:26380
    command: redis-sentinel /usr/local/etc/redis/sentinel-2.conf
    depends_on:
      - redis-master
      - redis-replica-1
      - redis-replica-2
    volumes:
      - ./conf:/usr/local/etc/redis
    network_mode: "host"

  sentinel-3:
    image: reg.zcore.local/redis/redis:7.2.4-bookworm
    container_name: redis-sentinel-3
    ports:
      - 26381:26381
    depends_on:
      - redis-master
      - redis-replica-1
      - redis-replica-2
    command: redis-sentinel /usr/local/etc/redis/sentinel-3.conf
    volumes:
      - ./conf:/usr/local/etc/redis
    network_mode: "host"

Obviously for redundancy and correct operation, we want to separate these instances and run each pair on a different VM.

So:

• VM 1 = Redis master + Sentinel 1
• VM 2 = Redis replica 1 + Sentinel 2
• VM 3 = Redis replica 2 + Sentinel 3

This is what our configuration files should look like:

1. Redis master

port 6379
requirepass <cluster-password>
masterauth <cluster-password>

2. Redis replica 1

port 6379
requirepass <cluster-password>
masterauth <cluster-password>
replica-priority 10
replicaof 192.168.10.11 6379

3. Redis replica 2

port 6379
requirepass <cluster-password>
masterauth <cluster-password>
replica-priority 20
replicaof 192.168.10.11 6379

4. Redis Sentinel config

port 26379
sentinel deny-scripts-reconfig yes
sentinel monitor mymaster 192.168.10.116379 2
sentinel auth-pass mymaster passs
sentinel down-after-milliseconds mymaster 3000
sentinel failover-timeout mymaster 60000

Redis Sentinel configuration is identical for each node. If you plan to run more than one Sentinel instance on each node, then you’ll have to specify different pots for each instance.

One important thing to keep in mind is that the conf/ directory should have global read/write/execute permissions since the redis instances need to update the config file. (chmod -R 777 conf/)

Connecting to Redis Sentinel

Once your Sentinel is running, in your application you should use the Redis Sentinel client to connect to the cluster. This document is the example for connecting to Redis Sentinel with Python.