Join us

Sharded Redis cluster in Kubernetes


This is what we are going to create here in Kubernetes,

A “redis cluster” sharded across 3 master nodes, with a replica (slave) for each master.

List of Kubernetes(K8s) Resources we need to create

1. Namespace: A namespace dedicated for “Redis” cluster and its resources.

2. ConfigMaps: Two configmaps, one with Redis cluster configuration and another for the user (Access
control List)ACL file.

3. Service: A headless service which will be used to get the cluster nodes.

4. StatefulSet: A StatefulSet with 6 replicas ( 3 masters and 3 replicas(slaves)

Set up


Create the namespace “redis” which will hold all the resources related to redis cluster.

                kubectl create namespace redis


Create first ConfigMap which is used as the configurations for redis cluster. Create a file called “redis-config.yaml” and add below ConfigMap definition to it.

There are key configurations to start the nodes in “cluser-enabled” mode, see comments in yaml for details.

File name : redis-config.yaml

                apiVersion: v1
kind: ConfigMap
  name: redis-config
  namespace: redis
  redis.conf: |
    # important configurations
    # ------------------------
    cluster-enabled yes
    # configuration file used by redis to store cluster info - do not create.
    # created by redis, just give a path with a filename
    cluster-config-file /conf/nodes-6379.conf
    # configurations for snaposhot and AOF
    # read more :
    dbfilename dump.rdb
    dir /data
    appendonly yes
    appendfilename "appendonly.aof"
    # Enabled ACL based auth. 
    protected-mode yes
    # This is used by the replics nodes to communicate with master to replicate the data.
    # we are using a user called "replication" for this, and the a strong pwd for the same is given in masterauth
    masterauth `5$!DfwSJ.Y(d:@M
    masteruser replication
    # this is the second ConfiMap will be mounted to. it has the list of uses needed.
    aclfile /conf/acl/users.acl
    # port, each redis nodes will be used
    port 6379
    # More configurations are optional, if not provided, redis will consider default values ------
    # ------ More details on configuration : ------

Apply the ConfigMap yaml file,

                kubectl apply -f redis-config.yaml

Next, create another ConfiMap for the user ACL, this is a list of users that need to be created along with their permissions

File name : redis-acl.yaml

                apiVersion: v1
kind: ConfigMap
  name: redis-acl
  namespace: redis
  users.acl: |
    # user "default" is the default user ( act as admin ) and user "replication" is used by the nodes for the replication.
    user default on #8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918 ~* &* +@all
    user replication on #65cf6f5f48186a4a6c5de02f156f1642b3873451d9de1607147799023dbf4ef8 +psync +replconf +ping
    user worker on #87eba76e7f3164534045ba922e7770fb58bbd14ad732bbf5ba6f11cc56989e6e ~* &* +@all -@dangerous

Users list created by ACL configuration.

  • default : default user with admin previleges. Access should be restricted to administrators. Note : This is needed for all the administrative purpose, use a strong password.
  • worker : user to be used by apps or clients connect to redis cluster.
  • replication : user used by replicas to AUTH and communicate with master.

User Password

The password can be configured either in plain text or SHA-256 encrypted. “#” indicates its SHA-256 hash value, “>” will be used for plain password.

More details : redis ACL

Create the ConfigMap,

                kubectl apply -f redis_acl.yaml


This is the key part, in this file we ill mount the ConfigMaps created above, and create the StatefulSet replicas based on these configurations.

File name : redis.yaml

                apiVersion: apps/v1
kind: StatefulSet
  name: redis
  namespace: redis
  serviceName: redis
  replicas: 6   # 6 replicas, 3 master and 3 replicas(slaves)
      app: redis
        app: redis
      - name: config
        image: redis:7.0.0-alpine
        command: [ "sh", "-c" ]
          - |
            if [ -f "/conf/redis.conf" ]; then
              echo "config exists /conf/redis.conf .. not creating new"
              echo "config doesn't exist copying to /conf/redis.conf"
              cp /tmp/conf/redis.conf /conf/redis.conf
        - name: storage
          subPath: conf
          mountPath: /conf
        - name: config
          mountPath: /tmp/conf/
      - name: redis
        image: redis:7.0.0-alpine
        command: ["redis-server"]
        args: ["/conf/redis.conf"]
            memory: "100M"
            memory: "2000M"
        - containerPort: 6379
          name: redis
        - name: storage
          mountPath: /data
          subPath: data
        - name: storage
          mountPath: /conf
          subPath: conf
        - name: config-acl
          mountPath: /conf/acl/
      - name: config
          name: redis-config
      - name: config-acl
          name: redis-acl
  - metadata:
      name: storage
      accessModes: [ "ReadWriteOnce" ]
          storage: 50Mi

There is a config container under “initContainers”. This will be created and terminated before the redis container starts. The sole purpose of this is to create/copy the configurations required for redis nodes.

Note: Here we are using volumeClaimTemplate, which creates PersistentVolume(PV) and PersistentVolumeClaim(PVC) dynamically for each node. See the link below for details on “How to make the Redis cluster use a pre-defined PV and PVC

Create the StatefulSet,

                kubectl apply -f redis.yaml

Headless Service

A headless service to connect to the 6 replicas ( redis nodes) of the StatefulSet.

Filename: redis-service.yaml

                apiVersion: v1
kind: Service
  name: redis
  namespace: redis
  clusterIP: None # "None" make it a headless service. No cluster IP.
  - port: 6379
    targetPort: 6379
    name: redis
    app: redis

create the headless service,

                kubectl apply -f redis-service.yaml

At this point, if all went well, there will be 6 pods running ( redis-0 to redis-5) in “redis” namespace and a headless service as below,

                > kubectl get all -n redis
NAME          READY   STATUS    RESTARTS   AGE     IP             
pod/redis-0   1/1     Running   0          3m32s   
pod/redis-1   1/1     Running   0          3m13s  
pod/redis-2   1/1     Running   0          9s  
pod/redis-3   1/1     Running   0          3m6s  
pod/redis-4   1/1     Running   0          3m3s 
pod/redis-5   1/1     Running   0          2m59s
service/redis   ClusterIP   None         <none>        6379/TCP   3m32s   app=redis
NAME                     READY   AGE     CONTAINERS   IMAGES
statefulset.apps/redis   6/6     3m33s   redis        redis:7.0.0-alpine

If you check the log of any pod, it should show printing like below, key part is where it says “Ready to accept connections”

                > kubectl logs -n redis pod/redis-0

1:C 05 May 2022 23:28:54.937 # oO0OoO0OoO0Oo Redis is starting 

1:C 05 May 2022 23:28:54.937 # Redis version=7.0.0, bits=64, 
commit=00000000, modified=0, pid=1, just started

1:C 05 May 2022 23:28:54.937 # Configuration loaded

1:M 05 May 2022 23:28:54.937 * monotonic clock: POSIX clock_gettime

1:M 05 May 2022 23:28:54.939 * Node configuration loaded, I'm 

1:M 05 May 2022 23:28:54.939 * Running mode=cluster, port=6379.

1:M 05 May 2022 23:28:54.939 # Server initialized

1:M 05 May 2022 23:28:54.942 * The AOF directory appendonlydir doesn't 

1:M 05 May 2022 23:28:54.946 * SYNC append only file rewrite performed

1:M 05 May 2022 23:28:54.948 * Ready to accept connections

Start Cluster

Now that all the nodes running, we need to start the cluster, This will assign 3 pods as masters and other 3 as replicas of each master,

“exec” into any running redis pod,

                kubectl exec -it -n redis pod/redis-0 -- sh

execute the below command to create the cluster,

                # Note : "-a admin" in the command is the "default" user password 
given in redis-acl.yaml (raw form, not SHA-256 hash value)
# Imp : use a strong password.

/data # redis-cli --cluster create redis-
0.redis.redis.svc.cluster.local:6379 redis-
1.redis.redis.svc.cluster.local:6379 redis-
2.redis.redis.svc.cluster.local:6379 redis-
3.redis.redis.svc.cluster.local:6379 redis-
4.redis.redis.svc.cluster.local:6379 redis-
5.redis.redis.svc.cluster.local:6379 -a admin --cluster-replicas 1

This will ask for a confirmation to accept default configuration, type “yes”. this will create a sharded cluster with 3 master and 3 replicas(salves)

Sample Output,

                /data # redis-cli --cluster create redis-
0.redis.redis.svc.cluster.local:6379 redis-
1.redis.redis.svc.cluster.local:6379 redis-
2.redis.redis.svc.cluster.local:6379 redis-
3.redis.redis.svc.cluster.local:6379 redis-4

.redis.redis.svc.cluster.local:6379 redis-
5.redis.redis.svc.cluster.local:6379 -a admin --cluster-replicas 1

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.

>>> Performing hash slots allocation on 6 nodes...

Master[0] -> Slots 0 - 5460

Master[1] -> Slots 5461 - 10922

Master[2] -> Slots 10923 - 16383

Adding replica redis-4.redis.redis.svc.cluster.local:6379 to redis-

Adding replica redis-5.redis.redis.svc.cluster.local:6379 to redis-

Adding replica redis-3.redis.redis.svc.cluster.local:6379 to redis-

M: 2e2ecf247ee64fabec434a3cbec12bb211824ce4 redis-

slots:[0-5460] (5461 slots) master

M: 4bf5dbf6146eff3355c796883338b9c1c91ba5f2 redis-

slots:[5461-10922] (5462 slots) master

M: 06bd7df09c25f9d4ea433c3b69293cd9adbed304 redis-

slots:[10923-16383] (5461 slots) master

S: 8abb9fda1a2659851f22b298b813d91b14851a8f redis-

replicates 06bd7df09c25f9d4ea433c3b69293cd9adbed304

S: 19cf9a0a04d7bd5182c306b94cf8f589d394577d redis-

replicates 2e2ecf247ee64fabec434a3cbec12bb211824ce4

S: c3eda2d867f5dd2f0781e6bb48627ca40b6066c7 redis-

replicates 4bf5dbf6146eff3355c796883338b9c1c91ba5f2

Can I set the above configuration? (type 'yes' to accept): yes

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join
>>> Performing Cluster Check (using node redis-

M: 2e2ecf247ee64fabec434a3cbec12bb211824ce4 redis-

slots:[0-5460] (5461 slots) master

1 additional replica(s)

S: c3eda2d867f5dd2f0781e6bb48627ca40b6066c7

slots: (0 slots) slave

replicates 4bf5dbf6146eff3355c796883338b9c1c91ba5f2

M: 06bd7df09c25f9d4ea433c3b69293cd9adbed304

slots:[10923-16383] (5461 slots) master

1 additional replica(s)

M: 4bf5dbf6146eff3355c796883338b9c1c91ba5f2

slots:[5461-10922] (5462 slots) master

1 additional replica(s)

S: 8abb9fda1a2659851f22b298b813d91b14851a8f

slots: (0 slots) slave

replicates 06bd7df09c25f9d4ea433c3b69293cd9adbed304

S: 19cf9a0a04d7bd5182c306b94cf8f589d394577d

slots: (0 slots) slave

replicates 2e2ecf247ee64fabec434a3cbec12bb211824ce4

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

At this point, the cluster is created, and its ready to store data. To see the cluster info, “exec” into a pod, then use “redis-cli -c”

                # Note : "-c" tells redis-cli that it is connecting to a cluster. "-a 
admin" is the default user password. 

/data # redis-cli -c -a admin

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.> CLUSTER NODES

myself,master - 0 1651805328000 1 connected 0-5460

master - 0 1651805330000 3 connected 10923-16383

master - 0 1651805330589 2 connected 5461-10922

slave e68bab90451817f0008bb8ec4c9dd10b849565e8 0 1651805329000 2 connected

slave 2690fabc3b37462a08a44148a000efb000d9841c 0 1651805330990 3 connected

slave 17fbef584bbbeac241bd33aa8e9d03b35ab4c94d 0 1651805329985 1 connected

Now, any “Cluster-aware” client can connect to redis, ( ex: jedis ).

There are two ways, a client can connect.

  • It can use the IP’s of the pods by using DNS of headless service
  • Or, more easy way, it can use the FQDN(Fully Qualified Domain Names) to connect. (ex:redis-n.redis.redis.svc.cluster.local)

Note: use the user “worker” (or similar), with less ACL privileges in the apps to connect and use redis.


Only registered users can post comments. Please, login or signup.

Start blogging about your favorite technologies, reach more readers and earn rewards!

Join other developers and claim your FAUN account now!


Jithin Scaria

Enterprise Architect

Enterprise Architect — Kubernetes(K8s) | Full-stack Engineer | Data Engineer | Development — Testing — Integration — Deployment | Cloud | Project Management
User Popularity



Total Hits