Redis Cluster Internals

1. Overview

Redis Cluster is Redis’s distributed solution, designed to achieve data sharding and high availability. By distributing data across multiple nodes, each responsible for a subset of hash slots, it supports horizontal scaling. At the same time, through master-slave replication and automatic failover, it ensures the system keeps running when a node fails.

2. Data Sharding

2.1 Hash Slots

  • Redis Cluster divides the entire keyspace into CLUSTER_SLOTS hash slots, i.e., 16384.
  • Each key is mapped to a hash slot by computing a CRC16 hash of its name and taking it modulo 16384.
  • Each master node (Master Node) is responsible for a set of hash slots.
  • In the source, the slots array in the clusterNode struct marks the slots handled by that node:
unsigned char slots[CLUSTER_SLOTS/8]; /* slots handled by this node */

2.2 Key-to-Slot Mapping

  • The mapping function keyHashSlot(key) implements the conversion from a key name to a slot number.
  • This design allows the cluster to migrate slots between nodes dynamically, without rehashing all data — only the data of specific slots needs to be migrated.

3. Node Communication and Discovery

3.1 Cluster Bus

  • All cluster nodes establish TCP connections via a port offset (default: client port + 10000), forming an internal binary-protocol communication channel called the “cluster bus”.
  • The cluster bus uses the Gossip protocol to propagate messages, ensuring eventual consistency of cluster state.

3.2 Gossip Protocol

  • Message types: nodes exchange PING, PONG, MEET and other messages, carrying their own state and known node information.
  • Member discovery: a new node joins the cluster via the CLUSTER MEET command; the receiver sends its own node list back, enabling automatic discovery.
  • State propagation: nodes periodically send PING to other nodes and receive PONG to detect node health.
  • Multiple message types are defined in the source:
#define CLUSTERMSG_TYPE_PING 0          /* Ping */
#define CLUSTERMSG_TYPE_PONG 1          /* Pong (reply to Ping) */
#define CLUSTERMSG_TYPE_MEET 2          /* Meet "let's join" message */

4. Failure Detection and Failover

4.1 Failure Detection

  • PFAIL (Possibly Fail): when a node does not receive a PONG reply from another node within a period (configured by node timeout), it marks that node as subjectively down.
  • FAIL (objectively down): when a node believes a master is subjectively down, it asks other masters for confirmation. If a majority of masters also consider the node down, consensus is reached and it is marked as objectively down. This requires agreement from quorum nodes.
  • The flags field in the clusterNode struct records the node’s status:
int flags;      /* CLUSTER_NODE_... */
#define CLUSTER_NODE_PFAIL 4      /* Failure? Need acknowledge */
#define CLUSTER_NODE_FAIL 8       /* The node is believed to be malfunctioning */

4.2 Failover

When a master is marked objectively down, its slave node initiates a failover and elects itself as the new master.

4.2.1 Election Process

  1. Eligibility check: the slave first checks whether it is eligible to participate in the election. For example, it must be sufficiently synchronized with the old master to avoid losing too much data.
  2. Request votes: an eligible slave sends a FAILOVER_AUTH_REQUEST message to all masters, requesting a vote.
  3. Voting decision: each master votes only once per election round, typically for the first slave that requested and met the conditions.
  4. Win the election: if the slave receives more than half (quorum) of the votes, the election succeeds and failover begins.

4.2.2 Source Analysis

  • clusterHandleSlaveFailover is the core logic that handles slave failover.
  • The failover_auth_sent flag prevents a slave from repeatedly sending vote requests.
  • failover_auth_count counts the number of votes received.

5. High Availability Workflow

5.1 Master-Slave Replication

  • Slaves continuously replicate the master’s data, ensuring redundancy.
  • When the master fails, a slave can take over, ensuring uninterrupted service.

5.2 Automatic Failover Flow

  1. Detection: a majority of nodes in the cluster confirm the master has failed (objectively down).
  2. Election: the most suitable slave (usually the one with the largest replication offset) becomes the new master through the election process above.
  3. Update configuration: the new master updates its configEpoch (configuration epoch), a monotonically increasing version number used to resolve split-brain. It broadcasts an UPDATE message to other nodes, notifying them of its new role and slot ownership.
  4. Redirect clients: other nodes and clients receive the MOVED redirection error and send subsequent requests to the new master.

5.3 Config Epoch

  • configEpoch is the key mechanism for resolving conflicts. When two nodes dispute ownership of the same slot, the one with the higher configEpoch wins.
  • This ensures that even under network partitions, a single authoritative node is elected, preventing data inconsistency.
  • This field is included in the clusterNode struct:
uint64_t configEpoch; /* Last configEpoch observed for this node */

6. Design Rationale

  • Decentralization: no single point of failure; all nodes are equal and reach consensus via the Gossip protocol.
  • High performance: clients can connect directly to the node responsible for the keys they need, reducing proxy-layer overhead.
  • Scalability: horizontal scaling is easy by adding nodes and migrating slots.
  • Eventual consistency: an eventual-consistency model is adopted instead of strong consistency, trading some consistency for higher availability and partition tolerance, consistent with the CAP theorem.
  • Simplicity: compared to external consistency-coordination services (e.g., ZooKeeper), Redis Cluster’s design is relatively simple, easy to deploy and maintain.