Question

When using consistent hashing to scale a cluster, what is the primary structural mechanism used to prevent the &#x27;thundering herd&#x27; effect when a single node joins or leaves the ring?

Accepted Answer

The primary structural mechanism used to prevent the thundering herd effect in consistent hashing is the use of virtual nodes. Consistent hashing maps both data items and physical server nodes onto a circular coordinate space called a ring. In a basic implementation, if a single physical node fails or is added, the immediate neighbor in the ring must absorb or shed all the keys previously assigned to that specific node, which creates a spike in traffic known as the thundering herd. Virtual nodes solve this by mapping a single physical node to multiple, non-contiguous points scattered across the ring. Because each physical node is represented by many virtual nodes, the keyspace is fragmented into smaller, non-adjacent segments. When a node leaves or joins, the responsibility for its keys is redistributed across many different physical neighbors rather than a single one. This distributes the load of migration across the entire cluster, ensuring that no single node experiences a sudden, overwhelming surge in requests or data transfer when the topology changes.

Home → All Courses → Engineering and Technology Courses → Distributed Systems for Internet-Scale Services → Flashcard

When using consistent hashing to scale a cluster, what is the primary structural mechanism used to prevent the 'thundering herd' effect when a single node joins or leaves the ring?