Consistent hashing and random trees
David R. Karger, Eric Lehman, Tom Leighton et al. · ACM Symposium on Theory of Computing (STOC) · 1997
We describe a family of caching protocols for distrib-uted networks that can be used to decrease or eliminate the occurrence of hot spots in the network. Our protocols are particularly designed for use with very large networks such as the Internet, where delays caused by hot spots can be severe, and where it is not feasible for every server to have complete information about the current state of the entire network. The protocols are easy to implement using existing network protocols such as TCP/IP, and require very little overhead. The protocols work with local control, make efficient use of existing resources, and scale gracefully as the network grows. Our caching protocols are based on a special kind of hashing that we call consistent hashing. Roughly speaking, a consistent hash function is one which changes minimally as the range of the function changes. Through the development of good consistent hash functions, we are able to develop caching protocols which do not require users to have a current or even consistent view of the network. We believe that consistent hash functions may eventually prove to be useful in other applications such as distributed name servers and/or quorum systems.
Cited by 1,972 works. doi:10.1145/258533.258660
Authors
References
- Efficient dispersal of information for security, load balancing, and fault tolerance
- A reliable multicast framework for light-weight sessions and application level framing
- A Hierarchical Internet Object Cache
- A reliable multicast framework for light-weight sessions and application level framing
- Implementing global memory management in a workstation cluster
- Making World Wide Web Caching Servers Cooperate
- LH*—a scalable, distributed data structure
- Chernoff–Hoeffding Bounds for Applications with Limited Independence
- Fast fault-tolerant concurrent access to shared objects
- The Availability of Quorum Systems