Consistent Hashing and Random Trees
A hash whose answers barely change when the set of machines does - invented for web caches, not for databases.
Read the original ↗Everyone in infrastructure can draw the ring. Almost nobody has read the paper, which is a shame, because it is not about databases, it is barely about the ring, and the word everybody repeats - consistent - means something other than what they think.
Included with a plan
This discussion is included with a plan. The first three lessons of every track are free, with their boards and videos. A plan unlocks every lesson, board, video, lab and paper for one price.
Related on chalk
The same idea as a board, lab, video or lesson.
- BoardSplitting the Ledgerthe ring itself, and what the ring is for
- BoardThe Slate Next to the Stovethe cache - which is the application the paper was actually written for
- BoardOpen Connect CDNhot pages served from many places, the 1997 problem at present-day scale
- BoardThe Shard Key That Liedthe hot key no amount of ring tuning moves
- BoardPartitioningthe textbook treatment, including the skew the ring does not fix
- LabThe ringmove a node and watch which keys move - the monotonicity property, run
- LabSplitting the ledgerwhat a badly chosen key does to an evenly balanced ring
- LabThe slate next to the stovethe cache whose hot spots this paper set out to relieve
- VideoSharding: splitting the ledgerwhy the keyspace gets cut up at all
- VideoCaching: the slate next to the stovethe cache, before anyone has to place it on a ring
- VideoCDN: how Netflix puts video inside your ISPcaching at the edge, the paper's own domain
- VideoThe shard key that lied: when a correct decision goes wrong on its ownan even ring and a lopsided load, which are not the same thing
- LessonKV: the ringUses the board Splitting the Ledger
- LessonSplitting the ledger (sharding)Uses the board Splitting the Ledger
- LessonThe slate next to the stove (caching)Uses the board The Slate Next to the Stove
- LessonCounting the crowd without a register (HyperLogLog, count-min, top-k)Uses the board The Slate Next to the Stove
- LessonNetflix - the case studyUses the board Open Connect CDN
- LessonThe branch near your house (CDNs and object storage)Uses the board Open Connect CDN
- LessonSlack - the case studyUses the board The Shard Key That Lied
- LessonPartitioningUses the board Partitioning
- LessonBuild Your Own DynamoUses the board Partitioning
Further reading
Read the original for the full argument and its evidence.