Large-scale cluster management at Google with Borg
A decade of running everything Google runs on shared machines, written up with the regrets left in.
Read the original ↗This is the paper behind Kubernetes, and the reason to read it is not the architecture - it is section 8, where the authors list what they got wrong. Very few systems papers have one. Almost none of them are this specific about it.
Included with a plan
This discussion is included with a plan. The first three lessons of every track are free, with their boards and videos. A plan unlocks every lesson, board, video, lab and paper for one price.
Related on chalk
The same idea as a board, lab, video or lesson.
- BoardImperative to Declarativethe shift Borg made twenty years ago, drawn as the choice it is
- BoardThe Reconcile Loopthe idea to steal, on one board
- BoardThe Schedulerfeasibility then scoring, which is Borg's algorithm
- BoardThe Harbor Masterthe control plane Borgmaster became the template for
- BoardServices & Ingressan address per pod - the direct fix to the paper's own first regret
- BoardThe Shipping Containerthe isolation primitive Borg used for packing, years before it meant an image format
- LabThe bouncer at the doorrejecting work at admission rather than queueing it, which is what Borg quota does
- LabThe order slip railthe pending queue batch work waits in while serving work goes first
- VideoKubernetes, part one: the objects, and the loop that reconciles themthe fleet, and why anything schedules at all
- VideoKubernetes, part two: scheduling, storage, and what breaks at three in the morningwhere a pod lands, what it may eat, and what survives it - Borg's chapter
- VideoContainers: namespaces, cgroups, and the layered filesystemcgroups and namespaces, the mechanism under a Borg task
- VideoAutoscaling groups: capacity that follows demanda control loop reacting to load, the idea at its smallest
- VideoInfrastructure as code: the state file is the whole storydeclare the outcome, let something reconcile - the steal, generalised off the cluster
- LessonKubernetes 01: Imperative to DeclarativeUses the board Imperative to Declarative
- LessonKubernetes 02: The Reconcile LoopUses the board The Reconcile Loop
- LessonKubernetes 05: The SchedulerUses the board The Scheduler
- LessonThe Harbor Master (pods, deployments, services)Uses the board The Harbor Master
- LessonServices & Ingress, from the insideUses the board Services & Ingress
- LessonThe Shipping Container (Docker)Uses the board The Shipping Container
- LessonContainer 03: A Mount Namespace and pivot_rootUses the board The Shipping Container
- LessonContainer 07: EverythingUses the board The Shipping Container
Further reading
Read the original for the full argument and its evidence.