Kubernetes 02: The Reconcile Loop

The loop that makes the last board’s declared fact keep being true. Follows Kubernetes 01: Imperative to Declarative, which said what a Kubernetes object is; this is the machine that enforces it, and it belongs to the build in Build Your Own Kubernetes - From Scratch.

Before you start. This series builds a small copy of Kubernetes: a program that keeps a fleet of machines running the programs you asked for, and restarts them when they die. So far our build can do one thing. It lets you write down what you want, say three copies of a program, as a plain file. Kubernetes calls one running copy a pod. Nothing yet checks that file against what is actually running. This lesson builds the part that closes that gap.

The idea in 30 seconds

A controller is a loop that reads the world, compares it to what’s wanted, and takes one step to close the gap - with no notion of ever being finished.
It never remembers what it did last time, on purpose: a crash mid-step costs nothing, because nothing was in flight to lose.
Level-triggered means reading the current value; edge-triggered means reacting to a change. Miss a level read and you’re late. Miss an edge and you’re wrong, forever.
A controller that remembers what it created can crash between creating and recording it, and duplicate the thing forever - the fix is to stop remembering and ask the world instead.
Idempotent means a reconcile against an already-correct world does nothing at all, which is what makes retrying one always safe.
Every controller can be individually correct and the whole system can still never settle, if two of them want different things from the same pods.

Read the story · Skip to the numbers