The Chai Stall That Got Famous

The chai stall that got famous — system design, lesson one new track. the networking arc gave you every part — now we build with them. (amazon shuffled its doors for us just now — proof on the right.) act 1 · one stall the stall (one server) perfect. cheap. fine — until the cricket match ends and 500 people want chai in the same minute. the stove is the ceiling, and there is exactly one of it. two ways out: · a bigger stove — works until stoves max out. and when the one stove dies, everything dies with it · more stalls — no ceiling at all. but now: who tells each customer which stall to walk to? act 2 · the token guy at the entrance token guy stall A stall B stall C his three strategies, by situation: · "next stall in turn" — the default, dumb and fair · "shortest line" — when some orders take far longer · "your usual wala" — when a stall remembers you. convenient… and the smell of trouble (see right →) the catch: the stall that remembers you if your running tab lives in one wala's head, you are chained to that wala. he goes home → your tab dies. his line grows → you wait anyway. remembering customers is what makes stalls unswappable. the fix: walas remember NOTHING. every tab lives in the shared register behind the stalls — any wala can serve anyone, and stalls become interchangeable. lesson 1's board said "stateless — scale horizontally" back when it was just a phrase. now you know why. (the shared register has its own problems. next lesson: the cache.) act 3 · you already know two more floors of this floor 1 — the phone book shuffles the doors: amazon gave us 3 addresses in a different order on every query (lesson 2's "two doors", industrial size) → floor 2 — the same address announced in every city: lesson 10's thousand doors floor 3 — the token guy at one building's entrance: today's lesson. big sites run all three floors at once. $ dig +short amazon.com (×3) 98.87.170.74 98.82.161.185 98.87.170.71 98.87.170.74 98.87.170.71 98.82.161.185 98.87.170.71 98.82.161.185 98.87.170.74 same three doors, shuffled every time — crowd split before anyone even leaves home where you feel this: IRCTC melting on tatkal morning = one stove. "you've been logged out" right after a site updates = a sticky wala went home mid-shift. game login queues = a token guy being honest about capacity. the jargon, now that you already own the story: the token guy = "load balancer" more stalls = "horizontal scaling" a bigger stove = "vertical scaling" "your usual wala" = "sticky sessions" forgetful walas = "stateless servers" the shuffled phone book = "DNS round-robin" THE TOKEN GUY'S PLAYBOOK -- LB algorithms, L4 vs L7, health checks 1) WHICH STALL? -- the algorithms round-robin: 1 -> 2 -> 3 -> 1 ... (even rotation) least-connections: -> the least-busy stall IP-hash: same customer -> same stall (sticky) round-robin is the default; least-conn suits uneven work; IP-hash gives stickiness without a shared session. 2) L4 vs L7 -- how smart is the token guy? L4 (transport): routes by IP : port. fast, cheap, blind to content. L7 (application): reads the HTTP host/path. routes /api vs /img, can terminate TLS. smarter, slightly costlier. 3) HEALTH CHECKS -- the safety net the token guy pings each stall's /health every few seconds. no reply -> pulled from rotation instantly, so no customer is sent to a dead stall. this stops orders reaching a dead stall — the lost tab still needs the shared register. ONE LEVEL DEEPER — THE LOAD BALANCER, DRAWN many clients → one LB → a pool of identical stateless servers client client client LOAD BALANCER srv 1 ✓ healthy srv 2 ✓ healthy srv 3 ✗ DOWN health check every ~2s: srv 3 fails → LB stops routing to it. self-healing. which server gets the next request? • round-robin 1,2,1,2,… — simplest, ignores load • least connections send to the server with fewest open conns • IP hash same client → same server (sticky) • weighted beefier box gets a bigger share L4 vs L7 — where the LB looks L4 (transport): routes by IP/port — fast, blind to content L7 (application): reads the HTTP path/headers → route /api → pool A, /img → pool B; TLS terminate; can cache stateless is the enabler: any server can serve any request, so 'add a box' = more capacity. session state → Redis, not the box. horizontal (add boxes) beats vertical (bigger box): no ceiling, and one death isn't an outage. this is the chai stall cloning its counter → shop runs exactly this in front of its pods ("Getting Traffic In").