The Slate Next to the Stove

The slate next to the stove — caching the chai stall saga, part two. (we caught wikipedia serving us from a slate just now — an 18.7-hour-old copy, used 544 times.) the problem we created last lesson: walas remember nothing (that's what made them swappable) — so now EVERY question walks to the register in the back room. "how much is one chai?" asked 500 times = 500 walks. the register became the queue. the fix: a slate, chalked next to the stove wala at the counter SLATE chai ......... ₹10 samosa ....... ₹15 garv's tab ... ₹120 the register (back room — slow, but always right) 1 · look here first 2 · only on a blank — walk to the back, chalk the answer on return found on the slate = instant. blank = the slow walk, then chalk it for the next person. the whole game is one number: what fraction of questions never leave the counter. the two curses of the slate curse 1 · staleness milk price rose in the register; the slate still says ₹10. customers get wrong answers CONFIDENTLY. cures: an expiry on every line (lesson 1's sticky note had one — 32 seconds!), or wiping the line the moment the register changes — easy to say, legendarily hard to get right everywhere. curse 2 · the slate is small chalk space runs out. when full: erase the line nobody has asked about lately, keep the crowd-pleasers. hot things live on slates; rare things always take the walk. (the old joke lands now: "there are only two hard things in computer science — cache invalidation and naming things.") caught in the act, just now $ curl wikipedia's logo (×2) x-cache-status: hit-front x-cache: cp5021 ... hit/544 age: 67343 translation: a slate named cp5021 answered. its chalked copy is 18.7 hours old and has been served 544 times. the register never heard from us. and it's slates all the way down: your browser keeps one (the back button is instant), the CDN keeps one (cp5021), the app servers keep one, and even the register keeps one internally. each layer answers what it can, passes on the rest — the concierge's sticky note, industrialized. where you feel this: the back button loading instantly. an old like-count on instagram = a stale slate somewhere. Ctrl+Shift+R = "ignore every slate, walk to the register". "clear your cache" advice = wiping a cursed slate. the jargon, now that you already own the story: the slate = "cache" found = "hit" · blank = "miss" the fraction found = "hit rate" line expiry = "TTL" wiping on change = "invalidation" erasing the least-asked = "LRU eviction" catch a slate yourself: curl -sI https://en.wikipedia.org/static/images/project-logos/enwiki.png | grep -i x-cache THE CACHE PATTERNS -- how writes & evictions actually work 1) READ/WRITE PATTERNS cache-aside (lazy): miss -> load DB -> fill cache the default. app owns the logic. write-through: write cache + DB together consistent, but slower writes. write-back: write cache now, DB later fast, but data loss risk on crash. 2) EVICTION -- the slate is small cache full -> something must go: LRU: drop least-Recently-used LFU: drop least-Frequently-used TTL: entries expire on a timer LRU is the common default -- keeps the hot set. 3) THE MATH & THE STAMPEDE 90% hit @1ms + 10% miss @50ms = 5.9ms avg every 1% of hit-rate matters at scale. STAMPEDE (thundering herd): a hot key expires -> 1000 requests all miss -> all slam the DB at once. fix: one loader (lock) + staggered TTLs. ONE LEVEL DEEPER — HIT, MISS, AND THE 3 WRITE STRATEGIES, DRAWN cache-aside: check the slate first app CACHE Redis DATABASE ~40ms 1 GET key HIT → 0.5ms, done 2 MISS → ask DB 3 fill cache (SET), return hit rate is everything: 95% hits → avg = .95×0.5ms + .05×40ms ≈ 2.5ms (16× faster). the whole game. 3 ways to keep the slate honest on WRITES write-through write cache + DB together — safe, slower writes write-back write cache now, DB later — fast, risks loss on crash write-around write DB only, let cache fill on next read — avoids cache churn and the two hard problems: • invalidation — stale data; TTL or explicit bust on write • thundering herd — key expires, 1000 misses stampede the DB → fix: lock/singleflight, stagger TTLs, or refresh-ahead when the slate is full: EVICTION picks who leaves • LRU — evict least-recently-used (default) • LFU — least-frequently-used • TTL — time-based expiry real: this board caught Wikipedia's CDN cache live (x-cache: HIT). shop fronts Postgres with Redis exactly this way — the slate next to the stove.