What a Data System Promises

WHAT A DATA SYSTEM PROMISES the three questions every data system must answer apps today are DATA-intensive, not CPU-intensive: you stitch together DB + cache + search index + queue + stream. you're a data-system designer. 1 · RELIABILITY "works correctly even when things go wrong." a FAULT = one part goes wrong. a FAILURE = the whole system stops. goal: fault-TOLERANT — keep a fault from ever becoming a failure. HARDWARE faults disks die (MTTF ~ years), power drops. fix: redundancy (RAID, spare nodes). SOFTWARE faults a systematic bug hits ALL nodes at once and cascades. harder to catch. HUMAN error — the #1 cause a wrong config. fix: easy undo, staging, fast rollback, good tooling. Chaos Monkey: kill things on PURPOSE to prove you tolerate it. 2 · SCALABILITY — the Twitter fan-out first, name your LOAD PARAMETER: the number that defines the load. for Twitter timelines it's FAN-OUT — how many followers a post must reach. A · fan-out on READ poster one big tweets table reader gather + merge everyone you follow, at read time reads do ALL the work → timeline reads >> tweet writes → dies. B · fan-out on WRITE poster a mailbox per follower — the tweet is PUSHED into each one read = cheap (one mailbox). but a celeb's post = millions of writes (write amplification → the HOT KEY from board 06). HYBRID (what Twitter actually did): fan-out on write for normal users; fetch celebrities at read time and merge the two. the load parameter (follower distribution) PICKS the design — that's the whole method: describe load with a parameter, then ask what happens to performance as it grows. 3 · PERFORMANCE IS A DISTRIBUTION — not a single number response time #reqs p50 (median) p95 p99 p99.9 p50 = the TYPICAL request. p99 = 1 request in 100 — often your heaviest, most valuable users (most data). the MEAN lies: a few huge outliers drag it around and it hides the tail entirely. always quote percentiles, not the average. SLO / SLA are written in percentiles (e.g. p99 < 200ms). tail-latency amplification: in a fan-out request, ONE slow backend makes the WHOLE call slow. 4 · COPING, & MAINTAINABILITY coping with load: · scale UP (vertical) — a bigger box, simple, limited · scale OUT (horizontal) — many boxes, shared-nothing · elastic — add/remove capacity automatically maintainability — for the humans who run & change it: · OPERABILITY — make ops easy: visibility, automation · SIMPLICITY — kill accidental complexity w/ abstraction · EVOLVABILITY — make change easy (the rest of the book) the biggest cost of software is maintenance, not the first build. JARGON, DECODED data-intensive = bottleneck is data (amount/complexity/change), not CPU · fault = one part wrong · failure = whole system stops fault-tolerant = a fault never becomes a failure · load parameter = the number that describes load (e.g. fan-out) · fan-out = reaching many targets from one op throughput = work/sec · response time = what the client sees (service time + queueing + network) · percentile pN = N% of requests are faster tail latency = the slow high-percentile requests · vertical vs horizontal = bigger box vs more boxes · shared-nothing = nodes share no memory/disk operability / simplicity / evolvability = the three faces of maintainability · accidental complexity = complexity from the tools, not the problem