YouTube - the case study

The whole machine on one canvas. Every stage in the story below is one box on it.

Before you start. You need to know that a website is a program running on rented computers, and that it keeps its facts in a database, a filing cabinet a program can write to and search. One more thing, and the whole lesson hangs off it: a video is enormous next to a row of text. It is a file of many millions of bytes, a byte being one small unit of storage, so YouTube spends its life moving bytes and only occasionally facts about bytes.

The idea in 30 seconds

A video is written once and watched up to a billion times, so every rupee of effort belongs at write time if it saves anything at read time.
A shard is a slice of the database holding some of the users, and slicing beats copying once the data no longer fits in memory, because copies multiply a dataset and shards divide it.
Vitess is a layer in front of the database that pools connections, blocks dangerous queries and hides the shards, which kept the database YouTube already knew how to run.
Transcoding turns one upload into a ladder of versions, and a better codec is a loan the view count repays, which is why the expensive one is reserved for hits.
Google Global Cache is a rack of YouTube’s machines inside your internet provider’s building, and it answers 70 to 90% of requests without the traffic ever leaving.
Watch time replaced clicks as the thing recommendations maximise, and the change lives inside the model’s arithmetic rather than in a policy.
A fingerprint is a compact summary of how content looks and sounds that survives cropping and re-encoding, which turns a copyright judgment into a search.

Read the story · Skip to the numbers