Instagram - the case study

The whole machine on one canvas. The Python program at the top is drawn once, because there was only ever one of it.

Before you start. You need to know that a website is a program running on rented computers, and that it keeps its information in a database, a filing cabinet a program writes to and searches. This is the story of a photo app that reached three billion users without ever rewriting the program at the top.

The idea in 30 seconds

Instagram never rewrote the Python program at the top. Every fix for fifteen years went in underneath it.
A shard is one of two thousand numbered pieces of the database, picked by the remainder when the user’s number is divided by two thousand, and a photo’s id contains the number of the shard it lives in.
Fanout is copying a new post’s id into every follower’s list, done by a background job, and skipped entirely for accounts with tens of millions of followers.
Denormalising is storing an answer you could have recalculated, which is why a like count is one number to read instead of millions of rows to count.
Copy-on-write lets dozens of worker processes share one copy of the program’s memory, until something writes to a page, and Python’s memory tidier was writing to every page it looked at.
A quorum waits for a majority of copies to confirm a write, which is cheap inside a region and ruinous across an ocean, so every record was given a home continent.

Read the story · Skip to the numbers