Object Storage and the CDN

The board’s wiring is right: storage (one source of truth) is a different job from distribution (cached copies near people), and an Origin Access Control (a setting that lets only the CDN, Amazon’s is called CloudFront, read the storage bucket) is what joins those two boxes safely. This lesson gives that wiring its missing half: what an object actually is, what a cache actually buys you, and every number both of them run on.

Note on names. Every hostname, distribution id and bucket name below is a placeholder (app.example.com, d111111abcdef8.cloudfront.net, assets-prod). Nothing here refers to a real account, bucket or client.

Before you start. You need to know what a web address is and that a server is just a computer that answers requests - you have used both today. A bucket is one named storage area in S3, Amazon’s file storage. Who may read what inside a bucket is The Locked Filing Cabinet - folder-level S3 access‘s subject; this lesson is about what the bucket’s contents actually are, and what a cache remembers about you.

The idea in 30 seconds

An object store like S3 has no real folders. Every file’s name, its key, is one entry in one flat list, and a folder view is just that list grouped for you on screen.
That list is called the index. Lose it, as happened on 28 February 2017, and every file is still on disk, completely untouched, and completely unfindable, for hours.
A CDN is a cache. It keeps copies near you and only goes back to the one real copy, the origin, when it does not already hold a match for your cache key.
The share of requests a cache answers without reaching the origin is the hit rate - that is what kept some sites alive when the 1999 Star Wars trailer took the rest of the internet down.
Neither system bills you for bytes moved. Both bill you for how many distinct keys you made.

Read the story · Skip to the numbers