End-to-End Arguments in System Design
A function that depends on the application's own definition of correct cannot be completely implemented by any layer that does not have it.
Read the original ↗Twelve pages from 1984 that answer one question - which layer does a function belong in - with an argument sharp enough that it settled the design of the internet and is still the fastest way to find the bug in a retry loop. It is also the most confidently misquoted paper in systems, usually as a slogan the authors did not write and would not have signed.
Included with a plan
This discussion is included with a plan. The first three lessons of every track are free, with their boards and videos. A plan unlocks every lesson, board, video, lab and paper for one price.
Related on chalk
The same idea as a board, lab, video or lesson.
- BoardThe Parcel's Journeythe layers the argument is about, and the question of which one a function belongs in
- BoardPostcards and Phone Callsreliability as a transport feature - the thing the paper says is a performance enhancement and not a guarantee
- BoardCharged Twicethe acknowledgement that never arrived: the paper's own example, in an application nobody was thinking about networks in
- BoardThe Whisper Codeencryption between the ends rather than inside the network, which is the paper's second-strongest case
- BoardOne Address A Thousand DoorsNAT, and the addressing assumption underneath every example in the paper quietly becoming false
- BoardTCP Handshakewhat the transport actually promises, read off the wire, so the gap to what an application needs is visible
- LabCharged twicethe reply that never came back - an end-to-end check is the only thing that resolves it, and the lab prices each answer
- LabThe TCP handshakethe round trips reliability costs, which is the performance half of the paper's own trade
- LabOne address, many doorsthe middlebox that broke the argument, run as arithmetic on a subnet
- VideoRouting: how a packet finds any house on earththe layers, in order, which is the thing the argument is choosing between
- VideoIdempotency keys: how a retry stops being a second chargethe application-level check the paper says has to exist, given a name and a key
- VideoCertificates: what TLS is proving, and who is vouchingtrusting the ends rather than the path - end-to-end encryption as deployed
- LessonHow a parcel finds any house on earthUses the board The Parcel's Journey
- LessonOne click, every layer (what happens when you load a page)Uses the board The Parcel's Journey
- LessonIt won't load: the ladder (network debugging)Uses the board The Parcel's Journey
- LessonInside one postal company (distance-vector, link-state, and ICMP)Uses the board The Parcel's Journey
- LessonBuild it: ping, and the parcel that dies on purpose (ICMP and traceroute)Uses the board The Parcel's Journey
- LessonTCP vs UDP, and the 3-way handshakeUses the board Postcards and Phone Calls
- LessonThe page that never arrived (TCP reliable delivery)Uses the board Postcards and Phone Calls
- LessonBuild it: the page that gets through (stop-and-wait, then a window)Uses the board Postcards and Phone Calls
- LessonBuild it: only the lost page (selective repeat, loss and reordering)Uses the board Postcards and Phone Calls
- LessonCharged Twice (idempotency & exactly-once)Uses the board Charged Twice
- LessonThe whisper code (how a call goes private)Uses the board The Whisper Code
- LessonOne address, a thousand doorsUses the board One Address A Thousand Doors
- LessonThe hold line (the listen queue, and the SYN flood)Uses the board TCP Handshake
Further reading
Read the original for the full argument and its evidence.
- The paper ↗ACM TOCS 2(4), November 1984, twelve pages, and readable in one sitting. The careful-file-transfer section is the whole argument; the rest is it being applied. Note the parenthesis on the first statement.
- Rethinking the Design of the Internet: The End-to-End Arguments vs. the Brave New World ↗Blumenthal and Clark, 2001. One of the original authors, seventeen years on, on trust, middleboxes and users who do not want to run their own security. The honest companion piece.
- RFC 9000: QUIC ↗What a transport looks like when it has to defend itself from the middle of the network by encryption. Read Section 1 and then think about why the header is authenticated.