The customer who won’t hang up (polling, SSE and WebSockets)

Four famous designs worked in one sitting. This lesson opens up the first of them, chat, whose hardest part is not storing messages but getting each one to a person the instant it exists.

Before you start. From The Contract at the Counter (API design): two programs meet at a counter, and every exchange so far was one question, one answer. From TCP vs UDP, and the 3-way handshake and HTTP Keep-Alive: opening a connection (the open line between two computers) costs a round of “hello” messages, and keeping one open saves paying it again. From HTTP 07: Chunked Encoding: a server can send its answer in pieces as they happen. From The order slip rail (queues & the utilization cliff): work that arrives in a bunch hurts far more than the same work spread out.

The idea in 30 seconds

Polling means asking on a timer; it wastes most requests and still delivers news late, so no polling speed is right.
Long polling holds each request open until there is news, which delivers instantly but makes the server hold many waiting requests.
A WebSocket is one connection kept open that carries messages both ways, so the connection cost is paid once.
Chat systems hold open connections on dedicated gateway machines and keep a list of which gateway holds each user.
A heartbeat is a small regular message that proves a connection is still alive, and presence (who is online) is often costlier than the messages because every change must reach many people.
When many clients lose their connection at once, they must wait random times before reconnecting, or they return in synchronised waves that knock the servers over again.

Read the story · Skip to the numbers