Real-time and location

How the server reaches the client, and how many connections one box holds

HTTP was built for the client to ask. A real-time system needs the server to speak first, and there are four ways to do it. Pick one out loud, and say what it costs per client. That cost, multiplied by the number of clients online at once, sizes the gateway tier.

Why polling loses at scale: 10M clients polling every 5 s is 10,000,000 / 5 = 2,000,000 requests a second, and nearly all of them come back empty. With an open connection the server only sends when something happened. The price is that it now holds state for every client online.

An idle connection costs a file descriptor and some kernel and app buffers. Assume about 10 KB each, so 100,000 connections is 100,000 x 10 KB = 1 GB of memory. That is why one box can hold hundreds of thousands of them, and why a few hundred boxes cover tens of millions of users. Ports are not the limit: a server tells connections apart by the (client IP, client port, server IP, server port) tuple, so every client can share one listening port. The 65,535 ceiling bites only on the outbound side, for example a load balancer opening connections to one backend.

Short polling

The client asks every N seconds. It is the simplest to build and works through any proxy. The cost is N/2 seconds of average delay plus one request per client per interval, and most of those requests return nothing.

Long polling

The client asks and the server holds the request open until it has something to send or a timeout fires, then the client asks again. It is close to real time over plain HTTP, but each delivery pays for a new request and its headers.

Server-Sent Events (SSE)

One long-lived HTTP response that the server keeps writing text events into, with reconnect built into the browser's EventSource. It only goes one way, which is enough for feeds, notifications and live scores, where the client rarely talks back.

WebSocket

The HTTP connection is upgraded to a full-duplex socket, so either side can send at any time with a few bytes of framing. This is the default for chat, presence, dispatch and collaborative editing. The cost is that you now run stateful gateways: sticky routing, heartbeats, and a plan for when a box dies with 100,000 sockets on it.

Questions

Answer each one yourself before you open it.

1. 01Design a chat app like WhatsApp or Slack, with online presence: the green dot next to each friend.

   The hinge is that presence, not messages, is the big traffic, so the design is a tier of stateful WebSocket gateways that hold every online user's connection, with a presence service that only tells online friends who are actually looking. Messages go client to gateway to a chat service that writes them to a store partitioned by conversation, then fan out to the recipients' gateways through a lookup of which gateway holds which user.

   Interview assumptions: 50M daily users, 10M online at peak, 40 messages each a day, a heartbeat every 30 s. Messages: 50M x 40 / 86,400 s ≈ 20,000 writes/s. Heartbeats: 10M / 30 s ≈ 300,000/s, so presence is about 15 times the message traffic before a single status change fans out to anyone. Connections: at 100,000 per gateway, 10M / 100,000 = 100 gateways, each using about 1 GB for idle sockets. The components are the gateways, a connection registry (user id to gateway, in memory), a chat service writing to a store partitioned by conversation id, and a presence service that keeps last-seen timestamps in memory and expires them. Replace the 30 s heartbeat interval first with a real number, because it directly multiplies the biggest line.

   Follow-up One gateway box dies with 100,000 users on it. What happens?

   All 100,000 clients reconnect at once, and without care they hit the rest of the fleet as a thundering herd and knock over a second box. Clients reconnect with exponential backoff plus random jitter. Spread over 30 s, that is 100,000 / 30 ≈ 3,000 new connections a second across 99 boxes, which is easy. Messages sent while they were away are safe because the chat store, not the gateway, is the source of truth: each client resyncs from its last-seen message id.

   Learn more WhatsApp - the case study · Slack - the case study · One address, a thousand doors (ports and connections)

2. 02Design the dispatch system for a ride-hailing app: a rider requests a ride and we match them to a nearby driver.

   The hinge is that driver location pings outnumber ride requests about a thousand to one, so live locations never go to the database. They sit in memory, indexed by map cell (a geohash or hexagonal grid), sharded by city. A ride request looks up the rider's cell and its neighbours, ranks the free drivers by estimated arrival time, and offers the trip to one driver at a time.

   Interview assumptions: 1M drivers online, one location ping every 4 s, 20M rides a day. Pings: 1,000,000 / 4 s = 250,000 writes/s. Ride requests: 20M / 86,400 s ≈ 200/s. So pings are about 1,000 times the request rate, and they are also disposable, because a 4-second-old location is worth nothing. Memory: 1M drivers x about 100 bytes = 100 MB, which fits in one box. You shard by city for the write rate and for failure isolation, not for space. The components are driver WebSocket connections into a location service (in-memory cell index), a matching service, a trip service with a durable store for trips and payments, and a push channel back to both phones. Replace the ping interval first: it multiplies the biggest number.

   Follow-up Two riders request at the same moment and both get matched to the same driver. How do you stop it?

   Each driver's state (free, offered, on trip) has exactly one owner, the location or matching shard for that city. An offer is a single atomic compare-and-set from free to offered, with a TTL of about 10 s. The second match loses the compare-and-set and moves on to its next candidate, and if the driver never answers, the TTL puts them back to free. Do not use a distributed lock across services, because one owner per driver is simpler and has nothing to deadlock.

   Learn more Uber - the case study · Splitting the ledger (sharding) · Who holds the pen (consensus & Raft)

3. 03Design "find nearby places": given my location, show the 20 closest restaurants within 5 km.

   The hinge is that this is read-heavy over data that barely changes, and the geo index is small enough to fit in memory. So there is no sharding: build one spatial index (a geohash prefix or a quadtree), copy it to a handful of read replicas, and answer each query by scanning the query's cell plus its eight neighbours, then sorting by true distance. Full place details sit in an ordinary database and cache, fetched by id for the 20 winners.

   Interview assumptions: 200M places, 100M daily users, 5 searches each a day, 1M place edits a day, one replica serving about 2,000 queries/s. Index: 200M x about 30 bytes (id, latitude, longitude) = 6 GB, which fits in RAM, while the full records are 200M x 1 KB = 200 GB and stay on disk. Reads: 100M x 5 / 86,400 s ≈ 6,000 queries/s, about 12,000 at a 2x peak, so 12,000 / 2,000 = 6 replicas. Writes: 1M / 86,400 s ≈ 12/s, so rebuilding or patching the index every few minutes is fine. Replace the queries-per-replica figure first, by benchmarking the index. The neighbours rule exists because a place 50 m away can sit across a cell boundary with a completely different geohash prefix.

   Follow-up Manhattan has 50,000 places in one cell and the desert has none. What changes?

   A fixed grid gives uneven cells, so switch to an adaptive one: a quadtree that splits any cell holding more than about 100 places, which makes dense cities deep and deserts shallow. Or keep the geohash and choose the precision per query, starting fine and widening ring by ring until 20 results are in hand. Either way, the cost of a query follows the number of results, not the size of the city.

   Learn more Google Maps - the case study · The copy clerks (database replication) · The slate, properly (caching to real depth)

4. 04Design a collaborative document editor like Google Docs, where several people type into the same document at once.

   The hinge is concurrent edits to the same text, so every document gets exactly one ordering authority: all editors of a document are routed to the one server process that owns it, over WebSockets, and that process orders and transforms operations (operational transformation) before broadcasting them. The document is stored as a snapshot plus an append-only log of operations, and a CRDT is the alternative when there is no central server.

   Interview assumptions: 5 editors typing 5 characters a second at the busiest, 2-hour sessions, about 100 bytes per operation. Per document: 5 x 5 = 25 operations/s, which is tiny, so one owner per document is not a bottleneck and you shard by document id. The log is what grows: one typist over 2 hours is 5 x 7,200 s = 36,000 operations, x 100 bytes ≈ 4 MB, so you snapshot every thousand operations or so and never replay from zero. The components are a gateway, a routing table (document id to owner process), the owner processes, an operation log, a snapshot store, and cursor presence sent over the same socket. Replace the per-operation size first, since batching keystrokes can shrink it several times over.

   Follow-up A user edits offline on a plane for an hour, then reconnects. How do their edits merge?

   With OT, the client sends its operations tagged with the last server version it saw, and the owner transforms them against everything committed since, one by one, then broadcasts the result. That works, but it can be slow and surprising after an hour of divergence. This is where CRDTs earn their memory overhead: each character carries a unique id and a position between its neighbours, so any two replicas merge the same way in any order with no server rebase. Say which one you would pick and why: OT when a server is always in the loop, a CRDT when offline or peer-to-peer editing is a real requirement.

   Learn more Figma - the case study · The ledger everyone reads (event-driven & the log) · Who holds the pen (consensus & Raft)

5. 05Design a video streaming service like YouTube or Netflix.

   The hinge is egress bandwidth, which dwarfs everything else, so viewing never touches your origin servers: videos are transcoded once into a ladder of bitrates, cut into short segments (HLS or DASH), and served from a CDN, while the player picks a bitrate segment by segment from its measured bandwidth. The upload side is a separate pipeline: upload to object storage, then a queue of parallel per-segment transcode jobs, then publish.

   Interview assumptions: 10M concurrent viewers at 5 Mbps, servers with 100 Gbps network cards, 100,000 hours uploaded a day, a bitrate ladder that sums to about 10 Mbps across its renditions. Egress: 10M x 5 Mbps = 50 Tbps, and 50 Tbps / 100 Gbps = 500 edge servers running flat out, which is why this is a CDN problem before it is anything else. Storage: 10 Mbps x 3,600 s / 8 = 4.5 GB per hour of video, so 100,000 hours x 4.5 GB = 450 TB a day of new storage. The components are an upload service, object storage, a transcode queue and workers, a metadata database, and the CDN serving segments and manifests. Replace the concurrent-viewer count first, since it alone sets the egress bill.

   Follow-up Now it is a live event: 10M people watching one stream at the same time. What breaks?

   Every viewer asks for the same new segment within seconds of it existing. With 4 s segments that is 10M / 4 s = 2.5M requests/s, and a cache miss at every edge would stampede the origin. Put an origin shield between the edges and the origin, and have each edge coalesce requests so that only one fetch per segment goes upstream. The origin then sees roughly 200 PoPs x 5 renditions / 4 s = 250 requests/s. Latency is the other cost: players buffer a few segments, so 4 s segments mean 10 to 30 s behind live, and low-latency HLS cuts that with partial segments.

   Learn more YouTube - the case study · Netflix - the case study · Object storage and the CDN