Learn / System Design
Continue where you left off ×
The write that has to happen twice (CDC and the outbox)
The Ledger Everyone Reads · Open board The book on the counter that many people read. That lesson stopped at one sentence about getting rows out of a database into the book. This one is that sentence, slowed down until it is a design.
Before you start. You need The Ledger Everyone Reads (event-driven & the log) : a log is a book you only add to. Each reader keeps its own bookmark in it, called an offset. A reader can go back and replay what the book still holds. You need Charged Twice (idempotency & exactly-once) : networks retry, so the same work can arrive twice. And you need The copy clerks (database replication) : a database copies its changes to followers by sending them a stream of what it just did.
The idea in 30 seconds
A dual write is one action that must land in two systems, and a crash in the gap leaves them disagreeing with no error.
The outbox writes the event as a row in the same database transaction as the business change, so both exist or neither does.
Change data capture reads the database’s own write-ahead log and turns each committed change into an event, in commit order.
The usual design combines them: write the outbox row, let CDC carry it into a log.
A new reader gets snapshot then stream: copy the table, remember the log position, then follow the log from that position.
Delivery can repeat, so every consumer must be idempotent, noting event ids to skip repeats.
Read the story
Lesson 26 of 73 in System Design
The rest of this lesson is part of a plan.
A plan opens every lesson, board, video, lab and paper on chalk. Yearly is $39/year , or $19.50 for your first yearLAUNCH50 , ends 1 Nov, 18:30 UTC .
See plans Already a member? Sign in
Or start free : a free account opens lessons 2 and 3 of every track.
In this lesson 2012: LinkedIn has a database and a search index that disagree The register and the carbon paper: why two writes cannot be made safe The outbox: the slip written in the same stroke Change data capture: reading the register’s own diary The new reader problem: snapshot, then stream The carrier crashes: the idempotent consumer Terms it explains: dual write, outbox, change data capture, snapshot then stream, idempotent consumer.
Review System Design
Mark as read Saved in this browser, and to your account when you sign in.
