S5 · Build a Message Log (Kafka-style)

S1’s file that only ever grows, with records added at the end and never edited. This project keeps that file and changes who reads it: many readers, each remembering how far down the file it has got.

Before you start. You’ve built S1, so an append-only log, a length in front of each record, and replay on startup are already yours. This project points that same log at a different job: carrying messages from the programs that write them to any number of programs that read them. The slow build of the same thing, one lesson per wall, is the From Scratch Kafka series starting at Build Your Own Kafka - From Scratch; this page turns it into one repo and does not repeat its detail. Enough Go is an optional first read.