S2 · Build a Load Balancer

One front door, a pool of identical servers behind it, and the two questions the door answers for every visitor: which one gets this request, and is it alive.

Before you start. You’ve met ports and connections in the networking track, and you can run a terminal command and write a little code, but you’ve never built a server. That’s fine - this is the first one. By the end you’ll have written about 130 lines of Go, watched it alternate between two servers, killed one on purpose, and watched traffic route around it with only one request ever failing.

Board: S2 - Build a Load Balancer Code: teaching/code/s2-loadbalancer/ (lb.go, main.go, passing lb_test.go)

1994: one address stops being one computer

In 1993 a team at the National Center for Supercomputing Applications (NCSA), at the University of Illinois, shipped Mosaic, the browser that made the web popular. A year later their own web server was buckling under the traffic Mosaic had created - one address, one machine, answering every visitor itself. Three of the team, Eric Katz, Michelle Butler and Rob McGrath, wrote up the 1994 fix: run several identical copies of the server, and let DNS, the internet’s address book, hand a different machine’s address to each new visitor in turn. Ask for the same name ten times and you’d get ten rotating answers - one name, many computers, taking turns.

It worked, for spreading the load. It broke the moment one machine crashed. DNS had already told browsers which addresses to use, and those answers are cached - kept and reused - for hours. A dead machine kept getting a share of every visitor, because nothing was watching it.

1995: “about ten customers,” and the box that watched

Late in 1995, an engineer named John Mayes sat in a meeting about his company being bought by Cisco. Across the table was Robert Andrews, who ran Netscape’s website. He guessed there were “probably 10 customers in the world with a load balancing problem.” The guess was far too low, but it started the work. Four months later, April 1996, Cisco shipped the LocalDirector: a box that sat directly in front of a company’s servers, forwarding every request itself, and - the one thing DNS could never do - checking whether each server behind it was still answering, cutting one off the moment it wasn’t. It is the direct ancestor of the load balancer in front of nearly every busy website today.

The two jobs behind the door: which one, and is it alive

A load balancer sits at one address and, for every request, does two things - picks one of several identical copies of your server, and never picks one that’s dead.

Each identical copy is called a backend. The load balancer itself has a name too: it receives a request meant for a backend, passes it on, and carries the answer back without the visitor ever learning the backend exists - a reverse proxy. (A forward proxy is the opposite, standing in front of the visitor, hiding them from the servers.)

Round and round: who gets the next visitor

The simplest way to pick a backend is to take turns. Picture two backends, A and B: request 1 to A, 2 to B, 3 back to A, 4 to B, an even split forever as long as both answer. This is round-robin - the same idea NCSA’s DNS rotation used, just done inside the load balancer, on every request, instead of once per name lookup.

One wrinkle: requests can arrive in the same instant, all needing the count to go up by exactly one, none stepping on another’s turn. The fix is a counter the machine can increase in a single, indivisible step - an atomic counter - so two requests asking “what’s next” at once still come away with two different answers. Nothing to agree on, nothing to lock.

The corpse DNS couldn’t see

Round-robin alone still has NCSA’s old problem: it will happily send a visitor to a backend that crashed a second ago. The fix is the other half of the job - a health check, any test that answers “is this backend alive right now?” The load balancer runs two: one asks on a timer, every couple of seconds, is anyone home; the other is simpler - when a real request fails outright, that failure is itself the answer, and traffic to that backend stops immediately, no timer to wait out.

That pairing is what a DNS answer, cached for hours, never could do: notice a backend go quiet and route around it. It’s what you’re about to write.

You now know

• A load balancer sits at one address and hands each request to one of several identical copies of your server, called backends.
• A load balancer is a reverse proxy: it forwards each request to a backend and carries the answer back, and the visitor never learns the backend exists.
• Round-robin sends requests to backends in turn, using an atomic counter so requests arriving at once never collide over whose turn is next.
• A health check tests whether a backend is alive; running one on a timer plus one that reacts to a real failed request lets the load balancer stop sending traffic to a dead backend within one request.
• Cisco’s LocalDirector, shipped in 1996, was the first dedicated box to pick a backend for each request and skip dead ones; software doing both now fronts nearly every busy website.

Stop here if you want. What follows is the same door in Go - the code, the two diagrams, and the arithmetic that says why a health check needs both a timer and a real failure to watch.

Second read: the code, the wire, and the arithmetic

Go’s standard library hands you the proxy for free - httputil.NewSingleHostReverseProxy(url) copies a request to a backend and streams the reply back.

Get the code

The cd teaching/code/s2-loadbalancer lines below point at a folder you do not have; every file in it is printed in full under Run it. On a machine with Go 1.22 or newer:

mkdir -p ~/portfolio/s2-loadbalancer && cd ~/portfolio/s2-loadbalancer
go mod init loadbalancer

Save the six files printed under Run it into that folder (lb.go, main.go, lb_test.go, stage1_test.go, stage2_test.go, stage3_test.go), then:

$ go test ./...
ok  	loadbalancer	0.409s

If go mod init is new to you, S1’s “Get the code” says what it does and what breaks without it. Measured 2026-10-17 in an empty folder, Go 1.27.1 on macOS. The same tests pass on Go 1.22.12, with one wrinkle on a recent macOS: Go 1.22 links test binaries the system refuses to load (dyld: missing LC_UUID load command), and go test -ldflags=-linkmode=external ./... gets round it. Go 1.27.1 needs no flag.

The three moving parts, in Go

1. Round-robin. (counter++) % len(pool), skipping DOWN backends, via Go’s sync/atomic.
2. Active check. A goroutine (a cheap thread Go runs thousands of) wakes on a time.Ticker and pings GET /health every 2 seconds, flipping the UP/DOWN flag.
3. Passive check. The proxy’s ErrorHandler fires the instant a real request errors - connection refused, timeout - marking DOWN instantly.

The counter plus an RWMutex-guarded flag are the whole concurrency story: many requests read the pool at once, the health loop writes to it rarely.

The three stages, each a commit

The same ladder as S1: one test file per stage, added one at a time, each a green checkpoint to commit before the next.

Stage 1 needs NewBackend, Pool.Add and Pool.ServeHTTP with a single backend; stage 2 adds the round-robin in NextPeer; stage 3 adds the ErrorHandler and HealthCheck. stage1_test.go holds the helpers the other two use, so it stays, and lb_test.go, the original test, goes in last. Against the reference code, 2026-10-17:

$ go test -v -run Stage ./...
--- PASS: TestStage1ProxyOneBackend (0.00s)
--- PASS: TestStage2RoundRobin (0.00s)
--- PASS: TestStage3PassiveCheck (0.00s)
--- PASS: TestStage3ActiveCheckRecovery (0.00s)
ok  	loadbalancer	0.432s

The last test is the one a passive check can never pass alone. A backend that comes back sends no error to notice, so only the probe, asking /health again, can flip it from DOWN to UP.

The architecture, drawn

Three parts, two arrows of information:

flowchart LR
  C[client] -->|"GET /"| LB[load balancer :8080]
  LB -->|"pick next ALIVE"| RR{round robin cursor}
  RR --> A[backend A :9001]
  RR --> B[backend B :9002]
  RR --> N[backend N]
  H[health loop every 2s] -.->|"GET /health"| A
  H -.->|"GET /health"| B
  H -.->|"GET /health"| N
  H -.->|"flip alive flag"| RR

The dotted arrows: the health loop never touches a client request. It writes one boolean per backend on its own timer, and the cursor reads those booleans on every request - probe writes, requests read, which is why a read-write mutex is enough.

One request, with a backend dying half-way

The sequence the test asserts. Read it top to bottom: A answers request 1, B answers request 2, then B is killed, and request 4 is the one that finds it dead:

sequenceDiagram
  participant C as client
  participant LB as load balancer
  participant A as backend A
  participant B as backend B
  C->>LB: request 1
  LB->>A: forward (cursor 0)
  A-->>LB: 200 hello from A
  LB-->>C: 200
  C->>LB: request 2
  LB->>B: forward (cursor 1)
  B-->>LB: 200 hello from B
  LB-->>C: 200
  Note over B: process killed
  C->>LB: request 3
  LB->>A: forward (cursor 0)
  A-->>LB: 200
  LB-->>C: 200
  C->>LB: request 4
  LB-xB: forward (cursor 1), connection refused
  Note over LB: ErrorHandler fires, B marked DOWN
  LB-->>C: 502 backend unavailable
  C->>LB: request 5
  LB->>A: forward (B skipped)
  A-->>LB: 200
  LB-->>C: 200
  Note over LB,B: health loop probes B, still DOWN, stays skipped

Request 4 is the only one a visitor ever sees fail, because the passive check can only learn a backend is dead by trying it. Everything after routes around B, first by the passive flag, then confirmed by the next probe.

The whole build in code: 65 lines of node, then the Go

The same three parts fit on one screen in node, with two toy backends started inside the file. It kills backend 9002 after request 4 and keeps sending:

// lb.mjs  -  node lb.mjs
import http from "node:http";

const backends = [
  { port: 9001, alive: true, server: null },
  { port: 9002, alive: true, server: null },
];

// Two toy backends that answer with their own port
for (const b of backends) {
  b.server = http.createServer((req, res) => {
    if (req.url === "/health") return res.end("ok");
    res.end(`hello from backend ${b.port}\n`);
  }).listen(b.port);
}

// 1. round-robin: a counter, modulo the pool, skipping DOWN backends
let cursor = 0;
function nextPeer() {
  for (let i = 0; i < backends.length; i++) {
    const b = backends[(cursor + i) % backends.length];
    if (b.alive) { cursor = (cursor + i + 1) % backends.length; return b; }
  }
  return null;
}

// 2. active health check: probe /health on every backend on a timer
async function healthCheck() {
  for (const b of backends) {
    try {
      const r = await fetch(`http://localhost:${b.port}/health`, { signal: AbortSignal.timeout(300) });
      b.alive = r.ok;
    } catch { b.alive = false; }
  }
  console.log("  [probe] " + backends.map(b => `${b.port}=${b.alive ? "UP" : "DOWN"}`).join(" "));
}

// 3. the reverse proxy: forward the request, stream the reply back
const lb = http.createServer((req, res) => {
  const b = nextPeer();
  if (!b) { res.statusCode = 503; return res.end("no healthy backends\n"); }
  const up = http.request({ port: b.port, path: req.url, method: req.method, headers: req.headers }, (r) => {
    res.writeHead(r.statusCode, r.headers);
    r.pipe(res);
  });
  up.on("error", () => {              // passive check: a real request failed
    b.alive = false;
    res.statusCode = 502; res.end("backend unavailable\n");
  });
  req.pipe(up);
});
lb.listen(8080);

const ask = async (i) => console.log(`req ${i}: ` + (await (await fetch("http://localhost:8080/")).text()).trim());

await healthCheck();
const probe = setInterval(healthCheck, 1000);
for (let i = 1; i <= 4; i++) await ask(i);
console.log("-- killing backend 9002 --");
backends[1].server.close();               // stop accepting new connections
await ask(5); await ask(6);                // the second lands on 9002: passive check catches it
await new Promise(r => setTimeout(r, 1100)); // let the probe run once
for (let i = 7; i <= 9; i++) await ask(i);
clearInterval(probe); lb.close(); backends[0].server.close();

The output, exactly as it printed (node v22):

Three lines matter: req 6: backend unavailable, the 9002=DOWN probe a second later, and req 7, already back on 9001.

  [probe] 9001=UP 9002=UP
req 1: hello from backend 9001
req 2: hello from backend 9002
req 3: hello from backend 9001
req 4: hello from backend 9002
-- killing backend 9002 --
req 5: hello from backend 9001
req 6: backend unavailable
  [probe] 9001=UP 9002=DOWN
req 7: hello from backend 9001
req 8: hello from backend 9001
req 9: hello from backend 9001

Same shape as the sequence diagram, with the kill two requests later: 1-4 alternate (the cursor); 6 is refused, error flips alive false, 502 answered; the probe agrees a second later, and 7-9 all go to 9001. Nothing in the file knows why 9002 died, only that talking to it failed.

The Go version is this file with types, its test asserting both halves:

$ go test -v ./...
=== RUN   TestRoundRobinAndFailover
--- PASS: TestRoundRobinAndFailover (0.01s)
PASS
ok  	loadbalancer	0.541s

The arithmetic of a health check

Why both checks? A timer alone isn’t enough. The probe runs every 2 seconds, so a dying backend is found on average 1 second later, worst case 2 - and with two backends, the cursor hands it half the traffic meanwhile:

requests failed before the probe notices  =  rate × share × detection delay

  100 req/s × 1/2 × 1 s (average)  =   50 failed requests
  100 req/s × 1/2 × 2 s (worst)    =  100 failed requests
1,000 req/s × 1/2 × 2 s (worst)    = 1,000 failed requests

With the passive check that becomes 1: the failing request flips the flag, and the next one skips the backend. The active check is still needed the other way - a recovered backend sends no error to notice.

One more number: round-robin’s cost when requests aren’t equal. Two backends, a request every 20 ms, six in seven cost 10 ms, one in seven 200 ms, against sending each request to whichever backend has fewest in flight:

$ node rr-vs-lc.mjs
round-robin       { avg: '72.6', p50: 70, p99: 160, max: 160 }   (ms waited in queue)
least-connections { avg: '53.9', p50: 10, p99: 180, max: 180 }

p50 is the middle request’s wait; p99 is the wait only 1 in 100 exceed. Where 70 comes from: each backend gets every other request, one every 40 ms, so after a 200 ms request its next arrivals wait 160, 130, 100, 70, 40, 10 ms. Least-connections gets the median to 10 ms by walking a fast request past the busy backend. The p99 does not improve (160 to 180 ms): the slow requests still happen somewhere. It fixes the crowd, not the outliers.

The script behind that table. The table was printed by a script this page never showed. Here it is, rebuilt: simulated time, no sockets, so it prints the same thing every run.

// rr-vs-lc.mjs  -  node rr-vs-lc.mjs
// Two backends, one request every 20 ms. Requests 0, 7, 14, ... cost 200 ms,
// the other six in seven cost 10 ms. Each backend serves one request at a
// time, in arrival order. Simulated time: no sockets, same output every run.
const N = 500, GAP = 20, FAST = 10, SLOW = 200;

function run(pick) {
  const freeAt = [0, 0];            // when each backend finishes its queue
  const done = [[], []];            // finish time of every request it took
  const waits = [];
  for (let i = 0; i < N; i++) {
    const t = i * GAP, cost = i % 7 === 0 ? SLOW : FAST;
    const inFlight = done.map((ends) => ends.filter((e) => e > t).length);
    const b = pick(i, inFlight);
    const start = Math.max(t, freeAt[b]);
    freeAt[b] = start + cost;
    done[b].push(freeAt[b]);
    waits.push(start - t);          // time spent queued behind earlier requests
  }
  const s = [...waits].sort((a, b) => a - b);
  const avg = (waits.reduce((a, b) => a + b, 0) / N).toFixed(1);
  return { avg, p50: s[Math.floor(N * 0.5)], p99: s[Math.floor(N * 0.99)], max: s[N - 1] };
}

const roundRobin = (i) => i % 2;
const leastConnections = (i, inFlight) => (inFlight[1] < inFlight[0] ? 1 : 0);

console.log("round-robin      ", run(roundRobin), "  (ms waited in queue)");
console.log("least-connections", run(leastConnections));

$ node rr-vs-lc.mjs
round-robin       { avg: '72.6', p50: 70, p99: 160, max: 160 }   (ms waited in queue)
least-connections { avg: '53.5', p50: 10, p99: 180, max: 180 }

Round-robin matches the table exactly. Least-connections matches on the median, the p99 and the max, and its average is 53.5 ms against the 53.9 printed above. The original script was not kept, and none of twenty variants of tie-breaking and of counting “in flight” gave 53.9, so read the 0.4 ms as the cost of rebuilding it, not as a finding. The point the table makes does not move: least-connections fixes the median, and the slow requests still wait as long. Run on node v26.10.0, 2026-10-17.

Run it

Commands to type yourself; the # lines are notes, not output:

cd teaching/code/s2-loadbalancer
go test ./...          # round-robin spread + failover, both asserted

# or drive it by hand: start two toy backends + the LB, then kill one
python3 -m http.server 9001 &     # backend A
python3 -m http.server 9002 &     # backend B
go run . -backends http://localhost:9001,http://localhost:9002
# in another shell:  curl localhost:8080  (watch it alternate; kill 9002, watch it route around)

// A concurrent HTTP load balancer: round-robin across a pool of backends,
// with active health checks (probe on a timer) and passive ones (mark a
// backend DOWN when a real proxied request to it fails). Built on the stdlib
// net/http/httputil reverse proxy — no dependencies.
//
// Companion lesson: teaching/lessons/portfolio-projects/0147-s2-build-a-load-balancer.md
package main

import (
	"net/http"
	"net/http/httputil"
	"net/url"
	"sync"
	"sync/atomic"
)

type Backend struct {
	URL   *url.URL
	proxy *httputil.ReverseProxy
	mu    sync.RWMutex
	alive bool
}

func (b *Backend) SetAlive(a bool) { b.mu.Lock(); b.alive = a; b.mu.Unlock() }
func (b *Backend) Alive() bool     { b.mu.RLock(); defer b.mu.RUnlock(); return b.alive }

// NewBackend wires a reverse proxy to target. Its ErrorHandler is the PASSIVE
// health check: if a live request to this backend errors, mark it DOWN so the
// pool stops routing to it immediately.
func NewBackend(target string) (*Backend, error) {
	u, err := url.Parse(target)
	if err != nil {
		return nil, err
	}
	b := &Backend{URL: u, alive: true}
	rp := httputil.NewSingleHostReverseProxy(u)
	rp.ErrorHandler = func(w http.ResponseWriter, r *http.Request, e error) {
		b.SetAlive(false)
		http.Error(w, "backend unavailable", http.StatusBadGateway)
	}
	b.proxy = rp
	return b, nil
}

type Pool struct {
	backends []*Backend
	current  uint64 // atomically incremented round-robin cursor
}

func (p *Pool) Add(b *Backend) { p.backends = append(p.backends, b) }

// NextPeer returns the next ALIVE backend in round-robin order, or nil if the
// whole pool is down. Skipping dead backends is what makes failover automatic.
func (p *Pool) NextPeer() *Backend {
	n := uint64(len(p.backends))
	if n == 0 {
		return nil
	}
	start := atomic.AddUint64(&p.current, 1)
	for i := uint64(0); i < n; i++ {
		if b := p.backends[(start+i)%n]; b.Alive() {
			return b
		}
	}
	return nil
}

// ServeHTTP makes the Pool itself the load balancer's http.Handler.
func (p *Pool) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	b := p.NextPeer()
	if b == nil {
		http.Error(w, "no healthy backends", http.StatusServiceUnavailable)
		return
	}
	b.proxy.ServeHTTP(w, r)
}

// HealthCheck is the ACTIVE probe: hit path on every backend and flip its
// UP/DOWN flag. Call this on a ticker.
func (p *Pool) HealthCheck(path string) {
	for _, b := range p.backends {
		res, err := http.Get(b.URL.String() + path)
		up := err == nil && res.StatusCode < 500
		if res != nil {
			res.Body.Close()
		}
		b.SetAlive(up)
	}
}

package main

import (
	"net/http"
	"net/http/httptest"
	"sync/atomic"
	"testing"
)

// The two behaviours that make it a load balancer: it SPREADS load round-robin,
// and it FAILS OVER when a backend dies (no request lands on the dead one).
func TestRoundRobinAndFailover(t *testing.T) {
	var hits [2]int32
	mk := func(i int) *httptest.Server {
		return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			atomic.AddInt32(&hits[i], 1)
			w.Write([]byte("ok"))
		}))
	}
	s0, s1 := mk(0), mk(1)
	defer s0.Close()

	pool := &Pool{}
	for _, s := range []*httptest.Server{s0, s1} {
		b, err := NewBackend(s.URL)
		if err != nil {
			t.Fatal(err)
		}
		pool.Add(b)
	}
	lb := httptest.NewServer(pool)
	defer lb.Close()

	get := func() { res, err := http.Get(lb.URL); if err == nil { res.Body.Close() } }

	for i := 0; i < 10; i++ {
		get()
	}
	if atomic.LoadInt32(&hits[0]) == 0 || atomic.LoadInt32(&hits[1]) == 0 {
		t.Fatalf("round-robin didn't spread load: %v", hits)
	}

	// Kill backend 1, run the active health check -> it gets marked DOWN.
	s1.Close()
	pool.HealthCheck("/")

	before := atomic.LoadInt32(&hits[0])
	for i := 0; i < 6; i++ {
		get()
	}
	if got := atomic.LoadInt32(&hits[0]) - before; got != 6 {
		t.Fatalf("failover failed: expected all 6 requests on backend0, it got %d", got)
	}
}

package main

import (
	"flag"
	"log"
	"net/http"
	"strings"
	"time"
)

// Run: go run . -addr :8080 -backends http://localhost:9001,http://localhost:9002
func main() {
	addr := flag.String("addr", ":8080", "address to listen on")
	list := flag.String("backends", "", "comma-separated backend base URLs")
	flag.Parse()

	pool := &Pool{}
	for _, t := range strings.Split(*list, ",") {
		if t = strings.TrimSpace(t); t == "" {
			continue
		}
		b, err := NewBackend(t)
		if err != nil {
			log.Fatalf("bad backend %q: %v", t, err)
		}
		pool.Add(b)
	}
	if len(pool.backends) == 0 {
		log.Fatal("need at least one -backends URL")
	}

	go func() {
		tick := time.NewTicker(2 * time.Second)
		for range tick.C {
			pool.HealthCheck("/health")
		}
	}()

	log.Printf("load balancer on %s -> %s", *addr, *list)
	log.Fatal(http.ListenAndServe(*addr, pool))
}

package main

import (
	"io"
	"net/http"
	"net/http/httptest"
	"testing"
)

// The stage tests. Each file checks one stage: save stage1_test.go, make it
// pass, commit; then add the next. `go test -run Stage2` runs one stage. The
// helpers below are used by every stage, so this file stays.

// Stage 1: a reverse proxy in front of ONE backend. The visitor talks only to
// the balancer and gets the backend's body and status back unchanged.
func TestStage1ProxyOneBackend(t *testing.T) {
	be := backend(t, "A", http.StatusOK)
	lb := balancer(t, be.URL)

	if code, body := get(t, lb.URL+"/"); code != 200 || body != "A" {
		t.Fatalf("got %d %q; want 200 \"A\"", code, body)
	}
	if code, _ := get(t, lb.URL+"/missing"); code != 404 {
		t.Fatalf("a backend 404 came back as %d; a proxy passes it through", code)
	}
}

// backend answers "/" with its name, "/missing" with 404, and "/health"
// with whatever *health holds, so a test can make it sick and well again.
func backend(t *testing.T, name string, health int) *httptest.Server {
	t.Helper()
	code := health
	s := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		switch r.URL.Path {
		case "/":
			io.WriteString(w, name)
		case "/health":
			w.WriteHeader(code)
		default:
			http.NotFound(w, r)
		}
	}))
	t.Cleanup(s.Close)
	return s
}

func balancer(t *testing.T, urls ...string) *httptest.Server {
	t.Helper()
	lb := httptest.NewServer(pool(t, urls...))
	t.Cleanup(lb.Close)
	return lb
}

func pool(t *testing.T, urls ...string) *Pool {
	t.Helper()
	p := &Pool{}
	for _, u := range urls {
		b, err := NewBackend(u)
		if err != nil {
			t.Fatal(err)
		}
		p.Add(b)
	}
	return p
}

func get(t *testing.T, url string) (int, string) {
	t.Helper()
	res, err := http.Get(url)
	if err != nil {
		t.Fatal(err)
	}
	defer res.Body.Close()
	b, _ := io.ReadAll(res.Body)
	return res.StatusCode, string(b)
}

package main

import (
	"net/http"
	"testing"
)

// Stage 2: round-robin. Two backends, ten requests: they must alternate, so
// no two requests in a row land on the same one, and each gets exactly five.
func TestStage2RoundRobin(t *testing.T) {
	a, b := backend(t, "A", http.StatusOK), backend(t, "B", http.StatusOK)
	lb := balancer(t, a.URL, b.URL)

	count := map[string]int{}
	prev := ""
	for i := 0; i < 10; i++ {
		_, body := get(t, lb.URL+"/")
		if body == prev {
			t.Fatalf("request %d went to %s twice in a row", i+1, body)
		}
		count[body]++
		prev = body
	}
	if count["A"] != 5 || count["B"] != 5 {
		t.Fatalf("split was %v; want 5 and 5", count)
	}
}

package main

import (
	"net/http"
	"net/http/httptest"
	"sync/atomic"
	"testing"
)

// Stage 3, passive check: kill B. Exactly one request finds it dead and gets
// a 502; ErrorHandler marks B DOWN, and every request after that goes to A.
func TestStage3PassiveCheck(t *testing.T) {
	a, b := backend(t, "A", http.StatusOK), backend(t, "B", http.StatusOK)
	lb := balancer(t, a.URL, b.URL)
	b.Close()

	failed := 0
	for i := 0; i < 10; i++ {
		code, body := get(t, lb.URL+"/")
		switch {
		case code == http.StatusBadGateway:
			failed++
		case body != "A":
			t.Fatalf("request %d got %d %q after B died", i+1, code, body)
		}
	}
	if failed != 1 {
		t.Fatalf("%d requests failed; the passive check should cost exactly 1", failed)
	}
}

// Stage 3, active check: the probe is the only thing that notices a backend
// come BACK. B's /health says 503, the probe marks it DOWN; it says 200 again,
// the next probe marks it UP, and B gets traffic again.
func TestStage3ActiveCheckRecovery(t *testing.T) {
	var health atomic.Int32 // written here, read by the server goroutine
	health.Store(http.StatusServiceUnavailable)
	sick := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		if r.URL.Path == "/health" {
			w.WriteHeader(int(health.Load()))
			return
		}
		w.Write([]byte("B"))
	}))
	defer sick.Close()
	a := backend(t, "A", http.StatusOK)
	p := pool(t, a.URL, sick.URL)
	lb := httptest.NewServer(p)
	defer lb.Close()

	p.HealthCheck("/health")
	for i := 0; i < 4; i++ {
		if _, body := get(t, lb.URL+"/"); body != "A" {
			t.Fatalf("B is DOWN but request %d got %q", i+1, body)
		}
	}

	health.Store(http.StatusOK)
	p.HealthCheck("/health")
	seen := map[string]bool{}
	for i := 0; i < 4; i++ {
		_, body := get(t, lb.URL+"/")
		seen[body] = true
	}
	if !seen["B"] {
		t.Fatal("B is healthy again but got no traffic: nothing flipped it back UP")
	}
}

Why python3 -m http.server passes the health check. It has no /health page at all: curl -s -o /dev/null -w "%{http_code}\n" localhost:9001/health prints 404. The Go probe still marks it UP, because HealthCheck counts any status below 500 as alive (up := err == nil && res.StatusCode < 500 in lb.go). A 404 proves the process is answering, which is all this probe asks. A stricter probe that wanted a 200 would mark both Python backends DOWN on its first tick, and every request would get no healthy backends. Driven by hand on 2026-10-17 with the commands above, then killing the Python on 9002:

$ for i in 1 2 3 4; do curl -s -o /dev/null -w "%{http_code} " localhost:8080/; done
200 200 200 200
$ for i in 1 2 3 4; do curl -s -o /dev/null -w "%{http_code} " localhost:8080/; done
502 200 200 200

The second loop ran with 9002 dead: one 502, the request that found it, then every request on 9001. That is the passive check, by hand.

Put it on GitHub

• Repo: golb. README leads with the board image and the “reverse proxy + policy + health check” framing.
• README gif: curl alternating, then kill one backend and show traffic route around it, with at most one 502 (the request that found the dead backend).
• Resume line: “Built a concurrent HTTP load balancer in Go - round-robin, active + passive health checks, reverse proxying, automatic failover.”

Ship it. The workflow, the badge, the README and the licence are the same four files as S1’s “Ship it”, with golb in place of gokv. Two parts of the README differ. The picture is the architecture flowchart above, redrawn in your words. “What I measured” is the failover rather than a speed: requests sent, requests failed, and which one, from your own hand run. Expect the gif to show one 502, not zero: the request that found the dead backend is the passive check’s whole price, and getting to zero would take a retry on another backend, which this build does not do. -race in the workflow earns its place here more than anywhere on the shelf, because the round-robin counter and the alive flags are shared by every request’s goroutine.

Try it

Five minutes, no Go for the first two:

1. Save the node sketch above as lb.mjs, run node lb.mjs. Watch 1-4 alternate, 6 fail, 7-9 land on 9001. Change setInterval(healthCheck, 1000) to 5000 and rerun: the passive check still catches request 6, the probe line just arrives later.
2. Comment out up.on("error", ...) and rerun. Now every second request after the kill is a 502 until the probe fires.
3. cd teaching/code/s2-loadbalancer && go test -v ./... - the same two facts, asserted.
4. Drive the Go one by hand, as in Run it, and kill a backend mid-loop.

Go deeper

• Nginx “Load Balancing” guide (algorithms and why): https://nginx.org/en/docs/http/load_balancing.html
• Go stdlib net/http/httputil.ReverseProxy - read Director/Rewrite and ErrorHandler: https://pkg.go.dev/net/http/httputil#ReverseProxy
• Flagship upgrade: swap round-robin for least-connections, add sticky sessions (hash the client IP) and weighted pools. Then put this LB in front of three copies of your S1 database - a distributed system, built end to end.

There is no flagship page on this site; that line names upgrades, not a lesson. Least-connections is the script above. The last idea, this balancer in front of three copies of a store, is S6, which starts with why the naive version loses keys.

Related

• The chai stall that got famous (load balancing) - the same idea as a story: one counter, many walas.

• HTTP 01: The Raw Socket through HTTP 08: Everything - build the HTTP server behind this balancer, from the socket up.

• TCP vs UDP, and the 3-way handshake - what “connection refused” is on the wire, the signal the passive check depends on.

• S1 · Build Your Own Database (for real) - the thing this balancer fronts once you run three copies of it.

• S3 · Build a Redis Server (that real clients can talk to) - the next brief, and the other thing to put behind this door.

• S4 · Build an HTTP Server (without net/http) - the server behind this door, as a portfolio project of its own.

• S6 · Build a Replicated Key-Value Store (Dynamo-style) - what goes wrong with three stores behind this balancer, and the ring that fixes it.

• Load Balancing & DNS (the operator's half) - the same door from the DevOps side, with DNS in front of it.

Retrieval practice

8 questions. Answer each one before you open it.