Queues: the order slip rail and the cliff

At the chai stall this was a rail of paper slips. In a system it is a queue, and the problem it solves is this. When taking work and doing work are the same step, one slow job freezes everyone standing behind it. So you split the step in two. The producer hands its work to a broker and gets an answer back immediately. Consumers pull from the other end, at whatever pace they can actually work. But a queue does not create capacity. It only smooths bursts. If work arrives faster than the consumers drain it for longer than a burst, the backlog grows forever, and no amount of buffering saves you. And the waiting does not grow gently. At fifty percent busy you wait about one cooking time. At ninety percent, nine. At ninety nine percent busy, a hundred and fifty. That is the utilization cliff. The broker does not trust the consumer either. A pulled job is hidden, not deleted, and it vanishes only once the consumer acknowledges it. Crash before that and it comes back. At least once, never exactly once. A job that always crashes would loop forever, so after a few tries it is moved to a dead letter queue. And when the rail is full, the kind answer is no. Shed the cheap work, throttle the loudest client, or reject. Healthy systems idle. The spare capacity you are always tempted to sell off is not waste. It is the shock absorber that keeps every wait on the flat part of that curve.