Entropy - why time only points forward

Counting arrangements, from a few coins to a whole box of gas, to show why messy so heavily outnumbers tidy that time only seems to run one way.

Before you start. You need to be comfortable counting: if you have three coins, how many ways can they land? That is the only maths in the story half, and every number is worked out as it comes up.

Big numbers below are written with powers of ten. 10^67 means a 1 followed by 67 zeros, so 8 × 10^67 is 8 followed by 67 zeros, and 2^10 means ten 2s multiplied together, which is 1,024.

1877: a physicist tries to count what he could not see

Shuffle a fresh deck of cards once, then try to shuffle it straight back to new-pack order, ace to king, suit by suit. You already know it will not work, not on the next try, not in a year of trying. Nothing about a shuffle forbids it: a shuffle is just cards moving, and run the exact same motions backward and you get a legal shuffle too. So why does new-pack order never come back?

In 1877 a physicist named Ludwig Boltzmann, working at the University of Graz in Austria, answered a version of that question for gases instead of cards. The laws that push molecules around are exactly as reversible as a card shuffle: rewind any single collision between two molecules and it is still a legal collision. Yet a warm room never spontaneously gathers its air into one corner. Boltzmann’s answer was not a new force. It was a count.

Microstates and macrostates: entropy is the count between them

Take just four cards and ask two different questions about the same deck. One question is coarse: does it look sorted, or does it look shuffled? The other is exact: which of the 24 possible orders is it actually in, 4 × 3 × 2 × 1 = 24, checked by hand. The coarse answer is what Boltzmann called a macrostate, the description you would actually give someone. The exact order is a microstate, one specific arrangement of every card.

flowchart LR
  D["4 cards, shuffled"] --> M{"which of the 24 orders?"}
  M -->|"1 way"| S["looks sorted"]
  M -->|"23 ways"| U["looks shuffled"]

Only 1 of the 24 microstates looks sorted. The other 23 all look shuffled, even though they are 23 different exact orders. Shuffle blind and you land on a shuffled-looking order 23 times as often as the sorted one, not because anything pushes the cards toward mess, but because mess has 23 times as many ways to happen.

That count of matching microstates is what Boltzmann named entropy. His formula for it, carved on his gravestone in Vienna’s Zentralfriedhof cemetery, is S = k · log W. In plain words: entropy (S) is a constant (k, just a unit-converter) times the count of matching microstates (W), and the “log” is there so that when W is too large a number to say out loud, S stays a normal-sized one.

The fight over whether atoms were even real

Boltzmann’s count only works if there really are separate molecules to count. Many respected scientists of the era did not believe there were. At a conference in Lübeck in 1895, two of them, the chemist Wilhelm Ostwald and Georg Helm, argued that energy, not matter, was the universe’s real substance, and that atoms were a convenient fiction. Boltzmann spent years defending an idea most of his field treated as unproven bookkeeping.

In September 1906 Boltzmann died. He never saw the proof arrive: in 1908 and 1909, Jean Perrin’s experiments with suspended particles, building on a 1905 paper by Albert Einstein, showed molecules really are there, in the numbers Boltzmann’s counting needed all along.

The second law: a bookmaker, not a commandment

Boltzmann’s count is the whole content of what is called the second law of thermodynamics. It does not command anything to happen. It says: of all the ways a system’s pieces could be arranged, the system will overwhelmingly likely be found in whichever arrangement has the most ways to look that way. Deal four cards and a shuffled-looking order shows up 23 times more often than the one sorted order, for exactly that reason. Deal a real room full of trillions of air molecules and the same counting applies: all the air gathering into one corner is not forbidden, only so outnumbered by “spread evenly” arrangements that it will not happen in the time the universe has existed. Not a wall. A landslide of ratio.

The arrow of time: why forward is forward

Because that count only ever grows and never shrinks, entropy is the one quantity in physics that tells past from future. Every other law reads the same played backward. Call the direction entropy grows “forward”, and you have named the arrow of time. It is why a memory can only be of the past: recording anything, even a thought in a brain, is a physical change, and every physical change adds to the running count. You remember yesterday and not tomorrow because “recording a memory” only runs in the direction the count is already moving.

Where the deck stops matching

Here is where the analogy runs out, and it matters. Leave a shuffled deck alone on a table and it stays exactly as shuffled as you left it: nothing reshuffles it without a hand. A room full of air never stops moving. Heat is exactly that motion, so the room reshuffles itself, on its own, trillions of times a second, always finding one of the astronomically more numerous messy arrangements. A deck needs a shuffler. A gas is its own shuffler, and that is the one way it behaves nothing like a deck of cards.

You now know

• Entropy is a count: how many exact arrangements (microstates) share one general appearance (a macrostate), not a force pushing anything anywhere.
• Messy outnumbers tidy by an astronomical margin, so systems drift toward messy for the same reason a shuffled deck lands on a shuffled-looking order far more often than the sorted one: there are simply more ways to be messy.
• The second law of thermodynamics is that drift stated plainly: a system moves toward whichever arrangement has the most ways to look that way.
• The arrow of time is the direction entropy grows. It is the only thing in physics that tells the past from the future, since every other law works the same played backward.
• Ludwig Boltzmann worked this out in 1877 and spent years defending it against physicists who did not believe atoms were real. He died in 1906, two years before experiments finally showed molecules are real.
• Recording a memory is a physical change, and physical changes only run in the direction entropy already grows, which is why you remember the past and not the future.

Stop here if you want. What follows is the same counting argument in numbers, for when you want to see exactly how large “astronomical” is.

Second read: the numbers behind “astronomical”

Two dice, then coins, then a real mole of gas: the same counting, scaled up, and turned into joules per kelvin with the real Boltzmann constant, S = k ln W applied exactly as written.

const k = 1.380649e-23;   // Boltzmann's constant, J/K (exact since 2019)
// 1. two dice: macrostate = the total you read; microstate = (die A, die B)
const dice = {};
for (let a = 1; a <= 6; a++) for (let b = 1; b <= 6; b++) dice[a + b] = (dice[a + b] || 0) + 1;
console.log("two dice: total -> ways (W)");
for (const t of Object.keys(dice)) console.log(`  ${String(t).padStart(2)} -> ${dice[t]}`);
console.log("  36 microstates, 11 macrostates; a 7 is", dice[7] / dice[2], "x likelier than a 2\n");
// 2. N coins: macrostate = how many heads; W = N choose h. "Ordered" = all heads. "Spread" = half heads.
const choose = (n, r) => { let x = 1n; for (let i = 1n; i <= BigInt(r); i++) x = x * (BigInt(n) - i + 1n) / i; return x; };
for (const N of [10, 100, 1000]) {
  const all = choose(N, 0), half = choose(N, N / 2), total = 2n ** BigInt(N);
  const digits = half.toString().length;
  console.log(`${N} coins: all heads W = ${all}   half heads W = ${digits > 12 ? half.toString().slice(0, 6) + "... (" + digits + " digits)" : half}   total arrangements 2^${N} = ${total.toString().length} digits`);
  const pHalf = Number(half * 10n ** 6n / total) / 1e6;
  console.log(`   chance of exactly half heads ${pHalf.toFixed(4)},  of all heads 1 in ${total.toString().length > 15 ? "10^" + (total.toString().length - 1) : total}`);
  // Boltzmann entropy difference between the two macrostates
  const dS = k * (Math.log(Number(half)) - Math.log(Number(all)));
  console.log(`   S(half) - S(all heads) = k ln(W_half / W_all) = ${dS.toExponential(3)} J/K   (ln W_half = ${Math.log(Number(half)).toFixed(2)})\n`);
}
// 3. a real object: one mole of gas (6.022e23 molecules) confined to the left half of a box vs spread over the whole box
const NA = 6.02214076e23;
const dSmole = NA * k * Math.log(2);           // each molecule has 2x the positions available
console.log(`one mole of gas, left half -> whole box:`);
console.log(`  W_whole / W_left = 2^${NA.toExponential(3)}  (a number with ${(NA * Math.log10(2)).toExponential(3)} digits)`);
console.log(`  dS = N k ln2 = ${dSmole.toFixed(3)} J/K   (= R ln 2, R = 8.314 J/mol/K)`);
console.log(`  odds of finding it all back in the left half: 1 in 10^${(NA * Math.log10(2)).toExponential(3)}`);
// 4. the egg, roughly: 1 J of heat spread into a 300 K room raises the room's entropy by dQ/T
console.log(`\n1 J of splat heat into a 300 K room: dS = 1/300 = ${(1 / 300).toExponential(3)} J/K, which is ln W growing by ${(1 / 300 / k).toExponential(3)} - W multiplies by e to that power`);

two dice: total -> ways (W)
   2 -> 1
   3 -> 2
   4 -> 3
   5 -> 4
   6 -> 5
   7 -> 6
   8 -> 5
   9 -> 4
  10 -> 3
  11 -> 2
  12 -> 1
  36 microstates, 11 macrostates; a 7 is 6 x likelier than a 2

10 coins: all heads W = 1   half heads W = 252   total arrangements 2^10 = 4 digits
   chance of exactly half heads 0.2461,  of all heads 1 in 1024
   S(half) - S(all heads) = k ln(W_half / W_all) = 7.634e-23 J/K   (ln W_half = 5.53)

100 coins: all heads W = 1   half heads W = 100891... (30 digits)   total arrangements 2^100 = 31 digits
   chance of exactly half heads 0.0796,  of all heads 1 in 10^30
   S(half) - S(all heads) = k ln(W_half / W_all) = 9.221e-22 J/K   (ln W_half = 66.78)

1000 coins: all heads W = 1   half heads W = 270288... (300 digits)   total arrangements 2^1000 = 302 digits
   chance of exactly half heads 0.0252,  of all heads 1 in 10^301
   S(half) - S(all heads) = k ln(W_half / W_all) = 9.519e-21 J/K   (ln W_half = 689.47)

one mole of gas, left half -> whole box:
  W_whole / W_left = 2^6.022e+23  (a number with 1.813e+23 digits)
  dS = N k ln2 = 5.763 J/K   (= R ln 2, R = 8.314 J/mol/K)
  odds of finding it all back in the left half: 1 in 10^1.813e+23

1 J of splat heat into a 300 K room: dS = 1/300 = 3.333e-3 J/K, which is ln W growing by 2.414e+20 - W multiplies by e to that power

Read it as four rungs of one ladder.

Two dice are the whole idea small enough to see: 36 microstates, 11 macrostates. A 7 comes up 6 times as often as a 2, for no reason except that six pairs of faces make 7 and only one pair makes 2.

Coins are where tidy gets rare fast. Ten coins: all-heads is 1 arrangement out of 1,024. A hundred coins: all-heads is 1 in 10^30 - flip a hundred coins once a second for the age of the universe (4 × 10^17 seconds) and you would still be short by a factor of 10^12. The entropy difference between messy and tidy grows far slower than W itself, because entropy is a log of W: a hundred times more coins bought about 125 times more entropy, while W itself grew by a factor of 10^300. That squeeze is why the same formula describes ten coins and a whole universe.

A mole of gas is the egg’s scale. 6 × 10^23 molecules, each with twice the room once let out of half a box, multiply W by 2^(6×10^23), a number with 1.8 × 10^23 digits. The entropy difference, N k ln 2 = 5.76 J/K, is the gas constant R times ln 2, the same number a chemistry textbook gets from dQ/T with no counting at all - which is the reason to trust the counting. The odds of the gas ever wandering back into half the box are 1 in 10^(1.8 × 10^23). “Never” is the honest word.

A splat, roughly. One joule of heat spread into a 300 K room raises the room’s entropy by 1/300 J/K. Divide by k and ln W has grown by 2.4 × 10^20, so the number of arrangements the room can be in has multiplied by e to that power. A real splat is a tenth of a joule per gram over tens of grams, into a floor and air that are each mole upon mole of molecules - which is why a reversed film reads as fake in under a second.

Being a pocket of low entropy, on purpose

A living body is locally low entropy, and that does not break the second law, because the law only bounds closed systems. A body eats concentrated order (food) and dumps more disorder (heat, waste) into its surroundings than it builds inside. An adult eating 2,000 kcal a day takes in 2,000 × 4,184 J = 8.4 MJ and radiates it as heat at 8.4 MJ / 86,400 s ≈ 97 W - a body is a hundred-watt heater. Heat leaving skin at about 306 K carries away entropy at 97 / 306 ≈ 0.32 J/K every second, about 27,000 J/K a day: several thousand mole-of-gas-expansions’ worth of entropy exported daily, to keep a few kilograms of protein folded. A fridge runs the identical trick; the arithmetic for it is in Cold Is a Lie - fridge, AC, heat pump.

Why time has a direction at all

The second law only gives time a direction because the universe started in an absurdly low-entropy state - the Big Bang was freakishly smooth. Entropy has had 13.8 billion years of room to grow since. If it ever reaches maximum entropy, heat death, no usable gradients left, no more events can happen, and the arrow of time effectively stops.

Where the argument stops, honestly

Reversible laws plus counting say a tidy state will almost certainly be messier a moment later. Run the same argument backward, though, and they say with equal confidence that it was almost certainly messier a moment earlier too. Counting alone predicts entropy rising in both directions away from now; it cannot pick out a past on its own. Physicist Josef Loschmidt raised exactly this objection in 1876.

flowchart TD
  A["microscopic laws, reversible in time"] --> B["counting: far more messy microstates than tidy ones"]
  B --> C["from ANY state, entropy is overwhelmingly likely to rise"]
  C --> D{"but the same counting says entropy was likely HIGHER in the past too"}
  D --> E["so add a boundary condition: the universe began in a very low entropy state"]
  E --> F["entropy rises from there, and that direction is what we call the future"]
  E -.->|"not explained by anything in this lesson"| G["why was the beginning so tidy?"]

What breaks the symmetry is a fact, not a theorem: the universe’s entropy was extraordinarily low 13.8 billion years ago and has been climbing since, a boundary condition physicists call the Past Hypothesis. Add it and everything above follows. Leave it out and the second law has no arrow at all.

Two more places the argument is thinner than it sounds. “Low entropy at the Big Bang” does not mean the early universe looked tidy in the everyday sense; it was an almost perfectly uniform hot gas, the messiest state possible for ordinary matter. The tidiness is gravitational, a uniform mass distribution being the low-entropy state for gravity, which wants to clump - why gravity’s books were open at that page is an open question in cosmology. And the second law is a statement about probabilities, not certainties: the fluctuation theorem (Evans and Searles, 1994, confirmed by pulling a plastic bead through a solution with a laser in 2002) gives the exact odds of entropy briefly running backward in a small system. The dice above show the same thing at toy scale - a 2 does come up, one roll in thirty-six. What makes the arrow feel absolute for everyday objects is only that they hold 10^23 particles, where the odds against a reversal are 1 in 10^(10^23) - not zero, merely a number no clock can wait for.

Try it

Predict first. Before you shuffle: if you shuffled once a second for the age of the universe, would you ever expect to see the same order twice?

1. Shuffle a deck once and try to intuit 8 × 10^67, the number of orders it could have landed in. You will never shuffle it back to new-pack order, not because it is illegal, only because of the count.
2. Ink in water. A drop of ink in still water always spreads, never un-spreads. Narrate it as you watch: more ways to be spread than clumped.
3. Melting ice. A solid (few matching arrangements) becomes a liquid (vastly more). Feel the room’s warmth flow into the ice, never out of it. Downhill, always.

And three at the keyboard in five minutes, starting from the listing above saved as microstates.mjs:

node microstates.mjs
node -e 'let f=1n; for (let i=1n;i<=52n;i++) f*=i; console.log(f.toString().length, "digits:", f.toString().slice(0,4))'
node -e 'const k=1.380649e-23; console.log((97/306).toFixed(3), "J/K per second from a 97 W body at 306 K")'

The second line is the deck of cards from the story: 52! is a 68-digit number starting 8065, so “8 × 10^67 shuffles” is not a round figure, it is the count. Edit the coins loop to [10, 20, 30] and watch the “1 in” column: every ten extra coins multiplies the odds against all-heads by about a thousand. That is the whole reason “never” arrives so quickly.

Go deeper

• Maxwell’s demon and Landauer’s principle. A thought-experiment demon seems to lower entropy for free by sorting molecules. The resolution: the demon must erase its memory to keep sorting, and erasing one bit of information costs at least kT · ln 2 of entropy (Landauer). This is the bridge between information and thermodynamics: entropy in physics and Shannon entropy in information theory are the same quantity, the idea behind DNS on the wire-style bit-counting.
• Statistical vs classical entropy. Clausius defined dS = dQ/T decades before Boltzmann explained why, with counting. Same S, two doors in.
• Dissipative structures (Ilya Prigogine): ordered structures like hurricanes, cells and cities form spontaneously because forming them accelerates entropy production elsewhere. Order as entropy’s own tool.
• Poincaré recurrence. A closed system will, technically, return arbitrarily close to its start, after roughly 10^(10^23) times the age of the universe. “Never” is really “not in any meaningful time”.

Related

• Cold Is a Lie - fridge, AC, heat pump runs the identical arithmetic for a fridge: order made here, more disorder exported there, paid for in electricity.
• Twice the Speed, Four Times the Stop - braking, crumple zones and seatbelts is the same one-way trip in a car: every stop turns ordered motion into brake heat that never turns back.
• Every Plug Ends at a Spinning Magnet - power stations, fission and the solar exception is the bill for this law at a power station: about two-thirds of the fuel’s heat must be thrown away.

Retrieval practice

8 questions. Answer each one before you open it.