Entropy is one of the most maligned and misunderstood concepts in science. Maybe you’ve heard it defined as the “amount of disorder” in a system. And the second law of thermodynamics says the entropy of a closed system always increases over time. So you might think, why should you clean up your office if it will only get messier?
That might be true, but you can’t blame it on entropy. The messy-room metaphor is often used to introduce the idea (it’s usually a teenager’s bedroom—can you relate?), but it’s misleading. See, disorder doesn’t mean messiness or chaos; it refers to the number of ways the parts of a system can be arranged without changing the overall state of the system.
For example, say your “system” is just a box full of air. Inside, at the microscopic level, the gas molecules are bouncing around like bumper cars. Now, imagine you could map the location and velocity of each particle at a given instant. That would be one possible arrangement, or microstate, but there are an infinitude of others, and they’re changing trillions of times a second.
Of course, you can’t really see this stuff. Instead, what you observe are overall, macro-level properties like air pressure; if you sealed the box at sea level, that would be 14.7 pounds per square inch. And unless you add energy to the system, say by heating it, that doesn’t change. So all those microstates correspond to the macrostate of 14.7 psi. Get it?
In other words, entropy is all about the link between the invisible atomic realm and the visible, measurable realm of objects, the world we inhabit. You could say it’s a conceptual and mathematical bridge between two levels of reality. I mean, c’mon, that’s pretty cool.
Now, out of all possible outcomes, which ones actually occur? That’s basically random, so it’s a matter of probability. In fact, probability is fundamental to the idea of entropy, and this is what the messy-room image fails to capture. So I’m going to use a different analogy: rolling dice. Einstein once said “God doesn’t play dice with the universe.” Let’s just see about that, shall we?
Rolling the Bones
Imagine you roll a six-sided game die. You get a number from 1 to 6, right? There are six possible outcomes, or states. If you roll the 20-sided die in Dungeons & Dragons, there are 20 possible states. If you want to wow your D&D pals, you could casually remark that this die has a higher entropy—because it has more possible outcomes.
Now say you’re determining a character’s abilities in D&D, and you roll three six-sided dice. The three values can add up to anything between 3 and 18, but the various sums are not equally likely. For maximum dexterity, say, you need an 18. Well, there’s only one way to achieve that: Each die must come up a 6.
But if moderate dexterity is enough for you, you might be fine with a level 10. That’s easier to get, because there are more combinations that add up to 10—six unique sets of numbers to be exact:
But wait. If we roll the dice one at a time and take sequence into account, there are even more permutations. Take 6-3-1 on the left. You could get the same three values in five other ways: 1-6-3, 3-1-6, 3-6-1, 6-1-3, 6-3-1. (Yes, these are different outcomes because time’s arrow moves in one direction.) All in all, there’s 27 different ways to roll a 10.
If we look at all possible results for three dice, there are 216 distinct microstates. But what matters for the game is the sum of the three values—that’s our macrostate. So the odds of rolling an 18 are 0.4 percent (1 out of 216), while the odds of rolling a 10 are 12.5 percent (27 out of 216).
We can say the 10 state has a higher entropy because there are more ways it can occur. And because there are more ways it can occur, it’s more likely to occur. See? There’s no mysterious force increasing the entropy of a system. It’s just that states with higher entropy have a higher probability. It’s actually kinda simple, really.
Opposite World
Now let’s think about this in terms of energy. Say you take a glass of cold water with a temperature of 50 degrees F, and you drop a hot, 120-degree ball of copper into it. What happens? Well, from experience, you’d expect the water to get warmer and the ball to get cooler, until they equalize at some temperature between 50 and 120.
But what do we mean when we say something gets warmer? We mean that its atoms and molecules increase in kinetic energy—they get more jiggly. Say the water gains 50 joules of thermal energy. Then, since energy is always conserved, we know that the copper ball cools off, losing the same 50 joules of energy.
But wait. What if, on a particular Tuesday, you dropped the hot ball in the cold water and the ball got hotter, increasing in thermal energy by 10 joules, while the water lost 10 joules and got colder? Did you just break physics? Nope. Energy is still conserved. You might find this disturbing, but it could happen. Why? Entropy. It’s one possible distribution of energy—just an extremely unlikely one. Basically, you won the Lotto.
An Object Lesson
Now imagine you have a tiny little solid. It's so tiny, it has only three atoms. (Remember the three-dice analogy?) Quantum mechanics tells us that atoms can only have certain energy levels—just like a die can roll a 2 or a 3 but not a 2.5. The point is that if the total energy of these three atoms is 10 units, then as we saw, there are 27 ways those 10 units of energy can be distributed.
Let’s take this just one step further: Say we have two tiny objects, A and B, with different amounts of thermal energy. Object A consists of two atoms (dice) and has a total energy of 3 units. B has three atoms (dice) and 7 units of energy. This puts the total energy in the system (A and B) at 10 units. Here, a picture will help:
Now suppose we put objects A and B in contact so that thermal energy can transfer between them. This means there could be many different arrangements of energy in the system, as long as the total remains 10 (for conservation of energy). Here are four of the many possible combos:
What if object A has just 2 units of energy? There's only one way this will work—each die must be a 1 (since 1 + 1 = 2). That means object B must have a total of 8, and there are 21 ways that could happen. What if A has 4 units and B has 6 units? In this case there are 30 possible combinations. You could say this state is “less ordered” or “more disordered,” and it’s more likely.
The Law of Large Numbers
This leads us to one of the definitions of entropy as a measure of the number of ways you can arrange energy in a system. We can write entropy (S) as the natural log of the number of microstates (Ω) multiplied by Boltzmann’s constant (kb):
.png)
Of course, our model with objects A and B is silly because most things aren't made of two or three atoms. For a reality check, a single drop of water contains about 1.7 sextillion (1.7 x 1021) H2O molecules. So what happens when we scale up the numbers by many orders of magnitude?
The same ideas still hold, but now we have a vastly larger number of microstates, and the probability distribution gets way more concentrated around the one with the highest entropy. So while the hot ball could get hotter in cold water, it’s so unlikely that it has never happened.
And the state of thermal equilibrium, where they end up at the same temperature is so entirely likely that, in practice, we treat it as a certainty. In fact, the second law of thermodynamics says heat always flows from a hotter object to a cooler object. But it isn’t a “law”; it’s just the odds. Turns out God really does play dice with the universe.
Comments
Back to top