Précis of epistemology/Computers and power consumption

Computers cannot run without consuming power. We prove it by thermodynamics, with the principle of the impossibility of perpetual motion of the second kind.

The perpetual motion of the second kind

A machine that could lift a weight or move a car without being supplied with energy could make a perpetual motion of the first kind. The law of conservation of energy, the first law of thermodynamics, forbids the existence of such a machine. It is one of the most fundamental laws of physics. All physicists would be wrong if we could invent such a machine, but no one has ever invented it.

Perpetual movement of the second kind does not contradict the law of conservation of energy. Any body can yield energy if it is cooled, unless it is at zero temperature, equal to 0 Kelvin = -273.15 °Celsius = -459.67 °Fahrenheit. So we can imagine a car, a boat or a plane that could advance without consuming fuel. It would only have to absorb air or water at room temperature and reject it at a colder temperature. The difference in energy would be used to run the engine.

For such an engine to work it would be necessary to be able to separate a system whose temperature is uniform in two parts, one warmer, the other colder. But it is forbidden by the second law of thermodynamics because it would reduce total entropy. The impossibility of the perpetual motion of the second kind thus results from the law of non-decrease of the total entropy, the second law of thermodynamics.

Maxwell's demon

A Maxwell's demon shows that information can be transformed into work:

Consider a gas in a container. A partition is placed in the middle. It is equipped with a small door controlled by a device that detects the speed of the incident molecules. It opens the door only if a molecule that comes from the left goes faster than average or if a molecule that comes from the right goes slower than average. In this way the right compartment is warmed while the left one is cooled (Maxwell 1871). This difference in temperature can be used to operate a heat engine.

The door opener is a Maxwell's demon. It acquires information that can be transformed into work. Information is therefore a kind of fuel.

Maxwell invented his "demon" to show that the law of entropy non-decrease is only a statistical truth that could be transgressed if one were able to modify the statistical equilibrium of the microscopic constituents. In its time, the existence of atoms and molecules was still very hypothetical. To consider the possibility of manipulating them was therefore out of the question. But as soon as the microscopic constituents of matter were better known, the possibility of a mechanical device that functions like a Maxwell's demon could be taken seriously.

To date, our ability to observe and manipulate microscopic constituents does not allow the device imagined by Maxwell to be realized, but scanning tunneling microscopy makes it possible to observe and manipulate atoms. One can then imagine a device that allows to recover work after reducing the entropy of the observed system, so a sort of theoretically feasible Maxwell's demon:

Consider a crystal that can accommodate atoms on its surface. It is assumed that initially $N_{A}$ atoms are randomly distributed on $N_{S}$ sites and that the temperature is low enough that they stay there. It is therefore a frozen disorder. We begin by observing the exact configuration of the surface atoms, which can be done with a scanning tunneling microscope, then we move them and collect them with the same microscope on a fraction ${\frac {N_{A}}{N_{S}}}$ of the surface. The activity of the microscope resembles an isothermal compression work on a gas, except that it is not a gas but a frozen disorder on the surface.

In principle, the displacement of atoms does not require any work because the work of tearing an atom can be recovered during redeposition.

To convert the activity of the scanning tunneling microscope into work, we put the surface of the crystal in contact with an empty container of volume $V$ whose other walls can not accommodate the atoms. This container is divided with a movable wall into two parts left and right whose volumes are respectively $V_{G}={\frac {N_{A}}{N_{S}}}V$ and $V_{D}={\frac {N_{S}-N_{A}}{N_{S}}}$. The crystal is heated to vaporize the atoms in the volume $V_{G}$. The resulting gas is then allowed to relax isothermally throughout the container, providing a work $W=N_{A}k_{B}T\ln {\frac {N_{S}}{N_{A}}}$ where $k_{B}$ is the Boltzmann constant and $T$ is the temperature. The crystal is then cooled to allow the atoms to redeposit on the surface of the crystal. If one proceeds reversibly, with a succession of thermal baths, the heat supplied during the heating by each thermal bath used is exactly equal to the heat it recovers during the cooling, because the specific heat at constant volume of a gas does not depend on its volume. The crystal and the thermal baths that were used to warm it have returned to their initial state.

The thermal bath which was used for the isothermal expansion gave up part of its heat which was converted into work. It therefore seems that we have achieved a perpetual motion of the second kind, since all the rest of the device has returned to its initial state.

The tunneling microscope which orders the atoms on the surface must be controlled by a computer programmed to fetch the atoms and deposit them at a suitable site. Since the second law of thermodynamics prohibits perpetual motion of the second kind, this computer must consume an energy at least equal to the work $W$ which was provided during the isothermal expansion. Hence a computer cannot function without consuming energy.

It has been supposed that an absorbing wall can make a perfect vacuum in an arbitrarily large volume. Such a wall can not exist otherwise one could make a perpetual motion of the second kind: the wall charged with atoms is placed in contact with an empty container, it is heated to a temperature sufficiently hot that all atoms are vaporised. The gas is then allowed to relax isothermally to provide work. The gas is then cooled to a temperature sufficiently cold that all atoms redeposit on the absorbing wall. If one proceeds reversibly the heat supplied during heating by each thermal bath is exactly equal to the heat it recovers during cooling. We could therefore return to the initial state after providing work by extracting heat from a single heat bath.

In order to make an exact calculation that is compatible with the laws of thermodynamics, it is necessary to take into account the equilibrium density of a gas in contact with an absorbing wall. This density can not be zero, but it can be very small, a priori as small as one wants if the wall is sufficiently absorbent. This is enough to justify the calculation above where this density is neglected.

The Szilard's engine

To better understand the transformation of information into work, Szilard (1929) invited us to reason about an engine that works with a "one molecule gas":

A molecule is enclosed in a container that can be separated by a removable wall. When the wall is put in place in the middle of the container, the presence of the molecule in one or the other of the separate compartments is detected. So a bit of information is acquired. The wall is then used as a piston on which the molecule can work. It is necessary to know where the molecule is to know in which direction of piston displacement a work can be recovered. In this way it is calculated that under optimal conditions an bit of information is used to recover a work equal to $k_{B}T\ln 2$. This is the work done by a molecule on a piston during an isothermal expansion at the temperature $T$ that doubles the accessible volume.

Szilard's engine seems to contradict thermodynamics because it suggests that we could make a perpetual motion of the second kind. If the piston can move in only one direction, it is not necessary to know the position of the molecule to recover work. Once in two the piston remains motionless because the molecule is on the wrong side of the piston, and we do not recover any work, but once in two it moves and we recover a work equal to $k_{B}T\ln 2$. By repeating the experiment many times we could thus obtain an arbitrary amount of work without spending any to know the position of the molecule.

But such a process requires a device that removes the piston and puts it back in place. Now there are two possible positions of the piston at the end of a cycle, either in the middle if it has not moved, or at one end, if it has moved. The device must therefore acquire a bit of information at the end of each cycle. To return to its initial state, it must erase this information. The cost of erasing information is thus here too at the origin of the impossibility of a perpetual motion of the second kind (Leff & Rex 1990).

A new proof of Szilard's theorem

On the surface of the crystal on which atoms are deposited, the number of possible configurations is equal to the number $(_{N_{S}}^{N_{A}})$ of ways to place $N_{A}$ atoms on $N_{S}$ sites . The quantity $I$ of information necessary to know the fixed disorder of atoms on the surface is therefore:

$I=\ln(_{N_{S}}^{N_{A}})=\ln {\frac {N_{S}!}{(N_{S}-N_{A})!N_{A}!}}=N_{S}\ln {\frac {N_{S}}{N_{S}-N_{A}}}+N_{A}\ln {\frac {N_{S}-N_{A}}{N_{A}}}$

where we used Stirling's approximation: $\ln N!\approx N\ln N-N$

Suppose ${\frac {N_{S}}{N_{A}}}\gg 1$. So

$I\approx N_{A}(\ln {\frac {N_{S}}{N_{A}}}+1)$

If in addition $\ln {\frac {N_{S}}{N_{A}}}\gg 1$, we obtain

$I\approx N_{A}\ln {\frac {N_{S}}{N_{A}}}$

Now this quantity of information can be used to provide a work $W=N_{A}k_{B}T\ln {\frac {N_{S}}{N_{A}}}$. We thus obtain an example of the following theorem:

A quantity $I$ of information can produce a work $W=k_{B}TI$ when used by a machine at the temperature $T$.

The minimum energy consumption by a computer

A computer that acquires an amount of information equal to $I$ must erase it to revert to its initial state. Now this quantity of information can be used to produce a work $W=k_{B}TI$. Since perpetual motion of the second kind is impossible, a computer that erases a quantity $I$ of information must consume an energy at least equal to $W=k_{B}TI$.

References

Bennett, Charles H., The thermodynamics of computation - a review (Int. J. Theor. Phys. 21, 905-40, 1982, reproduced in Leff & Rex 1990)

Diu, Bernard, Guthmann, Claudine, Lederer, Danielle, Roulet, Bernard, Physique statistique (1989)

Feynman, Richard P., Lectures on computation (1996)

Landauer, Rolf W., Irreversibility and heat generation in the computing process (IBM J. Res. Dev. 5, 183-91, 1961, reproduced in Leff & Rex 1990)

Leff, Harvey S., Rex, Andrew F., Maxwell's demon, entropy, information, computing (1990)

Maxwell, James Clerk, Theory of heat (1871)

Sethna, James P., Statistical mechanics : entropy, order parameters and complexity (2018)

Szilard, Leo, On the decrease of entropy in a thermodynamic system by the intervention of intelligent beings (Zeitschrift für Physik, 1929, 53, 840-856, translated in Leff & Rex 1990)