Entropy (thermodynamics)

"The first step came from W. Wien, whose displacement law of 1893 is embodied in the shift of the maximum of spectrum energy density, from red to violet, with increasing temperatures. Wien showed that a universal function of the ratio of temperature to frequency must here be in question. The determination of this universal function was the culmination of the insight and consistent labors of Planck (1900), who by postulating the energy quantum, became the creator of modern thermodynamics; for this energy element is a saucy reality, whose purpose is to stay. It not only tells us all we know of the distribution of energy in the black body spectrum in its thermal relations, but it gives us, indirectly, perhaps the most accurate data at hand of the number of molecules per normal cubic centimeter of the gas, of the mean translational energy of its molecules, of the molecular mass, of the Boltzmann entropy constant, even of the charge of the electron or electric atom itself."

"Of the Planck molecular oscillators... If operating continuously under the established electromagnetic laws they lead to the impossible distributions of energy in the spectrum investigated by Rayleigh and Jeans. But if emitting only, when their energy content is a whole number of energy elements, a case thus involving the entropy probability of Boltzmann, Wien's law and the numerical data referred to are deducible with astounding precision."

"We show that it is natural to introduce the concept of black-hole entropy as the measure of information about a black-hole interior which is inaccessible to an exterior observer. Considerations of simplicity and consistency, and dimensional arguments indicate that the black-hole entropy is equal to the ratio of the black-hole area to the square of the Planck length times a dimensionless constant of order unity. A different approach making use of the specific properties of Kerr black holes and of concepts from information theory leads to the same conclusion, and suggests a definite value for the constant."

"For well over a hundred years, a basic antithesis was noticed between inanimate and animate nature. The direction of physical events is prescribed by the second principle of thermodynamics... the general trend of physical happenings is toward most probable states, that is, maximum entropy and progressive destruction of differentiation and order. ...The system will tend toward thermal equilibrium ...a state of most probable distribution of molecules ...disappearance of the temperature gradient and uniform distribution ...maximum entropy. "Higher," directed forms of energy (e.g., mechanical, electric, chemical) are dissipated... progressively converted into the lowest form of energy, i.e., undirected heat movement of molecules; chemical systems tend toward equilibria with maximum entropy; machines wear out owing to friction; in communication channels, information can only be lost by conversion of messages into noise but not vice versa, and so forth."

"If for the entire universe we conceive the same magnitude to be determined, consistently and with due regard to all circumstances, which for a single body I have called entropy, and if at the same time we introduce the other and simpler conception of energy, we may express in the following manner the fundamental laws of the universe which correspond to the two fundamental theorems of the mechanical theory of heat. 1. The energy of the universe is constant. 2. The entropy of the universe tends to a maximum."

"All this prompts the question of why, from the infinite rage of possible values that Nature could have selected for the fundamental constants, and from the infinite variety of initial conditions that could have characterized the primeval universe, the actual values and conditions conspire to produce the particular range of special features that we observe. For clearly the universe is a very special place: exceedingly uniform on a large scale, yet not so precisely uniform that galaxies could not form; extremely low entropy per , and hence cool enough for chemistry to happen; almost zero cosmic propulsion and an expansion rate tuned to that energy content to unbelievable accuracy; values for the strengths of its forces that permit nuclei to exist, yet do not burn up all the cosmic , and many more apparent accidents of fortune."

"The fundamental problem about trying to define life in terms of physics is easily explained. If you go to a physics department... you'll be given a definition in terms of matter... force... energy... entropy... free energy, molecular binding affinities, and so on. If you go to a biology department... you'll be given a very different narrative in terms of... instructions, transcription, , translation, coding, signals... Biologists use information-speak... informational qualities... physicists define life in terms of physical quantities."

"Paul Davies, The Demon in the Machine (Sep. 7, 2019) 6th International FQXi Conference, "Mind Matters: Intelligence and Agency in the Physical World." A YouTube video source, 4:31."

"I know of no theorem that tells you... the maximum amount of change that agency can achieve in the universe, and what interests me... is agency at the end of the universe. If you end up in , which has a temperature and a horizon entropy, can you do anything with... those thermal fluctuations? Can you mine them... to extract energy?"

"Paul Davies, The Demon in the Machine (Sep. 7, 2019) 6th International FQXi Conference, "Mind Matters: Intelligence and Agency in the Physical World." A YouTube video source, 17:22."

"In the year 1900 Max Planck wrote... E = hv, where E is the energy of a light wave, v is its , and h is... . It said that energy and frequency are the same thing measured in different units. Plank's constant gives you a rate of exchange for for converting frequency into energy... But in the year 1900 this made no physical sense. Even Plank himself did not understand it. ...Now Hawking has written down an equation which looks rather like Plank's equation... S = kA, where S is the entropy of a black hole, A is the area of its surface, and k is... Hawking's constant. Entropy means roughly the same thing as the of an object. ...Hawking's equation says that entropy is really the same thing as area. The exchange rate... is given by Hawking's constant... But what does it really mean to say that entropy and area are the same thing? We are as far away from understanding that now as Planck was of understanding quantum mechanics in 1900. ...[T]his equation will emerge as a central feature of the still unborn theory which will tie together gravitation and quantum mechanics and thermodynamics."

"If I took a heavy weight on the floor here and pushed it, it would slide and stop. ... So, a frictional effect seems to be irreversible. ... a frictional effect ... is the result of enormous complexity of the interaction of the block with the wood ... the jiggling of the atoms inside the wood of the block is changed into disorganized irregular wiggle-waggles of the atoms in the wood."

"Newton and his theories were a step ahead of the technologies that would define his age. Thermodynamics, the grand theoretical vision of the nineteenth century, operated in the other direction with practice leading theory. The sweeping concepts of energy, , work and entropy, which thermodynamics (and its later form, statistical mechanics) would embrace, began first on the shop floor. Originally the domain of engineers, thermodynamics emerged from their engagement with machines. Only later did this study of heat and its transformation rise to the heights of abstract physics and, finally, to a new cosmological vision."

"Black holes have the universe's most inscrutable poker faces. ...When you've seen one black hole with a given mass, charge, and spin (though you've learned these thing indirectly, through their effect on surrounding gas and stars...) you've definitely seen them all. ...black holes contain the highest possible entropy ...a measure of the number of rearrangements of an object's internal constituents that have no effect on its appearance. ...Black holes have a monopoly on maximal disorder. ...As matter takes the plunge across a black hole's ravenous , not only does the black hole's entropy increase, but its size increases as well. ...the amount of entropy ...tells us something about space itself: the maximum entropy that can be crammed into a region of space—any region of space, anywhere, anytime—is equal to the entropy contained within a black hole whose size equals the region in question."

"A natural guess is that... a black hole's entropy is... proportional to its volume. But in the 1970s and Stephen Hawking discovered that this isn't right. Their... analyses showed that the entropy... is proportional to the area of its ... less than what we'd naïvely guess. ...Berkenstein and Hawking found that... each square being one by one Planck length... the black hole's entropy equals the number of such squares that can fit on its surface... each Planck square is a minimal unit of space, and each carries a minimal, single unit of entropy. This suggests that there is nothing, even in principle, that can take place within a Planck square, because any such activity could support disorder and hence the Planck square could contain more than a single unit of entropy... Once again... we are led to the notion of an elemental spatial entity."

"[F]or a physicist, the upper limit to entropy... is a critical, almost sacred quantity. ...the Bekenstein and Hawking result tells us that a theory that includes gravity is, in some sense, simpler than a theory that doesn't. ...If the maximum entropy in any given region of space is proportional to the region's surface area and not its volume, then perhaps the true, fundamental degrees of freedom—the attributes that have the potential to give rise to that disorder—actually reside on the region's surface and not within its volume. Maybe... the universe's physical processes take place on a thin, distant surface that surrounds us, and all we see and experience is merely a projection of those processes. Maybe... the universe is rather like a hologram."

"Just like a computer, we must remember things in the order in which entropy increases. This makes the second law of thermodynamics almost trivial. Disorder increases with time because we measure time in the direction in which disorder increases. You can’t have a safer bet than that!"

"The homeostatic principle does not apply literally to the functioning of all complex living systems, in that in counteracting entropy they move toward growth and expansion."

"Because entropy is not really a classical quantity, we must build quantum mechanics into the definition. ... It suffices to define entropy as the logarithm of the number of quantum states accessible to a system."

"As the natural sciences have developed to encompass increasingly complex systems, scientific rationality has become ever more statistical, or probabilistic. The deterministic classical mechanics of the enlightenment was revolutionized by the near-equilibrium statistical mechanics of late 19th century atomists, by quantum mechanics in the early 20th century, and by the far-from-equilibrium complexity theorists of the later 20th century. Mathematical , information theory, and quantitative social sciences compounded the trend. Forces, objects, and natural types were progressively dissolved into statistical distributions: heterogeneous clouds, entropy deviations, s, gene frequencies, noise-signal ratios and redundancies, dissipative structures, and complex systems at the edge of chaos."

"So if we're going to ask... What is life? ...Erwin Schrödinger wrote a famous book on that theme ...Two famous ideas ...emerged ...one ...was ...that genes are a code-script, and that was the first time anybody had used the word "code-script" or really thought in terms of information, in biology. ...This was before DNA was discovered. He was a direct inspiration to Watson and Crick and many others. The second theme... was how life maintains its organization over time, and why don't we just fall to pieces as entropy would tend to suggest... He talked about life feeding on negative entropy, or "negentropy"... [H]e talked about continually sucking order... from its environment. ...[I]t's a wonderful book. ...[H]e said, "If I had been catering for physicists alone I should have let the discussion turn on free energy instead." ...In more modern terms he's saying something like life is the harnessing of in such a way that the energy-harnessing device makes a copy of itself. ...[H]e's linking the two key themes of biology ...information and energy together."

"The new information technologies can be seen to drive societies toward increasingly dynamic high-energy regions further and further from thermodynamical equilibrium, characterized by decreasing specific entropy and increasingly dense free-energy flows, accessed and processed by more and more complex social, economic, and political structures."

"There is nothing supernatural about the process of to states of higher entropy; it is a general property of systems, regardless of their materials and origin. It does not violate the Second Law of thermodynamics since the decrease in entropy within an open system is always offset by the increase of entropy in its surroundings."

"After the invention of the steam-engine... by James Watt, the attention of engineers and of scientific men was directed to... its further improvement. ...Sadi Carnot, in 1824, published Réflexions sur la Puissance Motrice du Feu... [which] examined the relations between and the work done by heat used in an ideal engine, and by reducing the problem to its simplest form and avoiding...questions relating to details, he succeeded in establishing the conditions upon which the economical working of all heat-engines depends. ...Though the proof was invalid, the proposition remained true... Carnot's memoir remained for a long time unappreciated, and it was not until use was made of it by William Thomson... in 1848, to establish an absolute scale of temperature, that the merits of the method proposed in it were recognized. ...[H]e found that Carnot's proposition could no longer be proved by denying the possibility of "the ," and was led to lay down a second fundamental principle... now called the Second Law of Thermodynamics. ...It was published in March, 1851. In the previous year Clausias published a discussion of the same question... in which he lays down a principle for use in the demonstration of Carnot’s proposition, which, while not the same in form as Thomson’s, is the same in content, and ranks as another statement of the Second Law of Thermodynamics. Clausius followed up this paper by others, and subsequently published a book in which the subject of Thermodynamics was given a systematic treatment, and in which he introduced and developed the important function called by him the ."

"The most common way to describe entropy is as disorder... associated with things becoming more mixed, random and less ordered, but... the best way to think about entropy is as the tendency of energy to spread out. ...Most of the laws of physics work... the same... forwards or backwards in time. ...So how does this clear time dependence arise? ...[T]his is where Ludwig Boltzmann made an important insight. Heat flowing from cold to hot is not impossible, it's just improbable. ...In everyday solids there are about 100 trillion trillion atoms and even more energy packets, so heat flowing from cold to hot is just so unlikely that it never happens. ...[I]f the ...tendency is to spread out and for things to get messier, then how is it possible to have ...air conditioning, where the cold interior gets cooler and the hot exterior gets hotter? Energy is going from cold to hot, decreasing the entropy of the house. ...[T]his ...is only possible by increasing the entropy a greater amount ...at a power plant ...heating up the environment ...and creating waste heat in the fans and compressor [of the air conditioner]. ...How is there any structure left on earth? ...[I]f the earth were a the energy would spread out completely, meaning all life would cease, everything would decay and mix, and ...reach the same temperature. But luckily the earth is not a closed system, because we have the sun."

"You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, no one really knows what entropy really is, so in a debate you will always have the advantage."

"The equations of Newtonian mechanics are reversible in time and Poincaré proved that if a mechanical system is in a given state it will return infinitely often to a state arbitrarily close to the given one. Zermelo deduced that the Second Law of Thermodynamics is impossible in a mechanical system. Boltzmann asserted that entropy increases almost always, rather than always. However he believed that Poincaré's result, although correct in theory, was in practice impossible to observe since the time before a system returns to near its original state was too long."

"Let's talk some energy transfer principles. ...My Grandpappy always used to say "hot goes to cold." ...Things of a higher energy intensity state tend to equalize with things at a lower intensity energy state. ...Where there's differences, things tend towards equilibrium... You put a ball on top of a hill and you give it a chance to roll down the hill, that's what's going to happen... If you leave a big pile of sand outside long enough, it's going to flatten out. ...You take an ice cube and hold it in your hand ...heat goes out of your hand and melts the ice cube until that water becomes the same temperature as your hand. ...[V]oltage tends toward equilibrium ...If you have this ...high voltage [or current] ...stored in a battery ...[T]ake a wire and hook it from one side to the other ...It's going to equalize ...and the battery's going to be dead ..."

"Energy in the universe is constant and it can't be destroyed or created. [1st Law of Thermodynamics] ...You've got what's you've got. You've got entropy, which is a state of disorder, so things tend [from] order to disorder, but you're not going to destroy [the energy]. • Energy goes from organized and usable to disorganized and unusable, and it seeks equilibrium. That's the 2nd law and discusses entropy, which is just decay and disorder and death and destruction and... all that good stuff! All the happy stuff! When you get to be my age, you kind of look forward to it so it's not necessarily that bad. • Molecular motion stops, as does entropy, at . [3rd law] So when you get to absolute zero... nothing moves, which is why we can't really get there... because... we always take heat out of things by putting it into something else [at a lower temperature]... • Hot goes to cold. (Energy moves from higher temperature to lower temperature.) • High voltage goes to lower voltage. (Electrical current moves from high potential to low potential.) • High pressure goes to low pressure."

"Investigations of the entropy of substances at low temperatures have produced very important information regarding the structure of crystals, the work of Giauque and his collaborators being particularly noteworthy. For example, the observed entropy of crystalline hydrogen shows that even at very low temperatures the molecules of orthohydrogen in the crystal are rotating about as freely as in the gas; ... subsequent to this discovery the phenomenon of rotation of molecules in crystals was found to be not uncommon."

"Use "entropy" and you can never lose a debate, von Neumann told Shannon - because no one really knows what "entropy" is."

"My colleague Paul Glansdorff and I have investigated the problem as to if the results of near-equilibrium can be extrapolated to those of far - from-equilibrium situations and have arrived at a surprising conclusion: Contrary to what happens at equilibrium, or near equilibrium, systems far from equilibrium do not conform to any minimum principle that is valid for functions of free energy or entropy production."

"The functional order maintained within living systems seems to defy the Second Law; nonequilibrium thermodynamics describes how such systems come to terms with entropy."

"In an isolated system, which cannot exchange energy and matter with the surroundings, this tendency is expressed in terms of a function of the macroscopic state of the system: the entropy."

"Entropy is the price of structure."

"As far as we know, entropy increases throughout the portion of the universe observable from Earth. It does not seem probable to us, but in any case nothing excludes, that beyond the particle horizon which marks the maximum limit of observations there exist regions in which the arrow of time is reversed compared to ours and in which entropy decreases. I dare not think of the theoretical and observational complications that would arise if the matter contained in one of these anomalous regions began to interact with ours."

"My greatest concern was what to call it. I thought of calling it 'information,' but the word was overly used, so I decided to call it 'uncertainty.' When I discussed it with John von Neumann, he had a better idea. Von Neumann told me, 'You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, no one really knows what entropy really is, so in a debate you will always have the advantage.'"

"Prigogine was also concerned with the broader philosophical issues raised by his work. In the 19th century the discovery of the second law of thermodynamics, with its prediction of a relentless movement of the universe toward a state of maximum entropy, generated a pessimistic attitude about nature and science. Prigogine felt that his discovery of self-organizing systems constituted a more optimistic interpretation of the consequences of thermodynamics. In addition, his work led to a new view of the role of time in the physical sciences."

"Entropy... we shall use this property in a specific and limited manner. ...The following are two implications of this property: 1. If a gas or vapor is compressed or expanded frictionlessly without adding or removing heat during the process, the entropy of the substance remains constant. 2. In the process implied in implication 1, the change in represents the amount of work per unit mass required by the compression or delivered by the expansion. Possibly the greatest possible use we shall have for entropy is to read lines of constant entropy on graphs in computing the work of compression in cycles."

"Why is entropy at the beginning of time so low, and the entropy in a black hole so high? ...We ...don't know that the entropy was low ...We don't even know if there was a beginning of time. ...[E]ntropy ...is the physicist's measure of how messy things are, so my room ...tends to get higher and higher entropy, messier and messier. Why... eggs fall on the floor and break, and not... fly up and unbreak? People argued about that for a very long time until the shocking insight... that it was very low 13.4 billion years ago at the time when those... baby pictures of our universe were given off... the cosmic microwave background. ...So somehow, our flow of time towards greater messiness has something to do with our origin of our universe? That... we have learned. ...But now the question of why was that is something where many of my colleagues disagree violently... I have written a paper... which... has very little support... anyway, ...if you take seriously the idea of inflation and also this theory that the does not collapse, according to Hugh Everett, you can do some math and get an explanation... but... it's a wonderful mystery, and I'm open to all ideas... and black holes... is something else we know very little... ultimately where there are great truths yet to be discovered."

"One could... safely declare that 'Physics... can be defined as that subject which treats of the transformation of energy.' The philosophical version of Herakleitos and Empedokles... a continual cycle of changes and exchanges, had... crystallized into a quantitative physical theory. But this... picture... was... incomplete. For... there was a second, equally general and fundamental element in Nature—a directional one. This had first been formulated in the 1820s by the Mozart of modern physics, Sadi Carnot. ...Carnot started with the question: What proportion of the in any system is 'available' as a means of producing ? ...Carnot demonstrated ...a one-hundred-per-cent-efficient engine could exploit only a fraction of the heat supplied to it... A 'super-efficient' machine which could exploit all the heat supplied, would be (as Carnot's mathematics proved) a machine... one could get out of it more energy than was supplied... In an ... physical changes could at most be perfectly reversible; [but] in normal cases they would result in the progressive... 'degradation' of mechanical energy by the production of unavailable heat. To characterize this... Clausius coined the word ... [T]he directional principle of Carnot and Clausias (which gave precise expression to Newton's insight that 'motion is more easily lost than got, and is continually upon the decrease') became the Second Law of Thermodynamics."

"The third model regards mind as an information processing system. This is the model of mind subscribed to by cognitive psychologists and also to some extent by the ego psychologists. Since an acquisition of information entails maximization of negative entropy and complexity, this model of mind assumes mind to be an open system."

"It is my thesis that the physical functioning of the living individual and the operation of some of the newer communication machines are precisely parallel in their analogous attempts to control entropy through . Both of them have sensory receptors as one stage in their cycle of operation: that is, in both of them there exists a special apparatus for collecting information from the outer world at low energy levels, and for making it available in the operation of the individual or of the machine. In both cases these external messages are not taken neat, but through the internal transforming powers of the apparatus, whether it be alive or dead. The information is then turned into a new form available for the further stages of performance. In both the animal and the machine this performance is made to be effective on the outer world. In both of them, their performed action on the outer world, and not merely their intended action, is reported back to the central regulatory apparatus. This complex of behavior is ignored by the average man, and in particular does not play the role that it should in our habitual analysis of society; for just as individual physical responses may be seen from this point of view, so may the organic responses of society itself. I do not mean that the sociologist is unaware of the existence and complex nature of communications in society, but until recently he has tended to overlook the extent to which they are the cement which binds its fabric together."

"Progress imposes not only new possibilities for the future but new restrictions. It seems almost as if progress itself and our fight against the increase of entropy intrinsically must end in the downhill path from which we are trying to escape."

"He sat in the window thinking. Man has a for order. Keys in one pocket, change in another. Mandolins are tuned G D A E. The physical world has a tropism for disorder, entropy. Man against Nature . . . the battle of the centuries. Keys yearn to mix with change. Mandolins strive to get out of tune. Every order has within it the germ of destruction. All order is doomed, yet the battle is worth while."

"Revolution is everywhere, in everything. It is infinite. There is no final revolution, no final number. The social revolution is only one of an infinite number of numbers: the law of revolution is not a social law, but an immeasurably greater one. It is a cosmic, universal law—like the laws of the and of the dissipation of energy (entropy). Some day, an exact formula for the law of revolution will be established. And in this formula, nations, classes, stars—and books—will be expressed as numerical quantities."

"[M]y previous studies on the second law of thermodynamics served me here... in that my first impulse was to bring not the temperature but the entropy of the resonator into relation with its energy, more accurately not the entropy itself but its second derivative with respect to the energy... [T]his differential coefficient... has a direct physical significance for the irreversibility of the exchange of energy between the resonator and the radiation."

"But as I was... too much devoted to pure phenomenology to inquire more closely into the relation between entropy and probability, I felt compelled to limit myself to the available experimental results. Now, at that time... 1899, interest was centred on the law of the distribution of energy... proposed by W. Wien... On calculating the relation following from this law between the entropy and energy of a resonator the remarkable result is obtained that the reciprocal value of the above differential coeffcient... R, is proportional to the energy. This extremely simple relation can be regarded as an adequate expression of Wien's law..."

"I was... occupied with the task of giving it a real physical meaning, and this... led me, along Boltzmann's line... to the consideration of the relation between entropy and probability... after some weeks of the most intense work of my life clearness began to dawn... and an unexpected view revealed itself..."

"The choice (or accident) of s creates a sense of time directionality in a physical environment. The 'arrow' of entropy increase is a reflection of the improbability of those initial conditions which are entropy-decreasing in a closed physical system. ... Everywhere... in the Universe, we discern that closed physical systems evolve in the same sense from ordered states towards a state of complete disorder called . This cannot be a consequence of known laws of change, since... these laws are time symmetric—they permit... time-reverse... The initial conditions play a decisive role in endowing the world with its sense of temporal direction. ...some prescription for initial conditions is crucial if we are to understand... A Theory of Everything needs to be complemented by some such independent prescription which appeals to simplicity, economy, or some other equally metaphysical notion to underpin its credibility. The only radically different alternative... a belief that the type of mathematical description of Nature... —that of causal equations with starting conditions—is just an artefact of our own preferred categories of thought and merely an approximation... At a deeper level, a sharp divide between those aspects of reality that we habitually call 'laws' and... 'initial conditions' may simply not exist."