The 2nd Law is a tautology

This is a cute idea I had, which I thought was a brilliant innovation, until I read a paper by Stephen Hawking that said almost exactly the same thing. Anyhoo, here it is from my conception:

The word "time" used in stat-mech is used erroneously. What they actually mean when they say "the entropy increases with time" is "entropy increases with CLOCK TICKS" ; these are not the same. (Of course, by 'increases' I really mean 'does not decrease', and when I say 'the entropy' I mean the entropy of a closed/and or isolated system).

First, let us get an intuition for entropy. Entropy's increase means that work can be extracted from the system; we can see this easily, since the entropy of a system can only decrease if work is done on it. Work is extracted by allowing few degrees of freedom to become many. Thus entropy's increase with 'time' is a measure of how much 'work' can be extracted as the environment changes.

So, why does the second law have to mean 'clock ticks' not 'time' ? Because it only applies to closed systems, and in a truly closed system THERE IS NO TIME! That is, the energy is constant, so quantum mechanically, we known we cannot distinguish anything about the time (by the [E,t] uncertainty relation). Thus, the only way to have a 2nd law is to include a clock in your system. The 2nd law does not apply to a "closed" beaker, but it does apply to a closed beaker with a "perfect" clock stuck in it. (we neglect the fact that an observer might need to be included as well).

Now I will show that the 2nd law is a tautology. What does a clock measure? Not the ticks of 'time'!! Instead, a clock measures the increase of entropy! How can we see this? Because if entropy were not increasing, the clock would stop : it takes work to make a clock tick, and work can only be extracted from a system for which the entropy is increasing. (you can see this in the real world, because a clock runs on a battery, or perhaps by the decay of a radioactive element if you have a really fancy clock, but either way the increase of entropy is inherent in the clock's operation). Thus when we put the clock in a closed system, it will only tick if entropy is increasing. Thus the statement, "entropy increases with clock ticks", is simply a tautology, based on the operation of clocks!


On the Energy-Basis in Stat-Mech Entropy

First of all, I take the modern physicist's viewpoint that classical
statistical mechanics is just plain wrong, so:

S = ln W

is NOT true.  Instead, I use the quantum-mechanical-density matrix
viewpoint of statistical mechanics.  The "basis" (the set of "events"
which are quantized to measure probabilities) are given to use by
quantum mechanics : they are the energy eigenstates.  If we call
each state n, with energy E_n, then the probability of occupation of
that state is taken as a postulate : p_n = exp{-B E_n}/Z , where B = 1/kT
and Z = Sum[n] exp{-B E_n} , is the normalization factor.
Then the entropy is defined to be the Shannon Entropy (here, I mean
the "order-0 character entropy", or shannon's first entropy) :

S = Sum[n] - p_n ln( p_n )

(the use of ln is physics convention; Shannon preferred log2 ; this
is just a multiplicative constant).  Thus the mathematical relationship
between "modified Boltzmann" and Shannon Entropies is manifest.  If the
energies are all equal E_n = E, then p_n = 1/W , and S = lnW , the
classical Boltzmann entropy; clearly this is only an approximate
limitting result. (see Chris Hillman's many great pages on 
cases
where the entropies are equal)

> Timothy Murphy (tim@maths.tcd.ie) says [cut]
>Which is what I have been saying, they are closely related under certain
>conditions (where they are both defined), but mathematically speaking they
>certainly are not "the same".

Well, there is a certain disagreement here; while the Chaitin/Kolmogorov
(CK) entropy is the "best" measure of complexity and information, it is NOT
in any way directly related to the statistical-mechanics entropy; we
can think of this as the CK entropy being free to make the optimal choice
of basis (eg Fourier transforming images, or block-sorting text) prior to
counting the "order-0" Shannon entropy (plogp), while in stat-mech you
are not free to change your basis - you must use the energy basis.

In this sense, the special role of "time" in the second law is equivalent
to the fact that we are forced to use the energy basis to compute our
entropies (energy & time are conjugate).  This is in turn somehow related
to the definition of "temperature" and the stat-mech weighting function
in terms of energies;  In particular, quantum stat-mech surrenders the
Similarity Transform (unitary rotation of bases) which Dirac was so
adamant in retaining.



Chris Hillman writes:

On Tue, 23 Dec 1997, Charles Bloom wrote:
> >Really?  I had no idea, I thought that departures from ergodicity in
> >natural language were hard to detect.
> 
> Right, that was the common viewpoint among all theorists - but it had
> never really been tested!  Peter Fenwick and I tested it "experimentally"
> a few years back, and found to our surprise about a 5% deviation from
> 
> H(A : B) = H(B : A)
> (or)
> >  H(next letter/current letter) < H(previous letter/current letter)
> 
> We conjecture that it has something to do with learning the model, or
> simply more difficult modelling.  There is an apparrent asymmetry in
> text if you think about it : 'q' predicts 'u' very well, because 'qu'
> is very common, however 'u' predicts 'q' backwards only poorly (tub,rub,tune,etc.).  In fact, the many rules in english ('i' before 'e'
> except AFTER 'c', etc.) are asymmetric.  While these are all in theory
> detectable backwards, it simply takes longer for the model to learn these
> things backwards, so that ergodicity is not a great approximation for
> reasonable size sources ( < 8 million bits)

Very good example.  My brain feels like its full of treacle today, so let
me spell it out.  So, q* is almost certainly qu but *u could be tu, qu,
su, etc... which would mean that the uncertainly of the next letter, given
that the current letter is q, is LESS than the uncertainty of the previous
letter, given that the current letter is u.  If this trend holds true
generally we would have

  H(next/current ) < H(previous/current)
 
as I expect in asymmetric cases.

> The experimental results are in one of Peter's papers; see
> http://www.cs.auckland.ac.nz/~peter-f/
>
> How about directly in terms of a numerical source.  Imagine that in the
> Ath position we can have one of four values : (0,10,20,30).  Each of
> these deterministically predicts 1/10 of itself for the next position (B):
> (0,1,2,3).  Thus the Ath and Bth bytes will be :
> 
> A		 B
> (0,10,20,30) ; (0,1,2,3)
> 
> Here we have perfect symmetry : given A we predict B = A/10 , and given B
> we predict A = 10 B.
> But now, what if we coarse grain or smear, so that precision of less than
> 5 is lost, so that 0,1,2,3->0 , thus the observer/receiver apparently sees:
> 
> A		 B
> (0,10,20,30) ; (0,0,0,0)
>
> Now we have asymmetry.  Given A we predict B = 0, but given B, we can say
> nothing about A; the properties of B which allowed us to distinguish the
> A's has been destroyed, so B now gives no information about A.

Hmm... the trouble with your example is that this isn't a process (only
two times instead of an integer sequence of times) or information source,
even the alphabets are different at times 1 and 2. 

Can you come up with an example which is process?

I wonder if this has anything to do with the fact that quotients of a
Markov process are usually not Markov processes.


My the 2nd law and info-theory article.


Charles Bloom / cb at my domain
Send Me Email

Back to the Physics Index

The free web counter says you are visitor number