Entropy and Telecommunications Systems

The concept of entropy which first arose in the determination of the maximum efficiency attainable by heat engines in the early 19th century has turned out to have ramifications that extend far beyond its original domain of application. It is no exaggeration to claim that nearly every branch of science, engineering, and even the social sciences has been touched by entropy. Entropy has also enriched the field of Statistics where it has augmented traditional estimation methods such as least squares and maximum likelihood by a new methodology called the Maximum Entropy approach. While energy conservation is a fundamental law in Physics, it is not sufficient on its own to predict whether a physical process can occur spontaneously in nature, for there are many processes that would be permitted because they do not violate conservation of energy, but which cannot spontaneously occur in nature.

Historical Perspective

The roots of entropy lie in the field of thermodynamics where it arises in the analysis of the efficiency of heat engines, originating early in the 19th century. However, it was not till the 20th century that entropy began to appear as a fundamental tool of analysis in branch after branch of the sciences and engineering. As early as 1803, Lazare Carnot initiated the analysis of the efficiency of fundamental engines such as pulleys and inclined planes. This work eventually led to the idea of transformation-energy or energy lost to dissipation and friction. It is the transformation-energy idea that is now called entropy.

Implications of Entropy for Telecommunications

Entropy is a surprisingly deep and subtle concept and for that reason, perhaps there are more misconceptions and incorrect interpretations of entropy than any other physical concept. It is intimately connected to the second law of thermodynamics which can be stated in many forms, one of which is that heat always flows spontaneously from a hot object to a cold one. An equivalent version of the second law, applicable to those physical phenomena in which no heat flow takes place, is that the entropy of an isolated system never decreases. Since the universe in its entirety is an isolated system,this implies that the entropy of the universe is increasing. Returning to the heat flow example, note that the flow happens only when there is a temperature gradient-as the heat flows from the hot body to the cold, the temperature of the former drops and that of the latter increases until the two are at the same temperature. At that point, the two bodies are in thermal equilibrium and no more flow takes place. As another example, the energy content of a lake cannot be harnessed to produce useful work.