Theory of communication. Part 1: The analysis of information

Abstract

Hitherto communication theory was based on two alternative methods of signal analysis. One is the description of the signal as a function of time; the other is Fourier analysis. Both are idealizations, as the first method operates with sharply defined instants of time, the second with infinite wave-trains of rigorously defined frequencies. But our everyday experiences—especially our auditory sensations—insist on a description in terms of both time and frequency. In the present paper this point of view is developed in quantitative language. Signals are represented in two dimensions, with time and frequency as co-ordinates. Such two-dimensional representations can be called “information diagrams,” as areas in them are proportional to the number of independent data which they can convey. This is a consequence of the fact that the frequency of a signal which is not of infinite duration can be defined only with a certain inaccuracy, which is inversely proportional to the duration, and vice versa. This “uncertainty relation” suggests a new method of description, intermediate between the two extremes of time analysis and spectral analysis. There are certain “elementary signals” which occupy the smallest possible area in the information diagram. They are harmonic oscillations modulated by a “probability pulse.” Each elementary signal can be considered as conveying exactly one datum, or one “quantum of information.” Any signal can be expanded in terms of these by a process which includes time analysis and Fourier analysis as extreme cases.These new methods of analysis, which involve some of the mathematical apparatus of quantum theory, are illustrated by application to some problems of transmission theory, such as direct generation of single sidebands, signals transmitted in minimum time through limited frequency channels, frequency modulation and time-division multiplex telephony.