The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Universal noiseless coding
Abstract
Universal coding is any asymptotically optimum method of block-to-block memoryless source coding for sources with unknown parameters. This paper considers noiseless coding for such sources, primarily in terms of variable-length coding, with performance measured as a function of the coding redundancy relative to the per-letter conditional source entropy given the unknown parameter. It is found that universal (i.e., zero redundancy) coding in a weighted sense is possible if and only if the per-letter average mutual information between the parameter space and the message space is zero. Universal coding is possible in a maximin sense if and only if the channel capacity between the two spaces is zero. Universal coding is possible in a minimax sense if and only if a probability mass function exists, independent of the unknown parameter, for which the relative entropy of the known conditional-probability mass-function is zero. Several examples are given to illustrate the ideas. Particular attention is given to sources that are stationary and ergodic for any fixed parameter although the whole ensemble is not. For such sources, weighted universal codes always exist if the alphabet is finite, or more generally if the entropy is finite. Minimax universal codes result if an additional entropy stability constraint is applied. A discussion of fixed-rate universal coding is also given briefly with performance measured by a probability of error.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Three approaches to the quantitative definition of information*
Kolmogorov, A. N. (1968) · International Journal of Computer Mathematics · doi:10.1080/00207166808803030 · Cited by 2,031
Speech Analysis and Synthesis by Linear Prediction of the Speech Wave
Atal, B. S., & Hanauer, S. L. (1971) · The Journal of the Acoustical Society of America · doi:10.1121/1.1912679 · Cited by 1,209
Rate distortion theory : a mathematical basis for data compression
Berger, T. (1971) · Prentice Hall eBooks · Cited by 953
Rate Distortion Theory: A Mathematical Basis for Data Compression
Davisson, L. (1972) · IEEE Transactions on Communications · doi:10.1109/tcom.1972.1091311 · Cited by 895
Self-citationCover, T. (1973) · IEEE Transactions on Information Theory · doi:10.1109/tit.1973.1054929 · Cited by 504
Coding of sources with unknown statistics--I: Probability of encoding error
Ziv, J. (1972) · IEEE Transactions on Information Theory · doi:10.1109/tit.1972.1054830 · Cited by 38
Cited by
A universal algorithm for sequential data compression
Ziv, J., & Lempel, A. (1977) · IEEE Transactions on Information Theory · doi:10.1109/tit.1977.1055714 · Cited by 5,456
Universal codeword sets and representations of the integers
Elias, P. (1975) · IEEE Transactions on Information Theory · doi:10.1109/tit.1975.1055349 · Cited by 1,165
The minimum description length principle in coding and modeling
Barron, A., Rissanen, J., & Yu, B. (1998) · IEEE Transactions on Information Theory · doi:10.1109/18.720554 · Cited by 1,019
Universal coding, information, prediction, and estimation
Rissanen, J. (1984) · IEEE Transactions on Information Theory · doi:10.1109/tit.1984.1056936 · Cited by 898
Fisher information and stochastic complexity
Rissanen, J. J. (1996) · IEEE Transactions on Information Theory · doi:10.1109/18.481776 · Cited by 814
Ephraim, Y., & Merhav, N. (2002) · IEEE Transactions on Information Theory · doi:10.1109/tit.2002.1003838 · Cited by 798
Model Selection and the Principle of Minimum Description Length
Hansen, M. H., & Yu, B. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753168398 · Cited by 788
Recent results in the Shannon theory
Wyner, A. (1974) · IEEE Transactions on Information Theory · doi:10.1109/tit.1974.1055171 · Cited by 633
Information Theory and Statistics: A Tutorial
Csiszár, I., & Shields, P. C. (2004) · now publishers, Inc. eBooks · doi:10.1561/9781933019543 · Cited by 589
Information-theoretic determination of minimax rates of convergence
Yang, Y., & Barron, A. (1999) · The Annals of Statistics · doi:10.1214/aos/1017939142 · Cited by 555
The performance of universal encoding
Krichevsky, R., & Trofimov, V. (1981) · IEEE Transactions on Information Theory · doi:10.1109/tit.1981.1056331 · Cited by 535
A First Course in Information Theory
Yeung, R. W. (2002) · Information Technology: Transmission, Processing and Storage · doi:10.1007/978-1-4419-8608-5 · Cited by 533
Minimum complexity density estimation
Barron, A. R., & Cover, T. M. (1991) · IEEE Transactions on Information Theory · doi:10.1109/18.86996 · Cited by 497
Merhav, N., & Feder, M. (1998) · IEEE Transactions on Information Theory · doi:10.1109/18.720534 · Cited by 438
Information-theoretic asymptotics of Bayes methods
Clarke, B. S., & Barron, A. R. (1990) · IEEE Transactions on Information Theory · doi:10.1109/18.54897 · Cited by 431
Information Theory and Statistics: A Tutorial
Csiszár, I., & Shields, P. C. (2004) · Foundations and Trends™ in Communications and Information Theory · doi:10.1561/0100000004 · Cited by 420
Universal prediction of individual sequences
Feder, M., Merhav, N., & Gutman, M. (1992) · IEEE Transactions on Information Theory · doi:10.1109/18.144706 · Cited by 373
Grammar-based codes: a new class of universal lossless source codes
Kieffer, J. C., & Yang, E.-H. (2000) · IEEE Transactions on Information Theory · doi:10.1109/18.841160 · Cited by 352
Verdu, S. (1998) · IEEE Transactions on Information Theory · doi:10.1109/18.720531 · Cited by 316
Jeffreys' prior is asymptotically least favorable under entropy risk
Clarke, B. S., & Barron, A. R. (1994) · Journal of Statistical Planning and Inference · doi:10.1016/0378-3758(94)90153-8 · Cited by 292
Metadata
- Authors
- Davisson, L.
- Venue
- IEEE Transactions on Information Theory
- Published
- 1973-11
- Volume
- 19
- Issue
- 6
- Pages
- 783-795
- ISSN
- 0018-9448 (ISSN-L)
- 1557-9654
- OpenAlex
- W2131024393
- Cited by
- 365
- References
- 6
- FWCI
- 8.70
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.