The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
The minimum description length principle in coding and modeling
Abstract
We review the principles of minimum description length and stochastic complexity as used in data compression and statistical modeling. Stochastic complexity is formulated as the solution to optimum universal coding problems extending Shannon's basic source coding theorem. The normalized maximized likelihood, mixture, and predictive codings are each shown to achieve the stochastic complexity to within asymptotically vanishing terms. We assess the performance of the minimum description length criterion both from the vantage point of quality of data compression and accuracy of statistical inference. Context tree modeling, density estimation, and model selection in Gaussian linear regression serve as examples.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 36,808
Information Theory and an Extension of the Maximum Likelihood Principle
Akaike, H. (1998) · Springer Series in Statistics · doi:10.1007/978-1-4612-1694-0_15 · Cited by 17,781
Doob, J. L., & Doob, J. L. (1953) · CERN Document Server (European Organization for Nuclear Research) · Cited by 9,053
Modeling by shortest data description
Rissanen, J. (1978) · Automatica · doi:10.1016/0005-1098(78)90005-5 · Cited by 6,011
Self-citationOn the mathematical foundations of theoretical statistics
Fisher, R. A. (1922) · Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character · doi:10.1098/rsta.1922.0009 · Cited by 4,039
The Determination of the Order of an Autoregression
Hannan, E. J., & Quinn, B. G. (1979) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1979.tb01072.x · Cited by 2,992
Three approaches to the quantitative definition of information*
Kolmogorov, A. N. (1968) · International Journal of Computer Mathematics · doi:10.1080/00207166808803030 · Cited by 2,031
A formal theory of inductive inference. Part II
Solomonoff, R. J. (1964) · Information and Control · doi:10.1016/s0019-9958(64)90131-7 · Cited by 1,758
A formal theory of inductive inference. Part I
Solomonoff, R. J. (1964) · Information and Control · doi:10.1016/s0019-9958(64)90223-2 · Cited by 1,192
Dawid, A. P. (1984) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2981683 · Cited by 1,085
An Information Measure for Classification
Wallace, C. S., & Boulton, D. M. (1968) · The Computer Journal · doi:10.1093/comjnl/11.2.185 · Cited by 933
Stochastic Complexity and Modeling
Rissanen, J. (1986) · The Annals of Statistics · doi:10.1214/aos/1176350051 · Cited by 922
Self-citationThe context-tree weighting method: basic properties
Willems, F. M. J., Shtarkov, Y. M., & Tjalkens, T. J. (1995) · IEEE Transactions on Information Theory · doi:10.1109/18.382012 · Cited by 919
Universal coding, information, prediction, and estimation
Rissanen, J. (1984) · IEEE Transactions on Information Theory · doi:10.1109/tit.1984.1056936 · Cited by 898
Self-citationFisher information and stochastic complexity
Rissanen, J. J. (1996) · IEEE Transactions on Information Theory · doi:10.1109/18.481776 · Cited by 814
Self-citationA universal data compression system
Rissanen, J. (1983) · IEEE Transactions on Information Theory · doi:10.1109/tit.1983.1056741 · Cited by 631
Self-citationInformation-theoretic determination of minimax rates of convergence
Yang, Y., & Barron, A. (1999) · The Annals of Statistics · doi:10.1214/aos/1017939142 · Cited by 555
Self-citationA Course in Density Estimation.
Jones, M. C., & Devroye, L. (1988) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982771 · Cited by 503
Minimum complexity density estimation
Barron, A. R., & Cover, T. M. (1991) · IEEE Transactions on Information Theory · doi:10.1109/18.86996 · Cited by 497
Self-citationThe Estimation of the Order of an ARMA Process
Hannan, E. J. (1980) · The Annals of Statistics · doi:10.1214/aos/1176345144 · Cited by 489
Rissanen, J., & Langdon, G. (1981) · IEEE Transactions on Information Theory · doi:10.1109/tit.1981.1056282 · Cited by 454
Self-citationInformation-theoretic asymptotics of Bayes methods
Clarke, B. S., & Barron, A. R. (1990) · IEEE Transactions on Information Theory · doi:10.1109/18.54897 · Cited by 431
Self-citationOn some asymptotic properties of maximum likelihood estimates and related Bayes' estimates
Cam, L. L., & Lucien, M. (1953) · University of California Press eBooks · Cited by 388
Davisson, L. (1973) · IEEE Transactions on Information Theory · doi:10.1109/tit.1973.1055092 · Cited by 365
Yu, B. (1997) · Festschrift for Lucien Le Cam · doi:10.1007/978-1-4612-1880-7_29 · Cited by 309
Self-citationJeffreys' prior is asymptotically least favorable under entropy risk
Clarke, B. S., & Barron, A. R. (1994) · Journal of Statistical Planning and Inference · doi:10.1016/0378-3758(94)90153-8 · Cited by 292
Self-citationA universal finite memory source
Weinberger, M. J., Rissanen, J. J., & Feder, M. (1995) · IEEE Transactions on Information Theory · doi:10.1109/18.382011 · Cited by 228
Self-citationOn Predictive Least Squares Principles
Wei, C. Z. (1992) · The Annals of Statistics · doi:10.1214/aos/1176348511 · Cited by 194
Hypothesis Selection and Testing by the MDL Principle
Rissanen, J. (1999) · The Computer Journal · doi:10.1093/comjnl/42.4.260 · Cited by 159
Self-citationHartigan, J. A. (1983) · Springer series in statistics · doi:10.1007/978-1-4613-8242-3 · Cited by 151
Mutual information, metric entropy and cumulative relative entropy risk
Haussler, D. H., & Opper, M. (1997) · The Annals of Statistics · doi:10.1214/aos/1030741081 · Cited by 137
Density estimation by stochastic complexity
Rissanen, J., Speed, T. P., & Yu, B. (1992) · IEEE Transactions on Information Theory · doi:10.1109/18.119689 · Cited by 114
Self-citationA strong version of the redundancy-capacity theorem of universal coding
Merhav, N., & Feder, M. (1995) · IEEE Transactions on Information Theory · doi:10.1109/18.382017 · Cited by 101
A source matching approach to finding minimax codes
Davisson, L., & Leon‐Garcia, A. (1980) · IEEE Transactions on Information Theory · doi:10.1109/tit.1980.1056167 · Cited by 97
Model selection and prediction: Normal regression
Speed, T. P., & Yu, B. (1993) · Annals of the Institute of Statistical Mathematics · doi:10.1007/bf00773667 · Cited by 76
Self-citationPrequential Analysis, Stochastic Complexity and Bayesian Inference
Dawid, A. P. (1992) · Oxford University Press eBooks · doi:10.1093/oso/9780198522669.003.0007 · Cited by 70
Minimax noiseless universal coding for Markov sources
Davisson, L. (1983) · IEEE Transactions on Information Theory · doi:10.1109/tit.1983.1056652 · Cited by 64
Dawid, A. P. (1992) · Lecture notes-monograph series · doi:10.1214/lnms/1215458842 · Cited by 61
Model Selection and Forward Validation
Hjorth, U. (1982) · Cited by 47
Asymptotically minimax regret by Bayes mixtures
Takeuchi, J., & Barron, A. R. (2002) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.1998.708923 · Cited by 38
Self-citationData compression and histograms
Yu, B., & Speed, T. P. (1992) · Probability Theory and Related Fields · doi:10.1007/bf01194921 · Cited by 27
Self-citation
Cited by
Prediction, Learning, and Games
Cesa-Bianchi, N., & Lugosi, G. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511546921 · Cited by 3,162
Martinez, A. M. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1008382 · Cited by 813
Ephraim, Y., & Merhav, N. (2002) · IEEE Transactions on Information Theory · doi:10.1109/tit.2002.1003838 · Cited by 798
Model Selection and the Principle of Minimum Description Length
Hansen, M. H., & Yu, B. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753168398 · Cited by 788
Self-citationInformativeness of Genetic Markers for Inference of Ancestry*
Rosenberg, N. A., Li, L. M., Ward, R., & Pritchard, J. K. (2003) · The American Journal of Human Genetics · doi:10.1086/380416 · Cited by 719
Metadata
- Authors
- Barron, A., Rissanen, J., & Yu, B.
- Venue
- IEEE Transactions on Information Theory
- Published
- 1998
- Volume
- 44
- Issue
- 6
- Pages
- 2743-2760
- ISSN
- 0018-9448 (ISSN-L)
- OpenAlex
- W2102098892
- Cited by
- 1,019
- References
- 42
- FWCI
- 23.63
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.