The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
The minimum description length principle in coding and modeling
Abstract
We review the principles of minimum description length and stochastic complexity as used in data compression and statistical modeling. Stochastic complexity is formulated as the solution to optimum universal coding problems extending Shannon's basic source coding theorem. The normalized maximized likelihood, mixture, and predictive codings are each shown to achieve the stochastic complexity to within asymptotically vanishing terms. We assess the performance of the minimum description length criterion both from the vantage point of quality of data compression and accuracy of statistical inference. Context tree modeling, density estimation, and model selection in Gaussian linear regression serve as examples.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Estimating the Dimension of a Model
Schwarz, G. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344136 · Cited by 47,033
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 36,808
Information Theory and an Extension of the Maximum Likelihood Principle
Akaike, H. (1998) · Springer Series in Statistics · doi:10.1007/978-1-4612-1694-0_15 · Cited by 17,781
Doob, J. L., & Doob, J. L. (1953) · CERN Document Server (European Organization for Nuclear Research) · Cited by 9,053
Modeling by shortest data description
Rissanen, J. (1978) · Automatica · doi:10.1016/0005-1098(78)90005-5 · Cited by 6,011
Self-citationOn the mathematical foundations of theoretical statistics
Fisher, R. A. (1922) · Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character · doi:10.1098/rsta.1922.0009 · Cited by 4,039
The Determination of the Order of an Autoregression
Hannan, E. J., & Quinn, B. G. (1979) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1979.tb01072.x · Cited by 2,992
Three approaches to the quantitative definition of information*
Kolmogorov, A. N. (1968) · International Journal of Computer Mathematics · doi:10.1080/00207166808803030 · Cited by 2,031
A formal theory of inductive inference. Part II
Solomonoff, R. J. (1964) · Information and Control · doi:10.1016/s0019-9958(64)90131-7 · Cited by 1,758
A formal theory of inductive inference. Part I
Solomonoff, R. J. (1964) · Information and Control · doi:10.1016/s0019-9958(64)90223-2 · Cited by 1,192
Cited by
Statistical pattern recognition: a review
Jain, A. K., Duin, P. W., & Mao, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.824819 · Cited by 6,767
Prediction, Learning, and Games
Cesa-Bianchi, N., & Lugosi, G. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511546921 · Cited by 3,162
Martinez, A. M. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1008382 · Cited by 813
An information-theoretic framework for resolving community structure in complex networks
Rosvall, M., & Bergstrom, C. T. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0611034104 · Cited by 808
Ephraim, Y., & Merhav, N. (2002) · IEEE Transactions on Information Theory · doi:10.1109/tit.2002.1003838 · Cited by 798
Model Selection and the Principle of Minimum Description Length
Hansen, M. H., & Yu, B. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753168398 · Cited by 788
Self-citationInformativeness of Genetic Markers for Inference of Ancestry*
Rosenberg, N. A., Li, L. M., Ward, R., & Pritchard, J. K. (2003) · The American Journal of Human Genetics · doi:10.1086/380416 · Cited by 719
MIMO Radar for Advanced Driver-Assistance Systems and Autonomous Driving: Advantages and Challenges
Sun, S., Petropulu, A. P., & Poor, H. V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2978507 · Cited by 632
Reliable communication under channel uncertainty
Lapidoth, A., & Narayan, P. (1998) · IEEE Transactions on Information Theory · doi:10.1109/18.720535 · Cited by 519
Segmentation of Multivariate Mixed Data via Lossy Data Coding and Compression
Ma, Y., Derksen, H., Hong, W., & Wright, J. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1085 · Cited by 481
On the nature and use of models in network neuroscience
Bassett, D. S., Zurn, P., & Gold, J. I. (2018) · Nature Reviews Neuroscience · doi:10.1038/s41583-018-0038-8 · Cited by 385
Model Selection Techniques: An Overview
Ding, J., Tarokh, V., & Yang, Y. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2018.2867638 · Cited by 383
Freimer, N., & Sabatti, C. (2003) · Nature Genetics · doi:10.1038/ng0503-15 · Cited by 372
Saez‐Rodriguez, J., Alexopoulos, L. G., Epperlein, J., Samaga, R., Lauffenburger, D. A., Klamt, S., & Sorger, P. K. (2009) · Molecular Systems Biology · doi:10.1038/msb.2009.87 · Cited by 351
Elements of Information Theory
Yang, Y. (2008) · Journal of the American Statistical Association · doi:10.1198/jasa.2008.s218 · Cited by 339
Sparse Modeling for Image and Vision Processing
Mairal, J. (2014) · now publishers, Inc. eBooks · doi:10.1561/9781680830095 · Cited by 322
Verdu, S. (1998) · IEEE Transactions on Information Theory · doi:10.1109/18.720531 · Cited by 316
Yang, Y. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753168262 · Cited by 312
A tutorial introduction to the minimum description length principle
Grunwald, P. (2004) · arXiv · doi:10.48550/arxiv.math/0406077 · Cited by 293
Gaussian Processes for Machine Learning
Ounpraseuth, S. T. (2008) · Journal of the American Statistical Association · doi:10.1198/jasa.2008.s219 · Cited by 271
Human inference reflects a normative balance of complexity and accuracy
Tavoni, G., Doi, T., Pizzica, C., Balasubramanian, V., & Gold, J. I. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01357-z · Cited by 35
Metadata
- Authors
- Barron, A., Rissanen, J., & Yu, B.
- Venue
- IEEE Transactions on Information Theory
- Published
- 1998
- Volume
- 44
- Issue
- 6
- Pages
- 2743-2760
- ISSN
- 0018-9448 (ISSN-L)
- OpenAlex
- W2102098892
- Cited by
- 1,019
- References
- 42
- FWCI
- 23.63
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.