The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Minimum Message Length and Kolmogorov Complexity
Abstract
The notion of algorithmic complexity was developed by Kolmogorov (1965) and Chaitin (1966) independently of one another and of Solomonoff's notion (1964) of algorithmic probability. Given a Turing machine T, the (prefix) algorithmic complexity of a string S is the length of the shortest input to T which would cause T to output S and stop. The Solomonoff probability of S given T is the probability that a random binary string of 0s and 1s will result in T producing an output having S as a prefix. We attempt to establish a parallel between a restricted (two-part) version of the Kolmogorov model and the minimum message length approach to statistical inference and machine learning of Wallace and Boulton (1968), in which an ‘explanation’ of a data string is modelled as a two-part message, the first part stating a general hypothesis about the data and the second encoding details of the data not implied by the hypothesis. Solomonoff's model is tailored to prediction rather than inference in that it considers not just the most likely explanation, but it also gives weights to all explanations depending upon their posterior probability. However, as the amount of data increases, we typically expect the most likely explanation to have a dominant weighting in the prediction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Modeling by shortest data description
Rissanen, J. (1978) · Automatica · doi:10.1016/0005-1098(78)90005-5 · Cited by 6,011
An Introduction to Kolmogorov Complexity and Its Applications
Li, M., & Vitányi, P. (2019) · Texts in Computer Science · doi:10.1007/978-3-030-11298-1 · Cited by 3,681
Three approaches to the quantitative definition of information*
Kolmogorov, A. N. (1968) · International Journal of Computer Mathematics · doi:10.1080/00207166808803030 · Cited by 2,031
A formal theory of inductive inference. Part II
Solomonoff, R. J. (1964) · Information and Control · doi:10.1016/s0019-9958(64)90131-7 · Cited by 1,758
Stochastic Complexity in Statistical Inquiry
Rissanen, J. (1998) · WORLD SCIENTIFIC eBooks · doi:10.1142/0822 · Cited by 1,539
An Information Measure for Classification
Wallace, C. S., & Boulton, D. M. (1968) · The Computer Journal · doi:10.1093/comjnl/11.2.185 · Cited by 933
Self-citationOn the Length of Programs for Computing Finite Binary Sequences: statistical considerations
Chaitin, G. J. (1969) · Journal of the ACM · doi:10.1145/321495.321506 · Cited by 918
Estimation and Inference by Compact Coding
Wallace, C. S., & Freeman, P. R. (1987) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1987.tb01695.x · Cited by 523
Self-citation1. Bayesian Statistics a Review
Lindley, D. V. (1972) · Bayesian Statistics · doi:10.1137/1.9781611970654.ch1 · Cited by 521
Minimum complexity density estimation
Barron, A. R., & Cover, T. M. (1991) · IEEE Transactions on Information Theory · doi:10.1109/18.86996 · Cited by 497
Cited by
Unsupervised learning of finite mixture models
Figueiredo, M. A. T., & Jain, A. K. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.990138 · Cited by 2,125
Theory of Decision under Uncertainty
Gilboa, I. (2009) · Cambridge University Press eBooks · doi:10.1017/cbo9780511840203 · Cited by 600
An efficient k′-means clustering algorithm
Žalik, K. R. (2008) · Pattern Recognition Letters · doi:10.1016/j.patrec.2008.02.014 · Cited by 370
Answering Schrödinger's question: A free-energy formulation
Ramstead, M. J. D., Badcock, P. B., & Friston, K. J. (2018) · Physics of Life Reviews · doi:10.1016/j.plrev.2017.09.001 · Cited by 354
Active Inference, Curiosity and Insight
Friston, K. J., Lin, M., Frith, C. D., Pezzulo, G., Hobson, J. A., & Ondobaka, S. (2017) · Neural Computation · doi:10.1162/neco_a_00999 · Cited by 351
Metadata
- Authors
- Wallace, C. S.
- Venue
- The Computer Journal
- Publisher
- Oxford University Press (OUP)
- Published
- 1999-04-01
- Volume
- 42
- Issue
- 4
- Pages
- 270-283
- ISSN
- 0010-4620 (ISSN-L)
- 1460-2067
- OpenAlex
- W1997404989
- Cited by
- 347
- References
- 14
- FWCI
- 11.43
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Minimum Message Length and Kolmogorov Complexity. https://app.cochat.ai/papers/10.1093/comjnl/42.4.270
- The paper
- Wallace, C. S. (1999). Minimum Message Length and Kolmogorov Complexity. The Computer Journal. https://doi.org/10.1093/comjnl/42.4.270
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.