The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Properties of cross-entropy minimization
Abstract
The principle of minimum cross-entropy (minimum directed divergence, minimum discrimination information) is a general method of inference about an unknown probability density when there exists a prior estimate of the density and new information in the form of constraints on expected values. Various fundamental properties of cross-entropy minimization are proven and collected in one place. Cross-entropy's well-known properties as an information measure are extended and strengthened when one of the densities involved is the result of cross-entropy minimization. The interplay between properties of cross-entropy minimization as an inference procedure and properties of cross-entropy as an information measure is pointed out. Examples are included and general analytic and computational methods of finding minimum cross-entropy probability densities are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Information theory and statistical mechanics
Jaynes, E. T. (1957) · Physical Review · doi:10.1103/physrev.106.620 · Cited by 12,931
Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy
Shore, J., & Johnson, R. (1980) · IEEE Transactions on Information Theory · doi:10.1109/tit.1980.1056144 · Cited by 1,774
Self-citation$I$-Divergence Geometry of Probability Distributions and Minimization Problems
Csiszar, I. (1975) · The Annals of Probability · doi:10.1214/aop/1176996454 · Cited by 1,645
Information Theory and Statistics.
Dwass, M., & Kullback, S. (1960) · The American Mathematical Monthly · doi:10.2307/2309734 · Cited by 1,537
Markel, J. D., & Gray, A. H. (1976) · Communication and Cybernetics · doi:10.1007/978-3-642-66286-7 · Cited by 1,402
Regression and the Moore-Penrose Pseudoinverse
(1972) · Mathematics in Science and Engineering · doi:10.1016/s0076-5392(08)x6167-3 · Cited by 718
Speech coding based upon vector quantization
Buzo, A., Gray, A., Gray, R., & Markel, J. (1980) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1980.1163445 · Cited by 559
Rational Descriptions, Decisions and Designs
(1969) · Elsevier eBooks · doi:10.1016/c2013-0-01558-7 · Cited by 470
Distortion measures for speech processing
Gray, R., Buzo, A., Gray, A., & Matsuyama, Y. (1980) · IEEE Transactions on Acoustics, Speech, and Signal Processing · doi:10.1109/tassp.1980.1163421 · Cited by 388
An algorithm for finding the distribution of maximal entropy
Agmon, N., Alhassid, Y., & Levine, R. D. (1979) · Journal of Computational Physics · doi:10.1016/0021-9991(79)90102-5 · Cited by 244
Cited by
Principles of maximum entropy and maximum caliber in statistical physics
Pressé, S., Ghosh, K., Lee, J., & Dill, K. A. (2013) · Reviews of Modern Physics · doi:10.1103/revmodphys.85.1115 · Cited by 413
An iterative multiuser decoder for near-capacity communications
Moher, M. (1998) · IEEE Transactions on Communications · doi:10.1109/26.701309 · Cited by 360
Decomposing information into copying versus transformation
Kolchinsky, A., & Corominas-Murtra, B. (2020) · Journal of The Royal Society Interface · doi:10.1098/rsif.2019.0623 · Cited by 5
Metadata
- Authors
- Shore, J., & Johnson, R.
- Venue
- IEEE Transactions on Information Theory
- Published
- 1981-07
- Volume
- 27
- Issue
- 4
- Pages
- 472-482
- ISSN
- 0018-9448 (ISSN-L)
- 1557-9654
- OpenAlex
- W2068375500
- Cited by
- 394
- References
- 16
- FWCI
- 7.10
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Properties of cross-entropy minimization. https://app.cochat.ai/papers/10.1109/tit.1981.1056373
- The paper
- Shore, J., & Johnson, R. (1981). Properties of cross-entropy minimization. IEEE Transactions on Information Theory. https://doi.org/10.1109/tit.1981.1056373
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.