The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Entropy estimation of symbol sequences
Open accessAbstract
We discuss algorithms for estimating the Shannon entropy h of finite symbol sequences with long range correlations. In particular, we consider algorithms which estimate h from the code lengths produced by some compression algorithm. Our interest is in describing their convergence with sequence length, assuming no limits for the space and time complexities of the compression algorithms. A scaling law is proposed for extrapolation from finite sample lengths. This is applied to sequences of dynamical systems in non-trivial chaotic regimes, a 1-D cellular automaton, and to written English texts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 48,949
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193
A universal algorithm for sequential data compression
Ziv, J., & Lempel, A. (1977) · IEEE Transactions on Information Theory · doi:10.1109/tit.1977.1055714 · Cited by 5,456
Quantitative universality for a class of nonlinear transformations
Feigenbaum, M. J. (1978) · Journal of Statistical Physics · doi:10.1007/bf01020332 · Cited by 3,589
Compression of individual sequences via variable-rate coding
Ziv, J., & Lempel, A. (1978) · IEEE Transactions on Information Theory · doi:10.1109/tit.1978.1055934 · Cited by 3,432
On the Complexity of Finite Sequences
Lempel, A., & Ziv, J. (1976) · IEEE Transactions on Information Theory · doi:10.1109/tit.1976.1055501 · Cited by 2,717
Prediction and Entropy of Printed English
Shannon, C. E. (1951) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1951.tb01366.x · Cited by 2,686
Intermittent transition to turbulence in dissipative dynamical systems
Pomeau, Y., & Manneville, P. (1980) · Communications in Mathematical Physics · doi:10.1007/bf01197757 · Cited by 2,015
A philosophical essay on probabilities
(1952) · Journal of the Franklin Institute · doi:10.1016/0016-0032(52)90656-x · Cited by 1,136
Bell, T., Cleary, J. G., & Witten, I. H. (1990) · Cited by 988
Stochastic Complexity and Modeling
Rissanen, J. (1986) · The Annals of Statistics · doi:10.1214/aos/1176350051 · Cited by 922
The context-tree weighting method: basic properties
Willems, F. M. J., Shtarkov, Y. M., & Tjalkens, T. J. (1995) · IEEE Transactions on Information Theory · doi:10.1109/18.382012 · Cited by 919
On the Length of Programs for Computing Finite Binary Sequences: statistical considerations
Chaitin, G. J. (1969) · Journal of the ACM · doi:10.1145/321495.321506 · Cited by 918
On the Length of Programs for Computing Finite Binary Sequences
Chaitin, G. J. (1966) · Journal of the ACM · doi:10.1145/321356.321363 · Cited by 913
Ergodic Theory and Information
Révész, P., Billingsley, P., & Revesz, P. (1966) · Revue de l'Institut International de Statistique / Review of the International Statistical Institute · doi:10.2307/1401263 · Cited by 864
A universal data compression system
Rissanen, J. (1983) · IEEE Transactions on Information Theory · doi:10.1109/tit.1983.1056741 · Cited by 631
Logical basis for information theory and probability theory
Kolmogorov, A. (1968) · IEEE Transactions on Information Theory · doi:10.1109/tit.1968.1054210 · Cited by 592
The Estimation of Probabilities: An Essay on Modern Bayesian Methods
Sparks, J. N., & Good, I. J. (1967) · American Educational Research Journal · doi:10.2307/1162122 · Cited by 528
Random sequence generation by cellular automata
Wolfram, S. (1986) · Advances in Applied Mathematics · doi:10.1016/0196-8858(86)90028-x · Cited by 494
Dimensions and Entropies in Chaotic Systems: Quantification of Complex Behavior
(1986) · Springer Series in Synergetics · doi:10.1007/978-3-642-71001-8 · Cited by 475
Rissanen, J., & Langdon, G. (1981) · IEEE Transactions on Information Theory · doi:10.1109/tit.1981.1056282 · Cited by 454
Finite sample corrections to entropy and dimension estimates
Grassberger, P. (1988) · Physics Letters A · doi:10.1016/0375-9601(88)90193-4 · Cited by 287
Self-citationA universal finite memory source
Weinberger, M. J., Rissanen, J. J., & Feder, M. (1995) · IEEE Transactions on Information Theory · doi:10.1109/18.382011 · Cited by 228
A convergent gambling estimate of the entropy of English
Cover, T., & King, R. (1978) · IEEE Transactions on Information Theory · doi:10.1109/tit.1978.1055912 · Cited by 220
Estimating functions of probability distributions from a finite set of samples
Wolpert, D. H., & Wolf, D. R. (1995) · Physical Review E · doi:10.1103/physreve.52.6841 · Cited by 191
Entropy and Long-Range Correlations in Literary English
Ebeling, W., & Pöschel, T. (1993) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/26/4/001 · Cited by 183
Global dynamical behavior of the optical field in a ring cavity
Hammel, S. M., Moloney, J. V., & Jones, C. K. R. T. (1985) · Journal of the Optical Society of America B · doi:10.1364/josab.2.000552 · Cited by 160
Word frequency and entropy of symbolic sequences: a dynamical perspective
Ebeling, W., & Nìcolis, G. (1992) · Chaos Solitons & Fractals · doi:10.1016/0960-0779(92)90058-u · Cited by 138
LONG RANGE CORRELATION IN HUMAN WRITINGS
Schenkel, A., ZHANG, J., & Zhang, Y. (1993) · Fractals · doi:10.1142/s0218348x93000083 · Cited by 126
Universal Schemes for Prediction, Gambling and Portfolio Selection
Algoet, P. (1992) · The Annals of Probability · doi:10.1214/aop/1176989811 · Cited by 114
Finite sample effects in sequence analysis
Herzel, H., Schmitt, A. O., & Ebeling, W. (1994) · Chaos Solitons & Fractals · doi:10.1016/0960-0779(94)90020-5 · Cited by 112
Estimating the information content of symbol sequences and efficient codes
Grassberger, P. (1989) · IEEE Transactions on Information Theory · doi:10.1109/18.30993 · Cited by 107
Self-citationA note on the Ziv - Lempel model for compressing individual sequences (Corresp.)
Langdon, G. G. (1983) · IEEE Transactions on Information Theory · doi:10.1109/tit.1983.1056645 · Cited by 85
Cited by
Causality detection based on information-theoretic approaches in time series analysis
HLAVACKOVASCHINDLER, K., PALUS, M., VEJMELKA, M., & BHATTACHARYA, J. (2007) · Physics Reports · doi:10.1016/j.physrep.2006.12.004 · Cited by 800
Equitability, mutual information, and the maximal information coefficient
Kinney, J. B., & Atwal, G. S. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1309933111 · Cited by 667
A review of symbolic analysis of experimental data
Daw, C. S., Finney, C. E. A., & Tracy, E. R. (2003) · Review of Scientific Instruments · doi:10.1063/1.1531823 · Cited by 591
Meyer, P. E., Lafitte, F., & Bontempi, G. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-461 · Cited by 510
Regularities unseen, randomness observed: Levels of entropy convergence
Crutchfield, J. P., & Feldman, D. P. (2003) · Chaos: An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/1.1530990 · Cited by 368
Metadata
- Authors
- Schürmann, T., & Grassberger, P.
- Venue
- Chaos: An Interdisciplinary Journal of Nonlinear Science
- Publisher
- AIP Publishing
- Published
- 1996-09-01
- Volume
- 6
- Issue
- 3
- Pages
- 414-427
- ISSN
- 1054-1500 (ISSN-L)
- 1089-7682
- OpenAlex
- W2018891628
- Cited by
- 319
- References
- 33
- FWCI
- 1.92
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.