Estimation of Entropy and Mutual Information
The DOI matches its Crossref record; 62 of the 63 references it cites were found in a registry or an index.
Abstract
We present some new results on the nonparametric estimation of entropy and mutual information. First, we use an exact local expansion of the entropy function to prove almost sure consistency and central limit theorems for three of the most commonly used discretized information estimators. The setup is related to Grenander's method of sieves and places no assumptions on the underlying probability measure generating the data. Second, we prove a converse to these consistency theorems, demonstrating that a misapplication of the most common estimation techniques leads to an arbitrarily poor estimate of the true information, even given unlimited data. This “inconsistency” theorem leads to an analytical approximation of the bias, valid in surprisingly small sample regimes and more accurate than the usual [Formula: see text] formula of Miller and Madow over a large region of parameter space. The two most practical implications of these results are negative: (1) information estimates in a certain data regime are likely contaminated by bias, even if “bias-corrected” estimators are used, and (2) confidence intervals calculated by standard techniques drastically underestimate the error of the most common estimation methods. Finally, we note a very useful connection between the bias of entropy estimators and a certain polynomial approximation problem. By casting bias calculation problems in this approximation theory framework, we obtain the best possible generalization of known asymptotic bias results. More interesting, this framework leads to an estimator with some nice properties: the estimator comes equipped with rigorous bounds on the maximum error over all possible underlying probability distributions, and this maximum error turns out to be surprisingly small. We demonstrate the application of this new estimator on both real and simulated data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The information-theoretic neuroscience literature has long recognized the importance of correcting for the limited-sampling bias and has provided effective tools for correcting the Shannon information sampling bias.24,25,26,27,28”
From Sampling bias corrections for discrete and Gaussian partial information decompositions (2026) · cites this paper as [25]
In the citing paper
“Earlier applications of information theory to neuroscience have recognized that Shannon information measures from neural activity suffer from a prominent limited-sampling bias.24,25,26”
From Sampling bias corrections for discrete and Gaussian partial information decompositions (2026) · cites this paper as [25]
In the citing paper
“It is also well known that such histogram approach leads to a biased estimate of the entropy [58].”
From Maximizing the information content of experiments in systems biology (2013) · cites this paper as [58]
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 48,949
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 36,808
Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (1994) · Cited by 15,411
Numerical recipes in Pascal: the art of scientific computing
(1990) · Choice Reviews Online · doi:10.5860/choice.27-5158 · Cited by 11,811
The Mathematical Theory of Communication
Shannon, C. E., Weaver, W., & Wiener, N. (1950) · Physics Today · doi:10.1063/1.3067010 · Cited by 9,801
Loève, M. (1963) · Andalas University Repository (Andalas University) · Cited by 5,938
Approximation Theorems of Mathematical Statistics
Serfling, R. J. (1980) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316481 · Cited by 5,851
Weak Convergence and Empirical Processes
van der Vaart, A. W., & Wellner, J. A. (1996) · Springer Series in Statistics · doi:10.1007/978-1-4757-2545-2 · Cited by 5,014 · cited 1×
“This asymptotically deterministic behavior of the sorted histograms is perhaps surprising, given that there is no such corresponding deterministic behavior for the unsorted histograms (although, by the Glivenko-Cantelli theorem, van der Vaart & Wellner, 1996, there is well-known deterministic behavior for the integral…”
BackgroundInfluentialA Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations
Chernoff, H. (1952) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729330 · Cited by 3,550 · cited 2×
“One such slice through parameter space is given by the “central lines” of the m-simplex: these are the subsets formed by linearly interpolating between the trivial (minimal entropy) and flat (maximal entropy) distributions (there are m of these lines, by symmetry).”
BackgroundLarge Deviations Techniques and Applications
Schmock, U., Dembo, A., & Zeitouni, O. (2000) · Journal of the American Statistical Association · doi:10.2307/2669805 · Cited by 3,524
A Probabilistic Theory of Pattern Recognition
Devroye, L., Györfi, L., & Lugosi, G. (1996) · Stochastic Modelling and Applied Probability · doi:10.1007/978-1-4612-0711-5 · Cited by 3,274 · cited 13×
“We follow the terminology in Devroye et al. (1996), with a few obvious modications.”
BackgroundMethodInfluentialOn the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
Vapnik, V. N., & Chervonenkis, A. Y. (2015) · Measures of Complexity · doi:10.1007/978-3-319-21852-6_3 · Cited by 3,105 · cited 2×
“As a start, we have the following consistency result, stated in terms of the measure of the richness of a partition introduced by Vapnik and Chervonenkis (1971), N(AF ) (the shatter coefficient of the set of allowed partitions, defined in the appendix; m is, as in the preceding, the maximal number…”
BackgroundSpikes: Exploring the Neural Code
Rieke, F. M., Warland, D., van Steveninck, R. R. D. R., & Bialek, W. (1996) · Medical Entomology and Zoology · Cited by 2,844
The information bottleneck method
Tishby, N., Pereira, F. C., & Bialek, W. (2000) · arXiv · doi:10.48550/arxiv.physics/0004057 · Cited by 2,626 · cited 1×
“is due to Steele (1986) : Theorem (Steele’ s inequality).”
BackgroundOn the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
Vapnik, V. N., & Chervonenkis, A. Y. (1971) · Theory of Probability & Its Applications · doi:10.1137/1116025 · Cited by 2,448
On the mathematical foundations of learning
Cucker, F., & Smale, S. (2001) · Bulletin of the American Mathematical Society · doi:10.1090/s0273-0979-01-00923-5 · Cited by 1,545 · cited 3×
“For example, an application of theorem II.2.3 of Cucker and Smale (2002) gives the exact rate of decay of our L2 objective function, equation 6.3, in terms of the spectral properties of the discrete differential operator D, expressed in the Bernstein polynomial basis; however, it is unclear at…”
BackgroundMethodInfluentialThe Jackknife Estimate of Variance
Efron, B., & Stein, C. (1981) · The Annals of Statistics · doi:10.1214/aos/1176345462 · Cited by 1,436 · cited 2×
“Most common estimators of H, including ĤMLE, ĤMM, and ĤJK, share this permutation symmetry (in fact, one can show that there is some statistical justification for restricting our attention to this class of symmetric estimators; see appendix A).”
BackgroundMethodOn the method of bounded differences
McDiarmid, C. (1989) · Surveys in Combinatorics, 1989 · doi:10.1017/cbo9781107359949.008 · Cited by 1,425 · cited 1×
“McDiarmid (1989) and Devroye, Gyorfi, and Lugosi (1996) provided a proof and extended discussions.”
MethodOn Choosing and Bounding Probability Metrics
Gibbs, A. L., & Su, F. E. (2002) · International Statistical Review · doi:10.1111/j.1751-5823.2002.tb00178.x · Cited by 1,163 · cited 1×
“The second useful consequence of the local expansion follows by the next two well-known results (Gibbs & Su, 2002): 0 ≤ DKL(pN; p) ≤ log(1+ χ2(pN; p)), (4.3) where χ2 ≡ m∑ i=1 (pN,i − pi)2 p2i denotes Pearson’s chi-square functional, and Ep(χ2(pN; p)) = |supp(p)| − 1N ∀p, (4.4) where |supp(p)|…”
InfluentialDitzian, Z., & Totik, V. (1987) · Springer Series in Computational Mathematics · doi:10.1007/978-1-4612-4778-4 · Cited by 1,143 · cited 2×
“It can be shown, using the above bounds combined with a much deeper result from approximation theory (Devore & Lorentz, 1993; Ditzian & Totik, 1987), that there exists an aj,N such that the maximum (over all p) bias is O( mN2 ).”
BackgroundWeighted sums of certain dependent random variables
Azuma, K. (1967) · Tohoku Mathematical Journal · doi:10.2748/tmj/1178243286 · Cited by 1,084 · cited 1×
“Theorem (“McDiarmid’s inequality”; Chernoff, 1952; Azuma, 1967).”
Ergodic Theory and Information
S., D., & Billingsley, P. (1966) · Mathematics of Computation · doi:10.2307/2003618 · Cited by 1,070 · cited 2×
“SN; TN/ ! I; this follows from results in standard references, such as Billingsley (1965) and Kolmogorov (1993), if either 3⁄4S1;T1 μ 3⁄4S2 ;T2 μ ¢ ¢ ¢ μ 3⁄4SN ;TN μ ¢ ¢ ¢ and 3⁄4X;Y D [N3⁄4SN ;TN ; or sup A23⁄4SN ;TN ;B23⁄4X;Y 1⁄2.”
Schervish, M. J. (1995) · Springer Series in Statistics · doi:10.1007/978-1-4612-4250-5 · Cited by 1,046
Bialek, W., Rieke, F., de Ruyter van Steveninck, R. R., & Warland, D. (1991) · Science · doi:10.1126/science.2063199 · Cited by 992 · cited 3×
“The most successful approach taken to date in our field to circumvent these problems was introduced by Bialek and colleagues ( Bialek, Rieke, de Ruyter van Steveninck, & Warland, 1991; Strong et al., 1998).”
InfluentialErgodic Theory and Information
Révész, P., Billingsley, P., & Revesz, P. (1966) · Revue de l'Institut International de Statistique / Review of the International Statistical Institute · doi:10.2307/1401263 · Cited by 864
Entropy and information in neural spike trains
Strong, S. P., Koberle, R., de Ruyter van Steveninck, R. R., & Bialek, W. (1998) · Physical Review Letters · doi:10.1103/physrevlett.80.197 · Cited by 791 · cited 10×
“But if V/B2 < 1, bias plays a more important role, and estimators that are specifically designed to reduce the bias become competitive; previous work has demonstrated that ĤMM and ĤJK are effective in this regime (Panzeri & Treves, 1996; Strong et al., 1998).”
BackgroundMethodResultInfluential(1997) · SZTAKI Publication Repository (Hungarian Academy of Sciences) · Cited by 598
Estimation of the information by an adaptive partitioning of the observation space
Darbellay, G. A., & Vajda, I. (1999) · IEEE Transactions on Information Theory · doi:10.1109/18.761290 · Cited by 493 · cited 2×
“What are the benefits? Intuitively, we should gain in efficiency: we are putting the partitions where they do the most good ( Darbellay & Vajda, 1999 ).”
BackgroundEfficient Discrimination of Temporal Patterns by Motion-Sensitive Neurons in Primate Visual Cortex
Burac̆as, G. T., Zador, A. M., DeWeese, M. R., & Albright, T. D. (1998) · Neuron · doi:10.1016/s0896-6273(00)80477-8 · Cited by 383 · cited 1×
“Buracas, Zador, DeWeese, & Albright, 1998 ) is at first sight less dependent on assumptions on the prior distribtion on X . Here one discretizes the space of all spike trains on some interval into some finite number, m, of words w, and makes use of the information formula for discrete distributions,”
BackgroundThe Upward Bias in Measures of Information Derived from Limited Data Samples
Treves, A., & Panzeri, S. (1995) · Neural Computation · doi:10.1162/neco.1995.7.2.399 · Cited by 358 · cited 3×
“Treves and Panzeri (1995) and Victor (2000a) both admit that their methods (a divergent expansion of the logarithm in each case) are not rigorous.”
BackgroundMethodInfluentialAdvances in Quantitative Electroencephalogram Analysis Methods
Thakor, N. V., & Tong, S. (2004) · Annual Review of Biomedical Engineering · doi:10.1146/annurev.bioeng.5.040202.121601 · Cited by 305
Analyzing Neural Responses to Natural Signals: Maximally Informative Dimensions
Sharpee, T., Rust, N. C., & Bialek, W. (2004) · Neural Computation · doi:10.1162/089976604322742010 · Cited by 298
On a Statistical Estimate for the Entropy of a Sequence of Independent Random Variables
Basharin, G. P. (1959) · Theory of Probability & Its Applications · doi:10.1137/1104033 · Cited by 263 · cited 3×
“If m = o(N), then Ĥ → HN a.s. Note thatσ 2N in the statement of the CLT (theorem 2) is exactly the variance of the sum in expression 4.1, and corresponds to the asymptotic variance derived originally in Basharin (1959), by a similar local expansion.”
BackgroundInfluentialGeometrizing Rates of Convergence, II
Donoho, D. L., & Liu, R. C. (1991) · The Annals of Statistics · doi:10.1214/aos/1176348114 · Cited by 261 · cited 1×
“…approximation problem described in section 6 is essentially determined by a modulus of continuity of the function under question (Ditzian & Totik, 1987), and moduli of continuity pop up in apparently very different functional estimation contexts as well (Donoho & Liu, 1991; Jongbloed, 2000).”
BackgroundOn a Class of Problems Related to the Random Division of an Interval
Darling, D. A. (1953) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729030 · Cited by 256 · cited 4×
“…to those used in the proof of theorem 3 (sorting and linearity of expectation, effectively), and the asymptotic variance follows upon combining the formulas of Darling (1953) and Shao and Hahn (1995) with an approximation-in-measure argument similar, again, to that used to prove theorem 3.”
MethodInfluentialBinless strategies for estimation of information from neural data
Victor, J. D. (2002) · Physical Review E · doi:10.1103/physreve.66.051903 · Cited by 238
An Efron-Stein Inequality for Nonsymmetric Statistics
Steele, J. M. (1986) · The Annals of Statistics · doi:10.1214/aos/1176349952 · Cited by 226 · cited 3×
“The second variance bound comes from Steele (1986); see appendix B for the proof (again, a generalization of the corresponding result for Ĥ): Proposition 6.”
BackgroundConvergence properties of functional estimates for discrete distributions
Antos, A., & Kontoyiannis, I. (2001) · Random Structures & Algorithms · doi:10.1002/rsa.10019 · Cited by 224 · cited 36×
“Theorem 1 (Consistency).”
BackgroundResultInfluentialApproximation theory and numerical methods
Watson, G. A. (1980) · International Linear Collider · Cited by 212
Entropy and Inference, Revisited
Nemenman, I., Shafee, F., & Bialek, W. (2002) · Advances in Neural Information Processing Systems 14 · doi:10.7551/mitpress/1120.003.0065 · Cited by 211 · cited 1×
“... to the martingale E(H | x1 ,..., xi); the computation of the asymptotic mean follows by methods almost identical to those used in the proof of theorem 3 (sorting and linearity of expectation, effectively), and the asymptotic variance follows upon combining the formulas of Darling (1953) and Shao and Hahn (1995) wi…”
MethodAnalytical estimates of limited sampling biases in different information measures
Panzeri, S., & Treves, A. (1996) · Network Computation in Neural Systems · doi:10.1088/0954-898x/7/1/006 · Cited by 207
Entropy and information in neural spike trains: Progress on the sampling problem
Nemenman, I., Bialek, W., & de Ruyter van Steveninck, R. (2004) · Physical Review E · doi:10.1103/physreve.69.056111 · Cited by 196
Estimating functions of probability distributions from a finite set of samples
Wolpert, D. H., & Wolf, D. R. (1995) · Physical Review E · doi:10.1103/physreve.52.6841 · Cited by 191 · cited 3×
“Wolpert and Wolf (1995) give an explicit formula for the Bayes’ estimate of H and related statistics in the case of a uniform prior on the simplex.”
BackgroundConsistency of data-driven histogram methods for density estimation and classification
Lugosi, G., & Nobel, A. (1996) · The Annals of Statistics · doi:10.1214/aos/1032894460 · Cited by 169 · cited 1×
“Theorem (Lugosi & Nobel, 1996).”
Estimating the Entropy Rate of Spike Trains via Lempel-Ziv Complexity
Amigó, J. M., Szczepański, J., Wajnryb, E., & Sanchez-Vives, M. V. (2004) · Neural Computation · doi:10.1162/089976604322860677 · Cited by 160
Estimating Entropy on $m$ Bins Given Fewer Than $m$ Samples
Paninski, L. (2004) · IEEE Transactions on Information Theory · doi:10.1109/tit.2004.833360 · Cited by 148
Self-citationAnalytical estimates of limited sampling biases in different information measures
Panzeri, S., & Treves, A. (1996) · Network: Computation in Neural Systems · doi:10.1080/0954898x.1996.11978656 · Cited by 138 · cited 2×
“But if V/B2 < 1, bias plays a more important role, and estimators that are specifically designed to reduce the bias become competitive; previous work has demonstrated that ĤMM and ĤJK are effective in this regime (Panzeri & Treves, 1996; Strong et al., 1998).”
BackgroundMethodOn Choosing and Bounding Probability Metrics
Gibbs, A. L., & Su, F. E. (2002) · International Statistical Review / Revue Internationale de Statistique · doi:10.2307/1403865 · Cited by 136
Bias analysis in entropy estimation
Schürmann, T. (2004) · Journal of Physics A Mathematical and General · doi:10.1088/0305-4470/37/27/l02 · Cited by 111
Society for Neuroscience Abstracts
Chambers, D., Wingate, R. J. T., & Mason, I. J. (2000) · Cited by 96
On the bias of information estimates.
Carlton, A. G. (1969) · Psychological Bulletin · doi:10.1037/h0026857 · Cited by 95 · cited 2×
“Several articles (Miller, 1955; Carlton, 1969; Treves & Panzeri, 1995; Victor, 2000a) provide a series expansion for the bias, in the hope of estimating and subtracting out the bias directly.”
BackgroundAchieving Information Bounds in Non and Semiparametric Models
Ritov, Y., & Bickel, P. J. (1990) · The Annals of Statistics · doi:10.1214/aos/1176347633 · Cited by 93 · cited 1×
“In other words, with high probability, we have to estimate H given only 1 + α−1 numbers, namely {hj}0≤j≤α−1 , and it is not hard to see, given equation 5.1 and the usual Bayesian lower bounds on minimax error rates (see, e.g., Ritov & Bickel, 1990), that this is not enough to estimate H(p).”
BackgroundInfluentialHow the brain uses time to represent and process visual information
Victor, J. D. (2000) · Brain Research · doi:10.1016/s0006-8993(00)02751-7 · Cited by 85 · cited 3×
“(See, e.g., Victor, 2000b, for recent work along these lines.)”
BackgroundInformation theory and the theory of algorithms
Kolmogorov, A. N., Ширяев, А. Н., & Sosinskiĭ, A. B. (2010) · Kluwer Academic Publishers eBooks · Cited by 83
Second-order noiseless source coding theorems
Kontoyiannis, I. (1997) · IEEE Transactions on Information Theory · doi:10.1109/18.605604 · Cited by 77 · cited 2×
“N goes by the name of “minimal coding variance”; see Kontoyiannis, 1997, for more details).”
BackgroundUniversal Entropy Estimation Via Block Sorting
Cai, H., Kulkarni, S. R., & Verdú, S. (2004) · IEEE Transactions on Information Theory · doi:10.1109/tit.2004.830771 · Cited by 73
Analysis of neural coding through quantization with an information-based distortion measure
Dimitrov, A. G., Miller, J. P., Gedeon, T., Aldworth, Z., & Parker, A. E. (2003) · Network Computation in Neural Systems · doi:10.1080/net.14.1.151.176 · Cited by 49
Asymptotic Bias in Information Estimates and the Exponential (Bell) Polynomials
Victor, J. D. (2000) · Neural Computation · doi:10.1162/089976600300014728 · Cited by 46 · cited 4×
“(See, e.g., Victor, 2000b, for recent work along these lines.)”
BackgroundMethodInfluentialLimit theorems for the logarithm of sample spacings
Shao, Y., & Hahn, M. G. (1995) · Statistics & Probability Letters · doi:10.1016/0167-7152(94)00156-3 · Cited by 30 · cited 1×
“…to those used in the proof of theorem 3 (sorting and linearity of expectation, effectively), and the asymptotic variance follows upon combining the formulas of Darling (1953) and Shao and Hahn (1995) with an approximation-in-measure argument similar, again, to that used to prove theorem 3.”
MethodMinimax lower bounds and moduli of continuity
Jongbloed, G. (2000) · Statistics & Probability Letters · doi:10.1016/s0167-7152(00)00104-8 · Cited by 18 · cited 1×
“…approximation problem described in section 6 is essentially determined by a modulus of continuity of the function under question (Ditzian & Totik, 1987), and moduli of continuity pop up in apparently very different functional estimation contexts as well (Donoho & Liu, 1991; Jongbloed, 2000).”
BackgroundCramer-Rao type integral inequalities for general loss functions
Рао, Б. Л. С. П. (2001) · Test · doi:10.1007/bf02595826 · Cited by 4 · cited 2×
“HMLE is asymptotically minimax for fixed m as N →∞ (by “minimax,” we mean best in a worst-case sense; we discuss this concept in more detail below); see Prakasa Rao (2001) for the standard technique, a clever “local Bayesian” application of the Cramer-Rao inequality.”
Method
Research works printed but not linked: 1
- Beirlant J., 1997, International Journal of the Mathematical Statistics Sciences, 6, 17
Cited by
Chao, A., Gotelli, N. J., Hsieh, T. C., Sander, E. L., Ma, K. H., Colwell, R. K., & Ellison, A. M. (2014) · Ecological Monographs · doi:10.1890/13-0133.1 · Cited by 4,256
“Asymptotic estimators for Hill numbers with q ¼ 1 are closely related to the well-known entropy estimation (for reviews, see Paninski 2003, Chao et al. 2013).”
Opening the Black Box of Deep Neural Networks via Information
Shwartz-Ziv, R., & Tishby, N. (2017) · arXiv · doi:10.48550/arxiv.1703.00810
“Good overviews on methods for mutual information estimation can be found in Paninski (2003) and Kraskov et al. (2004).”
MethodA comprehensive survey on sentiment analysis: Approaches, challenges and trends
Birjali, M., Kasri, M., & Beni-Hssane, A. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107134 · Cited by 921
Conditional Likelihood Maximisation: A Unifying Framework for Mutual Information Feature Selection
Brown, G., Pocock, A., Zhao, M., & Luján, M. (2012) · Research Explorer (The University of Manchester) · Cited by 1,359
Mutual Information Neural Estimation.
Belghazi, M. I., Baratin, A., Rajeshwar, S., Ozair, S., Bengio, Y., Courville, A. C., & Hjelm, D. (2018) · International Conference on Machine Learning · Cited by 920
Metadata
- Authors
- Paninski, L.
- Affiliations
- New York University (US)
- Corresponding author
- Paninski, Liam
- ORCID
- Paninski, Liam
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2003-06-01
- Volume
- 15
- Issue
- 6
- Pages
- 1191-1253
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2114771311
- Cited by
- 1,442
- References
- 63
- Citations by year
- 2026: 88 · 2025: 84 · 2024: 73 · 2023: 93 · 2022: 84 · 2021: 100 · 2020: 90 · 2019: 100 · 2018: 83 · 2017: 52 · 2016: 62 · 2015: 60 · 2014: 52 · 2013: 56 · 2012: 51
- FWCI
- 28.49
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Computation (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Statistical Mechanics and Entropy
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Keywords
- entropy estimation, mutual information estimation, consistency, central limit theorem, confidence intervals, polynomial approximation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.