Reasoning in Reference Games: Individual- vs. Population-Level Probabilistic Modeling
Open accessThe DOI matches its Crossref record; 102 of the 108 references it cites were found in a registry or an index.
Abstract
Recent advances in probabilistic pragmatics have achieved considerable success in modeling speakers' and listeners' pragmatic reasoning as probabilistic inference. However, these models are usually applied to population-level data, and so implicitly suggest a homogeneous population without individual differences. Here we investigate potential individual differences in Theory-of-Mind related depth of pragmatic reasoning in so-called reference games that require drawing ad hoc Quantity implicatures of varying complexity. We show by Bayesian model comparison that a model that assumes a heterogenous population is a better predictor of our data, especially for comprehension. We discuss the implications for the treatment of individual differences in probabilistic models of language use.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




![Fig 5 Sketch of the full data-generating model as a probabilistic graphical model, using the conventions of [64]. The more abstract a parameter, the higher it is in the graph. Deterministic variables have double edges. Categorical variables have rectangular shape. Observed variables are shaded in gray. Boxes encircle the range of indices.](https://figures.cochat.ai/paper-figures/b5ebb41857d34d1793ee8e14603ef222/reasoning-in-reference-games-individual-vs-population-level-probabilistic-figure-5.jpg)


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Classically, the RSA framework is used to computationally model speakers’ and listeners’ behavior in communicative tasks such as the reference game [26,33,34,58].”
From The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model (2026) · cites this paper as [34]
The cited source says what the claim attributes to it.
In the citing paper
“The RSA model provides a framework to model different listener and speaker types that vary by the information they take into account to choose or interpret a message.”
From The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model (2026) · cites this paper as [34]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Grice, H. P. (1975) · Speech Acts · doi:10.1163/9789004368811_003 · Cited by 23,221
Inference from iterative simulation using multiple sequences
Gelman, A., & Rubin, D. B. (1992) · Statistical Science · doi:10.1214/ss/1177011136 · Cited by 16,627
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
Random effects structure for confirmatory hypothesis testing: Keep it maximal
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013) · Journal of Memory and Language · doi:10.1016/j.jml.2012.11.001 · Cited by 10,328
Jeffreys, H., Moran, P. A. P., & Chatfield, C. (1922) · Nature · doi:10.1093/oso/9780198503682.001.0001 · Cited by 7,878
Marr, D. (1982) · Cumulative Index of Computer Aided Architectural Design · Cited by 5,733
Barlow, H. B. (1983) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(83)90030-5 · Cited by 5,424
JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling
Plummer, M. (2003) · Cited by 4,720
Probability theory the logic of science
Jaynes, E. T., & Bretthorst, G. L. (2003) · Cited by 4,654
Probability Theory: The Logic of Science
Jaynes, E. T. (2003) · doi:10.1017/cbo9780511790423 · Cited by 4,045
Convention: A Philosophical Study.
Llewelyn, J. E., & Lewis, D. K. (1970) · The Philosophical Quarterly · doi:10.2307/2218418 · Cited by 3,121
Barto, A. G. (1998) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)38315-5 · Cited by 2,976
Individual Choice Behavior: A Theoretical Analysis.
Cane, V., & Luce, R. D. (1960) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2343282 · Cited by 2,504
The Adaptive Character of Thought
Massaro, D. W., Friedman, D., & Anderson, J. R. (1991) · The American Journal of Psychology · doi:10.2307/1423253 · Cited by 2,174
Syntax and Semantics:" Vol. 3, "Speech Acts
Parret, H. (1977) · Tijdschrift voor Philosophie · Cited by 1,694
A Cognitive Hierarchy Model of Games
Camerer, C. F., Ho, T.-H., & Chong, J.-K. (2004) · The Quarterly Journal of Economics · doi:10.1162/0033553041502225 · Cited by 1,556
Conceptual pacts and lexical choice in conversation.
Brennan, S. E., & Clark, H. H. (1996) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037//0278-7393.22.6.1482 · Cited by 1,457
(2013) · Cited by 1,351
JEFFREYS, H. (1922) · Nature · doi:10.1038/109132a0 · Cited by 1,217
Individual choice behavior: A theoretical analysis.
Luce, R. D. (2005) · doi:10.1037/14396-000 · Cited by 1,157
Studies on the semantics of questions and the pragmatics of answers
Groenendijk, J., & Stokhof, M. (1984) · UvA-DARE (University of Amsterdam) · Cited by 1,051
Farrell, J., & Rabin, M. (1996) · Journal of Economic Perspectives · doi:10.1257/jep.10.3.103 · Cited by 1,012
Predicting pragmatic reasoning in language games
Frank, M. C., & Goodman, N. D. (2012) · Science · doi:10.1126/science.1218633 · Cited by 868
Individual differences in rational thought.
Stanovich, K. E., & West, R. F. (1998) · Journal of Experimental Psychology General · doi:10.1037//0096-3445.127.2.161 · Cited by 867
Free Choice and the Theory of Scalar Implicatures
Fox, D. (2007) · Presupposition and Implicature in Compositional Semantics · doi:10.1057/9780230210752_4 · Cited by 751
Bayesian hypothesis testing for psychologists: A tutorial on the Savage–Dickey method
Wagenmakers, E.-J., Lodewyckx, T., Kuriyal, H., & Grasman, R. (2010) · Cognitive Psychology · doi:10.1016/j.cogpsych.2009.12.001 · Cited by 740
Some utterances are underinformative: The onset and time course of scalar inferences
Bott, L., & Noveck, I. A. (2004) · Journal of Memory and Language · doi:10.1016/j.jml.2004.05.006 · Cited by 665
The problem of inference from curves based on group data.
Estes, W. K. (1956) · Psychological Bulletin · doi:10.1037/h0045156 · Cited by 654
Robust speech perception: Recognize the familiar, generalize to the similar, and adapt to the novel.
Kleinschmidt, D. F., & Jaeger, T. F. (2015) · Psychological Review · doi:10.1037/a0038695 · Cited by 635
Syntax and Semantics Volume 3: Speech Acts
Geis, M. L., Cole, P., & Morgan, J. (1976) · The Modern Language Journal · doi:10.2307/324613 · Cited by 580
Jones, M., & Love, B. C. (2011) · Behavioral and Brain Sciences · doi:10.1017/s0140525x10003134 · Cited by 533
Meaning and Credibility in Cheap-Talk Games
Farrell, J. (1993) · Games and Economic Behavior · doi:10.1006/game.1993.1029 · Cited by 507
(1997) · An Introduction to Natural Computation · doi:10.7551/mitpress/3917.003.0017 · Cited by 492
Iterated Dominance and Iterated Best-Response in Experimental P-Beauty Contests
Ho, T., Weigelt, K., & Camerer, C. F. (1996) · CaltechAUTHORS (California Institute of Technology) · doi:10.7907/2wqmd-n6102 · Cited by 464
Breheny, R., Katsos, N., & Williams, J. (2006) · Cognition · doi:10.1016/j.cognition.2005.07.003 · Cited by 452
Quantal Choice Analaysis: A SurveyPreprint
McFadden, D. L. (1976) · RePEc: Research Papers in Economics · Cited by 397
Rapid Expectation Adaptation during Syntactic Comprehension
Fine, A. B., Jaeger, T. F., Farmer, T. A., & Qian, T. (2013) · PLoS ONE · doi:10.1371/journal.pone.0077661 · Cited by 397
Knowledge and Implicature: Modeling Language Understanding as Social Cognition
Goodman, N. D., & Stuhlmüller, A. (2013) · Topics in Cognitive Science · doi:10.1111/tops.12007 · Cited by 397
Lying for Strategic Advantage: Rational and Boundedly Rational Misrepresentation of Intentions
Crawford, V. P. (2003) · American Economic Review · doi:10.1257/000282803321455197 · Cited by 391
Pragmatic Meaning and Non-monotonic Reasoning: The Case of Exhaustive Interpretation
Schulz, K., & Van Rooij, R. (2006) · Linguistics and Philosophy · doi:10.1007/s10988-005-3760-4 · Cited by 375
Quantity implicatures, exhaustive interpretation, and rational conversation
Franke, M. (2011) · Semantics and Pragmatics · doi:10.3765/sp.4.1 · Cited by 319
Self-citationGesture in Spatial Cognition: Expressing, Communicating, and Thinking About Spatial Information
Alibali, M. W. (2005) · Spatial Cognition & Computation · doi:10.1207/s15427633scc0504_2 · Cited by 317
The philosophy of Bayes factors and the quantification of statistical evidence
Morey, R. D., Romeijn, J.-W., & Rouder, J. N. (2016) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2015.11.001 · Cited by 313
How cognitive modeling can benefit from hierarchical Bayesian models
Lee, M. D. (2011) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2010.08.013 · Cited by 307
Signal to act : game theory in pragmatics
Franke, M. (2009) · UvA-DARE (University of Amsterdam) · Cited by 303
Self-citationProcessing Scalar Implicature: A Constraint‐Based Approach
Degen, J., & Tanenhaus, M. K. (2015) · Cognitive Science · doi:10.1111/cogs.12171 · Cited by 292
Self-citationModel Comparison and the Principle of Parsimony
Vandekerckhove, J., Matzke, D., & Wagenmakers, E.-J. (2015) · Oxford Handbooks Online · doi:10.1093/oxfordhb/9780199957996.013.14 · Cited by 269
The conversational condition on horn scales
Matsumoto, Y. (1995) · Linguistics and Philosophy · doi:10.1007/bf00984960 · Cited by 268
10: Scalar Implicatures: Exhaustivity and Gricean Reasoning
Spector, B. (2007) · Questions in Dynamic Semantics · doi:10.1163/9780080470993_011 · Cited by 262
The Weighted Likelihood Ratio, Sharp Hypotheses about Chances, the Order of a Markov Chain
Dickey, J. M., & Lientz, B. P. (1970) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177697203 · Cited by 251
The role of executive function in perspective taking during online language comprehension
Brown-Schmidt, S. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.5.893 · Cited by 250
Proceedings of Sinn und Bedeutung 13
Riester, A., & Solstad, T. (2009) · OPUS Publication Server of the University of Stuttgart (University of Stuttgart) · doi:10.18419/opus-5718 · Cited by 237
Fatal Attraction: Salience, Naïveté, and Sophistication in Experimental “Hide-and-Seek” Games
Crawford, V. P., & Iriberri, N. (2007) · American Economic Review · doi:10.1257/aer.97.5.1731 · Cited by 230
Quantal Choice Analysis: A Survey
McFadden, D. L. (1976) · NBER Chapters · Cited by 226
From if to iff: Conditional perfection as pragmatic strengthening
Horn, L. R. (2000) · Journal of Pragmatics · doi:10.1016/s0378-2166(99)00053-3 · Cited by 216
How Robust Are Probabilistic Models of Higher-Level Cognition?
Marcus, G. F., & Davis, E. (2013) · Psychological Science · doi:10.1177/0956797613495418 · Cited by 195
What do you think I think you think?: Strategic reasoning in matrix games
Hedden, T., & Zhang, J. (2002) · Cognition · doi:10.1016/s0010-0277(02)00054-9 · Cited by 195
Context, scale structure, and statistics in the interpretation of positive-form adjectives
Lassiter, D., & Goodman, N. D. (2013) · Semantics and Linguistic Theory · doi:10.3765/salt.v23i0.2658 · Cited by 192
Nonliteral understanding of number words
Kao, J. T., Wu, J. Y., Bergen, L., & Goodman, N. D. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1407479111 · Cited by 189
A note on functional relations obtained from group data.
Sidman, M. (1952) · Psychological Bulletin · doi:10.1037/h0063643 · Cited by 184
What is the scanpath signature of syntactic reanalysis?
von der Malsburg, T., & Vasishth, S. (2011) · Journal of Memory and Language · doi:10.1016/j.jml.2011.02.004 · Cited by 173
The Processing and Acquisition of Reference
(2011) · The MIT Press eBooks · doi:10.7551/mitpress/9780262015127.001.0001 · Cited by 172
Communication between rational agents
Rabin, M. (1990) · Journal of Economic Theory · doi:10.1016/0022-0531(90)90055-o · Cited by 171
The Effect of Speaker-Specific Information on Pragmatic Inferences
Grodner, D. J., & Sedivy, J. (2011) · The MIT Press eBooks · doi:10.7551/mitpress/9780262015127.003.0010 · Cited by 170
Probabilistic pragmatics, or why Bayes’ rule is probably important for pragmatics
Franke, M., & Jäger, G. (2016) · Zeitschrift für Sprachwissenschaft · doi:10.1515/zfs-2016-0002 · Cited by 170
Self-citationTemporal anaphora across and inside sentences: the function of participles
Bary, C., & Haug, D. (2011) · Semantics and Pragmatics · doi:10.3765/sp.4.8 · Cited by 166
Estes, W. K., & Maddox, W. T. (2005) · Psychonomic Bulletin & Review · doi:10.3758/bf03193784 · Cited by 160
McGrane, J. A. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00431 · Cited by 154
Presupposition and Implicature in Compositional Semantics
Sauerland, U., & Stateva, P. (2007) · Palgrave Macmillan UK eBooks · doi:10.1057/9780230210752 · Cited by 144
Questions in Dynamic Semantics
(2007) · BRILL eBooks · doi:10.1163/9780080470993 · Cited by 136
Toward an explanation of the power law artifact: Insights from response surface analysis
Myung, I. J., Kim, C., & Pitt, M. A. (2000) · Memory & Cognition · doi:10.3758/bf03198418 · Cited by 126 · cited 1×
“Why should such a computational-level rational analysis care about individual differences? First, if only aggregate data are considered, it is possible that the phenomenon we are trying to explain is an artifact of averaging [76].”
BackgroundSaying what's on your mind: Working memory effects on sentence production.
Slevc, L. R. (2011) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/a0024350 · Cited by 112
Learning individual talkers’ structural preferences
Kamide, Y. (2012) · Cognition · doi:10.1016/j.cognition.2012.03.001 · Cited by 101
Probabilistic pragmatics, or why Bayes’ rule is probably important for pragmatics
Franke, M., & Jäger, G. (2016) · Zeitschrift für Sprachwissenschaft · doi:10.15496/publikation-14148 · Cited by 98
Self-citationVerbal working memory predicts co-speech gesture: Evidence from individual differences
Gillespie, M., James, A. N., Federmeier, K. D., & Watson, D. G. (2014) · Cognition · doi:10.1016/j.cognition.2014.03.012 · Cited by 93
Pogue, A., Kurumada, C., & Tanenhaus, M. K. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.02035 · Cited by 89
Communicating with Cost-based Implicature: a Game-Theoretic Approach to Ambiguity
Rohde, H., Seyfarth, S., Clark, B., Jaeger, G., & Kaufmann, S. C. (2012) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 77
Cost-Based Pragmatic Inference about Referential Expressions
Degen, J., Franke, M., Jäger, G., Knauff, M., Pauen, M., Sebanz, N., & Wachsmuth, I. (2013) · UvA-DARE (University of Amsterdam) · Cited by 73
Self-citationOptimal assertions, and what they implicate. A uniform game theoretic approach
Benz, A., & van Rooij, R. (2007) · Topoi · doi:10.1007/s11245-006-9007-3 · Cited by 71
Term answers and contextual change
von Stechow, A., & Zimmermann, T. E. (1984) · Linguistics · doi:10.1515/ling.1984.22.1.3 · Cited by 66
Meaning and Use of Gradable Adjectives: Formal Modeling Meets Empirical Data
Qing, C., & Franke, M. (2014) · eScholarship (California Digital Library) · Cited by 66
Self-citationWhat Eye Movements Can Tell about Theory of Mind in a Strategic Game
Meijering, B., van Rijn, H., Taatgen, N., & Verbrugge, R. (2012) · PLoS ONE · doi:10.1371/journal.pone.0045961 · Cited by 62
Formalizing the Pragmatics of Metaphor Understanding
Kao, J., Bergen, L., & Goodman, N. D. (2014) · eScholarship (California Digital Library) · Cited by 54
Fatal Attraction: Focality, Naivete, and Sophistication in Experimental "Hide-and-Seek" GamesPreprint
Crawford, V. P., & Iriberri, N. (2004) · RePEc: Research Papers in Economics · Cited by 50
Ad-hoc scalar implicature in adults and children
Stiller, A. J., Goodman, N. D., & Frank, M. C. (2011) · eScholarship (California Digital Library) · Cited by 38
Variations on a Bayesian Theme: Comparing Bayesian Models of Referential Reasoning
Qing, C., & Franke, M. (2015) · Language, cognition and mind · doi:10.1007/978-3-319-17064-0_9 · Cited by 37
Self-citationOptimal Reasoning About Referential Expressions
Degen, J., Franke, M., Brown‐Schmidt, S., Ginzburg, J., & Larsson, S. (2012) · UvA-DARE (University of Amsterdam) · Cited by 36
Self-citationGAME THEORY AND SCALAR IMPLICATURES
Rothschild, D. (2013) · Philosophical Perspectives · doi:10.1111/phpe.12024 · Cited by 34
Fraundorf, S. H., Watson, D. G., & Benjamin, A. S. (2011) · Psychology and Aging · doi:10.1037/a0024138 · Cited by 33
Bayesian Natural Language Semantics and Pragmatics
Zeevat, H., & Schmitz, H.-C. (2015) · Language, cognition and mind · doi:10.1007/978-3-319-17064-0 · Cited by 31
Pragmatic Back-and-Forth Reasoning
Franke, M., & Jäger, G. (2014) · Palgrave Macmillan UK eBooks · doi:10.1057/9781137333285_7 · Cited by 31
Self-citationA game-theoretic account of implicature
Parikh, P. (1992) · Cited by 29
Are we Bayesian referring expression generators
Albert Gatt, A., van Gompel, R. P. G., van Deemter, K., & Krahmer, E. (2013) · Research portal (Tilburg University) · Cited by 22
Jäger, G. (2013) · Erkenntnis · doi:10.1007/s10670-013-9462-3 · Cited by 20
Benz, A. (2011) · Journal of Logic Language and Information · doi:10.1007/s10849-011-9149-6 · Cited by 18
Testing a rational account of pragmatic reasoning: The case of spatial language
Carstensen, A., Kon, E., & Regier, T. (2014) · eScholarship (California Digital Library) · Cited by 12
Typical use of quantifiers: A probabilistic speaker model
Franke, M. (2014) · eScholarship (California Digital Library) · Cited by 10
Self-citationScalar implicatures and iterated admissibility
Pavan, S. (2013) · Linguistics and Philosophy · doi:10.1007/s10988-013-9138-0 · Cited by 9
Context, scale structure, and statistics in the interpretation of positive-form adjectives
Lassiter, D., & Goodman, N. D. (2015) · Proceedings from Semantics and Linguistic Theory · doi:10.3765/salt.v0i0.2658 · Cited by 8
Overspecification and the Cost of Pragmatic Reasoning about Referring Expressions
Baumann, P., Clark, B., & Kaufmann, S. (2014) · eScholarship (California Digital Library) · Cited by 7
Explaining Quantity Implicatures
van Rooij, R., & de Jager, T. (2012) · Journal of Logic Language and Information · doi:10.1007/s10849-012-9163-3 · Cited by 6
Some arguments are probably valid: Syllogistic reasoning as communication
Tessler, M. H., & Goodman, N. D. (2014) · eScholarship (California Digital Library) · Cited by 5
Research works printed but not linked: 6
- P Baumann, 2014, Proceedings of CogSci
- JT Kao, 2014, Proceedings of CogSci
- C Qing, 2014, Proceedings of CogSci, 1204
- MH Tessler, 2014, Proceedings of CogSci
- M Franke, 2014, Proceedings of CogSci, 487
- M Franke, 2014, Semantics, Pragmatics and the Case of Scalar Implicatures. Palgrave Studies in Pragmatics Language and Cognition, 170
Cited by
Pragmatic Language Interpretation as Probabilistic Inference
Goodman, N. D., & Frank, M. C. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.08.005 · Cited by 594
“More recent studies [26], however, showed some evidence of deeper recursion for a subpopulation of participants (approx.”
Pragmatic reasoning through semantic inference
Bergen, L., Lévy, R., & Goodman, N. D. (2016) · Semantics and Pragmatics · doi:10.3765/sp.9.20 · Cited by 211
Noveck, I. (2018) · The Oxford Handbook of Psycholinguistics · doi:10.1017/9781316027073 · Cited by 160
The Oxford Handbook of Experimental Semantics and Pragmatics
Cummins, C., & Katsos, N. (2019) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198791768.001.0001 · Cited by 129
Experimental Pragmatics: The making of a cognitive science
Noveck, I. A. (2018) · HAL · Cited by 105
Metadata
- Authors
- Franke, M., & Degen, J.
- Affiliations
- University of Tübingen (DE)
- Stanford University (US)
- Corresponding author
- Franke, Michael
- Degen, Judith
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-05-05
- Volume
- 11
- Issue
- 5
- Pages
- e0154854
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4858259
- OpenAlex
- W2346168649
- Cited by
- 115
- References
- 108
- Citations by year
- 2026: 4 · 2025: 3 · 2024: 11 · 2023: 2 · 2022: 6 · 2021: 13 · 2020: 3 · 2019: 41 · 2018: 24 · 2017: 4 · 2016: 4
- FWCI
- 154.90
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- Deutsche Forschungsgemeinschaft (ZUK 63, DFG Schwerpunktprogramm 1727)
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (Early Postdoc.Mobility Award)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
- Social Sciences › Psychology › Developmental and Educational Psychology › Child and Animal Learning Development
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Categorization, perception, and language
- Keywords
- reference games, individual differences, theory of mind, quantity implicature, Bayesian model selection, population heterogeneity, pragmatic reasoning, comprehension, language use
- MeSH terms
- Bayes Theorem, Decision Making, Games, Experimental, Humans, Models, Statistical, Comprehension
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.