The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model
Open accessThe DOI matches its Crossref record; 76 of the 107 references it cites were found in a registry or an index, and 22 are not checked yet; 42 of the 120 sources its claims cite were checked for support.
Abstract
We model pragmatic mechanisms of referential communication and categorization in a multi-agent framework of emergent communication. Pragmatic theories and experimental work predict that speakers consider the context in their choice of referring expressions. In addition to this context-based reasoning, utility-based pragmatic reasoning about the listener’s likely interpretation of an utterance influences the speaker’s production choices. We aim to investigate these two factors and their role in referring expression generation and categorization in a computational model of language emergence and language use. We model communication in interaction and consider an efficiency tradeoff between speaker and listener utilities. Our results show that an emerging language becomes more effective, ambiguous, and efficient when speakers and listeners communicate in a shared context. This is achieved by an efficient tradeoff between production and comprehension where languages can afford to be simpler in production when they are sufficiently informative in context, placing more burden on the listener’s side. We further demonstrate that incorporating utility-based pragmatics, as modeled with the Rational Speech Acts framework, improves the linguistic efficiency of language use only in languages that emerged with a shared context between interlocutors, but not in languages that emerged without such contextual information during training. We conclude that context-based pragmatics plays a role in referential communication and categorization by shaping an emerging language. Efficient reference in a communicative situation can benefit especially from utility-based pragmatics if the language that is being used has emerged in context. This might suggest that utility-based pragmatics hinges on mechanisms that naturally emerge when context is available during the evolution of a language. In summary, we show that human-like language and category systems emerge as an optimized tradeoff between speaker and listener needs in interaction, i.e., under efficiency considerations of simplicity and informativeness.
Verified 2026-11-19 · 22 references not checked yet
Overview
CoChat Summary
TL;DR
In an emergent communication model with concept-level reference games, a shared context between speaker and listener produced more ambiguous, shorter, and more efficient languages that generalized better. Adding Rational Speech Acts (RSA) utility-based pragmatics improved efficiency mainly for languages that had emerged with shared context, not for context-unaware languages.
Key findings
- Context-aware agents achieved higher validation accuracy than context-unaware agents (validation accuracies ≥0.96 vs. ≥0.86; main effect of condition M = −0.03, CrI [−0.04, −0.02], pd = 100%, 0% in ROPE). Test accuracies were also higher in the context-aware condition across datasets (e.g., D(3,4): 0.93 ± 0.02 vs. 0.79 ± 0.02).
- Context-aware languages were shorter and more ambiguous: messages in the context-unaware condition tended to be at least six symbols longer, and entropy-based scores were higher in the context-unaware condition for NMI (M = 0.14, CrI [0.13, 0.16]), effectiveness (M = 0.22, CrI [0.20, 0.25]), and consistency (M = 0.04, CrI [0.03, 0.05]), all pd = 100%, 0% in ROPE.
- In Experiment 2, RSA shortened messages and improved lexicon-level efficiency especially for context-aware agents: context-unaware generic-concept messages averaged 10.5 symbols and specific-concept messages 17.2 symbols; with RSA these dropped to 5.5 and 10. Context-aware messages averaged 3.6 symbols for generic and 3.9 for specific concepts, with RSA reducing lengths by about 1 symbol.
- RSA did not improve test accuracy for context-unaware agents and sometimes worsened it (e.g., D(3,16): 0.95 ± 0.04 without RSA vs. 0.72 ± 0.03 with RSA), while context-aware + RSA remained high (e.g., D(3,16): 0.96 ± 0.07 vs. 0.95 ± 0.09). RSA reduced lexicon sizes mainly for context-aware agents (interaction M = −41.42, CrI [−58.62, −24.36], pd = 100%, 2.00% in ROPE), and message distributions became more Zipf-like as more pragmatic mechanisms were available.
Methods
- Multi-agent emergent communication model using a concept-level reference game: a speaker sends a symbolic message about a target concept, and a listener labels objects as target or distractor. Agents were implemented with feed-forward layers and GRUs (hidden size 128), trained with batch size 16, learning rate 0.001, Gumbel-Softmax, a message-length cost, and early stopping in the EGG framework.
- Six symbolic datasets D(n,k) with 3–5 attributes taking 4–16 values; concepts ranged from generic to specific, and contexts ranged from coarse to fine. Data were split 60% train, 20% validation, 20% test; game size was 10 targets and 10 distractors, with scaling factor 10.
- Experiment 1 compared context-unaware speakers with context-aware speakers sharing context with listeners. Experiment 2 used trained models at inference and compared context-unaware, context-unaware + RSA, context-aware, and context-aware + RSA speakers. RSA was a level-1 speaker using an internal listener model and a length cost. Results are means and standard deviations over five runs; Bayesian hierarchical models were fit in R with brms and bayestestR (2000 iterations, 1000 warm-up; all Rhat < 1.01, ESS > 1000).
Limitations
- Stated in the paper: RSA reasoning was implemented only for the speaker at inference, not for the listener, because listener-side inference over possible world states would be computationally costly. RSA agents could only use messages that had been produced during training, so they could not generate novel shorter messages even when more efficient. The authors also state they cannot draw inferences about listener-side processing costs from these simulations.
- Stated in the paper: the RSA framework presupposes a small fixed set of possible messages, which is cognitively implausible and intractable in full form; the model approximates this by using training utterances.
- Evident from the text: the findings come from symbolic toy datasets and artificial neural-network agents, not human language users, and are based on five runs per condition across six controlled datasets. The context-aware condition uses a specific shared-context manipulation in which speaker and listener distractors differ from targets in the same concept-defining attributes, so generalization to richer natural communicative settings is not directly tested.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures











Verification
Claims the paper makes
24 Valid · 2 Not valid · 36 Not checked · 195 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“If there is a communicative need to talk about subcategories in an environment, then the category system will be more fine-grained than if there is no such need [6].”
§ The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model > Introduction
What the sources say
- Partly supportedcites [6] T Regier, 2016, Languages Support Efficient Communication about the Environment: Words for Snow Revisited, PLoS One sourceExperimental — accuracy not yet measured
The Introduction predicts that fine-grained categories will tend to appear where distinctions are frequently referenced, and the Conclusions say local communicative needs can shape category systems. The claim’s unqualified “will” overstates that tendency as an inevitable outcome. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“It has been proposed that, due to cognitive constraints, both languages and category systems tend to find an optimal tradeoff between simplicity and informativeness [6,8,16–20].”
§ The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model > Introduction
What the sources say
- Supportedcites [6] T Regier, 2016, Languages Support Efficient Communication about the Environment: Words for Snow Revisited, PLoS One source
The Introduction describes a general hypothesis that language is shaped by a need for informative, precise communication with minimal effort. It applies that tradeoff to semantic category systems, explaining that finer categories provide more information but require more effort to store and process. The claim is framed as a proposal, consistent with the paper’s framing as a hypothesis. (LLM, conf=0.88)
- Not checked — no full text of the sourcecites [8] J Winters, 2018, Contextual predictability shapes signal autonomy, Cognition source
- Not checked — no full text of the sourcecites [16] ST Piantadosi, 2012, The communicative function of ambiguity in language, Cognition source
- Not matchedcites [17] GK Zipf, 1949
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites [18] C Kemp, 2012, Kinship Categories Across Languages Reflect General Communicative Principles, Science source
- Not checked — no full text of the sourcecites [19] E Rosch, 1976, Basic objects in natural categories, Cognitive Psychology source
- Not checked — no full text of the sourcecites [20] E Rosch, 1978, Principles of Categorization source
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Modeling pragmatic agents based on neural network architectures can help to scale the applicability of the RSA framework [56,65–67].”
§ The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model > Introduction > Theoretical and empirical background
What the sources say
- Not matchedcites [56] White J, Mu J, Goodman ND. Learning to refer informatively by amortizing pragmatic reasoning. In: Proceedings of the 42nd Annual Meeting of the Cognitive Science Society (CogSci), 2020. 994–1000. https://www.cognitivesciencesociety.org/cogsci20/papers/0175/0175.pdf
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites [65] Andreas J, Klein D. Reasoning about Pragmatics with Neural Listeners and Speakers. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016. https://doi.org/10.18653/v1/d16-1125 source
The Introduction identifies derived pragmatic models as rational speech acts models and says their application has been restricted to simple reference games. It then states that the paper’s goal is a derived model that scales to real-world datasets without domain engineering, using neural modules for its listener and speaker. (LLM, conf=0.91)
- Supportedcites [66] W Monroe, 2017, Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding, Transactions of the Association for Computational Linguistics source
The Introduction describes a scalable RSA-based model whose literal agents are neural networks that scale to large datasets with diverse utterances. The “Listener-based listener” section says replacing literal agents with RNN-based agents addresses obstacles to applying RSA to large datasets. (LLM, conf=0.94)
- Partly supportedcites [67] Cohn-Gordon R, Goodman ND, Potts C. An Incremental Iterated Response Model of Pragmatics. In: Proceedings of the Society for Computation in Linguistics (SCiL). New York City, New York; 2019. p. 81–90. sourceExperimental — accuracy not yet measured
Section 5 (p. 8) says an incremental RSA approach offers a scalable alternative if its incremental semantics is learned. Section 2.4 (p. 5) clarifies that the model presented here is not itself more tractable, and our copy does not specify neural-network architectures as the basis for scaling it. (LLM, conf=0.86)
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The emergent communication framework is well-suited to model the role of pragmatics in communication [36,41,56,57].”
§ The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model > Introduction > Theoretical and empirical background
What the sources say
- Partly supportedcites [36] Lazaridou A, Hermann KM, Tuyls K, Clark S. Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input. In: International Conference on Learning Representations (ICML); 2018. https://openreview.net/forum?id=HJGv1Z-AW sourceExperimental — accuracy not yet measured
Section 3.2 (p. 5) uses referential games to test how the distribution of distractors affects emergent messages, supporting the framework’s ability to model a contextual aspect of communication. The paper does not explicitly evaluate whether the framework is well-suited to modeling pragmatics more broadly. (LLM, conf=0.77)
- Not matchedcites [41] Mu J, Goodman N. Emergent Communication of Generalizations. Advances in Neural Information Processing Systems (NeurIPS). 2021. p. 17994–8007.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [56] White J, Mu J, Goodman ND. Learning to refer informatively by amortizing pragmatic reasoning. In: Proceedings of the 42nd Annual Meeting of the Cognitive Science Society (CogSci), 2020. 994–1000. https://www.cognitivesciencesociety.org/cogsci20/papers/0175/0175.pdf
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [57] Fang F, Sinha K, Goodman ND, Potts C, Kreiss E. In: Proceedings of the Annual Meeting of the Cognitive Science Society (CogSci), 2022. 1796–803. https://escholarship.org/uc/item/9kn7n6qb
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Instead of the computationally costly Bayesian inference in the classical RSA paradigm, neural-network-based approaches for RSA-based referring expression generation approximate the inference, for example by directly taking into account the interpretation of an internal listener model [56,57,65–67].”
§ The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model > Introduction > Theoretical and empirical background
What the sources say
- Not matchedcites [56] White J, Mu J, Goodman ND. Learning to refer informatively by amortizing pragmatic reasoning. In: Proceedings of the 42nd Annual Meeting of the Cognitive Science Society (CogSci), 2020. 994–1000. https://www.cognitivesciencesociety.org/cogsci20/papers/0175/0175.pdf
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [57] Fang F, Sinha K, Goodman ND, Potts C, Kreiss E. In: Proceedings of the Annual Meeting of the Cognitive Science Society (CogSci), 2022. 1796–803. https://escholarship.org/uc/item/9kn7n6qb
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites [65] Andreas J, Klein D. Reasoning about Pragmatics with Neural Listeners and Speakers. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016. https://doi.org/10.18653/v1/d16-1125 source
Section 3.4 (Reasoning model; page not specified) says exhaustive optimization over possible descriptions is impractical and describes a sampling-based alternative: a neural speaker proposes descriptions, which are scored using an internal neural listener’s predicted interpretation. This supports the attributed example of approximate inference for referring-expression generation. (LLM, conf=0.91)
- Supportedcites [66] W Monroe, 2017, Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding, Transactions of the Association for Computational Linguistics source
The Introduction says the pragmatic speaker produces utterances based on a literal RNN listener. Section 4.3 (page unavailable) defines that speaker using the neural listener’s interpretation probabilities and describes sampling utterances to make the otherwise costly normalization tractable. (LLM, conf=0.91)
- Partly supportedcites [67] Cohn-Gordon R, Goodman ND, Potts C. An Incremental Iterated Response Model of Pragmatics. In: Proceedings of the Society for Computation in Linguistics (SCiL). New York City, New York; 2019. p. 81–90. sourceExperimental — accuracy not yet measured
Section 2.1 (p. 2) says the RSA speaker reasons about a listener’s interpretation, and Section 2.3 (p. 4) describes efficient, word-by-word referring-expression generation. But the paper presents an incremental RSA model, not a neural-network-based approach that approximates classical RSA inference; Section 2.4 (p. 5) describes its predictions as distinct from the global model’s. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
127 more claims · 2 with their cited sources checked
Create a free account to see all 257 claims and to verify the 13 references still open.
- Probably not a substantial claim. 65
- It may state two separate claims as one. 76
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model > Abstract
“incorporating utility-based pragmatics, as modeled with the Rational Speech Acts framework, improves the linguistic efficiency of language use only in languages that emerged with a shared context between interlocutors, but not in languages that emerged without such contextual information during training.”
§ The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model > Discussion
“Despite these performance issues, context-unaware speakers made more efficient use of their language when they reasoned about their interlocutor’s likely interpretation of their message and observed a cost on longer messages in the context-unaware + RSA simulations.”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model > General discussion > The role of utility-based pragmatics in language use
“We found that both context-based and utility-based pragmatics improve generalization performance.”
§ The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model > Discussion > Experiment 2: Utility-based pragmatic reasoning
“Regarding utility-based pragmatic reasoning, accuracies in the context-unaware condition did not improve with RSA but RSA seemed to worsen performance of context-unaware agents. In the context-aware condition, however, utility-based pragmatics lead to similar performance.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model > Discussion
“This translates into a high consistency score, i.e., low meaning uncertainty.”
§ The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model > Results > Experiment 1: Context-based pragmatics
“If the consistency score is high, this means that there is low signal uncertainty.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model > Methods > Dataset
“coarse contexts, in which one attribute is shared between targets and distractors.”
§ The role of pragmatic mechanisms in referential communication and categorization: An emergent communication model > Methods > Dataset
“B: An example for a specific concept LARGE BLUE CIRCLE in a coarse context (no attribute shared).”
Citations
Cites
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 1×
“Both our speaker and listener agents are implemented as neural networks with feed-forward (dense) layers for embedding the input objects and Gated-Recurrent-Unit (GRU) [87] cells to encode (speaker) or decode (listener) the messages.”
Grice, H. P. (1975) · Speech Acts · doi:10.1163/9789004368811_003 · Cited by 23,221
brms : An R Package for Bayesian Multilevel Models Using Stan
Bürkner, P.-C. (2017) · Journal of Statistical Software · doi:10.18637/jss.v080.i01 · Cited by 9,476
Basic objects in natural categories
Rosch, E., Mervis, C. B., Gray, W. D., Johnson, D. M., & Boyes-Braem, P. (1976) · Cognitive Psychology · doi:10.1016/0010-0285(76)90013-x · Cited by 5,643 · cited 5×
“A category system with few categories is optimized for cognitive economy because fewer categories need to be learned and stored in memory [19,20].”
InfluentialPresumptive Meanings: The Theory of Generalized Conversational Implicature
Levinson, S. C. (2000) · doi:10.7551/mitpress/5526.001.0001 · Cited by 2,404
Makowski, D., Ben-Shachar, M., & Lüdecke, D. (2019) · Journal of Open Source Software · doi:10.21105/joss.01541 · Cited by 1,483 · cited 1×
“For the statistical analyses, we used R version 4.4.1 [72] and the R-packages brms [73] and bayestestR [74].”
InfluentialConceptual pacts and lexical choice in conversation.
Brennan, S. E., & Clark, H. H. (1996) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037//0278-7393.22.6.1482 · Cited by 1,457 · cited 1×
“On the other hand, the field of pragmatics is concerned with the role of context and utility-based pragmatic mechanisms in concrete communicative situations and repeated turns, e.g., [26,30,33,81].”
Do humans have two systems to track beliefs and belief-like states
Apperly, I. A., & Butterfill, S. A. (2009) · Psychological Review · doi:10.1037/a0016923 · Cited by 1,261 · cited 1×
“Contrary to this assumption, pragmatic reasoning, such as the derivation of implicatures, reasoning about an interlocutor and general Theory of Mind, is typically believed to bear a cognitive cost and be effortful and slow [61,82].”
Learning to Communicate with Deep Multi-Agent Reinforcement Learning
Foerster, J. N., Assael, Y. M., de Freitas, N., & Whiteson, S. (2016) · arXiv · doi:10.48550/arxiv.1605.06676 · Cited by 1,217 · cited 1×
“Our approach is based on the integration of Deep Neural Networks and Multi-agent Reinforcement Learning to study the interactions of neural-network-based agent which allows for improved scalability and flexibility [35,53–55].”
Relevance: Communication and Cognition
Meara, P., Sperber, D., & Wilson, D. (1989) · The Modern Language Review · doi:10.2307/3731158 · Cited by 1,189
Rosch, E. (1988) · Readings in Cognitive Science · doi:10.1016/b978-1-4832-1446-7.50028-5 · Cited by 946 · cited 4×
“As predicted by Rosch et al. [19,20], the category system of a language can afford to be simpler when the meanings of categories are interpreted in context.”
InfluentialPredicting pragmatic reasoning in language games
Frank, M. C., & Goodman, N. D. (2012) · Science · doi:10.1126/science.1218633 · Cited by 868
Word lengths are optimized for efficient communication
Piantadosi, S. T., Tily, H., & Gibson, E. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1012551108 · Cited by 695 · cited 1×
“Second, word lengths are determined by their frequency, where the most frequent words tend to be the shortest in a language [91].”
Some utterances are underinformative: The onset and time course of scalar inferences
Bott, L., & Noveck, I. A. (2004) · Journal of Memory and Language · doi:10.1016/j.jml.2004.05.006 · Cited by 665
The communicative function of ambiguity in language
Piantadosi, S. T., Tily, H., & Gibson, E. (2012) · Cognition · doi:10.1016/j.cognition.2011.10.004 · Cited by 628 · cited 10×
“This principle was proposed as the Principle of Least Effort by Zipf [17] and the idea has been also taken up later [16,21].”
InfluentialPragmatic Language Interpretation as Probabilistic Inference
Goodman, N. D., & Frank, M. C. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.08.005 · Cited by 594 · cited 2×
“We implement S RSA as a speaker that maximizes a utility function following standard RSA models [33,34,88]: where U is the utility function, typically defined as U ( m | w ) = log p L 0 ( w | m ) [88].”
Least effort and the origins of scaling in human language
Cancho, R. F. I., & Solé, R. V. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0335980100 · Cited by 588
Achieving incremental semantic interpretation through contextual representation
Sedivy, J. C., K. Tanenhaus, M., Chambers, C. G., & Carlson, G. N. (1999) · Cognition · doi:10.1016/s0010-0277(99)00025-6 · Cited by 553 · cited 1×
“…pragmatics, it has been found that speakers are more likely to choose a more informative referring expression, i.e., a subordinate referring expression (“dalmatian”), or a modified nominal referring expression (“tall glass”), if an object of the same type is present in the context [42–45].”
Kirby, S. (2001) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.918430 · Cited by 530 · cited 2×
“Focusing on computational modeling, the iterated learning computational framework and game theoretic approaches were very influential [46–48].”
How Efficiency Shapes Human Language
Gibson, E., Futrell, R., Piantadosi, S. P., Dautriche, I., Mahowald, K., Bergen, L., & Levy, R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.003 · Cited by 485 · cited 1×
“This adds to the argument on the efficiency of ambiguity made in the literature and adds evolutionary plausibility to it [14,16,77].”
Common ground at the understanding of demonstrative reference
Clark, H. H., Schreuder, R., & Buttrick, S. (1983) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(83)90189-5 · Cited by 445 · cited 1×
“The amount of shared context between two interlocutors is thought to influence communication in pragmatics theories and has been called mutual cognitive environment [70] or common ground [71].”
A Self-Organizing Spatial Vocabulary
Steels, L. (1995) · Artificial Life · doi:10.1162/artl.1995.2.3.319 · Cited by 422 · cited 1×
“…research tradition following Steel’s Talking Heads experiments focuses on the joint evolution of conceptualizations, i.e., mappings between perception and meanings, and lexicalizations, i.e., mappings between meanings and words, in the communicative interaction between autonomous agents [47,51,52].”
Building Large Monolingual Dictionaries at the Leipzig Corpora Collection: From 100 to 200 Languages
Goldhahn, D., Eckart, T., & Quasthoff, U. (2012) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4e5i253k8v5a · Cited by 345 · cited 1×
“We use the Leipzig Corpora Collection [93] and specifically the News copora with 10K words for English (2024) and Arabic (2022) including words and their frequencies in the corpus.”
Kinship Categories Across Languages Reflect General Communicative Principles
Kemp, C., & Regier, T. (2012) · Science · doi:10.1126/science.1218811 · Cited by 284
Multi-Agent Cooperation and the Emergence of (Natural) Language
Lazaridou, A., Peysakhovich, A., & Baroni, M. (2016) · arXiv · doi:10.48550/arxiv.1612.07182 · Cited by 231 · cited 4×
“Our research makes use of an emergent communication model [35–38].”
InfluentialSAPIR, E. (1912) · American Anthropologist · doi:10.1525/aa.1912.14.2.02a00020 · Cited by 224 · cited 2×
“The idea that language shapes cognition in this way has become popular as the (weak) Sapir-Whorf hypothesis, or linguistic relativity hypothesis [1,2].”
The Rational Speech Act Framework
Degen, J. (2022) · Annual Review of Linguistics · doi:10.1146/annurev-linguistics-031220-010811 · Cited by 205 · cited 1×
“Observing the effect of RSA-based reasoning in context is in line with experimental paradigms and theoretical models in the RSA literature [83].”
Word meanings across languages support efficient communication
Regier, T., Kemp, C., & Kay, P. (2015) · The Handbook of Language Emergence · doi:10.1002/9781118346136.ch11 · Cited by 196
Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols
Havrylov, S., & Titov, I. (2017) · arXiv · doi:10.48550/arxiv.1705.11192 · Cited by 195
Efficient compression in color naming and its evolution
Zaslavsky, N., Kemp, C., Regier, T., & Tishby, N. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1800521115 · Cited by 178 · cited 1×
“These findings connect to the literature that investigates the influence of efficiency on the evolution of category systems such as color and kinship in language [13,23,84].”
Kanwal, J., Smith, K., Culbertson, J., & Kirby, S. (2017) · Cognition · doi:10.1016/j.cognition.2017.05.001 · Cited by 176 · cited 2×
“This is in line with the efficient communication hypothesis which predicts that human languages are optimized for efficient communication, a claim that has gained traction in many subfields of linguistics such as language evolution [7,8], pragmatics [9–12] and more general proposals [6,13–17].”
Steels, L. (2015) · doi:10.17169/fudocs_document_000000022455 · Cited by 149
Degen, J., & Tanenhaus, M. K. (2015) · Cognitive Science · doi:10.1111/cogs.12227 · Cited by 147
Languages Support Efficient Communication about the Environment: Words for Snow Revisited
Regier, T., Carstensen, A., & Kemp, C. (2016) · PLOS ONE · doi:10.1371/journal.pone.0151138 · Cited by 136 · cited 6×
“They relate this finding to the lower communicative need to talk about snow and ice in these regions [6].”
InfluentialRubio-Fernández, P. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00153 · Cited by 131 · cited 1×
“But recent proposals have shown that the redundant expressions provide information that helps listeners to identify the target referent, e.g., by mentioning atypical features or providing salient information in complex scenes that facilitate visual search [9,10,26,31,32].”
Bayesian intractability is not an ailment that approximation can cure
Kwisthout, J., Wareham, T., & van Rooij, I. (2011) · Cognitive Science · doi:10.1111/j.1551-6709.2011.01182.x · Cited by 122 · cited 1×
“This is cognitively implausible and quickly becomes intractable [56,64].”
Reasoning in Reference Games: Individual- vs. Population-Level Probabilistic Modeling
Franke, M., & Degen, J. (2016) · PLOS ONE · doi:10.1371/journal.pone.0154854 · Cited by 115 · cited 5×
“We model an RSA speaker following the Rational Speech Acts (RSA) framework [33,34,88].”
InfluentialReasoning about Pragmatics with Neural Listeners and Speakers
Andreas, J., & Klein, D. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1125 · Cited by 115 · cited 2×
“Instead of the computationally costly Bayesian inference in the classical RSA paradigm, neural-network-based approaches for RSA-based referring expression generation approximate the inference, for example by directly taking into account the interpretation of an internal listener model [56,57,65–67].”
Ambiguity Avoidance is Overrated
Wasow, T. (2015) · De Gruyter eBooks · doi:10.1515/9783110403589-003 · Cited by 108
Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding
Monroe, W., Hawkins, R. X. D., Goodman, N. D., & Potts, C. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00064 · Cited by 94
When redundancy is useful: A Bayesian approach to “overinformative” referring expressions
Degen, J., Hawkins, R. D., Graf, C., Kreiss, E., & Goodman, N. D. (2020) · Psychological Review · doi:10.1037/rev0000186 · Cited by 90 · cited 6×
“This can be explained by the idea that speakers take into account the listener’s needs and want to guide the listener toward identifying the correct referent as quickly as possible [26,32].”
InfluentialLanguages adapt to their contextual niche
WINTERS, J., KIRBY, S., & SMITH, K. (2015) · Language and Cognition · doi:10.1017/langcog.2014.35 · Cited by 87 · cited 6×
“This measure is used to capture the synonymy of messages [38,50].”
InfluentialEmergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input
Lazaridou, A., Hermann, K. M., Tuyls, K., & Clark, S. (2018) · arXiv · doi:10.48550/arxiv.1804.03984 · Cited by 86
Anti-efficient encoding in emergent communication
Chaabouni, R., Kharitonov, E., Dupoux, E., & Baroni, M. (2019) · arXiv · doi:10.48550/arxiv.1905.12561 · Cited by 85 · cited 1×
“We investigate the following two relationships proposed by Zipf’s law [17,75] which seem to be a common principle of natural languages on the lexicon level [90–92].”
Categories, Concepts, and Conceptual Development
Sloutsky, V. M., & Deng, W. (. (2019) · Language, Cognition and Neuroscience · doi:10.1080/23273798.2017.1391398 · Cited by 77 · cited 2×
“This makes languages that emerged in the context-aware setting likely more similar to natural languages, where concepts are thought to be organized in a hierarchical structure [85].”
Ambiguity, Accessibility, and a Division of Labor for Communicative Success
Ferreira, V. S. (2008) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)00006-6 · Cited by 73 · cited 2×
“Ambiguity in human languages has been related to pragmatic considerations [16,28,29].”
Contextual predictability shapes signal autonomy
Winters, J., Kirby, S., & Smith, K. (2018) · Cognition · doi:10.1016/j.cognition.2018.03.002 · Cited by 54 · cited 6×
“For example, it has been shown that abstract references, i.e., references that can be used to refer to more than one object, emerge in coarse contexts [39,50] and that contextual predictability shapes signal autonomy [8].”
InfluentialCo-evolution of language and agents in referential games
Dagan, G., Hupkes, D., & Bruni, E. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.260 · Cited by 43 · cited 1×
“On the one hand, the emergent communication paradigm is used to investigate the emergence of novel communication systems between agents and general mechanisms of language evolution, e.g., [35,79,80].”
Self-citationEGG: a toolkit for research on Emergence of lanGuage in Games
Kharitonov, E., Chaabouni, R., Bouchacourt, D., & Baroni, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations · doi:10.18653/v1/d19-3010 · Cited by 40 · cited 1×
“…joint binary cross entropy (BCE) loss maximizing the probability that the listener agent correctly identifies the target objects in their input: where ˆ m ∼ p S ( m | T S , D S ) and p L ( y We implement the game using the EGG framework [86] developed for implementing emergent communication games.”
Communicating artificial neural networks develop efficient color-naming systems
Chaabouni, R., Kharitonov, E., Dupoux, E., & Baroni, M. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016569118 · Cited by 39
Rubio-Fernandez, P., Mollica, F., & Jara-Ettinger, J. (2021) · Journal of Experimental Psychology: General · doi:10.1037/xge0000963 · Cited by 37
Fider, N. A., & Komarova, N. L. (2019) · Palgrave Communications · doi:10.1057/s41599-019-0341-7 · Cited by 28 · cited 1×
“Evidence for the efficient communication hypothesis has been found in the context of kinship [18] and color [22–25] systems.”
Usage context influences the evolution of overspecification in iterated learning
Tinits, P., Nölle, J., & Hartmann, S. (2017) · Journal of Language Evolution · doi:10.1093/jole/lzx011 · Cited by 28 · cited 2×
“Focusing on experimental studies, artificial language learning experiments have been used to study how the context in which target objects need to be communicated, can explain properties of an emerging lexicon [8,39,49,50].”
Rational over-specification in visually-situated comprehension and production
Tourtouri, E. N., Delogu, F., Sikos, L., & Crocker, M. W. (2019) · Journal of Cultural Cognitive Science · doi:10.1007/s41809-019-00032-6 · Cited by 27 · cited 2×
“But recent proposals have shown that the redundant expressions provide information that helps listeners to identify the target referent, e.g., by mentioning atypical features or providing salient information in complex scenes that facilitate visual search [9,10,26,31,32].”
(2021) · doi:10.48550/arxiv.2105.09867 · Cited by 27
An Incremental Iterated Response Model of Pragmatics
Cohn-Gordon, R., Goodman, N., & Potts, C. (2019) · University of Massachusetts Amherst · doi:10.7275/cprc-8x17 · Cited by 26
The evolution of color naming reflects pressure for efficiency: Evidence from the recent past
Zaslavsky, N., Garvin, K., Kemp, C., Tishby, N., & Regier, T. (2022) · Journal of Language Evolution · doi:10.1093/jole/lzac001 · Cited by 23
Signalling under Uncertainty: Interpretative Alignment without a Common Prior
Brochhagen, T. (2020) · The British Journal for the Philosophy of Science · doi:10.1093/bjps/axx058 · Cited by 22 · cited 3×
“This adds to the argument on the efficiency of ambiguity made in the literature and adds evolutionary plausibility to it [14,16,77].”
Language games meet multi-agent reinforcement learning: A case study for the naming game
Eecke, P. V., Beuls, K., Ekila, J. B., & Rădulescu, R. (2022) · Journal of Language Evolution · doi:10.1093/jole/lzad001 · Cited by 18
Rubio-Fernandez, P. (2021) · Humanities and Social Sciences Communications · doi:10.1057/s41599-021-00818-6 · Cited by 16 · cited 2×
“But recent proposals have shown that the redundant expressions provide information that helps listeners to identify the target referent, e.g., by mentioning atypical features or providing salient information in complex scenes that facilitate visual search [9,10,26,31,32].”
Learning about others: Modeling social inference through ambiguity resolution
Achimova, A., Scontras, G., Stegemann–Philipps, C., Lohmann, J., & Butz, M. V. (2021) · Cognition · doi:10.1016/j.cognition.2021.104862 · Cited by 14 · cited 1×
“Ambiguity in human languages has been related to pragmatic considerations [16,28,29].”
The Interactions of Rational, Pragmatic Agents Lead to Efficient Language Structure and Use
Peloquin, B. N., Goodman, N. D., & Frank, M. C. (2020) · Topics in Cognitive Science · doi:10.1111/tops.12489 · Cited by 11
PyArabic: A Python package for Arabic text
Zerrouki, T. (2023) · The Journal of Open Source Software · doi:10.21105/joss.04886 · Cited by 9
Towards a Theory of Communicative Efficiency in Human Languages
Levshina, N. (2018) · Zenodo · doi:10.5281/zenodo.1542856 · Cited by 7
Mutual influence between language and perception in multi-agent communication games
Ohmer, X., Marino, M., Franke, M., & König, P. (2022) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1010658 · Cited by 6
Self-citationConversation as a Site of Category Learning and Category Use
Barr, D. J., & Kronmüller, E. (2006) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(06)47006-7 · Cited by 4 · cited 2×
“One instance of the claim that language shapes cognition is that conversation shapes category structure [3].”
Emergence of Hierarchical Reference Systems in Multi-agent Communication
Ohmer, X., Duda, M., & Bruni, E. (2022) · arXiv · doi:10.48550/arxiv.2203.13176 · Cited by 4
Self-citationZarrieß, S., Buschmeier, H., Han, T., & Schüz, S. (2021) · Proceedings of the 14th International Conference on Natural Language Generation · doi:10.18653/v1/2021.inlg-1.41 · Cited by 3
Agents generalize to novel levels of abstraction by using adaptive linguistic strategies
Kobrock, K., Ohmer, X. I., Bruni, E., & Gotzner, N. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.455 · Cited by 3 · cited 1×
“It differs from the context-aware condition presented in previous works [37,68,69] by assuming a shared context between the interlocutors.”
Self-citationWhy do objects have many names? A study on word informativeness in language use and lexical systems
Gualdoni, E., & Boleda, G. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.1009
The Efficiency of Ambiguity: Abstract References Emerge in Coarse ContextsPreprint
Briotto, A., Kobrock, K., & Bruni, E. (2025) · doi:10.31234/osf.io/3btvj_v2
Self-citationBotoko Ekila, J., Nevens, J., Verheyen, L., Beuls, K., & Van Eecke, P. (2024) · International Joint Conference on Autonomous Agents and Multiagent Systems · doi:10.65109/dlfy8542
Context Shapes Emergent Communication about Concepts at Different Levels of Abstraction
Kobrock, K., Ohmer, X. I., Bruni, E., & Gotzner, N. (2024) · Proceedings of the Language Resources and Evaluation Conference · doi:10.63317/4odxjsoap6qe · cited 9×
“To prevent agents from communicating about lower-level features that are not relevant from a human perspective, we use a concept-level reference game for training [37,41].”
InfluentialSelf-citationLanguage evolution 'in silico'
Brochhagen, T. (2024) · Mètode: Revista de difusió de la investigació de la Universitat de Valencia · doi:10.7203/metode.15.27692
Ekila, J. B., Verheyen, L., Nevens, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.08461
The Efficiency of Ambiguity: Abstract References Emerge in Coarse Contexts
Briotto, A., Kobrock, K., & Bruni, E. (2024) · doi:10.31234/osf.io/3btvj
Self-citationThe evolution of color naming reflects pressure for efficiency: Evidence from the recent past
Zaslavsky, N., Garvin, K., Kemp, C., Tishby, N., & Regier, T. (2021) · doi:10.1101/2021.11.03.467047
Language, thought, and reality: selected writings
Whorf, B. (1956)
UnresolvedIntroduction
Boas, F. (1911)
UnresolvedKaplan L. Inuit or Eskimo: Which name to use?. https://www.uaf.edu/anlc/research-and-resources/resources/archives/inuit_or_eskimo.php 2025.
(n.d.)
UnresolvedMentioning atypical properties of objects is communicatively efficient
Kreiss, E. (2017) · Cognitive Science
UnresolvedTowards a Theory of Communicative Efficiency in Human Languages
Levshina, N. (2018)
UnresolvedHuman behavior and the principle of least effort: An introduction to human ecology
Zipf, G. (1949)
UnresolvedLazaridou A, Peysakhovich A, Baroni M. Multi-Agent Cooperation and the Emergence of (Natural) Language. In: International Conference on Learning Representations (ICML); 2017. https://openreview.net/forum?id=Hk8N3Sclg
(n.d.)
UnresolvedLazaridou A, Hermann KM, Tuyls K, Clark S. Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input. In: International Conference on Learning Representations (ICML); 2018. https://openreview.net/forum?id=HJGv1Z-AW
(n.d.)
UnresolvedOhmer X, Duda M, Bruni E. Emergence of Hierarchical Reference Systems in Multi-agent Communication. In: Proceedings of the 29th International Conference on Computational Linguistics, 2022. 5689–706. https://aclanthology.org/2022.coling-1.501
(n.d.)
UnresolvedHawkins RD, Franke M, Smith K, Goodman ND. Emerging abstractions: Lexical conventions are shaped by communicative context. In: Proceedings of the 40th annual conference of the cognitive science society (CogSci), 2018. 463–8. http://cocolab.stanford.edu/papers/HawkinsEtAl2018-Cogsci.pdf
(n.d.)
UnresolvedMu J, Goodman N. Emergent Communication of Generalizations. Advances in Neural Information Processing Systems (NeurIPS). 2021. p. 17994–8007.
(n.d.)
UnresolvedKobrock K, Uhlemann C, Gotzner N. In: Proceedings of the Annual Meeting of the Cognitive Science Society, 2024. 518–25. https://escholarship.org/uc/item/31n5d3p6
(n.d.)
UnresolvedGraf C, Degen J, Hawkins RD, Goodman ND. Animal, dog, or dalmatian? Level of abstraction in nominal referring expressions. In: Proceedings of the 38th Annual Conference of the Cognitive Science Society (CogSci); 2016. p. 2261–2266. http://cocolab.stanford.edu/papers/GrafEtAl2016-Cogsci.pdf
(n.d.)
UnresolvedEvaluating explanations for referential context effects: Evidence for Gricean mechanisms in online language interpretation
Sedivy, J. (2005)
UnresolvedConvention
Lewis, D. (1969)
UnresolvedFoerster J, Assael IA, de Freitas N, Whiteson S. Learning to Communicate with Deep Multi-Agent Reinforcement Learning. In: Advances in Neural Information Processing Systems 29 (NIPS 2016), 2016. https://proceedings.neurips.cc/paper/2016/hash/c7635bfd99248a2cdef8249ef7bfbef4-Abstract.html
(n.d.)
UnresolvedHavrylov S, Titov I. Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols. In: Advances in Neural Information Processing Systems, 2017. https://papers.nips.cc/paper_files/paper/2017/hash/70222949cc0db89ab32c9969754d4758-Abstract.html
(n.d.)
UnresolvedWhite J, Mu J, Goodman ND. Learning to refer informatively by amortizing pragmatic reasoning. In: Proceedings of the 42nd Annual Meeting of the Cognitive Science Society (CogSci), 2020. 994–1000. https://www.cognitivesciencesociety.org/cogsci20/papers/0175/0175.pdf
(n.d.)
UnresolvedFang F, Sinha K, Goodman ND, Potts C, Kreiss E. In: Proceedings of the Annual Meeting of the Cognitive Science Society (CogSci), 2022. 1796–803. https://escholarship.org/uc/item/9kn7n6qb
(n.d.)
UnresolvedBaumann P, Clark B, Kaufmann S. In: Proceedings of the Annual Meeting of the Cognitive Science Society, 2014. 1898–903.
(n.d.)
UnresolvedToward a new taxonomy for pragmatic inference: Q-based and R-based implicature
Horn, L. (1984)
UnresolvedCohn-Gordon R, Goodman ND, Potts C. An Incremental Iterated Response Model of Pragmatics. In: Proceedings of the Society for Computation in Linguistics (SCiL). New York City, New York; 2019. p. 81–90.
(n.d.)
UnresolvedR Core Team. R: A Language and Environment for Statistical Computing. 2024.
(n.d.)
Unresolvedbrms : An R Package for Bayesian Multilevel Models Using Stan
Bürkner, P.-C. (2017) · J Stat Soft
UnresolvedThe psycho-biology of language
Zipf, G. (1935)
UnresolvedSignaling under Uncertainty
Brochhagen, T. (2018)
UnresolvedChoi E, Lazaridou A, Freitas N d. In: Vancouver, Canada, 2018. https://openreview.net/forum?id=rknt2Be0-
(n.d.)
UnresolvedLanguage evolution ’in silico’: From large-scale data to artificial agents creating languages from scratch
Brochhagen, T. (2025) · Metode Science Studies Journal
UnresolvedJang E, Gu S, Poole B. Categorical Reparameterization with Gumbel-Softmax. In: International Conference on Learning Representations (ICML); 2017. https://openreview.net/forum?id=rkE3y85ee
(n.d.)
UnresolvedChaabouni R, Kharitonov E, Dupoux E, Baroni M. Anti-efficient encoding in emergent communication. In: Advances in Neural Information Processing Systems, 2019. https://proceedings.neurips.cc/paper/2019/hash/31ca0ca71184bbdb3de7b20a51e88e90-Abstract.html
(n.d.)
Unresolved
Cited by
On the Optimality of Kinship Naming: an Information-theoretic Approach
Le, P.-H., Lindeman, M., & Alhama, R. G. (2025) · doi:10.48550/arxiv.2511.19120
Metadata
- Authors
- Kobrock, K., Ohmer, X., Bruni, E., & Gotzner, N.
- Affiliations
- Osnabrück University (DE)
- Corresponding author
- Kobrock, Kristina
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2026-05-26
- Volume
- 22
- Issue
- 5
- Pages
- e1014326
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC13252840
- OpenAlex
- W4411393719
- Cited by
- 0
- References
- 94
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Update notices
- new_version (2026-06-10)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- Funders
- Deutsche Forschungsgemeinschaft (DFG-GRK 2340, 441607011)
- Topics
- Social Sciences › Cultural Studies › Language and cultural evolution
- Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Categorization, perception, and language
- Keywords
- emergent communication, referential communication, categorization, referring expressions, Rational Speech Act, language emergence, shared context, multi-agent simulation
- MeSH terms
- Communication, Computer Simulation, Humans, Language, Speech, Speech Perception, Computational Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.