Rational integration of noisy evidence and prior semantic expectations in sentence interpretation
Open accessThe DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
Sentence processing theories typically assume that the input to our language processing mechanisms is an error-free sequence of words. However, this assumption is an oversimplification because noise is present in typical language use (for instance, due to a noisy environment, producer errors, or perceiver errors). A complete theory of human sentence comprehension therefore needs to explain how humans understand language given imperfect input. Indeed, like many cognitive systems, language processing mechanisms may even be “well designed”–in this case for the task of recovering intended meaning from noisy utterances. In particular, comprehension mechanisms may be sensitive to the types of information that an idealized statistical comprehender would be sensitive to. Here, we evaluate four predictions about such a rational (Bayesian) noisy-channel language comprehender in a sentence comprehension task: ( i ) semantic cues should pull sentence interpretation towards plausible meanings, especially if the wording of the more plausible meaning is close to the observed utterance in terms of the number of edits; ( ii ) this process should asymmetrically treat insertions and deletions due to the Bayesian “size principle”; such nonliteral interpretation of sentences should ( iii ) increase with the perceived noise rate of the communicative situation and ( iv ) decrease if semantically anomalous meanings are more likely to be communicated. These predictions are borne out, strongly suggesting that human language relies on rational statistical inference over a noisy channel.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“a human’s ease of processing a sentence is influenced by that sentence’s probability (66–68)”
From Embers of autoregression show how large language models are shaped by the problem they are trained to solve (2024) · cites this paper as [68]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 48,949
Binary Codes capable of currecting deletions, insertions, and reversals
Levenshtein, V. I. (1965) · Soviet physics. Doklady · Cited by 15,130
Information theory, inference, and learning algorithms
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5949 · Cited by 6,362
Information Theory, Inference, and Learning Algorithms
(2004) · Kybernetes · doi:10.1108/03684920410534506 · Cited by 4,937
Reading Senseless Sentences: Brain Potentials Reflect Semantic Incongruity
Kutas, M., & Hillyard, S. A. (1980) · Science · doi:10.1126/science.7350657 · Cited by 4,888
Integration of Visual and Linguistic Information in Spoken Language Comprehension
Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995) · Science · doi:10.1126/science.7777863 · Cited by 2,929
Linguistic complexity: locality of syntactic dependencies
Gibson, E. (1998) · Cognition · doi:10.1016/s0010-0277(98)00034-1 · Cited by 2,537
Self-citationEnglish Verb Classes and Alternations: A Preliminary Investigation
Tenny, C., & Levin, B. (1995) · Language · doi:10.2307/415968 · Cited by 2,299
Expectation-based syntactic comprehension
Levy, R. (2008) · Cognition · doi:10.1016/j.cognition.2007.05.006 · Cited by 2,202
Event-related brain potentials elicited by syntactic anomaly
Osterhout, L., & Holcomb, P. J. (1992) · Journal of Memory and Language · doi:10.1016/0749-596x(92)90039-z · Cited by 1,856
The sausage machine: A new two-stage parsing model
Frazier, L., & Fodor, J. D. (1978) · Cognition · doi:10.1016/0010-0277(78)90002-1 · Cited by 1,400
The syntactic positive shift (sps) as an erp measure of syntactic processing
Hagoort, P., Brown, C., & Groothusen, J. (1993) · Language and Cognitive Processes · doi:10.1080/01690969308407585 · Cited by 1,260
Semantic Influences On Parsing: Use of Thematic Role Information in Syntactic Ambiguity Resolution
Trueswell, J. C., Tanenhaus, M. K., & Garnsey, S. M. (1994) · Journal of Memory and Language · doi:10.1006/jmla.1994.1014 · Cited by 1,175
A probabilistic earley parser as a psycholinguistic model
Hale, J. (2001) · Second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies 2001 - NAACL '01 · doi:10.3115/1073336.1073357 · Cited by 1,150
Good-Enough Representations in Language Comprehension
Ferreira, F., Bailey, K. G. D., & Ferraro, V. (2002) · Current Directions in Psychological Science · doi:10.1111/1467-8721.00158 · Cited by 1,147
Reflections of the Environment in Memory
Anderson, J. R., & Schooler, L. J. (1991) · Psychological Science · doi:10.1111/j.1467-9280.1991.tb00174.x · Cited by 1,085
Neural mechanisms of language comprehension: Challenges to syntax
Kuperberg, G. R. (2007) · Brain Research · doi:10.1016/j.brainres.2006.12.063 · Cited by 1,041
Word learning as Bayesian inference.
Xu, F., & Tenenbaum, J. B. (2007) · Psychological Review · doi:10.1037/0033-295x.114.2.245 · Cited by 1,033
Continuous speech recognition by statistical methods
Jelinek, F. (1976) · Proceedings of the IEEE · doi:10.1109/proc.1976.10159 · Cited by 920
Expect the Unexpected: Event-related Brain Response to Morphosyntactic Violations
Coulson, S., King, J. W., & Kutas, M. (1998) · Language and Cognitive Processes · doi:10.1080/016909698386582 · Cited by 920
The misinterpretation of noncanonical sentences
Ferreira, F. (2003) · Cognitive Psychology · doi:10.1016/s0010-0285(03)00005-7 · Cited by 871
Aylett, M., & Turk, A. (2004) · Language and Speech · doi:10.1177/00238309040470010201 · Cited by 781
A probabilistic model of lexical and syntactic access and disambiguation
Jurafsky, D. (1996) · Cognitive Science · doi:10.1016/s0364-0213(99)80005-6 · Cited by 714
Event-Related Brain Potentials Elicited by Failure to Agree
Osterhout, L., & Mobley, L. A. (1995) · Journal of Memory and Language · doi:10.1006/jmla.1995.1033 · Cited by 705
The independence of combinatory semantic processing: Evidence from event-related potentials
Kim, A., & Osterhout, L. (2005) · Journal of Memory and Language · doi:10.1016/j.jml.2004.10.002 · Cited by 702
Structure and limited capacity in verbal working memory: A study with event-related potentials
Kolk, H. H. J., Chwilla, D. J., van Herten, M., & Oor, P. J. W. (2003) · Brain and Language · doi:10.1016/s0093-934x(02)00548-5 · Cited by 522
Cue validity and sentence interpretation in English, German, and Italian
MacWhinney, B., Bates, E., & Kliegl, R. (1984) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(84)90093-8 · Cited by 498
Sequential ideal-observer analysis of visual discriminations.
Geisler, W. S. (1989) · Psychological Review · doi:10.1037//0033-295x.96.2.267 · Cited by 383
Perception of speech reflects optimal use of probabilistic speech cues
Clayards, M., Tanenhaus, M. K., Aslin, R. N., & Jacobs, R. A. (2008) · Cognition · doi:10.1016/j.cognition.2008.04.004 · Cited by 350
The need for quantitative methods in syntax and semantics research
Gibson, E., & Fedorenko, E. (2013) · Language and Cognitive Processes · doi:10.1080/01690965.2010.515080 · Cited by 297
Self-citationEye movement evidence that readers maintain and act on uncertainty about past linguistic input
Levy, R., Bicknell, K., Slattery, T., & Rayner, K. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0907664106 · Cited by 288
Ten years of the rational analysis of cognition
Chater, N., & Oaksford, M. (1999) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(98)01273-x · Cited by 267
A computational theory of human linguistic processing: memory limitations and processing breakdown
Gibson, E. A. F. (1991) · Cited by 249
Self-citationvan Herten, M., Chwilla, D. J., & Kolk, H. H. J. (2006) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2006.18.7.1181 · Cited by 226
Münte, T. F., Heinze, H.-J., Matzke, M., Wieringa, B. M., & Johannes, S. (1998) · Neuropsychologia · doi:10.1016/s0028-3932(97)00119-x · Cited by 208
Altering Context Speech Rate Can Cause Words to Appear or Disappear
Dilley, L. C., & Pitt, M. A. (2010) · Psychological Science · doi:10.1177/0956797610384743 · Cited by 187
Crowdsourcing and language studies: the new generation of linguistic data
Munro, R., Bethard, S. J., Kuperman, V., Lai, V. T., Melnick, R., Potts, C., Schnoebelen, T., & Tily, H. J. (2010) · Max Planck Institute for Plasma Physics · Cited by 183
Ferreira, F., Christianson, K., & Hollingworth, A. (2001) · Journal of Psycholinguistic Research · doi:10.1023/a:1005290706460 · Cited by 174
A noisy-channel model of rational human sentence comprehension under uncertain input
Levy, R. (2008) · Proceedings of the Conference on Empirical Methods in Natural Language Processing - EMNLP '08 · doi:10.3115/1613715.1613749 · Cited by 171
Neural correlates of processing syntactic, semantic, and thematic relationships in sentences
Kuperberg, G. R., Caplan, D., Sitnikova, T., Eddy, M., & Holcomb, P. J. (2006) · Language and Cognitive Processes · doi:10.1080/01690960500094279 · Cited by 142
Blackwell, A., & Bates, E. (1995) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1995.7.2.228 · Cited by 138
A Bayesian approach to the evolution of perceptual and cognitive systems
Geisler, W. (2003) · Cognitive Science · doi:10.1016/s0364-0213(03)00009-0 · Cited by 124
An ERP study of syntactic processing in English and nonsense sentences
Yamada, Y., & Neville, H. J. (2006) · Brain Research · doi:10.1016/j.brainres.2006.10.052 · Cited by 72
Sentence comprehension in aphasia in two clear case-marking languages
MacWhinney, B., Osmán-Sági, J., & Slobin, D. I. (1991) · Brain and Language · doi:10.1016/0093-934x(91)90154-s · Cited by 56
A maximum likelihood procedure for the analysis of group and individual data in aphasia research*1
Bates, E. A., McDonald, J., MacWhinney, B., & Appelbaum, M. (1991) · Brain and Language · doi:10.1016/0093-934x(91)90126-l · Cited by 27
Frank, M. C., Arnon, I., Tily, H. J., & Goldwater, S. J. (2010) · Proceedings of the Annual Meeting of the Cognitive Science Society · Cited by 16
(1993) · Americanae (AECID Library)
Research works printed but not linked: 5
- ACKNOWLEDGMENTS. Thanks to Roger Levy, Melissa Kline, Kyle Mahowald, Harry Tily, Nathaniel Smith, audiences at Architectures and Mechanisms for Language Processing 2011 in Paris, France, and the CUNY Conference on Human Sentence Processing 2012 in New York. Thanks also to Eunice Lim and Peter Graff who helped us extensively in running experiment 3 in our laboratory. And a special thanks to Ev Fedorenko, who gave us very detailed comments and suggestions on this work at multiple stages of this project. This work was supported by National Science Foundation Grant 0844472 from the Linguistics Program (to E.G.).
- 10.1207/s15516709cog2002_1 · doi:10.1207/s15516709cog2002_1 · Paper page
- D Marr Vision (Freeman, New York, 1982).
- 10.1037/0033-295X.96.2.267 · doi:10.1037/0033-295x.96.2.267 · Paper page
- 10.1207/s15516709cog2703_3 · doi:10.1207/s15516709cog2703_3 · Paper page
Cited by
The Now-or-Never bottleneck: A fundamental constraint on language
Christiansen, M. H., & Chater, N. (2016) · Behavioral and Brain Sciences · doi:10.1017/s0140525x1500031x · Cited by 733
Predicting while comprehending language: A theory and review.
Pickering, M. J., & Gambi, C. (2018) · Psychological Bulletin · doi:10.1037/bul0000158 · Cited by 518
The neural architecture of language: Integrative modeling converges on predictive processing
Schrimpf, M., Blank, I. A., Tuckute, G., Kauf, C., Hosseini, E. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2105646118 · Cited by 517
How Efficiency Shapes Human Language
Gibson, E., Futrell, R., Piantadosi, S. P., Dautriche, I., Mahowald, K., Bergen, L., & Levy, R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.003 · Cited by 485
“Another theory for the distribution of SOV and SVO order arises from principles of robust sentence understanding through a noisy communicative channel [74, 75, 76].”
Self-citationThe language network as a natural kind within the broader landscape of the human brain
Fedorenko, E., Ivanova, A. A., & Regev, T. I. (2024) · Nature Reviews Neuroscience · doi:10.1038/s41583-024-00802-4 · Cited by 293
“For comprehension, the bottom-up processing of linguistic input (solid arrows) is complemented by top-down influences (dashed arrows) of both general knowledge and reasoning on linguistic interpretation 319–322 (Box 2), and linguistic knowledge on perceptual processes 249–251 .”
Background
Metadata
- Authors
- Gibson, E., Bergen, L., & Piantadosi, S. T.
- Affiliations
- Massachusetts Institute of Technology (US)
- Departments of aBrain and Cognitive Sciences and
- University of Rochester (US)
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2013-05
- Volume
- 110
- Issue
- 20
- Pages
- 8051-8056
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC3657782
- OpenAlex
- W2132684680
- Cited by
- 407
- References
- 49
- Citations by year
- 2026: 22 · 2025: 37 · 2024: 33 · 2023: 45 · 2022: 76 · 2021: 48 · 2020: 29 · 2019: 27 · 2018: 20 · 2017: 24 · 2016: 23 · 2015: 14 · 2014: 7 · 2013: 2
- FWCI
- 8.41
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
- Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
- Social Sciences › Psychology › Developmental and Educational Psychology › Language Development and Disorders
- Keywords
- sentence comprehension, noisy channel model, Bayesian inference, semantic expectation, prior knowledge, edit distance, indels, size principle, perceived noise, statistical language processing
- MeSH terms
- Bayes Theorem, Cognition, Communication, Humans, Language, Natural Language Processing, Psycholinguistics, Semantics, Reproducibility of Results, Comprehension
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.