The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Large-scale replication study reveals a limit on probabilistic prediction in language comprehension
Open accessAbstract
Do people routinely pre-activate the meaning and even the phonological form of upcoming words? The most acclaimed evidence for phonological prediction comes from a 2005 Nature Neuroscience publication by DeLong, Urbach and Kutas, who observed a graded modulation of electrical brain potentials (N400) to nouns and preceding articles by the probability that people use a word to continue the sentence fragment (‘cloze’). In our direct replication study spanning 9 laboratories (N=334), pre-registered replication-analyses and exploratory Bayes factor analyses successfully replicated the noun-results but, crucially, not the article-results. Pre-registered single-trial analyses also yielded a statistically significant effect for the nouns but not the articles. Exploratory Bayesian single-trial analyses showed that the article-effect may be non-zero but is likely far smaller than originally reported and too small to observe without very large sample sizes. Our results do not support the view that readers routinely pre-activate the phonological form of predictable words.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Random effects structure for confirmatory hypothesis testing: Keep it maximal
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013) · Journal of Memory and Language · doi:10.1016/j.jml.2012.11.001 · Cited by 10,328
Self-citationMixed-effects modeling with crossed random effects for subjects and items
Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008) · Journal of Memory and Language · doi:10.1016/j.jml.2007.12.005 · Cited by 8,540
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 2×
“In this line of thought, language is yet another domain in which the brain acts as a prediction machine (Clark, 2013; Van Berkum, 2010; see also Friston, 2005, 2010; Summerfield and de Lange, 2014), hard-wired to continuously match sensory inputs with top-down, grammatical or probabilistic…”
BackgroundPower failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1×
“Inspired by recent demonstrations for the need for large subject-samples in psychology and neuroscience research (Button et al., 2013; Open Science Collaboration, 2015), our replication spanned nine laboratories each with a sample size equal to or greater than that of the original.”
BackgroundWhatever next? Predictive brains, situated agents, and the future of cognitive science
Clark, A. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12000477 · Cited by 5,924
Reading Senseless Sentences: Brain Potentials Reflect Semantic Incongruity
Kutas, M., & Hillyard, S. A. (1980) · Science · doi:10.1126/science.7350657 · Cited by 4,888
A theory of cortical responses.
Friston, K. (2005) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2005.1622 · Cited by 4,578
Kutas, M., & Federmeier, K. D. (2011) · Annual Review of Psychology · doi:10.1146/annurev.psych.093008.131123 · Cited by 4,377
The role of knowledge in discourse comprehension: A construction-integration model.
Kintsch, W. (1988) · Psychological Review · doi:10.1037/0033-295x.95.2.163 · Cited by 4,363
Removing electroencephalographic artifacts by blind source separation
Jung, T., Makeig, S., Humphries, C., Lee, T.-W., McKeown, M. J., Iragui, V. J., & Sejnowski, T. J. (2000) · Psychophysiology · doi:10.1017/s0048577200980259 · Cited by 3,334
Foundations of Language: Brain, Meaning, Grammar, Evolution
Jackendoff, R. (2002) · doi:10.1093/acprof:oso/9780198270126.001.0001 · Cited by 3,244
“Cloze Procedure”: A New Tool for Measuring Readability
Taylor, W. L. (1953) · Journalism Quarterly · doi:10.1177/107769905303000401 · Cited by 2,332
The language-as-fixed-effect fallacy: A critique of language statistics in psychological research
Clark, H. H. (1973) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(73)80014-3 · Cited by 2,222
Brain potentials during reading reflect word expectancy and semantic association
Kutas, M., & Hillyard, S. A. (1984) · Nature · doi:10.1038/307161a0 · Cited by 2,218
Foundations of Language: Brain, Meaning, Grammar, Evolution
Jackendoff, R. S. (2002) · Cited by 1,858
Event-related brain potentials elicited by syntactic anomaly
Osterhout, L., & Holcomb, P. J. (1992) · Journal of Memory and Language · doi:10.1016/0749-596x(92)90039-z · Cited by 1,856
Incremental interpretation at verbs: restricting the domain of subsequent reference
Altmann, G. T. M., & Kamide, Y. (1999) · Cognition · doi:10.1016/s0010-0277(99)00059-1 · Cited by 1,735
A cortical network for semantics: (de)constructing the N400
Lau, E. F., Phillips, C., & Poeppel, D. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2532 · Cited by 1,700
An integrated theory of language production and comprehension
Pickering, M. J., & Garrod, S. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12001495 · Cited by 1,569
The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant
Gelman, A., & Stern, H. (2006) · The American Statistician · doi:10.1198/000313006x152649 · Cited by 1,407 · cited 1×
“Crucially, the difference between significant and not-significant itself may not be significant (Gelman and Stern, 2006), log-transformation does not yield higher t-values consistently across laboratories, does not necessarily improve model fit, and does not yield higher t-values or improve model…”
BackgroundThe temporal structure of spoken language understanding
Marslen-Wilson, W., & Tyler, L. K. (1980) · Cognition · doi:10.1016/0010-0277(80)90015-3 · Cited by 1,295
DeLong, K. A., Urbach, T. P., & Kutas, M. (2005) · Nature Neuroscience · doi:10.1038/nn1504 · Cited by 1,240
How to get statistically significant effects in any ERP experiment (and why you shouldn't)
Luck, S. J., & Gaspelin, N. (2017) · Psychophysiology · doi:10.1111/psyp.12639 · Cited by 1,195
What do we mean by prediction in language comprehension?
Kuperberg, G. R., & Jaeger, T. F. (2016) · Language, Cognition and Neuroscience · doi:10.1080/23273798.2015.1102299 · Cited by 1,087
Anticipating Upcoming Words in Discourse: Evidence From ERPs and Reading Times.
Van Berkum, J. J. A., Brown, C. M., Zwitserlood, P., Kooijman, V., & Hagoort, P. (2005) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.31.3.443 · Cited by 930
A Rose by Any Other Name: Long-Term Memory Structure and Sentence Processing
Federmeier, K. D., & Kutas, M. (1999) · Journal of Memory and Language · doi:10.1006/jmla.1999.2660 · Cited by 905
Expectation in perceptual decision making: neural and computational mechanisms
Summerfield, C., & de Lange, F. P. (2014) · Nature Reviews Neuroscience · doi:10.1038/nrn3838 · Cited by 832
The Processing Nature of the N400: Evidence from Masked Priming
Brown, C., & Hagoort, P. (1993) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1993.5.1.34 · Cited by 783
Prediction during language comprehension: Benefits, costs, and ERP components
Van Petten, C., & Luka, B. J. (2012) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2011.09.015 · Cited by 777
The effect of word predictability on reading time is logarithmic
Smith, N. J., & Levy, R. (2013) · Cognition · doi:10.1016/j.cognition.2013.02.013 · Cited by 725
Do people use language production to make predictions during comprehension?
Pickering, M. J., & Garrod, S. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.12.002 · Cited by 642
Semantic Integration in Sentences and Discourse: Evidence from the N400
Berkum, J. J. A. V., Hagoort, P., & Brown, C. M. (1999) · Journal of Cognitive Neuroscience · doi:10.1162/089892999563724 · Cited by 642
The time course of visual word recognition as revealed by linear regression analysis of ERP data
Hauk, O., Davis, M. H., Ford, M., Pulvermüller, F., & Marslen-Wilson, W. D. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2005.11.048 · Cited by 586
Wicha, N. Y. Y., Moreno, E. M., & Kutas, M. (2004) · Journal of Cognitive Neuroscience · doi:10.1162/0898929041920487 · Cited by 516
The P-chain: relating sentence production and its disorders to comprehension and acquisition
Dell, G. S., & Chang, F. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2012.0394 · Cited by 496
Connolly, J. F., & Phillips, N. A. (1994) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1994.6.3.256 · Cited by 449
The N400 as a function of the level of processing
CHWILLA, D. J., BROWN, C. M., & HAGOORT, P. (1995) · Psychophysiology · doi:10.1111/j.1469-8986.1995.tb02956.x · Cited by 430
The locus of the effects of sentential-semantic context in spoken-word processing
Zwitserlood, P. (1989) · Cognition · doi:10.1016/0010-0277(89)90013-9 · Cited by 429
Four central questions about prediction in language processing
Huettig, F. (2015) · Brain Research · doi:10.1016/j.brainres.2015.02.014 · Cited by 405 · cited 1×
“However, predictions in such studies are scaffolded onto already-available visual context, and therefore do not measure purely pre-activation, but perhaps re-activation of word information previously activated by the visual object itself (Huettig, 2015).”
BackgroundSelf-citationTime course of word identification and semantic integration in spoken language.
Van Petten, C., Coulson, S., Rubin, S., Plante, E., & Parks, M. (1999) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.25.2.394 · Cited by 374
Incrementality and Prediction in Human Sentence Processing
Altmann, G. T. M., & Mirković, J. (2009) · Cognitive Science · doi:10.1111/j.1551-6709.2009.01022.x · Cited by 369
Tanner, D., Morgan‐Short, K., & Luck, S. J. (2015) · Psychophysiology · doi:10.1111/psyp.12437 · Cited by 354
Bilinguals reading in their second language do not predict upcoming words as native readers do
Martin, C. D., Thierry, G., Kuipers, J.-R., Boutonnet, B., Foucart, A., & Costa, A. (2013) · Journal of Memory and Language · doi:10.1016/j.jml.2013.08.001 · Cited by 348
Dissociating N400 Effects of Prediction from Association in Single-word Contexts
Lau, E. F., Holcomb, P. J., & Kuperberg, G. R. (2013) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00328 · Cited by 284
The balance between memory and unification in semantics: A dynamic account of the N400
Baggio, G., & Hagoort, P. (2011) · Language and Cognitive Processes · doi:10.1080/01690965.2010.542671 · Cited by 222
Lexical integration: Sequential effects of syntactic and semantic information
Friederici, A. D., Steinhauer, K., & Frisch, S. (1999) · Memory & Cognition · doi:10.3758/bf03211539 · Cited by 207
Saccade size control in reading: Evidence for the linguistic control hypothesis
O’Regan, K. (1979) · Perception & Psychophysics · doi:10.3758/bf03213829 · Cited by 206
The statistical significance filter leads to overoptimistic expectations of replicability
Vasishth, S., Mertzen, D., Jäger, L. A., & Gelman, A. (2018) · Journal of Memory and Language · doi:10.1016/j.jml.2018.07.004 · Cited by 204
Getting ahead: forward models and their place in cognitive architecture
Pickering, M. J., & Clark, A. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.05.006 · Cited by 190
Goals and strategies influence lexical prediction during sentence comprehension
Brothers, T. A., Swaab, T. Y., & Traxler, M. J. (2016) · Journal of Memory and Language · doi:10.1016/j.jml.2016.10.002 · Cited by 180
Discourse-Based Word Anticipation During Language Processing: Prediction or Priming?
Otten, M., & Van Berkum, J. J. A. (2008) · Discourse Processes · doi:10.1080/01638530802356463 · Cited by 176
Wlotko, E. W., & Federmeier, K. D. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.054 · Cited by 172
Nieuwland, M. S., & Martin, A. E. (2012) · Cognition · doi:10.1016/j.cognition.2011.09.001 · Cited by 170
Self-citationPredicting form and meaning: Evidence from brain potentials
Ito, A., Corley, M., Pickering, M. J., Martin, A. E., & Nieuwland, M. S. (2016) · Journal of Memory and Language · doi:10.1016/j.jml.2015.10.007 · Cited by 170
Self-citationWlotko, E. W., & Federmeier, K. D. (2015) · Cortex · doi:10.1016/j.cortex.2015.03.014 · Cited by 150
The core and beyond in the language-ready brain
Hagoort, P. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.01.048 · Cited by 150
Berkum, V. E. C., & Alferes, J. J. (2010) · The Italian Journal of Linguistics · Cited by 145
Great expectations: Specific lexical anticipation influences the processing of spoken language
Otten, M., Nieuwland, M. S., & Berkum, J. J. A. V. (2007) · BMC Neuroscience · doi:10.1186/1471-2202-8-89 · Cited by 141
Self-citationHow to quantify the evidence for the absence of a correlation
Wagenmakers, E., Verhagen, J., & Ly, A. (2015) · Behavior Research Methods · doi:10.3758/s13428-015-0593-0 · Cited by 133
Ito, A., Martin, A. E., & Nieuwland, M. S. (2016) · Language Cognition and Neuroscience · doi:10.1080/23273798.2016.1242761 · Cited by 123
Self-citationAn Event-Related Brain Potential Analysis of Visual Word Priming Effects
Brown, C. M., Peter Hagoort, A., & Chwilla, D. J. (2000) · Brain and Language · doi:10.1006/brln.1999.2284 · Cited by 122
Does working memory capacity affect the ability to predict upcoming words in discourse?
Otten, M., & Berkum, J. J. A. V. (2009) · Brain Research · doi:10.1016/j.brainres.2009.07.042 · Cited by 111
Nieuwland, M. S. (2015) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/xlm0000173 · Cited by 105
Self-citationA Direct Comparison of N400 Effects of Predictability and Incongruity in Adjective-Noun Combination
Lau, E. F., Namyst, A. M., Fogel, A. R., & Delgado, T. (2016) · Collabra · doi:10.1525/collabra.40 · Cited by 80
Ito, A., Martin, A. E., & Nieuwland, M. S. (2017) · Language Cognition and Neuroscience · doi:10.1080/23273798.2017.1323112 · Cited by 46
Self-citationNieuwland, M. S., Politzer‐Ahles, S., Heyselaar, E., Segaert, K., Darley, E., Kazanina, N., Wolfsthurn, S. V. G. Z., Bartolozzi, F., Kogan, V. V., Ito, A., et al. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/111807 · Cited by 40
Self-citation
Cited by
A hierarchy of linguistic predictions during natural language comprehension
Heilbron, M., Armeni, K., Schoffelen, J.-M., Hagoort, P., & de Lange, F. P. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2201968119 · Cited by 343
Hasson, U., Egidi, G., Marelli, M., & Willems, R. M. (2018) · Cognition · doi:10.1016/j.cognition.2018.06.018 · Cited by 157
Interference patterns in subject-verb agreement and reflexives revisited: A large-sample study
Jäger, L. A., Mertzen, D., Van Dyke, J. A., & Vasishth, S. (2020) · Journal of Memory and Language · doi:10.1016/j.jml.2019.104063 · Cited by 142
A hierarchy of linguistic predictions during natural language comprehension
(2020) · doi:10.1101/2020.12.03.410399 · Cited by 63
Localizing syntactic predictions using recurrent neural network grammars
Brennan, J. R., Dyer, C., Kuncoro, A., & Hale, J. T. (2020) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2020.107479 · Cited by 58
Metadata
- Authors
- Nieuwland, M. S., Politzer-Ahles, S., Heyselaar, E., Segaert, K., Darley, E., Kazanina, N., Von Grebmer Zu Wolfsthurn, S., Bartolozzi, F., Kogan, V., Ito, A., Mézière, D., Barr, D. J., Rousselet, G. A., Ferguson, H. J., Busch-Moreno, S., Fu, X., Tuomainen, J., Kulakova, E., Husband, E. M., Donaldson, D. I., Kohút, Z., Rueschemeyer, S.-A., & Huettig, F.
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2018-04-03
- Volume
- 7
- ISSN
- 2050-084X (ISSN-L)
- OpenAlex
- W2796350793
- Cited by
- 299
- References
- 67
- FWCI
- 18.40
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.