On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior
Open accessThe DOI matches its OpenAlex record; 7 of the 7 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 2×
“Recently, however, neural network models such as Long Short-Term Memory Recurrent Neural Networks (LSTM-RNNs; Elman, 1990; Hochreiter & Schmidhuber, 1997) and Transformers (Vaswani et al., 2017) have set new standards in natural language processing, achieving state-of-the-art per-plexity results.”
BackgroundBuilding a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
A probabilistic earley parser as a psycholinguistic model
Hale, J. (2001) · Second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies 2001 - NAACL '01 · doi:10.3115/1073336.1073357 · Cited by 1,150 · cited 1×
“A large body of evidence suggests that humans are expectation-based language processors, insofar as real-time language comprehension involves making predictions about upcoming material (Levy, 2008; Hale, 2001).”
BackgroundNeural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1162 · Cited by 974
Neural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2015) · arXiv · doi:10.48550/arxiv.1508.07909 · Cited by 950
The effect of word predictability on reading time is logarithmic
Smith, N. J., & Levy, R. (2013) · Cognition · doi:10.1016/j.cognition.2013.02.013 · Cited by 725
Recurrent Neural Network Grammars
Dyer, C., Kuncoro, A., Ballesteros, M., & Smith, N. A. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1024 · Cited by 517 · cited 1×
“• Recurrent Neural Network Grammars (RNNGs; Dyer, Kuncoro, Ballesteros, & Smith, 2016) model the joint probability of a sequence of words as well as its syntactic structure.”
Background
Cited by
Yenduri, G., Ramalingam, M., Selvi, G. C., Supriya, Y., Srivastava, G., Maddikunta, P. K. R., Raj, G. D., Jhaveri, R. H., Prabadevi, B., Wang, W., et al. (2024) · IEEE Access · doi:10.1109/access.2024.3389497 · Cited by 596
The neural architecture of language: Integrative modeling converges on predictive processing
Schrimpf, M., Blank, I. A., Tuckute, G., Kauf, C., Hosseini, E. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2105646118 · Cited by 517
Over-reliance on English hinders cognitive science
Blasi, D. E., Henrich, J., Adamou, E., Kemmerer, D., & Majid, A. (2022) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2022.09.015 · Cited by 396
“It has been claimed that Language Models that are ef fi cient in solving this task are also better at predicting human behavior (e.g., reading times [82]) and, vice versa, those with the most brain-like representations perform the best at this task [83], all of which reinforces the idea that the…”
A hierarchy of linguistic predictions during natural language comprehension
Heilbron, M., Armeni, K., Schoffelen, J.-M., Hagoort, P., & de Lange, F. P. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2201968119 · Cited by 343
The neural architecture of language: Integrative modeling converges on predictive processing
(2020) · doi:10.1101/2020.06.26.174482 · Cited by 129
Oh, B.-D., & Schuler, W. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00548 · Cited by 107
“, 2017) models have dominated many NLP tasks, both large pre-trained and smaller ‘trained-from-scratch’ Transformer-based LMs have also been evaluated as models of processing difficulty (Wilcox et al., 2020; Hao et al., 2020; Merkx and Frank, 2021; Schrimpf et al., 2021).”
BackgroundHosseini, E. A., Schrimpf, M., Zhang, Y., Bowman, S., Zaslavsky, N., & Fedorenko, E. (2024) · Neurobiology of Language · doi:10.1162/nol_a_00137 · Cited by 63
“…network (ANN) language models were recently shown to be effective at predicting human brain activity and behavior during language processing (Caucheteux & King, 2022; Gauthier & Levy, 2019; Goldstein et al., 2022; Jain & Huth, 2018; Schrimpf et al., 2021; Toneva & Wehbe, 2019; Wilcox et al., 2020).”
BackgroundLanguage models are not naysayers: an analysis of language models on negation benchmarks
Truong, T. H., Baldwin, T., Verspoor, K., & Cohn, T. (2023) · arXiv · doi:10.48550/arxiv.2306.08189 · Cited by 34
Lampinen, A. (2024) · Computational Linguistics · doi:10.1162/coli_a_00525 · Cited by 32
“Some recent work has explored whether networks trained on natural language can learn syntactic structures (e.g. Futrell et al., 2019; Wilcox et al., 2020), and in particular recursive dependencies (Wilcox et al.”
Lampinen, A. K. (2022) · arXiv · doi:10.48550/arxiv.2210.15303 · Cited by 32
Steuer, J., Mosbach, M., & Klakow, D. (2023) · Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning · doi:10.18653/v1/2023.conll-babylm.12 · Cited by 14
Bigger is not always better: The importance of human-scale language modeling for psycholinguistics
Wilcox, E. G., Hu, M. Y., Mueller, A., Warstadt, A., Choshen, L., Zhuang, C., Williams, A., Cotterell, R., & Linzen, T. (2025) · Journal of Memory and Language · doi:10.1016/j.jml.2025.104650 · Cited by 9
Self-citationWhen Can AI Models Explain Learning? Validity Criteria for AI as Cognitive Models in Education
Wang, P., Viberg, O., Law, E. L.-C., & Greiff, S. (2026) · Educational Psychology Review · doi:10.1007/s10648-026-10202-y · Cited by 3
“Emerging studies report partial alignment between certain LLM behaviors and human patterns in domains such as reading time prediction (Wilcox et al., 2020 ), reasoning heuristics (Dasgupta et al., 2024 ), and performance on standardized academic assess - ments (OpenAI et al., 2024 ).”
Metadata
- Authors
- Wilcox, E., Gauthier, J., Hu, J., Qian, P., & Lévy, R.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2020-06-02
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2006.01912
- OpenAlex
- W3033254023
- Cited by
- 105
- References
- 7
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.