Artificial neural network language models predict human brain responses to language even after a developmentally realistic amount of training
Open accessPublished version of a preprint
Abstract
Artificial neural networks have emerged as computationally plausible models of human language processing. A major criticism of these models is that the amount of training data they receive far exceeds that of humans during language learning. Here, we use two complementary approaches to ask how the models’ ability to capture human fMRI responses to sentences is affected by the amount of training data. First, we evaluate GPT-2 models trained on 1 million, 10 million, 100 million, or 1 billion words against an fMRI benchmark. We consider the 100-million-word model to be developmentally plausible in terms of the amount of training data given that this amount is similar to what children are estimated to be exposed to during the first 10 years of life. Second, we test the performance of a GPT-2 model trained on a 9-billion-token dataset to reach state-of-the-art next-word prediction performance on the human benchmark at different stages during training. Across both approaches, we find that (i) the models trained on a developmentally plausible amount of data already achieve near-maximal performance in capturing fMRI responses to sentences. Further, (ii) lower perplexity—a measure of next-word prediction performance—is associated with stronger alignment with human data, suggesting that models that have received enough training to achieve sufficiently high next-word prediction performance also acquire representations of sentences that are predictive of human fMRI responses. In tandem, these findings establish that although some training is necessary for the models’ predictive ability, a developmentally realistic amount of training (∼100 million words) may suffice.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,695 · see citing works
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1× · see citing works
“In line with increasing emphasis in the field on robustness and replicability (Button et al., 2013; Ioannidis et al., 2014; Poldrack et al., 2017; Simmons et al., 2011), we adopt two complementary approaches (Figure 1).”
MethodBuilding a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405 · see citing works
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
Pointer Sentinel Mixture ModelsPreprint
Merity, S., Xiong, C., Bradbury, J., & Socher, R. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1609.07843 · Cited by 5,051 · see citing works
Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015) · arXiv · doi:10.48550/arxiv.1506.06724 · Cited by 2,264 · see citing works
Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., & Fidler, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.11 · Cited by 2,264 · see citing works
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · Cited by 1,749 · cited 2× · see citing works
“…and amount of computing power are the main contributors to model performance after training, as measured in loss for predicting the next token (Kaplan et al., 2020), and that model size is the main contributor to model performance in predicting fMRI responses to language (Antonello et al.,…”
BackgroundResultProceedings of the 2nd International Conference on Neural Information Processing Systems
Touretzky, D. S. (1989) · Cited by 1,413 · see citing works
Scanning the horizon: towards transparent and reproducible neuroimaging research
Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafò, M. R., Nichols, T. E., Poline, J.-B., Vul, E., & Yarkoni, T. (2017) · Nature Reviews Neuroscience · doi:10.1038/nrn.2016.167 · Cited by 1,410 · see citing works
BERT Rediscovers the Classical NLP Pipeline
Tenney, I., Das, D., & Pavlick, E. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1452 · Cited by 1,307 · see citing works
Proceedings of the 3rd International Conference on Neural Information Processing Systems
Lippmann, R. P., Moody, J. E., & Touretzky, D. S. (1990) · Cited by 1,204 · see citing works
Mariani, J. (2018) · HAL · Cited by 1,151 · see citing works
The brain’s default network: updated anatomy, physiology and evolving insights
Buckner, R. L., & DiNicola, L. M. (2019) · Nature Reviews Neuroscience · doi:10.1038/s41583-019-0212-7 · Cited by 1,058 · see citing works
Discovering Event Structure in Continuous Narrative Perception and Memory
Baldassano, C., Chen, J., Zadbood, A., Pillow, J. W., Hasson, U., & Norman, K. A. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.041 · Cited by 952 · see citing works
Paradigms and processes in reading comprehension.
Just, M. A., Carpenter, P. A., & Woolley, J. D. (1982) · Journal of Experimental Psychology General · doi:10.1037//0096-3445.111.2.228 · Cited by 875 · see citing works
Topographic mapping of a hierarchy of temporal receptive windows using a narrated story
Lerner, Y., Honey, C. J., Silbert, L. J., & Hasson, U. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3684-10.2011 · Cited by 834 · see citing works
Paradigms and processes in reading comprehension.
Just, M. A., Carpenter, P. A., & Woolley, J. D. (1982) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.111.2.228 · Cited by 830 · see citing works
The effect of word predictability on reading time is logarithmic
Smith, N. J., & Levy, R. (2013) · Cognition · doi:10.1016/j.cognition.2013.02.013 · Cited by 725 · see citing works
Cortical representation of the constituent structure of sentences
Pallier, C., Devauchelle, A.-D., & Dehaene, S. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1018711108 · Cited by 691 · see citing works
New method for fMRI investigations of language: defining ROIs functionally in individual subjects
Fedorenko, E., Hsieh, P.-J., Nieto-Castañón, A., Whitfield-Gabrieli, S., & Kanwisher, N. (2010) · Journal of Neurophysiology · doi:10.1152/jn.00032.2010 · Cited by 686 · see citing works
Self-citation(2019) · doi:10.18653/v1/n19-1419 · Cited by 601 · see citing works
Functional specificity for high-level linguistic processing in the human brain
Fedorenko, E., Behr, M. K., & Kanwisher, N. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1112937108 · Cited by 595 · cited 3× · see citing works
“…selected the checkpoint with the best perplexity on the validation set and tested how well the model representations capture human neural (fMRI) responses to sentences in the language-selective network (Fedorenko et al., 2011) and human behavioral responses in a self-paced reading task (Figure 1D).”
BackgroundMethodSelf-citationHart, B. M., & Risley, T. (1992) · Developmental Psychology · doi:10.1037/0012-1649.28.6.1096 · Cited by 595 · see citing works
Dynamic reconfiguration of the default mode network during narrative comprehension
Simony, E., Honey, C. J., Chen, J., Lositsky, O., Yeshurun, Y., Wiesel, A., & Hasson, U. (2016) · Nature Communications · doi:10.1038/ncomms12141 · Cited by 588 · see citing works
A critique of pure learning and what artificial neural networks can learn from animal brains
Zador, A. M. (2019) · Nature Communications · doi:10.1038/s41467-019-11786-6 · Cited by 553 · see citing works
The neural architecture of language: Integrative modeling converges on predictive processing
Schrimpf, M., Blank, I. A., Tuckute, G., Kauf, C., Hosseini, E. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2105646118 · Cited by 517 · see citing works
Self-citationMapping the Early Language Environment Using All-Day Recordings and Automated Analysis
Gilkerson, J., Richards, J. A., Warren, S. F., Montgomery, J. K., Greenwood, C. R., Kimbrough Oller, D., Hansen, J. H. L., & Paul, T. D. (2017) · American Journal of Speech-Language Pathology · doi:10.1044/2016_ajslp-15-0169 · Cited by 472 · see citing works
Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention
Ioannidis, J. P. A., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.02.010 · Cited by 471 · see citing works
Shared computational principles for language processing in humans and deep language models
Goldstein, A., Zada, Z., Buchnik, E., Schain, M., Price, A., Aubrey, B., Nastase, S. A., Feder, A., Emanuel, D., Cohen, A., et al. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01026-4 · Cited by 447 · see citing works
Perplexity—a measure of the difficulty of speech recognition tasks
Jelinek, F., Mercer, R. L., Bahl, L. R., & Baker, J. K. (1977) · The Journal of the Acoustical Society of America · doi:10.1121/1.2016299 · Cited by 405 · see citing works
The ERP response to the amount of information conveyed by words in sentences
Frank, S. L., Otten, L. J., Galli, G., & Vigliocco, G. (2014) · Brain and Language · doi:10.1016/j.bandl.2014.10.006 · Cited by 382 · see citing works
A hierarchy of linguistic predictions during natural language comprehension
Heilbron, M., Armeni, K., Schoffelen, J.-M., Hagoort, P., & de Lange, F. P. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2201968119 · Cited by 343 · see citing works
Toward a universal decoder of linguistic meaning from brain activation
Pereira, F., Lou, B., Pritchett, B., Ritter, S., Gershman, S. J., Kanwisher, N., Botvinick, M., & Fedorenko, E. (2018) · Nature Communications · doi:10.1038/s41467-018-03068-4 · Cited by 324 · see citing works
Self-citationBrains and algorithms partially converge in natural language processing
Caucheteux, C., & King, J.-R. (2022) · Communications Biology · doi:10.1038/s42003-022-03036-1 · Cited by 318 · see citing works
What do Neural Machine Translation Models Learn about Morphology?
Belinkov, Y., Durrani, N., Dalvi, F., Sajjad, H., & Glass, J. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1080 · Cited by 307 · see citing works
DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale
Aminabadi, R. Y., Rajbhandari, S., Zhang, M., Awan, A. A., Li, C., Li, D., Zheng, E., Rasley, J., Smith, S., Ruwase, O., et al. (2022) · arXiv · doi:10.48550/arxiv.2207.00032 · Cited by 297 · see citing works
DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale
Aminabadi, R. Y., Rajbhandari, S., Awan, A. A., Li, C., Li, D., Zheng, E., Ruwase, O., Smith, S., Zhang, M., Rasley, J., et al. (2022) · SC22: International Conference for High Performance Computing, Networking, Storage and Analysis · doi:10.1109/sc41404.2022.00051 · Cited by 297 · see citing works
Modelling the N400 brain potential as change in a probabilistic representation of meaning
Rabovsky, M., Hansen, S. S., & McClelland, J. L. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0406-4 · Cited by 290 · see citing works
Prediction during natural language comprehension
Willems, R. M., Frank, S. L., Nijhof, A. D., Hagoort, P., & van den Bosch, A. (2016) · Cerebral Cortex · doi:10.1093/cercor/bhv075 · Cited by 288 · see citing works
Language acquisition in the absence of explicit negative evidence: how important is starting small?
Rohde, D. L. T., & Plaut, D. C. (1999) · Cognition · doi:10.1016/s0010-0277(99)00031-1 · Cited by 272 · see citing works
The neurobiology of language beyond single-word processing
Peter Hagoort, A. (2019) · Science · doi:10.1126/science.aax0289 · Cited by 256 · see citing works
fMRI reveals language-specific predictive coding during naturalistic sentence comprehension
Shain, C., Blank, I. A., van Schijndel, M., Schuler, W., & Fedorenko, E. (2020) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2019.107307 · Cited by 215 · see citing works
Self-citationEmmanuel Dupoux, A. (2018) · Cognition · doi:10.1016/j.cognition.2017.11.008 · Cited by 208 · see citing works
Abstract linguistic structure correlates with temporal activity during naturalistic comprehension
Brennan, J., Stabler, E. P., Wagenen, S. E. V., Luh, W., & Hale, J. (2016) · Brain and Language · doi:10.1016/j.bandl.2016.04.008 · Cited by 201 · see citing works
BERT Rediscovers the Classical NLP PipelinePreprint
Tenney, I., Das, D., & Pavlick, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.05950 · Cited by 192 · see citing works
Lack of selectivity for syntax relative to word meanings throughout the language network
Fedorenko, E., Blank, I. A., Siegelman, M., & Mineroff, Z. (2020) · Cognition · doi:10.1016/j.cognition.2020.104348 · Cited by 170 · see citing works
Self-citationLanguage structure in the brain: A fixation-related fMRI study of syntactic surprisal in reading
Henderson, J. M., Choi, W., Lowder, M. W., & Ferreira, F. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.02.050 · Cited by 155 · see citing works
Incorporating Context into Language Encoding Models for fMRI
Jain, S., & Huth, A. G. (2018) · doi:10.1101/327601 · Cited by 155 · see citing works
Toneva, M., & Wehbe, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11833 · Cited by 152 · see citing works
Probabilistic atlas for the language network based on precision fMRI data from >800 individuals
Lipkin, B., Tuckute, G., Affourtit, J., Small, H., Mineroff, Z., Kean, H., Jouravlev, O., Rakocevic, L. I., Pritchett, B., Siegelman, M., et al. (2022) · Scientific Data · doi:10.1038/s41597-022-01645-3 · Cited by 149 · see citing works
Self-citationAn ecologically motivated image dataset for deep learning yields better models of human vision
Mehrer, J., Spoerer, C. J., Jones, E. C., Kriegeskorte, N., & Kietzmann, T. C. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2011417118 · Cited by 139 · see citing works
A Systematic Assessment of Syntactic Generalization in Neural Language Models
Hu, J., Gauthier, J., Qian, P., Wilcox, E., & Levy, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.158 · Cited by 134 · see citing works
Individual differences among deep neural network models
Mehrer, J., Spoerer, C. J., Kriegeskorte, N., & Kietzmann, T. C. (2020) · Nature Communications · doi:10.1038/s41467-020-19632-w · Cited by 133 · see citing works
On the Predictive Power of Neural Language Models for Human Real-Time Comprehension BehaviorPreprint
Wilcox, E., Gauthier, J., Hu, J., Qian, P., & Lévy, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.01912 · Cited by 125 · see citing works
What Artificial Neural Networks Can Tell Us about Human Language Acquisition
Warstadt, A., & Bowman, S. R. (2022) · Algebraic Structures in Natural Language · doi:10.1201/9781003205388-2 · Cited by 111 · see citing works
Self-citationOh, B.-D., & Schuler, W. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00548 · Cited by 107 · see citing works
Parallel processing in speech perception with local and global representations of linguistic context
Brodbeck, C. J., Bhattasali, S., Heredia, A. A. L. C., Resnik, P., Simon, J. Z., & Lau, E. F. (2022) · eLife · doi:10.7554/elife.72056 · Cited by 97 · see citing works
Shared functional specialization in transformer-based language models and the human brain
Kumar, S., Sumers, T. R., Yamakoshi, T., Goldstein, A., Hasson, U., Norman, K. A., Griffiths, T. L., Hawkins, R. D., & Nastase, S. A. (2024) · Nature Communications · doi:10.1038/s41467-024-49173-5 · Cited by 94 · see citing works
(2021) · doi:10.1162/jocn_a_01755 · Cited by 85 · see citing works
Driving and suppressing the human language network using large language models
Tuckute, G., Sathe, A., Srikant, S., Taliaferro, M., Wang, M., Schrimpf, M., Kay, K., & Fedorenko, E. G. (2024) · Nature Human Behaviour · doi:10.1038/s41562-023-01783-7 · Cited by 85 · see citing works
Self-citationEvaluating information-theoretic measures of word prediction in naturalistic sentence reading
Aurnhammer, C., & Frank, S. L. (2019) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2019.107198 · Cited by 79 · see citing works
Lopopolo, A., Frank, S. L., van den Bosch, A. P. J., & Willems, R. M. (2017) · PLoS ONE · doi:10.1371/journal.pone.0177794 · Cited by 78 · see citing works
The neural basis for human syntax: Broca's area and beyond
Friederici, A. D. (2018) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2018.03.004 · Cited by 72 · see citing works
No evidence for differences among language regions in their temporal receptive windows
Blank, I., & Fedorenko, E. G. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116925 · Cited by 72 · see citing works
Self-citationBridging the data gap between children and large language models
Frank, M. C. (2023) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.08.007 · Cited by 72 · see citing works
Linking artificial and human neural representations of language
Gauthier, J., & Lévy, R. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1050 · Cited by 72 · see citing works
GPT-NeoX-20B: An Open-Source Autoregressive Language Model
Black, S., Biderman, S., Hallahan, E., Anthony, Q., Gao, L., Golding, L., He, H., Leahy, C., McDonell, K., Phang, J., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.06745 · Cited by 67 · see citing works
Quantity doesn’t buy quality syntax with neural language models
van Schijndel, M., Mueller, A., & Linzen, T. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1592 · Cited by 65 · see citing works
Futrell, R., Gibson, E. A. F., Tily, H. J., Blank, I., Vishnevetsky, A., Piantadosi, S. T., & Fedorenko, E. G. (2020) · Language Resources and Evaluation · doi:10.1007/s10579-020-09503-7 · Cited by 63 · see citing works
Self-citationReassessing hierarchical correspondences between brain and deep networks through direct interface
Sexton, N. J., & Love, B. C. (2022) · Science Advances · doi:10.1126/sciadv.abm2219 · Cited by 58 · see citing works
Distinct patterns of syntactic agreement errors in recurrent networks and humansPreprint
Linzen, T., & Leonard, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.06882 · Cited by 51 · see citing works
One model for the learning of language
Yang, Y., & Piantadosi, S. T. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2021865119 · Cited by 50 · see citing works
Antonello, R., & Huth, A. G. (2022) · Neurobiology of Language · doi:10.1162/nol_a_00087 · Cited by 49 · see citing works
Kauf, C., Tuckute, G., Lévy, R., Andreas, J., & Fedorenko, E. G. (2023) · Neurobiology of Language · doi:10.1162/nol_a_00116 · Cited by 41 · see citing works
Self-citationLinguistic Analysis of Pretrained Sentence Encoders with Acceptability JudgmentsPreprint
Warstadt, A., & Bowman, S. R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.03438 · Cited by 31 · see citing works
Self-citationShared functional specialization in transformer-based language models and the human brainPreprint
Kumar, S., Sumers, T. R., Yamakoshi, T., et al. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.06.08.495348 · Cited by 31 · see citing works
Futrell, R., Gibson, E. A. F., Tily, H. J., Blank, I., Vishnevetsky, A., Piantadosi, S. T., & Fedorenko, E. G. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2nyub6u36pvm · Cited by 31 · see citing works
Self-citationWord Acquisition in Neural Language Models
Chang, T. A., & Bergen, B. K. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00444 · Cited by 30 · see citing works
A Systematic Assessment of Syntactic Generalization in Neural Language ModelsPreprint
Hu, J., Gauthier, J., Qian, P., Wilcox, E., & Lévy, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.03692 · Cited by 27 · see citing works
Scaling laws for language encoding models in fMRI
Antonello, R., Vaidya, A. R., & Huth, A. G. (2023) · PubMed · doi:10.48550/arxiv.2305.11863 · Cited by 27 · see citing works
How much pretraining data do language models need to learn syntax?
Pérez-Mayos, L., Ballesteros, M., & Wanner, L. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.118 · Cited by 26 · see citing works
What Artificial Neural Networks Can Tell Us About Human Language AcquisitionPreprint
Warstadt, A., & Bowman, S. R. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.07998 · Cited by 24 · see citing works
Self-citationOh, B.-D., & Schuler, W. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.128 · Cited by 21 · see citing works
Neural Language Models are not Born Equal to Fit Brain Data, but Training HelpsPreprint
Pasquiou, A., Lakretz, Y., Hale, J., Thirion, B., & Pallier, C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.03380 · Cited by 14 · see citing works
Bridging the data gap between children and large language modelsPreprint
Frank, M. C. (2023) · doi:10.31234/osf.io/qzbgx · Cited by 13 · see citing works
Recurrent babbling: evaluating the acquisition of grammar from limited input data
Pannitto, L., & Herbelot, A. (2020) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2020.conll-1.13 · Cited by 12 · see citing works
Graded sensitivity to structure and meaning throughout the human language networkPreprint
Shain, C., Kean, H., Casto, C., Lipkin, B., Affourtit, J., Siegelman, M., Mollica, F., & Fedorenko, E. G. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.11.12.467812 · Cited by 12 · see citing works
Self-citationWiring Up Vision: Minimizing Supervised Synaptic Updates Needed to Produce a Primate Ventral StreamPreprint
Geiger, F., Schrimpf, M., Marques, T., & DiCarlo, J. J. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.06.08.140111 · Cited by 10 · see citing works
Self-citationHow Well Do Unsupervised Learning Algorithms Model Human Real-time and Life-long Learning?
Zhuang, C., Xiang, V., Bai, Y., Jia, X., Turk‐Browne, N. B., Norman, K. A., DiCarlo, J. J., & Yamins, D. L. K. (2022) · PubMed · Cited by 8 · see citing works
Oh, B.-D., & Schuler, W. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.11389 · Cited by 7 · see citing works
Oh, B.-D., & Schuler, W. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.12131 · Cited by 6 · see citing works
How Well Do Contrastive Learning Algorithms Model Human Real-time and Life-long Learning?
Zhuang, C., Xiang, V., Bai, Y., Jia, X., DiCarlo, J. J., & Yamins, D. L. K. (2022) · Conference on Cognitive Computational Neuroscience (CCN) · doi:10.32470/ccn.2022.1155-0 · Cited by 6 · see citing works
How much pretraining data do language models need to learn syntax?Preprint
Pérez-Mayos, L., Ballesteros, M., & Wanner, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.03160 · Cited by 5 · see citing works
How much pretraining data do language models need to learn syntax?Preprint
P\'erez-Mayos, L., Ballesteros, M., & Wanner, L. (2021) · arXiv (Cornell University) · Cited by 4 · see citing works
Linking artificial and human neural representations of languagePreprint
Gauthier, J., & Lévy, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01244 · Cited by 4 · see citing works
Scaffolded input promotes atomic organization in the recurrent neural network language model
Huebner, P. A., & Willits, J. A. (2021) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2021.conll-1.32 · Cited by 3 · see citing works
Quantity doesn't buy quality syntax with neural language modelsPreprint
van Schijndel, M., Mueller, A., & Linzen, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.00111 · Cited by 3 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
Cited by
Sorted by citations
Dissociating language and thought in large language models
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.011 · Cited by 345 · see citing works
Self-citationDo Large Language Models Know What Humans Know?
Trott, S., Jones, C., Chang, T., Michaelov, J., & Bergen, B. (2023) · Cognitive Science · doi:10.1111/cogs.13309 · Cited by 99 · see citing works
Driving and suppressing the human language network using large language models
Tuckute, G., Sathe, A., Srikant, S., Taliaferro, M., Wang, M., Schrimpf, M., Kay, K., & Fedorenko, E. G. (2024) · Nature Human Behaviour · doi:10.1038/s41562-023-01783-7 · Cited by 85 · see citing works
Self-citationLanguage in Brains, Minds, and Machines
Tuckute, G., Kanwisher, N., & Fedorenko, E. G. (2024) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-120623-101142 · Cited by 69 · see citing works
Self-citationContextual feature extraction hierarchies converge in large language models and the brain
Mischler, G., Li, Y. A., Bickel, S., Mehta, A. D., & Mesgarani, N. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00925-4 · Cited by 58 · see citing works
Event Knowledge in Large Language Models: The Gap Between the Impossible and the Unlikely
Kauf, C., Ivanova, A. A., Rambelli, G., Chersoni, E., She, J. S., Chowdhury, Z., Fedorenko, E. G., & Lenci, A. (2023) · Cognitive Science · doi:10.1111/cogs.13386 · Cited by 56 · see citing works
Self-citationGoldstein, A., Ham, E., Schain, M., Nastase, S. A., Aubrey, B., Zada, Z., Grinstein-Dabush, A., Gazula, H., Feder, A., Doyle, W., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-65518-0 · Cited by 43 · see citing works
Kauf, C., Tuckute, G., Lévy, R., Andreas, J., & Fedorenko, E. G. (2023) · Neurobiology of Language · doi:10.1162/nol_a_00116 · Cited by 41 · see citing works
Self-citationLampinen, A. (2024) · Computational Linguistics · doi:10.1162/coli_a_00525 · Cited by 32 · see citing works
van Dijk, B., Kouwenhoven, T., Spruit, M. R., & van Duijn, M. J. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.779 · Cited by 22 · see citing works
Large Language Models and the Wisdom of Small Crowds
Trott, S. (2024) · Open Mind · doi:10.1162/opmi_a_00144 · Cited by 22 · see citing works
A population code for semantics in human hippocampus
Franch, M. C., Mickiewicz, E. A., Belanger, J. L., Joiner, B., Katlowitz, K. A., Zhu, H., Chavez, A. G., Chericoni, A., Goldman, A. M., Krishnan, V., et al. (2026) · Nature Neuroscience · doi:10.1038/s41593-026-02436-4 · Cited by 20 · see citing works
Large Language Models Are Human-Like Internally
Kuribayashi, T., Oseki, Y., Taieb, S. B., Inui, K., & Baldwin, T. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.58 · Cited by 19 · see citing works
Huber, E., Sauppe, S., Isasi-Isasmendi, A., Bornkessel‐Schlesewsky, I., Merlo, P., & Bickel, B. (2023) · Neurobiology of Language · doi:10.1162/nol_a_00121 · Cited by 18 · see citing works
Cognitive Modeling Using Artificial Intelligence
Frank, M. C., & Goodman, N. D. (2025) · Annual Review of Psychology · doi:10.1146/annurev-psych-030625-040748 · Cited by 17 · see citing works
Hadidi, N., Feghhi, E., Song, B. H., Blank, I., & Kao, J. C. (2026) · Nature Communications · doi:10.1038/s41467-026-72253-7 · Cited by 17 · see citing works
Mueller, A., & Linzen, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.629 · Cited by 14 · see citing works
AI Detection's High False Positive Rates and the Psychological and Material Impacts on Students
Gegg-Harrison, W. M., & Quarterman, C. (2024) · Advances in educational marketing, administration, and leadership book series · doi:10.4018/979-8-3693-0240-8.ch011 · Cited by 14 · see citing works
Context, not grammar, is key to structural priming
Nielsen, Y. A., & Christiansen, M. H. (2025) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2025.05.016 · Cited by 7 · see citing works
A Python Toolbox for Representational Similarity AnalysisPreprint
van den Bosch, J. J. F., Golan, T., Peters, B., Taylor, J., Shahbazi, M., Lin, B., Charest, I., Diedrichsen, J., Kriegeskorte, N., Mur, M., et al. (2025) · eLife · doi:10.7554/elife.107828.1 · Cited by 7 · see citing works
Shared representations of human actions across vision and language
Dima, D. C., Janarthanan, S., Culham, J. C., & Mohsenzadeh, Y. (2024) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2024.108962 · Cited by 6 · see citing works
Social Learning in Neural Agent-Based Models
Douven, I. (2024) · Philosophy of Science · doi:10.1017/psa.2024.33 · Cited by 6 · see citing works
Natural syntax, artificial intelligence and language acquisition
O’Grady, W., & Lee, M. (2023) · Information · doi:10.3390/info14070418 · Cited by 5 · see citing works
Keshishian, M., Mischler, G., Thomas, S., Kingsbury, B. E. D., Bickel, S., Mehta, A. D., & Mesgarani, N. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-026-01185-0 · Cited by 5 · see citing works
Hong, Z., Wang, H., Zada, Z., Gazula, H., Turner, D. D., Aubrey, B., Niekerken, L., Doyle, W., Devore, S., Dugan, P. C., et al. (2024) · eLife · doi:10.7554/elife.101204.1 · Cited by 5 · see citing works
Bruera, A., & Poesio, M. (2024) · PLoS ONE · doi:10.1371/journal.pone.0291099 · Cited by 4 · see citing works
Does word knowledge account for the effect of world knowledge on pronoun interpretation?
Jones, C. R., & Bergen, B. K. (2024) · Language and Cognition · doi:10.1017/langcog.2024.2 · Cited by 4 · see citing works
Brain-model neural similarity reveals abstractive summarization performance
Zhang, Z., Guo, S.-L., Zhou, W. H., Luo, Y., Zhu, Y., Zhang, L., & Li, L. (2025) · Scientific Reports · doi:10.1038/s41598-024-84530-w · Cited by 4 · see citing works
de Varda, A. G., Malik-Moraleda, S., Tuckute, G., & Fedorenko, E. G. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.02.01.636044 · Cited by 4 · see citing works
Self-citationSchepens, J. J., Woloszyn, H., Marx, N., & Gagl, B. (2025) · Open Mind · doi:10.1162/opmi.a.30 · Cited by 3 · see citing works
A Python Toolbox for Representational Similarity AnalysisPreprint
van den Bosch, J. J. F., Golan, T., Peters, B., Taylor, J., Shahbazi, M., Lin, B., Charest, I., Diedrichsen, J., Kriegeskorte, N., Mur, M., et al. (2025) · eLife · doi:10.7554/elife.107828 · Cited by 3 · see citing works
Pernencar, C. A. D. C., dos Santos, R. F. G., & Ribeiro, N. (2025) · Advances in wireless technologies and telecommunication book series · doi:10.4018/979-8-3693-7147-3.ch015 · Cited by 2 · see citing works
Do Language Models Know Who Did What to Whom?
Denning, J. M., Guo, X., Snefjella, B., & Blank, I. (2026) · Open Mind · doi:10.1162/opmi.a.365 · Cited by 2 · see citing works
Cardiac Heterogeneity Prediction by Cardio-Neural Network Simulation
Mehmood, A., Ilyas, A., & Ilyas, H. (2025) · Neuroinformatics · doi:10.1007/s12021-025-09717-6 · Cited by 2 · see citing works
Comparing Children and Large Language Models in Word Sense Disambiguation: Insights and ChallengesPreprint
Cabiddu, F., Nikolaus, M., & Fourtassi, A. (2023) · doi:10.31234/osf.io/zgy7v · Cited by 2 · see citing works
Hosseini, E. A., & Fedorenko, E. G. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.11.05.564832 · Cited by 2 · see citing works
Self-citationWhat language models can tell us about learning adjectives
Gotowski, M., & Davis, F. (2024) · Proceedings of the Linguistic Society of America · doi:10.3765/plsa.v9i1.5693 · Cited by 1 · see citing works
Puffay, C., Mischler, G., Choudhari, V., Vanthornhout, J., Bickel, S., Mehta, A. D., Schevon, C. A., McKhann, G. M., hamme, H. V., Francart, T., et al. (2026) · Imaging Neuroscience · doi:10.1162/imag.a.1227 · Cited by 1 · see citing works
A deep neural network model of audiovisual speech recognition reports the McGurk effect
Ma, H., Wang, Z., Zhang, X., Magnotti, J. F., & Beauchamp, M. S. (2026) · Psychonomic Bulletin & Review · doi:10.3758/s13423-025-02846-8 · Cited by 1 · see citing works
Unifying the structures of language in a neural population code
Nastase, S. A., Zada, Z., Goldberg, A., & Hasson, U. (2026) · Neuron · doi:10.1016/j.neuron.2026.07.024 · Cited by 1 · see citing works
Fodor, J., Murawski, C., & Suzuki, S. (2026) · eLife · doi:10.1101/2025.07.19.665701 · Cited by 1 · see citing works
CogMemLM: Human-Like Memory Mechanisms Improve Performance and Cognitive Plausibility of LLMs
Thoma, L., Weyers, I., Çano, E., Schweter, S., Mueller, J. L., & Roth, B. (2023) · BabyLM Challenge · doi:10.18653/v1/2023.conll-babylm.15 · Cited by 1 · see citing works
Larger language models better align with neural representations of natural languagePreprint
Hong, Z., Wang, H., Zada, Z., Gazula, H., Turner, D. D., Aubrey, B., Niekerken, L., Doyle, W., Devore, S., Dugan, P. C., et al. (2024) · eLife · doi:10.7554/elife.101204 · Cited by 1 · see citing works
Christiansen, M. H., & Kallens, P. C. (2022) · doi:10.64628/aai.qy449s577 · Cited by 1 · see citing works
Can an emerging field called ‘neural systems understanding’ explain the brain?
Musser, G. S. (2024) · The Transmitter · doi:10.53053/ogps4091
Rao, C. V. G., Banu, S. B., & Kapila, D. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-8031-0_116
Shimizu, R., Antonello, R., Singh, C., & Mesgarani, N. (2026) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1014906
Fodor, J., Murawski, C., & Suzuki, S. (2025) · eLife · doi:10.7554/elife.108442.1
Fodor, J., Murawski, C., & Suzuki, S. (2025) · eLife · doi:10.7554/elife.108442
Li, C., & Chen, J. (2025) · International Symposium on Computer and Information Processing Technology (ISCIPT) · doi:10.1109/iscipt67144.2025.11265684
Karami, B., Zandi, F., & Hatami, J. (2025) · Research Square · doi:10.21203/rs.3.rs-8270539/v1
Using computational semantics to study meaning in the brain
Fodor, J., & Suzuki, S. (2025) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2025.106514
Bruera, A., & Hartwigsen, G. (2026) · Neurobiology of Language · doi:10.1162/nol.a.244
Kissane, H., Tziridis, K., Schilling, A., Krauß, P., & Herbst, T. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1676341
The gap between humans and language models: Analyzing adjective performance
Gotowski, M., & Davis, F. (2026) · Language Acquisition · doi:10.1080/10489223.2026.2613909
Khan, M. P., Chang, C., Raja, M. A. Z., Shoaib, M., & Shu, C. (2026) · ZAMM ‐ Journal of Applied Mathematics and Mechanics / Zeitschrift für Angewandte Mathematik und Mechanik · doi:10.1002/zamm.70464
Human-like sequential sound-to-meaning transfer drives artificial speech comprehensionPreprint
Zhang, S., Li, S., Yang, R., Chen, G., Tian, X., Wang, Q., & Fang, F. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.05.13.723203
Quantifiers that are more monotone are easier to learn
Haberland, C., & Steinert‐Threlkeld, S. (2025) · Proceedings from Semantics and Linguistic Theory · doi:10.3765/6qa95941
Курьян, В. Е. (2026) · Vestnik psihofiziologii. · doi:10.34985/g1876-0680-0528-l
Position-related centro-parietal ERP responses during contextual processing across symbolic domains
Yanez-Ramos, M. G., de Rivera, D. Z. E., & Garrido, A. A. G. (2026) · Biological Psychology · doi:10.1016/j.biopsycho.2026.109374
Large Language Models as Distributional Baselines for Language Tasks
Trott, S., Michaelov, J. A., Jones, C., Chang, T. A., & Bergen, B. K. (2026) · Open Mind · doi:10.1162/opmi.a.373
Low-dimensional GPT-2 features improve cross-narrative prediction of shared fMRI responses
Karaman, B. (2026) · Journal of Neurolinguistics · doi:10.1016/j.jneuroling.2026.101381
Fodor, J., Murawski, C., & Suzuki, S. (2026) · eLife · doi:10.7554/elife.108442.3
Fodor, J., Murawski, C., & Suzuki, S. (2026) · eLife · doi:10.7554/elife.108442.3.sa0
- Authors
- Hosseini, E. A., Schrimpf, M., Zhang, Y., Bowman, S., Zaslavsky, N., & Fedorenko, E.
- Affiliations
- McGovern Institute for Brain Research (US)
- Stanford University (US)
- New York University (US)
- Harvard University (US)
- Corresponding author
- Hosseini, Eghbal A.
- Fedorenko, Evelina
- Venue
- Neurobiology of Language
- Publisher
- MIT Press
- Published
- 2024
- Volume
- 5
- Issue
- 1
- Pages
- 43-63
- ISSN
- 2641-4368 (ISSN-L)
- PMC
- PMC11025646
- OpenAlex
- W4391100869
- Cited by
- 63
- References
- 94
- Citations by year
- 2026: 18 · 2025: 17 · 2024: 15 · 2023: 11 · 2022: 2
- FWCI
- 12.12
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Neurobiology of Language (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Funders
- National Institute of Neurological Disorders and Stroke (U01-NS121471)
- McGovern Institute for Brain Research, Massachusetts Institute of Technology (Friends of McGovern graduate fellowship, K. Lisa Young ICoN Center postdoctoral fellowship)
- National Institute on Deafness and Other Communication Disorders (DC016607, R01-DC016950)
- Simons Center for the Social Brain, Massachusetts Institute of Technology
- Massachusetts Institute of Technology (Middleton Professorship)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
- Social Sciences › Psychology › Developmental and Educational Psychology › Language Development and Disorders
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- GPT-2, human language processing, training data size, next-word prediction, perplexity, sentence representation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
