The DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index.
Localizing syntactic predictions using recurrent neural network grammars
Open accessAbstract
Brain activity in numerous perisylvian brain regions is modulated by the expectedness of linguistic stimuli. We leverage recent advances in computational parsing models to test what representations guide the processes reflected by this activity. Recurrent Neural Network Grammars (RNNGs) are generative models of (tree, string) pairs that use neural networks to drive derivational choices. Parsing with them yields a variety of incremental complexity metrics that we evaluate against a publicly available fMRI data-set recorded while participants simply listen to an audiobook story. Surprisal, which captures a word's un-expectedness, correlates with a wide range of temporal and frontal regions when it is calculated based on word-sequence information using a top-performing LSTM neural network language model. The explicit encoding of hierarchy afforded by the RNNG additionally captures activity in left posterior temporal areas. A separate metric tracking the number of derivational steps taken between words correlates with activity in the left temporal lobe and inferior frontal gyrus. This pattern of results narrows down the kinds of linguistic representations at play during predictive processing across the brain's language network.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“it is sensitive to abstract hierarchical rules in isolated phrases and sentences [31, 108, 109, 110], in naturalistic narratives [111, 112, 113, 114], and in syntactically well-formed but semantically empty (”
From Dissociating language and thought in large language models (2024) · cites this paper as [112]
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,893
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Building a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390
Accurate unlexicalized parsing
Klein, D., & Manning, C. D. (2003) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1075096.1075150 · Cited by 3,055
Integration of Visual and Linguistic Information in Spoken Language Comprehension
Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995) · Science · doi:10.1126/science.7777863 · Cited by 2,929
Statistical methods for speech recognition
Jelinek, F. (1997) · Journal of the American Statistical Association · doi:10.2307/2670189 · Cited by 2,376
Electrophysiology reveals semantic memory use in language comprehension.
Kutas, M., & Federmeier, K. D. (2000) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01560-6 · Cited by 2,124
A probabilistic earley parser as a psycholinguistic model
Hale, J. (2001) · Second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies 2001 - NAACL '01 · doi:10.3115/1073336.1073357 · Cited by 1,150
Self-citationTransition-Based Dependency Parsing with Stack Long Short-Term Memory
Dyer, C., Ballesteros, M., Ling, W., Matthews, A., & Smith, N. A. (2015) · doi:10.3115/v1/p15-1033 · Cited by 848
Self-citationAssessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies
Linzen, T., Emmanuel Dupoux, A., & Goldberg, Y. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00115 · Cited by 820
The Cortical Representation of Speech
Mazoyer, B. M., Tzourio‐Mazoyer, N., Frak, V., Syrota, A., Murayama, N., Levrier, O., Salamon, G., Dehaene, S., Cohen, L., & Mehler, J. (1993) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1993.5.4.467 · Cited by 731
A Probabilistic Model of Lexical and Syntactic Access and Disambiguation
Jurafsky, D. S. (1996) · Cognitive Science · doi:10.1207/s15516709cog2002_1 · Cited by 718
Cortical representation of the constituent structure of sentences
Pallier, C., Devauchelle, A.-D., & Dehaene, S. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1018711108 · Cited by 691
Data from eye-tracking corpora as evidence for theories of syntactic processing complexity
Demberg, V., & Keller, F. (2008) · Cognition · doi:10.1016/j.cognition.2008.07.008 · Cited by 606
Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing
Lapata, M., & Ng, H. T. (2008) · Cited by 567
Colorless Green Recurrent Networks Dream Hierarchically
Gulordava, K., Bojanowski, P., Grave, É., Linzen, T., & Baroni, M. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1108 · Cited by 538
Recurrent Neural Network Grammars
Dyer, C., Kuncoro, A., Ballesteros, M., & Smith, N. A. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1024 · Cited by 517
Self-citationSyntactic and Semantic Modulation of Neural Activity during Auditory Sentence Comprehension
Humphries, C., Binder, J. R., Medler, D. A., & Liebenthal, E. (2006) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2006.18.4.665 · Cited by 463
Sentence Perception as an Interactive Parallel Process
Marslen‐Wilson, W. D. (1975) · Science · doi:10.1126/science.189.4198.226 · Cited by 453
The Response of Left Temporal Cortex to Sentences
Vandenberghe, R. R., Nobre, A. C., & Price, C. J. (2002) · Journal of Cognitive Neuroscience · doi:10.1162/08989290260045800 · Cited by 445
The ERP response to the amount of information conveyed by words in sentences
Frank, S. L., Otten, L. J., Galli, G., & Vigliocco, G. (2014) · Brain and Language · doi:10.1016/j.bandl.2014.10.006 · Cited by 382
Large-scale replication study reveals a limit on probabilistic prediction in language comprehension
Nieuwland, M. S., Politzer‐Ahles, S., Heyselaar, E., Segaert, K., Darley, E., Kazanina, N., Wolfsthurn, S. V. G. Z., Bartolozzi, F., Kogan, V. V., Ito, A., et al. (2018) · eLife · doi:10.7554/elife.33468 · Cited by 375
Reconciling time, space and function: A new dorsal–ventral stream model of sentence comprehension
Bornkessel‐Schlesewsky, I., & Schlesewsky, M. (2013) · Brain and Language · doi:10.1016/j.bandl.2013.01.010 · Cited by 343
Neurophysiological dynamics of phrase-structure building during sentence processing
Nelson, M. J., El Karoui, I., Giber, K., Yang, X., Cohen, L., Koopman, H., Cash, S. S., Naccache, L., Hale, J. T., Pallier, C., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1701590114 · Cited by 319
Self-citationBemis, D. K., & Liina Pylkkänen, A. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.5003-10.2011 · Cited by 294
Prediction during natural language comprehension
Willems, R. M., Frank, S. L., Nijhof, A. D., Hagoort, P., & van den Bosch, A. (2016) · Cerebral Cortex · doi:10.1093/cercor/bhv075 · Cited by 288
Language: Structure, Processing, and Disorders
Caplan, D. N. (1992) · Cited by 279
Syntactic structure building in the anterior temporal lobe during natural story listening
Brennan, J., Nir, Y., Hasson, U., Malach, R., Heeger, D. J., & Pylkkänen, L. (2012) · Brain and Language · doi:10.1016/j.bandl.2010.04.002 · Cited by 279
Self-citationParsing costs as predictors of reading difficulty: An evaluation using the Potsdam Sentence Corpus
Boston, M. F., Hale, J., Kliegl, R. M., Patil, U., & Vasishth, S. (2008) · Journal of Eye Movement Research · doi:10.16910/jemr.2.1.1 · Cited by 267
Self-citationA computational theory of human linguistic processing: memory limitations and processing breakdown
Gibson, E. A. F. (1991) · Cited by 250
Poeppel, D. (2015) · Princeton University Press eBooks · doi:10.1515/9781400851935-018 · Cited by 250
Snijders, T. M., Vosse, T., Kempen, G., Berkum, J. J. A. V., Petersson, K. M., & Peter Hagoort, A. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhn187 · Cited by 248
Rogalsky, C., & Hickok, G. S. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhn126 · Cited by 240
Insensitivity of the Human Sentence-Processing System to Hierarchical Structure
Frank, S. L., & Bod, R. (2011) · Psychological Science · doi:10.1177/0956797611409589 · Cited by 219
Broca’s Area and Language Processing: Evidence for the Cognitive Control Connection
Novick, J. M., Trueswell, J. C., & Thompson‐Schill, S. L. (2010) · Language and Linguistics Compass · doi:10.1111/j.1749-818x.2010.00244.x · Cited by 218
Just, M. A., & Varma, S. (2007) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/cabn.7.3.153 · Cited by 216
Aligning Grammatical Theories and Language Processing Models
Lewis, S., & Phillips, C. (2014) · Journal of Psycholinguistic Research · doi:10.1007/s10936-014-9329-z · Cited by 210
Localizing components of a complex task
Stowe, L. A., Broere, C. A. J., Paans, A. M. J., Wijers, A. A., Mulder, G., Vaalburg, W., & Zwarts, F. (1998) · Neuroreport · doi:10.1097/00001756-199809140-00014 · Cited by 207
Abstract linguistic structure correlates with temporal activity during naturalistic comprehension
Brennan, J., Stabler, E. P., Wagenen, S. E. V., Luh, W., & Hale, J. (2016) · Brain and Language · doi:10.1016/j.bandl.2016.04.008 · Cited by 205
Self-citationBuilding by Syntax: The Neural Basis of Minimal Linguistic Structures
Zaccarella, E., Meyer, L., Makuuchi, M., & Friederici, A. D. (2015) · Cerebral Cortex · doi:10.1093/cercor/bhv234 · Cited by 197
Information‐theoretical Complexity Metrics
Hale, J. (2016) · Language and Linguistics Compass · doi:10.1111/lnc3.12196 · Cited by 186
Self-citationBrian Roark, A., Bachrach, A., Cardenas, C., & Pallier, C. (2009) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1699510.1699553 · Cited by 178
Language structure in the brain: A fixation-related fMRI study of syntactic surprisal in reading
Henderson, J. M., Choi, W., Lowder, M. W., & Ferreira, F. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.02.050 · Cited by 153
Predicting language: MEG evidence for lexical preactivation
Dikker, S., & Liina Pylkkänen, A. (2012) · Brain and Language · doi:10.1016/j.bandl.2012.08.004 · Cited by 146
Finding syntax in human encephalography with beam search
Hale, J., Dyer, C., Kuncoro, A., & Brennan, J. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1254 · Cited by 143
Self-citationLanguage: Structure, processing, and disorders
Taylor, M. M., & Taylor, I. K. (1993) · Biological Psychology · doi:10.1016/0301-0511(93)90072-g · Cited by 135
The time-course and spatial distribution of brain activity associated with sentence processing
Brennan, J., & Liina Pylkkänen, A. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.030 · Cited by 131
Self-citationWhat Do Recurrent Neural Network Grammars Learn About Syntax?
Kuncoro, A., Ballesteros, M., Kong, L., Dyer, C., Neubig, G., & Smith, N. A. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-1117 · Cited by 131
Self-citationAugmented transition networks as psychological models of sentence comprehension
Kaplan, R. M. (1972) · Artificial Intelligence · doi:10.1016/0004-3702(72)90043-4 · Cited by 130
Hierarchical structure guides rapid linguistic predictions during naturalistic listening
Brennan, J., & Hale, J. (2019) · PLoS ONE · doi:10.1371/journal.pone.0207741 · Cited by 125
Self-citationEye typing in application: A comparison of two interfacing systems with ALS patients
Pannasch, S., Helmert, J. R., Malischke, S., Storch, A., & Velichkovsky, B. M. (2008) · Journal of Eye Movement Research · Cited by 117
Building phrases in language production: An MEG study of simple composition
Liina Pylkkänen, A., Bemis, D. K., & Blanco-Elorrieta, E. (2014) · Cognition · doi:10.1016/j.cognition.2014.07.001 · Cited by 107
Lexical Predictability During Natural Reading: Effects of Surprisal and Entropy Reduction
Lowder, M. W., Choi, W., Ferreira, F., & Henderson, J. M. (2018) · Cognitive Science · doi:10.1111/cogs.12597 · Cited by 86
Lopopolo, A., Frank, S. L., van den Bosch, A. P. J., & Willems, R. M. (2017) · PLoS ONE · doi:10.1371/journal.pone.0177794 · Cited by 78
The LATL as locus of composition: MEG evidence from English and Arabic
Westerlund, M., Kastner, I., Kaabi, M. A., & Liina Pylkkänen, A. (2015) · Brain and Language · doi:10.1016/j.bandl.2014.12.003 · Cited by 78
MEG Evidence for Incremental Sentence Composition in the Anterior Temporal Lobe
Brennan, J., & Liina Pylkkänen, A. (2016) · Cognitive Science · doi:10.1111/cogs.12445 · Cited by 75
Self-citationMolinaro, N., Barraza, P., & Carreiras, M. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.01.031 · Cited by 65
Effective Inference for Generative Neural Parsing
Stern, M., Fried, D., & Klein, D. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1178 · Cited by 60
Automaton Theories of Human Sentence Comprehension
Hale, J. (2014) · Cited by 52
Self-citationBlanco-Elorrieta, E., & Liina Pylkkänen, A. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.08.049 · Cited by 32
Brian Roark, A. (2004) · Natural Language Engineering · doi:10.1017/s1351324903003152 · Cited by 26
Hierarchical Representation in Neural Language Models: Suppression and Recovery of Expectations
Wilcox, E., Lévy, R., & Futrell, R. (2019) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w19-4819 · Cited by 26
Predictive sentence comprehension during story-listening in autism spectrum disorder
Brennan, J. R., Lajiness-O'Neill, R., Bowyer, S., Kovelman, I., & Hale, J. T. (2019) · Language, Cognition and Neuroscience · doi:10.1080/23273798.2018.1560483 · Cited by 19
Self-citationProceedings of the 15th Workshop on Cognitive Modeling and Computational Linguistics
(2026) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4pvp2rzzk9vq · Cited by 17
Composition of Complex Meaning
Liina Pylkkänen, A. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-407794-2.00050-x · Cited by 16
Composition of Complex Meaning: Interdisciplinary perspectives on the left anterior temporal lobe
Pylkkanen, M. (2015) · Cited by 15
Entropy Reduction correlates with temporal lobe activity
Nelson, M. J., Dehaene, S., Pallier, C., & Hale, J. (2017) · Workshop on Cognitive Modeling and Computational Linguistics (CMCL) · doi:10.18653/v1/w17-0701 · Cited by 14
Self-citationSyntax and the Cognitive Neuroscience of Syntactic Structure Building
Sprouse, J., & Hornstein, N. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-407794-2.00014-6 · Cited by 13
Effective Inference for Generative Neural ParsingPreprint
Stern, M., Fried, D., & Klein, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.08976 · Cited by 9
Research works printed but not linked: 1
- Bates, 2009, lme4: linear mixed-effects models using S4 classes, R package
Cited by
Sorted by citations
Dissociating language and thought in large language models
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.011 · Cited by 345
A hierarchy of linguistic predictions during natural language comprehension
Heilbron, M., Armeni, K., Schoffelen, J.-M., Hagoort, P., & de Lange, F. P. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2201968119 · Cited by 343
Brodbeck, C. J., & Simon, J. Z. (2020) · Current Opinion in Physiology · doi:10.1016/j.cophys.2020.07.014 · Cited by 127
Shared functional specialization in transformer-based language models and the human brain
Kumar, S., Sumers, T. R., Yamakoshi, T., Goldstein, A., Hasson, U., Norman, K. A., Griffiths, T. L., Hawkins, R. D., & Nastase, S. A. (2024) · Nature Communications · doi:10.1038/s41467-024-49173-5 · Cited by 108
Semantic encoding during language comprehension at single-cell resolution
Jamali, M., Grannan, B. L., Cai, J., Khanna, A. R., Muñoz, W., Caprara, I., Paulk, A. C., Cash, S. S., Fedorenko, E. G., & Williams, Z. M. (2024) · Nature · doi:10.1038/s41586-024-07643-2 · Cited by 76
Metadata
- Authors
- Brennan, J. R., Dyer, C., Kuncoro, A., & Hale, J. T.
- Affiliations
- University of Michigan (US)
- Google DeepMind (United Kingdom) (GB)
- University of Oxford (GB)
- University of Georgia (US)
- Corresponding author
- Brennan, Jonathan R.
- ORCID
- Brennan, Jonathan R.
- Venue
- Neuropsychologia
- Publisher
- Elsevier BV
- Published
- 2020-09
- Volume
- 146
- Pages
- 107479
- ISSN
- 0028-3932 (ISSN-L)
- OpenAlex
- W3024404575
- Cited by
- 58
- References
- 72
- Citations by year
- 2026: 9 · 2025: 14 · 2024: 12 · 2023: 14 · 2022: 4 · 2021: 2 · 2020: 3
- FWCI
- 3.09
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Neuropsychologia (acceptedVersion)
- PubMed (publishedVersion)
- Funders
- National Science Foundation
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
- Social Sciences › Psychology › Developmental and Educational Psychology › Language Development and Disorders
- Social Sciences › Psychology › Developmental and Educational Psychology › Reading and Literacy Development
- Keywords
- syntactic analysis, predictive processing, language comprehension, functional MRI, inferior frontal gyrus, derivational complexity, linguistic representation
- MeSH terms
- Brain Mapping, Humans, Language, Linguistics, Magnetic Resonance Imaging, Temporal Lobe, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
