Aligning context-based statistical models of language with brain activity during reading
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Many statistical models for natural language processing exist, including context-based neural networks that (1) model the previously seen context as a latent feature vector, (2) integrate successive words into the context using some learned representation (embedding), and (3) compute output probabilities for incoming words given the context.On the other hand, brain imaging studies have suggested that during reading, the brain (a) continuously builds a context from the successive words and every time it encounters a word it (b) fetches its properties from memory and (c) integrates it with the previous context with a degree of effort that is inversely proportional to how probable the word is.This hints to a parallelism between the neural networks and the brain in modeling context (1 and a), representing the incoming words (2 and b) and integrating it (3 and c).We explore this parallelism to better understand the brain processes and the neural networks representations.We study the alignment between the latent vectors used by neural networks and brain activity observed via Magnetoencephalography (MEG) when subjects read a story.For that purpose we apply the neural network to the same text the subjects are reading, and explore the ability of these three vector representations to predict the observed word-by-word brain activity.Our novel results show that: before a new word i is read, brain activity is well predicted by the neural network latent representation of context and the predictability decreases as the brain integrates the word and changes its own representation of context.Secondly, the neural network embedding of word i can predict the MEG activity when word i is presented to the subject, revealing that it is correlated with the brain's own representation of word i.Moreover, we obtain that the activity is predicted in different regions of the brain with varying delay.The delay is consistent with the placement of each region on the processing pathway that starts in the visual cortex and moves to higher level regions.Finally, we show that the output probability computed by the neural networks agrees with the brain's own assessment of the probability of word i, as it can be used to predict the brain activity after the word i's properties have been fetched from memory and the brain is in the process of integrating it into the context.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Further, a previous study suggests that representations learned by an LM can be useful for modeling MEG brain responses to reading (23).”
From Incorporating Context into Language Encoding Models for fMRI (2018) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2010) · International Conference on Machine Learning · Cited by 13,995
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490 · cited 1×
“We control the false discovery rate using (Benjamini and Yekutieli, 2001) to adjust for the testing at multiple locations and time windows.”
MethodGeneralized Cross-Validation as a Method for Choosing a Good Ridge Parameter
Golub, G. H., Heath, M., & Wahba, G. (1979) · Technometrics · doi:10.1080/00401706.1979.10489751 · Cited by 3,934
Towards a neural basis of auditory sentence processing
Friederici, A. D. (2002) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01839-8 · Cited by 2,231
Information-based functional brain mapping
Kriegeskorte, N., Goebel, R., & Bandettini, P. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0600244103 · Cited by 2,113 · cited 1×
“Restricting the analysis spatially: a searchlight equivalent We adapt the searchlight method (Kriegeskorte et al., 2006) to MEG.”
MethodSpatiotemporal signal space separation method for rejecting nearby interference in MEG measurements
Taulu, S., & Simola, J. (2006) · Physics in Medicine and Biology · doi:10.1088/0031-9155/51/7/008 · Cited by 1,549 · cited 1×
“For preprocessing, we used Signal Space Separation method (SSS, (Taulu et al., 2004)), followed by its temporal extension (tSSS, (Taulu and Simola, 2006)).”
MethodStatistical Methods for Research Workers
(1926) · Journal of Comparative Pathology and Therapeutics · doi:10.1016/s0368-1742(26)80018-9 · Cited by 1,283
Suppression of Interference and Artifacts by the Signal Space Separation Method
Taulu, S., Kajola, M., & Simola, J. (2004) · Brain Topography · doi:10.1023/b:brat.0000032864.93890.f9 · Cited by 541 · cited 1×
“For preprocessing, we used Signal Space Separation method (SSS, (Taulu et al., 2004)), followed by its temporal extension (tSSS, (Taulu and Simola, 2006)).”
MethodHagoort, P. (2003) · NeuroImage · doi:10.1016/j.neuroimage.2003.09.013 · Cited by 352 · cited 1×
“Less is understood about the cortical dynamics of word integration, as multiple theories exist (Friederici, 2002; Hagoort, 2003).”
Background
Research works printed but not linked: 1
- Tomas Mikolov. 2012. Statistical Language Models Based on Neural Networks. Ph.D. thesis, Brno Uni- versity of Technology.
Cited by
The neural architecture of language: Integrative modeling converges on predictive processing
Schrimpf, M., Blank, I. A., Tuckute, G., Kauf, C., Hosseini, E. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2105646118 · Cited by 517
Shared computational principles for language processing in humans and deep language models
Goldstein, A., Zada, Z., Buchnik, E., Schain, M., Price, A., Aubrey, B., Nastase, S. A., Feder, A., Emanuel, D., Cohen, A., et al. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01026-4 · Cited by 447
“These results agree with concurrent studies demonstrating that contextual embeddings model neural responses to words better than static semantic embeddings 15,16,45,46 .”
ResultBrains and algorithms partially converge in natural language processing
Caucheteux, C., & King, J.-R. (2022) · Communications Biology · doi:10.1038/s42003-022-03036-1 · Cited by 318
“First, our work complements previous studies 26,27,30 – 34 and con fi rms that the activations of deep language models signi fi cantly map onto the brain responses to written sentences (Fig.”
ResultEvidence of a predictive coding hierarchy in the human brain listening to speech
Caucheteux, C., Gramfort, A., & King, J.-R. (2023) · Nature Human Behaviour · doi:10.1038/s41562-022-01516-2 · Cited by 308
Incorporating Context into Language Encoding Models for fMRI
Jain, S., & Huth, A. G. (2018) · doi:10.1101/327601 · Cited by 155
“Further, previous studies suggest that representations learned by LMs can be useful for modeling brain responses (23; 19).”
Background
Metadata
- Authors
- Wehbe, L., Vaswani, A., Knight, K., & Mitchell, T.
- Affiliations
- Carnegie Mellon University (US)
- University of Southern California (US)
- Venue
- Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Published
- 2014
- Pages
- 233-243
- OpenAlex
- W2170167574
- Cited by
- 95
- References
- 21
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 2 · 2023: 15 · 2022: 10 · 2021: 16 · 2020: 14 · 2019: 15 · 2018: 7 · 2017: 2 · 2016: 5 · 2015: 2
- FWCI
- 5.57
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
- Social Sciences › Psychology › Developmental and Educational Psychology › Reading and Literacy Development
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- reading, context modeling, latent representations, word embeddings, magnetoencephalography, word predictability, word integration, memory retrieval, natural language processing models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Aligning context-based statistical models of language with brain activity during reading. https://app.cochat.ai/papers/10.3115/v1/d14-1030
- The paper
- Wehbe, L., Vaswani, A., Knight, K., & Mitchell, T. (2014). Aligning context-based statistical models of language with brain activity during reading. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). https://doi.org/10.3115/v1/d14-1030
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.