Predicting Human Brain Activity Associated with the Meanings of Nouns
Open accessAbstract
The question of how the human brain represents conceptual knowledge has been debated in many scientific fields. Brain imaging studies have shown that different spatial patterns of neural activation are associated with thinking about different semantic categories of pictures and words (for example, tools, buildings, and animals). We present a computational model that predicts the functional magnetic resonance imaging (fMRI) neural activation associated with words for which fMRI data are not yet available. This model is trained with a combination of data from a trillion-word text corpus and observed fMRI data associated with viewing several dozen concrete nouns. Once trained, the model predicts fMRI activation for thousands of other concrete nouns in the text corpus, with highly significant accuracies over the 60 nouns for which we currently have fMRI data.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Current word embedding-based language encoding approaches treat each stimulus word independently and ignore context.
“Current word embedding-based approaches treat each stimulus word independently and thus ignore the influence of context on language understanding.”
From Incorporating Context into Language Encoding Models for fMRI (2018) · cites this paper as [15]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The fusiform face area: a module in human extrastriate cortex specialized for face perception
Kanwisher, N., McDermott, J., & Chun, M. M. (1997) · The Journal of Neuroscience · doi:10.1523/jneurosci.17-11-04302.1997 · Cited by 7,738 · see citing works
Beyond mind-reading: multi-voxel pattern analysis of fMRI data
Norman, K. A., Polyn, S. M., Detre, G. J., & Haxby, J. V. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.07.005 · Cited by 2,271 · see citing works
Flagged first, then by citations
Research works printed but not linked: 28
- References 12 November 2007; accepted 3 April 2008 10.1126/science.ll52876 References and Notes 1. 3-V. Haxby et ai, Science 293, 2425 (2001). · DOI
- A. Ishai, L G. Ungedeider, A. Martin, J. L Scheuten, ]. V. Haxby, Proc Nati Acad. Sei. USA. 96, 9379 (1999).
- T, A. Carlson, P. Schrater, S. He, /. Cogn. Neurosa 15, 704 (2003).
- D. D. Cox, R. I. Savoy, Neuroimage 19, 261 (2003).
- S. ]. Hanson, T. Matsuka, ). V. Haxby, Neuroimage 23, 156 (2004).
- S. M. Polyn, V. S. Natu,}. D. Cohen, K. A. Norman, Science 310, 1963 (2005).
- A. ]. O'Toole, F. Jiang, H. Abdi? ]. V. Haxby, j. Cogn. Neurosa 17, 580 (2005).
- K. Kipper, A. Korhonen, H. Ryant? M. Palmer? Proceedings of the 5th International Conference on Language Resources and Evaluation, Genoa, Italy, 24 to 26 May 2006,
- G. Milter, R. Beckwith, C Fe?baum, D. Gross, K. j. Miller, ?nt I Lexicography 3, 235 (1990).
- C. Fellbaum? Ed., WordNet: An Electronic Lexical Database (Massachusetts Institute of Technology Press, Cambridge? MA, 1998).
- K. W. Church, P. Hanks, Comput Linguist 16, 22 (1990).
- T. K. Landauer, S. T. Dumais, PsychoL Rev. 104, 211 (1997).
- D. Lin, 5. Zhao? I. Oin, M. Zhou, Proceedings of the 18th International Joint Conference on Artificial intelligence, Acapulco, Mexico, August 2003 (Morgan Kaufmann, San Francisco, 2003), pp. 1492-1493.
- R, Snow, D. jurafsky, A. Ng? Proceedings of the 44th Annual Meeting of the Association for Computational Linguistics, Sydney, Australia, 17 to 21 July 2006.
- S. ]. Crutch? E. K. Warrington, Brain 126, 1821 (2003).
- D. Samson? A. Pillon, Brain Lang, 91, 252 (2004).
- A. Martin, L L Chao, Curr. Opin. Neurobioi. 11,194 (2001).
- R. F. Goldberg, C. A. Perfetti, W. Schneider? J. Neurosci. 26, 4917 (2006).
- B. Z. Mahon, A. Caramazza, in The Encyclopedia of Language and Linguistics, K. Brown? Ed. (Elsevier Science, Amsterdam? ed. 2, 2005).
- K. J. Friston er al, Hum. Brain Mapp. 2, 189 (1995).
- N. Tzourio-Mazoyer et ai, Neuroimage 15, 273 (2002).
- A. Martin, L G. Ungerteider, ]. V. Haxby, in The New Cognitive Neurosciences, M. 5. Gazzainga, Ed, (Massachusetts Institute of Technology Press, Cambridge, MA, ed. 2, 2000), pp. 1023-1036.
- K. A. Peiphrey, ]. P. Morris, C. R. Michelich, T. Allison, G. McCarthy, Cereb. Cortex 15, 1866 (2005).
- L M. Vaina, ]. Solomon, S. Chowdhury, P. Sinha, J. Belliveau, Proc Nati Acad Sei USA. 98, 11656 (2001).
- K. Sakai er ai, Magn. Reson. Med. 33, 736 (1995).
- D. R. Hardoon, j. Mourao-Miranda, M. Brammer? J. Shawe-Taylor, Neuroimage 37, 1250 (2007).
- J. D. Haynes, G. Rees, Nat Rev. Neurosa, 7, 523 (2006).
- K. A. Norman, S. M. Polyn, G. ). Detre, ]. V. Haxby, Trends Cogn. Sei 10, 424 (2006).
Cited by
Sorted by citations
Large-scale automated synthesis of human functional neuroimaging data.
Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C., & Wager, T. D. (2011) · Nature Methods · doi:10.1038/nmeth.1635 · Cited by 3,686 · see citing works
The brain basis of emotion: a meta-analytic review
Lindquist, K. A., Wager, T. D., Kober, H., Bliss-Moreau, E., & Barrett, L. F. (2012) · Behavioral and Brain Sciences · doi:10.1017/s0140525x11000446 · Cited by 2,320 · see citing works
“…“machine”) learning, could in principle be used to discover which kinds of psychological categories are most consistently and specifically respected in the brain (Mitchell et al. 2008; Yarkoni et al. 2011), potentially providing new tests of locationist versus constructionist approaches to emotion.”
Prediction of individual brain maturity using fMRI
Dosenbach, N. U. F., Nardos, B., Cohen, A. L., Fair, D. A., Power, J. D., Church, J. A., Nelson, S. M., Wig, G. S., Vogel, A. C., Lessov-Schlaggar, C. N., et al. (2010) · Science · doi:10.1126/science.1194144 · Cited by 2,183 · see citing works
Machine learning classifiers and fMRI: A tutorial overview
Pereira, F., Mitchell, T., & Botvinick, M. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.11.007 · Cited by 1,624 · see citing works
Self-citationMachine learning for medical imaging
Erickson, B. J., Korfiatis, P., Akkus, Z., & Kline, T. L. (2017) · RadioGraphics · doi:10.1148/rg.2017160130 · Cited by 1,672 · see citing works
Natural speech reveals the semantic maps that tile human cerebral cortex
Huth, A. G., de Heer, W. A., Griffiths, T. L., Theunissen, F. E., & Gallant, J. L. (2016) · Nature · doi:10.1038/nature17637 · Cited by 1,504 · see citing works
Deep supervised, but not unsupervised, models may explain IT cortical representation
Khaligh-Razavi, S.-M., & Kriegeskorte, N. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003915 · Cited by 1,204 · see citing works
Kriegeskorte, N. (2015) · Annual Review of Vision Science · doi:10.1146/annurev-vision-082114-035447 · Cited by 1,093 · see citing works
Building better biomarkers: brain models in translational neuroimaging
Woo, C.-W., Chang, L. J., Lindquist, M. A., & Wager, T. D. (2017) · Nature Neuroscience · doi:10.1038/nn.4478 · Cited by 1,056 · see citing works
Inferring mental states from neuroimaging data: from reverse inference to large-scale decoding
Poldrack, R. A. (2011) · Neuron · doi:10.1016/j.neuron.2011.11.001 · Cited by 821 · see citing works
Shared memories reveal shared structure in neural activity across individuals
Chen, J., Leong, Y. C., Honey, C. J., Yong, C. H., Norman, K. A., & Hasson, U. (2017) · Nature Neuroscience · doi:10.1038/nn.4450 · Cited by 645 · see citing works
Decoding Neural Representational Spaces Using Multivariate Pattern Analysis
Haxby, J. V., Connolly, A. C., & Guntupalli, J. S. (2014) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-062012-170325 · Cited by 717 · see citing works
Reconstructing Speech from Human Auditory Cortex
Pasley, B. N., David, S. V., Mesgarani, N., Flinker, A., Shamma, S. A., Crone, N. E., Knight, R. T., & Chang, E. F. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001251 · Cited by 604 · see citing works
Cognitive computational neuroscience
Kriegeskorte, N., & Douglas, P. K. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0210-5 · Cited by 535 · see citing works
A Primer on Pattern-Based Approaches to fMRI: Principles, Pitfalls, and Perspectives
Haynes, J.-D. (2015) · Neuron · doi:10.1016/j.neuron.2015.05.025 · Cited by 402 · see citing works
Semantic reconstruction of continuous language from non-invasive brain recordings
Tang, J., LeBel, A., Jain, S., & Huth, A. G. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01304-9 · Cited by 442 · see citing works
The ontogeny of the cortical language network
Skeide, M. A., & Friederici, A. D. (2016) · Nature Reviews Neuroscience · doi:10.1038/nrn.2016.23 · Cited by 391 · see citing works
Decoding Patterns of Human Brain Activity
Tong, F., & Pratte, M. S. (2012) · Annual Review of Psychology · doi:10.1146/annurev-psych-120710-100412 · Cited by 375 · see citing works
Toward a universal decoder of linguistic meaning from brain activation
Pereira, F., Lou, B., Pritchett, B., Ritter, S., Gershman, S. J., Kanwisher, N., Botvinick, M., & Fedorenko, E. (2018) · Nature Communications · doi:10.1038/s41467-018-03068-4 · Cited by 324 · see citing works
Attention during natural vision warps semantic representation across the human brain
Çukur, T., Nishimoto, S., Huth, A. G., & Gallant, J. L. (2013) · Nature Neuroscience · doi:10.1038/nn.3381 · Cited by 351 · see citing works
Brains and algorithms partially converge in natural language processing
Caucheteux, C., & King, J.-R. (2022) · Communications Biology · doi:10.1038/s42003-022-03036-1 · Cited by 318 · see citing works
On staying grounded and avoiding quixotic dead ends
Barsalou, L. W. (2016) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1028-3 · Cited by 331 · see citing works
Wager, T. D., Atlas, L. Y., Leotti, L. A., & Rilling, J. K. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.3420-10.2011 · Cited by 328 · see citing works
Distributed Representations in Memory: Insights from Functional Brain Imaging
Rissman, J., & Wagner, A. D. (2012) · Annual Review of Psychology · doi:10.1146/annurev-psych-120710-100344 · Cited by 299 · see citing works
Identifying emotions on the basis of neural activation
Kassam, K. S., Markey, A. R., Cherkassky, V. L., Loewenstein, G., & Just, M. A. (2013) · PLoS ONE · doi:10.1371/journal.pone.0066032 · Cited by 306 · see citing works
Self-citationMental imagery of speech and movement implicates the dynamics of internal forward models
Tian, X. (2010) · Frontiers in Psychology · doi:10.3389/fpsyg.2010.00166 · Cited by 259 · see citing works
“This same data-analytic approach has been applied, as well, in fMRI research (Mitchell et al., 2008; Braze et al., 2010) and neural network modeling (Hinton and Shallice, 1991).”
Pei, X., Barbour, D. L., Leuthardt, E. C., & Schalk, G. (2011) · Journal of Neural Engineering · doi:10.1088/1741-2560/8/4/046028 · Cited by 237 · see citing works
Deniz, F., Nunez-Elizalde, A. O., Huth, A. G., & Gallant, J. L. (2019) · The Journal of Neuroscience · doi:10.1523/jneurosci.0675-19.2019 · Cited by 214 · see citing works
Just, M. A., Pan, L., Cherkassky, V. L., McMakin, D. L., Cha, C., Nock, M. K., & Brent, D. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0234-y · Cited by 210 · see citing works
RetractedA Direct Brain-to-Brain Interface in Humans
Rao, R. P. N., Stocco, A., Bryan, M., Sarma, D., Youngquist, T. M., Wu, J., & Prat, C. S. (2014) · PLoS ONE · doi:10.1371/journal.pone.0111332 · Cited by 210 · see citing works
The representation of self and person knowledge in the medial prefrontal cortex
Wagner, D. D., Haxby, J. V., & Heatherton, T. F. (2012) · WIREs Cognitive Science · doi:10.1002/wcs.1183 · Cited by 201 · see citing works
Deep neural networks in computational neuroscience
Kietzmann, T. C., McClure, P., & Kriegeskorte, N. (2019) · Oxford Research Encyclopedia of Neuroscience · doi:10.1093/acrefore/9780190264086.013.46 · Cited by 186 · see citing works
Generative embedding for model-based classification of fMRI data
Brodersen, K. H., Schofield, T. M., Leff, A. P., Ong, C. S., Lomakina, E. I., Buhmann, J. M., & Stephan, K. E. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002079 · Cited by 165 · see citing works
Incorporating Context into Language Encoding Models for fMRI
Jain, S., & Huth, A. G. (2018) · doi:10.1101/327601 · Cited by 155 · see citing works
Modality-Independent Decoding of Semantic Information from the Human Brain
Simanova, I., Hagoort, P., Oostenveld, R., & van Gerven, M. A. J. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhs324 · Cited by 138 · see citing works
Measuring neural representations with fMRI: practices and pitfalls
Davis, T., & Poldrack, R. A. (2013) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.12156 · Cited by 132 · see citing works
Ellis, N. C., O'Donnell, M. B., & Römer, U. (2014) · Cognitive Linguistics · doi:10.1515/cog-2013-0031 · Cited by 113 · see citing works
Brain2Char: a deep architecture for decoding text from brain recordings
Sun, P., Anumanchipalli, G. K., & Chang, E. F. (2020) · Journal of Neural Engineering · doi:10.1088/1741-2552/abc742 · Cited by 77 · see citing works
Unification of sentence processing via ear and eye: An fMRI study
Braze, D., Mencl, W. E., Tabor, W., Pugh, K. R., Todd Constable, R., Fulbright, R. K., Magnuson, J. S., Van Dyke, J. A., & Shankweiler, D. P. (2011) · Cortex · doi:10.1016/j.cortex.2009.11.005 · Cited by 69 · see citing works
Anderson, A. J., Kiela, D., Binder, J. R., Fernandino, L., Humphries, C. J., Conant, L. L., Raizada, R. D. S., Grimm, S., & Lalor, E. C. (2021) · The Journal of Neuroscience · doi:10.1523/jneurosci.1152-20.2021 · Cited by 70 · see citing works
Goldstein, A., Grinstein-Dabush, A., Schain, M., Wang, H., Hong, Z., Aubrey, B., Nastase, S. A., Zada, Z., Ham, E., Feder, A., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-46631-y · Cited by 64 · see citing works
A cryptography-based approach for movement decoding
Dyer, E. L., Gheshlaghi Azar, M., Perich, M. G., Fernandes, H. L., Naufel, S., Miller, L. E., & Körding, K. P. (2017) · Nature Biomedical Engineering · doi:10.1038/s41551-017-0169-7 · Cited by 70 · see citing works
Contributions of cortical feedback to sensory processing in primary visual cortex
Petro, L. S., Vizioli, L., & Muckli, L. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01223 · Cited by 61 · see citing works
Leveraging shared connectivity to aggregate heterogeneous datasets into a common response space
Nastase, S. A., Liu, Y.-F., Hillman, H., Norman, K. A., & Hasson, U. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116865 · Cited by 42 · see citing works
Concepts in the Brain: The View From Cross-linguistic Diversity
Kemmerer, D. (2019) · doi:10.1093/oso/9780190682620.001.0001 · Cited by 42 · see citing works
Inpatient suicide in psychiatric settings: Evaluation of current prevention measures
Chammas, F., Januel, D., & Bouaziz, N. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.997974 · Cited by 38 · see citing works
Model-based feature construction for multivariate decoding
Brodersen, K. H., Haiss, F., Ong, C. S., Jung, F., Tittgemeyer, M., Buhmann, J. M., Weber, B., & Stephan, K. E. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.04.036 · Cited by 36 · see citing works
The promise of awake behaving infant fMRI as a deep measure of cognition
Yates, T. S., Ellis, C. T., & Turk-Browne, N. B. (2021) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2020.11.007 · Cited by 29 · see citing works
The brain of the beholder: honouring individual representational idiosyncrasies
Charest, I., & Kriegeskorte, N. (2015) · Language, Cognition and Neuroscience · doi:10.1080/23273798.2014.1002505 · Cited by 27 · see citing works
Simistira Liwicki, F., Gupta, V., Saini, R., De, K., Abid, N., Rakesh, S., Wellington, S., Wilson, H., Liwicki, M., & Eriksson, J. (2023) · Scientific Data · doi:10.1038/s41597-023-02286-w · Cited by 28 · see citing works
- Authors
- Mitchell, T. M., Shinkareva, S. V., Carlson, A., Chang, K.-M., Malave, V. L., Mason, R. A., & Just, M. A.
- Affiliations
- University of South Carolina (US)
- Corresponding author
- Mitchell, Tom M.
- Venue
- Science
- Published
- 2008-05-30
- Volume
- 320
- Issue
- 5880
- Pages
- 1191-1195
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2168217710
- Cited by
- 1,180
- References
- 2
- Citations by year
- 2026: 36 · 2025: 51 · 2024: 42 · 2023: 46 · 2022: 62 · 2021: 73 · 2020: 87 · 2019: 91 · 2018: 74 · 2017: 70 · 2016: 71 · 2015: 72 · 2014: 74 · 2013: 65 · 2012: 72
- FWCI
- 33.52
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- other-oa
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
- KiltHub Repository (submittedVersion)
- Figshare (submittedVersion, cc-by)
- KiltHub Repository
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Categorization, perception, and language
- Social Sciences › Psychology › Developmental and Educational Psychology › Reading and Literacy Development
- Keywords
- semantic representation, concrete nouns, word meaning, functional MRI, computational model, text corpora, semantic categories, conceptual knowledge
- MeSH terms
- Adolescent, Adult, Brain, Brain Mapping, Female, Humans, Language, Magnetic Resonance Imaging, Male, Models, Neurological, Semantics, Speech Perception, Models, Statistical, Computational Biology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
