A Neurosemantic Theory of Concrete Noun Representation Based on the Underlying Brain Codes
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
This article describes the discovery of a set of biologically-driven semantic dimensions underlying the neural representation of concrete nouns, and then demonstrates how a resulting theory of noun representation can be used to identify simple thoughts through their fMRI patterns. We use factor analysis of fMRI brain imaging data to reveal the biological representation of individual concrete nouns like apple, in the absence of any pictorial stimuli. From this analysis emerge three main semantic factors underpinning the neural representation of nouns naming physical objects, which we label manipulation, shelter, and eating. Each factor is neurally represented in 3-4 different brain locations that correspond to a cortical network that co-activates in non-linguistic tasks, such as tool use pantomime for the manipulation factor. Several converging methods, such as the use of behavioral ratings of word meaning and text corpus characteristics, provide independent evidence of the centrality of these factors to the representations. The factors are then used with machine learning classifier techniques to show that the fMRI-measured brain representation of an individual concrete noun like apple can be identified with good accuracy from among 60 candidate words, using only the fMRI activity in the 16 locations associated with these factors. To further demonstrate the generativity of the proposed account, a theory-based model is developed to predict the brain activation patterns for words to which the algorithm has not been previously exposed. The methods, findings, and theory constitute a new approach of using brain activity for understanding how object concepts are represented in the mind.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., Mazoyer, B., & Joliot, M. (2002) · NeuroImage · doi:10.1006/nimg.2001.0978 · Cited by 16,319
Indexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
Recognition-by-components: A theory of human image understanding.
Biederman, I. (1987) · Psychological Review · doi:10.1037/0033-295x.94.2.115 · Cited by 5,621
Distributed and overlapping representations of faces and objects in ventral temporal cortex
Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., & Pietrini, P. (2001) · Science · doi:10.1126/science.1063736 · Cited by 3,940 · cited 1×
“a given category, such as houses) whose visual forms were being represented (probably in an abstract form) in secondary visual processing areas, particularly ventral temporal cortex, and the activation patterns were then identified as being associated with a particular category [8].”
Miller, G. A., & Johnson-Laird, P. N. (1976) · Harvard University Press eBooks · doi:10.4159/harvard.9780674421288 · Cited by 2,629 · cited 1×
“There is a rich history of applying dimension-reduction techniques, such as factor analysis and multidimensional scaling, to behavioral data to recover the underlying dimensions of meanings of words, including classic studies of color terms [4], verbs of motion [5], animals [6], and a variety of different domains [7].”
Beyond mind-reading: multi-voxel pattern analysis of fMRI data
Norman, K. A., Polyn, S. M., Detre, G. J., & Haxby, J. V. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.07.005 · Cited by 2,271 · cited 1×
“Moreover, by applying machine learning or pattern-based classification methods to fMRI data (reviewed in [11–13]), such studies have succeeded in finding a mapping between multivariate patterns of brain activity and a given object category.”
Somatotopic Representation of Action Words in Human Motor and Premotor Cortex
Hauk, O., Johnsrude, I., & Pulvermüller, F. (2004) · Neuron · doi:10.1016/s0896-6273(03)00838-9 · Cited by 2,197 · cited 1×
“, and the representation is distributed across a number of relevant brain areas; for example, the information about the physical actions that can be applied to an object are represented in cortical areas related to control of hand actions [1,2].”
Decoding mental states from brain activity in humans
Haynes, J.-D., & Rees, G. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1931 · Cited by 1,817 · cited 1×
“Moreover, by applying machine learning or pattern-based classification methods to fMRI data (reviewed in [11–13]), such studies have succeeded in finding a mapping between multivariate patterns of brain activity and a given object category.”
Predicting Human Brain Activity Associated with the Meanings of Nouns
Mitchell, T. M., Shinkareva, S. V., Carlson, A., Chang, K.-M., Malave, V. L., Mason, R. A., & Just, M. A. (2008) · Science · doi:10.1126/science.1152876 · Cited by 1,180 · cited 7×
“In the case of discrete physical objects, the neural representations can be related to verbs of perception and action that apply to the objects [1].”
InfluentialSelf-citationCox, D. D., & Savoy, R. L. (2003) · NeuroImage · doi:10.1016/s1053-8119(03)00049-1 · Cited by 1,053
Cited by
Natural speech reveals the semantic maps that tile human cerebral cortex
Huth, A. G., de Heer, W. A., Griffiths, T. L., Theunissen, F. E., & Gallant, J. L. (2016) · Nature · doi:10.1038/nature17637 · Cited by 1,504
Huth, A. G., Nishimoto, S., Vu, A. T., & Gallant, J. L. (2012) · Neuron · doi:10.1016/j.neuron.2012.10.014 · Cited by 896
Distributed Representations in Memory: Insights from Functional Brain Imaging
Rissman, J., & Wagner, A. D. (2012) · Annual Review of Psychology · doi:10.1146/annurev-psych-120710-100344 · Cited by 299
Toward a universal decoder of linguistic meaning from brain activation
Pereira, F., Lou, B., Pritchett, B., Ritter, S., Gershman, S. J., Kanwisher, N., Botvinick, M., & Fedorenko, E. (2018) · Nature Communications · doi:10.1038/s41467-018-03068-4 · Cited by 324
Identifying emotions on the basis of neural activation
Kassam, K. S., Markey, A. R., Cherkassky, V. L., Loewenstein, G., & Just, M. A. (2013) · PLoS ONE · doi:10.1371/journal.pone.0066032 · Cited by 306
“They include a large number of anterior frontal and orbital frontal voxels, a prevalence uncommon in neurosemantic classification studies of physical objects (Just et al., 2010).”
BackgroundSelf-citationThe Hierarchical Cortical Organization of Human Speech Processing
de Heer, W. A., Huth, A. G., Griffiths, T. L., Gallant, J. L., & Theunissen, F. E. (2017) · The Journal of Neuroscience · doi:10.1523/jneurosci.3267-16.2017 · Cited by 277
The Conceptual mind: new directions in the study of concepts
(2016) · Choice Reviews Online · doi:10.5860/choice.193092 · Cited by 230
Just, M. A., Pan, L., Cherkassky, V. L., McMakin, D. L., Cha, C., Nock, M. K., & Brent, D. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0234-y · Cited by 210
RetractedModality-Independent Decoding of Semantic Information from the Human Brain
Simanova, I., Hagoort, P., Oostenveld, R., & van Gerven, M. A. J. (2014) · Cerebral Cortex · doi:10.1093/cercor/bhs324 · Cited by 138
“Some studies addressed decoding of the semantic category of visually presented words (Just et al. 2010, Shinkareva et al. 2011), or drawings of objects with captions (Mitchell et al. 2008).”
Ellis, N. C., O'Donnell, M. B., & Römer, U. (2014) · Cognitive Linguistics · doi:10.1515/cog-2013-0031 · Cited by 113
“There are promising developments in categorizing lexical concepts through their commonality in brain imaging that would be exciting to apply also (Just et al. 2010; Mitchell et al. 2008).”
BackgroundAnderson, A. J., Bruni, E., Lopopolo, A., Poesio, M., & Baroni, M. (2015) · NeuroImage · doi:10.1016/j.neuroimage.2015.06.093 · Cited by 77
“9 Materials and Methods fMRI Experiment We re-analyzed fMRI data for a popular set of stimuli (Table 1) consisting of 60 concrete nouns belonging to 12 taxonomic categories (because of limitations in the ImageNet database we use to construct our image-based computational model, we had coverage for 51 words and 11 clas…”
BackgroundMethodInfluentialAnderson, A. J., Kiela, D., Binder, J. R., Fernandino, L., Humphries, C. J., Conant, L. L., Raizada, R. D. S., Grimm, S., & Lalor, E. C. (2021) · The Journal of Neuroscience · doi:10.1523/jneurosci.1152-20.2021 · Cited by 70
Inpatient suicide in psychiatric settings: Evaluation of current prevention measures
Chammas, F., Januel, D., & Bouaziz, N. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.997974 · Cited by 38
“This approach is based on recent advances in cognitive neuroscience that use machine-learning techniques to identify a concept from its signature in functional MRI (104, 106, 107).”
BackgroundMethodInfluentialHidden Stages of Cognition Revealed in Patterns of Brain Activation
Anderson, J. R., Pyke, A. A., & Fincham, J. M. (2016) · Psychological Science · doi:10.1177/0956797616654912 · Cited by 29
“There is a rich history in cognitive psychology of trying to infer the stages of a task by using effects of manipulations on total time and their interactions, such as in the additive-factors logic employed by Sternberg (1969). While such endeavors have had some success, they have been limited by the impoverished natu…”
Background
Metadata
- Authors
- Just, M. A., Cherkassky, V. L., Aryal, S., & Mitchell, T. M.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2010-01-13
- Volume
- 5
- Issue
- 1
- Pages
- e8622
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2042684628
- Cited by
- 207
- References
- 26
- FWCI
- 11.04
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.