Conceptual grounding of language in action and perception: a neurocomputational model of the emergence of category specificity and semantic hubs
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
Current neurobiological accounts of language and cognition offer diverging views on the questions of ‘where’ and ‘how’ semantic information is stored and processed in the human brain. Neuroimaging data showing consistent activation of different multi‐modal areas during word and sentence comprehension suggest that all meanings are processed indistinctively, by a set of general semantic centres or ‘hubs’. However, words belonging to specific semantic categories selectively activate modality‐preferential areas; for example, action‐related words spark activity in dorsal motor cortex, whereas object‐related ones activate ventral visual areas. The evidence for category‐specific and category‐general semantic areas begs for a unifying explanation, able to integrate the emergence of both. Here, a neurobiological model offering such an explanation is described. Using a neural architecture replicating anatomical and neurophysiological features of frontal, occipital and temporal cortices, basic aspects of word learning and semantic grounding in action and perception were simulated. As the network underwent training, distributed lexico‐semantic circuits spontaneously emerged. These circuits exhibited different cortical distributions that reached into dorsal‐motor or ventral‐visual areas, reflecting the correlated category‐specific sensorimotor patterns that co‐occurred during action‐ or object‐related semantic grounding, respectively. Crucially, substantial numbers of neurons of both types of distributed circuits emerged in areas interfacing between modality‐preferential regions, i.e. in multimodal connection hubs, which therefore became loci of general semantic binding. By relating neuroanatomical structure and cellular‐level learning mechanisms with system‐level cognitive function, this model offers a neurobiological account of category‐general and category‐specific semantic areas based on the different cortical distributions of the underlying semantic circuits.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
A spreading-activation theory of semantic processing.
Collins, A. M., & Loftus, E. F. (1975) · Psychological Review · doi:10.1037/0033-295x.82.6.407 · Cited by 8,141 · cited 5×
“The authors would also like to thank the HPC Service of ZEDAT, Freie Universit€at Berlin, for computing time, and Malte Schomers, Mir o Herrmann for their help at proof revision stage.”
BackgroundMethodInfluentialRizzolatti, G., & Craighero, L. (2004) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144230 · Cited by 7,433 · cited 2×
“The authors would also like to thank the HPC Service of ZEDAT, Freie Universit€at Berlin, for computing time, and Malte Schomers, Mir o Herrmann for their help at proof revision stage.”
BackgroundMethodSearle, J. R. (1980) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00005756 · Cited by 6,954
Landauer, T. K., & Dumais, S. T. (1997) · Psychological Review · doi:10.1037/0033-295x.104.2.211 · Cited by 6,282
Barsalou, L. W. (2008) · Annual Review of Psychology · doi:10.1146/annurev.psych.59.103006.093639 · Cited by 5,936 · cited 7×
“The authors would also like to thank the HPC Service of ZEDAT, Freie Universit€at Berlin, for computing time, and Malte Schomers, Mir o Herrmann for their help at proof revision stage.”
BackgroundMethodInfluentialHarnad, S. (1990) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(90)90087-6 · Cited by 4,235 · cited 2×
“Using a neural architecture replicating anatomical and neurophysiological features of frontal, occipital and temporal cortices, basic aspects of word learning and semantic grounding in action and perception were simulated.”
BackgroundFrom Neuropsychology to Mental Structure
Shallice, T. (1988) · doi:10.1017/cbo9780511526817 · Cited by 4,072
Malenka, R. C., & Bear, M. F. (2004) · Neuron · doi:10.1016/j.neuron.2004.09.012 · Cited by 3,968
Cattaneo, L., & Rizzolatti, G. (2009) · Archives of Neurology · doi:10.1001/archneurol.2009.41 · Cited by 3,137
Object vision and spatial vision: Two cortical pathways
Mishkin, M., Ungerleider, L. G., & Macko, K. A. (1983) · Trends in Neurosciences · doi:10.1016/0166-2236(83)90190-x · Cited by 2,947
Where do you know what you know? The representation of semantic knowledge in the human brain
Patterson, K., Nestor, P. J., & Rogers, T. T. (2007) · Nature Reviews Neuroscience · doi:10.1038/nrn2277 · Cited by 2,706
Network hubs in the human brain
van den Heuvel, M. P., & Sporns, O. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.09.012 · Cited by 2,123 · cited 1×
“…likely present not only in language and motor systems but also in the ventral visual ‘what’ stream (e.g. if the same object is being repeatedly grasped; Ungerleider & Mishkin, 1982; Mishkin et al., 1983; Ungerleider & Haxby, 1994), as well as in the dorsal parieto-occipital visual ‘where’ stream…”
BackgroundThe neurobiology of semantic memory
Binder, J. R., & Desai, R. H. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.10.001 · Cited by 2,114
‘What’ and ‘where’ in the human brain
Ungerleider, L. G., & Haxby, J. V. (1994) · Current Opinion in Neurobiology · doi:10.1016/0959-4388(94)90066-3 · Cited by 1,913 · cited 2×
“The authors would also like to thank the HPC Service of ZEDAT, Freie Universit€at Berlin, for computing time, and Malte Schomers, Mir o Herrmann for their help at proof revision stage.”
BackgroundMethodMeta-analyzing left hemisphere language areas: phonology, semantics, and sentence processing.
Vigneau, M., Beaucousin, V., Hervé, P. Y., Duffau, H., Crivello, F., Houdé, O., Mazoyer, B., & Tzourio-Mazoyer, N. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2005.11.002 · Cited by 1,851 · cited 2×
“Here, a neurobiological model offering such an explanation is described.”
BackgroundPerisylvian language networks of the human brain
Catani, M., Jones, D. K., & ffytche, D. H. (2005) · Annals of Neurology · doi:10.1002/ana.20319 · Cited by 1,832 · cited 1×
“…(and premotor) cortex (PFi) are realised by the arcuate and uncinate fascicles (Makris et al., 1999; Romanski et al., 1999b; Petrides & Pandya, 2001, 2009; Catani et al., 2005; Parker et al., 2005; Romanski, 2007; Rilling et al., 2008; Makris & Pandya, 2009; Petrides et al., 2012; Rilling, 2014).”
BackgroundIntrinsic functional architecture in the anaesthetized monkey brain
Vincent, J. L., Patel, G. H., Fox, M. D., Snyder, A. Z., Baker, J. T., Van Essen, D. C., Zempel, J. M., Snyder, L. H., Corbetta, M., & Raichle, M. E. (2007) · Nature · doi:10.1038/nature05758 · Cited by 1,786 · cited 2×
“The authors would also like to thank the HPC Service of ZEDAT, Freie Universit€at Berlin, for computing time, and Malte Schomers, Mir o Herrmann for their help at proof revision stage.”
BackgroundMethodRethinking Innateness: A Connectionist Perspective on Development
Elman, J., Bates, E., Johnson, M. H., Karmiloff-Smith, A., Parisi, D., & Plunkett, K. (1996) · doi:10.7551/mitpress/5929.001.0001 · Cited by 1,511
Grasping objects: the cortical mechanisms of visuomotor transformation
Jeannerod, M., Arbib, M. A., Rizzolatti, G., & Sakata, H. (1995) · Trends in Neurosciences · doi:10.1016/0166-2236(95)93921-j · Cited by 1,399
Pulvermüller, F. (1999) · Behavioral and Brain Sciences · doi:10.1017/s0140525x9900182x · Cited by 1,284
Cortex: Statistics and Geometry of Neuronal Connectivity
Braitenberg, V., & Schüz, A. (1998) · doi:10.1007/978-3-662-03733-1 · Cited by 1,178 · cited 4×
“…into account (Husain et al., 2004; Guenther et al., 2006; Ueno et al., 2011), they either did not incorporate learning mechanisms, or made use of ones (e.g. back-propagation) whose neurobiological plausibility is questionable (Mazzoni et al., 1991; Braitenberg & Sch€uz, 1998; O’Reilly, 1998).”
BackgroundMethodInfluentialSchmahmann, J. D., Pandya, D. N., Wang, R., Dai, G., D'Arceuil, H. E., de Crespigny, A. J., & Wedeen, V. J. (2007) · Brain · doi:10.1093/brain/awl359 · Cited by 1,067 · cited 2×
“…linked to anterior and inferior temporal regions (AT; Pandya & Barnes, 1987; Ungerleider et al., 1989; Webster et al., 1994), as well as to the superior temporal cortex (PB) via the extreme capsule (Pandya & Barnes, 1987; Romanski et al., 1999a,b; Schmahmann et al., 2007; Dick et al., 2014).”
BackgroundPetrides, M., & Pandya, D. N. (2002) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.2001.02090.x · Cited by 882 · cited 1×
“The authors would also like to thank the HPC Service of ZEDAT, Freie Universit€at Berlin, for computing time, and Malte Schomers, Mir o Herrmann for their help at proof revision stage.”
MethodMotor areas in the frontal lobe of the primate
DUM, R., & STRICK, P. (2002) · Physiology & Behavior · doi:10.1016/s0031-9384(02)00929-0 · Cited by 653
How neurons make meaning: brain mechanisms for embodied and abstract-symbolic semantics
Pulvermüller, F. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.06.004 · Cited by 631 · cited 3×
“Using a neural architecture replicating anatomical and neurophysiological features of frontal, occipital and temporal cortices, basic aspects of word learning and semantic grounding in action and perception were simulated.”
BackgroundTopographic Maps are Fundamental to Sensory Processing
Kaas, J. H. (1997) · Brain Research Bulletin · doi:10.1016/s0361-9230(97)00094-4 · Cited by 478
Distinct parietal and temporal pathways to the homologues of Broca's area in the monkey
Petrides, M., & Pandya, D. N. (2009) · PLoS Biology · doi:10.1371/journal.pbio.1000170 · Cited by 396 · cited 2×
“…auditory and visual (as well as prefrontal) cortices is much more complex than that realised here (Felleman & Van Essen, 1991; Kaas & Hackett, 2000; Petrides & Pandya, 2001, 2009; Vincent et al., 2007; Rauschecker & Scott, 2009; for a recent discussion on perisylvian connectivity, see also…”
BackgroundThe modular organization of projections from areas V1 and V2 to areas V4 and TEO in macaques
Nakamura, H., Gattass, R., Desimone, R., & Ungerleider, L. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-09-03681.1993 · Cited by 211 · cited 2×
“Furthermore, two separate three-way ANOVAs were run on the data from the six extrasylvian and six perisylvian areas (factors ‘FrontoTemp’, ‘ModSpecificity’ and ‘WordType’, as above).”
MethodBrain mechanisms for perceptual and reward-related decision-making
Deco, G., Rolls, E. T., Albantakis, L., & Romo, R. (2013) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2012.01.010 · Cited by 196
The role of metaplasticity mechanisms in regulating memory destabilization and reconsolidation
Finnie, P. S. B., & Nader, K. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.03.008 · Cited by 189
Cited by
Challenges and opportunities for grounding cognition.
Barsalou, L. W. (2020) · Journal of Cognition · doi:10.5334/joc.116 · Cited by 319
Semantic memory: A review of methods, models, and current challenges
Kumar, A. A. (2021) · Psychonomic Bulletin & Review · doi:10.3758/s13423-020-01792-x · Cited by 314
Neural reuse of action perception circuits for language, concepts and communication
Pulvermüller, F. (2018) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2017.07.001 · Cited by 288
Self-citationWords have a weight: language as a source of inner grounding and flexibility in abstract concepts
Dove, G., Barca, L., Tummolini, L., & Borghi, A. M. (2022) · Psychological Research · doi:10.1007/s00426-020-01438-6 · Cited by 80
“They hold that concepts rely on a hierarchy of neural circuits that extend from modality-specific areas up to multimodal areas located within association cortices (Binder, 2016; Ferandino et al., 2016; Garagnani & Pulvermüller, 2016; Simmons & Barsalou, 2003).”
BackgroundConcepts in the Brain: The View From Cross-linguistic Diversity
Kemmerer, D. (2019) · doi:10.1093/oso/9780190682620.001.0001 · Cited by 42
Metadata
- Authors
- Garagnani, M., & Pulvermüller, F.
- Venue
- European Journal of Neuroscience
- Publisher
- Wiley
- Published
- 2016-02-09
- Volume
- 43
- Issue
- 6
- Pages
- 721-737
- ISSN
- 0953-816X (ISSN-L)
- 1460-9568
- OpenAlex
- W2192773587
- Cited by
- 130
- References
- 31
- FWCI
- 4.24
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.