Interactions between Auditory and Visual Semantic Stimulus Classes: Evidence for Common Processing Networks for Speech and Body Actions
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Incongruencies between auditory and visual signals negatively affect human performance and cause selective activation in neuroimaging studies; therefore, they are increasingly used to probe audiovisual integration mechanisms. An open question is whether the increased BOLD response reflects computational demands in integrating mismatching low-level signals or reflects simultaneous unimodal conceptual representations of the competing signals. To address this question, we explore the effect of semantic congruency within and across three signal categories (speech, body actions, and unfamiliar patterns) for signals with matched low-level statistics. In a localizer experiment, unimodal (auditory and visual) and bimodal stimuli were used to identify ROIs. All three semantic categories cause overlapping activation patterns. We find no evidence for areas that show greater BOLD response to bimodal stimuli than predicted by the sum of the two unimodal responses. Conjunction analysis of the unimodal responses in each category identifies a network including posterior temporal, inferior frontal, and premotor areas. Semantic congruency effects are measured in the main experiment. We find that incongruent combinations of two meaningful stimuli (speech and body actions) but not combinations of meaningful with meaningless stimuli lead to increased BOLD response in the posterior STS (pSTS) bilaterally, the left SMA, the inferior frontal gyrus, the inferior parietal lobule, and the anterior insula. These interactions are not seen in premotor areas. Our findings are consistent with the hypothesis that pSTS and frontal areas form a recognition network that combines sensory categorical representations (in pSTS) with action hypothesis generation in inferior frontal gyrus/premotor areas. We argue that the same neural networks process speech and body actions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The episodic buffer: a new component of working memory?
Baddeley, A. (2000) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01538-2 · Cited by 7,359 · cited 1×
“, Baddeley, 2000). Baddeley (2000) proposed three working memory buffers under the control of a central executive system.”
BackgroundRizzolatti, G., & Arbib, M. A. (1998) · Trends in Neurosciences · doi:10.1016/s0166-2236(98)01260-0 · Cited by 2,858
Hickok, G., & Poeppel, D. (2004) · Cognition · doi:10.1016/j.cognition.2003.10.011 · Cited by 2,212
Brain mechanisms linking language and action
Pulvermüller, F. (2005) · Nature Reviews Neuroscience · doi:10.1038/nrn1706 · Cited by 1,828 · cited 3×
“The pSTS demarcates the lower bank of Wernickeʼs area so that there is a wealth of literature linking speech perception deficits to lesions at this site (for a review, see, e.g., Damasio & Geschwind, 1984; but see Pulvermüller, 2005).”
BackgroundResultHearing Sounds, Understanding Actions: Action Representation in Mirror Neurons
Kohler, E., Keysers, C., Umiltà, M. A., Fogassi, L., Gallese, V., & Rizzolatti, G. (2002) · Science · doi:10.1126/science.1070311 · Cited by 1,801 · cited 1×
“Although the response is selective for specific actions, the modality of observation is not critical (Keysers et al., 2003; Kohler et al., 2002).”
BackgroundCrossmodal Processing in the Human Brain: Insights from Functional Neuroimaging Studies
Calvert, G. A. (2001) · Cerebral Cortex · doi:10.1093/cercor/11.12.1110 · Cited by 1,012
Calvert, G. A., Campbell, R., & Brammer, M. J. (2000) · Current Biology · doi:10.1016/s0960-9822(00)00513-3 · Cited by 954
Listening to speech activates motor areas involved in speech production
Wilson, S. M., Saygin, A. P., Sereno, M. I., & Iacoboni, M. (2004) · Nature Neuroscience · doi:10.1038/nn1263 · Cited by 885
Electrophysiology and brain imaging of biological motion
Puce, A., & Perrett, D. (2003) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.2002.1221 · Cited by 693
Separate neural subsystems within ‘Wernicke's area’
Wise, R. J. S., Scott, S. K., Blank, S. C., Mummery, C. J., Murphy, K., & Warburton, E. A. (2001) · Brain · doi:10.1093/brain/124.1.83 · Cited by 572 · cited 2×
“This view is consistent with the suggestion that the pSTS may be a phonological buffer (Wise et al., 2001) and more recent proposals by Jacquemot and Scott (2006), who propose a role of the pSTS in short-term perceptual memory.”
Result
Cited by
Diffusion tensor imaging shows white matter tracts between human auditory and visual cortex
Beer, A. L., Plank, T., & Greenlee, M. W. (2011) · Experimental Brain Research · doi:10.1007/s00221-011-2715-y · Cited by 124
Self-citationAn fMRI investigation of the relationship between future imagination and cognitive flexibility
Roberts, R. P., Wiebels, K., Sumner, R. L., van Mulukom, V., Grady, C. L., Schacter, D. L., & Addis, D. R. (2017) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2016.11.019 · Cited by 58
Beer, A. L., Plank, T., Meyer, G., & Greenlee, M. W. (2013) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2013.00005 · Cited by 53
“Several studies showing multisensory responses in the STS also reported multisensory activity in the inferior occipito-temporal cortex, anterior insula, and ventrolateral frontal cortex (Calvert et al., 2000; Beauchamp et al., 2004b; Hein et al., 2007; Meyer et al., 2011; Nath and Beauchamp, 2012).”
BackgroundMethodResultInfluentialNaumer, M. J., van den Bosch, J. J. F., Wibral, M., Kohler, A., Singer, W., Kaiser, J., van de Ven, V., & Muckli, L. (2011) · Experimental Brain Research · doi:10.1007/s00221-011-2669-0 · Cited by 11
Metadata
- Authors
- Meyer, G. F., Greenlee, M., & Wuerger, S.
- Venue
- Journal of Cognitive Neuroscience
- Publisher
- MIT Press - Journals
- Published
- 2011-09-01
- Volume
- 23
- Issue
- 9
- Pages
- 2291-2308
- ISSN
- 0898-929X (ISSN-L)
- 1530-8898
- OpenAlex
- W2127431490
- Cited by
- 52
- References
- 17
- FWCI
- 6.39
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Interactions between Auditory and Visual Semantic Stimulus Classes: Evidence for Common Processing Networks for Speech and Body Actions. https://app.cochat.ai/papers/10.1162/jocn.2010.21593
- The paper
- Meyer, G. F., Greenlee, M., & Wuerger, S. (2011). Interactions between Auditory and Visual Semantic Stimulus Classes: Evidence for Common Processing Networks for Speech and Body Actions. Journal of Cognitive Neuroscience. https://doi.org/10.1162/jocn.2010.21593
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.