Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Iconic gestures are spontaneous hand movements that illustrate certain contents of speech and, as such, are an important part of face-to-face communication. This experiment targets the brain bases of how iconic gestures and speech are integrated during comprehension. Areas of integration were identified on the basis of two classic properties of multimodal integration, bimodal enhancement and inverse effectiveness (i.e., greater enhancement for unimodally least effective stimuli). Participants underwent fMRI while being presented with videos of gesture-supported sentences as well as their unimodal components, which allowed us to identify areas showing bimodal enhancement. Additionally, we manipulated the signal-to-noise ratio of speech (either moderate or good) to probe for integration areas exhibiting the inverse effectiveness property. Bimodal enhancement was found at the posterior end of the superior temporal sulcus and adjacent superior temporal gyrus (pSTS/STG) in both hemispheres, indicating that the integration of iconic gestures and speech takes place in these areas. Furthermore, we found that the left pSTS/STG specifically showed a pattern of inverse effectiveness, i.e., the neural enhancement for bimodal stimulation was greater under adverse listening conditions. This indicates that activity in this area is boosted when an iconic gesture accompanies an utterance that is otherwise difficult to comprehend. The neural response paralleled the behavioral data observed. The present data extends results from previous gesture-speech integration studies in showing that pSTS/STG plays a key role in the facilitation of speech comprehension through simultaneous gestural input.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Spatial registration and normalization of images
Friston, K. J., Ashburner, J., Frith, C. D., Poline, J. ‐., Heather, J. D., & Frackowiak, R. S. J. (1995) · Human Brain Mapping · doi:10.1002/hbm.460030303 · Cited by 4,028
Visual Contribution to Speech Intelligibility in Noise
Sumby, W. H., & Pollack, I. (1954) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907309 · Cited by 2,583 · cited 1×
“Notably, Willems et al. (2009) insights into differential roles of left IFG and pSTS/MTG were largely based on the use of semantically incongruent combinations of speech and hand movements.”
McNeill, D. (2005) · doi:10.7208/chicago/9780226514642.001.0001 · Cited by 2,157 · cited 1×
“Addressees not only process the speech of a speaker, but also continuously integrate gestures with speech during comprehension (Goldin-Meadow, 2006; McNeill, 2005).”
Is neocortex essentially multisensory?
GHAZANFAR, A., & SCHROEDER, C. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.04.008 · Cited by 1,463 · cited 1×
“In summary, the existing studies are difficult to reconcile with the notion that the brain contains a single area for integration of information from multiple domains, as has been suggested previously (Hagoort, 2005, 2007; but see Ghazanfar & Schroeder, 2006).”
On Broca, brain, and binding: A new framework
Hagoort, P. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.07.004 · Cited by 1,411 · cited 1×
“In summary, the existing studies are difficult to reconcile with the notion that the brain contains a single area for integration of information from multiple domains, as has been suggested previously (Hagoort, 2005, 2007; but see Ghazanfar & Schroeder, 2006).”
Calvert, G. A., Campbell, R., & Brammer, M. J. (2000) · Current Biology · doi:10.1016/s0960-9822(00)00513-3 · Cited by 954
Unraveling multisensory integration: patchy organization within human STS multisensory cortex
Beauchamp, M. S., Argall, B. D., Bodurka, J., Duyn, J. H., & Martin, A. (2004) · Nature Neuroscience · doi:10.1038/nn1333 · Cited by 513
Superior Temporal Sulcus—It's My Area: Or Is It?
Hein, G., & Knight, R. T. (2008) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2008.20148 · Cited by 486 · cited 1×
“We attribute this to the left-lateralization of language-related integration processes in the brain (Friederici et al., in press; Hein & Knight, 2008; Obleser et al., 2007).”
Multisensory integration: methodological approaches and emerging principles in the human brain.
Calvert, G. A., & Thesen, T. (2004) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2004.03.018 · Cited by 425
Obleser, J., Wise, R. J. S., Alex Dresner, M., & Scott, S. K. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.4663-06.2007 · Cited by 354 · cited 2×
“For instance, speech is more difficult to comprehend in noisy environments, that is, when the SNR of speech is only moderate (e.g., Miller, 1951; Obleser et al., 2007).”
Self-citation
Cited by
Price, C. J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.062 · Cited by 2,159
The anatomy of language: a review of 100 fMRI studies published in 2009
Price, C. J. (2010) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2010.05444.x · Cited by 1,387
Superior Temporal Sulcus—It's My Area: Or Is It?
Hein, G., & Knight, R. T. (2008) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2008.20148 · Cited by 486
Brain regions involved in human movement perception: A quantitative voxel‐based meta‐analysis
Grosbras, M., Beaton, S., & Eickhoff, S. B. (2012) · Human Brain Mapping · doi:10.1002/hbm.21222 · Cited by 284
Dricu, M., & Frühholz, S. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.10.020 · Cited by 102
Metadata
- Authors
- Holle, H., Obleser, J., Rueschemeyer, S.-A., & Gunter, T. C.
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2010-01
- Volume
- 49
- Issue
- 1
- Pages
- 875-884
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W2012441310
- Cited by
- 153
- References
- 10
- FWCI
- 10.04
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions. https://app.cochat.ai/papers/10.1016/j.neuroimage.2009.08.058
- The paper
- Holle, H., Obleser, J., Rueschemeyer, S.-A., & Gunter, T. C. (2010). Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions. NeuroImage. https://doi.org/10.1016/j.neuroimage.2009.08.058
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.