Co‐speech gestures influence neural activity in brain regions associated with processing semantic information
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Everyday communication is accompanied by visual information from several sources, including co‐speech gestures, which provide semantic information listeners use to help disambiguate the speaker's message. Using fMRI, we examined how gestures influence neural activity in brain regions associated with processing semantic information. The BOLD response was recorded while participants listened to stories under three audiovisual conditions and one auditory‐only (speech alone) condition. In the first audiovisual condition, the storyteller produced gestures that naturally accompany speech. In the second, the storyteller made semantically unrelated hand movements. In the third, the storyteller kept her hands still. In addition to inferior parietal and posterior superior and middle temporal regions, bilateral posterior superior temporal sulcus and left anterior inferior frontal gyrus responded more strongly to speech when it was further accompanied by gesture, regardless of the semantic relation to speech. However, the right inferior frontal gyrus was sensitive to the semantic import of the hand movements, demonstrating more activity when hand movements were semantically unrelated to the accompanying speech. These findings show that perceiving hand movements during speech modulates the distributed pattern of neural activation involved in both biological motion perception and discourse comprehension, suggesting listeners attempt to find meaning, not only in the words speakers produce, but also in the hand movements that accompany speech. Hum Brain Mapp, 2009. © 2009 Wiley‐Liss, Inc.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Desikan, R. S., Ségonne, F., Fischl, B., Quinn, B. T., Dickerson, B. C., Blacker, D., Buckner, R. L., Dale, A. M., Maguire, R. P., Hyman, B. T., et al. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.021 · Cited by 13,324 · cited 3×
“3 and 4) were defined on each individual surface representation using an automated parcellation procedure [Desikan et al., 2006; Fischl et al., 2004].”
Cortical Surface-Based Analysis
Dale, A. M., Fischl, B., & Sereno, M. I. (1999) · NeuroImage · doi:10.1006/nimg.1998.0395 · Cited by 10,910
Fischl, B., Salat, D. H., Busa, E., Albert, M., Dieterich, M., Haselgrove, C., van der Kouwe, A., Killiany, R., Kennedy, D., Klaveness, S., et al. (2002) · Neuron · doi:10.1016/s0896-6273(02)00569-x · Cited by 8,610
Rizzolatti, G., & Craighero, L. (2004) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144230 · Cited by 7,433
Nonparametric permutation tests for functional neuroimaging: A primer with examples
Nichols, T. E., & Holmes, A. P. (2002) · Human Brain Mapping · doi:10.1002/hbm.1058 · Cited by 6,256
Cortical Surface-Based Analysis
Fischl, B., Sereno, M. I., & Dale, A. M. (1999) · NeuroImage · doi:10.1006/nimg.1998.0396 · Cited by 6,147 · cited 4×
“Participants gave written informed consent and the Institutional Review Board of the Biological Sciences Division of The University of Chicago approved the study.”
InfluentialThe Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding
EKMAN, P., & FRIESEN, W. V. (1969) · Semiotica · doi:10.1515/semi.1969.1.1.49 · Cited by 3,316 · cited 2×
“The remaining gestures were categorized as beat or deictic gestures, and none were categorized as codified emblems [Ekman and Friesen, 1969].”
Neurophysiological mechanisms underlying the understanding and imitation of action
Rizzolatti, G., Fogassi, L., & Gallese, V. (2001) · Nature Reviews Neuroscience · doi:10.1038/35090060 · Cited by 3,302
Cattaneo, L., & Rizzolatti, G. (2009) · Archives of Neurology · doi:10.1001/archneurol.2009.41 · Cited by 3,137 · cited 1×
“A number of studies have identified brain regions, including ventral premotor cortex, IFG, and IPL, whose functional properties resemble those of macaque ‘‘mirror neurons,’’ that fire during both execution of one’s own actions and the observation of the same actions of others [for reviews see Iacoboni, 2005; Rizzolatt…”
McNeill, D. (2005) · doi:10.7208/chicago/9780226514642.001.0001 · Cited by 2,157 · cited 7×
“Listeners not only process the words that speakers produce, but also continuously integrate gestures with speech and with other visual information (e.g., the speaker’s lips, mouth, and eyes) to arrive at the speaker’s meaning [GoldinMeadow, 2006; Kendon, 1994; McNeill, 2005].”
Influential
Cited by
Price, C. J. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.04.062 · Cited by 2,159
An integrated theory of language production and comprehension
Pickering, M. J., & Garrod, S. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12001495 · Cited by 1,569
The anatomy of language: a review of 100 fMRI studies published in 2009
Price, C. J. (2010) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2010.05444.x · Cited by 1,387
The Neural Bases of Social Cognition and Story Comprehension
Mar, R. A. (2011) · Annual Review of Psychology · doi:10.1146/annurev-psych-120709-145406 · Cited by 950
The Simulation of Smiles (SIMS) model: Embodied simulation and the meaning of facial expression
Niedenthal, P. M., Mermillod, M., Maringer, M., & Hess, U. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x10000865 · Cited by 618
The impact of iconic gestures on foreign language word learning and its neural substrate
Macedonia, M., Müller, K., & Friederici, A. D. (2011) · Human Brain Mapping · doi:10.1002/hbm.21084 · Cited by 277
Whitney, C., Kirk, M., O'Sullivan, J., Lambon Ralph, M. A., & Jefferies, E. (2012) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00123 · Cited by 261
N400 ERPs for actions: building meaning in context
Amoruso, L., Gelormini, C., Aboitiz, F., Alvarez González, M., Manes, F., Cardona, J. F., & Ibanez, A. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00057 · Cited by 125
“Consistent with these findings, recent work on language and gesture processing (Willems et al., 2007; Holle et al., 2008; Dick et al., 2009; Hubbard et al., 2009; Kircher et al., 2009) also points to the engagement of temporal areas (STS, STG, MTG), inferior parietal (AG), IFG, and premotor regions…”
ResultGesture in the developing brain
Dick, A. S., Goldin‐Meadow, S., Solodkin, A., & Small, S. L. (2012) · Developmental Science · doi:10.1111/j.1467-7687.2011.01100.x · Cited by 60
“In previous work, we found that the STSp responds more strongly to speech with hand movements than to audiovisual speech without hand movements (Dick et al., 2009), and that there are no age-related differences in left STSp for processing audiovisual speech without hand movements (Dick et al.”
BackgroundInfluentialSelf-citationSpeech-accompanying gestures are not processed by the language-processing mechanisms
Jouravlev, O., Zheng, D., Balewski, Z., Le Arnz Pongos, A., Levan, Z., Goldin-Meadow, S., & Fedorenko, E. (2019) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2019.107132 · Cited by 39
“), also known as self-adaptor movements (Ekman and Friesen, 1969; Dick et al., 2009).”
Interpretation-mediated changes in neural activity during language comprehension
Cooper, E. A., Hasson, U., & Small, S. L. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.01.003 · Cited by 31
Self-citation
Metadata
- Authors
- Dick, A. S., Goldin‐Meadow, S., Hasson, U., Skipper, J. I., & Small, S. L.
- Venue
- Human Brain Mapping
- Publisher
- Wiley
- Published
- 2009-04-21
- Volume
- 30
- Issue
- 11
- Pages
- 3509-3526
- ISSN
- 1065-9471 (ISSN-L)
- 1097-0193
- OpenAlex
- W2105902013
- Cited by
- 153
- References
- 29
- FWCI
- 16.01
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.