Eye’m talking to you: speakers’ gaze direction modulates co-speech gesture processing in the right MTG
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Recipients process information from speech and co-speech gestures, but it is currently unknown how this processing is influenced by the presence of other important social cues, especially gaze direction, a marker of communicative intent. Such cues may modulate neural activity in regions associated either with the processing of ostensive cues, such as eye gaze, or with the processing of semantic information, provided by speech and gesture. Participants were scanned (fMRI) while taking part in triadic communication involving two recipients and a speaker. The speaker uttered sentences that were and were not accompanied by complementary iconic gestures. Crucially, the speaker alternated her gaze direction, thus creating two recipient roles: addressed (direct gaze) vs unaddressed (averted gaze) recipient. The comprehension of Speech&Gesture relative to SpeechOnly utterances recruited middle occipital, middle temporal and inferior frontal gyri, bilaterally. The calcarine sulcus and posterior cingulate cortex were sensitive to differences between direct and averted gaze. Most importantly, Speech&Gesture utterances, but not SpeechOnly utterances, produced additional activity in the right middle temporal gyrus when participants were addressed. Marking communicative intent with gaze direction modulates the processing of speech-gesture utterances in cerebral areas typically associated with the semantic processing of multi-modal communicative acts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Other minds in the brain: a functional imaging study of “theory of mind” in story comprehension.
Fletcher, P. (1995) · Cognition · doi:10.1016/0010-0277(95)00692-r · Cited by 1,522 · cited 2×
“, 2012), but one potential interpretation is that its activation was associated with perspective taking (Ruby and Decety, 2001), or mind-reading (Fletcher et al., 1995; Brunet et al., 2000), as participants may have engaged more in perspective taking when the other recipient was addressed rather than themselves.”
BackgroundEye contact detection in humans from birth
Farroni, T., Csibra, G., Simion, F., & Johnson, M. H. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.152159999 · Cited by 1,442 · cited 3×
“, 2010) and one of the first ones humans attend to (Farroni et al., 2002).”
BackgroundOn Broca, brain, and binding: A new framework
Hagoort, P. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.07.004 · Cited by 1,411 · cited 4×
“…with the sound it makes or the word ‘to write’ with a pantomimic depiction of writing), LIFG seems to be the predominant neural region for unification (i.e. the integration of lexical items, or words and gestures, into coherent sentential representations) (Hagoort, 2005; Willems et al., 2009).”
BackgroundInfluentialEffect of subjective perspective taking during simulation of action: a PET investigation of agency
Ruby, P., & Decety, J. (2001) · Nature Neuroscience · doi:10.1038/87510 · Cited by 1,182
The eye contact effect: mechanisms and development
Senju, A., & Johnson, M. H. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.11.009 · Cited by 797
Echoes of the brain within the posterior cingulate cortex
Leech, R., Braga, R., & Sharp, D. J. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.3689-11.2012 · Cited by 595 · cited 2×
“The role of this region is rather unclear (Leech et al., 2012), but one potential interpretation is that its activation was associated with perspective taking (Ruby and Decety, 2001), or mind-reading (Fletcher et al.”
BackgroundWhen Strangers Pass: Processing of Mutual and Averted Social Gaze in the Superior Temporal Sulcus
Pelphrey, K. A., Viola, R. J., & McCarthy, G. (2004) · Psychological Science · doi:10.1111/j.0956-7976.2004.00726.x · Cited by 464 · cited 1×
“For Permissions, please email: journals.permissions@oup.com Downloaded from https://academic.oup.com/scan/article-abstract/10/2/255/1655029 by guest on 29 July 2018 and the initiation of interaction (e.g. Pelphrey et al., 2004; Schilbach et al., 2006).”
MethodKampe, K. K. W., Frith, C. D., & Frith, U. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-12-05258.2003 · Cited by 406
Brain activation evoked by perception of gaze shifts: the influence of context
Pelphrey, K. A., Singerman, J. D., Allison, T., & McCarthy, G. (2003) · Neuropsychologia · doi:10.1016/s0028-3932(02)00146-x · Cited by 341
Mapping the information flow from one brain to another during gestural communication
Schippers, M. B., Roebroeck, A., Renken, R., Nanetti, L., & Keysers, C. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1001791107 · Cited by 323
How iconic gestures enhance communication: An ERP study
Wu, Y. C., & Coulson, S. (2007) · Brain and Language · doi:10.1016/j.bandl.2006.12.003 · Cited by 150
Cited by
Multimodal Language Processing in Human Communication
Holler, J., & Levinson, S. C. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.05.006 · Cited by 441
Self-citationSpeech-accompanying gestures are not processed by the language-processing mechanisms
Jouravlev, O., Zheng, D., Balewski, Z., Le Arnz Pongos, A., Levan, Z., Goldin-Meadow, S., & Fedorenko, E. (2019) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2019.107132 · Cited by 39
“language processing (e.g., Dick et al., 2009; Holle et al., 2008; Holler et al., 2015; Hubbard et al., 2009; Weisberg et al., 2016) and interpreted those responses as evidence of shared processing mechanisms.”
Exploring the boundaries of gesture-speech integration during language comprehension
Kelly, S. D. (2017) · Gesture Studies · doi:10.1075/gs.7.12kel · Cited by 30
Self-citationBluet, A., Reynaud, E., Federico, G., Bryche, C., Lesourd, M., Fournel, A., Lamberton, F., Ibarrola, D., Rossetti, Y., & Osiurak, F. (2025) · iScience · doi:10.1016/j.isci.2025.111870 · Cited by 11
Metadata
- Authors
- Holler, J., Kokal, I., Toni, I., Hagoort, P., Kelly, S. D., & Özyürek, A.
- Venue
- Social Cognitive and Affective Neuroscience
- Publisher
- Oxford University Press (OUP)
- Published
- 2014-04-16
- Volume
- 10
- Issue
- 2
- Pages
- 255-261
- ISSN
- 1749-5016 (ISSN-L)
- 1749-5024
- OpenAlex
- W2165271834
- Cited by
- 37
- References
- 11
- FWCI
- 2.38
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.