CoChat
Sign Up
Cited by 70Neurobiology of Language and BilingualismAction Observation and SynchronizationMultisensory perception and integration

Gesture Facilitates the Syntactic Analysis of Speech

Open access
Holle, Henning, Obermeier, Christian, Schmidt-Kassow, Maren, Friederici, Angela D., Ward, Jamie, Gunter, Thomas C.·2012·Frontiers in Psychology
DOI 10.3389/fpsyg.2012.00074PMC PMC3307377
Open this paper in CoChat — free accountOpen PDF

Abstract

Recent research suggests that the brain routinely binds together information from gesture and speech. However, most of this research focused on the integration of representational gestures with the semantic content of speech. Much less is known about how other aspects of gesture, such as emphasis, influence the interpretation of the syntactic relations in a spoken message. Here, we investigated whether beat gestures alter which syntactic structure is assigned to ambiguous spoken German sentences. The P600 component of the Event Related Brain Potential indicated that the more complex syntactic structure is easier to process when the speaker emphasizes the subject of a sentence with a beat. Thus, a simple flick of the hand can change our interpretation of who has been doing what to whom in a spoken sentence. We conclude that gestures and speech are integrated systems. Unlike previous studies, which have shown that the brain effortlessly integrates semantic information from gesture and speech, our study is the first to demonstrate that this integration also occurs for syntactic information. Moreover, the effect appears to be gesture-specific and was not found for other stimuli that draw attention to certain parts of speech, including prosodic emphasis, or a moving visual stimulus with the same trajectory as the gesture. This suggests that only visual emphasis produced with a communicative intention in mind (that is, beat gestures) influences language comprehension, but not a simple visual movement lacking such an intention.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fpsyg.2012.00074

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 36

  1. Alibali M. W. Heath D. C. Myers H. J. (2001). Effects of visibility between speaker and listener on gesture production: some gestures are meant to be seen. J. Mem. Lang. 44, 169– 188 10.1006/jmla.2000.2752 · DOI
  2. Bader M. Meng M. (1999). Subject-object ambiguities in German embedded clauses: an across-the-board comparison. J. Psycholinguist. Res. 28, 121– 143 10.1023/A:1023206208142 · DOI
  3. Bavelas J. Gerwing J. Sutton C. Prevost D. (2008). Gesturing on the telephone: independent effects of dialogue and visibility. J. Mem. Lang. 58, 495– 520 10.1016/j.jml.2007.02.004 · DOI
  4. Crocker M. W. Knoeferle P. Mayberry M. R. (2010). Situated sentence processing: the coordinated interplay account and a neurobehavioral model. Brain Lang. 112, 189– 201 10.1016/j.bandl.2009.03.004 19450874 · DOI
  5. Friederici A. D. (2002). Towards a neural basis of auditory sentence processing. Trends Cogn. Sci. (Regul. Ed.) 6, 78– 84 10.1016/S1364-6613(00)01839-8 15866191 · DOI · Paper page
  6. Friederici A. D. Pfeifer E. Hahne A. (1993). Event-related brain potentials during natural speech processing: effects of semantic, morphological and syntactic violations. Brain Res. Cogn. Brain Res. 1, 183– 192 10.1016/0926-6410(93)90026-2 8257874 · DOI · Paper page
  7. Gentilucci M. Bernardis P. Crisi G. Dalla Volta R. (2006). Repetitive transcranial magnetic stimulation of Broca’s area affects verbal responses to gesture observation. J. Cogn. Neurosci. 18, 1059– 1074 10.1162/jocn.2006.18.7.1059 16839281 · DOI
  8. Grice H. P. (1975). “Logic and conversation,” in Speech Acts: Syntax and Semantics, eds Cole P. Morgan J. L. (New York: Academic Press), 41– 58
  9. Hagoort P. Brown C. Groothusen J. (1993). The syntactic positive shift (SPS) as an ERP measure of syntactic processing. Lang. Cogn. Process. 8, 439– 483 10.1080/01690969308407585 · DOI
  10. Haupt F. S. Schlesewsky M. Roehm D. Friederici A. D. Bornkessel-Schlesewsky I. (2008). The status of subject-object reanalyses in the language comprehension architecture. J. Mem. Lang. 59, 54– 96 10.1016/j.jml.2008.02.003 · DOI
  11. Holle H. Gunter T. C. (2007). The role of iconic gestures in speech disambiguation: ERP evidence. J. Cogn. Neurosci. 19, 1175– 1192 10.1162/jocn.2007.19.7.1175 17583993 · DOI
  12. Holle H. Gunter T. C. Ruschemeyer S. A. Hennenlotter A. Iacoboni M. (2008). Neural correlates of the processing of co-speech gestures. Neuroimage 39, 2010– 2024 10.1016/j.neuroimage.2007.10.055 18093845 · DOI
  13. Holle H. Obleser J. Rueschemeyer S. A. Gunter T. C. (2010). Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions. Neuroimage 49, 875– 884 10.1016/j.neuroimage.2009.08.058 19733670 · DOI
  14. Hubbard A. L. Wilson S. M. Callan D. E. Dapretto M. (2009). Giving speech a hand: gesture modulates activity in auditory cortex during speech perception. Hum. Brain Mapp. 30, 1028– 1037 10.1002/hbm.20565 18412134 PMC2644740 · DOI
  15. Kaiser E. Trueswell J. C. (2004). The role of discourse context in the processing of a flexible word-order language. Cognition 94, 113– 147 10.1016/j.cognition.2004.01.002 15582623 · DOI
  16. Kelly S. D. Creigh P. Bartolotti J. (2010a). Integrating speech and iconic gestures in a Stroop-like task: evidence for automatic processing. J. Cogn. Neurosci. 22, 683– 694 10.1162/jocn.2009.21254 19413483 · DOI
  17. Kelly S. D. Özyürek A. Maris E. (2010b). Two sides of the same coin: speech and gesture mutually interact to enhance comprehension. Psychol. Sci. 21, 260– 267 10.1177/0956797609357327 20424055 · DOI
  18. Kempen G. Harbusch K. (2003). An artificial opposition between grammaticality and frequency: comment on Bornkessel, Schlesewsky, and Friederici. (2002). Cognition 90, 205– 210; discussion 211–203, 215. 14599755
  19. Kita S. Özyürek A. (2003). What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. J. Mem. Lang. 48, 16– 32 10.1016/S0749-596X(02)00505-3 · DOI
  20. Knoeferle P. Habets B. Crocker M. W. Munte T. F. (2008). Visual scenes trigger immediate syntactic reanalysis: evidence from ERPs during situated spoken comprehension. Cereb. Cortex 18, 789– 795 10.1093/cercor/bhm121 17644830 · DOI
  21. Krahmer E. Swerts M. (2007). The effects of visual beats on prosodic prominence: acoustic analyses, auditory perception and visual perception. J. Mem. Lang. 57, 396– 414 10.1016/j.jml.2007.06.005 · DOI
  22. Krauss R. M. (1998). Why do we gesture when we speak. Curr. Dir. Psychol. Sci. 7, 54– 60 10.1111/1467-8721.ep13175642 · DOI
  23. Krauss R. M. Dushay R. A. Chen Y. S. Rauscher F. (1995). The communicative value of conversational hand gestures. J. Exp. Soc. Psychol. 31, 533– 552 10.1006/jesp.1995.1024 · DOI
  24. Krauss R. M. Morrel-Samuels P. Colasante C. (1991). Do conversational hand gestures communicate? J. Pers. Soc. Psychol. 61, 743– 754 10.1037/0022-3514.61.5.743 1753329 · DOI
  25. Levelt W. J. M. Richardson G. la Heij W. (1985). Pointing and voicing in deictic expressions. J. Mem. Lang. 24, 133– 164 10.1016/0749-596X(85)90021-X · DOI
  26. McNeill D. (1992). Hand and Mind - What Gestures Reveal about Thought. Chicago: The University of Chicago Press
  27. Obermeier C. Holle H. Gunter T. C. (2011). What iconic gesture fragments reveal about gesture-speech integration: when synchrony is lost, memory can help. J. Cogn. Neurosci. 23, 1648– 1663 10.1162/jocn.2010.21498 20350188 · DOI
  28. Oldfield R. C. (1971). The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9, 97– 113 10.1016/0028-3932(71)90067-4 5146491 · DOI · Paper page
  29. Osterhout L. Holcomb P. J. (1992). Event-related brain potentials elicited by syntactic anomaly. J. Mem. Lang. 31, 785– 806 10.1016/0749-596X(92)90039-Z · DOI · Paper page
  30. Özyürek A. Willems R. M. Kita S. Hagoort P. (2007). On-line integration of semantic information from speech and gesture: insights from event-related brain potentials. J. Cogn. Neurosci. 19, 605– 616 10.1162/jocn.2007.19.4.605 17381252 · DOI
  31. Schriefers H. Friederici A. D. Kuhn K. (1995). The processing of locally ambiguous relative clauses in German. J. Mem. Lang. 34, 499– 520 10.1006/jmla.1995.1023 · DOI
  32. van de Meerendonk N. Kolk H. H. Vissers C. T. Chwilla D. J. (2010). Monitoring in language perception: mild and strong conflicts elicit different ERP patterns. J. Cogn. Neurosci. 22, 67– 82 10.1162/jocn.2008.21170 19199401 · DOI
  33. Willems R. M. Özyürek A. Hagoort P. (2007). When language meets action: the neural integration of gesture and speech. Cereb. Cortex 17, 2322– 2333 10.1093/cercor/bhl141 17159232 · DOI
  34. Willems R. M. Özyürek A. Hagoort P. (2009). Differential roles for left inferior frontal and superior temporal cortex in multimodal integration of action and language. Neuroimage 47, 1992– 2004 10.1016/S1053-8119(09)71943-3 19497376 · DOI
  35. Wu Y. C. Coulson S. (2010). Gestures modulate speech processing early in utterances. Neuroreport 21, 522– 526 10.1097/WNR.0b013e32833904bb 20375745 PMC2929818 · DOI
  36. Xu J. Gannon P. J. Emmorey K. Smith J. F. Braun A. R. (2009). Symbolic gestures and spoken language are processed by a common neural system. Proc. Natl. Acad. Sci. U.S.A. 106, 20664– 20669 10.1073/pnas.0900883106 19923436 PMC2779203 · DOI

Cited by

Sorted by citations

Authors
Holle, H., Obermeier, C., Schmidt-Kassow, M., Friederici, A. D., Ward, J., & Gunter, T. C.
Affiliations
  • Max Planck Society (DE)
  • University of Sussex (GB)
Corresponding author
Holle, Henning
ORCID
Venue
Frontiers in Psychology
Publisher
Frontiers Media SA
Published
2012
Volume
3
Pages
74-74
ISSN
1664-1078 (ISSN-L)
DOI
10.3389/fpsyg.2012.00074
PMC
PMC3307377
OpenAlex
W2122198827
Cited by
70
References
0
Citations by year
2026: 2 · 2025: 3 · 2024: 1 · 2023: 4 · 2022: 2 · 2021: 7 · 2020: 4 · 2019: 6 · 2018: 9 · 2017: 6 · 2016: 6 · 2015: 6 · 2014: 7 · 2013: 3 · 2012: 4
FWCI
3.28
Type
journal article
Language
en
Open access
Licence
cc-by-nc
Locations
  • Frontiers in Psychology (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • Repository@Hull (Worktribe) (University of Hull) (submittedVersion, cc-by-nc)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Publication Server of Goethe University Frankfurt am Main (Goethe University Frankfurt) (submittedVersion, cc-by-nc)
  • MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
Funders
  • Volkswagen Foundation
  • Max-Planck-Gesellschaft
  • Economic and Social Research Council (RES-062-23-, RES-062-23-1150, RES-062)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
  • Social Sciences › Psychology › Social Psychology › Action Observation and Synchronization
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
Keywords
gesture-speech integration, beat gestures, syntactic ambiguity, P600, event-related potentials, German syntax, language comprehension, syntactic analysis, communicative intention

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review