The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
On-line Integration of Semantic Information from Speech and Gesture: Insights from Event-related Brain Potentials
Open accessAbstract
During language comprehension, listeners use the global semantic representation from previous sentence or discourse context to immediately integrate the meaning of each upcoming word into the unfolding message-level representation. Here we investigate whether communicative gestures that often spontaneously co-occur with speech are processed in a similar fashion and integrated to previous sentence context in the same way as lexical meaning. Event-related potentials were measured while subjects listened to spoken sentences with a critical verb (e.g., knock), which was accompanied by an iconic co-speech gesture (i.e., KNOCK). Verbal and/or gestural semantic content matched or mismatched the content of the preceding part of the sentence. Despite the difference in the modality and in the specificity of meaning conveyed by spoken words and gestures, the latency, amplitude, and topographical distribution of both word and gesture mismatches are found to be similar, indicating that the brain integrates both types of information simultaneously. This provides evidence for the claim that neural processing in language comprehension involves the simultaneous incorporation of information coming from a broader domain of cognition than only verbal semantics. The neural evidence for similar integration of information from speech and gesture emphasizes the tight interconnection between speech and co-speech gestures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183
Reading Senseless Sentences: Brain Potentials Reflect Semantic Incongruity
Kutas, M., & Hillyard, S. A. (1980) · Science · doi:10.1126/science.7350657 · Cited by 4,888
Gesture Visible Action as Utterance
Kendon, A. (2004) · Cambridge University Press eBooks · doi:10.1017/cbo9780511807572 · Cited by 3,891
Integration of Visual and Linguistic Information in Spoken Language Comprehension
Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995) · Science · doi:10.1126/science.7777863 · Cited by 2,929
Brain potentials during reading reflect word expectancy and semantic association
Kutas, M., & Hillyard, S. A. (1984) · Nature · doi:10.1038/307161a0 · Cited by 2,218
On Broca, brain, and binding: A new framework
Hagoort, P. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.07.004 · Cited by 1,411 · cited 4×
“This area has been claimed to be crucial for the integration of semantic information into previous context (Hagoort, 2003b, 2005; Hagoort et al., 2004).”
BackgroundInfluentialIntegration of word meaning and world knowledge in language comprehension
Hagoort, P., Hald, L., Bastiaansen, M., & Petersson, K. M. (2004) · Science · doi:10.1126/science.1095455 · Cited by 1,396
Hearing gesture: how our hands help us think
(2004) · Choice Reviews Online · doi:10.5860/choice.41-4463 · Cited by 1,140
Kita, S., & Özyürek, A. (2003) · Journal of Memory and Language · doi:10.1016/s0749-596x(02)00505-3 · Cited by 955
Self-citation(2000) · Cambridge University Press eBooks · doi:10.1017/cbo9780511620850 · Cited by 949
McNeill, D. (2011) · Advances in Visual Semiotics · doi:10.1515/9783110874259.351 · Cited by 927
Semantic Integration in Sentences and Discourse: Evidence from the N400
Berkum, J. J. A. V., Hagoort, P., & Brown, C. M. (1999) · Journal of Cognitive Neuroscience · doi:10.1162/089892999563724 · Cited by 642
Self-citationHagoort, P. (2003) · Journal of Cognitive Neuroscience · doi:10.1162/089892903322370807 · Cited by 437
Self-citationGanis, G., Kutas, M., & Sereno, M. I. (1996) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1996.8.2.89 · Cited by 367
Seeing speech: visual information from lip movements modifies activity in the human auditory cortex
Sams, M., Aulanko, R., Hämäläinen, M., Hari, R., Lounasmaa, O. V., Lu, S.-T., & Simola, J. (1991) · Neuroscience Letters · doi:10.1016/0304-3940(91)90914-f · Cited by 367
When and how do listeners relate a sentence to the wider discourse? Evidence from the N400 effect
van Berkum, J. J. A., Zwitserlood, P., Hagoort, P., & Brown, C. M. (2003) · Cognitive Brain Research · doi:10.1016/s0926-6410(03)00196-4 · Cited by 367
Self-citationHagoort, P. (2003) · NeuroImage · doi:10.1016/j.neuroimage.2003.09.013 · Cited by 352
Self-citationAn electrophysiological investigation of semantic priming with pictures of real objects
McPherson, W. B., & Holcomb, P. J. (1999) · Psychophysiology · doi:10.1017/s0048577299971196 · Cited by 345
When Language Meets Action: The Neural Integration of Gesture and Speech
Willems, R. M., Özyürek, A., & Hagoort, P. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhl141 · Cited by 342
Self-citationChildren Learn When Their Teacher's Gestures and Speech Differ
Singer, M. A., & Goldin-Meadow, S. (2005) · Psychological Science · doi:10.1111/j.0956-7976.2005.00786.x · Cited by 333
Event-related potentials and the semantic matching of pictures
Barrett, S. E., & Rugg, M. D. (1990) · Brain and Cognition · doi:10.1016/0278-2626(90)90029-n · Cited by 317
Do conversational hand gestures communicate?
Krauss, R. M., Morrel-Samuels, P., & Colasante, C. (1991) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.61.5.743 · Cited by 265
An electrophysiological study of scene effects on object identification
Ganis, G., & Kutas, M. (2003) · Cognitive Brain Research · doi:10.1016/s0926-6410(02)00244-6 · Cited by 264
Speech and gesture share the same communication system
Bernardis, P., & Gentilucci, M. (2006) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2005.05.007 · Cited by 263
Event-related potentials during discourse-level semantic integration of complex pictures
West, W. C., & Holcomb, P. J. (2002) · Cognitive Brain Research · doi:10.1016/s0926-6410(01)00129-x · Cited by 263
Word familiarity predicts temporal asynchrony of hand gestures and speech.
Morrel-Samuels, P., & Krauss, R. M. (1992) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037//0278-7393.18.3.615 · Cited by 250
BEATTIE, G., & SHOVELTON, H. (1999) · semi · doi:10.1515/semi.1999.123.1-2.1 · Cited by 239
Semantic integration in videos of real–world events: An electrophysiological investigation
Sitnikova, T., Kuperberg, G., & Holcomb, P. J. (2003) · Psychophysiology · doi:10.1111/1469-8986.00016 · Cited by 235
Event-Related Brain Potentials Reflect Semantic Priming in an Object Decision Task
Holcomb, P. J., & Mcpherson, W. B. (1994) · Brain and Cognition · doi:10.1006/brcg.1994.1014 · Cited by 235
Neural correlates of bimodal speech and gesture comprehension
Kelly, S. D., Kravitz, C., & Hopkins, M. (2004) · Brain and Language · doi:10.1016/s0093-934x(03)00335-3 · Cited by 234
The Role of Iconic Gestures in Speech Disambiguation: ERP Evidence
Holle, H., & Gunter, T. C. (2007) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2007.19.7.1175 · Cited by 232
Gesture and Silence as Indicators of Planning in Speech
Butterworth, B., & Beattie, G. (1978) · Recent Advances in the Psychology of Language · doi:10.1007/978-1-4684-2532-1_19 · Cited by 222
N400 to Semantically Anomalous Pictures and Words
Nigam, A., Hoffman, J. E., & Simons, R. F. (1992) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1992.4.1.15 · Cited by 222
Gesture and the communicative intention of the speaker
Melinger, A., & Levelt, W. J. M. (2005) · Gesture · doi:10.1075/gest.4.2.02mel · Cited by 214
A Cross-Cultural Study of the Communication of Extra-Verbal Meaning by Gesture
Graham, J. A., & Argyle, M. (1975) · International Journal of Psychology · doi:10.1080/00207597508247319 · Cited by 210
Federmeier, K. D., & Kutas, M. (2001) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.27.1.202 · Cited by 205
Neural correlates of the processing of co-speech gestures
Holle, H., Gunter, T. C., Rüschemeyer, S.-A., Hennenlotter, A., & Iacoboni, M. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2007.10.055 · Cited by 205
Meaningful gestures: Electrophysiological indices of iconic gesture comprehension
Wu, Y. C., & Coulson, S. (2005) · Psychophysiology · doi:10.1111/j.1469-8986.2005.00356.x · Cited by 187 · cited 2×
“The present study, together with the studies by Wu and Coulson (2005) and Kelly et al. (2004), point to the fact that iconic gestures trigger semantic processing, as is indicated by the presence of N400 effects.”
BackgroundResultKelly, S. D., Manning, S. M., & Rodak, S. (2008) · Language and Linguistics Compass · doi:10.1111/j.1749-818x.2008.00067.x · Cited by 186
Communicative Effects of Speech-Mismatched Gestures
McNeill, D., Cassell, J., & McCullough, K.-E. (1994) · Research on Language & Social Interaction · doi:10.1207/s15327973rlsi2703_4 · Cited by 183
Özyürek, A. (2002) · Journal of Memory and Language · doi:10.1006/jmla.2001.2826 · Cited by 178
Self-citationColin, C. (2002) · Clinical Neurophysiology · doi:10.1016/s1388-2457(02)00024-x · Cited by 173
Processing of changes in visual speech in the human auditory cortex
Möttönen, R., Krause, C. M., Tiippana, K. M., & Sams, M. E. (2002) · Cognitive Brain Research · doi:10.1016/s0926-6410(02)00053-8 · Cited by 171
Brain responses to lexical ambiguity resolution and parsing.
Peter Hagoort, A., & Brown, C. M. (1994) · Max Planck Digital Library · Cited by 160
Giving speech a hand: Gesture modulates activity in auditory cortex during speech perception
Hubbard, A. L., Wilson, S. M., Callan, D. E., & Dapretto, M. (2008) · Human Brain Mapping · doi:10.1002/hbm.20565 · Cited by 157
Kelly, S. D., & Church, R. B. (1998) · Child Development · doi:10.1111/j.1467-8624.1998.tb06135.x · Cited by 141
Event-related brain potentials and human language
Osterhout, L., McLaughlin, J. A., & Bersick, M. (1997) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(97)01073-5 · Cited by 135
Gestures convey substantive information about a child’s thoughts to ordinary listeners
Susan Goldin-meadow, A., & Sandhofer, C. M. (1999) · Developmental Science · doi:10.1111/1467-7687.00056 · Cited by 135
Willems, R. M., Özyürek, A., & Hagoort, P. (2008) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2008.20085 · Cited by 102
Self-citationStraube, B., Green, A., Weis, S., Chatterjee, A. K., & Kircher, T. T. J. (2008) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2009.21053 · Cited by 93
The Meaning of Gestures - What Can Be Understood Without Speech
Feyereisen, P., Vandewiele, M., & Dubois, F. (1988) · DIAL (Catholic University of Leuven) · Cited by 77
Is the Sky 2? Contextual Priming in Grapheme-Color Synaesthesia
Brang, D., Edwards, L. A., Ramachandran, V. S., & Seana Coulson, A. (2008) · Psychological Science · doi:10.1111/j.1467-9280.2008.02103.x · Cited by 67
McNeill (2008) · Cited by 64
Bernardis, P., Salillas, E., & Caramelli, N. (2008) · Cognitive Neuropsychology · doi:10.1080/02643290801921707 · Cited by 62
Willems, R. M., Oostenveld, R., & Peter Hagoort, A. (2008) · Brain Research · doi:10.1016/j.brainres.2008.04.065 · Cited by 46
Self-citation
Cited by
Kutas, M., & Federmeier, K. D. (2011) · Annual Review of Psychology · doi:10.1146/annurev.psych.093008.131123 · Cited by 4,377
Two Sides of the Same Coin: Speech and Gesture Mutually Interact to Enhance Comprehension
Kelly, S. D., Özyürek, A., & Maris, E. (2010) · Psychological Science · doi:10.1177/0956797609357327 · Cited by 669
Self-citationGesture and speech in interaction: An overview
Wagner, P., Malisz, Z., & Kopp, S. (2014) · Speech Communication · doi:10.1016/j.specom.2013.09.008 · Cited by 456
Neural evidence for the interplay between language, gesture, and action: A review
Willems, R. M., & Hagoort, P. (2007) · Brain and Language · doi:10.1016/j.bandl.2007.03.004 · Cited by 393
Self-citationThe bridge of iconicity: from a world of experience to the experience of language
Perniss, P., & Vigliocco, G. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0300 · Cited by 345
Metadata
- Authors
- Özyürek, A., Willems, R. M., Kita, S., & Hagoort, P.
- Venue
- Journal of Cognitive Neuroscience
- Publisher
- MIT Press - Journals
- Published
- 2007-04-01
- Volume
- 19
- Issue
- 4
- Pages
- 605-616
- ISSN
- 0898-929X (ISSN-L)
- 1530-8898
- OpenAlex
- W2100271981
- Cited by
- 303
- References
- 55
- FWCI
- 8.35
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.