The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Visual speech speeds up the neural processing of auditory speech
Open accessAbstract
Synchronous presentation of stimuli to the auditory and visual systems can modify the formation of a percept in either modality. For example, perception of auditory speech is improved when the speaker's facial articulatory movements are visible. Neural convergence onto multisensory sites exhibiting supra-additivity has been proposed as the principal mechanism for integration. Recent findings, however, have suggested that putative sensory-specific cortices are responsive to inputs presented through a different modality. Consequently, when and where audiovisual representations emerge remain unsettled. In combined psychophysical and electroencephalography experiments we show that visual speech speeds up the cortical processing of auditory signals early (within 100 ms of signal onset). The auditory–visual interaction is reflected as an articulator-specific temporal facilitation (as well as a nonspecific amplitude reduction). The latency facilitation systematically depends on the degree to which the visual signal predicts possible auditory targets. The observed auditory–visual data support the view that there exist abstract internal representations that constrain the analysis of subsequent speech inputs. This is evidence for the existence of an “analysis-by-synthesis” mechanism in auditory–visual speech perception.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bootstrap methods: another look at the jackknife
Efron, B. (1979) · The Annals of Statistics · doi:10.1214/aos/1176344552 · Cited by 17,602
Neurophysiological investigation of the basis of the fMRI signal
Logothetis, N. K., Pauls, J., Augath, M., Trinath, T., & Oeltermann, A. (2001) · Nature · doi:10.1038/35084005 · Cited by 6,551
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483
Interplay between Cortical Spreading Depolarization and Seizures
Kramer, D. R., Fujii, T., Ohiorhenuan, I., & Liu, C. Y. (2017) · Stereotactic and Functional Neurosurgery · doi:10.1159/000452841 · Cited by 3,696
An internal model for sensorimotor integration
Wolpert, D. M., Ghahramani, Z., & Jordan, M. I. (1995) · Science · doi:10.1126/science.7569931 · Cited by 3,553
Näätänen, R., & Picton, T. (1987) · Psychophysiology · doi:10.1111/j.1469-8986.1987.tb00311.x · Cited by 3,321
The motor theory of speech perception revised
Liberman, A. M., & Mattingly, I. G. (1985) · Cognition · doi:10.1016/0010-0277(85)90021-6 · Cited by 2,861
Risto, N. (2018) · Routledge eBooks · doi:10.4324/9780429487354 · Cited by 2,024
Merging the senses into a robust percept
Ernst, M. O., & Bülthoff, H. H. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.02.002 · Cited by 1,906
(1994) · Choice Reviews Online · doi:10.5860/choice.31-4365 · Cited by 1,882
Cited by
A cortical network for semantics: (de)constructing the N400
Lau, E. F., Phillips, C., & Poeppel, D. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2532 · Cited by 1,700
Self-citationAn integrated theory of language production and comprehension
Pickering, M. J., & Garrod, S. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12001495 · Cited by 1,569
Is neocortex essentially multisensory?
GHAZANFAR, A., & SCHROEDER, C. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.04.008 · Cited by 1,463
Cortical oscillations and sensory predictions
Arnal, L. H., & Giraud, A.-L. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.05.003 · Cited by 1,114
Sensorimotor Integration in Speech Processing: Computational Basis and Neural Organization
Hickok, G., Houde, J., & Rong, F. (2011) · Neuron · doi:10.1016/j.neuron.2011.01.019 · Cited by 786
Kononowicz, T. W., van Rijn, H., & Meck, W. H. (2018) · Stevens' Handbook of Experimental Psychology and Cognitive Neuroscience · doi:10.1002/9781119170174.epcn114 · Cited by 756
Two Sides of the Same Coin: Speech and Gesture Mutually Interact to Enhance Comprehension
Kelly, S. D., Özyürek, A., & Maris, E. (2010) · Psychological Science · doi:10.1177/0956797609357327 · Cited by 669
Robust speech perception: Recognize the familiar, generalize to the similar, and adapt to the novel.
Kleinschmidt, D. F., & Jaeger, T. F. (2015) · Psychological Review · doi:10.1037/a0038695 · Cited by 635
The Natural Statistics of Audiovisual Speech
Chandrasekaran, C., Trubanova, A., Stillittano, S., Caplier, A., & Ghazanfar, A. A. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000436 · Cited by 630
Temporal window of integration in auditory-visual speech perception
van Wassenhove, V., Grant, K. W., & Poeppel, D. (2007) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2006.01.001 · Cited by 621
Self-citationMultisensory Integration of Dynamic Faces and Voices in Rhesus Monkey Auditory Cortex
Ghazanfar, A. A., Maier, J. X., Hoffman, K. L., & Logothetis, N. K. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.0799-05.2005 · Cited by 544
Visual Modulation of Neurons in Auditory Cortex
Kayser, C., Petkov, C. I., & Logothetis, N. K. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhm187 · Cited by 509
Neuronal oscillations and visual amplification of speech
Schroeder, C. E., Lakatos, P., Kajikawa, Y., Partan, S., & Puce, A. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.01.002 · Cited by 498
Speech perception at the interface of neurobiology and linguistics
Poeppel, D., Idsardi, W. J., & van Wassenhove, V. (2008) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2007.2160 · Cited by 497
Self-citationMultimodal interaction: A review
Turk, M. (2014) · Pattern Recognition Letters · doi:10.1016/j.patrec.2013.07.003 · Cited by 496
Predictive top-down integration of prior knowledge during speech perception
Sohoglu, E., Peelle, J. E., Carlyon, R. P., & Davis, M. H. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.5069-11.2012 · Cited by 388
“This account, which is gathering increasing experimental support (Murray et al., 2002; van Wassenhove et al., 2005; Alink et al., 2010; Arnal et al., 2011), proposes that top-down predictions are compared with incoming sensory input and only unexplained activity (or error) propagated through the…”
BackgroundIntegration of Touch and Sound in Auditory Cortex
Kayser, C., Petkov, C. I., Augath, M., & Logothetis, N. K. (2005) · Neuron · doi:10.1016/j.neuron.2005.09.018 · Cited by 388
How does the bilingual experience sculpt the brain?
Costa, A., & Sebastián-Gallés, N. (2014) · Nature Reviews Neuroscience · doi:10.1038/nrn3709 · Cited by 377
Skipper, J. I., van Wassenhove, V., Nusbaum, H. C., & Small, S. L. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhl147 · Cited by 369
Self-citationTransitions in neural oscillations reflect prediction errors generated in audiovisual speech
Arnal, L. H., Wyart, V., & Giraud, A.-L. (2011) · Nature Neuroscience · doi:10.1038/nn.2810 · Cited by 353
The bridge of iconicity: from a world of experience to the experience of language
Perniss, P., & Vigliocco, G. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0300 · Cited by 345
Neural Correlates of Multisensory Integration of Ecologically Valid Audiovisual Events
Stekelenburg, J. J., & Vroomen, J. (2007) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2007.19.12.1964 · Cited by 336
Multisensory Processing in Review: from Physiology to Behaviour
Alais, D., Newell, F., & Mamassian, P. (2010) · Seeing and Perceiving · doi:10.1163/187847510x488603 · Cited by 324
Implicit multisensory associations influence voice recognition
von Kriegstein, K., & Giraud, A.-L. (2006) · PLoS Biology · doi:10.1371/journal.pbio.0040326 · Cited by 309
Crossmodal binding: Evaluating the “unity assumption” using audiovisual speech stimuli
Vatakis, A., & Spence, C. (2007) · Perception & Psychophysics · doi:10.3758/bf03193776 · Cited by 298
Multisensory synesthetic interactions in the speeded classification of visual size
Gallace, A., & Spence, C. (2006) · Perception & Psychophysics · doi:10.3758/bf03193720 · Cited by 296
Blau, V., van Atteveldt, N., Ekkebus, M., Goebel, R., & Blomert, L. (2009) · Current Biology · doi:10.1016/j.cub.2009.01.065 · Cited by 289
The processing of audio-visual speech: empirical and neural bases
Campbell, R. (2008) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2007.2155 · Cited by 282
Effects of hearing and vision impairments on the Montreal Cognitive Assessment
Dupuis, K., Pichora-Fuller, M. K., Chasteen, A. L., Marchuk, V., Singh, G., & Smith, S. L. (2015) · Aging, Neuropsychology, and Cognition · doi:10.1080/13825585.2014.968084 · Cited by 278
Fontolan, L., Morillon, B., Liegeois-Chauvel, C., & Giraud, A.-L. (2014) · Nature Communications · doi:10.1038/ncomms5694 · Cited by 276
Multisensory Integration: Flexible Use of General Operations
van Atteveldt, N., Murray, M. M., Thut, G., & Schroeder, C. E. (2014) · Neuron · doi:10.1016/j.neuron.2014.02.044 · Cited by 272
Mental imagery of speech and movement implicates the dynamics of internal forward models
Tian, X. (2010) · Frontiers in Psychology · doi:10.3389/fpsyg.2010.00166 · Cited by 259
“…can be induced by visual presentation of a face and that following auditory stimuli can be affected (see Jaaskelainen, 2010 for recent discussion of AV integration models; Sams et al., 1991; van Wassenhove et al., 2005; Von Kriegstein and Giraud, 2006; Skipper et al., 2007; Poeppel et al., 2008).”
Spence, C., & Parise, C. (2010) · Consciousness and Cognition · doi:10.1016/j.concog.2009.12.001 · Cited by 267
Attention and prediction in human audition: a lesson from cognitive psychophysiology
Schröger, E., Marzecová, A., & SanMiguel, I. (2015) · European Journal of Neuroscience · doi:10.1111/ejn.12816 · Cited by 262
Hickok, G. (2012) · Journal of Communication Disorders · doi:10.1016/j.jcomdis.2012.06.004 · Cited by 260
Visual Input Enhances Selective Speech Envelope Tracking in Auditory Cortex at a “Cocktail Party”
Zion Golumbic, E., Cogan, G. B., Schroeder, C. E., & Poeppel, D. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3675-12.2013 · Cited by 258
Self-citationDual Neural Routing of Visual Facilitation in Speech Processing
Arnal, L. H., Morillon, B., Kell, C. A., & Giraud, A.-L. (2009) · The Journal of Neuroscience · doi:10.1523/jneurosci.3194-09.2009 · Cited by 241
Reflections on mirror neurons and speech perception
Lotto, A. J., Hickok, G. S., & Holt, L. L. (2009) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.11.008 · Cited by 247
Auditory-visual fusion in speech perception in children with cochlear implants
Schorr, E. A., Fox, N. A., van Wassenhove, V., & Knudsen, E. I. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0508862102 · Cited by 220
Dynamic causal modelling for EEG and MEG
Kiebel, S. J., Garrido, M. I., Moran, R. J., & Friston, K. J. (2008) · Cognitive Neurodynamics · doi:10.1007/s11571-008-9038-0 · Cited by 211
“…are many analysis schemes available, which look at different components of the signal: analysis of evoked responses (Kiebel and Friston 2004; van Wassenhove et al. 2005), of single trials using multivariate techniques like independent component analysis, analysis S. J. Kiebel (&) M. I. Garrido…”
BackgroundMethodTian, X., & Poeppel, D. (2013) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00381 · Cited by 172
Self-citationPrediction, Bayesian inference and feedback in speech recognition
Norris, D., McQueen, J. M., & Cutler, A. (2016) · Language, Cognition and Neuroscience · doi:10.1080/23273798.2015.1081703 · Cited by 170
Tian, X., & Poeppel, D. (2012) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2012.00314 · Cited by 161
Self-citationHow prior expectations shape multisensory perception
Gau, R., & Noppeney, U. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2015.09.045 · Cited by 109
“…loss, auditory weight is reduced (Ernst and Banks, 2002; Helbig et al., 2012)), general sensory dominance or potentially inter-sensory attention (Foxe et al., 2013; Ross et al., 2007; Rouger et al., 2008; Schwartz, 2010; Senkowski et al., 2008; Talsma et al., 2007; van Wassenhove et al., 2005).”
BackgroundPredictive Coding in Sensory Cortex
Kok, P., & de Lange, F. P. (2015) · An Introduction to Model-Based Cognitive Neuroscience · doi:10.1007/978-1-4939-2236-9_11 · Cited by 98
Brief Report: Arrested Development of Audiovisual Speech Perception in Autism Spectrum Disorders
Stevenson, R. A., Siemann, J. K., Woynaroski, T. G., Schneider, B. C., Eberly, H. E., Camarata, S. M., & Wallace, M. T. (2014) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-013-1992-7 · Cited by 84
The corollary discharge in humans is related to synchronous neural oscillations
Chen, C.-M. A., Mathalon, D. H., Roach, B. J., Cavus, I., Spencer, D. D., & Ford, J. M. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2010.21589 · Cited by 86
Kauramäki, J., Jääskeläinen, I. P., Hari, R., Möttönen, R., Rauschecker, J. P., & Sams, M. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.1950-09.2010 · Cited by 47
Vaden, K. I., Piquado, T., & Hickok, G. (2011) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2011.21620 · Cited by 36
Temporal synchrony and audiovisual integration of speech and object stimuli in autism
Smith, E., Zhang, S., & Bennetto, L. (2017) · Research in Autism Spectrum Disorders · doi:10.1016/j.rasd.2017.04.001 · Cited by 33
Exploring the boundaries of gesture-speech integration during language comprehension
Kelly, S. D. (2017) · Gesture Studies · doi:10.1075/gs.7.12kel · Cited by 30
Borowiak, K., Schelinski, S., & von Kriegstein, K. (2018) · NeuroImage: Clinical · doi:10.1016/j.nicl.2018.09.019 · Cited by 17
Borgolte, A., Roy, M., Sinke, C., Wiswede, D., Stephan, M., Bleich, S., Münte, T. F., & Szycik, G. R. (2021) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2021.108022 · Cited by 13
Fridriksson (2009) · doi:10.1044/1092-4388(2008/06-0197
Metadata
- Authors
- van Wassenhove, V., Grant, K. W., & Poeppel, D.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 2005-01-12
- Volume
- 102
- Issue
- 4
- Pages
- 1181-1186
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- OpenAlex
- W2158387459
- Cited by
- 854
- References
- 46
- FWCI
- 37.48
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.