The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Visual speech speeds up the neural processing of auditory speech
Open accessAbstract
Synchronous presentation of stimuli to the auditory and visual systems can modify the formation of a percept in either modality. For example, perception of auditory speech is improved when the speaker's facial articulatory movements are visible. Neural convergence onto multisensory sites exhibiting supra-additivity has been proposed as the principal mechanism for integration. Recent findings, however, have suggested that putative sensory-specific cortices are responsive to inputs presented through a different modality. Consequently, when and where audiovisual representations emerge remain unsettled. In combined psychophysical and electroencephalography experiments we show that visual speech speeds up the cortical processing of auditory signals early (within 100 ms of signal onset). The auditory–visual interaction is reflected as an articulator-specific temporal facilitation (as well as a nonspecific amplitude reduction). The latency facilitation systematically depends on the degree to which the visual signal predicts possible auditory targets. The observed auditory–visual data support the view that there exist abstract internal representations that constrain the analysis of subsequent speech inputs. This is evidence for the existence of an “analysis-by-synthesis” mechanism in auditory–visual speech perception.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bootstrap methods: another look at the jackknife
Efron, B. (1979) · The Annals of Statistics · doi:10.1214/aos/1176344552 · Cited by 17,602
Neurophysiological investigation of the basis of the fMRI signal
Logothetis, N. K., Pauls, J., Augath, M., Trinath, T., & Oeltermann, A. (2001) · Nature · doi:10.1038/35084005 · Cited by 6,551
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483
Interplay between Cortical Spreading Depolarization and Seizures
Kramer, D. R., Fujii, T., Ohiorhenuan, I., & Liu, C. Y. (2017) · Stereotactic and Functional Neurosurgery · doi:10.1159/000452841 · Cited by 3,696
An internal model for sensorimotor integration
Wolpert, D. M., Ghahramani, Z., & Jordan, M. I. (1995) · Science · doi:10.1126/science.7569931 · Cited by 3,553
Näätänen, R., & Picton, T. (1987) · Psychophysiology · doi:10.1111/j.1469-8986.1987.tb00311.x · Cited by 3,321
The motor theory of speech perception revised
Liberman, A. M., & Mattingly, I. G. (1985) · Cognition · doi:10.1016/0010-0277(85)90021-6 · Cited by 2,861
Risto, N. (2018) · Routledge eBooks · doi:10.4324/9780429487354 · Cited by 2,024
Merging the senses into a robust percept
Ernst, M. O., & Bülthoff, H. H. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.02.002 · Cited by 1,906
(1994) · Choice Reviews Online · doi:10.5860/choice.31-4365 · Cited by 1,882
Poeppel, D. (2003) · Speech Communication · doi:10.1016/s0167-6393(02)00107-3 · Cited by 1,315
Self-citationVisual properties of neurons in a polysensory area in superior temporal sulcus of the macaque.
Bruce, C., Desimone, R., & Gross, C. G. (1981) · Journal of Neurophysiology · doi:10.1152/jn.1981.46.2.369 · Cited by 1,227
Immediate perceptual response to intersensory discrepancy.
Welch, R. B., & Warren, D. H. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.88.3.638 · Cited by 1,176
Giard, M. H., & Peronnet, F. (1999) · Journal of Cognitive Neuroscience · doi:10.1162/089892999563544 · Cited by 1,072
Cross-modal plasticity: where and how?
Bavelier, D., & Neville, H. J. (2002) · Nature Reviews Neuroscience · doi:10.1038/nrn848 · Cited by 1,040
Activation of Auditory Cortex During Silent Lipreading
Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C. R., McGuire, P. K., Woodruff, P. W. R., Iversen, S. D., & David, A. S. (1997) · Science · doi:10.1126/science.276.5312.593 · Cited by 1,025
Crossmodal Processing in the Human Brain: Insights from Functional Neuroimaging Studies
Calvert, G. A. (2001) · Cerebral Cortex · doi:10.1093/cercor/11.12.1110 · Cited by 1,012
The neural basis of functional brain imaging signals
Attwell, D., & Iadecola, C. (2002) · Trends in Neurosciences · doi:10.1016/s0166-2236(02)02264-6 · Cited by 957
Calvert, G. A., Campbell, R., & Brammer, M. J. (2000) · Current Biology · doi:10.1016/s0960-9822(00)00513-3 · Cited by 954
Anatomical evidence of multimodal integration in primate striate cortex
Falchier, A., Clavagnier, S., Barone, P., & Kennedy, H. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-13-05749.2002 · Cited by 892
Large-scale neuronal theories of the brain
(1995) · Choice Reviews Online · doi:10.5860/choice.32-4182 · Cited by 624
Toward a model for lexical access based on acoustic landmarks and distinctive features
Stevens, K. N. (2002) · The Journal of the Acoustical Society of America · doi:10.1121/1.1458026 · Cited by 597
Multisensory convergence in calcarine visual areas in macaque monkey
Rockland, K. S., & Ojima, H. (2003) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(03)00121-1 · Cited by 564
Multisensory perception: Beyond modularity and convergence
Driver, J., & Spence, C. (2000) · Current Biology · doi:10.1016/s0960-9822(00)00740-5 · Cited by 465
Auditory-Somatosensory Multisensory Processing in Auditory Association Cortex: An fMRI Study
Foxe, J. J., Wylie, G. R., Martinez, A., Schroeder, C. E., Javitt, D. C., Guilfoyle, D., Ritter, W., & Murray, M. M. (2002) · Journal of Neurophysiology · doi:10.1152/jn.2002.88.1.540 · Cited by 426
Somatosensory Input to Auditory Association Cortex in the Macaque Monkey
Schroeder, C. E., Lindsley, R. W., Specht, C., Marcovici, A., Smiley, J. F., & Javitt, D. C. (2001) · Journal of Neurophysiology · doi:10.1152/jn.2001.85.3.1322 · Cited by 419
Deactivation of Sensory-Specific Cortex by Cross-Modal Stimuli
Laurienti, P. J., Burdette, J. H., Wallace, M. T., Yen, Y.-F., Field, A. S., & Stein, B. E. (2002) · Journal of Cognitive Neuroscience · doi:10.1162/089892902317361930 · Cited by 369
Seeing speech: visual information from lip movements modifies activity in the human auditory cortex
Sams, M., Aulanko, R., Hämäläinen, M., Hari, R., Lounasmaa, O. V., Lu, S.-T., & Simola, J. (1991) · Neuroscience Letters · doi:10.1016/0304-3940(91)90914-f · Cited by 367
Multisensory integration, perception and ecological validity
De Gelder, B., & Bertelson, P. (2003) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2003.08.014 · Cited by 366
Temporal constraints on the McGurk effect
Munhall, K. G., Gribble, P., Sacco, L., & Ward, M. (1996) · Perception & Psychophysics · doi:10.3758/bf03206811 · Cited by 358
Speaking in shorthand – A syllable-centric perspective for understanding pronunciation variation
Greenberg, S. (1999) · Speech Communication · doi:10.1016/s0167-6393(99)00050-3 · Cited by 353
A revised view of sensory cortical parcellation
Wallace, M. T., Ramachandran, R., & Stein, B. E. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0305697101 · Cited by 343
Response amplification in sensory-specific cortices during crossmodal binding
Calvert, G. A., Brammer, M. J., Bullmore, E. T., Campbell, R., Iversen, S. D., & David, A. S. (1999) · NeuroReport · doi:10.1097/00001756-199908200-00033 · Cited by 331
Auditory Cortical Neurons Respond to Somatosensory Stimulation
Fu, K.-M. G., Johnston, T. A., Shah, A. S., Arnold, L., Smiley, J., Hackett, T. A., Garraghty, P. E., & Schroeder, C. E. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-20-07510.2003 · Cited by 325
Braddick, O. J., O’Brien, J. M. D., Wattam-Bell, J., Atkinson, J., & Turner, R. (2000) · Current Biology · doi:10.1016/s0960-9822(00)00540-6 · Cited by 296
Audiovisual Integration of Letters in the Human Brain
Raij, T., Uutela, K., & Hari, R. (2000) · Neuron · doi:10.1016/s0896-6273(00)00138-0 · Cited by 282
Polysensory Interactions along Lateral Temporal Regions Evoked by Audiovisual Speech
Wright, T. M. (2003) · Cerebral Cortex · doi:10.1093/cercor/13.10.1034 · Cited by 277
Neural correlates of cross-modal binding
Bushara, K. O., Hanakawa, T., Immisch, I., Toma, K., Kansaku, K., & Hallett, M. (2003) · Nature Neuroscience · doi:10.1038/nn993 · Cited by 226
Woods, D. L., Alho, K., & Algazi, A. (1992) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(92)90004-2 · Cited by 224
On the neuronal basis for multisensory convergence: a brief overview
Meredith, M. A. (2002) · Cognitive Brain Research · doi:10.1016/s0926-6410(02)00059-9 · Cited by 197
Measures of auditory–visual integration in nonsense syllables and sentences
Grant, K. W., & Seitz, P. F. (1998) · The Journal of the Acoustical Society of America · doi:10.1121/1.423751 · Cited by 193
Self-citationEye Position Affects Activity in Primary Auditory Cortex of Primates
Werner-Reiss, U., Kelly, K. A., Trause, A. S., Underhill, A. M., & Groh, J. M. (2003) · Current Biology · doi:10.1016/s0960-9822(03)00168-4 · Cited by 176
Colin, C. (2002) · Clinical Neurophysiology · doi:10.1016/s1388-2457(02)00024-x · Cited by 173
Processing of changes in visual speech in the human auditory cortex
Möttönen, R., Krause, C. M., Tiippana, K. M., & Sams, M. E. (2002) · Cognitive Brain Research · doi:10.1016/s0926-6410(02)00053-8 · Cited by 171
Cortical operational synchrony during audio–visual speech integration
Fingelkurts, A. A., Fingelkurts, A. A., Krause, C. M., Möttönen, R., & Sams, M. E. (2003) · Brain and Language · doi:10.1016/s0093-934x(03)00059-2 · Cited by 74
Watanabe, J., & Iwai, E. (1991) · Brain Research Bulletin · doi:10.1016/0361-9230(91)90099-6 · Cited by 30
Cited by
A cortical network for semantics: (de)constructing the N400
Lau, E. F., Phillips, C., & Poeppel, D. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2532 · Cited by 1,700
Self-citationAn integrated theory of language production and comprehension
Pickering, M. J., & Garrod, S. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12001495 · Cited by 1,569
Is neocortex essentially multisensory?
GHAZANFAR, A., & SCHROEDER, C. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.04.008 · Cited by 1,463
Cortical oscillations and sensory predictions
Arnal, L. H., & Giraud, A.-L. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.05.003 · Cited by 1,114
Sensorimotor Integration in Speech Processing: Computational Basis and Neural Organization
Hickok, G., Houde, J., & Rong, F. (2011) · Neuron · doi:10.1016/j.neuron.2011.01.019 · Cited by 786
Metadata
- Authors
- van Wassenhove, V., Grant, K. W., & Poeppel, D.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 2005-01-12
- Volume
- 102
- Issue
- 4
- Pages
- 1181-1186
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- OpenAlex
- W2158387459
- Cited by
- 854
- References
- 46
- FWCI
- 37.48
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.