Neural processing of asynchronous audiovisual speech perception
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
The temporal synchrony of auditory and visual signals is known to affect the perception of an external event, yet it is unclear what neural mechanisms underlie the influence of temporal synchrony on perception. Using parametrically varied levels of stimulus asynchrony in combinations with BOLD fMRI, we identified two anatomically distinct subregions of multisensory superior temporal cortex (mSTC) that showed qualitatively distinct BOLD activation patterns. A synchrony-defined subregion of mSTC (synchronous > asynchronous) responded only when auditory and visual stimuli were synchronous, whereas a bimodal subregion of mSTC (auditory > baseline and visual > baseline) showed significant activation to all presentations, but showed monotonically increasing activation with increasing levels of asynchrony. The presence of two distinct activation patterns suggests that the two subregions of mSTC may rely on different neural mechanisms to integrate audiovisual sensory signals. An additional whole-brain analysis revealed a network of regions responding more with synchronous than asynchronous speech, including right mSTC, and bilateral superior colliculus, fusiform gyrus, lateral occipital cortex, and extrastriate visual cortex. The spatial location of individual mSTC ROIs was much more variable in the left than right hemisphere, suggesting that individual differences may contribute to the right lateralization of mSTC in a group SPM. These findings suggest that bilateral mSTC is composed of distinct multisensory subregions that integrate audiovisual speech signals through qualitatively different mechanisms, and may be differentially sensitive to stimulus properties including, but not limited to, temporal synchrony.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183
Forman, S. D., Cohen, J. D., Fitzgerald, M., Eddy, W. F., Mintun, M. A., & Noll, D. C. (1995) · Magnetic Resonance in Medicine · doi:10.1002/mrm.1910330508 · Cited by 3,202
Visual Contribution to Speech Intelligibility in Noise
Sumby, W. H., & Pollack, I. (1954) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907309 · Cited by 2,583 · cited 4×
“…et al., 2004; Vroomen et al., 2001) or stimulus quality may be modulated to investigatemultisensory enhancement at low signal-to-noise ratios (James et al., 2009; Kim and James, in press; Stevenson and James, 2009; Sumby and Pollack, 1954; Summerfield, 1987; Werner and Noppeney, in press).”
InfluentialCalvert, G. A., Campbell, R., & Brammer, M. J. (2000) · Current Biology · doi:10.1016/s0960-9822(00)00513-3 · Cited by 954
Analysis of a large fMRI cohort: statistical and methodological issues for group analyses
Thirion, B., Pinel, P., Mériaux, S., Roche, A., Dehaene, S., & Poline, J.-B. (2007) · NeuroImage · doi:10.1016/j.neuroimage.2006.11.054 · Cited by 563 · cited 2×
“The cluster-threshold correction technique used here controls false positives, with a relative sparing of statistical power (Forman et al., 1995; Thirion et al., 2007), and has previously been used to define mSTC (Stevenson et al., 2009).”
Divide and conquer: A defense of functional localizers
Saxe, R., Brett, M., & Kanwisher, N. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2005.12.062 · Cited by 544 · cited 2×
“6) is known to occur (Saxe et al., 2006), and has been reported previously in mSTC and other multisensory regions (Kim and James, in press; Puce et al.”
Unraveling multisensory integration: patchy organization within human STS multisensory cortex
Beauchamp, M. S., Argall, B. D., Bodurka, J., Duyn, J. H., & Martin, A. (2004) · Nature Neuroscience · doi:10.1038/nn1333 · Cited by 513 · cited 7×
“Previous studies using similar functional definitions have suggested that mSTC may be functionally heterogeneous, in particular, that there may be a patchy organization of subregions with small unisensory subregions that feed forward into integrative subregions (Beauchamp et al., 2004a).”
InfluentialTemporal constraints on the McGurk effect
Munhall, K. G., Gribble, P., Sacco, L., & Ward, M. (1996) · Perception & Psychophysics · doi:10.3758/bf03206811 · Cited by 358 · cited 2×
“The drop in perceived synchrony was greater with V-A than with A-V stimulus presentations, replicating earlier studies of asynchrony detection (Conrey and Pisoni, 2006; Conrey and Pisoni, 2004; Dixon and Spitz, 1980; Grant and Greenberg, 2001; Grant et al., 2004; McGrath and Summerfield, 1985; Miller and D'Esposito, 2…”
Intermodal timing relations and audio-visual speech recognition by normal-hearing adults
McGrath, M., & Summerfield, Q. (1985) · The Journal of the Acoustical Society of America · doi:10.1121/1.392336 · Cited by 201 · cited 4×
“The drop in perceived synchrony was greater with V-A than with A-V stimulus presentations, replicating earlier studies of asynchrony detection (Conrey and Pisoni, 2006; Conrey and Pisoni, 2004; Dixon and Spitz, 1980; Grant and Greenberg, 2001; Grant et al., 2004; McGrath and Summerfield, 1985; Miller and D'Esposito, 2…”
Influential
Cited by
Multisensory temporal integration in autism spectrum disorders
Stevenson, R. A., Siemann, J. K., Schneider, B. C., Eberly, H. E., Woynaroski, T. G., Camarata, S. M., & Wallace, M. T. (2014) · The Journal of Neuroscience · doi:10.1523/jneurosci.3615-13.2014 · Cited by 482
“…similar substrates have been implicated in the processing of audiovisual stimuli and the fine temporal structure of sensory stimuli (Miller and D’Esposito, 2005; Stevenson et al., 2010), as well as in the integration of audiovisual speech stimuli (Stevenson et al., 2011; Nath and Beauchamp, 2012).”
BackgroundSelf-citationBaum, S. H., Stevenson, R. A., & Wallace, M. T. (2015) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2015.09.007 · Cited by 370
Self-citationWallace, M. T., & Stevenson, R. A. (2014) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2014.08.005 · Cited by 315
Self-citationStevenson, R. A., Zemtsov, R. K., & Wallace, M. T. (2012) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/a0027339 · Cited by 269
“Thus, variations in the temporal disparity between cross-modal stimuli have been shown to modulate the amount of activity measured in pSTS in a number of studies (Macaluso et al., 2004; Miller & D’Esposito, 2005; Stevenson et al., 2010; Stevenson et al., 2011).”
BackgroundInfluentialCostantini, M., Robinson, J., Migliorati, D., Donno, B., Ferri, F., & Northoff, G. (2016) · Cognition · doi:10.1016/j.cognition.2016.08.010 · Cited by 107
Metadata
- Authors
- Stevenson, R. A., Altieri, N. A., Kim, S., Pisoni, D. B., & James, T. W.
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2010-02
- Volume
- 49
- Issue
- 4
- Pages
- 3308-3318
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W1981312896
- Cited by
- 117
- References
- 9
- FWCI
- 9.64
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Neural processing of asynchronous audiovisual speech perception. https://app.cochat.ai/papers/10.1016/j.neuroimage.2009.12.001
- The paper
- Stevenson, R. A., Altieri, N. A., Kim, S., Pisoni, D. B., & James, T. W. (2010). Neural processing of asynchronous audiovisual speech perception. NeuroImage. https://doi.org/10.1016/j.neuroimage.2009.12.001
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.