Multisensory Integration Sites Identified by Perception of Spatial Wavelet Filtered Visual Speech Gesture Information
The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Perception of speech is improved when presentation of the audio signal is accompanied by concordant visual speech gesture information. This enhancement is most prevalent when the audio signal is degraded. One potential means by which the brain affords perceptual enhancement is thought to be through the integration of concordant information from multiple sensory channels in a common site of convergence, multisensory integration (MSI) sites. Some studies have identified potential sites in the superior temporal gyrus/sulcus (STG/S) that are responsive to multisensory information from the auditory speech signal and visual speech movement. One limitation of these studies is that they do not control for activity resulting from attentional modulation cued by such things as visual information signaling the onsets and offsets of the acoustic speech signal, as well as activity resulting from MSI of properties of the auditory speech signal with aspects of gross visual motion that are not specific to place of articulation information. This fMRI experiment uses spatial wavelet bandpass filtered Japanese sentences presented with background multispeaker audio noise to discern brain activity reflecting MSI induced by auditory and visual correspondence of place of articulation information that controls for activity resulting from the above-mentioned factors. The experiment consists of a low-frequency (LF) filtered condition containing gross visual motion of the lips, jaw, and head without specific place of articulation information, a midfrequency (MF) filtered condition containing place of articulation information, and an unfiltered (UF) condition. Sites of MSI selectively induced by auditory and visual correspondence of place of articulation information were determined by the presence of activity for both the MF and UF conditions relative to the LF condition. Based on these criteria, sites of MSI were found predominantly in the left middle temporal gyrus (MTG), and the left STG/S (including the auditory cortex). By controlling for additional factors that could also induce greater activity resulting from visual motion information, this study identifies potential MSI sites that we believe are involved with improved speech perception intelligibility.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183
Rizzolatti, G., & Arbib, M. A. (1998) · Trends in Neurosciences · doi:10.1016/s0166-2236(98)01260-0 · Cited by 2,858
Cortical mechanisms of human imitation
Iacoboni, M., Woods, R. P., Brass, M., Bekkering, H., Mazziotta, J. C., & Rizzolatti, G. (1999) · Science · doi:10.1126/science.286.5449.2526 · Cited by 2,683
Visual Contribution to Speech Intelligibility in Noise
Sumby, W. H., & Pollack, I. (1954) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907309 · Cited by 2,583 · cited 2×
“It is well known that the addition of visual speech information improves speech intelligibility over that of auditory speech information alone (Grant & Braida 1991; Sumby & Pollack, 1954).”
BackgroundSocial perception from visual cues: Role of the STS region
Allison, T., Puce, A., & McCarthy, G. (2000) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01501-1 · Cited by 2,461 · cited 5×
“, 1997; Calvert & Campbell, 2003) and nonspeech stimuli (Allison et al., 2000).”
BackgroundInfluentialHuman temporal lobe activation by speech and nonspeech sounds
Binder, J. R. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.5.512 · Cited by 1,345 · cited 2×
“The superior temporal gyrus and sulcus (STG/S) responds to auditory speech stimulation (Wise et al., 2001; Binder et al., 2000; Scott, Blank, Rosen, &Wise, 2000), visual speech stimulation (‘‘speechreading’’) (Olson, Gatenby, &Gore, 2002; Campbell et al.”
BackgroundA new brain region for coordinating speech articulation
Dronkers, N. F. (1996) · Nature · doi:10.1038/384159a0 · Cited by 1,192 · cited 1×
“The anterior insula has been implicated with processes related to speech production planning (Dronkers, 1996).”
BackgroundGiard, M. H., & Peronnet, F. (1999) · Journal of Cognitive Neuroscience · doi:10.1162/089892999563544 · Cited by 1,072
Crossmodal Processing in the Human Brain: Insights from Functional Neuroimaging Studies
Calvert, G. A. (2001) · Cerebral Cortex · doi:10.1093/cercor/11.12.1110 · Cited by 1,012
Calvert, G. A., Campbell, R., & Brammer, M. J. (2000) · Current Biology · doi:10.1016/s0960-9822(00)00513-3 · Cited by 954
Cited by
Integrating face and voice in person perception
Campanella, S., & Belin, P. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.10.001 · Cited by 527
Integration of Touch and Sound in Auditory Cortex
Kayser, C., Petkov, C. I., Augath, M., & Logothetis, N. K. (2005) · Neuron · doi:10.1016/j.neuron.2005.09.018 · Cited by 388
Neural Correlates of Multisensory Integration of Ecologically Valid Audiovisual Events
Stekelenburg, J. J., & Vroomen, J. (2007) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2007.19.12.1964 · Cited by 336
Reading Speech from Still and Moving Faces: The Neural Substrates of Visible Speech
Calvert, G. A., & Campbell, R. (2003) · Journal of Cognitive Neuroscience · doi:10.1162/089892903321107828 · Cited by 332
Perceptual Fusion and Stimulus Coincidence in the Cross-Modal Integration of Speech
Miller, L. M., & D'Esposito, M. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.0896-05.2005 · Cited by 322
Metadata
- Authors
- Callan, D. E., Jones, J. A., Munhall, K., Kroos, C., Callan, A. M., & Vatikiotis-Bateson, E.
- Venue
- Journal of Cognitive Neuroscience
- Publisher
- MIT Press - Journals
- Published
- 2004-06-01
- Volume
- 16
- Issue
- 5
- Pages
- 805-816
- ISSN
- 0898-929X (ISSN-L)
- 1530-8898
- OpenAlex
- W2127819611
- Cited by
- 115
- References
- 17
- FWCI
- 8.36
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Multisensory Integration Sites Identified by Perception of Spatial Wavelet Filtered Visual Speech Gesture Information. https://app.cochat.ai/papers/10.1162/089892904970771
- The paper
- Callan, D. E., Jones, J. A., Munhall, K., Kroos, C., Callan, A. M., & Vatikiotis-Bateson, E. (2004). Multisensory Integration Sites Identified by Perception of Spatial Wavelet Filtered Visual Speech Gesture Information. Journal of Cognitive Neuroscience. https://doi.org/10.1162/089892904970771
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.