Benefits of Stimulus Congruency for Multisensory Facilitation of Visual Learning
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Studies of perceptual learning have largely focused on unisensory stimuli. However, multisensory interactions are ubiquitous in perception, even at early processing stages, and thus can potentially play a role in learning. Here, we examine the effect of auditory-visual congruency on visual learning. METHODOLOGY/PRINCIPLE FINDINGS: Subjects were trained over five days on a visual motion coherence detection task with either congruent audiovisual, or incongruent audiovisual stimuli. Comparing performance on visual-only trials, we find that training with congruent audiovisual stimuli produces significantly better learning than training with incongruent audiovisual stimuli or with only visual stimuli. CONCLUSIONS/SIGNIFICANCE: This advantage from stimulus congruency during training suggests that the benefits of multisensory training may result from audiovisual interactions at a perceptual rather than cognitive level.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

![Figure 2 Data from each training session for congruent audiovisual group (green), unisensory visual group (red), and incongruent audiovisual group (blue). Ordinate is proportion correct averaged across three signal levels, abscissa reflects training session number. Solid lines reflect performance on visual-only trials over the first third of each session; dashed lines represent performance on audiovisual trials over the first third of each session. Error bars reflect within-group standard error [19].](https://figures.cochat.ai/paper-figures/348bf40fcc8c46829b8d80347094a802/benefits-of-stimulus-congruency-for-multisensory-facilitation-of-visual-learning-figure-2.jpg)

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In fact, after training with audiovisual feedback, learning of motion perception tasks was still enhanced even though auditory feedback was withdrawn (Kim, Seitz, & Shams, 2008;Seitz, Kim, & Shams, 2006).”
From Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review (2013) · cites this paper as Kim, Seitz, & Shams, 2008
The cited source says what the claim attributes to it.
In the citing paper
“Researchers have assumed that, after training with multimodal stimuli, multimodal processing is activated even if only unimodal stimuli are present (Kim et al., 2008;Seitzetal.,2006;Shams & Seitz, 2008).”
From Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review (2013) · cites this paper as Kim et al., 2008
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using confidence intervals in within-subject designs
Loftus, G. R., & Masson, M. E. J. (1994) · Psychonomic Bulletin & Review · doi:10.3758/bf03210951 · Cited by 2,448 · cited 1×
“Error bars reflect within-group standard error [19].”
The analysis of visual motion: a comparison of neuronal and psychophysical performance
Britten, K., Shadlen, M., Newsome, W., & Movshon, J. (1992) · The Journal of Neuroscience · doi:10.1523/jneurosci.12-12-04745.1992 · Cited by 1,958 · cited 1×
“A Movshon/Newsome-type motion algorithm [17] was employed with white dots (0.”
MethodIs neocortex essentially multisensory?
GHAZANFAR, A., & SCHROEDER, C. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.04.008 · Cited by 1,463 · cited 1×
“However, crossmodal interactions are ubiquitous in human perception, and can occur even at early stages of processing in areas previously viewed as ‘‘sensory-specific’’ [5–11].”
BackgroundRecanzone, G., Schreiner, C., & Merzenich, M. (1993) · The Journal of Neuroscience · doi:10.1523/jneurosci.13-01-00087.1993 · Cited by 1,266 · cited 1×
“This type of learning is particularly interesting because it seems to demonstrate a surprising degree of cortical plasticity even in adult primary visual [1–2], auditory [3] and somotosensory areas [4].”
BackgroundActivation of Auditory Cortex During Silent Lipreading
Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C. R., McGuire, P. K., Woodruff, P. W. R., Iversen, S. D., & David, A. S. (1997) · Science · doi:10.1126/science.276.5312.593 · Cited by 1,025
Anatomical evidence of multimodal integration in primate striate cortex
Falchier, A., Clavagnier, S., Barone, P., & Kennedy, H. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-13-05749.2002 · Cited by 892 · cited 1×
“While these results cannot speak to the exact mechanism by which multisensory training mediates visual learning, anatomical studies have found evidence for input to lower level visual areas both directly from primary and parabelt auditory cortex [25,26], and also from the superior temporal polysensory area [25].”
ResultPractising orientation identification improves orientation coding in V1 neurons
Schoups, A., Vogels, R., Qian, N., & Orban, G. (2001) · Nature · doi:10.1038/35087601 · Cited by 856 · cited 1×
“This type of learning is particularly interesting because it seems to demonstrate a surprising degree of cortical plasticity even in adult primary visual [1–2], auditory [3] and somotosensory areas [4].”
BackgroundVisuo-haptic object-related activation in the ventral visual pathway
Amedi, A., Malach, R., Hendler, T., Peled, S., & Zohary, E. (2001) · Nature Neuroscience · doi:10.1038/85201 · Cited by 630 · cited 1×
“. To date, most studies of perceptual learning have focused on the learning of sensory features with a single modality. However, crossmodal interactions are ubiquitous in human perception, and can occur even at early stages of processing in areas previously viewed as “sensory-specific””
BackgroundSensory modalities are not separate modalities: plasticity and interactions
Shimojo, S. (2001) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(00)00241-5 · Cited by 598 · cited 1×
“Furthermore, crossmodal interactions are known to play an important role in the development of perceptual systems [12–14].”
BackgroundMultisensory convergence in calcarine visual areas in macaque monkey
Rockland, K. S., & Ojima, H. (2003) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(03)00121-1 · Cited by 564 · cited 1×
“While these results cannot speak to the exact mechanism by which multisensory training mediates visual learning, anatomical studies have found evidence for input to lower level visual areas both directly from primary and parabelt auditory cortex [25,26], and also from the superior temporal polysensory area [25].”
ResultInvolvement of visual cortex in tactile discrimination of orientation
Zangaladze, A., Epstein, C. M., Grafton, S. T., & Sathian, K. (1999) · Nature · doi:10.1038/44139 · Cited by 508 · cited 1×
“However, crossmodal interactions are ubiquitous in human perception, and can occur even at early stages of processing in areas previously viewed as ‘‘sensory-specific’’ [5–11].”
BackgroundPerceptual learning and top-down influences in primary visual cortex
Li, W., Piëch, V., & Gilbert, C. D. (2004) · Nature Neuroscience · doi:10.1038/nn1255 · Cited by 473 · cited 1×
“This type of learning is particularly interesting because it seems to demonstrate a surprising degree of cortical plasticity even in adult primary visual [1–2], auditory [3] and somotosensory areas [4].”
BackgroundFoxe, J. J., Morocz, I. A., Murray, M. M., Higgins, B. A., Javitt, D. C., & Schroeder, C. E. (2000) · Cognitive Brain Research · doi:10.1016/s0926-6410(00)00024-0 · Cited by 395 · cited 1×
“However, crossmodal interactions are ubiquitous in human perception, and can occur even at early stages of processing in areas previously viewed as ‘‘sensory-specific’’ [5–11].”
BackgroundSound alters activity in human V1 in association with illusory visual perception
Watkins, S., Shams, L., Tanaka, S., Haynes, J.-D., & Rees, G. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.016 · Cited by 343 · cited 1×
“However, crossmodal interactions are ubiquitous in human perception, and can occur even at early stages of processing in areas previously viewed as ‘‘sensory-specific’’ [5–11].”
BackgroundSelf-citationPharmacological Modulation of Perceptual Learning and Associated Cortical Reorganization
Dinse, H. R., Ragert, P., Pleger, B., Schwenkreis, P., & Tegenthoff, M. (2003) · Science · doi:10.1126/science.1085423 · Cited by 285 · cited 1×
“This type of learning is particularly interesting because it seems to demonstrate a surprising degree of cortical plasticity even in adult primary visual [1–2], auditory [3] and somotosensory areas [4].”
BackgroundMotion Opponency in Visual Cortex
Heeger, D. J., Boynton, G. M., Demb, J. B., Seidemann, E., & Newsome, W. T. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-16-07162.1999 · Cited by 282 · cited 1×
“For instance, inhibitory interactions at the sensory level are well established between opposite directions of visual motion [20–22].”
BackgroundA common framework for perceptual learning
Seitz, A. R., & Dinse, H. R. (2007) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2007.02.004 · Cited by 276 · cited 1×
“This would be consistent with the proposal that for learning to take place it requires that neuronal activity exceeds a threshold [24].”
BackgroundSelf-citationTransparent motion perception as detection of unbalanced motion signals. I. Psychophysics
Qian, N., Andersen, R., & Adelson, E. (1994) · The Journal of Neuroscience · doi:10.1523/jneurosci.14-12-07357.1994 · Cited by 270 · cited 1×
“For instance, inhibitory interactions at the sensory level are well established between opposite directions of visual motion [20–22].”
BackgroundSound Facilitates Visual Learning
Seitz, A. R., Kim, R., & Shams, L. (2006) · Current Biology · doi:10.1016/j.cub.2006.05.048 · Cited by 243 · cited 3×
“To explore the possible benefit of multisensory interactions in adult perceptual learning, we recently compared multisensory with unisensory training [15] using a coherent motion detection and discrimination task.”
BackgroundMethodResultInfluentialSelf-citationIntersensory redundancy facilitates discrimination of tempo in 3‐month‐old infants
Bahrick, L. E., Flom, R., & Lickliter, R. (2002) · Developmental Psychobiology · doi:10.1002/dev.10049 · Cited by 185 · cited 1×
“Furthermore, crossmodal interactions are known to play an important role in the development of perceptual systems [12–14].”
BackgroundEarly modulation of visual cortex by sound: an MEG study
Shams, L. B., Iwaki, S., Chawla, A., & Bhattacharya, J. (2005) · Neuroscience Letters · doi:10.1016/j.neulet.2004.12.035 · Cited by 151 · cited 1×
“However, crossmodal interactions are ubiquitous in human perception, and can occur even at early stages of processing in areas previously viewed as ‘‘sensory-specific’’ [5–11].”
BackgroundSelf-citationDavis, E. P., Donzella, B., Krueger, W. K., & Gunnar, M. R. (1999) · Developmental Psychobiology · doi:10.1002/(sici)1098-2302(199911)35:3<188::aid-dev3>3.0.co;2-k · Cited by 129
Task-specific disruption of perceptual learning
Seitz, A. R., Yamagishi, N., Werner, B., Goda, N., Kawato, M., & Watanabe, T. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0505765102 · Cited by 114
Self-citationInhibition and disinhibition of direction-specific mechanisms in human vision
Levinson, E., & Sekuler, R. (1975) · Nature · doi:10.1038/254692a0 · Cited by 54 · cited 1×
“For instance, inhibitory interactions at the sensory level are well established between opposite directions of visual motion [20–22].”
BackgroundAdaptive processing of visual motion.
Ball, K., & Sekuler, R. (1981) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.7.4.780 · Cited by 27 · cited 1×
“Because perception of cardinal directions may be robust to training [18], we chose to train 190u (instead of 180u) for leftward motion.”
BackgroundAdaptive processing of visual motion.
Ball, K., & Sekuler, R. (1981) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/0096-1523.7.4.780 · Cited by 22
Cited by
Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review
Sigrist, R., Rauter, G., Riener, R., & Wolf, P. (2013) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0333-8 · Cited by 1,416
“Evidence for this conclusion is given by experiments showing that congruent audiovisual feedback was effective for learning the task, whereas incongruent audiovisual feedback was not (Kim et al., 2008).”
BackgroundResultBenefits of multisensory learning
Shams, L., & Seitz, A. R. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.07.006 · Cited by 1,178
Self-citationCrossmodal influences on visual perception
Shams, L., & Kim, R. (2010) · Physics of Life Reviews · doi:10.1016/j.plrev.2010.04.006 · Cited by 227
“Crossmodal sensory signals appear to be used to recalibrate vision [1,6,35,36,112], adjust the relative weight of visual cues [5,22], and to enhance perceptual learning in low-level visual tasks [42,78,85].”
BackgroundInfluentialSelf-citationvan Kesteren, M. T. R., Rijpkema, M., Ruiter, D. J., & Fernández, G. (2010) · The Journal of Neuroscience · doi:10.1523/jneurosci.2674-10.2010 · Cited by 229
“DOI:10.1523/JNEUROSCI.2674-10.2010 Copyright © 2010 the authors 0270-6474/10/3015888-07$15.00/0 rience (Kim et al., 2008; Yuval-Greenberg and Deouell, 2009).”
InfluentialPerspectives on sensory processing disorder: a call for translational research
Miller, L. (2009) · Frontiers in Integrative Neuroscience · doi:10.3389/neuro.07.022.2009 · Cited by 183
Metadata
- Authors
- Kim, R. S., Seitz, A. R., & Shams, L.
- Affiliations
- University of California, Los Angeles (US)
- Boston University (US)
- Corresponding author
- Shams, Ladan
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2008-01-30
- Volume
- 3
- Issue
- 1
- Pages
- e1532
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC2211398
- OpenAlex
- W1966714841
- Cited by
- 148
- References
- 25
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 8 · 2023: 13 · 2022: 9 · 2021: 14 · 2020: 7 · 2019: 9 · 2018: 11 · 2017: 10 · 2016: 8 · 2015: 8 · 2014: 6 · 2013: 7 · 2012: 7
- FWCI
- 10.45
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- OpenBU (Boston University) (submittedVersion)
- Funders
- National Science Foundation
- Office of Naval Research
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Keywords
- perceptual learning, multisensory integration, audiovisual congruency, multisensory training, auditory-visual interaction
- MeSH terms
- Adult, Humans, Learning, Motion, Photic Stimulation, Task Performance and Analysis, Vision, Ocular
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.