‘When Birds of a Feather Flock Together’: Synesthetic Correspondences Modulate Audiovisual Integration in Non-Synesthetes
Open accessAbstract
BACKGROUND: Synesthesia is a condition in which the stimulation of one sense elicits an additional experience, often in a different (i.e., unstimulated) sense. Although only a small proportion of the population is synesthetic, there is growing evidence to suggest that neurocognitively-normal individuals also experience some form of synesthetic association between the stimuli presented to different sensory modalities (i.e., between auditory pitch and visual size, where lower frequency tones are associated with large objects and higher frequency tones with small objects). While previous research has highlighted crossmodal interactions between synesthetically corresponding dimensions, the possible role of synesthetic associations in multisensory integration has not been considered previously. METHODOLOGY: Here we investigate the effects of synesthetic associations by presenting pairs of asynchronous or spatially discrepant visual and auditory stimuli that were either synesthetically matched or mismatched. In a series of three psychophysical experiments, participants reported the relative temporal order of presentation or the relative spatial locations of the two stimuli. PRINCIPAL FINDINGS: The reliability of non-synesthetic participants' estimates of both audiovisual temporal asynchrony and spatial discrepancy were lower for pairs of synesthetically matched as compared to synesthetically mismatched audiovisual stimuli. CONCLUSIONS: Recent studies of multisensory integration have shown that the reduced reliability of perceptual estimates regarding intersensory conflicts constitutes the marker of a stronger coupling between the unisensory signals. Our results therefore indicate a stronger coupling of synesthetically matched vs. mismatched stimuli and provide the first psychophysical evidence that synesthetic congruency can promote multisensory integration. Synesthetic crossmodal correspondences therefore appear to play a crucial (if unacknowledged) role in the multisensory integration of auditory and visual information.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
According to Bayesian decision theory, the brain uses prior knowledge about what sensory signals go together to integrate appropriate sensory stimuli to maximise reliability of perceived information and reduce cognitive load.
“According to Bayesian decision theory, the brain uses prior knowledge about what sensory signals go together-whether inborn or explicitly learned-to integrate appropriate sensory stimuli with the goal of maximising the reliability of perceived information [139-141] and, presumably, to reduce cognitive load by combining disparate sensory cues into a single object.”
From The Role of Intrinsic and Extrinsic Sensory Factors in Sweetness Perception of Food and Beverages: A Review (2019) · cites this paper as [140]
The Introduction and Discussion (pages not available in our copy) support the role of prior knowledge about which signals go together in determining whether they are integrated. They do not establish that integration aims to maximise the reliability of perceived information; instead, the Introduction describes reduced reliability of estimates of crossmodal conflicts as a cost of integration. The sentence’s suggestion about reducing cognitive load follows the citations as the citing authors’ own presumption. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Quest: A Bayesian adaptive psychometric method
Watson, A. B., & Pelli, D. G. (1983) · Perception & Psychophysics · doi:10.3758/bf03202828 · Cited by 2,398 · see citing works
The psychometric function: I. Fitting, sampling, and goodness of fit
Wichmann, F. A., & Hill, N. J. (2001) · Perception & Psychophysics · doi:10.3758/bf03194544 · Cited by 2,359 · see citing works
The Ventriloquist Effect Results from Near-Optimal Bimodal Integration
Alais, D., & Burr, D. (2004) · Current Biology · doi:10.1016/j.cub.2004.01.029 · Cited by 1,936 · see citing works
A Method for Obtaining and Analyzing Sensitivity Data
Dixon, W. J., & Mood, A. M. (1948) · Journal of the American Statistical Association · doi:10.1080/01621459.1948.10483254 · Cited by 1,223 · see citing works
Immediate perceptual response to intersensory discrepancy.
Welch, R. B., & Warren, D. H. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.88.3.638 · Cited by 1,176 · see citing works
A study in phonetic symbolism.
Sapir, E. (1929) · Journal of Experimental Psychology · doi:10.1037/h0070931 · Cited by 1,013 · see citing works
Spatial and Temporal Contrast-Sensitivity Functions of the Visual System
Robson, J. G. (1966) · Journal of the Optical Society of America · doi:10.1364/josa.56.001141 · Cited by 853 · see citing works
The Bagplot: A Bivariate Boxplot
Rousseeuw, P. J., Ruts, I., & Tukey, J. W. (1999) · The American Statistician · doi:10.2307/2686061 · Cited by 568 · see citing works
Combining Sensory Information: Mandatory Fusion Within, but Not Between, Senses
Hillis, J. M., Ernst, M. O., Banks, M. S., & Landy, M. S. (2002) · Science · doi:10.1126/science.1075396 · Cited by 486 · see citing works
Auditory capture of vision: examining temporal ventriloquism
Morein‐Zamir, S., Soto‐Faraco, S., & Kingstone, A. F. (2003) · Cognitive Brain Research · doi:10.1016/s0926-6410(03)00089-2 · Cited by 462 · see citing works
Temporal and spatial dependency of the ventriloquism effect
Slutsky, D. A., & Recanzone, G. H. (2001) · Neuroreport · doi:10.1097/00001756-200101220-00009 · Cited by 344 · see citing works
Learning to integrate arbitrary signals from vision and touch
Ernst, M. O. (2007) · Journal of Vision · doi:10.1167/7.5.7 · Cited by 332 · see citing works
Martino, G., & Marks, L. E. (2001) · Current Directions in Psychological Science · doi:10.1111/1467-8721.00116 · Cited by 318 · see citing works
Shore, D. I., Spence, C. J., & Klein, R. M. (2001) · Psychological Science · doi:10.1111/1467-9280.00337 · Cited by 316 · see citing works
Self-citationCrossmodal binding: Evaluating the “unity assumption” using audiovisual speech stimuli
Vatakis, A., & Spence, C. J. (2007) · Perception & Psychophysics · doi:10.3758/bf03193776 · Cited by 298 · see citing works
Self-citationMultisensory synesthetic interactions in the speeded classification of visual size
Gallace, A., & Spence, C. J. (2006) · Perception & Psychophysics · doi:10.3758/bf03193720 · Cited by 297 · see citing works
Self-citationOn cross-modal similarity: Auditory-visual interactions in speeded discrimination.
Marks, L. E. (1987) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.13.3.384 · Cited by 290 · see citing works
Automatic visual bias of perceived auditory location
Bertelson, P., & Aschersleben, G. (1998) · Psychonomic Bulletin & Review · doi:10.3758/bf03208826 · Cited by 256 · see citing works
Interaction between synesthetically corresponding dimensions.
Melara, R. D., & O'Brien, T. P. (1987) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.116.4.323 · Cited by 223 · see citing works
Dzn, H. D. L. (1954) · Journal of the Optical Society of America · doi:10.1364/josa.44.000380 · Cited by 222 · see citing works
Vision and touch are automatically integrated for the perception of sequences of events
Bresciani, J., Dammeier, F., & Ernst, M. O. (2006) · Journal of Vision · doi:10.1167/6.5.2 · Cited by 180 · see citing works
Effects of some variations in auditory input upon visual choice reaction time.
Bernstein, I. H., & Edelstein, B. A. (1971) · Journal of Experimental Psychology · doi:10.1037/h0030524 · Cited by 170 · see citing works
Cross-Modal Interactions in Speeded Classification
Marks, L. E. (2004) · The MIT Press eBooks · doi:10.7551/mitpress/3422.003.0009 · Cited by 145 · see citing works
Cross-Modal Interaction between Vision and Touch: The Role of Synesthetic Correspondence
Martino, G., & Marks, L. E. (2000) · Perception · doi:10.1068/p2984 · Cited by 142 · see citing works
Can synaesthesia research inform cognitive science?
Kadosh, R. C., & Henik, A. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.01.003 · Cited by 135 · see citing works
Vatakis, A., & Spence, C. J. (2007) · Acta Psychologica · doi:10.1016/j.actpsy.2006.12.002 · Cited by 124 · see citing works
Self-citationKnowledge about a Common Source Can Promote Visual — Haptic Integration
Helbig, H. B., & Ernst, M. O. (2007) · Perception · doi:10.1068/p5851 · Cited by 106 · see citing works
Synesthetic congruency modulates the temporal ventriloquism effect
Parise, C., & Spence, C. J. (2008) · Neuroscience Letters · doi:10.1016/j.neulet.2008.07.010 · Cited by 99 · see citing works
Self-citationOn cross-modal similarity: The perceptual structure of pitch, loudness, and brightness.
Marks, L. E. (1989) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.15.3.586 · Cited by 94 · see citing works
Facilitation of multisensory integration by the "unity effect" reveals that speech is special
Vatakis, A., Ghazanfar, A. A., & Spence, C. J. (2008) · Journal of Vision · doi:10.1167/8.9.14 · Cited by 94 · see citing works
Self-citationStroop Interference Based on the Synaesthetic Qualities of Auditory Pitch
Walker, P., & Smith, S. B. (1984) · Perception · doi:10.1068/p130075 · Cited by 90 · see citing works
The method of constant stimuli is inefficient
Watson, A. B., & Fitzhugh, A. E. (1990) · Perception & Psychophysics · doi:10.3758/bf03208169 · Cited by 79 · see citing works
King, R. B., & Oldfield, S. R. (1997) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543895 · Cited by 63 · see citing works
Temporal mechanisms of multimodal binding
Burr, D. C., Silva, O., Cicchini, G. M., Banks, M. S., & Morrone, M. C. (2009) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2008.1899 · Cited by 62 · see citing works
Stroop Interference Based on the Multimodal Correlates of Haptic Size and Auditory Pitch
Walker, P., & Smith, S. B. (1985) · Perception · doi:10.1068/p140729 · Cited by 58 · see citing works
Implications for phonetic symbolism: The relationship between pure tones and geometric figures
O’Boyle, M. W., & Tarte, R. D. (1980) · Journal of Psycholinguistic Research · doi:10.1007/bf01068115 · Cited by 54 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [19]Ernst M 2005 A Bayesian view on multimodal cue integration. Knoblich G Thornton I Grosejan M Shiffrar M Perception of the human body from the inside out New York Oxford University Press 105 131
- [30]Ernst M 2007 Learning to integrate arbitrary signals from vision and touch. Journal of Vision 7 1 14 18217847
- [38]Calvert GA Spence C Stein BE 2004 The handbook of multisensory processes Cambridge, MA MIT Press
Cited by
Sorted by citations
Crossmodal correspondences: A tutorial review
Spence, C. (2011) · Attention, Perception, & Psychophysics · doi:10.3758/s13414-010-0073-7 · Cited by 1,618 · see citing works
Self-citationChen, Y.-C., & Spence, C. (2010) · Cognition · doi:10.1016/j.cognition.2009.10.012 · Cited by 221 · see citing works
Self-citationCrisinel, A.-S., & Spence, C. (2010) · Attention, Perception & Psychophysics · doi:10.3758/app.72.7.1994 · Cited by 184 · see citing works
Self-citationWhat sound does that taste? Cross-modal mappings across gustation and audition
Simner, J., Cuskley, C., & Kirby, S. (2010) · Perception · doi:10.1068/p6591 · Cited by 174 · see citing works
Wang, Q. J., Mielby, L. A., Junge, J. Y., Bertelsen, A. S., Kidmose, U., Spence, C., & Byrne, D. V. (2019) · Foods · doi:10.3390/foods8060211 · Cited by 168 · see citing works
Self-citationImplicit association between basic tastes and pitch
Crisinel, A.-S., & Spence, C. (2009) · Neuroscience Letters · doi:10.1016/j.neulet.2009.08.016 · Cited by 141 · see citing works
Self-citationThat Sounds Sweet: Using Cross‐Modal Correspondences to Communicate Gustatory Attributes
Knoeferle, K. M., Woods, A., Käppler, F., & Spence, C. (2015) · Psychology & Marketing · doi:10.1002/mar.20766 · Cited by 137 · see citing works
- Authors
- Parise, C. V., & Spence, C.
- Affiliations
- Max Planck Society (DE)
- University of Oxford (GB)
- Corresponding author
- Parise, Cesare Valerio
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2009-05-27
- Volume
- 4
- Issue
- 5
- Pages
- e5664
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC2680950
- OpenAlex
- W1545018887
- Cited by
- 272
- References
- 36
- Citations by year
- 2026: 10 · 2025: 13 · 2024: 10 · 2023: 22 · 2022: 16 · 2021: 20 · 2020: 11 · 2019: 22 · 2018: 18 · 2017: 14 · 2016: 22 · 2015: 19 · 2014: 12 · 2013: 16 · 2012: 17
- FWCI
- 19.89
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- PUB – Publications at Bielefeld University (Bielefeld University) (submittedVersion)
- Publikationen an der Universität Bielefeld (Universität Bielefeld) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neuroscience and Music Perception
- Keywords
- synesthesia, multisensory integration, audiovisual integration, cross-modal correspondence, temporal order judgment, audiovisual asynchrony
- MeSH terms
- Acoustic Stimulation, Humans, Photic Stimulation, Pitch Discrimination, Space Perception, Time Factors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
