CoChat
Sign Up
Cited by 1,618Multisensory perception and integrationTactile and Sensory InteractionsMultimodal Machine Learning Applications

Crossmodal correspondences: A tutorial review

Open access
Spence, Charles·2011·Attention, Perception, & Psychophysics
DOI 10.3758/s13414-010-0073-7
Open this paper in CoChat — free accountOpen PDF

Abstract

In many everyday situations, our senses are bombarded by many different unisensory signals at any given time. To gain the most veridical, and least variable, estimate of environmental stimuli/properties, we need to combine the individual noisy unisensory perceptual estimates that refer to the same object, while keeping those estimates belonging to different objects or events separate. How, though, does the brain "know" which stimuli to combine? Traditionally, researchers interested in the crossmodal binding problem have focused on the roles that spatial and temporal factors play in modulating multisensory integration. However, crossmodal correspondences between various unisensory features (such as between auditory pitch and visual size) may provide yet another important means of constraining the crossmodal binding problem. A large body of research now shows that people exhibit consistent crossmodal correspondences between many stimulus features in different sensory modalities. For example, people consistently match high-pitched sounds with small, bright objects that are located high up in space. The literature reviewed here supports the view that crossmodal correspondences need to be considered alongside semantic and spatiotemporal congruency, among the key constraints that help our brains solve the crossmodal binding problem.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3758/s13414-010-0073-7

Cites

254 of 254 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • According to Bayesian decision theory, the brain uses prior knowledge about what sensory signals go together to integrate appropriate sensory stimuli to maximise reliability of perceived information and reduce cognitive load.

    “According to Bayesian decision theory, the brain uses prior knowledge about what sensory signals go together-whether inborn or explicitly learned-to integrate appropriate sensory stimuli with the goal of maximising the reliability of perceived information [139-141] and, presumably, to reduce cognitive load by combining disparate sensory cues into a single object.”

    From The Role of Intrinsic and Extrinsic Sensory Factors in Sweetness Perception of Food and Beverages: A Review (2019) · cites this paper as [141]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 12

  1. Belli, F., & Sagrillo, F. (2001). Qual è Takete? Qual è Maluma? La psicolinguistica applicata alla comunicazione pubblicitaria [What is Takete? What is Maluma? Psycholinguistics applied to advertising] (2nd ed.). Milan: Franco Angeli.
  2. Berlin, B. (1994). Evidence for pervasive synthetic sound symbolism in ethnozoological nomenclature. In L. Hinton, J. Nicholls, & J. J. Ohala (Eds.), Sound symbolism (pp. 76-93). New York: Cambridge University Press.
  3. Gal, D., Wheeler, S. C., & Shiv, B. (2011). Cross-modal influences on gustatory perception. Manuscript submitted for publication. Available at http://ssrn.com/abstract=1030197
  4. Hinton, L., Nichols, J., & Ohala, J. J. (Eds.). (1994). Sound symbolism. Cambridge: Cambridge University Press.
  5. Kusnir, F., & Thut, G. (2010, June). When letters evoke colours: Probing for induction of synaesthetic behaviour in non-synaesthetes through explicit versus implicit grapheme-colour associations. Poster presented at the 11th International Multisensory Research Forum meeting, Liverpool, U.K.
  6. Pan, J.-Y., Yang, H.-J., Faloutsos, C., & Duygulu, P. (2004). Automatic multimedia crossmodal correlation discovery. In Proceedings of the Tenth ACM SIGKDD International Confer- ence on Knowledge Discovery and Data Mining (pp. 653-658). Seattle: ACM Press.
  7. Piesse, C. H. (1891). Piesse's art of perfumery (5th ed.). London: Piesse and Lubin. Plato. (1961). In E. Hamilton & H. Cairns (Eds.), The collected dialogues. New York: Pantheon.
  8. Rader, C. M., & Tellegen, A. (1987). An investigation of synaesthesia. Journal of Personality and Social Psychology, 52, 981-987.
  9. Shore, D. I., Barnes, M. E., & Spence, C. (2006). The temporal evolution of the crossmodal congruency effect. Neuroscience Letters, 392, 96-100.
  10. Simner, J., & Ludwig, V. (2009). What colour does that feel? Cross- modal correspondences from touch to colour. Paper presented at the Third International Conference of Synaesthesia and Art, Granada, Spain, April.
  11. In M. H. Zadeh (Ed.), Advances in haptics (pp. 47-74). InTech. Available at www.sciyo.com/articles/show/title/solving-the-corre spondence-problem-in-haptic-multisensory-interface-design
  12. Vatakis, A., Ghazanfar, A., & Spence, C. (2008). Facilitation of multisensory integration by the "unity assumption": Is speech special? Journal of Vision, 8(9), 14:1-11.

Cited by

Sorted by citations

Related

Authors
Spence, C.
Affiliations
University of Oxford (GB)
Corresponding author
Spence, Charles
ORCID
Spence, Charles
Venue
Attention, Perception, & Psychophysics
Publisher
Springer Science and Business Media LLC
Published
2011-01-19
Volume
73
Issue
4
Pages
971-995
ISSN
  • 1943-3921 (ISSN-L)
  • 1943-393X
DOI
10.3758/s13414-010-0073-7
OpenAlex
W2010927954
Cited by
1,618
References
254
Citations by year
2026: 144 · 2025: 121 · 2024: 128 · 2023: 139 · 2022: 109 · 2021: 124 · 2020: 125 · 2019: 160 · 2018: 123 · 2017: 98 · 2016: 79 · 2015: 95 · 2014: 65 · 2013: 58 · 2012: 33
FWCI
7.16
Type
journal article
Language
en
Open access
Licence
www.springer.com/tdm (tdm)
Locations
  • Attention Perception & Psychophysics (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
cross-modal correspondence, multisensory integration, auditory-visual correspondence, semantic congruence, perceptual binding, feature binding
MeSH terms
Association, Attention, Auditory Perception, Bayes Theorem, Discrimination, Psychological, Humans, Psychophysics, Research, Symbolism, Time Perception, Visual Perception

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review