Assessing the Role of the ‘Unity Assumption’ on Multisensory Integration: A Review
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
There has been longstanding interest from both experimental psychologists and cognitive neuroscientists in the potential modulatory role of various top-down factors on multisensory integration/perception in humans. One such top-down influence, often referred to in the literature as the 'unity assumption,' is thought to occur in those situations in which an observer considers that various of the unisensory stimuli that they have been presented with belong to one and the same object or event (Welch and Warren, 1980). Here, we review the possible factors that may lead to the emergence of the unity assumption. We then critically evaluate the evidence concerning the consequences of the unity assumption from studies of the spatial and temporal ventriloquism effects, from the McGurk effect, and from the Colavita visual dominance paradigm. The research that has been published to date using these tasks provides support for the claim that the unity assumption influences multisensory perception under at least a subset of experimental conditions. We then consider whether the notion has been superseded in recent years by the introduction of priors in Bayesian causal inference models of human multisensory perception. We suggest that the prior of common cause (that is, the prior concerning whether multisensory signals originate from the same source or not) offers the most useful way to quantify the unity assumption as a continuous cognitive variable.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Fodor, J. A. (1983) · doi:10.7551/mitpress/4737.001.0001 · Cited by 7,734
Hearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183
A theory of cortical responses.
Friston, K. (2005) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2005.1622 · Cited by 4,578 · cited 1×
“This view suggests that human brain is proactive in predicting the state of the outside world and so the incoming sensory signals; the received sensory signals, in turn, provide sensory evidence to verify the predictions (e.g., Friston, 2005; Bar, 2007; Friston et al., 2015; Kok and de Lange, 2015).”
BackgroundThe Ventriloquist Effect Results from Near-Optimal Bimodal Integration
Alais, D., & Burr, D. (2004) · Current Biology · doi:10.1016/j.cub.2004.01.029 · Cited by 1,936 · cited 2×
“…studies that have wanted to Frontiers in Psychology | www.frontiersin.org 3 March 2017 | Volume 8 | Article 445 test optimal statistical integration based on Bayesian models (e.g., Alais and Burr, 2004, to ask their participants to “think the display as a ball thudding onto the screen,” p. 260).”
BackgroundMethodCrossmodal correspondences: A tutorial review
Spence, C. (2011) · Attention, Perception, & Psychophysics · doi:10.3758/s13414-010-0073-7 · Cited by 1,618 · cited 9×
“The effects of crossmodal correspondences and semantic congruency often occur between stimuli in a larger temporal disparity over hundreds of ms that are represented as two distinct events, such as the studies demonstrating crossmodal semantic priming (Chen and Spence, 2011b, 2013).”
BackgroundInfluentialSelf-citationShams, L., Kamitani, Y., & Shimojo, S. (2000) · Nature · doi:10.1038/35048669 · Cited by 1,262
Cognition does not affect perception: Evaluating the evidence for “top-down” effects
Firestone, C., & Scholl, B. J. (2016) · Behavioral and Brain Sciences · doi:10.1017/s0140525x15000965 · Cited by 1,200
Immediate perceptual response to intersensory discrepancy.
Welch, R. B., & Warren, D. H. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.88.3.638 · Cited by 1,176 · cited 10×
“…top–down influence, often referred to in the literature as the ‘unity assumption,’ is thought to occur in those situations in which an observer considers that various of the unisensory stimuli that they have been presented with belong to one and the same object or event (Welch and Warren, 1980).”
BackgroundMethodInfluentialThe proactive brain: using analogies and associations to generate predictions
Bar, M. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.05.005 · Cited by 1,144
Causal Inference in Multisensory Perception
Körding, K. P., Beierholm, U., Ma, W. J., Quartz, S., Tenenbaum, J. B., & Shams, L. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000943 · Cited by 1,069 · cited 6×
“Frontiers in Psychology | www.frontiersin.org 8 March 2017 | Volume 8 | Article 445 light (that is, a counter-ventriloquism effect was obtained; see also Körding et al., 2007; Rohe and Noppeney, 2015b).”
BackgroundMethodInfluentialThe Handbook of Multisensory Processes
(2004) · doi:10.7551/mitpress/3422.001.0001 · Cited by 948
Anatomical evidence of multimodal integration in primate striate cortex
Falchier, A., Clavagnier, S., Barone, P., & Kennedy, H. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-13-05749.2002 · Cited by 892 · cited 1×
“…compared to central visual field (e.g., Charbonneau et al., 2013; Gleiss and Kayser, 2013; Chen et al., submitted), which is thought to be partly attributable to the more extensive neural connectivity across sensory-dominant areas in the periphery (Falchier et al., 2002; Rockland and Ojima, 2003).”
BackgroundThe multifaceted interplay between attention and multisensory integration.
Talsma, D., Senkowski, D., Soto-Faraco, S., & Woldorff, M. G. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.06.008 · Cited by 828
Vision and Touch: An Experimentally Created Conflict between the Two Senses
Rock, I., & Victor, J. (1964) · Science · doi:10.1126/science.143.3606.594 · Cited by 702 · cited 1×
“When the participants were instructed that “they would be seeing and feeling identical halves of the same object” (p. 121), their performance on the shape matching task demonstrated visual dominance (e.g., Rock and Victor, 1964).”
BackgroundMultisensory convergence in calcarine visual areas in macaque monkey
Rockland, K. S., & Ojima, H. (2003) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(03)00121-1 · Cited by 564
Bayesian just-so stories in psychology and neuroscience.
Bowers, J. S., & Davis, C. J. (2012) · Psychological Bulletin · doi:10.1037/a0026450 · Cited by 436 · cited 1×
“Some, for instance, have argued that Bayesian models simply re-express the cognitive model in mathematical language (see Murphy, 1993; Jones and Love, 2011; Bowers and Davis, 2012).”
BackgroundTalsma, D., Doty, T. J., & Woldorff, M. G. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhk016 · Cited by 434 · cited 1×
“…research has demonstrated that the effect of multisensory integration is more pronounced when the participants attend to stimuli in both sensory modalities rather than focusing on just the stimulus in one modality (e.g., Talsma et al., 2007; Mozolic et al., 2008; Fairhall and Macaluso, 2009).”
BackgroundCausal inference in perception
Shams, L., & Beierholm, U. R. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.07.001 · Cited by 403 · cited 2×
“…and Spence (2007, 2008; Vatakis et al., 2008) were revisiting the unity effect in human behavior, other researchers were examining this issue using the computational modeling approach based on Bayes’ rule (e.g., Ernst, 2007; Körding et al., 2007; see Shams and Beierholm, 2010, 2011, for reviews).”
BackgroundMethodTemporal constraints on the McGurk effect
Munhall, K. G., Gribble, P., Sacco, L., & Ward, M. (1996) · Perception & Psychophysics · doi:10.3758/bf03206811 · Cited by 358 · cited 3×
“In one of the above studies, Munhall et al. (1996) also presented the visual and auditory consonant that could lead to the McGurk effect being sandwiched between either matched or mismatched vowels.”
BackgroundCortical Hierarchies Perform Bayesian Causal Inference in Multisensory Perception
Rohe, T., & Noppeney, U. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002073 · Cited by 344
Learning to integrate arbitrary signals from vision and touch
Ernst, M. O. (2007) · Journal of Vision · doi:10.1167/7.5.7 · Cited by 332
Audio–visual simultaneity judgments
Zampini, M., Guest, S., Shore, D. I., & Spence, C. (2005) · Perception & Psychophysics · doi:10.3758/bf03193329 · Cited by 280 · cited 1×
“8This feature is consistent with the fact that the window in which audiovisual simultaneity is perceived is typically wider when vision leads than when audition leads (Zampini et al., 2005).”
BackgroundSelf-citationParise, C. V., & Spence, C. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005664 · Cited by 272 · cited 4×
“The results of the three experiments reported by Parise and Spence (2009) are consistent with the view that more pronounced multisensory integration occurs for congruent visual and auditory stimuli than for those pairs of stimuli that happen to be incongruent.”
BackgroundResultInfluentialSelf-citationProbability matching as a computational strategy used in perception
Wozny, D. R., Beierholm, U. R., & Shams, L. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000871 · Cited by 265
Green, K. P., Kuhl, P. K., Meltzoff, A. N., & Stevens, E. B. (1991) · Perception & Psychophysics · doi:10.3758/bf03207536 · Cited by 239 · cited 2×
“In addition, the congruency of stimulus identity modulates the extent to which people integrate multisensory speech stimuli (Easton and Basala, 1982; Munhall et al., 1996; though see Green et al., 1991).”
BackgroundSpeech perception as a multimodal phenomenon
Rosenblum, L. D. (2008) · Current Directions in Psychological Science · doi:10.1111/j.1467-8721.2008.00615.x · Cited by 193 · cited 1×
“Before accepting the undoubtedly controversial claim that speech really is ‘special’ (see also Saldaña and Rosenblum, 1993; Tuomainen et al., 2005; though see Rosenblum, 2008; Vroomen and Stekelenburg, 2011), a number of alternative explanations for these null results need to be ruled out first.”
ResultMultisensory Integration in Complete Unawareness: Evidence From Audiovisual Congruency Priming
Faivre, N., Mudrik, L., Schwartz, N., & Koch, C. (2014) · Psychological Science · doi:10.1177/0956797614547916 · Cited by 127
How prior expectations shape multisensory perception
Gau, R., & Noppeney, U. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2015.09.045 · Cited by 109 · cited 3×
“Such a contextual effect on segregating visual and auditory stimuli in the incongruent context is associated with increased activities at the left inferior frontal sulcus (Gau and Noppeney, 2016).”
BackgroundMethod
Cited by
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
Senses of place: architectural design for the multisensory mind
Spence, C. (2020) · Cognitive Research: Principles and Implications · doi:10.1186/s41235-020-00243-4 · Cited by 380
Self-citationOral referral: On the mislocalization of odours to the mouth
Spence, C. (2016) · Food Quality and Preference · doi:10.1016/j.foodqual.2016.02.006 · Cited by 80
Self-citation
Metadata
- Authors
- Chen, Y.-C., & Spence, C.
- Venue
- Frontiers in Psychology
- Publisher
- Frontiers Media SA
- Published
- 2017-03-31
- Volume
- 8
- Pages
- 445-445
- ISSN
- 1664-1078 (ISSN-L)
- OpenAlex
- W2601305433
- Cited by
- 185
- References
- 28
- FWCI
- 22.42
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.