Correlation detection as a general mechanism for multisensory integration
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
The brain efficiently processes multisensory information by selectively combining related signals across the continuous stream of multisensory inputs. To do so, it needs to detect correlation, lag and synchrony across the senses; optimally integrate related information; and dynamically adapt to spatiotemporal conflicts across the senses. Here we show that all these aspects of multisensory perception can be jointly explained by postulating an elementary processing unit akin to the Hassenstein–Reichardt detector—a model originally developed for visual motion perception. This unit, termed the multisensory correlation detector (MCD), integrates related multisensory signals through a set of temporal filters followed by linear combination. Our model can tightly replicate human perception as measured in a series of empirical studies, both novel and previously published. MCDs provide a unified general theory of multisensory processing, which simultaneously explains a wide spectrum of phenomena with a simple, yet physiologically plausible model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Humans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844
Self-citationSpatiotemporal energy models for the perception of motion
Adelson, E. H., & Bergen, J. R. (1985) · Journal of the Optical Society of America A · doi:10.1364/josaa.2.000284 · Cited by 3,442
The Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588
The Ventriloquist Effect Results from Near-Optimal Bimodal Integration
Alais, D., & Burr, D. (2004) · Current Biology · doi:10.1016/j.cub.2004.01.029 · Cited by 1,936
Normalization as a canonical neural computation
Carandini, M., & Heeger, D. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3136 · Cited by 1,906
Merging the senses into a robust percept
Ernst, M. O., & Bülthoff, H. H. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.02.002 · Cited by 1,906 · cited 1×
“, age range 22–35 years, one female) observed the stimuli and had to report (1) whether or not the visual and auditory sequences appeared to be causally related and formed perceptual unity (causality judgment, also known as ‘relatedness’ task17), and (2) the relative temporal order of the two sequences (temporal order…”
BackgroundSelf-citationBayesian inference with probabilistic population codes
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006) · Nature Neuroscience · doi:10.1038/nn1790 · Cited by 1,457
A visual motion detection circuit suggested by Drosophila connectomics
Takemura, S.-Y., Bharioke, A., Lu, Z., Nern, A., Vitaladevuni, S., Rivlin, P. K., Katz, W. T., Olbris, D. J., Plaza, S. M., Winston, P., et al. (2013) · Nature · doi:10.1038/nature12450 · Cited by 686
Causal inference in perception
Shams, L., & Beierholm, U. R. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.07.001 · Cited by 403
Processing properties of ON and OFF pathways for Drosophila motion detection
Behnia, R., Clark, D. A., Carter, A. G., Clandinin, T. R., & Desplan, C. (2014) · Nature · doi:10.1038/nature13427 · Cited by 262
Cited by
How multisensory neurons solve causal inference.
Rideaux, R., Storrs, K. R., Maiello, G., & Welchman, A. E. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2106235118 · Cited by 65
Which hand is mine? Discriminating body ownership perception in a two-alternative forced-choice task
Chancel, M., & Ehrsson, H. H. (2020) · Attention, Perception, & Psychophysics · doi:10.3758/s13414-020-02107-x · Cited by 73
Leveraging Nonhuman Primate Multisensory Neurons and Circuits in Assessing Consciousness Theory
Noel, J.-P., Ishizawa, Y., Patel, S. R., Eskandar, E. N., & Wallace, M. T. (2019) · The Journal of Neuroscience · doi:10.1523/jneurosci.0934-19.2019 · Cited by 30
Proscription supports robust perceptual integration by suppression in human visual cortex
Rideaux, R., & Welchman, A. E. (2018) · Nature Communications · doi:10.1038/s41467-018-03400-y · Cited by 27
Sensory augmentation: integration of an auditory compass signal into human perception of space
Schumann, F., & O’Regan, J. K. (2017) · Scientific Reports · doi:10.1038/srep42197 · Cited by 25
Metadata
- Authors
- Parise, C. V., & Ernst, M. O.
- Venue
- Nature Communications
- Published
- 2016-06-06
- Volume
- 7
- Issue
- 1
- Pages
- 11543-11543
- ISSN
- 2041-1723 (ISSN-L)
- OpenAlex
- W1503282170
- Cited by
- 185
- References
- 10
- FWCI
- 30.91
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.