Models and processes of multisensory cue combination
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
Fundamental to our perception of a unified and stable environment is the capacity to combine information across the senses. Although this process appears seamless as an adult, the brain’s ability to successfully perform multisensory cue combination takes years to develop and relies on a number of complex processes including cue integration, cue calibration, causal inference, and reference frame transformations. Further complexities exist because multisensory cue combination is implemented by populations of noisy neurons. In this review, we discuss recent behavioral studies exploring how the brain combines information from different sensory systems, neurophysiological studies relating behavior to neuronal activity, and a theory of neural sensory encoding that can account for many of these experimental findings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Humans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844
The Ventriloquist Effect Results from Near-Optimal Bimodal Integration
Alais, D., & Burr, D. (2004) · Current Biology · doi:10.1016/j.cub.2004.01.029 · Cited by 1,936
Normalization as a canonical neural computation
Carandini, M., & Heeger, D. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3136 · Cited by 1,906
Roitman, J. D., & Shadlen, M. N. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-21-09475.2002 · Cited by 1,589 · cited 1×
“Previous work using this paradigm showed that observers make trade-offs between speed and accuracy [38] and that more reliable evidence leads to faster decisions [39], suggesting that perceptual evidence is accumulated over time until a decision boundary is reached.”
Bayesian inference with probabilistic population codes
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006) · Nature Neuroscience · doi:10.1038/nn1790 · Cited by 1,457 · cited 2×
“This led to the investigation of how the brain can simultaneously represent multiple pieces of sensory information along with their reliabilities, and combine them optimally to implement Bayesian cue integration [14].”
Perception as Bayesian Inference
(1996) · doi:10.1017/cbo9780511984037 · Cited by 1,425
Correlated neuronal discharge rate and its implications for psychophysical performance
Zohary, E., Shadlen, M. N., & Newsome, W. T. (1994) · Nature · doi:10.1038/370140a0 · Cited by 1,237 · cited 1×
“This generalizes a widely used model which assumes that neurons’ firing rates are independent and governed by Poisson statistics to allow for, among other things, correlated neural variability [16, 17] and different mean–variance relationships in firing rates [18].”
Causal Inference in Multisensory Perception
Körding, K. P., Beierholm, U., Ma, W. J., Quartz, S., Tenenbaum, J. B., & Shams, L. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000943 · Cited by 1,069 · cited 1×
“To examine how this inference is performed, one study presented human subjects with synchronized visual flashes and auditory clicks that originated from either the same or different locations, and asked them to indicate both the locations of the stimuli and whether they had one or two causes [12].”
Representation of Confidence Associated with a Decision by Neurons in the Parietal Cortex
Kiani, R., & Shadlen, M. N. (2009) · Science · doi:10.1126/science.1169405 · Cited by 1,027
A common reference frame for movement plans in the posterior parietal cortex
Cohen, Y. E., & Andersen, R. A. (2002) · Nature Reviews Neuroscience · doi:10.1038/nrn873 · Cited by 695
The effect of stimulus strength on the speed and accuracy of a perceptual decision
Palmer, J., Huk, A. C., & Shadlen, M. N. (2005) · Journal of Vision · doi:10.1167/5.5.1 · Cited by 691 · cited 1×
“Previous work using this paradigm showed that observers make trade-offs between speed and accuracy [38] and that more reliable evidence leads to faster decisions [39], suggesting that perceptual evidence is accumulated over time until a decision boundary is reached.”
Probabilistic population codes for Bayesian decision making
Beck, J. M., Ma, W. J., Kiani, R., Hanks, T., Churchland, A. K., Roitman, J., Shadlen, M. N., Latham, P. E., & Pouget, A. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.021 · Cited by 645
Neural correlates of multisensory cue integration in macaque MSTd
Gu, Y., Angelaki, D. E., & DeAngelis, G. C. (2008) · Nature Neuroscience · doi:10.1038/nn.2191 · Cited by 554
Self-citationYoung Children Do Not Integrate Visual and Haptic Form Information
Gori, M., Del Viva, M., Sandini, G., & Burr, D. C. (2008) · Current Biology · doi:10.1016/j.cub.2008.04.036 · Cited by 547
Reference frames for representing visual and tactile locations in parietal cortex
Avillac, M., Denève, S., Olivier, E., Pouget, A., & Duhamel, J.-R. (2005) · Nature Neuroscience · doi:10.1038/nn1480 · Cited by 464
Neural correlates of reliability-based cue weighting during multisensory integration
Fetsch, C. R., Pouget, A., DeAngelis, G. C., & Angelaki, D. E. (2012) · Nature Neuroscience · doi:10.1038/nn.2983 · Cited by 459
Self-citationCausal inference in perception
Shams, L., & Beierholm, U. R. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.07.001 · Cited by 403 · cited 1×
“Behavior in this, and a number of other tasks, can be largely accounted for by a model of Bayesian causal inference in which the probability that two sensory cues have the same underlying cause is computed first, and then Bayesian cue integration is performed taking into account the observer’s belief about the number…”
Variance as a signature of neural computations during decision making
Churchland, A. K., Kiani, R., Chaudhuri, R., Wang, X.-J., Pouget, A., & Shadlen, M. N. (2011) · Neuron · doi:10.1016/j.neuron.2010.12.037 · Cited by 350
Probabilistic Interpretation of Population Codes
Zemel, R. S., Dayan, P., & Pouget, A. (1998) · Neural Computation · doi:10.1162/089976698300017818 · Cited by 333
Multisensory Space Representations in the Macaque Ventral Intraparietal Area
Schlack, A., Sterbing-D'Angelo, S. J., Hartung, K., Hoffmann, K.-P., & Bremmer, F. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.0455-05.2005 · Cited by 318
Multisensory Integration in Macaque Visual Cortex Depends on Cue Reliability
Morgan, M. L., DeAngelis, G. C., & Angelaki, D. E. (2008) · Neuron · doi:10.1016/j.neuron.2008.06.024 · Cited by 208
Self-citationGori, M., Sandini, G., Martinoli, C., & Burr, D. (2010) · Current Biology · doi:10.1016/j.cub.2009.11.069 · Cited by 205 · cited 1×
“Recent studies have shown that children with congenital visual deficits have an impaired ability to determine the object’s orientation by touch [47], and children with movement disorders have an impaired ability to visually discriminate the object’s size [48].”
Multisensory Integration of Natural Odors and Sounds in the Auditory Cortex
Cohen, L., Rothschild, G., & Mizrahi, A. (2011) · Neuron · doi:10.1016/j.neuron.2011.08.019 · Cited by 192
Diverse Spatial Reference Frames of Vestibular Signals in Parietal Cortex
Chen, X., DeAngelis, G. C., & Angelaki, D. E. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.006 · Cited by 94
Self-citationLearning Multisensory Integration and Coordinate Transformation via Density Estimation
Makin, J. G., Fellows, M. R., & Sabes, P. N. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003035 · Cited by 53
Cited by
The connectome of an insect brain
Winding, M., Pedigo, B. D., Barnes, C. L., Patsolic, H. G., Park, Y., Kazimiers, T., Fushiki, A., Andrade, I. V., Khandelwal, A., Valdes-Aleman, J., et al. (2023) · Science · doi:10.1126/science.add9330 · Cited by 435
Baum, S. H., Stevenson, R. A., & Wallace, M. T. (2015) · Progress in Neurobiology · doi:10.1016/j.pneurobio.2015.09.007 · Cited by 370
A computational perspective on autism
Rosenberg, A., Patterson, J. S., & Angelaki, D. E. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1510583112 · Cited by 156
“One possibility is that population activity is decoded to determine the most probable stimulus, and the appropriate behavioral response is then selected accordingly (17, 38, 42).”
BackgroundSelf-citationCharacterizing the roles of alpha and theta oscillations in multisensory attention
Keller, A. S., Payne, L., & Sekuler, R. (2017) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2017.02.021 · Cited by 106
“Given that numerous behavioral studies using concurrent auditory and visual stimuli have demonstrated audio-visual integration (Seilheimer et al., 2014; Driver and Noesselt, 2008; Senkowski et al., 2008; Stein et al., 2014), it is unsurprising that an electroencephalographic correlate of auditory processing would be e…”
Khalvati, K., Kiani, R., & Rao, R. P. N. (2021) · Nature Communications · doi:10.1038/s41467-021-25419-4 · Cited by 64
Metadata
- Authors
- Seilheimer, R. L., Rosenberg, A., & Angelaki, D. E.
- Venue
- Current Opinion in Neurobiology
- Publisher
- Elsevier BV
- Published
- 2014-04
- Volume
- 25
- Pages
- 38-46
- ISSN
- 0959-4388 (ISSN-L)
- OpenAlex
- W2036189952
- Cited by
- 107
- References
- 25
- FWCI
- 1.77
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.