Multisensory Integration in Macaque Visual Cortex Depends on Cue Reliability
Open accessAbstract
Responses of multisensory neurons to combinations of sensory cues are generally enhanced or depressed relative to single cues presented alone, but the rules that govern these interactions have remained unclear. We examined integration of visual and vestibular self-motion cues in macaque area MSTd in response to unimodal as well as congruent and conflicting bimodal stimuli in order to evaluate hypothetical combination rules employed by multisensory neurons. Bimodal responses were well fit by weighted linear sums of unimodal responses, with weights typically less than one (subadditive). Surprisingly, our results indicate that weights change with the relative reliabilities of the two cues: visual weights decrease and vestibular weights increase when visual stimuli are degraded. Moreover, both modulation depth and neuronal discrimination thresholds improve for matched bimodal compared to unimodal stimuli, which might allow for increased neural sensitivity during multisensory stimulation. These findings establish important new constraints for neural models of cue integration.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Congruent neuron activity is highly predictive of behavioral choice on vestibular-visual combination tasks.
“Congruent neurons appear to combine vestibular–visual signals in a way that assumes a common cause, and their activity is highly predictive of behavioral choice on vestibular–visual combination tasks (8–10).”
From How multisensory neurons solve causal inference. (2021) · cites this paper as [10]
The Discussion’s “Modulation depth and discrimination threshold” section reports improved neural discriminability for congruent cells under aligned visual–vestibular stimulation, but explicitly says that a parallel improvement in behavioral sensitivity during multimodal discrimination remains to be demonstrated. It does not establish that their activity is highly predictive of behavioral choice. The indexed passages provide no page numbers. (LLM, conf=0.86)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredCongruent neurons combine vestibular-visual signals in a way that assumes a common cause.
“Congruent neurons appear to combine vestibular–visual signals in a way that assumes a common cause, and their activity is highly predictive of behavioral choice on vestibular–visual combination tasks (8–10).”
From How multisensory neurons solve causal inference. (2021) · cites this paper as [10]
The Results describe congruent MSTd neurons combining visual and vestibular inputs, and the Discussion reports that weighted linear sums describe bimodal responses. The paper does not establish that these neurons assume a common cause; its Results also say the monkeys only maintained fixation, so it does not test whether neural activity predicts behavioral choice. (LLM, conf=0.84)
Experimental — accuracy not yet measuredWhen a small conflict is introduced between two cues, an observer's judgment incorporates both signals but depends more on the more reliable signal.
“In particular, if a small conflict is introduced between two cues, an observer’s judgment will incorporate both signals but depend more on the signal that is more reliable (i.e., has the lowest estimator variance) (9, 10, 21).”
From How multisensory neurons solve causal inference. (2021) · cites this paper as [10]
The Introduction (page not provided) says humans often combine cue estimates with weights proportional to their relative reliabilities. The Results report cue-conflict stimuli, but the monkeys only maintained fixation; the paper measures neuronal responses, not observers’ judgments under a small cue conflict. That specific condition is not established by this paper’s findings. (LLM, conf=0.84)
Experimental — accuracy not yet measuredAsymmetries in sensitivity to different cues are common among macaque MSTd neurons.
“We found that this was made possible by asymmetries in sensitivity to different cues, which are also common among macaque MSTd neurons (6, 8, 10, 11).”
From How multisensory neurons solve causal inference. (2021) · cites this paper as [10]
Vestibular and visual signals are combined in macaque MSTd.
“Neurophysiological work in macaque monkeys indicates that vestibular and visual signals are combined on the dorsal subdivision of the medial superior temporal area (MSTd) (6–11) and in the ventral intraparietal area (VIP) (12, 13).”
From How multisensory neurons solve causal inference. (2021) · cites this paper as [10]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Humans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844 · see citing works
The Ventriloquist Effect Results from Near-Optimal Bimodal Integration
Alais, D., & Burr, D. (2004) · Current Biology · doi:10.1016/j.cub.2004.01.029 · Cited by 1,936 · see citing works
Multisensory integration: current issues from the perspective of the single neuron
Stein, B. E., & Stanford, T. R. (2008) · Nature Reviews Neuroscience · doi:10.1038/nrn2331 · Cited by 1,598 · see citing works
Bayesian inference with probabilistic population codes
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006) · Nature Neuroscience · doi:10.1038/nn1790 · Cited by 1,457 · see citing works
Meredith, M. A., & Stein, B. E. (1986) · Journal of Neurophysiology · doi:10.1152/jn.1986.56.3.640 · Cited by 1,183 · see citing works
Neuronal oscillations and multisensory interaction in primary auditory cortex
Lakatos, P., Chen, C.-M., O'Connell, M. N., Mills, A., & Schroeder, C. E. (2007) · Neuron · doi:10.1016/j.neuron.2006.12.011 · Cited by 961 · see citing works
Interactions Among Converging Sensory Inputs in the Superior Colliculus
Meredith, M. A., & Stein, B. E. (1983) · Science · doi:10.1126/science.6867718 · Cited by 850 · see citing works
Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors
Meredith, M. A., Nemitz, J. W., & Stein, B. (1987) · Journal of Neuroscience · doi:10.1523/jneurosci.07-10-03215.1987 · Cited by 763 · see citing works
Essen, D. C. V., Maunsell, J. H. R., & Bixby, J. L. (1981) · The Journal of Comparative Neurology · doi:10.1002/cne.901990302 · Cited by 692 · see citing works
Simple models for reading neuronal population codes.
Seung, H. S., & Sompolinsky, H. (1993) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.90.22.10749 · Cited by 623 · see citing works
Multiple visual areas in the caudal superior temporal sulcus of the macaque
Desimone, R., & Ungerleider, L. G. (1986) · The Journal of Comparative Neurology · doi:10.1002/cne.902480203 · Cited by 588 · see citing works
Neural correlates of multisensory cue integration in macaque MSTd
Gu, Y., Angelaki, D. E., & DeAngelis, G. C. (2008) · Nature Neuroscience · doi:10.1038/nn.2191 · Cited by 554 · see citing works
Self-citationMultisensory Integration of Dynamic Faces and Voices in Rhesus Monkey Auditory Cortex
Ghazanfar, A. A., Maier, J. X., Hoffman, K. L., & Logothetis, N. K. (2005) · Journal of Neuroscience · doi:10.1523/jneurosci.0799-05.2005 · Cited by 531 · see citing works
Visual Modulation of Neurons in Auditory Cortex
Kayser, C., Petkov, C. I., & Logothetis, N. K. (2008) · Cerebral Cortex · doi:10.1093/cercor/bhm187 · Cited by 502 · see citing works
Calvert, G. A., Hansen, P. C., Iversen, S. D., & Brammer, M. J. (2001) · NeuroImage · doi:10.1006/nimg.2001.0812 · Cited by 464 · see citing works
Bayesian integration of visual and auditory signals for spatial localization
Battaglia, P., Jacobs, R. A., & Aslin, R. Ν. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001391 · Cited by 426 · see citing works
Representation and integration of multiple sensory inputs in primate superior colliculus
Wallace, M. T., Wilkinson, L. K., & Stein, B. E. (1996) · Journal of Neurophysiology · doi:10.1152/jn.1996.76.2.1246 · Cited by 415 · see citing works
Spatial factors determine the activity of multisensory neurons in cat superior colliculus
Meredith, M. A., & Stein, B. E. (1986) · Brain Research · doi:10.1016/0006-8993(86)91648-3 · Cited by 393 · see citing works
Physiological and Anatomical Evidence for Multisensory Interactions in Auditory Cortex
Bizley, J. K., Nodal, F. R., Bajo, V. M., Nelken, I., & King, A. J. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhl128 · Cited by 341 · see citing works
Visual-vestibular interactive responses in the macaque ventral intraparietal area (VIP)
Bremmer, F., Klam, F., Duhamel, J., Ben Hamed, S., & Graf, W. (2002) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.2002.02206.x · Cited by 331 · see citing works
Spatial determinants of multisensory integration in cat superior colliculus neurons
Meredith, M. A., & Stein, B. E. (1996) · Journal of Neurophysiology · doi:10.1152/jn.1996.75.5.1843 · Cited by 327 · see citing works
Barraclough, N. E., Xiao, D., Baker, C. I., Oram, M. W., & Perrett, D. I. (2005) · Journal of Cognitive Neuroscience · doi:10.1162/0898929053279586 · Cited by 320 · see citing works
Gu, Y., Watkins, P. V., Angelaki, D. E., & DeAngelis, G. C. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.2356-05.2006 · Cited by 316 · cited 11× · see citing works
“First, what is the combination rule used by neurons to integrate sensory signals from two different sources? Second, how does this rule depend on the relative reliabilities of the sensory cues? We addressed these issues by examining visual-vestibular interactions in the dorsal portion of the medial superior temporal a…”
BackgroundMethodInfluentialSelf-citationEvaluating the Operations Underlying Multisensory Integration in the Cat Superior Colliculus
Stanford, T. R., Quessy, S., & Stein, B. E. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.5095-04.2005 · Cited by 314 · see citing works
Maunsell, J. H. R., & Essen, D. C. V. (1987) · The Journal of Comparative Neurology · doi:10.1002/cne.902660407 · Cited by 308 · see citing works
A functional link between area MSTd and heading perception based on vestibular signals
Gu, Y., DeAngelis, G. C., & Angelaki, D. E. (2007) · Nature Neuroscience · doi:10.1038/nn1935 · Cited by 297 · see citing works
Self-citationMultisensory Integration in the Ventral Intraparietal Area of the Macaque Monkey.
Avillac, M., Ben Hamed, S., & Duhamel, J.-R. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.2646-06.2007 · Cited by 281 · see citing works
MST Neurons Respond to Optic Flow and Translational Movement
Duffy, C. J. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.80.4.1816 · Cited by 277 · see citing works
Local precision of visuotopic organization in the middle temporal area (MT) of the macaque
Albright, T. D., & Desimone, R. (1987) · Experimental Brain Research · doi:10.1007/bf00235981 · Cited by 277 · see citing works
Sugihara, T., Diltz, M. D., Averbeck, B. B., & Romanski, L. M. (2006) · Journal of Neuroscience · doi:10.1523/jneurosci.3550-06.2006 · Cited by 273 · see citing works
Multisensory Integration: Space, Time and Superadditivity
Holmes, N. P., & Spence, C. J. (2005) · Current Biology · doi:10.1016/j.cub.2005.08.058 · Cited by 253 · see citing works
Statistical Criteria in fMRI Studies of Multisensory Integration
Beauchamp, M. S. (2005) · Neuroinformatics · doi:10.1385/ni:3:2:093 · Cited by 227 · see citing works
The contribution of spike threshold to the dichotomy of cortical simple and complex cells
Priebe, N. J., Mechler, F., Carandini, M., & Ferster, D. (2004) · Nature Neuroscience · doi:10.1038/nn1310 · Cited by 217 · see citing works
Laurienti, P. J., Perrault, T. J., Stanford, T. R., Wallace, M. T., & Stein, B. E. (2005) · Experimental Brain Research · doi:10.1007/s00221-005-2370-2 · Cited by 187 · see citing works
Nover, H., Anderson, C. H., & DeAngelis, G. C. (2005) · Journal of Neuroscience · doi:10.1523/jneurosci.1661-05.2005 · Cited by 177 · see citing works
Self-citationRomanski, L. M. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhm099 · Cited by 165 · see citing works
Takahashi, K., Gu, Y., May, P. J., Newlands, S. D., DeAngelis, G. C., & Angelaki, D. E. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.0817-07.2007 · Cited by 162 · see citing works
Self-citationSchlack, A., Hoffmann, K., & Bremmer, F. (2002) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.2002.02251.x · Cited by 159 · see citing works
Perrault, T. J., Vaughan, J. W., Stein, B. E., & Wallace, M. T. (2005) · Journal of Neurophysiology · doi:10.1152/jn.00926.2004 · Cited by 158 · see citing works
Superadditivity in multisensory integration: putting the computation in context
Stanford, T. R., & Stein, B. E. (2007) · Neuroreport · doi:10.1097/wnr.0b013e3280c1e315 · Cited by 156 · see citing works
Heading Representation in MST: Sensory Interactions and Population Encoding
Page, W. K., & Duffy, C. J. (2003) · Journal of Neurophysiology · doi:10.1152/jn.00493.2002 · Cited by 146 · see citing works
Narrow Versus Wide Tuning Curves: What's Best for a Population Code?
Pouget, A., Denève, S., Ducom, J.-C., et al. (1999) · Neural Computation · doi:10.1162/089976699300016818 · Cited by 146 · see citing works
Visual-auditory interactions modulate saccade-related activity in monkey superior colliculus
Frens, M. A., & Opstal, A. J. V. (1998) · Brain Research Bulletin · doi:10.1016/s0361-9230(98)00007-0 · Cited by 132 · see citing works
Fetsch, C. R., Wang, S., Gu, Y., DeAngelis, G. C., & Angelaki, D. E. (2007) · Journal of Neuroscience · doi:10.1523/jneurosci.3553-06.2007 · Cited by 130 · see citing works
Self-citationBimodal Interactions in the Superior Colliculus of the Behaving Cat
Populin, L. C., & Yin, T. C. T. (2002) · Journal of Neuroscience · doi:10.1523/jneurosci.22-07-02826.2002 · Cited by 105 · see citing works
Neuron-Specific Response Characteristics Predict the Magnitude of Multisensory Integration
Perrault, T. J., Vaughan, J. W., Stein, B. E., & Wallace, M. T. (2003) · Journal of Neurophysiology · doi:10.1152/jn.00494.2003 · Cited by 95 · see citing works
Bell, A. H., Corneil, B. D., Munoz, D. P., & Meredith, M. A. (2003) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2003.02976.x · Cited by 44 · see citing works
Skaliora, I., Doubell, T. P., Holmes, N. P., Nodal, F. R., & King, A. J. (2004) · Journal of Neurophysiology · doi:10.1152/jn.00450.2004 · Cited by 44 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- Gattass R Gross CG Visual topography of striate projection zone (MT) in posterior superior temporal sulcus of the macaque J Neurophysiol 1981 46 621 638 7299437
- Kadunce DC Vaughan JW Wallace MT Benedek G Stein BE Mechanisms of within- and cross-modality suppression in the superior colliculus J Neurophysiol 1997 78 2834 2847 9405504
- Robinson DA A Method Of Measuring Eye Movement Using A Scleral Search Coil In A Magnetic Field IEEE Trans Biomed Eng 1963 10 137 145 14121113
- Zeki SM Functional organization of a visual area in the posterior bank of the superior temporal sulcus of the rhesus monkey J Physiol 1974 236 549 573 4207129 PMC1350849
Cited by
Sorted by citations
Normalization as a canonical neural computation
Carandini, M., & Heeger, D. J. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3136 · Cited by 1,906 · see citing works
Crossmodal correspondences: A tutorial review
Spence, C. (2011) · Attention, Perception, & Psychophysics · doi:10.3758/s13414-010-0073-7 · Cited by 1,618 · see citing works
Neural correlates of reliability-based cue weighting during multisensory integration
Fetsch, C. R., Pouget, A., DeAngelis, G. C., & Angelaki, D. E. (2012) · Nature Neuroscience · doi:10.1038/nn.2983 · Cited by 459 · see citing works
Self-citationMultisensory integration: psychophysics, neurophysiology, and computation
Angelaki, D. E., Gu, Y., & DeAngelis, G. C. (2009) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2009.06.008 · Cited by 383 · see citing works
Self-citationCortical Hierarchies Perform Bayesian Causal Inference in Multisensory Perception
Rohe, T., & Noppeney, U. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002073 · Cited by 344 · see citing works
A normalization model of multisensory integration
Ohshiro, T., Angelaki, D. E., & DeAngelis, G. C. (2011) · Nature Neuroscience · doi:10.1038/nn.2815 · Cited by 320 · see citing works
Self-citationNeural Coding of Uncertainty and Probability
Ma, W. J., & Jazayeri, M. (2014) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071013-014017 · Cited by 290 · see citing works
A learning-based approach to artificial sensory feedback leads to optimal integration
Dadarlat, M. C., O'Doherty, J. E., & Sabes, P. N. (2015) · Nature Neuroscience · doi:10.1038/nn.3883 · Cited by 210 · see citing works
Tian, X., & Poeppel, D. (2012) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2012.00314 · Cited by 161 · see citing works
Fetsch, C. R., DeAngelis, G. C., & Angelaki, D. E. (2010) · European Journal of Neuroscience · doi:10.1111/j.1460-9568.2010.07207.x · Cited by 154 · see citing works
“However, it remains unclear how these changes in weights (Morgan et al., 2008) might be related to our preliminary findings of a single-cell correlate of behavioral cue reweighting (measured with ROC-based neurometric functions, as discussed above).”
BackgroundResultConvergence of Vestibular and Visual Self-Motion Signals in an Area of the Posterior Sylvian Fissure
Chen, A., DeAngelis, G. C., & Angelaki, D. E. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.1266-11.2011 · Cited by 141 · see citing works
Self-citationThe neural encoding of self-motion
Cullen, K. E. (2011) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2011.05.022 · Cited by 80 · see citing works
Efficient probabilistic inference in generic neural networks trained with non-probabilistic feedback
Orhan, A. E., & Ma, W. J. (2017) · Nature Communications · doi:10.1038/s41467-017-00181-8 · Cited by 77 · see citing works
How multisensory neurons solve causal inference.
Rideaux, R., Storrs, K. R., Maiello, G., & Welchman, A. E. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2106235118 · Cited by 65 · see citing works
“We found that this was made possible by asymmetries in sensitivity to different cues, which are also common among macaque MSTd neurons (6, 8, 10, 11).”
Signal detection theory, uncertainty, and Poisson-like population codes
Ma, W. J. (2010) · Vision Research · doi:10.1016/j.visres.2010.08.035 · Cited by 60 · see citing works
Zhang, W.-H., Wang, H., Chen, A., Gu, Y., Lee, T. S., Wong, K. M., & Wu, S. (2019) · eLife · doi:10.7554/elife.43753 · Cited by 45 · see citing works
Joint Representation of Depth from Motion Parallax and Binocular Disparity Cues in Macaque Area MT
Nadler, J. W., Barbash, D., Kim, H. R., Shimpi, S., Angelaki, D. E., & DeAngelis, G. C. (2013) · Journal of Neuroscience · doi:10.1523/jneurosci.0251-13.2013 · Cited by 37 · see citing works
Self-citationProscription supports robust perceptual integration by suppression in human visual cortex
Rideaux, R., & Welchman, A. E. (2018) · Nature Communications · doi:10.1038/s41467-018-03400-y · Cited by 27 · see citing works
Kim, H. R., Pitkow, X., Angelaki, D. E., & DeAngelis, G. C. (2016) · Journal of Neurophysiology · doi:10.1152/jn.00005.2016 · Cited by 32 · see citing works
Self-citation
- Authors
- Morgan, M. L., DeAngelis, G. C., & Angelaki, D. E.
- Affiliations
- Washington University in St. Louis (US)
- University of Rochester (US)
- Corresponding author
- Angelaki, Dora E.
- Venue
- Neuron
- Publisher
- Elsevier BV
- Published
- 2008-08
- Volume
- 59
- Issue
- 4
- Pages
- 662-673
- ISSN
- 0896-6273 (ISSN-L)
- PMC
- PMC2601653
- OpenAlex
- W2122904188
- Cited by
- 208
- References
- 48
- Citations by year
- 2026: 7 · 2025: 16 · 2024: 5 · 2023: 8 · 2022: 11 · 2021: 9 · 2020: 10 · 2019: 11 · 2018: 15 · 2017: 17 · 2016: 15 · 2015: 13 · 2014: 11 · 2013: 10 · 2012: 12
- FWCI
- 17.24
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Neuron (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institutes of Health
- National Eye Institute (R01EY017866, EY-017866, EY-016178, R01EY016178)
- National Institute on Deafness and Other Communication Disorders (DC04260., R01DC004260)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Motor Control and Adaptation
- Keywords
- multisensory integration, cue reliability, visual-vestibular integration, self-motion perception, macaque visual cortex, bimodal stimuli, cue weighting, stimulus degradation
- MeSH terms
- Analysis of Variance, Animals, Cues, Discrimination, Psychological, Evoked Potentials, Kinesthesis, Macaca mulatta, Male, Mental Processes, Models, Neurological, Motion Perception, Neurons, Sensory Thresholds, Temporal Lobe, Vestibule, Labyrinth, Visual Cortex
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review