Combining Sensory Information: Mandatory Fusion Within, but Not Between, Senses
Abstract
Humans use multiple sources of sensory information to estimate environmental properties. For example, the eyes and hands both provide relevant information about an object's shape. The eyes estimate shape using binocular disparity, perspective projection, etc. The hands supply haptic shape information by means of tactile and proprioceptive cues. Combining information across cues can improve estimation of object properties but may come at a cost: loss of single-cue information. We report that single-cue information is indeed lost when cues from within the same sensory modality (disparity and texture gradients in vision) are combined, but not when different modalities (vision and haptics) are combined.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Adults' integration of sensory information comes at a cost of losing access to the individual estimates that feed into the integrated percept ("sensory fusion").
“In adults, sensory integration can lead to mandatory “fusion,” in which the ability to judge the individual component estimates is lost (12, 13).”
From Fusion of visual cues is not mandatory in children (2010) · cites this paper as [12]
Cites
Humans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844 · see citing works
Self-citationde Berg, M., van Kreveld, M. J., Overmars, M., et al. (2000) · Springer eBooks · doi:10.1007/978-3-662-04245-8 · Cited by 1,473 · see citing works
Measurement and modeling of depth cue combination: in defense of weak fusion
Landy, M. S., Maloney, L. T., Johnston, E. B., & Young, M. (1995) · Vision Research · doi:10.1016/0042-6989(94)00176-m · Cited by 996 · see citing works
Self-citationData Fusion for Sensory Information Processing Systems
Clark, J. J., & Yuille, A. (1990) · Springer eBooks · doi:10.1007/978-1-4757-2076-1 · Cited by 519 · see citing works
Horizontal and vertical disparity, eye position, and stereoscopic slant perception
Backus, B. T., Banks, M. S., van Ee, R., & Crowell, J. A. (1999) · Vision Research · doi:10.1016/s0042-6989(98)00139-4 · Cited by 219 · see citing works
Self-citationIntegration of stereopsis and motion shape cues
Johnston, E. B., Cumming, B. G., & Landy, M. S. (1994) · Vision Research · doi:10.1016/0042-6989(94)90106-6 · Cited by 186 · see citing works
Self-citationComputational models of sensorimotor integration
Zoubin Ghahramani, A., Wolpert, D. M., & Jordan, M. I. (1997) · Advances in psychology · doi:10.1016/s0166-4115(97)80006-4 · Cited by 159 · see citing works
Discrimination of planar surface slant from texture: human and ideal observers compared
Knill, D. C. (1998) · Vision Research · doi:10.1016/s0042-6989(97)00325-8 · Cited by 148 · see citing works
Ideal cue combination for localizing texture-defined edges
Landy, M. S., & Kojima, H. (2001) · Journal of the Optical Society of America A · doi:10.1364/josaa.18.002307 · Cited by 137 · see citing works
Self-citationPerceived distance, shape and size
Brenner, E., & Damme, W. J. M. V. (1999) · Vision Research · doi:10.1016/s0042-6989(98)00162-x · Cited by 132 · see citing works
Quantifying sensory channels: generalizing colorimetry to orientation and texture, touch, and tones.
Richards, W. (1979) · PubMed · Cited by 50 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Bayesian integration in sensorimotor learning
Körding, K. P., & Wolpert, D. M. (2004) · Nature · doi:10.1038/nature02169 · Cited by 2,004 · see citing works
Merging the senses into a robust percept
Ernst, M. O., & Bülthoff, H. H. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.02.002 · Cited by 1,906 · see citing works
Self-citationObject perception as Bayesian inference
Kersten, D., Mamassian, P., & Yuille, A. (2004) · Annual Review of Psychology · doi:10.1146/annurev.psych.55.090902.142005 · Cited by 1,269 · see citing works
Relevance of Error: What Drives Motor Adaptation?
Wei, K., & Körding, K. (2009) · Journal of Neurophysiology · doi:10.1152/jn.90545.2008 · Cited by 422 · see citing works
Slant from texture and disparity cues: Optimal cue combination
Hillis, J. M., Watt, S. J., Landy, M. S., & Banks, M. S. (2004) · Journal of Vision · doi:10.1167/4.12.1 · Cited by 421 · see citing works
Self-citationThe statistical determinants of adaptation rate in human reaching
Burge, J., Ernst, M. O., & Banks, M. S. (2008) · Journal of Vision · doi:10.1167/8.4.20 · Cited by 400 · see citing works
Self-citationSuboptimality in perceptual decision making
Rahnev, D., & Denison, R. N. (2018) · Behavioral and Brain Sciences · doi:10.1017/s0140525x18000936 · Cited by 342 · see citing works
Learning to integrate arbitrary signals from vision and touch
Ernst, M. O. (2007) · Journal of Vision · doi:10.1167/7.5.7 · Cited by 332 · see citing works
Self-citationNeural Coding of Uncertainty and Probability
Ma, W. J., & Jazayeri, M. (2014) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071013-014017 · Cited by 290 · see citing works
Parise, C. V., & Spence, C. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005664 · Cited by 272 · see citing works
A Bayesian View on Multimodal Cue Integration
Ernst, M. O. (2005) · Human Body Perception From The Inside Out · doi:10.1093/oso/9780195178371.003.0006 · Cited by 209 · see citing works
Self-citation3D shape perception from combined depth cues in human visual cortex
Welchman, A. E., Deubelius, A., Conrad, V., Bülthoff, H. H., & Kourtzi, Z. (2005) · Nature Neuroscience · doi:10.1038/nn1461 · Cited by 183 · see citing works
Fusion of visual cues is not mandatory in children
Nardini, M., Bedford, R., & Mareschal, D. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1001699107 · Cited by 161 · see citing works
Does Confidence Use a Common Currency Across Two Visual Tasks?
de Gardelle, V., & Mamassian, P. (2014) · Psychological Science · doi:10.1177/0956797614528956 · Cited by 131 · see citing works
Cue combination in human spatial navigation
Chen, X., McNamara, T. P., Kelly, J. W., & Wolbers, T. (2017) · Cognitive Psychology · doi:10.1016/j.cogpsych.2017.04.003 · Cited by 122 · see citing works
Causal inference accounts for heading perception in the presence of object motion
Dokka, K., Park, H., Jansen, M., DeAngelis, G. C., & Angelaki, D. E. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1820373116 · Cited by 80 · see citing works
Confidence as a Common Currency between Vision and Audition
de Gardelle, V., Le Corre, F., & Mamassian, P. (2016) · PLOS ONE · doi:10.1371/journal.pone.0147901 · Cited by 95 · see citing works
Model metamers reveal divergent invariances between biological and artificial neural networks
Feather, J., Leclerc, G., Mądry, A., & McDermott, J. H. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01442-0 · Cited by 59 · see citing works
Cue combination on the circle and the sphere
Murray, R. F., & Morgenstern, Y. (2010) · Journal of Vision · doi:10.1167/10.11.15 · Cited by 32 · see citing works
Proscription supports robust perceptual integration by suppression in human visual cortex
Rideaux, R., & Welchman, A. E. (2018) · Nature Communications · doi:10.1038/s41467-018-03400-y · Cited by 27 · see citing works
Suboptimal human multisensory cue combination
Arnold, D. H., Petrie, K., Murray, C., & Johnston, A. (2019) · Scientific Reports · doi:10.1038/s41598-018-37888-7 · Cited by 23 · see citing works
Experimentally disambiguating models of sensory cue integration
Scarfe, P. (2022) · Journal of Vision · doi:10.1167/jov.22.1.5 · Cited by 17 · see citing works
Minimal theory of 3D vision: New approach to visual scale and visual shape.
Linton, P. (2023) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2021.0455 · Cited by 13 · see citing works
- Authors
- Hillis, J. M., Ernst, M. O., Banks, M. S., & Landy, M. S.
- Affiliations
- University of California, Berkeley (US)
- Max Planck Institute for Biological Cybernetics (DE)
- New York University (US)
- Corresponding author
- Hillis, J. M.
- Ernst, M. O.
- Venue
- Science
- Published
- 2002-11-22
- Volume
- 298
- Issue
- 5598
- Pages
- 1627-1630
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W1977083296
- Cited by
- 486
- References
- 11
- Citations by year
- 2026: 6 · 2025: 12 · 2024: 12 · 2023: 24 · 2022: 22 · 2021: 25 · 2020: 19 · 2019: 21 · 2018: 26 · 2017: 21 · 2016: 21 · 2015: 27 · 2014: 13 · 2013: 32 · 2012: 21
- FWCI
- 18.61
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Max Planck Institute for Plasma Physics (submittedVersion)
- PUB – Publications at Bielefeld University (Bielefeld University) (submittedVersion)
- Publikationen an der Universität Bielefeld (Universität Bielefeld) (submittedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Funders
- National Eye Institute (R01EY012851, EY 08266, EY12851, R01EY008266)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Keywords
- multisensory integration, binocular disparity, texture gradient, tactile cues, proprioception, object shape estimation
- MeSH terms
- Cues, Form Perception, Humans, Mathematics, Stereognosis, Touch, Visual Perception, Vision Disparity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
