Suboptimal human multisensory cue combination
Open accessAbstract
Information from different sensory modalities can interact, shaping what we think we have seen, heard, or otherwise perceived. Such interactions can enhance the precision of perceptual decisions, relative to those based on information from a single sensory modality. Several computational processes could account for such improvements. Slight improvements could arise if decisions are based on multiple independent sensory estimates, as opposed to just one. Still greater improvements could arise if initially independent estimates are summed to form a single integrated code. This hypothetical process has often been described as optimal when it results in bimodal performance consistent with a summation of unimodal estimates weighted in proportion to the precision of each initially independent sensory code. Here we examine cross-modal cue combination for audio-visual temporal rate and spatial location cues. While suggestive of a cross-modal encoding advantage, the degree of facilitation falls short of that predicted by a precision weighted summation process. These data accord with other published observations, and suggest that precision weighted combination is not a general property of human cross-modal perception.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Deviations from optimal integration have been observed in tasks with time constraints or novel stimuli, suggesting the Bayesian fit may reflect task-specific tuning rather than a general cognitive strategy.
“Moreover, deviations from optimal integration have been observed in tasks with time constraints or novel stimuli (Arnold et al. 2019; Stanford et al. 2005), suggesting that the Bayesian fit may reflect task-specific tuning rather than a general cognitive strategy.”
From The myth of the Bayesian brain (2025) · cites this paper as Arnold et al. 2019
The Discussion reports deviations from precision-weighted integration, and the Implications section suggests that rarely encountered temporal modulations might not trigger that process. This lends some support to the task-specific interpretation, but the paper presents it as a possibility and does not attribute the deviations to time constraints or explicitly to novel stimuli. No page numbers are available in our copy. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Brainard, D. H. (1997) · Spatial Vision · doi:10.1163/156856897x00357 · Cited by 19,308 · see citing works
Journal of experimental psychology: Human Perception and performance
Jong, R. D., Coles, M. G. H., Logan, G. D., & Gratton, G. (1987) · Journal of Sound and Vibration · doi:10.1016/0022-460x(87)90423-8 · Cited by 5,411 · see citing works
Humans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844 · see citing works
Kleiner, M., Brainard, D., & Pelli, D. G. (2007) · MPG.PuRe (Max Planck Society) · Cited by 3,751 · see citing works
Journal of Experimental Psychology: Human Perception and Performance.
Intons-Peterson, M. J. (1981) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/h0078016 · Cited by 3,261 · see citing works
The Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588 · see citing works
The Ventriloquist Effect Results from Near-Optimal Bimodal Integration
Alais, D., & Burr, D. (2004) · Current Biology · doi:10.1016/j.cub.2004.01.029 · Cited by 1,936 · see citing works
Immediate perceptual response to intersensory discrepancy.
Welch, R. B., & Warren, D. H. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.88.3.638 · Cited by 1,176 · see citing works
Feature binding, attention and object perception
Treisman, A. (1998) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.1998.0284 · Cited by 754 · see citing works
Integration of Proprioceptive and Visual Position-Information: An Experimentally Supported Model
van Beers, R. J., Sittig, A. C., & Gon, J. J. D. V. D. (1999) · Journal of Neurophysiology · doi:10.1152/jn.1999.81.3.1355 · Cited by 742 · see citing works
Combining Sensory Information: Mandatory Fusion Within, but Not Between, Senses
Hillis, J. M., Ernst, M. O., Banks, M. S., & Landy, M. S. (2002) · Science · doi:10.1126/science.1075396 · Cited by 486 · see citing works
Neural correlates of reliability-based cue weighting during multisensory integration
Fetsch, C. R., Pouget, A., DeAngelis, G. C., & Angelaki, D. E. (2012) · Nature Neuroscience · doi:10.1038/nn.2983 · Cited by 459 · see citing works
Sound enhances visual perception: Cross-modal effects of auditory organization on vision.
Vroomen, J., & de Gelder, B. (2000) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.26.5.1583 · Cited by 433 · see citing works
Bayesian integration of visual and auditory signals for spatial localization
Battaglia, P., Jacobs, R. A., & Aslin, R. Ν. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001391 · Cited by 426 · see citing works
Temporal and spatial dependency of the ventriloquism effect
Slutsky, D. A., & Recanzone, G. H. (2001) · Neuroreport · doi:10.1097/00001756-200101220-00009 · Cited by 344 · see citing works
Neural Correlates of Multisensory Integration of Ecologically Valid Audiovisual Events
Stekelenburg, J. J., & Vroomen, J. (2007) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2007.19.12.1964 · Cited by 339 · see citing works
A normalization model of multisensory integration
Ohshiro, T., Angelaki, D. E., & DeAngelis, G. C. (2011) · Nature Neuroscience · doi:10.1038/nn.2815 · Cited by 320 · see citing works
Optimal integration of shape information from vision and touch
Helbig, H. B., & Ernst, M. O. (2007) · Experimental Brain Research · doi:10.1007/s00221-006-0814-y · Cited by 266 · see citing works
Roach, N. W., Heron, J., & McGraw, P. V. (2006) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2006.3578 · Cited by 210 · see citing works
Binocular and Uniocular Threshold of Vision
Pirenne, M. H. (1943) · Nature · doi:10.1038/152698a0 · Cited by 175 · see citing works
Temporal Ventriloquism: Sound Modulates the Flash-Lag Effect.
Vroomen, J., & de Gelder, B. (2004) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/0096-1523.30.3.513 · Cited by 151 · see citing works
Low-level integration of auditory and visual motion signals requires spatial co-localisation
Meyer, G. F., Wuerger, S. M., Röhrbein, F., & Zetzsche, C. (2005) · Experimental Brain Research · doi:10.1007/s00221-005-2394-7 · Cited by 106 · see citing works
Audio–visual integration in temporal perception
Wada, Y., Kitagawa, N., & Noguchi, K. (2003) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(03)00128-4 · Cited by 102 · see citing works
Multimodal signal detection: Independent decisions vs. integration
Mulligan, R. M., & Shaw, M. L. (1980) · Perception & Psychophysics · doi:10.3758/bf03204892 · Cited by 58 · see citing works
Crossmodal attention switching: Auditory dominance in temporal discrimination tasks
Lukas, S., Philipp, A. M., & Koch, I. (2014) · Acta Psychologica · doi:10.1016/j.actpsy.2014.10.003 · Cited by 47 · see citing works
Modeling Multisensory Integration
van Dam, L. C. J., Parise, C., & Ernst, M. O. (2014) · The MIT Press eBooks · doi:10.7551/mitpress/9780262027786.003.0010 · Cited by 43 · see citing works
Can Computational Goals Inform Theories of Vision?
Anderson, B. L. (2015) · Topics in Cognitive Science · doi:10.1111/tops.12136 · Cited by 37 · see citing works
Independent Detectors for Expansion and Rotation, and for Orthogonal Components of Deformation
Meese, T. S., & Harris, M. (2001) · Perception · doi:10.1068/p3196 · Cited by 22 · see citing works
Audio-Visual Speech Cue Combination
Arnold, D. H., Tear, M. J., Schindel, R., & Roseboom, W. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010217 · Cited by 20 · see citing works
Self-citationBall, D. M., Arnold, D. H., & Yarrow, K. (2017) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/xhp0000368 · Cited by 16 · see citing works
Self-citationBimodal sensory discrimination is finer than dual single modality discrimination
Koene, A. R., Arnold, D. H., & Johnston, A. (2007) · Journal of Vision · doi:10.1167/7.11.14 · Cited by 11 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Experimentally disambiguating models of sensory cue integration
Scarfe, P. (2022) · Journal of Vision · doi:10.1167/jov.22.1.5 · Cited by 17 · see citing works
The myth of the Bayesian brain
Mangalam, M. (2025) · European Journal of Applied Physiology · doi:10.1007/s00421-025-05855-6 · Cited by 14 · see citing works
“Moreover, deviations from optimal integration have been observed in tasks with time constraints or novel stimuli (Arnold et al. 2019; Stanford et al. 2005), suggesting that the Bayesian fit may reflect task-specific tuning rather than a general cognitive strategy.”
- Authors
- Arnold, D. H., Petrie, K., Murray, C., & Johnston, A.
- Affiliations
- The University of Queensland (AU)
- University of Nottingham (GB)
- Corresponding author
- Arnold, Derek H.
- Venue
- Scientific Reports
- Published
- 2019-03-26
- Volume
- 9
- Issue
- 1
- Pages
- 5155-5155
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC6435731
- OpenAlex
- W2923249668
- Cited by
- 23
- References
- 31
- Citations by year
- 2025: 3 · 2024: 6 · 2023: 4 · 2022: 3 · 2021: 1 · 2020: 3 · 2019: 3
- FWCI
- 2.94
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Scientific Reports (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Repository@Nottingham (University of Nottingham) (publishedVersion, cc-by)
- Repository@Nottingham (University of Nottingham) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Australian Research Council (FT130100605)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Keywords
- cross-modal perception, audiovisual integration, perceptual decision making
- MeSH terms
- Acoustic Stimulation, Auditory Perception, Calibration, Computer Simulation, Cues, Humans, Probability, Sensation, Task Performance and Analysis, Visual Perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
