Sensory reliability shapes perceptual inference via two mechanisms
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
To obtain a coherent percept of the environment, the brain should integrate sensory signals from common sources and segregate those from independent sources. Recent research has demonstrated that humans integrate audiovisual information during spatial localization consistent with Bayesian Causal Inference (CI). However, the decision strategies that human observers employ for implicit and explicit CI remain unclear. Further, despite the key role of sensory reliability in multisensory integration, Bayesian CI has never been evaluated across a wide range of sensory reliabilities. This psychophysics study presented participants with spatially congruent and discrepant audiovisual signals at four levels of visual reliability. Participants localized the auditory signals (implicit CI) and judged whether auditory and visual signals came from common or independent sources (explicit CI). Our results demonstrate that humans employ model averaging as a decision strategy for implicit CI; they report an auditory spatial estimate that averages the spatial estimates under the two causal structures weighted by their posterior probabilities. Likewise, they explicitly infer a common source during the common-source judgment when the posterior probability for a common source exceeds a fixed threshold of 0.5. Critically, sensory reliability shapes multisensory integration in Bayesian CI via two distinct mechanisms: First, higher sensory reliability sensitizes humans to spatial disparity and thereby sharpens their multisensory integration window. Second, sensory reliability determines the relative signal weights in multisensory integration under the assumption of a common source. In conclusion, our results demonstrate that Bayesian CI is fundamental for integrating signals of variable reliabilities.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bayesian model selection in social research
Raftery, A. E. (1995) · Sociological Methodology · doi:10.2307/271063 · Cited by 6,823
A note on a general definition of the coefficient of determination
NAGELKERKE, N. J. D. (1991) · Biometrika · doi:10.1093/biomet/78.3.691 · Cited by 5,927 · cited 1×
“The model fit was assessed by the coefficient of determination (Nagelkerke, 1991).”
MethodHumans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844
The Ventriloquist Effect Results from Near-Optimal Bimodal Integration
Alais, D., & Burr, D. (2004) · Current Biology · doi:10.1016/j.cub.2004.01.029 · Cited by 1,936 · cited 7×
“…effect; Figure 2C), we observed not only a main effect of visual reliability, F(1.6, 38.7)¼ 46.147, p , 0.001, g2¼ 0.649, as predicted by forced-fusion models (Alais & Burr, 2004; Ernst & Banks, 2002), but also a main effect of absolute disparity, F(1.5, 37.8)¼ 21.339, p , 0.001, g2 ¼ 0.460.”
BackgroundMethodResultInfluentialBayesian inference with probabilistic population codes
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006) · Nature Neuroscience · doi:10.1038/nn1790 · Cited by 1,457
Immediate perceptual response to intersensory discrepancy.
Welch, R. B., & Warren, D. H. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.88.3.638 · Cited by 1,176 · cited 1×
“…only be integrated when they are close in time, space, and structure (Lee & Noppeney, 2014; Lewald & Guski, 2003; Lewis & Noppeney, 2010; Noppeney, Ostwald, & Werner, 2010; Roach, Heron, & McGraw, 2006; Rohe & Noppeney, 2015; Slutsky & Recanzone, 2001; Wallace et al., 2004; Welch & Warren, 1980).”
Causal Inference in Multisensory Perception
Körding, K. P., Beierholm, U., Ma, W. J., Quartz, S., Tenenbaum, J. B., & Shams, L. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000943 · Cited by 1,069 · cited 16×
“We employed a CI model of audiovisual perception (Koerding et al., 2007).”
BackgroundMethodResultInfluentialComparing families of dynamic causal models
Penny, W. D., Stephan, K. E., Daunizeau, J., Rosa, M. J., Friston, K. J., Schofield, T. M., & Leff, A. P. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000709 · Cited by 688 · cited 3×
“Likewise, family inference (Penny et al., 2010) demonstrated the highest posterior probability for the model averaging strategy for the auditory localization task and the threshold (CITh–0.”
BackgroundInfluentialNeural correlates of reliability-based cue weighting during multisensory integration
Fetsch, C. R., Pouget, A., DeAngelis, G. C., & Angelaki, D. E. (2012) · Nature Neuroscience · doi:10.1038/nn.2983 · Cited by 459
Causal inference in perception
Shams, L., & Beierholm, U. R. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.07.001 · Cited by 403 · cited 3×
“…forcedfusion models that enforce signals to be integrated in a mandatory fashion (Ernst & Banks, 2002) but requires Bayesian Causal Inference (CI) that explicitly models the potential causal structures that could have generated the sensory signals (Koerding et al., 2007; Shams & Beierholm, 2010).”
BackgroundCortical Hierarchies Perform Bayesian Causal Inference in Multisensory Perception
Rohe, T., & Noppeney, U. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002073 · Cited by 344 · cited 2×
“A recent fMRI study suggests that a cortical hierarchy performs Bayesian CI for spatial localization by representing multiple spatial estimates, which are closely related to forced-fusion (e.g., ŜAV,C¼1), fullsegregation (e.g., ŜAV,C¼2), and Bayesian CI (e.g., ŜA; Rohe & Noppeney, 2015).”
BackgroundSelf-citationProbability matching as a computational strategy used in perception
Wozny, D. R., Beierholm, U. R., & Shams, L. (2010) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000871 · Cited by 265
The smart potential behind probability matching
Gaissmaier, W., & Schooler, L. J. (2008) · Cognition · doi:10.1016/j.cognition.2008.09.007 · Cited by 223
A Bayesian View on Multimodal Cue Integration
Ernst, M. O. (2005) · Human Body Perception From The Inside Out · doi:10.1093/oso/9780195178371.003.0006 · Cited by 209
Multisensory Integration in Macaque Visual Cortex Depends on Cue Reliability
Morgan, M. L., DeAngelis, G. C., & Angelaki, D. E. (2008) · Neuron · doi:10.1016/j.neuron.2008.06.024 · Cited by 208
A new perceptual illusion reveals mechanisms of sensory decoding
Jazayeri, M., & Movshon, J. A. (2007) · Nature · doi:10.1038/nature05739 · Cited by 173 · cited 1×
“…previously shown that prior categorization of motion direction may affect subsequent estimation of motion direction by biasing the perceived motion direction away from the decision boundary applied in the categorization task (though see Jazayeri & Movshon, 2007, for an alternative explanation).”
BackgroundTrial-to-trial, uncertainty-based adjustment of decision boundaries in visual categorization
Qamar, A. T., Cotton, R. J., George, R. G., Beck, J. M., Prezhdo, E., Laudano, A., Tolias, A. S., & Ma, W. J. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1219756110 · Cited by 64 · cited 1×
“In other words, visual noise biased participants’ common-source judgments (for related sensory noise-dependent biases in visual categorization tasks, see Qamar et al., 2013).”
Background
Cited by
Hallucinations and strong priors
Corlett, P. R., Horga, G., Fletcher, P. C., Alderson-Day, B., Schmack, K., & Powers, A. R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.001 · Cited by 534
“Audiovisual integration of sensory cues is critically dependent upon the relative precisions of the information to be integrated [62].”
The neural dynamics of hierarchical Bayesian causal inference in multisensory perception
Rohe, T., Ehlis, A.-C., & Noppeney, U. (2019) · Nature Communications · doi:10.1038/s41467-019-09664-2 · Cited by 187
Self-citationAcerbi, L., Dokka, K., Angelaki, D. E., & Ma, W. J. (2018) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006110 · Cited by 111
“Depending on the experimental setup, the bisensory estimation/discrimination and unity judgment tasks might be performed in the same trial (a ‘dual task’ setup; see for example [2, 3]).”
BackgroundHow prior expectations shape multisensory perception
Gau, R., & Noppeney, U. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2015.09.045 · Cited by 109
Self-citationTo integrate or not to integrate: Temporal dynamics of hierarchical Bayesian causal inference
Aller, M., & Noppeney, U. (2019) · PLOS Biology · doi:10.1371/journal.pbio.3000210 · Cited by 98
Self-citation
Metadata
- Authors
- Rohe, T., & Noppeney, U.
- Venue
- Journal of Vision
- Published
- 2015-04-27
- Volume
- 15
- Issue
- 5
- Pages
- 22
- ISSN
- 1534-7362 (ISSN-L)
- OpenAlex
- W783654334
- Cited by
- 125
- References
- 17
- FWCI
- 12.72
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.