Vision as Bayesian inference: analysis by synthesis?
Open accessAbstract
We argue that the study of human vision should be aimed at determining how humans perform natural tasks with natural images. Attempts to understand the phenomenology of vision from artificial stimuli, although worthwhile as a starting point, can lead to faulty generalizations about visual systems, because of the enormous complexity of natural images. Dealing with this complexity is daunting, but Bayesian inference on structured probability distributions offers the ability to design theories of vision that can deal with the complexity of natural images, and that use 'analysis by synthesis' strategies with intriguing similarities to the brain. We examine these strategies using recent examples from computer vision, and outline some important implications for cognitive science.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Bayesian probability theory dictates that agents should form and update their beliefs on the basis of not only sensory evidence but also the prior probability of hypotheses about what is currently present in the world.
“Bayesian probability theory provides a principled way of making such inferences, dictating that agents should form and update their beliefs on the basis of not only the evidence provided by the senses but also the prior probability of the various hypotheses about what is currently present in the world (i.e., expectations) [101].”
From How Do Expectations Shape Perception? (2018) · cites this paper as [101]
The Introduction (“Perception as inference,” page unavailable) describes Bayesian visual inference as combining bottom-up processing with a top-down generative component. “The need for Bottom-up and Top-down processing” likewise describes hypotheses informed by image cues and high-level models. These passages support using more than sensory evidence, but the copy does not explicitly state that agents update beliefs using the prior probabilities of hypotheses; its discussion is also principally about vision rather than agents generally. (LLM, conf=0.77)
Experimental — accuracy not yet measuredBayesian inference is a normative theory, prescribing the best possible inference a system can make given the data.
“Importantly, Bayesian inference is a normative theory, prescribing the best possible inference a system can make given the data.”
From How Do Expectations Shape Perception? (2018) · cites this paper as [101]
The general motif of these theories is that perceptual inference involves a top-down generative component that predicts and constrains the processing of bottom-up input over time.
“The general motif of these theories is the notion that perceptual inference involves a top-down generative component that predicts and constrains the processing of bottom-up input over time [101].”
From How Do Expectations Shape Perception? (2018) · cites this paper as [101]
Bayesian models of cognition have primarily flourished as explanations of the complex dependency of behavior on the environment in domains including intuitive physics, causal learning, perception, motor control, and language.
“Bayesian models of cognition have primarily flourished as explanations of the complex dependency of behavior on the environment in domains including intuitive physics, causal learning, perception, motor control, and language [1-10]”
From Bayesian Brains without Probabilities (2016) · cites this paper as [10]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 39 · Cited by 849
Research works printed but not linked: 10
- Chen X & Yuille AL. "Detecting and Reading Text in Natural Scenes". CVPR (2) (2004): 366-373
- Dayan, P. and Yu, A. "Norepinephrine and Neural Interrupts" NIPS Advances in Neural Infor- mation Processing Systems 18. 2005.
- Duda, R. O., Hart, P. E., and Stork, D. G. (2001). Pattern Classification. John Wiley and Sons, Inc.
- Grenander U. 1996. Elements of pattern theory. Baltimore: Johns Hopkins University Press. xiii, 222 pp.
- Dalmatian photo by R. C. James, in R. L. Gregory, The Intelligent Eye (McGraw-Hill, New York, 1973), p. 14.
- Kersten D, & Schrater PW. 2002. Pattern Inference Theory: A Probabilistic Approach to Vision. In Perception and the Physical World, ed. R Mausfeld, & D Heyer. Chichester: John Wiley & Sons, Ltd.
- Mamassian P, Landy MS, & Maloney LT. (2002). Bayesian modelling of visual perception. In Probabilistic Models of the Brain. R. Rao, B. Olshausen and M. Lewicki (Eds), pp. 13-36. Cambridge, MA: MIT Press
- Palmer SE. Vision Science: Photons to Phenomenology pBradford Books, MIT Press, Cambridge, MA. 1999.
- Ullman S. 1996. High-level Vision: Object Recognition and Visual Cognition. Cambridge, Mas- sachusetts: MIT Press
- Yuille AL, & Bülthoff HH. 1996. Bayesian decision theory and psychophysics. In Perception as Bayesian Inference, ed. DC Knill & W Richards, pp. 123-161. Cambridge, UK: Cambridge University Press
- Authors
- Yuille, A., & Kersten, D.
- Affiliations
- University of California, Los Angeles (US)
- University of Minnesota (US)
- Corresponding author
- Yuille, Alan
- Kersten, Daniel
- Venue
- Trends in Cognitive Sciences
- Publisher
- Elsevier BV
- Published
- 2006-07
- Volume
- 10
- Issue
- 7
- Pages
- 301-308
- ISSN
- 1364-6613 (ISSN-L)
- OpenAlex
- W1533374248
- Cited by
- 849
- References
- 39
- Citations by year
- 2026: 29 · 2025: 40 · 2024: 40 · 2023: 69 · 2022: 34 · 2021: 40 · 2020: 48 · 2019: 36 · 2018: 43 · 2017: 50 · 2016: 47 · 2015: 35 · 2014: 41 · 2013: 41 · 2012: 40
- FWCI
- 2.89
- Type
- journal article
- Language
- en
- Open access
- green
- escholarship.org
- Locations
- Trends in Cognitive Sciences (publishedVersion)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Department of Statistics, UCLA
- Funders
- National Institutes of Health (RO1 EY015261, EY015261)
- Office of Naval Research (N00014)
- National Eye Institute (R01EY015261)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- Bayesian inference, human vision, analysis-by-synthesis, natural images, visual perception, computational models of vision, artificial stimuli, phenomenology of vision, computer vision
- MeSH terms
- Algorithms, Bayes Theorem, Brain, Brain Mapping, Field Dependence-Independence, Humans, Markov Chains, Monte Carlo Method, Orientation, Pattern Recognition, Visual, Probability Theory, Vision, Ocular, Visual Pathways, Models, Statistical, Signal Detection, Psychological
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
