Comparing Bayesian and non-Bayesian accounts of human confidence reports
Open accessPublished version of a preprint
The DOI matches its Crossref record; 107 of the 110 references it cites were found in a registry or an index; 19 of the 38 sources its claims cite were checked for support.
Abstract
Humans can meaningfully report their confidence in a perceptual or cognitive decision. It is widely believed that these reports reflect the Bayesian probability that the decision is correct, but this hypothesis has not been rigorously tested against non-Bayesian alternatives. We use two perceptual categorization tasks in which Bayesian confidence reporting requires subjects to take sensory uncertainty into account in a specific way. We find that subjects do take sensory uncertainty into account when reporting confidence, suggesting that brain areas involved in reporting confidence can access low-level representations of sensory uncertainty, a prerequisite of Bayesian inference. However, behavior is not fully consistent with the Bayesian hypothesis and is better described by simple heuristic models that use uncertainty in a non-Bayesian way. Both conclusions are robust to changes in the uncertainty manipulation, task, response modality, model comparison metric, and additional flexibility in the Bayesian model. Our results suggest that adhering to a rational account of confidence behavior may require incorporating implementational constraints.
Overview
CoChat Summary
## TL;DR
This paper tests whether human confidence reports in perceptual categorization are Bayesian, using two tasks that require subjects to use trial-to-trial sensory uncertainty. The authors find that people do use uncertainty, but their confidence behavior is better fit by simple non-Bayesian heuristic models than by Bayesian models.
## Key findings
- In Experiment 1, the Bayes model beat the Fixed model by a summed LOO difference of 2265 [498, 4253], showing that subjects take trial-to-trial sensory uncertainty into account rather than relying only on a point estimate.
- Adding decision noise improved the Bayesian fit by 804 [510, 1134], but the Bayesian model with decision noise (Bayes-dN) was still outperformed by heuristic models: Lin by 1398 [571, 2644] and Quad by 1167 [858, 2698].
- The heuristic models remained better after multiple robustness checks: with non-parametric reliability–uncertainty mapping, Quad still beat Bayes-dN by 1676 [839, 2730]; with incorrect generative-model assumptions, Quad still beat Bayes-dN by 768 [399, 1144].
- Results were consistent across the paper’s controls and replications: in Task A and Task B separately, Quad beat Bayes-dN by 581 [278, 938] and 978 [406, 1756], respectively; in a separate-response experiment, Lin beat Bayes-dN by 396 [186, 622]; and in a Task-B-only experiment, Quad beat Bayes-dN by 1361 [777, 2022].
## Methods
- The paper ran three experiments on perceptual category judgment and confidence: Experiment 1 used a single button press for combined category-confidence reports; Experiment 2 separated category and confidence responses and added trial-by-trial feedback; Experiment 3 used Task B only.
- Experiment 1 had 11 subjects; Experiment 2 had 8 subjects; Experiment 3 had 15 subjects. Tasks used drifting Gabors or ellipses, with reliability manipulated by contrast or ellipse elongation, and participants were trained extensively before testing.
- The authors fit 75 models from 6 distinct model families using MCMC, compared models mainly with leave-one-out cross-validated log likelihood (LOO), and also checked AIC, BIC, AICc, and WAIC; they performed model recovery and several robustness analyses.
## Limitations
- The paper states that, because of the short presentation time, the results may not generalize to tasks requiring integration of evidence over time.
- The authors state that, because the stimuli are very low-level, they cannot say much about high-level stimuli like faces.
- They also state that they only considered explicit confidence ratings, so the findings may not extend to implicit confidence measures. As evident from the text, the tasks were tightly controlled lab perceptual tasks with trained subjects and specific stimulus manipulations, so broader ecological generalization remains limited.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures












Verification
Claims the paper makes
12 Valid · 2 Not valid · 7 Not checked · 96 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“Recent studies have attempted to test the BCH [19, 20] but, because of their experimental designs, are unable to meaningfully distinguish the Bayesian model from any other model of confidence.”
§ Comparing Bayesian and non-Bayesian accounts of human confidence reports > Introduction
What the sources say
- Disputedcites [19] L Aitchison, 2015, Doubly Bayesian analysis of confidence in perceptual decision-making, PLoS Comput Biol sourceExperimental — accuracy not yet measured
The Introduction says that one-dimensional designs cannot distinguish heuristic from Bayes-optimal confidence, but explicitly describes this study’s multidimensional design as a way to test that distinction. Results > Model selection reports that the two-response data significantly favored the Bayesian model over the Max and Difference models. The paper therefore makes the contrary point about its own experimental design, while acknowledging that it cannot test every possible heuristic model. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
- Partly supportedcites [20] JI Sanders, 2016, Signatures of a statistical computation in the human sense of confidence, Neuron sourceExperimental — accuracy not yet measured
The Discussion reports that the study compared human confidence with predictions from a statistical confidence model, supporting its identification as an attempt to test Bayesian confidence. It also notes that other algorithms can approximate Bayes-optimal predictions, but does not say the experimental design was unable to meaningfully distinguish the Bayesian model from any other confidence model. The cited passages have section labels but no page numbers. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Because we are interested in subjects’ intrinsic computation of confidence, we did not instruct or incentivize them to assign probability ranges to each button (e.g., by using a scoring rule [34–36]).”
§ Comparing Bayesian and non-Bayesian accounts of human confidence reports > Results > Experiment 1 During each session, each subject completed two orientation categorization tasks, Tasks A and B. On each trial, a category C was selected randomly (both categories were equally probable), and a stimulus s was drawn from the corresponding stimulus distribution and displayed. The subject categorized the stimulus and simultaneously reported their confidence on a 4-point scale, with a single button press (Fig 1a). Using a single button press for choice and confidence may prevent post-choice influences on the confidence judgment ([25], but see [26]) and emphasized that confidence should reflect the observer’s perception rather than a preceding motor response. The categories were defined by normal distributions on orientation, which differed by task (Fig 1b). In Task A, the distributions had different means (±μC) and the same standard deviation (σC); leftward-tilting stimuli were more likely to be from category 1. Variants of Task A are common in decision-making studies [27]. In Task B, the distributions had the same mean (0°) and different standard deviations (σ1, σ2); stimuli around the horizontal were more likely to be from category 1. Variants of Task B are less common [28–30] but have some properties of perceptual organization tasks; for example, a subject may have to detect when a stimulus belongs to a narrow category (e.g., in which two line segments are collinear) that is embedded in a a broader category (e.g., in which two line segments are unrelated). Subjects were highly trained on the categories; during training, we only used highest-reliability stimuli, and we provided trial-to-trial category correctness feedback. Subjects were then tested with 6 different reliability levels, which were chosen randomly on each trial. During testing, correctness feedback was withheld to avoid the possibility that confidence simply reflects a learned mapping between stimulus orientation and reliability and the probability of being correct [30–33].
What the sources say
- Supportedcites [34] S Massoni, 2014, Confidence measurement in the light of signal detection theory, Front Psychol source
The citation identifies a scoring rule as an example of a way to incentivize confidence reports; the citing authors’ statement about what they did not do is their own methodological description. In “Elicitation Rules,” the source explicitly identifies and uses a Quadratic Scoring Rule, so it supports the cited example. (LLM, conf=0.91)
- Not checked — no full text of the sourcecites [35] GW Brier, 1950, Verification of forecasts expressed in terms of probability, Monthly Weather Review doi:10.1175/1520-0493(1950)078
- Not checked — no full text of the sourcecites [36] T Gneiting, 2007, Strictly proper scoring rules, prediction, and estimation, J Am Stat Assoc source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“After fitting our core models, our results did not differ substantially from experiment 1: Lin fits better than Bayes-dN by 396 [186, 622] and fits better than Fixed by 2095 [1344, 2889] (S6 Fig and S6 Table).”
§ Comparing Bayesian and non-Bayesian accounts of human confidence reports > Results > Model comparison
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 3 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This feature improves Bayes-dN’s fit by 908 [318, 1661], but it still underperforms Quad by 768 [399, 1144] (S1 Table).”
§ Comparing Bayesian and non-Bayesian accounts of human confidence reports > Results > Model comparison
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This feature improves Bayes-dN’s fit by 1310 [580, 2175], but it still underperforms Quad by 362 [162, 602] (S1 Table).”
§ Comparing Bayesian and non-Bayesian accounts of human confidence reports > Results > Model comparison
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
75 more claims
Create a free account to see all 117 claims and every source verdict.
- It may state two separate claims as one. 14
- Probably not a substantial claim. 26
- Borderline: one of the tests for a substantial claim is unclear. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Other work has revealed how people adjust their confidence criteria in the face of changing stimulus uncertainty (Adler & Ma 2018, Aitchison et al. 2015, Denison et al. 2018).”
From Metacognition and Confidence: A Review and Synthesis (2024) · cites this paper as Adler & Ma 2018
Citations
Cites
Brainard, D. H. (1997) · Spatial Vision · doi:10.1163/156856897x00357 · Cited by 19,702
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
The VideoToolbox software for visual psychophysics: transforming numbers into movies
Pelli, D. G. (1997) · Spatial Vision · doi:10.1163/156856897x00366 · Cited by 11,098
(1966) · Medical Entomology and Zoology · Cited by 10,293
Signal detection theory and psychophysics
Ingleby, J. D. (1967) · Journal of Sound and Vibration · doi:10.1016/0022-460x(67)90197-6 · Cited by 8,509
Strictly Proper Scoring Rules, Prediction, and Estimation
Gneiting, T., & Raftery, A. E. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001437 · Cited by 5,793
(2015) · doi:10.1007/s11222-016-9696-4 · Cited by 5,611 · cited 1×
“−1295 [−2051,−734] −2105 [−3353,−1281] −77 [−1881, 1321] −953 [−2922, 565] −3545 [−5177,−2401] −3799 [−5455,−2521] −361 [−675,−135] −1581 [−2477,−986] −2401 [−3750,−1551] −290 [−2170, 1090] −1172 [−3186, 358] −3549 [−5189,−2402] −4136 [−5848,−2777] 1910 [72, 3813] −847 [−2103, 660] −2361 [−3442,−1517] −3043 [−4489,−19…”
Interplay between Cortical Spreading Depolarization and Seizures
Kramer, D. R., Fujii, T., Ohiorhenuan, I., & Liu, C. Y. (2017) · Stereotactic and Functional Neurosurgery · doi:10.1159/000452841 · Cited by 3,696
An Energy Budget for Signaling in the Grey Matter of the Brain
Attwell, D., & Laughlin, S. B. (2001) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1097/00004647-200110000-00001 · Cited by 3,543 · cited 1×
“−345 [−933, 262] −3497 [−4860,−1817] −2836 [−4544,−1122] −3127 [−4549,−1575] −2449 [−4200,−969] 606 [6, 1269]”
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.2307/2291091 · Cited by 2,312
Research works printed but not linked: 1
- Orhan AE, Jacobs RA. Are performance limitations in visual short-term memory tasks due to capacity limitations of model mismatch? arXiv. 2014 Jul;.
Cited by
Items 101–125 of 125
The interplay of uncertainty, relevance and learning influences auditory categorization
Sheth, J. K., Collina, J. S., Piasini, E., Kording, K. P., Cohen, Y. E., & Geffen, M. N. (2025) · Scientific Reports · doi:10.1038/s41598-025-86856-5 · Cited by 2
Rong, Y., & Peters, M. A. K. (2023) · doi:10.31234/osf.io/7pga2 · Cited by 1
Certainty and Singular Causal JudgmentPreprint
O’Neill, K. J. (2021) · Thesis Commons (OSF Preprints) · doi:10.31237/osf.io/d697e
On why we lack confidence in signal-detection-based analyses of confidencePreprint
Arnold, D. H., Johnston, A., Adie, J. M., & Yarrow, K. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.11.07.515537
“Other studies have enjoyed considerable success in explaining human behaviour, in studies of confidence, by describing how confidence criteria might be subject to noise (e.g. Adler & Ma, 2018; Desender et al., 2022; Mamassian & de Gardelle, 2021; Miyoshi & Lau, 2020; Shekhar & Rahnev, 2021).”
BackgroundAnalyzing the Dynamics Between Theory of Mind, Speech Disorders, and Brain Rewiring in Aphasia
Lopez-Soto, T. (2023) · Logic, argumentation & reasoning · doi:10.1007/978-3-031-46742-4_9 · Cited by 1
Confidence and metacognitionPreprint
Miyoshi, K., Webb, T. W., Rahnev, D., & Lau, H. C. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/y3drq · Cited by 2
Subjective confidence modulates individual BOLD patterns of predictive processingPreprint
Hechler, A., de Lange, F. P., & Riedl, V. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.02.23.581697
Jiang, H., Pan, K., Guo, X., Zheng, Z., Zhu, Z., Wu, H., Zhu, J., Santos-Pata, D., Zhang, J., & Kwok, S. C. (2024) · Research Square · doi:10.21203/rs.3.rs-3978190/v1
Green, M. L., & Pratte, M. S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4820539
Raising Social Stakes Raises ConfidencePreprint
Esmaily, J., Abharzad, E., Knogler, S., Deroy, O. D., & Bahrami, B. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4844364
Priors for Natural Image Statistics Inform Confidence in Perceptual DecisionsPreprint
West, R. K., A-Izzeddin, E. J., Sewell, D. K., & Harrison, W. J. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4882462
Suboptimal but intact integration of Bayesian components during perceptual decision-making in autismPreprint
Fazioli, L., Hadad, B. S., Denison, R. N., & Yashar, A. (2024) · doi:10.31234/osf.io/249rv
Mason, L., Woelk, S. P., Eldar, E., & Rutledge, R. B. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.11.18.624111 · Cited by 1
“As a further robustness check, given prior work indicating that confidence is best captured as the squared difference in value and probability 19,28,29 , we compared model variants in which confidence was formalized as the squared difference between stimulus Q values.”
A common algorithm for confidence judgements across visual, auditory and audio-visual decisionsPreprint
West, R. K., Matthews, N. L., Mattingley, J. B., & Sewell, D. K. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.02.12.637846
Xue, K., Fung, H., & Rahnev, D. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5387422
Common psychiatric treatments alter affective dynamicsPreprint
Dercon, Q., Huys, Q. J. M., Rutledge, R. B., & Nord, C. L. (2025) · eLife · doi:10.7554/elife.107269
Common psychiatric treatments alter affective dynamicsPreprint
Dercon, Q., Huys, Q. J. M., Rutledge, R. B., & Nord, C. L. (2025) · eLife · doi:10.7554/elife.107269.1 · Cited by 1
Green, M. L., & Pratte, M. S. (2024) · Cognition · doi:10.1016/j.cognition.2025.106347
Long-term perceptual priors drive confidence bias that favors prior-congruent evidence
Constant, M., Filevich, E., & Mamassian, P. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1013826 · Cited by 2
“This work is also pertinent to an ongoing discussion in the literature about whether confidence computations are Bayesian [5,6,8,56].”
Accumulated evidence as an additive performance measure based on confidence ratings
Meyen, S., Lin, L., Schrenk, C., & Franz, V. H. (2026) · New Ideas in Psychology · doi:10.1016/j.newideapsych.2026.101249
From neural coding to decision-making
Berlemont, K. (2020) · theses.fr (ABES) · doi:10.70675/98ad90cazb967z4ef3zb51az8213de1b171f
Confidence Judgments Reflect the Standard Error of Noisy Evidence Samples Across DomainsPreprint
West, R. K., Sewell, D. K., & Scheibehenne, B. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.04.20.719573
“We also compared these models to a Bayesian observer model (Adler & Ma, 2018; Aitchison et al., 2015; West et al., 2023) that computed a posterior probability ratio using all sample statistics and prior category information.”
dynConfiR: An R package for sequential sampling models of decision confidence
Hellmann, S., Zehetleitner, M., & Rausch, M. (2026) · Behavior Research Methods · doi:10.3758/s13428-026-03013-0 · Cited by 1
“Although response times may be incorporated into thesemodelstoaccountforthedynamicpropertiesofthegen-eration of evidence on which the decisions are based, e.g., by scaling both the magnitude and noise with the response time, they do not provide an explanation for how response times are generated as…”
Suboptimal but intact integration of reward information during perceptual decision-making in autism
Fazioli, L., Hadad, B., Denison, R. N., & Yashar, A. (2026) · Cognition · doi:10.1016/j.cognition.2026.106716
Metadata
- Authors
- Adler, W. T., & Ma, W. J.
- Affiliations
- New York University (US)
- Corresponding author
- Adler, William T.
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2018-11-13
- Volume
- 14
- Issue
- 11
- Pages
- e1006572
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC6258566
- OpenAlex
- W2950975704
- Cited by
- 118
- References
- 110
- Citations by year
- 2026: 5 · 2025: 19 · 2024: 19 · 2023: 23 · 2022: 12 · 2021: 12 · 2020: 14 · 2019: 8 · 2018: 6
- FWCI
- 5.95
- Type
- journal article
- Language
- en
- Update notices
- new_version (2018-11-27)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- National Science Foundation (DGE-1342536)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- confidence reports, Bayesian inference, sensory uncertainty, perceptual categorization, decision-making, heuristics, rational model, metacognition
- MeSH terms
- Adult, Bayes Theorem, Behavior, Decision Making, Female, Humans, Male, Probability, Models, Statistical, Observer Variation, Normal Distribution, Poisson Distribution, Uncertainty, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.