Comparing Bayesian and non-Bayesian accounts of human confidence reports
Open accessPublished version of a preprint
The DOI matches its Crossref record; 107 of the 110 references it cites were found in a registry or an index; 19 of the 38 sources its claims cite were checked for support.
Abstract
Humans can meaningfully report their confidence in a perceptual or cognitive decision. It is widely believed that these reports reflect the Bayesian probability that the decision is correct, but this hypothesis has not been rigorously tested against non-Bayesian alternatives. We use two perceptual categorization tasks in which Bayesian confidence reporting requires subjects to take sensory uncertainty into account in a specific way. We find that subjects do take sensory uncertainty into account when reporting confidence, suggesting that brain areas involved in reporting confidence can access low-level representations of sensory uncertainty, a prerequisite of Bayesian inference. However, behavior is not fully consistent with the Bayesian hypothesis and is better described by simple heuristic models that use uncertainty in a non-Bayesian way. Both conclusions are robust to changes in the uncertainty manipulation, task, response modality, model comparison metric, and additional flexibility in the Bayesian model. Our results suggest that adhering to a rational account of confidence behavior may require incorporating implementational constraints.
Overview
CoChat Summary
## TL;DR
This paper tests whether human confidence reports in perceptual categorization are Bayesian, using two tasks that require subjects to use trial-to-trial sensory uncertainty. The authors find that people do use uncertainty, but their confidence behavior is better fit by simple non-Bayesian heuristic models than by Bayesian models.
## Key findings
- In Experiment 1, the Bayes model beat the Fixed model by a summed LOO difference of 2265 [498, 4253], showing that subjects take trial-to-trial sensory uncertainty into account rather than relying only on a point estimate.
- Adding decision noise improved the Bayesian fit by 804 [510, 1134], but the Bayesian model with decision noise (Bayes-dN) was still outperformed by heuristic models: Lin by 1398 [571, 2644] and Quad by 1167 [858, 2698].
- The heuristic models remained better after multiple robustness checks: with non-parametric reliability–uncertainty mapping, Quad still beat Bayes-dN by 1676 [839, 2730]; with incorrect generative-model assumptions, Quad still beat Bayes-dN by 768 [399, 1144].
- Results were consistent across the paper’s controls and replications: in Task A and Task B separately, Quad beat Bayes-dN by 581 [278, 938] and 978 [406, 1756], respectively; in a separate-response experiment, Lin beat Bayes-dN by 396 [186, 622]; and in a Task-B-only experiment, Quad beat Bayes-dN by 1361 [777, 2022].
## Methods
- The paper ran three experiments on perceptual category judgment and confidence: Experiment 1 used a single button press for combined category-confidence reports; Experiment 2 separated category and confidence responses and added trial-by-trial feedback; Experiment 3 used Task B only.
- Experiment 1 had 11 subjects; Experiment 2 had 8 subjects; Experiment 3 had 15 subjects. Tasks used drifting Gabors or ellipses, with reliability manipulated by contrast or ellipse elongation, and participants were trained extensively before testing.
- The authors fit 75 models from 6 distinct model families using MCMC, compared models mainly with leave-one-out cross-validated log likelihood (LOO), and also checked AIC, BIC, AICc, and WAIC; they performed model recovery and several robustness analyses.
## Limitations
- The paper states that, because of the short presentation time, the results may not generalize to tasks requiring integration of evidence over time.
- The authors state that, because the stimuli are very low-level, they cannot say much about high-level stimuli like faces.
- They also state that they only considered explicit confidence ratings, so the findings may not extend to implicit confidence measures. As evident from the text, the tasks were tightly controlled lab perceptual tasks with trained subjects and specific stimulus manipulations, so broader ecological generalization remains limited.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures












Verification
Claims the paper makes
12 Valid · 2 Not valid · 7 Not checked · 96 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“Recent studies have attempted to test the BCH [19, 20] but, because of their experimental designs, are unable to meaningfully distinguish the Bayesian model from any other model of confidence.”
§ Comparing Bayesian and non-Bayesian accounts of human confidence reports > Introduction
What the sources say
- Disputedcites [19] L Aitchison, 2015, Doubly Bayesian analysis of confidence in perceptual decision-making, PLoS Comput Biol sourceExperimental — accuracy not yet measured
The Introduction says that one-dimensional designs cannot distinguish heuristic from Bayes-optimal confidence, but explicitly describes this study’s multidimensional design as a way to test that distinction. Results > Model selection reports that the two-response data significantly favored the Bayesian model over the Max and Difference models. The paper therefore makes the contrary point about its own experimental design, while acknowledging that it cannot test every possible heuristic model. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
- Partly supportedcites [20] JI Sanders, 2016, Signatures of a statistical computation in the human sense of confidence, Neuron sourceExperimental — accuracy not yet measured
The Discussion reports that the study compared human confidence with predictions from a statistical confidence model, supporting its identification as an attempt to test Bayesian confidence. It also notes that other algorithms can approximate Bayes-optimal predictions, but does not say the experimental design was unable to meaningfully distinguish the Bayesian model from any other confidence model. The cited passages have section labels but no page numbers. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Because we are interested in subjects’ intrinsic computation of confidence, we did not instruct or incentivize them to assign probability ranges to each button (e.g., by using a scoring rule [34–36]).”
§ Comparing Bayesian and non-Bayesian accounts of human confidence reports > Results > Experiment 1 During each session, each subject completed two orientation categorization tasks, Tasks A and B. On each trial, a category C was selected randomly (both categories were equally probable), and a stimulus s was drawn from the corresponding stimulus distribution and displayed. The subject categorized the stimulus and simultaneously reported their confidence on a 4-point scale, with a single button press (Fig 1a). Using a single button press for choice and confidence may prevent post-choice influences on the confidence judgment ([25], but see [26]) and emphasized that confidence should reflect the observer’s perception rather than a preceding motor response. The categories were defined by normal distributions on orientation, which differed by task (Fig 1b). In Task A, the distributions had different means (±μC) and the same standard deviation (σC); leftward-tilting stimuli were more likely to be from category 1. Variants of Task A are common in decision-making studies [27]. In Task B, the distributions had the same mean (0°) and different standard deviations (σ1, σ2); stimuli around the horizontal were more likely to be from category 1. Variants of Task B are less common [28–30] but have some properties of perceptual organization tasks; for example, a subject may have to detect when a stimulus belongs to a narrow category (e.g., in which two line segments are collinear) that is embedded in a a broader category (e.g., in which two line segments are unrelated). Subjects were highly trained on the categories; during training, we only used highest-reliability stimuli, and we provided trial-to-trial category correctness feedback. Subjects were then tested with 6 different reliability levels, which were chosen randomly on each trial. During testing, correctness feedback was withheld to avoid the possibility that confidence simply reflects a learned mapping between stimulus orientation and reliability and the probability of being correct [30–33].
What the sources say
- Supportedcites [34] S Massoni, 2014, Confidence measurement in the light of signal detection theory, Front Psychol source
The citation identifies a scoring rule as an example of a way to incentivize confidence reports; the citing authors’ statement about what they did not do is their own methodological description. In “Elicitation Rules,” the source explicitly identifies and uses a Quadratic Scoring Rule, so it supports the cited example. (LLM, conf=0.91)
- Not checked — no full text of the sourcecites [35] GW Brier, 1950, Verification of forecasts expressed in terms of probability, Monthly Weather Review doi:10.1175/1520-0493(1950)078
- Not checked — no full text of the sourcecites [36] T Gneiting, 2007, Strictly proper scoring rules, prediction, and estimation, J Am Stat Assoc source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“After fitting our core models, our results did not differ substantially from experiment 1: Lin fits better than Bayes-dN by 396 [186, 622] and fits better than Fixed by 2095 [1344, 2889] (S6 Fig and S6 Table).”
§ Comparing Bayesian and non-Bayesian accounts of human confidence reports > Results > Model comparison
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 3 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This feature improves Bayes-dN’s fit by 908 [318, 1661], but it still underperforms Quad by 768 [399, 1144] (S1 Table).”
§ Comparing Bayesian and non-Bayesian accounts of human confidence reports > Results > Model comparison
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This feature improves Bayes-dN’s fit by 1310 [580, 2175], but it still underperforms Quad by 362 [162, 602] (S1 Table).”
§ Comparing Bayesian and non-Bayesian accounts of human confidence reports > Results > Model comparison
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
75 more claims
Create a free account to see all 117 claims and every source verdict.
- It may state two separate claims as one. 14
- Probably not a substantial claim. 26
- Borderline: one of the tests for a substantial claim is unclear. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Other work has revealed how people adjust their confidence criteria in the face of changing stimulus uncertainty (Adler & Ma 2018, Aitchison et al. 2015, Denison et al. 2018).”
From Metacognition and Confidence: A Review and Synthesis (2024) · cites this paper as Adler & Ma 2018
Citations
Cites
Brainard, D. H. (1997) · Spatial Vision · doi:10.1163/156856897x00357 · Cited by 19,702
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
The VideoToolbox software for visual psychophysics: transforming numbers into movies
Pelli, D. G. (1997) · Spatial Vision · doi:10.1163/156856897x00366 · Cited by 11,098
(1966) · Medical Entomology and Zoology · Cited by 10,293
Signal detection theory and psychophysics
Ingleby, J. D. (1967) · Journal of Sound and Vibration · doi:10.1016/0022-460x(67)90197-6 · Cited by 8,509
Strictly Proper Scoring Rules, Prediction, and Estimation
Gneiting, T., & Raftery, A. E. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001437 · Cited by 5,793
(2015) · doi:10.1007/s11222-016-9696-4 · Cited by 5,611 · cited 1×
“−1295 [−2051,−734] −2105 [−3353,−1281] −77 [−1881, 1321] −953 [−2922, 565] −3545 [−5177,−2401] −3799 [−5455,−2521] −361 [−675,−135] −1581 [−2477,−986] −2401 [−3750,−1551] −290 [−2170, 1090] −1172 [−3186, 358] −3549 [−5189,−2402] −4136 [−5848,−2777] 1910 [72, 3813] −847 [−2103, 660] −2361 [−3442,−1517] −3043 [−4489,−19…”
Interplay between Cortical Spreading Depolarization and Seizures
Kramer, D. R., Fujii, T., Ohiorhenuan, I., & Liu, C. Y. (2017) · Stereotactic and Functional Neurosurgery · doi:10.1159/000452841 · Cited by 3,696
An Energy Budget for Signaling in the Grey Matter of the Brain
Attwell, D., & Laughlin, S. B. (2001) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1097/00004647-200110000-00001 · Cited by 3,543 · cited 1×
“−345 [−933, 262] −3497 [−4860,−1817] −2836 [−4544,−1122] −3127 [−4549,−1575] −2449 [−4200,−969] 606 [6, 1269]”
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.2307/2291091 · Cited by 2,312
Understanding predictive information criteria for Bayesian models
Gelman, A., Hwang, J., & Vehtari, A. (2014) · Statistics and Computing · doi:10.1007/s11222-013-9416-2 · Cited by 2,104
Bayesian integration in sensorimotor learning
Körding, K. P., & Wolpert, D. M. (2004) · Nature · doi:10.1038/nature02169 · Cited by 2,004 · cited 1×
“−678 [−1261,−291] 1591 [639, 2624] 525 [161, 945] −439 [−773,−172] −1072 [−1730,−641] 1007 [42, 1957] 2262 [1284, 3399]”
The analysis of visual motion: a comparison of neuronal and psychophysical performance
Britten, K., Shadlen, M., Newsome, W., & Movshon, J. (1992) · The Journal of Neuroscience · doi:10.1523/jneurosci.12-12-04745.1992 · Cited by 1,958
Bayesian inference with probabilistic population codes
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006) · Nature Neuroscience · doi:10.1038/nn1790 · Cited by 1,457
Self-citationPerception as Bayesian Inference
(1996) · doi:10.1017/cbo9780511984037 · Cited by 1,425
Bayesian model selection for group studies
Stephan, K. E., Penny, W. D., Daunizeau, J., Moran, R. J., & Friston, K. J. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2009.03.025 · Cited by 1,423
Fleming, S. M., & Lau, H. C. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00443 · Cited by 1,408
Neal, R. M. (2003) · The Annals of Statistics · doi:10.1214/aos/1056562461 · Cited by 1,361
Slice SamplingPreprint
Neal, R. M. (2000) · arXiv (Cornell University) · Cited by 1,160
The cost of cortical computation.
Lennie, P. (2003) · Current Biology · doi:10.1016/s0960-9822(03)00135-0 · Cited by 1,134
Representation of Confidence Associated with a Decision by Neurons in the Parietal Cortex
Kiani, R., & Shadlen, M. N. (2009) · Science · doi:10.1126/science.1169405 · Cited by 1,027
Maniscalco, B., & Lau, H. (2012) · Consciousness and Cognition · doi:10.1016/j.concog.2011.09.021 · Cited by 883
S‐potentials from colour units in the retina of fish (Cyprinidae)
Naka, K. I., & Rushton, W. A. H. (1966) · The Journal of Physiology · doi:10.1113/jphysiol.1966.sp008001 · Cited by 825
Neural correlates, computation and behavioural impact of decision confidence
Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008) · Nature · doi:10.1038/nature07200 · Cited by 809
Relating Introspective Accuracy to Individual Differences in Brain Structure
Fleming, S. M., Weil, R. S., Nagy, Z., Dolan, R. J., & Rees, G. (2010) · Science · doi:10.1126/science.1191883 · Cited by 809 · cited 1×
“−533 [−920,−283] 1184 [23, 2129] 846 [−609, 2092] −2303 [−3578,−921] −1670 [−3268,−39]”
The neural basis of metacognitive ability
Fleming, S. M., & Dolan, R. J. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2011.0417 · Cited by 747
Bahrami, B., Olsen, K., Latham, P. E., Roepstorff, A., Rees, G., & Frith, C. D. (2010) · Science · doi:10.1126/science.1185718 · Cited by 738
Two-stage dynamic signal detection: A theory of choice, decision time, and confidence.
Pleskac, T. J., & Busemeyer, J. R. (2010) · Psychological Review · doi:10.1037/a0019737 · Cited by 722
Bayesian model selection for group studies — Revisited
Rigoux, L., Stephan, K. E., Friston, K. J., & Daunizeau, J. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.08.065 · Cited by 666
S‐potentials from luminosity units in the retina of fish (Cyprinidae)
Naka, K. I., & Rushton, W. A. H. (1966) · The Journal of Physiology · doi:10.1113/jphysiol.1966.sp008003 · Cited by 631
Cardinal rules: visual orientation perception reflects knowledge of environmental statistics
Girshick, A. R., Landy, M. S., & Simoncelli, E. P. (2011) · Nature Neuroscience · doi:10.1038/nn.2831 · Cited by 613
Jones, M., & Love, B. C. (2011) · Behavioral and Brain Sciences · doi:10.1017/s0140525x10003134 · Cited by 533
Confidence and certainty: Distinct probabilistic quantities for different goals
Pouget, A., Drugowitsch, J., & Kepecs, A. (2016) · Nature Neuroscience · doi:10.1038/nn.4240 · Cited by 528
Relative blindsight in normal observers and the neural correlate of visual consciousness
Lau, H. C., & Passingham, R. E. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0607716103 · Cited by 518
A review of the tip-of-the-tongue experience
Brown, A. S. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.109.2.204 · Cited by 517
Decision Processes in Visual Perception
(1979) · Elsevier eBooks · doi:10.1016/c2013-0-11654-6 · Cited by 502
Choice Certainty Is Informed by Both Evidence and Decision Time
Kiani, R., Corthell, L., & Shadlen, M. N. (2014) · Neuron · doi:10.1016/j.neuron.2014.12.015 · Cited by 470
The evolutionary origins of modularity
Clune, J., Mouret, J.-B., & Lipson, H. (2013) · Proceedings of the Royal Society B: Biological Sciences · doi:10.1098/rspb.2012.2863 · Cited by 466
Gigerenzer, G., Hertwig, R., & Pachur, T. (2011) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199744282.001.0001 · Cited by 455
Post-decision wagering objectively measures awareness
Persaud, N., McLeod, P., & Cowey, A. (2007) · Nature Neuroscience · doi:10.1038/nn1840 · Cited by 454
The Importance of Falsification in Computational Cognitive Modeling
Palminteri, S., Wyart, V., & Koechlin, E. (2017) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.03.011 · Cited by 448
Bayesian just-so stories in psychology and neuroscience.
Bowers, J. S., & Davis, C. J. (2012) · Psychological Bulletin · doi:10.1037/a0026450 · Cited by 436
Heuristics: The Foundations of Adaptive Behavior
Gigerenzer, G., Hertwig, R., & Pachur, T. (2015) · Cited by 424
Rounis, E., Maniscalco, B., Rothwell, J. C., Passingham, R. E., & Lau, H. (2010) · Cognitive Neuroscience · doi:10.1080/17588921003632529 · Cited by 407
Prefrontal Contributions to Metacognition in Perceptual Decision Making
Fleming, S. M., Huijgen, J., & Dolan, R. J. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.6489-11.2012 · Cited by 391
MAPS IN THE BRAIN: What Can We Learn from Them?
Chklovskii, D. B., & Koulakov, A. A. (2004) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144226 · Cited by 366
On small differences in sensation
Peirce, C. S., & Jastrow, J. (1884) · PhilPapers (PhilPapers Foundation) · Cited by 362
Confidence as Bayesian probability: From neural origins to behavior
Meyniel, F., Sigman, M., & Mainen, Z. F. (2015) · Neuron · doi:10.1016/j.neuron.2015.09.039 · Cited by 349
Not Noisy, Just Wrong: The Role of Suboptimal Inference in Behavioral Variability
Beck, J. M., Ma, W. J., Pitkow, X., Latham, P. E., & Pouget, A. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.016 · Cited by 316
Self-citationDecision theory: what “should” the nervous system do?
Körding, K. (2007) · Science · doi:10.1126/science.1142998 · Cited by 313
Factorial comparison of working memory models.
van den Berg, R., Awh, E., & Ma, W. J. (2014) · Psychological Review · doi:10.1037/a0035234 · Cited by 311
Self-citationA computational framework for the study of confidence in humans and animals
Kepecs, A., & Mainen, Z. F. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2012.0037 · Cited by 305 · cited 1×
“2041 [385, 3623] 1188 [−588, 2704] −1398 [−2644,−571] −1667 [−2698,−858] 1763 [981, 2774] 535 [131, 1006] −306 [−505,−120] 1823 [84, 3478] 966 [−876, 2526] −1401 [−2652,−573] −1996 [−3163,−1026] 4073 [2379, 5781] 1296 [34, 2933] −234 [−1088, 650] −908 [−1661,−318] 1370 [−100, 2776] 298 [−645, 1325] −667 [−1425, 36] −1…”
The construction of confidence in a perceptual decision
Zylberberg, A., Barttfeld, P., & Sigman, M. (2012) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2012.00079 · Cited by 297
Neural Coding of Uncertainty and Probability
Ma, W. J., & Jazayeri, M. (2014) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-071013-014017 · Cited by 290
Self-citationA common mechanism underlies changes of mind about decisions and confidence
van den Berg, R., Anandalingam, K., Zylberberg, A., Kiani, R., Shadlen, M. N., & Wolpert, D. M. (2016) · eLife · doi:10.7554/elife.12192 · Cited by 270
Dissociable prior influences of signal probability and relevance on visual contrast sensitivity
Wyart, V., Nobre, A. C., & Summerfield, C. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1120118109 · Cited by 268
Schmidts, M., Hou, Y., Cortés, C. R., Mans, D. A., Huber, C., Boldt, K., Patel, M., van Reeuwijk, J., Plaza, J.-M., van Beersum, S. E. C., et al. (2016) · Nature Communications · doi:10.1038/ncomms11270 · Cited by 260
Signatures of a Statistical Computation in the Human Sense of Confidence
Sanders, J. I., Hangya, B., & Kepecs, A. (2016) · Neuron · doi:10.1016/j.neuron.2016.03.025 · Cited by 253
Perceptual confidence neglects decision-incongruent evidence in the brain
Peters, M. A. K., Thesen, T., Ko, Y. D., Maniscalco, B., Carlson, C., Davidson, M., Doyle, W., Kuzniecky, R., Devinsky, O., Halgren, E., et al. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0139 · Cited by 252
Attention induces conservative subjective biases in visual perception
Rahnev, D., Maniscalco, B., Graves, T., Huang, E., de Lange, F. P., & Lau, H. (2011) · Nature Neuroscience · doi:10.1038/nn.2948 · Cited by 241
Responses of pulvinar neurons reflect a subject's confidence in visual categorization
Komura, Y., Nikkuni, A., Hirashima, N., Uetake, T., & Miyamoto, A. (2013) · Nature Neuroscience · doi:10.1038/nn.3393 · Cited by 226
Dissociation Between Memory Accuracy and Memory Confidence Following Bilateral Parietal Lesions
Simons, J. S., Peers, P. V., Mazuz, Y. S., Berryhill, M. E., & Olson, I. R. (2010) · Cerebral Cortex · doi:10.1093/cercor/bhp116 · Cited by 222
Spatial attention increases performance but not subjective confidence in a discrimination task
Wilimzig, C., Tsuchiya, N., Fahle, M., Einhäuser, W., & Koch, C. (2008) · Journal of Vision · doi:10.1167/8.5.7 · Cited by 196
Attentional Enhancement via Selection and Pooling of Early Sensory Responses in Human Visual Cortex
Pestilli, F., Carrasco, M., Heeger, D. J., & Gardner, J. L. (2011) · Neuron · doi:10.1016/j.neuron.2011.09.025 · Cited by 190
Effects of cortical microstimulation on confidence in a perceptual decision
Fetsch, C. R., Kiani, R., Newsome, W. T., & Shadlen, M. N. (2014) · Neuron · doi:10.1016/j.neuron.2014.07.011 · Cited by 185
Explicit representation of confidence informs future value-based decisions
Folke, T., Jacobsen, C., Fleming, S. M., & Martino, B. D. (2016) · Nature Human Behaviour · doi:10.1038/s41562-016-0002 · Cited by 183
Bayesian decision theory as a model of human visual perception: Testing Bayesian transfer
MALONEY, L. T., & MAMASSIAN, P. (2009) · Visual Neuroscience · doi:10.1017/s0952523808080905 · Cited by 180
Organizing probabilistic models of perception
Ma, W. J. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.08.010 · Cited by 171
Self-citationObject classification for human and ideal observers
Liu, Z., Knill, D. C., & Kersten, D. (1995) · Vision Research · doi:10.1016/0042-6989(94)00150-k · Cited by 164
Internal Representations of Temporal Statistics and Feedback Calibrate Motor-Sensory Interval Timing
Acerbi, L., Wolpert, D. M., & Vijayakumar, S. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002771 · Cited by 162
Impairing the useful field of view in natural scenes: Tunnel vision versus general interference
Ringer, R. V., Throneburg, Z., Johnson, A. P., Kramer, A. F., & Loschky, L. C. (2016) · Journal of Vision · doi:10.1167/16.2.7 · Cited by 159
Multivoxel neurofeedback selectively modulates confidence without changing perceptual performance
Cortese, A., Amano, K., Koizumi, A., Kawato, M., & Lau, H. (2016) · Nature Communications · doi:10.1038/ncomms13669 · Cited by 157
Direct injection of noise to the visual cortex decreases accuracy but increases decision confidence
Rahnev, D. A., Maniscalco, B., Luber, B., Lau, H., & Lisanby, S. H. (2012) · Journal of Neurophysiology · doi:10.1152/jn.00985.2011 · Cited by 156
Representation of retrieval confidence by single neurons in the human medial temporal lobe
Rutishauser, U., Ye, S., Koroma, M., Tudusciuc, O., Ross, I. B., Chung, J. M., & Mamelak, A. N. (2015) · Nature Neuroscience · doi:10.1038/nn.4041 · Cited by 147
On the Origins of Suboptimality in Human Probabilistic Inference
Acerbi, L., Vijayakumar, S., & Wolpert, D. M. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003661 · Cited by 139
Doubly Bayesian Analysis of Confidence in Perceptual Decision-Making
Aitchison, L., Bang, D., Bahrami, B., & Latham, P. E. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004519 · Cited by 131
Maniscalco, B., Peters, M. A. K., & Lau, H. (2016) · Attention, Perception, & Psychophysics · doi:10.3758/s13414-016-1059-x · Cited by 129
Humans incorporate attention-dependent uncertainty into perceptual decisions and confidence
Denison, R. N., Adler, W. T., Carrasco, M., & Ma, W. J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1717720115 · Cited by 122
Self-citationWhat failure in collective decision-making tells us about metacognition
Bahrami, B., Olsen, K., Bang, D., Roepstorff, A., Rees, G., & Frith, C. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2011.0420 · Cited by 118
The idiosyncratic nature of confidence
Navajas, J., Hindocha, C., Foda, H., Keramati, M., Latham, P. E., & Bahrami, B. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0215-1 · Cited by 117
Does perceptual confidence facilitate cognitive control?
Koizumi, A., Maniscalco, B., & Lau, H. C. (2015) · Attention Perception & Psychophysics · doi:10.3758/s13414-015-0843-3 · Cited by 115
A Mathematical Framework for Statistical Decision Confidence
Hangya, B., Sanders, J. I., & Kepecs, A. (2016) · Neural Computation · doi:10.1162/neco_a_00864 · Cited by 113
Acerbi, L., Dokka, K., Angelaki, D. E., & Ma, W. J. (2018) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006110 · Cited by 111
Self-citationBang, D., Fusaroli, R., Tylén, K., Olsen, K., Latham, P. E., Lau, J. Y. F., Roepstorff, A., Rees, G., Frith, C. D., & Bahrami, B. (2014) · Consciousness and Cognition · doi:10.1016/j.concog.2014.02.002 · Cited by 107
Samaha, J., Barrett, J. J., Sheldon, A. D., LaRocque, J. J., & Postle, B. R. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00851 · Cited by 99
Uncovering mental representations with Markov chain Monte Carlo
Sanborn, A. N., Griffiths, T. L., & Shiffrin, R. M. (2009) · Cognitive Psychology · doi:10.1016/j.cogpsych.2009.07.001 · Cited by 96
Post-decisional accounts of biases in confidence
Navajas, J., Bahrami, B., & Latham, P. E. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.05.005 · Cited by 96
Evaluation of Objective Uncertainty in the Visual System
Barthelmé, S., & Mamassian, P. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000504 · Cited by 90
Flexible mechanisms underlie the evaluation of visual confidence
Barthelmé, S., & Mamassian, P. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1007704107 · Cited by 82
Efficient probabilistic inference in generic neural networks trained with non-probabilistic feedback
Orhan, A. E., & Ma, W. J. (2017) · Nature Communications · doi:10.1038/s41467-017-00181-8 · Cited by 77
Self-citationBressler, D. W., & Silver, M. A. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.06.063 · Cited by 75
Relation between belief and performance in perceptual decision making
Drugowitsch, J., Moreno-Bote, R., & Pouget, A. (2014) · PLoS ONE · doi:10.1371/journal.pone.0096511 · Cited by 72
Probabilistic Computation in Human Perception under Variability in Encoding Precision
Keshvari, S., van den Berg, R., & Ma, W. J. (2012) · PLoS ONE · doi:10.1371/journal.pone.0040216 · Cited by 64
Self-citationTrial-to-trial, uncertainty-based adjustment of decision boundaries in visual categorization
Qamar, A. T., Cotton, R. J., George, R. G., Beck, J. M., Prezhdo, E., Laudano, A., Tolias, A. S., & Ma, W. J. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1219756110 · Cited by 64
Self-citationA decisional account of subjective inflation of visual perception at the periphery
Solovey, G., Graney, G. G., & Lau, H. (2015) · Attention, Perception, & Psychophysics · doi:10.3758/s13414-014-0769-1 · Cited by 63
Erratum to: Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
Vehtari, A., Gelman, A., & Gabry, J. (2017) · Statistics and Computing · doi:10.1007/s11222-016-9709-3 · Cited by 62
Signal detection theory, uncertainty, and Poisson-like population codes
Ma, W. J. (2010) · Vision Research · doi:10.1016/j.visres.2010.08.035 · Cited by 60
Self-citationConfidence measurement in the light of signal detection theory
Massoni, S., Gajdos, T., & Vergnaud, J.-C. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01455 · Cited by 58
Visibility Is Not Equivalent to Confidence in a Low Contrast Orientation Discrimination Task
Rausch, M., & Zehetleitner, M. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00591 · Cited by 57
Limitations of Proposed Signatures of Bayesian Confidence
Adler, W. T., & Ma, W. J. (2018) · Neural Computation · doi:10.1162/neco_a_01141 · Cited by 45
Self-citationImplicit and explicit processes in category-based induction: Is induction best when we don’t think?
Chen, S. Y., Ross, B. H., & Murphy, G. L. (2013) · Journal of Experimental Psychology General · doi:10.1037/a0032064 · Cited by 33
Limitations of proposed signatures of Bayesian confidencePreprint
Adler, W. T., & Ma, W. J. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/218222 · Cited by 16
Self-citationAcerbi, L., Dokka, K., Angelaki, D. E., et al. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/150052 · Cited by 13
Comparing models of contrast gain using psychophysical experiments
DiMattina, C. (2016) · Journal of Vision · doi:10.1167/16.9.1 · Cited by 12
The idiosyncratic nature of confidencePreprint
Navajas, J., Hindocha, C., Foda, H., Keramati, M., Latham, P. E., & Bahrami, B. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/102269 · Cited by 11
VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY
BRIER, G. W. (1950) · Monthly Weather Review · doi:10.1175/1520-0493(1950)078<0001:vofeit>2.0.co;2
Verification of forecasts expressed in terms of probability
BRIER, G. W. (1950) · Monthly Weather Review · doi:10.1175/1520-0493(1950)078%3c0001:vofeit%3e2.0.co;2
Research works printed but not linked: 1
- Orhan AE, Jacobs RA. Are performance limitations in visual short-term memory tasks due to capacity limitations of model mismatch? arXiv. 2014 Jul;.
Cited by
Suboptimality in perceptual decision making
Rahnev, D., & Denison, R. N. (2018) · Behavioral and Brain Sciences · doi:10.1017/s0140525x18000936 · Cited by 342
Metacognition and Confidence: A Review and Synthesis
Fleming, S. M. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-022423-032425 · Cited by 256
“Other work has revealed how people adjust their confidence criteria in the face of changing stimulus uncertainty (Aitchison et al., 2015, Adler & Ma, 2018, Denison et al., 2018).”
Sources of Metacognitive Inefficiency
Shekhar, M., & Rahnev, D. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.10.007 · Cited by 119
“Of course, it is possible that some of them are also due to processing or resource limitations, especially when computing the probability of being correct is complex or resource demanding [75,77].”
BackgroundHumans incorporate attention-dependent uncertainty into perceptual decisions and confidence
Denison, R. N., Adler, W. T., Carrasco, M., & Ma, W. J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1717720115 · Cited by 122
“In the Bayesian model, decisions depend on the relative posterior probabilities of the two categories, leading the observer to shift his or her decision boundaries in measurement space, based on the attention condition (4, 5) (Fig.”
BackgroundMethodResultInfluentialSelf-citationAcerbi, L., Dokka, K., Angelaki, D. E., & Ma, W. J. (2018) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1006110 · Cited by 111
Self-citation
Metadata
- Authors
- Adler, W. T., & Ma, W. J.
- Affiliations
- New York University (US)
- Corresponding author
- Adler, William T.
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2018-11-13
- Volume
- 14
- Issue
- 11
- Pages
- e1006572
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC6258566
- OpenAlex
- W2950975704
- Cited by
- 118
- References
- 110
- Citations by year
- 2026: 5 · 2025: 19 · 2024: 19 · 2023: 23 · 2022: 12 · 2021: 12 · 2020: 14 · 2019: 8 · 2018: 6
- FWCI
- 5.95
- Type
- journal article
- Language
- en
- Update notices
- new_version (2018-11-27)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- National Science Foundation (DGE-1342536)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- confidence reports, Bayesian inference, sensory uncertainty, perceptual categorization, decision-making, heuristics, rational model, metacognition
- MeSH terms
- Adult, Bayes Theorem, Behavior, Decision Making, Female, Humans, Male, Probability, Models, Statistical, Observer Variation, Normal Distribution, Poisson Distribution, Uncertainty, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.