CoChat
Sign Up
Cited by 131Visual perception and processing mechanismsDecision-Making and Behavioral EconomicsNeural and Behavioral Psychology Studies

Doubly Bayesian Analysis of Confidence in Perceptual Decision-Making

Open access
Aitchison, Laurence, Bang, Dan, Bahrami, Bahador, Latham, Peter E.·2015·PLOS Computational Biology
DOI 10.1371/journal.pcbi.1004519PMC PMC4627723
Open this paper in CoChat — free accountOpen PDF

Abstract

Humans stand out from other animals in that they are able to explicitly report on the reliability of their internal operations. This ability, which is known as metacognition, is typically studied by asking people to report their confidence in the correctness of some decision. However, the computations underlying confidence reports remain unclear. In this paper, we present a fully Bayesian method for directly comparing models of confidence. Using a visual two-interval forced-choice task, we tested whether confidence reports reflect heuristic computations (e.g. the magnitude of sensory data) or Bayes optimal ones (i.e. how likely a decision is to be correct given the sensory data). In a standard design in which subjects were first asked to make a decision, and only then gave their confidence, subjects were mostly Bayes optimal. In contrast, in a less-commonly used design in which subjects indicated their confidence and decision simultaneously, they were roughly equally likely to use the Bayes optimal strategy or to use a heuristic but suboptimal strategy. Our results suggest that, while people's confidence reports can reflect Bayes optimal computations, even a small unusual twist or additional element of complexity can prevent optimality.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pcbi.1004519

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Recent studies attempted to test the BCH but, because of their experimental designs, are unable to meaningfully distinguish the Bayesian model from any other model of confidence.

    “Recent studies have attempted to test the BCH [19, 20] but, because of their experimental designs, are unable to meaningfully distinguish the Bayesian model from any other model of confidence.”

    From Comparing Bayesian and non-Bayesian accounts of human confidence reports (2018) · cites this paper as [19]

    The Introduction says that one-dimensional designs cannot distinguish heuristic from Bayes-optimal confidence, but explicitly describes this study’s multidimensional design as a way to test that distinction. Results > Model selection reports that the two-response data significantly favored the Bayesian model over the Max and Difference models. The paper therefore makes the contrary point about its own experimental design, while acknowledging that it cannot test every possible heuristic model. (LLM, conf=0.90)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • People adjust their confidence criteria in the face of changing stimulus uncertainty.

    “Other work has revealed how people adjust their confidence criteria in the face of changing stimulus uncertainty (Adler & Ma 2018, Aitchison et al. 2015, Denison et al. 2018).”

    From Metacognition and Confidence: A Review and Synthesis (2024) · cites this paper as Aitchison et al. 2015

    The Results (‘Differences between models’; page unavailable) examine confidence reports at four stimulus-contrast levels, and the paper reports that subjects’ confidence often reflected a Bayesian computation. But the cited paper does not show that people adjust their confidence criteria when stimulus uncertainty changes; its Methods describe confidence reports using fitted thresholds, not changes in those thresholds across uncertainty conditions. (LLM, conf=0.79)

    Experimental — accuracy not yet measured

  • Aitchison et al. [19] found that confidence appears more Bayesian when subjects use separate responses.

    “Aitchison et al. [19] found that confidence appears more Bayesian when subjects use separate responses.”

    From Comparing Bayesian and non-Bayesian accounts of human confidence reports (2018) · cites this paper as [19]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 31

  1. [1]Kepecs A, Uchida N, Zariwala HA, Mainen ZF. Neural correlates, computation and behavioral impact of decision confidence. Nature. 2008; 455 (7210). 10.1038/nature07200 18690210 · DOI · Paper page
  2. [3]Komura Y, Nikkuni A, Hirashima N, Uetake T, Miyamoto A. Responses of pulvinar neurons reflect a subject’s confidence in visual categorization. Nature Neuroscience. 2013; 16(6). 10.1038/nn.3393 23666179 · DOI · Paper page
  3. [4]Bahrami B, Olsen K, Latham PE, Roepstorff A, Rees G, Frith CD. Optimally interacting minds. Science. 2010; 329 (5995). 10.1126/science.1185718 20798320 PMC3371582 · DOI · Paper page
  4. [5]Fusaroli R, Bahrami B, Olsen K, Roepstorff A, Rees G, Frith C, et al Coming to terms: quantifying the benefits of linguistic coordination. Psychological Science. 2012; 23(8). 10.1177/0956797612436816 22810169 · DOI
  5. [6]Shea N, Boldt A, Bang D, Yeung N, Heyes C, Frith C. Supra-personal cognitive control and metacognition. Trends in Cognitive Sciences. 2014; 18(4). 10.1016/j.tics.2014.01.006 24582436 PMC3989995 · DOI · Paper page
  6. [7]Fleming SM, Dolan RJ, Frith CD. Metacognition: computation, biology and function. Philosophical Transactions of the Royal Society B: Biological Sciences. 2012; 367 (1594). 10.1098/rstb.2012.0021 PMC3318771 22492746 · DOI · Paper page
  7. [8]Broihanne MH, Merli M, Roger P. Overconfidence, risk perception and the risk-taking behavior of finance professionals. Finance Research Letters. 2014; 11(2). 10.1016/j.frl.2013.11.002 · DOI
  8. [9]Berner ES, Graber ML. Overconfidence as a cause of diagnostic error in medicine. The American Journal of Medicine. 2008; 121(5). 10.1016/j.amjmed.2008.01.001 18440350 · DOI · Paper page
  9. [10]Tenney ER, MacCoun RJ, Spellman BA, Hastie R. Calibration trumps confidence as a basis for witness credibility. Psychological Science. 2007; 18(1). 10.1111/j.1467-9280.2007.01847.x 17362377 · DOI
  10. [13]Green DM, Swets JA. Signal Detection Theory and Psychophysics. New York: Wiley; 1966.
  11. [14]Galvin S, Podd J, Drga V, Whitmore J. Type 2 tasks in the theory of signal detectability: discrimination between correct and incorrect decisions. Psychonomic Bulletin and Review. 2003; 10(4). 10.3758/BF03196546 15000533 · DOI · Paper page
  12. [16]Maniscalco B, Lau H. A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings. Consciousness and Cognition. 2012; 21 10.1016/j.concog.2011.09.021 22071269 · DOI · Paper page
  13. [18]Vickers D. Decision Processes in Visual Perception. London: Academic Press; 1979.
  14. [19]de Martino B, Fleming SM, Garrett N, Dolan RJ. Confidence in value-based choice. Nature Neuroscience. 2013; 16(1). 10.1038/nn.3279 23222911 PMC3786394 · DOI · Paper page
  15. [22]Lavie N. Distracted and confused: selective attention under load. Trends in Cognitive Sciences. 2005; (2). 15668100
  16. [23]Kahneman D. Attention and Effort. Englewood Cliffs, New Jersey: Prentice Hall; 1973.
  17. [24]Pashler H. Dual-task interference in simple tasks: data and theory. Psychological Bulletin. 1994; (2). 7972591
  18. [25]Huszár F, Noppeney U, Lengyel M. Mind reading by machine learning: A doubly Bayesian method for inferring mental representations. In: Proceedings of the Thirty-Second Annual Conference of the Cognitive Science Society; 2010. p. 2810–2815.
  19. [26]Stephan KE, Penny WD, Daunizeau J, Moran RJ, Friston KJ. Bayesian model selection for group studies. Neuroimage. 2009; (4). PMC2703732 19306932
  20. [27]Barthelmé S, Mamassian P. Evaluation of objective Uncertainty in the visual system. PLoS Computational Biology. 2009; 5(9). 10.1371/journal.pcbi.1000504 19750003 PMC2730538 · DOI
  21. [28]Barthelmé S, Mamassian P. Flexible mechanisms underlie the evaluation of visual confidence. Proceedings of the National Academy of Sciences. 2010; 107(48). PMC2996459 21076036
  22. [29]de Gardelle V, Mamassian P. Does confidence use a common currency across two visual tasks? Psychological Science. 2014; 25(6). 10.1177/0956797614528956 24699845 · DOI · Paper page
  23. [30]Harvey N. Confidence in judgement. Trends in Cognitive Sciences. 1997; 1(2). 10.1016/S1364-6613(97)01014-0 21223868 · DOI · Paper page
  24. [31]Sniezek JA, Henry RA. Accuracy and confidence in group judgment. Organizational Behavior and Human Decision Processes. 1989; 43(1). 10.1016/0749-5978(89)90055-1 · DOI
  25. [32]Bang D, Fusaroli R, Tylén K, Olsen K, Latham PE, Lau JYF, et al Does interaction matter? Testing whether a confidence heuristic can replace interaction in collective decision-making. Consciousness and Cognition. 2014; 26(1). 10.1016/j.concog.2014.02.002 24650632 PMC4029078 · DOI
  26. [33]Mahmoodi A, Bang D, Ahmadabadi MN, Bahrami B. Learning to make collective decisions: the impact of confidence escalation. PloS One. 2013; 8(12). 10.1371/journal.pone.0081195 24324677 PMC3855698 · DOI
  27. [34]Fleming SM, Dolan RJ. Effects of loss aversion on post-decision wagering: implications for measures of awareness. Consciousness and Cognition. 2010; 19(1). 10.1016/j.concog.2009.11.002 20005133 PMC2842936 · DOI
  28. [35]Barber BM, Odean T. Boys Will be boys: gender, overconfidence, and common stock investment. The Quarterly Journal of Economics. 2001; 116(1). 10.1162/003355301556400 · DOI · Paper page
  29. [36]Mann L, Radford M, Burnett P, Ford S, Bond M, Leung K, et al Cross-cultural differences in self-reported decision-making style and confidence. International Journal of Psychology. 1998; 33(5). 10.1080/002075998400213 · DOI
  30. [38]Moore DA, Healy PJ. The trouble with overconfidence. Psychological Review. 2008; (2). 18426301
  31. [39]Zylberberg AD, Roelfsema PR, Sigman M. Variance misperception explains illusions of confidence in simple perceptual decisions. Consciousness & Cognition. 2014; 27(1). 24951943

Cited by

Sorted by citations

Authors
Aitchison, L., Bang, D., Bahrami, B., & Latham, P. E.
Affiliations
  • Gatsby Computational Neuroscience Unit (GB)
  • Aarhus University (DK)
Corresponding author
Aitchison, Laurence
ORCID
Venue
PLOS Computational Biology
Publisher
Public Library of Science (PLoS)
Published
2015-10-30
Volume
11
Issue
10
Pages
e1004519
ISSN
  • 1553-734X (ISSN-L)
  • 1553-7358
DOI
10.1371/journal.pcbi.1004519
PMC
PMC4627723
OpenAlex
W1923426392
Cited by
131
References
8
Citations by year
2026: 9 · 2025: 13 · 2024: 8 · 2023: 21 · 2022: 11 · 2021: 20 · 2020: 13 · 2019: 10 · 2018: 7 · 2017: 9 · 2016: 10
FWCI
5.21
Type
journal article
Language
en
Open access
Licence
Locations
  • PLoS Computational Biology (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • UCL Discovery (University College London) (submittedVersion)
  • Figshare (submittedVersion, cc-by)
  • Scopus (Elsevier) (submittedVersion, other-oa)
  • Bristol Research (University of Bristol) (submittedVersion)
Funders
  • Gatsby Charitable Foundation
  • FP7 Ideas: European Research Council (309865)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
  • Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
Keywords
metacognition, confidence reports, perceptual decision making, model comparison
MeSH terms
Heuristics, Bayes Theorem, Choice Behavior, Computer Simulation, Decision Support Techniques, Humans, Visual Perception, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review