A Mathematical Framework for Statistical Decision Confidence
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Decision confidence is a forecast about the probability that a decision will be correct. From a statistical perspective, decision confidence can be defined as the Bayesian posterior probability that the chosen option is correct based on the evidence contributing to it. Here, we used this formal definition as a starting point to develop a normative statistical framework for decision confidence. Our goal was to make general predictions that do not depend on the structure of the noise or a specific algorithm for estimating confidence. We analytically proved several interrelations between statistical decision confidence and observable decision measures, such as evidence discriminability, choice, and accuracy. These interrelationships specify necessary signatures of decision confidence in terms of externally quantifiable variables that can be empirically tested. Our results lay the foundations for a mathematically rigorous treatment of decision confidence that can lead to a common framework for understanding confidence across different research domains, from human and animal behavior to neural representations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Merging the senses into a robust percept
Ernst, M. O., & Bülthoff, H. H. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.02.002 · Cited by 1,906 · cited 1×
“Bayesian theories have been used to account for how brains can perform sensory processing (Beck et al., 2008; Ernst and Bülthoff, 2004; Graf et al., 2005) and motor control (Körding and Wolpert, 2004, 2006).”
MethodBayesian inference with probabilistic population codes
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006) · Nature Neuroscience · doi:10.1038/nn1790 · Cited by 1,457
Representation of Confidence Associated with a Decision by Neurons in the Parietal Cortex
Kiani, R., & Shadlen, M. N. (2009) · Science · doi:10.1126/science.1169405 · Cited by 1,027 · cited 1×
“Perhaps the most intuitive extension is within the race model framework, where the difference between decision variables for the winning and losing races provides an estimate of confidence (Kepecs et al., 2008; Merkle and Van Zandt, 2006; Moreno-Bote, 2010; Vickers, 1979; Zylberberg et al., 2012).”
BackgroundA comparison of current measures of the accuracy of feeling-of-knowing predictions
Nelson, T. O. (1984) · Psychological Bulletin · doi:10.1037/0033-2909.95.1.109 · Cited by 976 · cited 1×
“…confidence could be computed, signal detection theory has also been used as a starting point for evaluating the metacognitive sensitivity of human confidence reports (Ferrell and McGoey, 1980; Higham and Arnold, 2007; Higham et al., 2009; Kunimoto et al., 2001; Lachman et al., 1979; Nelson, 1984).”
BackgroundManiscalco, B., & Lau, H. (2012) · Consciousness and Cognition · doi:10.1016/j.concog.2011.09.021 · Cited by 883
Neural correlates, computation and behavioural impact of decision confidence
Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008) · Nature · doi:10.1038/nature07200 · Cited by 809 · cited 6×
“Perhaps the most intuitive extension is within the race model framework, where the difference between decision variables for the winning and losing races provides an estimate of confidence (Vickers, 1979; Kepecs et al., 2008; Moreno-Bote, 2010; Pleskac & Busemeyer, 2010; Zylberberg, Barttfeld, & Sigman, 2012; Drugowit…”
BackgroundMethodInfluentialTwo-stage dynamic signal detection: A theory of choice, decision time, and confidence.
Pleskac, T. J., & Busemeyer, J. R. (2010) · Psychological Review · doi:10.1037/a0019737 · Cited by 722
Principles of Statistical Inference
Cox, D. R. (2006) · doi:10.1017/cbo9780511813559 · Cited by 670
Probabilistic population codes for Bayesian decision making
Beck, J. M., Ma, W. J., Kiani, R., Hanks, T., Churchland, A. K., Roitman, J., Shadlen, M. N., Latham, P. E., & Pouget, A. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.021 · Cited by 645
Confidence and certainty: Distinct probabilistic quantities for different goals
Pouget, A., Drugowitsch, J., & Kepecs, A. (2016) · Nature Neuroscience · doi:10.1038/nn.4240 · Cited by 528
Cited by
Confidence and certainty: Distinct probabilistic quantities for different goals
Pouget, A., Drugowitsch, J., & Kepecs, A. (2016) · Nature Neuroscience · doi:10.1038/nn.4240 · Cited by 528
Signatures of a Statistical Computation in the Human Sense of Confidence
Sanders, J. I., Hangya, B., & Kepecs, A. (2016) · Neuron · doi:10.1016/j.neuron.2016.03.025 · Cited by 253
Self-citationStriatal dopamine mediates hallucination-like perception in mice
Schmack, K., Bosc, M., Ott, T., Sturgill, J. F., & Kepecs, A. (2021) · Science · doi:10.1126/science.abf4740 · Cited by 253
Self-citationMidbrain dopamine neurons signal belief in choice accuracy during a perceptual decision
Lak, A., Nomoto, K., Keramati, M., Sakagami, M., & Kepecs, A. (2017) · Current Biology · doi:10.1016/j.cub.2017.02.026 · Cited by 170
Comparing Bayesian and non-Bayesian accounts of human confidence reports
Adler, W. T., & Ma, W. J. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006572 · Cited by 118
Behavior- and Modality-General Representation of Confidence in Orbitofrontal Cortex
Masset, P., Ott, T., Lak, A., Hirokawa, J., & Kepecs, A. (2020) · Cell · doi:10.1016/j.cell.2020.05.022 · Cited by 109
“Our core observations that OFC neurons represent decision confidence and predict confidence reports fit into a statistical framework of decision confidence (Hangya et al., 2016; Ott et al., 2018).”
BackgroundMethodInfluentialSubjective confidence reflects representation of Bayesian probability in cortex
Geurts, L. S., Cooke, J. R. H., van Bergen, R. S., & Jehee, J. F. M. (2022) · Nature Human Behaviour · doi:10.1038/s41562-021-01247-w · Cited by 104
“This revealed 13 a significant effect of confidence on BOLD activity in all three regions (dAI: F(3,93) = 25.”
Sensory noise increases metacognitive efficiency
Bang, J. W., Shekhar, M., & Rahnev, D. (2019) · Journal of Experimental Psychology: General · doi:10.1037/xge0000511 · Cited by 94
Confidence reflects a noisy decision reliability estimate
Boundy-Singer, Z. M., Ziemba, C. M., & Goris, R. L. T. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01464-x · Cited by 71
Khalvati, K., Kiani, R., & Rao, R. P. N. (2021) · Nature Communications · doi:10.1038/s41467-021-25419-4 · Cited by 64
Perceptual confidence judgments reflect self-consistency
Caziot, B., & Mamassian, P. (2021) · Journal of Vision · doi:10.1167/jov.21.12.8 · Cited by 38
Yeon, J., Shekhar, M., & Rahnev, D. (2020) · Scientific Reports · doi:10.1038/s41598-020-77820-6 · Cited by 30
Haarsma, J., Deveci, N., Corbin, N., Callaghan, M. F., & Kok, P. (2023) · The Journal of Neuroscience · doi:10.1523/jneurosci.0998-23.2023 · Cited by 26
Metadata
- Authors
- Hangya, B., Sanders, J. I., & Kepecs, A.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2016-09
- Volume
- 28
- Issue
- 9
- Pages
- 1840-1858
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2465126939
- Cited by
- 113
- References
- 21
- FWCI
- 7.45
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.