A Mathematical Framework for Statistical Decision Confidence
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Decision confidence is a forecast about the probability that a decision will be correct. From a statistical perspective, decision confidence can be defined as the Bayesian posterior probability that the chosen option is correct based on the evidence contributing to it. Here, we used this formal definition as a starting point to develop a normative statistical framework for decision confidence. Our goal was to make general predictions that do not depend on the structure of the noise or a specific algorithm for estimating confidence. We analytically proved several interrelations between statistical decision confidence and observable decision measures, such as evidence discriminability, choice, and accuracy. These interrelationships specify necessary signatures of decision confidence in terms of externally quantifiable variables that can be empirically tested. Our results lay the foundations for a mathematically rigorous treatment of decision confidence that can lead to a common framework for understanding confidence across different research domains, from human and animal behavior to neural representations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Merging the senses into a robust percept
Ernst, M. O., & Bülthoff, H. H. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.02.002 · Cited by 1,906 · cited 1×
“Bayesian theories have been used to account for how brains can perform sensory processing (Beck et al., 2008; Ernst and Bülthoff, 2004; Graf et al., 2005) and motor control (Körding and Wolpert, 2004, 2006).”
MethodBayesian inference with probabilistic population codes
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006) · Nature Neuroscience · doi:10.1038/nn1790 · Cited by 1,457
Representation of Confidence Associated with a Decision by Neurons in the Parietal Cortex
Kiani, R., & Shadlen, M. N. (2009) · Science · doi:10.1126/science.1169405 · Cited by 1,027 · cited 1×
“Perhaps the most intuitive extension is within the race model framework, where the difference between decision variables for the winning and losing races provides an estimate of confidence (Kepecs et al., 2008; Merkle and Van Zandt, 2006; Moreno-Bote, 2010; Vickers, 1979; Zylberberg et al., 2012).”
BackgroundA comparison of current measures of the accuracy of feeling-of-knowing predictions
Nelson, T. O. (1984) · Psychological Bulletin · doi:10.1037/0033-2909.95.1.109 · Cited by 976 · cited 1×
“…confidence could be computed, signal detection theory has also been used as a starting point for evaluating the metacognitive sensitivity of human confidence reports (Ferrell and McGoey, 1980; Higham and Arnold, 2007; Higham et al., 2009; Kunimoto et al., 2001; Lachman et al., 1979; Nelson, 1984).”
BackgroundNeural correlates, computation and behavioural impact of decision confidence
Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008) · Nature · doi:10.1038/nature07200 · Cited by 809 · cited 6×
“Perhaps the most intuitive extension is within the race model framework, where the difference between decision variables for the winning and losing races provides an estimate of confidence (Vickers, 1979; Kepecs et al., 2008; Moreno-Bote, 2010; Pleskac & Busemeyer, 2010; Zylberberg, Barttfeld, & Sigman, 2012; Drugowit…”
BackgroundMethodInfluentialTwo-stage dynamic signal detection: A theory of choice, decision time, and confidence.
Pleskac, T. J., & Busemeyer, J. R. (2010) · Psychological Review · doi:10.1037/a0019737 · Cited by 722
Probabilistic population codes for Bayesian decision making
Beck, J. M., Ma, W. J., Kiani, R., Hanks, T., Churchland, A. K., Roitman, J., Shadlen, M. N., Latham, P. E., & Pouget, A. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.021 · Cited by 645
Confidence and certainty: Distinct probabilistic quantities for different goals
Pouget, A., Drugowitsch, J., & Kepecs, A. (2016) · Nature Neuroscience · doi:10.1038/nn.4240 · Cited by 528
Galvin, S. J., Podd, J. V., Drga, V., & Whitmore, J. (2003) · Psychonomic Bulletin & Review · doi:10.3758/bf03196546 · Cited by 405
The construction of confidence in a perceptual decision
Zylberberg, A., Barttfeld, P., & Sigman, M. (2012) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2012.00079 · Cited by 297 · cited 1×
“Perhaps the most intuitive extension is within the race model framework, where the difference between decision variables for the winning and losing races provides an estimate of confidence (Kepecs et al., 2008; Merkle and Van Zandt, 2006; Moreno-Bote, 2010; Vickers, 1979; Zylberberg et al., 2012).”
BackgroundConfidence and accuracy of near-threshold discrimination responses
Kunimoto, C., Miller, J., & Pashler, H. (2001) · Consciousness and Cognition · doi:10.1006/ccog.2000.0494 · Cited by 291
Harvey, N. (1997) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(97)01014-0 · Cited by 239
Responses of pulvinar neurons reflect a subject's confidence in visual categorization
Komura, Y., Nikkuni, A., Hirashima, N., Uetake, T., & Miyamoto, A. (2013) · Nature Neuroscience · doi:10.1038/nn.3393 · Cited by 226
Orbitofrontal Cortex Is Required for Optimal Waiting Based on Decision Confidence
Lak, A., Costa, G. M., Romberg, E., Koulakov, A. A., Mainen, Z. F., & Kepecs, A. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.039 · Cited by 209
Confidence-Related Decision Making
Insabato, A., Pannunzi, M., Rolls, E. T., & Deco, G. (2010) · Journal of Neurophysiology · doi:10.1152/jn.01068.2009 · Cited by 95
Relation between belief and performance in perceptual decision making
Drugowitsch, J., Moreno-Bote, R., & Pouget, A. (2014) · PLoS ONE · doi:10.1371/journal.pone.0096511 · Cited by 72
Higham, P. A., Perfect, T. J., & Bruno, D. (2009) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/a0013865 · Cited by 70 · cited 2×
“Beyond a description of how confidence could be computed, signal detection theory has also been used as a starting point for evaluating the metacognitive sensitivity of human confidence reports (Ferrell and McGoey, 1980; Higham and Arnold, 2007; Higham et al., 2009; Kunimoto et al., 2001; Lachman et al., 1979; Nelson,…”
BackgroundMethodEvaluation of a 'bias-free' measure of awareness
Azzopardi, P., & Evans, S. (2007) · Spatial Vision · doi:10.1163/156856807779369742 · Cited by 68 · cited 1×
“When these assumptions are violated, type-2 d’ can be confounded by biases (Barrett et al., 2013; Evans and Azzopardi, 2007; Galvin et al., 2003; Kunimoto et al., 2001; Maniscalco and Lau, 2012).”
BackgroundSignal detection theory, uncertainty, and Poisson-like population codes
Ma, W. J. (2010) · Vision Research · doi:10.1016/j.visres.2010.08.035 · Cited by 60 · cited 1×
“For instance, a signal detection theory framework is often employed for probing decisions under uncertainty, and can provide a strong basis for understanding decision confidence as well (Fleming and Dolan, 2010; Kepecs et al., 2008; Ma, 2010; Maniscalco and Lau, 2012; Ratcliff and Starns, 2009).”
Method
Cited by
Confidence and certainty: Distinct probabilistic quantities for different goals
Pouget, A., Drugowitsch, J., & Kepecs, A. (2016) · Nature Neuroscience · doi:10.1038/nn.4240 · Cited by 528
Signatures of a Statistical Computation in the Human Sense of Confidence
Sanders, J. I., Hangya, B., & Kepecs, A. (2016) · Neuron · doi:10.1016/j.neuron.2016.03.025 · Cited by 253
Self-citationStriatal dopamine mediates hallucination-like perception in mice
Schmack, K., Bosc, M., Ott, T., Sturgill, J. F., & Kepecs, A. (2021) · Science · doi:10.1126/science.abf4740 · Cited by 253
Self-citationMidbrain dopamine neurons signal belief in choice accuracy during a perceptual decision
Lak, A., Nomoto, K., Keramati, M., Sakagami, M., & Kepecs, A. (2017) · Current Biology · doi:10.1016/j.cub.2017.02.026 · Cited by 170
Comparing Bayesian and non-Bayesian accounts of human confidence reports
Adler, W. T., & Ma, W. J. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006572 · Cited by 118
Metadata
- Authors
- Hangya, B., Sanders, J. I., & Kepecs, A.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2016-09
- Volume
- 28
- Issue
- 9
- Pages
- 1840-1858
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2465126939
- Cited by
- 113
- References
- 19
- FWCI
- 7.45
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.