Relation between Belief and Performance in Perceptual Decision Making
Open accessThe DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
In an uncertain and ambiguous world, effective decision making requires that subjects form and maintain a belief about the correctness of their choices, a process called meta-cognition. Prediction of future outcomes and self-monitoring are only effective if belief closely matches behavioral performance. Equality between belief and performance is also critical for experimentalists to gain insight into the subjects' belief by simply measuring their performance. Assuming that the decision maker holds the correct model of the world, one might indeed expect that belief and performance should go hand in hand. Unfortunately, we show here that this is rarely the case when performance is defined as the percentage of correct responses for a fixed stimulus, a standard definition in psychophysics. In this case, belief equals performance only for a very narrow family of tasks, whereas in others they will only be very weakly correlated. As we will see it is possible to restore this equality in specific circumstances but this remedy is only effective for a decision-maker, not for an experimenter. We furthermore show that belief and performance do not match when conditioned on task difficulty--as is common practice when plotting the psychometric curve--highlighting common pitfalls in previous neuroscience work. Finally, we demonstrate that miscalibration and the hard-easy effect observed in humans' and other animals' certainty judgments could be explained by a mismatch between the experimenter's and decision maker's expected distribution of task difficulties. These results have important implications for experimental design and are of relevance for theories that aim to unravel the nature of meta-cognition.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“This is because the quantity that is accumulated, termed a decision variable (DV), when combined with elapsed decision time, maps to the probability that a decision rendered on its value will be correct (Kiani and Shadlen, 2009; Drugowitsch et al., 2014).”
From A common mechanism underlies changes of mind about decisions and confidence (2016) · cites this paper as Drugowitsch et al., 2014
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
(1966) · Medical Entomology and Zoology · Cited by 10,293
The Fokker-Planck Equation: Methods of Solution and Applications
Risken, H. (1996) · Springer Series in Synergetics · doi:10.1007/978-3-642-61544-3 · Cited by 9,056
Judgment Under Uncertainty: Heuristics and Biases.
Shafer, G. R., Kahnerman, D., Slovic, P., & Tversky, A. (1984) · Journal of the American Statistical Association · doi:10.2307/2288362 · Cited by 5,657
Humans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844
Ratcliff, R. (1978) · Psychological Review · doi:10.1037/0033-295x.85.2.59 · Cited by 4,513
The Theory of Stochastic Processes
Cox, D. R., Miller, H. D., Weiss, G., & Shuler, K. (1966) · Physics Today · doi:10.1063/1.3047872 · Cited by 3,324
The psychometric function: I. Fitting, sampling, and goodness of fit
Wichmann, F. A., & Hill, N. J. (2001) · Perception & Psychophysics · doi:10.3758/bf03194544 · Cited by 2,358
The trouble with overconfidence.
Moore, D. A., & Healy, P. J. (2008) · Psychological Review · doi:10.1037/0033-295x.115.2.502 · Cited by 2,358
The analysis of visual motion: a comparison of neuronal and psychophysical performance
Britten, K., Shadlen, M., Newsome, W., & Movshon, J. (1992) · The Journal of Neuroscience · doi:10.1523/jneurosci.12-12-04745.1992 · Cited by 1,958
The Ventriloquist Effect Results from Near-Optimal Bimodal Integration
Alais, D., & Burr, D. (2004) · Current Biology · doi:10.1016/j.cub.2004.01.029 · Cited by 1,936
Metacognition: Knowing about Knowing
(1994) · The MIT Press eBooks · doi:10.7551/mitpress/4561.001.0001 · Cited by 1,720
Probabilistic mental models: a Brunswikian theory of confidence
Gigerenzer, G., Hoffrage, U., & Kleinbölting, H. (1991) · Psychological Review · doi:10.1037/0033-295x.98.4.506 · Cited by 1,506 · cited 2×
“This potential explanation for miscalibration has already been suggested conceptually in [12], but here we made its statement more quantitative.”
BackgroundNeuronal correlates of a perceptual decision
Newsome, W. T., Britten, K. H., & Movshon, J. A. (1989) · Nature · doi:10.1038/341052a0 · Cited by 1,177
Optimum Character of the Sequential Probability Ratio Test
Wald, A., & Wolfowitz, J. (1948) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177730197 · Cited by 1,162
Do those who know more also know more about how much they know?
Lichtenstein, S., & Fischhoff, B. (1977) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(77)90001-0 · Cited by 1,081
Representation of Confidence Associated with a Decision by Neurons in the Parietal Cortex
Kiani, R., & Shadlen, M. N. (2009) · Science · doi:10.1126/science.1169405 · Cited by 1,027 · cited 6×
“This performance measure is standard in the psychophysics and perceptual decision making literature [5,8,21].”
BackgroundMethodInfluentialManiscalco, B., & Lau, H. (2012) · Consciousness and Cognition · doi:10.1016/j.concog.2011.09.021 · Cited by 883
Neural correlates, computation and behavioural impact of decision confidence
Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008) · Nature · doi:10.1038/nature07200 · Cited by 809
Two-stage dynamic signal detection: A theory of choice, decision time, and confidence.
Pleskac, T. J., & Busemeyer, J. R. (2010) · Psychological Review · doi:10.1037/a0019737 · Cited by 722
The effect of stimulus strength on the speed and accuracy of a perceptual decision
Palmer, J., Huk, A. C., & Shadlen, M. N. (2005) · Journal of Vision · doi:10.1167/5.5.1 · Cited by 691 · cited 1×
“Reaction times and choice probability were computed analytically using standard results for bounded diffusion models [48–49].”
MethodProbabilistic population codes for Bayesian decision making
Beck, J. M., Ma, W. J., Kiani, R., Hanks, T., Churchland, A. K., Roitman, J., Shadlen, M. N., Latham, P. E., & Pouget, A. (2008) · Neuron · doi:10.1016/j.neuron.2008.09.021 · Cited by 645
Self-citationSimultaneous over- and underconfidence: the role of error in judgment processes
Erev, I., Wallsten, T. S., & Budescu, D. V. (1994) · Psychological Review · doi:10.1037/0033-295x.101.3.519 · Cited by 644
The Cost of Accumulating Evidence in Perceptual Decision Making
Drugowitsch, J., Moreno-Bote, R., Churchland, A. K., Shadlen, M. N., & Pouget, A. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.4010-11.2012 · Cited by 577
Self-citationConfidence in value-based choice
De Martino, B., Fleming, S. M., Garrett, N., & Dolan, R. J. (2013) · Nature Neuroscience · doi:10.1038/nn.3279 · Cited by 534
Decision Processes in Visual Perception
(1979) · Elsevier eBooks · doi:10.1016/c2013-0-11654-6 · Cited by 502
Neural correlates of reliability-based cue weighting during multisensory integration
Fetsch, C. R., Pouget, A., DeAngelis, G. C., & Angelaki, D. E. (2012) · Nature Neuroscience · doi:10.1038/nn.2983 · Cited by 459
Self-citationBayesian integration of visual and auditory signals for spatial localization
Battaglia, P. W., Jacobs, R. A., & Aslin, R. N. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001391 · Cited by 425
Galvin, S. J., Podd, J. V., Drga, V., & Whitmore, J. (2003) · Psychonomic Bulletin & Review · doi:10.3758/bf03196546 · Cited by 405
The comparative psychology of uncertainty monitoring and metacognition
Smith, J. D., Shields, W. E., & Washburn, D. A. (2003) · Behavioral and Brain Sciences · doi:10.1017/s0140525x03000086 · Cited by 402
A Neural Representation of Categorization Uncertainty in the Human Brain
Grinband, J., Hirsch, J., & Ferrera, V. P. (2006) · Neuron · doi:10.1016/j.neuron.2006.01.032 · Cited by 336
Not Noisy, Just Wrong: The Role of Suboptimal Inference in Behavioral Variability
Beck, J. M., Ma, W. J., Pitkow, X., Latham, P. E., & Pouget, A. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.016 · Cited by 316
Self-citationA computational framework for the study of confidence in humans and animals
Kepecs, A., & Mainen, Z. F. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2012.0037 · Cited by 305
The construction of confidence in a perceptual decision
Zylberberg, A., Barttfeld, P., & Sigman, M. (2012) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2012.00079 · Cited by 297
Multihypothesis sequential probability ratio tests .I. Asymptotic optimality
Draglia, V. P., Tartakovsky, A. G., & Veeravalli, V. V. (1999) · IEEE Transactions on Information Theory · doi:10.1109/18.796383 · Cited by 291
How humans combine simultaneous proprioceptive and visual position information
van Beers, R. J., Sittig, A. C., & van der Gon Denier, J. J. (1996) · Experimental Brain Research · doi:10.1007/bf00227302 · Cited by 279
Stochastic Dynamic Models of Response Time and Accuracy: A Foundational Primer
Smith, P. L. (2000) · Journal of Mathematical Psychology · doi:10.1006/jmps.1999.1260 · Cited by 272
A sequential procedure for multihypothesis testing
Baum, C. W., & Veeravalli, V. V. (1994) · IEEE Transactions on Information Theory · doi:10.1109/18.340472 · Cited by 268
Modeling confidence and response time in recognition memory.
Ratcliff, R., & Starns, J. J. (2009) · Psychological Review · doi:10.1037/a0014086 · Cited by 256
A model of calibration for subjective probabilities
Ferrell, W. R., & McGoey, P. J. (1980) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(80)90045-8 · Cited by 192
Evaluating and Combining Subjective Probability Estimates
WALLSTEN, T. S., BUDESCU, D. V., EREV, I., & DIEDERICH, A. (1997) · Journal of Behavioral Decision Making · doi:10.1002/(sici)1099-0771(199709)10:3<243::aid-bdm268>3.0.co;2-m · Cited by 145
JUSLIN, P., OLSSON, H., & BJÖRKMAN, M. (1997) · Journal of Behavioral Decision Making · doi:10.1002/(sici)1099-0771(199709)10:3<189::aid-bdm258>3.0.co;2-4 · Cited by 145
Are We Overconfident in the Belief That Probability Forecasters Are Overconfident?
Pfeifer, P. E. (1994) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1994.1034 · Cited by 106
Moreno-Bote, R. (2010) · Neural Computation · doi:10.1162/neco.2010.12-08-930 · Cited by 99
Self-citationNearly-Optimal Sequential Tests for Finitely Many Parameter Values
Lorden, G. (1977) · The Annals of Statistics · doi:10.1214/aos/1176343737 · Cited by 91
Realism of confidence in sensory discrimination
Baranski, J. V., & Petrusic, W. M. (1999) · Perception & Psychophysics · doi:10.3758/bf03206187 · Cited by 80
An application of the Poisson race model to confidence calibration.
Merkle, E. C., & Van Zandt, T. (2006) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.135.3.391 · Cited by 75
The disutility of the hard-easy effect in choice confidence
Merkle, E. C. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.1.204 · Cited by 51
Ferrell, W. R. (1995) · Perception & Psychophysics · doi:10.3758/bf03206511 · Cited by 34
Research works printed but not linked: 2
- Wald A (1947) Sequential analysis. New York, London: J. Wiley & sons, Chapman & Hall. <volume>xii</volume>: , 212 p. p.
- MM Fishman, 1987, Average duration of asymptotically optimal multialternative sequential proceure for recognition of processes, Soviet Journal of Communication and Technology of Electronics, 30, 2541
Cited by
Self-Evaluation of Decision-Making: A General Bayesian Framework for Metacognitive Computation
Fleming, S. M., & Daw, N. D. (2017) · Psychological Review · doi:10.1037/rev0000045 · Cited by 584
Confidence and certainty: Distinct probabilistic quantities for different goals
Pouget, A., Drugowitsch, J., & Kepecs, A. (2016) · Nature Neuroscience · doi:10.1038/nn.4240 · Cited by 528
Self-citationChoice Certainty Is Informed by Both Evidence and Decision Time
Kiani, R., Corthell, L., & Shadlen, M. N. (2014) · Neuron · doi:10.1016/j.neuron.2014.12.015 · Cited by 470
“Indeed, if certainty is at all meaningful, it ought to reflect accuracy, on average, even if imperfectly (Drugowitsch et al., 2014).”
Suboptimality in perceptual decision making
Rahnev, D., & Denison, R. N. (2018) · Behavioral and Brain Sciences · doi:10.1017/s0140525x18000936 · Cited by 342
A common mechanism underlies changes of mind about decisions and confidence
van den Berg, R., Anandalingam, K., Zylberberg, A., Kiani, R., Shadlen, M. N., & Wolpert, D. M. (2016) · eLife · doi:10.7554/elife.12192 · Cited by 270
Metadata
- Authors
- Drugowitsch, J., Moreno-Bote, R., & Pouget, A.
- Affiliations
- University of Geneva (CH)
- Corresponding author
- Drugowitsch, Jan
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-05-09
- Volume
- 9
- Issue
- 5
- Pages
- e96511
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4016031
- OpenAlex
- W1980160354
- Cited by
- 72
- References
- 48
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 4 · 2023: 5 · 2022: 4 · 2021: 5 · 2020: 9 · 2019: 8 · 2018: 6 · 2017: 5 · 2016: 10 · 2015: 4 · 2014: 3
- FWCI
- 4.09
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Archive ouverte UNIGE (University of Geneva) (publishedVersion, other-oa)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Repositori digital de la UPF (Universitat Pompeu Fabra) (submittedVersion, cc-by)
- Archive ouverte UNIGE (University of Geneva) (submittedVersion, cc-by)
- Funders
- National Science Foundation (1109366, 0446730)
- James S. McDonnell Foundation
- European Commission (276795)
- Multidisciplinary University Research Initiative (N00014-07-1-0937)
- FP7 People: Marie-Curie Actions
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Keywords
- metacognition, perceptual decision making, confidence judgments, calibration error, task difficulty, psychometric curves, belief formation, experimental design
- MeSH terms
- Algorithms, Choice Behavior, Cognition, Culture, Decision Making, Humans, Models, Psychological, Problem Solving, Psychomotor Performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.