Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.

Relation between Belief and Performance in Perceptual Decision Making

Open access
Drugowitsch, Jan, Moreno-Bote, Rubén, Pouget, Alexandre · 2014 · PLoS ONE

Abstract

In an uncertain and ambiguous world, effective decision making requires that subjects form and maintain a belief about the correctness of their choices, a process called meta-cognition. Prediction of future outcomes and self-monitoring are only effective if belief closely matches behavioral performance. Equality between belief and performance is also critical for experimentalists to gain insight into the subjects' belief by simply measuring their performance. Assuming that the decision maker holds the correct model of the world, one might indeed expect that belief and performance should go hand in hand. Unfortunately, we show here that this is rarely the case when performance is defined as the percentage of correct responses for a fixed stimulus, a standard definition in psychophysics. In this case, belief equals performance only for a very narrow family of tasks, whereas in others they will only be very weakly correlated. As we will see it is possible to restore this equality in specific circumstances but this remedy is only effective for a decision-maker, not for an experimenter. We furthermore show that belief and performance do not match when conditioned on task difficulty--as is common practice when plotting the psychometric curve--highlighting common pitfalls in previous neuroscience work. Finally, we demonstrate that miscalibration and the hard-easy effect observed in humans' and other animals' certainty judgments could be explained by a mismatch between the experimenter's and decision maker's expected distribution of task difficulties. These results have important implications for experimental design and are of relevance for theories that aim to unravel the nature of meta-cognition.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0096511

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Illustration of framework with a three-sided coin example. (a) In each trial of a sequence, a hidden state is picked by the experimenter, based on which the observation is generated. The decision maker only observes but not and chooses option where is a deterministic function that maps observations into decisions. In this 2-AFC example there are two possible hidden state, causing to be sampled either according to a biased 3-sided coin , or a fair 3-sided coin . (b) For the given decision function, which maximizes the number of correct decisions for and , the resulting belief and performance are shown for either choice/hidden state. Belief and performance only match if , that is, when .
Figure 1 Illustration of framework with a three-sided coin example. (a) In each trial of a sequence, a hidden state is picked by the experimenter, based on which the observation is generated. The decision maker only observes but not and chooses option where is a deterministic function that maps observations into decisions. In this 2-AFC example there are two possible hidden state, causing to be sampled either according to a biased 3-sided coin , or a fair 3-sided coin . (b) For the given decision function, which maximizes the number of correct decisions for and , the resulting belief and performance are shown for either choice/hidden state. Belief and performance only match if , that is, when .CC BY · PubMed Central
Figure 2 The diffusion model (DM) and 2-race model. (a) In a DM, a particle drifts and diffuses over time. A decision is performed as soon as this particle reaches one of the two boundaries. The mean drift rate , which is unknown to the decision maker, determines which of the two choices is correct. In this illustration, the drift is towards the upper boundary, corresponding to hidden state , such that is the correct choice. We show eight (solid) trajectories leading to the correct choice () and two (dashed) trajectories leading to the wrong choice (). Our framework allows for time-varying boundaries, as shown here and used to generate Figs. 3a/b and 4a/b. (b) A race model features races (here ) that compete against each other in a race towards a boundary of height . The race that first reaches its associated boundary determines the decision. The set of all races is described by a drifting/diffusing particle in -dimensional space. In our illustration this particle drifts towards the upper boundary (thus ) and diffuses in both dimensions. Thus, four (solid) trajectories lead to the correct choice (), and one (dashed) trajectory leads to the incorrect choice ().
Figure 2 The diffusion model (DM) and 2-race model. (a) In a DM, a particle drifts and diffuses over time. A decision is performed as soon as this particle reaches one of the two boundaries. The mean drift rate , which is unknown to the decision maker, determines which of the two choices is correct. In this illustration, the drift is towards the upper boundary, corresponding to hidden state , such that is the correct choice. We show eight (solid) trajectories leading to the correct choice () and two (dashed) trajectories leading to the wrong choice (). Our framework allows for time-varying boundaries, as shown here and used to generate Figs. 3a/b and 4a/b. (b) A race model features races (here ) that compete against each other in a race towards a boundary of height . The race that first reaches its associated boundary determines the decision. The set of all races is described by a drifting/diffusing particle in -dimensional space. In our illustration this particle drifts towards the upper boundary (thus ) and diffuses in both dimensions. Thus, four (solid) trajectories lead to the correct choice (), and one (dashed) trajectory leads to the incorrect choice ().CC BY · PubMed Central
Figure 3 Relationship between belief and performance in diffusion models (DMs) and race models. (a) In a DM with uniform priors, , and symmetric boundaries, belief (data points) and performance (line) are equivalent. In the DM used to generate this figure, the boundaries collapse over time, causing a drop in belief/performance with time. If the boundaries were time-invariant instead, both belief and performance would be independent of time. (b) In the same DM with the same symmetric boundaries, but a non-uniform prior of , this equivalence fails to hold. It appears as if the decision maker were overconfident in her choices. (c) Simulations for a 2-race model with uniform priors, in which the winning race determines the choice, feature a strong fluctuation of the trial-by-trial belief around the decision maker's performance. It appears as if the decision maker features a belief that is idiosyncratic, fluctuating very strongly at each trial, although on average it equals her performance. In all panels the performance (with 95% CI) is estimated in bins of 250 ms, each containing data from 500 trials. The performance is measured as a fraction of trials in which option 1 was chosen when this choice was correct. For each of these bins, 10 examples (50 for the 2-race model) for the trial-by-trial belief when choosing option 1 are shown. This trial-by-trial belief is assumed to be either reported by the decision maker, or to be estimated from neural population activity. Details of how the models were simulated are in Methods: Generating Figures 3 and 4.
Figure 3 Relationship between belief and performance in diffusion models (DMs) and race models. (a) In a DM with uniform priors, , and symmetric boundaries, belief (data points) and performance (line) are equivalent. In the DM used to generate this figure, the boundaries collapse over time, causing a drop in belief/performance with time. If the boundaries were time-invariant instead, both belief and performance would be independent of time. (b) In the same DM with the same symmetric boundaries, but a non-uniform prior of , this equivalence fails to hold. It appears as if the decision maker were overconfident in her choices. (c) Simulations for a 2-race model with uniform priors, in which the winning race determines the choice, feature a strong fluctuation of the trial-by-trial belief around the decision maker's performance. It appears as if the decision maker features a belief that is idiosyncratic, fluctuating very strongly at each trial, although on average it equals her performance. In all panels the performance (with 95% CI) is estimated in bins of 250 ms, each containing data from 500 trials. The performance is measured as a fraction of trials in which option 1 was chosen when this choice was correct. For each of these bins, 10 examples (50 for the 2-race model) for the trial-by-trial belief when choosing option 1 are shown. This trial-by-trial belief is assumed to be either reported by the decision maker, or to be estimated from neural population activity. Details of how the models were simulated are in Methods: Generating Figures 3 and 4.CC BY · PubMed Central
Figure 4 Comparing estimated belief with performance and trial-by-trial belief. (a) A DM with a non-uniform prior of as in Fig. 3b. Trial-by-trial belief differs from performance because of the asymmetric prior. By contrast, the estimated belief using Eq. (13) matches the trial-by-trial belief, because the decision maker's state is fully observable in a DM. (b) A two race model with uniform priors as in Fig 3c. This time, the decision maker's state is not fully observable because the state of the losing race is unknown to the experimenter. As a consequence, the belief estimated by Eq. (13) no longer matches the trial-by-trial belief of the observer but only the averaged belief, where the average is performed over the state of the losing race. Details of the model simulations are described in Methods: Generating Figures 3 and 4.
Figure 4 Comparing estimated belief with performance and trial-by-trial belief. (a) A DM with a non-uniform prior of as in Fig. 3b. Trial-by-trial belief differs from performance because of the asymmetric prior. By contrast, the estimated belief using Eq. (13) matches the trial-by-trial belief, because the decision maker's state is fully observable in a DM. (b) A two race model with uniform priors as in Fig 3c. This time, the decision maker's state is not fully observable because the state of the losing race is unknown to the experimenter. As a consequence, the belief estimated by Eq. (13) no longer matches the trial-by-trial belief of the observer but only the averaged belief, where the average is performed over the state of the losing race. Details of the model simulations are described in Methods: Generating Figures 3 and 4.CC BY · PubMed Central
Figure 5 Mismatch between average belief and performance when conditioning on task difficulty: the hard-easy effect and miscalibration. We simulated a task with varying difficulty given by a diffusion model with a drift rate whose magnitude and sign varied across trials, while being constant within each trial. (a) The top graph shows the across-trials point-wise prior on the drift rate used in the simulation that roughly approximates a zero-mean Gaussian (dashed line). We computed the decision maker's belief by either using this point-wise prior directly, or by assuming it to follow a too-wide zero-mean Gaussian (dotted line). The bottom graph shows that the point-wise prior corresponds to the 10th, 20th, …, 90th percentile of the Gaussian it approximates. (b) The decision maker's chronometric (top) and psychometric (bottom) function over task difficulty (magnitude of ) for non-negative drift rates. Correct choices here correspond to hitting the upper bound of the diffusion model if the drift rate is positive, and the lower bound otherwise. The bottom graph also shows the decision maker's average belief over for both correct and error trials (dots exactly one top of each other, as confidence for correct and error trials is identical) based on the correct, point-wise prior (squares, +/− 2SD) and on the incorrect Gaussian prior (crosses). In both cases, the mismatch between average belief and performance when conditioning on task difficulty is clearly visible. (c) The calibration curves, showing the probability of performing correct choices as a function of the decision maker's belief. When binning trials by difficulty (that is, drift rate magnitude), this choice probability is constant while the decision maker's belief varies across trials. This results in flat calibration curves (dashed/dotted lines), caricaturizing the frequently observed hard-easy effect. Once we stop conditioning on task difficulty, the calibration curve reveals perfect calibration (solid line). (d) Calibration curves for a mismatch between the actual distribution of task difficulties and that assumed by the decision maker to compute her belief. We consider the case in which the decision maker's distribution is too narrow (that is, has too small standard deviation; dotted line) or too wide (too large standard deviation; solid line). Both cases feature a clear miscalibration of the decision maker's belief.
Figure 5 Mismatch between average belief and performance when conditioning on task difficulty: the hard-easy effect and miscalibration. We simulated a task with varying difficulty given by a diffusion model with a drift rate whose magnitude and sign varied across trials, while being constant within each trial. (a) The top graph shows the across-trials point-wise prior on the drift rate used in the simulation that roughly approximates a zero-mean Gaussian (dashed line). We computed the decision maker's belief by either using this point-wise prior directly, or by assuming it to follow a too-wide zero-mean Gaussian (dotted line). The bottom graph shows that the point-wise prior corresponds to the 10th, 20th, …, 90th percentile of the Gaussian it approximates. (b) The decision maker's chronometric (top) and psychometric (bottom) function over task difficulty (magnitude of ) for non-negative drift rates. Correct choices here correspond to hitting the upper bound of the diffusion model if the drift rate is positive, and the lower bound otherwise. The bottom graph also shows the decision maker's average belief over for both correct and error trials (dots exactly one top of each other, as confidence for correct and error trials is identical) based on the correct, point-wise prior (squares, +/− 2SD) and on the incorrect Gaussian prior (crosses). In both cases, the mismatch between average belief and performance when conditioning on task difficulty is clearly visible. (c) The calibration curves, showing the probability of performing correct choices as a function of the decision maker's belief. When binning trials by difficulty (that is, drift rate magnitude), this choice probability is constant while the decision maker's belief varies across trials. This results in flat calibration curves (dashed/dotted lines), caricaturizing the frequently observed hard-easy effect. Once we stop conditioning on task difficulty, the calibration curve reveals perfect calibration (solid line). (d) Calibration curves for a mismatch between the actual distribution of task difficulties and that assumed by the decision maker to compute her belief. We consider the case in which the decision maker's distribution is too narrow (that is, has too small standard deviation; dotted line) or too wide (too large standard deviation; solid line). Both cases feature a clear miscalibration of the decision maker's belief.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 48 references

Research works printed but not linked: 2

  1. Wald A (1947) Sequential analysis. New York, London: J. Wiley & sons, Chapman & Hall. <volume>xii</volume>: , 212 p. p.
  2. MM Fishman, 1987, Average duration of asymptotically optimal multialternative sequential proceure for recognition of processes, Soviet Journal of Communication and Technology of Electronics, 30, 2541

Cited by

See all 72

Metadata

Authors
Drugowitsch, J., Moreno-Bote, R., & Pouget, A.
Affiliations
University of Geneva (CH)
Corresponding author
Drugowitsch, Jan
Venue
PLoS ONE
Published
2014-05-09
Volume
9
Issue
5
Pages
e96511
ISSN
1932-6203 (ISSN-L)
OpenAlex
W1980160354
Cited by
72
References
48
Citations by year
2026: 3 · 2025: 6 · 2024: 4 · 2023: 5 · 2022: 4 · 2021: 5 · 2020: 9 · 2019: 8 · 2018: 6 · 2017: 5 · 2016: 10 · 2015: 4 · 2014: 3
FWCI
4.09
Type
journal article
Language
en
Open access
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, other-oa)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Archive ouverte UNIGE (University of Geneva) (publishedVersion, other-oa)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Repositori digital de la UPF (Universitat Pompeu Fabra) (submittedVersion, cc-by)
  • Archive ouverte UNIGE (University of Geneva) (submittedVersion, cc-by)
Funders
  • National Science Foundation (1109366, 0446730)
  • James S. McDonnell Foundation
  • European Commission (276795)
  • Multidisciplinary University Research Initiative (N00014-07-1-0937)
  • FP7 People: Marie-Curie Actions
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
  • Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
Keywords
metacognition, perceptual decision making, confidence judgments, calibration error, task difficulty, psychometric curves, belief formation, experimental design
MeSH terms
Algorithms, Choice Behavior, Cognition, Culture, Decision Making, Humans, Models, Psychological, Problem Solving, Psychomotor Performance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Relation between Belief and Performance in Perceptual Decision Making. https://app.cochat.ai/papers/10.1371/journal.pone.0096511
The paper
Drugowitsch, J., Moreno-Bote, R., & Pouget, A. (2014). Relation between Belief and Performance in Perceptual Decision Making. PLoS ONE. https://doi.org/10.1371/journal.pone.0096511

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.