A common mechanism underlies changes of mind about decisions and confidence
Open accessAbstract
Decisions are accompanied by a degree of confidence that a selected option is correct. A sequential sampling framework explains the speed and accuracy of decisions and extends naturally to the confidence that the decision rendered is likely to be correct. However, discrepancies between confidence and accuracy suggest that confidence might be supported by mechanisms dissociated from the decision process. Here we show that this discrepancy can arise naturally because of simple processing delays. When participants were asked to report choice and confidence simultaneously, their confidence, reaction time and a perceptual decision about motion were explained by bounded evidence accumulation. However, we also observed revisions of the initial choice and/or confidence. These changes of mind were explained by a continuation of the mechanism that led to the initial choice. Our findings extend the sequential sampling framework to vacillation about confidence and invites caution in interpreting dissociations between confidence and accuracy.
AI snapshot
Written from the paper’s full text.
TL;DR
Changes of mind about choice and confidence in a motion-discrimination task are explained by continued bounded evidence accumulation after the initial decision, using the same evidence stream. This post-decision processing can produce an apparent dissociation between confidence and accuracy without invoking separate metacognitive mechanisms.
Key findings
- Four subjects made simultaneous direction and confidence reports by reaching to one of four targets; changes of mind occurred on 2.0–8.8% of trials, and confidence changes were more frequent than decision changes (p<1e-4 for all participants).
- Changes of decision were more likely after errors and with stronger motion; for three of four subjects they corrected an initial error more often than they spoiled a correct choice (P<0.001, P<0.001, P=0.084, P<0.001). Changes of confidence were more likely after low-confidence initial choices, and changes of mind earned 0.72–1.12 additional points per change trial.
- Motion-energy kernels showed that initial choice and confidence were supported by a common evidence stream, with information used until ~400 ms before movement onset. Post-decision information explained revisions: low-to-high confidence changes were supported by evidence for the initial choice, whereas high-to-low confidence changes were associated with evidence for the opposite direction.
- Comparing odds ratios from initial versus final confidence produced a systematic discrepancy (P<1e-4, sign test; reliable for 13 of 19 points, P<0.05 bootstrap), which the authors argue can arise from continued first-order evidence accumulation rather than a separate second-order confidence process.
Methods
- Six naïve right-handed subjects (ages 21–34) participated; two were excluded, leaving four analyzed. Subjects used a vBOT handle to judge random-dot motion direction and report confidence simultaneously, with coherence levels ±0, ±3.2, ±6.4, ±12.8, ±25.6, and ±51.2%; the stimulus was extinguished at movement onset. Each subject completed 12 sessions (9024–9216 trials), except Subject 4, for whom only the first 5000 trials were analyzed because of nonstationary behavior.
- Behavioral analyses used logistic regressions, ANOVA, Fisher exact tests, and motion-energy psychophysical kernels. The main model was a race between two accumulators receiving bivariate Gaussian evidence, with decision bound B, non-decision time, and a log-odds threshold θ for confidence; changes of mind were modeled by continued accumulation for a post-initiation period tpip with shifted thresholds δθ1–δθ3.
- Model parameters were fit by maximum likelihood to initial choice, confidence, and reaction-time data, then the change-of-mind model was fit with κ and θ carried over from the initial fits.
Limitations
- Stated: the model is imperfect—it overestimates confidence at the lowest motion strength and shows discrepancies at 0% coherence for three of four subjects; the flat bound does not capture RT distributions or mean RT on error trials, so only correct-trial mean RTs and 0% coherence trials were used in the cost function.
- Stated: basing changes of decision on log odds is a simplification, and the frequency of changes of mind was too small to evaluate more flexible post-initiation termination rules.
- Evident: the main conclusions rest on four analyzed subjects in a specific random-dot motion task with stationary evidence samples; the paper notes that other perceptual and memory-based decisions involve less well-characterized evidence streams.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 43 cited sources checked · 0 supported · 0 flagged
- Findingown data
Subjects reported high confidence more often on correct choices than on errors.
“In addition, subjects reported high confidence more often on correct choices than on errors (Figure 2c), and they tended to be more confident on those errors associated with stronger motion (P<1e-5 for 3 subjects and P = 0.50 for S3).”
Quote in the paperNot a substantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
- Findingown data
Stronger motion supported faster decisions.
“Not surprisingly, stronger motion supported more accurate (Figure 2a) and faster decisions (Figure 2b), that tended to be assigned the higher confidence (Figure 2c).”
Quote in the paperNot a substantial claim - Findingown data
Choice, confidence, and reaction time are explained by a sequential sampling mechanism operating on a common evidence stream.
“We show that choice, confidence, and reaction time are explained by a sequential sampling mechanism operating on a common evidence stream.”
- Also stated in: § Abstract · plain
“When participants were asked to report choice and confidence simultaneously, their confidence, reaction time and a perceptual decision about motion were explained by bounded evidence accumulation.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The same information was used to reach an initial choice about direction and confidence.
- Also stated in: § Abstract · plain
- Findingown data
When subjects were more confident, they tended to be more accurate.
“When subjects were more confident, they tended to be more accurate (Figure 2a; P<0.0001 for all subjects) and faster (Figure 2b; P<0.0001 for all subjects).”
Quote in the paperNot a substantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
- Findingown data
Stronger motion tended to be assigned higher confidence.
“Not surprisingly, stronger motion supported more accurate (Figure 2a) and faster decisions (Figure 2b), that tended to be assigned the higher confidence (Figure 2c).”
Quote in the paperNot a substantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
- Findingown data
On errors, subjects tended to be more confident when the errors were associated with stronger motion.
“In addition, subjects reported high confidence more often on correct choices than on errors (Figure 2c), and they tended to be more confident on those errors associated with stronger motion (P<1e-5 for 3 subjects and P = 0.50 for S3).”
Quote in the paperNot a substantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
- Findingown data
Stronger motion supported more accurate decisions.
“Not surprisingly, stronger motion supported more accurate (Figure 2a) and faster decisions (Figure 2b), that tended to be assigned the higher confidence (Figure 2c).”
Quote in the paperNot a substantial claim - Findingown data
When subjects were more confident, they tended to be faster.
“When subjects were more confident, they tended to be more accurate (Figure 2a; P<0.0001 for all subjects) and faster (Figure 2b; P<0.0001 for all subjects).”
Quote in the paperNot a substantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
- Findingown data
For three of the four subjects, changes of decision corrected an initial error more often than they spoiled an initially correct choice.
“For three of the four subjects, changes of decision corrected an initial error more often than they spoiled an initially correct choice (P<0.001, P<0.001, P = 0.084, P<0.001), consistent with previous reports (Resulaj et al., 2009; Burk et al., 2014).”
Quote in the paperNot a substantial claim0 of 2 sources checked - Findingown data
Similar hysteresis is seen for initial high confidence decisions.
“Similar hysteresis is seen for initial high confidence decisions (Figure 6, bottom row).”
Quote in the paperSubstantial claim- The authors state this as an assumption: Evidence was accumulated past the point of the initial decision for a fixed amount of time (tpip) that is less than the non-decision time.
- The authors state this as an assumption: The state of the belief at tpip determines the final confidence and choice.
- The authors state this as an assumption: Subjects might not apply identical criteria for confidence and choice in the pre- and post-initiation epoch (δθ parameters).
- The authors state this as an assumption: The model parameterization requires the pink and red regions to be same for both initial high and low confidence decisions.
- Findingown data
These values are consistent with the estimates of non-decision time (tnd) obtained from the model fits to the initial choices and RTs.
“These values are consistent with the estimates of non-decision time (tnd) obtained from the model fits to the initial choices and RTs (Table 1).”
Quote in the paperNot a substantial claim - Findingown data
Changes from high to low confidence were associated with motion information in support of the direction opposite to the one chosen.
“changes from high to low confidence were associated with motion information in support of the direction opposite to the one chosen (Figure 4b, middle)”
Quote in the paperSubstantial claim - Findingown data
All subjects required more evidence to change their initial decision about direction than a simple reversal in sign of the evidence (δθ2>0).
“Moreover, all subjects required more evidence to change their initial decision about direction than a simple reversal in sign of the evidence (δθ2>0).”
Quote in the paperSubstantial claim- The authors state this as an assumption: Evidence was accumulated past the point of the initial decision for a fixed amount of time (tpip) that is less than the non-decision time.
- The authors state this as an assumption: The state of the belief at tpip determines the final confidence and choice.
- The authors state this as an assumption: Subjects might not apply identical criteria for confidence and choice in the pre- and post-initiation epoch (δθ parameters).
- The authors state this as an assumption: The model parameterization requires the pink and red regions to be same for both initial high and low confidence decisions.
- Findingown data
Those double changes that did occur tended to move from high to low confidence compared to trials from low to high.
“Those double changes that did occur tended to move from high to low confidence (1.03, 0.35, 0.12, 1.36%) compared to trials from low to high (0.26, 0.04, 0.04, 0.26%; p<0.0001 for three subjects and p<0.05 for S3).”
Quote in the paperNot a substantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
- Findingown data
Subjects relied on information from the beginning of the display until ~400 ms before movement onset to guide both their direction choices and their confidence.
“Moreover, by discounting the lag and smoothing introduced by the motion filter (inset), it is apparent that subjects relied on information from the beginning of the display until ~400 ms before movement onset (arrows; see Materials and methods and Figure 4—figure supplement 1) to guide both their direction choices and their confidence.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The same information was used to reach an initial choice about direction and confidence.
- Findingown data
Changes of decision were more likely if the initial decision was an error than if it was correct.
“Changes of decision were more likely if the initial decision was an error than if it was correct (Figure 5a).”
Quote in the paperNot a substantial claim - Findingown data
A simple extension of the bounded accumulation model explains the frequency of changes of confidence and decision as well as their dependency on features of the stimulus and the subject's initial report.
“A simple extension of the bounded accumulation model explains the frequency of these changes as well as their dependency on features of the stimulus and the subject’s initial report.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Evidence was accumulated past the point of the initial decision for a fixed amount of time (tpip) that is less than the non-decision time.
- The authors state this as an assumption: The state of the belief at tpip determines the final confidence and choice.
- The authors state this as an assumption: Subjects might not apply identical criteria for confidence and choice in the pre- and post-initiation epoch (δθ parameters).
- The authors state this as an assumption: The model parameterization requires the pink and red regions to be same for both initial high and low confidence decisions.
- Findingown data
The curves in Figure 5 are fits of the model that capture the main features of the subjects' revisions of both choice and confidence.
“The curves in Figure 5 are fits of the model, which capture the main features of the subjects’ revisions of both choice and confidence.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Evidence was accumulated past the point of the initial decision for a fixed amount of time (tpip) that is less than the non-decision time.
- The authors state this as an assumption: The state of the belief at tpip determines the final confidence and choice.
- The authors state this as an assumption: Subjects might not apply identical criteria for confidence and choice in the pre- and post-initiation epoch (δθ parameters).
- The authors state this as an assumption: The model parameterization requires the pink and red regions to be same for both initial high and low confidence decisions.
- Findingown data
Changes of mind were beneficial to the participants in that on changes of mind trials the gain in points over not changing one's mind ranged from 0.72-1.12 points across the subjects.
“Changes of mind were beneficial to the participants in that on changes of mind trials (either decision, confidence or both) the gain in points over not changing one’s mind ranged from 0.72–1.12 points across the subjects (Table 2).”
Quote in the paperNot a substantial claim - Findingown data
There is a systematic discrepancy between the odds ratio pairs calculated from initial and final confidence ratings (P<1e-4, sign test).
“Clearly, there is a systematic discrepancy between the OR pairs (P<1e-4, sign test).”
Quote in the paperSubstantial claim - Findingown data
The variations in the participants' behavior are accounted for by the different settings of the model parameters.
“The variations in the participants’ behavior are accounted for by the different settings of the model parameters”
Quote in the paperSubstantial claim- The authors state this as an assumption: Evidence was accumulated past the point of the initial decision for a fixed amount of time (tpip) that is less than the non-decision time.
- The authors state this as an assumption: The state of the belief at tpip determines the final confidence and choice.
- The authors state this as an assumption: Subjects might not apply identical criteria for confidence and choice in the pre- and post-initiation epoch (δθ parameters).
- The authors state this as an assumption: The model parameterization requires the pink and red regions to be same for both initial high and low confidence decisions.
- Findingown data
When subjects changed their confidence rating from low to high, the motion information in the post-decision period supported the initial choice.
“when subjects changed their confidence rating from low to high, the motion information in the post-decision period supported the initial choice (Figure 4b, bottom)”
Quote in the paperSubstantial claim - Findingown data
After an initial low confidence decision, all subjects required more belief to change to a high confidence decision than would have been required for an initial high confidence decision (δθ1>0).
“After an initial low confidence decision (Figure 6, top row), all subjects required more belief to change to a high confidence decision than would have been required for an initial high confidence decision (δθ1>0).”
Quote in the paperSubstantial claim- The authors state this as an assumption: Evidence was accumulated past the point of the initial decision for a fixed amount of time (tpip) that is less than the non-decision time.
- The authors state this as an assumption: The state of the belief at tpip determines the final confidence and choice.
- The authors state this as an assumption: Subjects might not apply identical criteria for confidence and choice in the pre- and post-initiation epoch (δθ parameters).
- The authors state this as an assumption: The model parameterization requires the pink and red regions to be same for both initial high and low confidence decisions.
- Findingown data
Changes to high confidence were more likely when the initial low confidence accompanied a stronger motion.
“These changes to high confidence were more likely when the initial low confidence accompanied a stronger motion.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
- Findingnone given
These corrections were more likely if the motion information was stronger.
“These corrections were more likely if the motion information was stronger.”
Quote in the paperNot a substantial claim - Findingown data
The influence of motion fluctuations on choice and confidence becomes negligible a few hundred milliseconds before movement onset.
“The influence of motion fluctuations on choice and confidence becomes negligible a few hundred milliseconds before movement onset (Figure 4a).”
Quote in the paperSubstantial claim - Findingown data
The trend of motion information supporting the opposite direction for high-to-low confidence changes was amplified for changes of mind about direction.
“this trend was amplified for changes of mind about direction (Figure 4b, top)”
Quote in the paperSubstantial claim - Findingown data
Changes of both decision and confidence were less common (0.4-1.6%).
“Changes of both decision and confidence were less common (0.4–1.6%), which may be partly due to an energy cost associated with crossing the workspace (Burk et al., 2014; Moher and Song, 2014).”
Quote in the paperNot a substantial claim - Findingown data
Changes of confidence were more likely if the initial decision was low confidence than if it was high confidence.
“Further, changes of confidence were more likely if the initial decision was low confidence than if it was high confidence (Figure 5b).”
Quote in the paperSubstantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
- Findingown data
For all subjects, changes of confidence were more frequent than changes of decision.
“For all subjects, changes of confidence were more frequent than changes of decision (Table 2 and Figure 5c; p<1e-4 for all participants).”
Quote in the paperSubstantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
- Findingown data
All participants relaxed their belief criterion when they changed their direction decision while either maintaining or changing to high confidence (δθ3<0).
“Finally, all participants relaxed their belief criterion when they changed their direction decision while either maintaining or changing to high confidence (δθ3<0).”
Quote in the paperSubstantial claim- The authors state this as an assumption: Evidence was accumulated past the point of the initial decision for a fixed amount of time (tpip) that is less than the non-decision time.
- The authors state this as an assumption: The state of the belief at tpip determines the final confidence and choice.
- The authors state this as an assumption: Subjects might not apply identical criteria for confidence and choice in the pre- and post-initiation epoch (δθ parameters).
- The authors state this as an assumption: The model parameterization requires the pink and red regions to be same for both initial high and low confidence decisions.
- Interpretationown data
The discrepancy between confidence and accuracy can arise naturally because of simple processing delays.
“Here we show that this discrepancy can arise naturally because of simple processing delays.”
Quote in the paperSubstantial claim- The authors state this as an assumption: There is additional information available to the decision maker after committing to a choice and confidence category, despite the disappearance of the random dot motion upon response initiation.
- Premisereasoning
The first process to reach an upper bound determines the choice and decision time.
“The first process to reach an upper bound determines the choice and decision time.”
Quote in the paperSubstantial claim - Interpretationcitation
These regularities are consistent with a common mechanism of bounded evidence accumulation to support choice, reaction time and confidence.
“These regularities are consistent with a common mechanism of bounded evidence accumulation to support choice, reaction time and confidence (Kiani et al., 2014a).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Kiani et al., 2014a
Not checked in the free run.
- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
- The authors state this as an assumption: The same information was used to reach an initial choice about direction and confidence.
- Premisereasoning
Since the winning process is at a fixed bound, the confidence is adequately summarized by the height of this bound, decision time, and the state of the losing accumulator.
“Since the winning process is at a fixed bound, the confidence is adequately summarized by the height of this bound, decision time, and the state of the losing accumulator.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
- Premisenone given
Changes of mind occur occasionally even when there appears to be no additional information available after the initial decision has been rendered.
“Such changes of mind occur occasionally, even when there appears to be no additional information available after the initial decision has been rendered.”
Quote in the paperSubstantial claim - Interpretationown data
The similar time course of the confidence and choice kernels implies that the initial direction and confidence choices were supported by a common evidence stream.
“The similar time course of the confidence and choice kernels implies that the initial direction and confidence choices were supported by a common evidence stream.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The same information was used to reach an initial choice about direction and confidence.
- Premisecitation
The main distinction of the model is the recognition that the time to decision influences confidence.
“Its main distinction is the recognition that the time to decision influences confidence (Fetsch et al., 2014; Kiani et al., 2014a).”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Fetsch et al., 2014 source
Not checked in the free run.
- cites Kiani et al., 2014a
Not checked in the free run.
- Interpretationreasoning
Changes of confidence arising through post-decision processing can contribute to an apparent dissociation between confidence and decision.
“However, changes of confidence arising through post-decision processing can contribute to an apparent dissociation between confidence and decision.”
Quote in the paperSubstantial claim- The authors state this as an assumption: There is additional information available to the decision maker after committing to a choice and confidence category, despite the disappearance of the random dot motion upon response initiation.
- Interpretationown data
The changes of mind were explained by a continuation of the mechanism that led to the initial choice.
“These changes of mind were explained by a continuation of the mechanism that led to the initial choice.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The same information was used to reach an initial choice about direction and confidence.
- The authors state this as an assumption: There is additional information available to the decision maker after committing to a choice and confidence category, despite the disappearance of the random dot motion upon response initiation.
- Premisecitation
The same sequential sampling framework explains the confidence that a decision is correct.
“The same framework also explains the confidence that a decision is correct (Kiani and Shadlen, 2009; Kiani et al., 2014a).”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Kiani and Shadlen, 2009 source
Not checked in the free run.
- cites Kiani et al., 2014a
Not checked in the free run.
- Premisecitation
The process involves a race between a mechanism that accumulates evidence for right, and against left, and a process that integrates evidence for left, and against right.
“The process involves a race between a mechanism that accumulates evidence for right, and against left, and a process that integrates evidence for left, and against right (Usher and McClelland, 2001; Mazurek et al., 2003).”
Quote in the paperSubstantial claim0 of 2 sources checked - Premisereasoning
The sequential sampling framework offers a natural account of changes of mind because the mechanism incorporates processing delays, which leave open the possibility that the brain might have access to additional evidence that did not influence the initial decision and which might instead influence a revision.
“The sequential sampling framework (e.g., bounded evidence accumulation) offers a natural account of this phenomenon, because the mechanism incorporates processing delays, which leave open the possibility that the brain might have access to additional evidence that did not influence the initial decision and which might instead influence a revision.”
Quote in the paperSubstantial claim - Premisereasoning
The state of both the winning and losing processes as well as decision time confer an expectation that a decision rendered on the evidence is likely to be correct, what we term confidence or belief.
“The state of both the winning and losing processes as well as decision time (Figure 3b shows the races on the same plot) confer an expectation that a decision rendered on the evidence is likely to be correct, what we term confidence or belief.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
- Premisereasoning
Since these processes obtain evidence originating in the same stimulus, they will tend to accumulate noisy evidence of opposite sign (i.e., anti-correlated).
“Since these processes obtain evidence originating in the same stimulus, they will tend to accumulate noisy evidence of opposite sign (i.e., anti-correlated).”
Quote in the paperSubstantial claim - Premisecitation
When the decision maker terminates deliberation, the choice is accompanied by a degree of certainty (confidence) based on the same stream of evidence that supported that decision.
“Thus, when the decision maker terminates deliberation, the choice is accompanied by a degree of certainty (i.e., confidence), based on the same stream of evidence that supported that decision (Fetsch et al., 2014).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Fetsch et al., 2014 source
Not checked in the free run.
- Premisereasoning
The degree of belief is based on the balance of evidence (i.e., difference) between winning and losing DVs, as well as the decision time.
“Thus the degree of belief is based on the balance of evidence (i.e., difference) between winning and losing DVs, as well as the decision time.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
- Interpretationown data
The findings extend the sequential sampling framework to vacillation about confidence.
“Our findings extend the sequential sampling framework to vacillation about confidence and invites caution in interpreting dissociations between confidence and accuracy.”
Quote in the paperSubstantial claim - Interpretationreasoning
Had subjects been asked only for their confidence rating after they indicated their initial choice-without an opportunity to revise those choices-a different relationship between confidence and accuracy would have been detected.
“In other words, had we asked our subjects only for their confidence rating after they indicated their initial choice—without an opportunity to revise those choices—we would have detected a different relationship between confidence and accuracy (Figure 7).”
Quote in the paperSubstantial claim - Interpretationown data
The model demonstrates that on many trials, the final confidence judgments are clearly based on additional information that did not affect the initial choice.
“but it demonstrates, nonetheless, that on many trials, the final confidence judgments are clearly based on additional information that did not affect the initial choice.”
Quote in the paperSubstantial claim - Interpretationcitation
The initial choice and confidence are explained using the identical stream of information.
“We explain the initial choice and confidence using the identical stream of information, as in a previous study (Kiani et al., 2014a).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Kiani et al., 2014a
Not checked in the free run.
- Interpretationown data
Deliberation in the post-initiation epoch is as rich a process as the deliberation preceding initiation.
“First, it establishes that deliberation in the post-initiation epoch is as rich a process as the deliberation preceding initiation.”
Quote in the paperSubstantial claim - Interpretationown data
Confidence and direction reports remain coupled via a common mechanism at the time of both the initial and final decision.
“We wish to emphasize that confidence and direction reports remain coupled via a common mechanism at the time of both the initial and final decision.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Choice and confidence are related but processed as if separate attributes, via a correspondence between decision variable and belief.
- Interpretationown data
Confidence, choice and reaction time can be understood in a common framework of bounded evidence accumulation.
“We conclude that confidence, choice and reaction time can be understood in a common framework of bounded evidence accumulation.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Choice and confidence are related but processed as if separate attributes, via a correspondence between decision variable and belief.
- Interpretationown data
The finding explains a potential dissociation between the information that decision makers might use to guide confidence and choice.
“Second, the finding explains a potential dissociation between the information that decision makers might use to guide confidence and choice.”
Quote in the paperSubstantial claim - Interpretationreasoning
When confidence is assessed after the initial choice, choice and confidence are explained by only partially overlapping sources of information.
“When confidence is assessed after the initial choice, as it is in most experiments, then it ought to come as no surprise that choice and confidence are explained by only partially overlapping sources of information.”
Quote in the paperSubstantial claim - Premisecitation
The state of the winning and losing race together with decision time maps directly to the log-odds of being correct.
“The state of both the winning and losing race together with decision time map directly to the log-odds of being correct (see Kiani et al., 2014a).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Kiani et al., 2014a
Not checked in the free run.
- The authors state this as an assumption: Momentary evidence is drawn from a bivariate Gaussian with mean (κC, -κC).
- The authors state this as an assumption: The two accumulators share noise with negative covariance ρ.
- The authors state this as an assumption: A decision is made when one of the two races crosses a decision bound B.
- Interpretationown data
In our study, additional information does not affect confidence in a manner fundamentally different from the process that ties choice and confidence together.
“In our study, the answer seems to be negative”
Quote in the paperSubstantial claim - Interpretationreasoning
The present study extends previous observations to the metacognitive operation of assigning a degree of belief that the decision is correct.
“The present study extends these observations to the metacognitive operation of assigning a degree of belief that the decision is correct.”
Quote in the paperSubstantial claim
- Classified as probably not a substantial claim. 32
- May combine two claims. 26
- The quote only partly states this claim. 1
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataSubjects differed in their sensitivity, speed/accuracy and confidence which was accounted for by the parameters of the fits.
“Naturally, subjects differed in their sensitivity, speed/accuracy and confidence which was accounted for by the parameters of the fits (Table 1).”
Quote in the paperNot a substantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataSubjects differed in their criteria for categorizing degree of belief into high and low.
“They also differed in their criteria for categorizing degree of belief into high and low, which can be appreciated by the separation of choice functions by confidence report.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
Classified as probably not a substantial claim.
Findingown dataThe smooth curves in Figure 2 are fits of the model (4 parameters) which capture the main features of the subjects' choices.
“The smooth curves in Figure 2 are fits of the model (4 parameters; see Materials and methods), which capture the main features of the subjects’ choices.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
May combine two claims.
Findingown dataThe inferred criteria for all the subjects were close to this optimal criterion, although all subjects were somewhat risk-averse.
“Interestingly the inferred criteria for all the subjects were close to this optimal criterion, although all subjects were somewhat risk-averse (probability thresholds of 0.71, 0.78, 0.71 and 0.75; thresholds in log-odd units shown in Table 1).”
Quote in the paperNot a substantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
May combine two claims.
Findingown dataThere are some noteworthy discrepancies between model and data, but the model captures the trend in the confidence ratings on error trials.
“There are some noteworthy discrepancies between model and data (e.g., proportion of high confidence choices at 0% coherence for 3 of 4 subjects), but the model captures the trend in the confidence ratings on error trials, mentioned above (3 of 4 subjects; Figure 2c), even though the fits themselves are dominated by the more numerous correct choices.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
May combine two claims.
Findingown dataOn a small fraction of trials, subjects changed their initial decision about the direction of motion and, more frequently, about their confidence.
“On a small fraction of trials, subjects changed their initial decision about the direction of motion and, more frequently, about their confidence.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataConfidence and choice are informed by the same evidence, both at the initial choice and on any subsequent revision.
“We show that confidence and choice are informed by the same evidence, both at the initial choice and on any subsequent revision.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The same information was used to reach an initial choice about direction and confidence.
- The authors state this as an assumption: There is additional information available to the decision maker after committing to a choice and confidence category, despite the disappearance of the random dot motion upon response initiation.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThere was a subtle bias for the bottom targets (50.6, 51.9, 52.5 and 51.4%, respectively), possibly due to kinematic factors, but the orientation was balanced across the experiment.
“There was a subtle bias for the bottom targets (50.6, 51.9, 52.5 and 51.4%, respectively), possibly due to kinematic factors, but recall that the orientation was balanced across the experiment.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataSubject 4 showed a qualitative change in behavior on the second half of their data, reporting high confidence on nearly all trials.
“Subject 4 showed a qualitative change in behavior on the second half of their data (after ~5000 trials; Figure 2—figure supplement 2a), reporting high confidence on nearly all trials.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenThe model overestimates confidence at the lowest motion strength.
“The model is by no means perfect (e.g., it overestimates confidence at the lowest motion strength)”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataJust a few hundred milliseconds of additional evidence led to changes in the confidence associated with the initial choice, ranging from 1.5-6.3% of trials across subjects, and this was enough to induce a clear departure from the level of confidence that ought to be associated with the evidence available at the time of the initial choice.
“Just a few hundred milliseconds of additional evidence led to changes in the confidence associated with the initial choice, ranging from 1.5–6.3% of trials across our subjects, and this was enough to induce a clear departure from the level of confidence that ought to be associated with the evidence available at the time of the initial choice.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataErrors were less frequent with stronger motion and less frequent than correct responses.
“Of course, errors were less frequent with stronger motion and less frequent than correct responses (Figure 2a).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe model could still fit the initial choices for the omitted second half of Subject 4's data.
“However, our model could still fit the initial choices for the omitted second half of this participant's data (Figure 2—figure supplement 2b).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe frequency of changes of mind varied between subjects, from 2.0 to 8.8% of trials.
“The frequency of these changes of mind varied between subjects, from 2.0 to 8.8% of trials.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe model can fit the initial choices of the excluded subject despite idiosyncrasy in their data.
“We found that despite the idiosyncrasy in this subject’s data, our model can fit their initial choices (Figure 2—figure supplement 1).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataTime-dependent confidence bounds improve the fits for all subjects but do not affect the figures visibly or the conclusions.
“We have also fit data using time-dependent confidence bounds, which improve the fits for all subjects but does not affect the figures visibly and does not affect our conclusions.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningThe model explains why confidence associated with errors increases with stronger coherences: stronger motion induces faster decision times for correct and errors alike.
“For example, this explains why confidence associated with errors increases with stronger coherences: stronger motion induces faster decision times for correct and errors alike.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
Classified as probably not a substantial claim.
PremisecitationEvidence for a process of post-decision evidence accumulation was adduced to explain reversals of perceptual decisions in humans and monkeys.
“Evidence for such a process was adduced to explain reversals of perceptual decisions in humans and monkeys (Rabbitt, 1966; Rabbitt and Vyas, 1981; McPeek et al., 2000; Caspi and Beutter, 2004; Van Zandt and Maldonado-Molina, 2004; Resulaj et al., 2009; Burk et al., 2014; Kiani et al., 2014b; Moher and Song, 2014).”
Quote in the paperSubstantial claim0 of 3 sources checkedClassified as probably not a substantial claim.
PremisecitationConfidence in a decision and the decision itself are dissociable in many instances.
“This last point remains controversial, however, for there are many instances when the confidence in a decision and the decision itself are dissociable.”
Quote in the paperSubstantial claim0 of 3 sources checked- cites Fischoff et al., 1982
Not checked in the free run.
- cites Baranski and Petrusic, 1994
Not checked in the free run.
- cites Kunimoto et al., 2001 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationThe model has been tested in neurophysiology and a previous report in humans.
“This model has been tested in neurophysiology and a previous report in humans (Kiani et al., 2014a) which extends ideas from signal detection theory, race, and Bayesian models (Vickers, 1979; Kepecs et al., 2008; Maniscalco and Lau, 2012; Drugowitsch et al., 2014).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Kiani et al., 2014a
Not checked in the free run.
Classified as probably not a substantial claim.
PremisecitationSequential sampling with optional stopping models explains the tradeoff between speed and accuracy in decisions.
“The ensuing tradeoff between speed and accuracy is explained by sequential sampling with optional stopping models in which evidence is accumulated to some stopping criterion or bound (Link, 1975; Ratcliff and Rouder, 1998).”
Quote in the paperSubstantial claim0 of 2 sources checkedProbably not a substantial claim: may not pass S3 (load-bearing).
PremisereasoningTo maximize the number of points, given the reward structure, subjects should choose high confidence if they believe the probability of a correct choice is greater than two-thirds, which corresponds to a log-odds of 0.69.
“To maximize the number of points, given the reward structure (see Materials and methods), subjects should choose high confidence if they believe the probability of a correct choice is greater than two-thirds, which corresponds to a log-odds of 0.69.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
PremisecitationThe decision variable, when combined with elapsed decision time, maps to the probability that a decision rendered on its value will be correct.
“This is because the quantity that is accumulated, termed a decision variable (DV), when combined with elapsed decision time, maps to the probability that a decision rendered on its value will be correct (Kiani and Shadlen, 2009; Drugowitsch et al., 2014).”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Kiani and Shadlen, 2009
Not checked in the free run.
- cites Drugowitsch et al., 2014 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationHuman decision makers can perform above chance level yet report they are guessing.
“and they can perform above chance level yet report they are guessing (Kunimoto et al., 2001).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Kunimoto et al., 2001 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationHuman decision makers tend to overestimate their certainty about choices based on truly ambiguous evidence.
“For example, human decision makers tend to overestimate their certainty about choices based on truly ambiguous evidence (Fischoff et al., 1982; Baranski and Petrusic, 1994; Erev et al., 1994; Drugowitsch et al., 2014; Kiani et al., 2014a)”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites Fischoff et al., 1982
Not checked in the free run.
- cites Baranski and Petrusic, 1994
Not checked in the free run.
- cites Erev et al., 1994 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationThe fixed non-decision time standard deviation of 60 ms is within the normal range from a previous study and is supported by an analysis of motion energy.
“To further reduce the degrees of freedom of the model to four we fixed both the covariance ρ to -0.5 ≈ -0.71 and non-decision time standard deviation σtnd to 60 ms (within the normal range from a previous study (Burk et al., 2014) and supported by an analysis of motion energy; see Figure 1—figure supplement 1).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Burk et al., 2014 source
Not checked in the free run.
- The authors state this as an assumption: Reaction time is determined by the time to reach the decision bound plus a non-decision time modeled as a normal distribution with mean μtnd and standard deviation σtnd.
May combine two claims.
Interpretationown dataThe observation supports a parsimonious account of decision confidence which is somewhat at odds with recent literature on meta-cognition.
“The observation supports a parsimonious account of decision confidence which is somewhat at odds with recent literature on meta-cognition (see Discussion).”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductcitationA flat decision bound does not capture the shape of the reaction-time distributions and the mean reaction time on error trials.
“For analytic simplicity (see below), we used a flat bound (i.e., stationary rather than collapsing), which does not capture the shape of the RT distributions and the mean RT on error trials (Drugowitsch et al., 2012).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Drugowitsch et al., 2012 source
Not checked in the free run.
May combine two claims.
Interpretationown dataThe discrepancy between odds ratios might be regarded as a sign of 2nd-order confidence, whereas it is simply the result of a continuation of the 1st-order process that couples choice and confidence, via bounded evidence accumulation.
“This discrepancy might be regarded as a sign of 2nd-order confidence, whereas it is simply the result of a continuation of the 1st-order process that couples choice and confidence, via bounded evidence accumulation.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataThe optimization procedure converged to a global maximum, as indicated by minimal variability across runs.
“The variability across runs was minimal, suggesting that the optimization procedure converged to global maximum.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductown dataThe arrangement of high/low confidence targets at top/bottom had no detectable effects on choice accuracy and only small effects on reaction time for 2 subjects.
“We show combined data for the two target configurations (high/low confidence targets at top/bottom), having reassured ourselves that the arrangement had no detectable effects on choice accuracy (p≥0.4; Fisher’s exact test) and only small effects on reaction time (RT) for 2 subjects (magnitudes<3%, p<0.01; t-test).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationThe class of bounded sequential sampling models explains the tradeoff between speed and accuracy, and many of the essential steps have correlates in neurophysiology.
“The class of bounded sequential sampling models, including many variants (Wald, 1947; Stone, 1960; Laming, 1968; Link, 1975; Good, 1979; Luce, 1986; Ratcliff and Rouder, 1998) explains the tradeoff between speed and accuracy, and many of the essential steps have correlates in neurophysiology (Romo et al., 2002; Heekeren et al., 2004; Ploran et al., 2007; Heitz and Schall, 2012; White et al., 2012;”
Quote in the paperNot a substantial claim0 of 3 sources checkedMay combine two claims.
InterpretationreasoningTermination has a kind of specificity; it marks the end of deliberation for purposes of response initiation, but it does not actually terminate all deliberation.
“This implies that termination has a kind of specificity; it marks the end of deliberation for purposes of response initiation, but it does not actually terminate all deliberation.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
PremisecitationIn previous studies, the motor system receives a continuous updating of the state of the evidence leading to a choice and can utilize late arriving information to revise an initial choice by continuing the process of deliberation.
“In previous studies, we showed that the motor system receives a continuous updating of the state of the evidence leading to a choice (Gold and Shadlen, 2000; Selen et al., 2012) and can utilize the late arriving information to revise an initial choice by continuing the process of deliberation (Resulaj et al., 2009; Burk et al., 2014).”
Quote in the paperSubstantial claim0 of 3 sources checkedMay combine two claims.
Interpretationown dataThe present study establishes that confidence assessment evolves in time and, like the choice itself, can undergo revision with additional evidence.
“The present study establishes that this assessment evolves in time and, like the choice itself, can undergo revision with additional evidence.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationUsing the analytic method precluded the use of collapsing decision bounds.
“Using this analytic method precluded the use of collapsing decision bounds (and no lower reflecting bounds; cf., Kiani et al., 2014a).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Kiani et al., 2014a
Not checked in the free run.
May combine two claims.
InterpretationcitationThis does not imply a dissociation between confidence and the decision, as the latter can also be revised based on the same additional information.
“Importantly, however, this does not imply a dissociation between confidence and the decision (Pleskac and Busemeyer, 2010), as the latter can also be revised based on the same additional information.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Pleskac and Busemeyer, 2010
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims. The quote only partly states this claim.
Study design and conductown dataBasing the change of decision about direction on log odds is a simplification, and the frequency of change of mind about choice and confidence is too small to evaluate a more flexible mechanism.
“In principle, such a mechanism could terminate post-initiation processing more flexibly, but the frequency of change of mind about choice and confidence is too small to evaluate this possibility.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: Once the initial decision has been made, the accumulators continue to integrate information that was not accessible at the time of the initial decision for a further post-initiation period tpip.
- The authors state this as an assumption: A change in confidence or choice occurs if at the end of the post-initiation period the log odds crosses a confidence or choice threshold.
May combine two claims.
InterpretationcitationThe operations leading to confidence seem neither mysterious nor dissociable from the decision process.
“Yet the operations leading to confidence seem neither mysterious nor dissociable from the decision process (cf. De Martino et al., 2013).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites De Martino et al., 2013 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationcitationThe evaluation of additional evidence may explain why confidence ratings are more strongly correlated with accuracy when they are reported with less time pressure, and why some individuals appear to be better than others at discriminating correct from incorrect decisions.
“The evaluation of additional evidence may explain why confidence ratings are more strongly correlated with accuracy when they are reported with less time pressure (Yu et al., 2015), and why some individuals appear to be better than others at discriminating correct from incorrect decisions (e.g. Ais et al., 2016).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Yu et al., 2015
Not checked in the free run.
- cites Ais et al., 2016 source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningThe odds ratios calculated from the initial confidence report establish a baseline analogous to meta-d′ equal to d′ in the signal detection theory framework.
“The ORs calculated from the initial confidence report establish a baseline: the breakdown of confidence associated with the information that explains the accuracy at each motion strength, analogous to meta-d′ equal to d′ in the SDT framework described above.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Premisenone givenThe meta-d′ statistic is not well behaved in the sequential sampling framework, but the basic logic remains applicable.
“The meta-d′ statistic is not well behaved in the sequential sampling framework, but the basic logic remains applicable (see Materials and methods).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe chosen covariance of -0.5 is within the normal range of fitted covariances.
“This choice of covariance assumes that about half the noise is shared between the two accumulators (i.e. arises from the stimulus) and is within the normal range of fitted covariances (Kiani et al., 2014a).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Kiani et al., 2014a
Not checked in the free run.
- The authors state this as an assumption: About half the noise is shared between the two accumulators (arises from the stimulus).
Probably not a substantial claim: may not pass S2 (checkable).
PremisereasoningThe signal detection theory framework must be extended to account for reaction time and choice.
“However, the SDT framework must be extended to account for RT and choice.”
Quote in the paperNot a substantial claim- Rests on an assumption: Under sequential sampling models, meta-d′ ≠ d′ even when choice and confidence are explained by the same decision variable.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationEvidence for a meta-cognitive process is adduced from a dissociation between the signal-to-noise properties of the evidence that would support a level of choice accuracy versus the level associated with a degree of confidence.
“Evidence for a meta-cognitive process is adduced from a dissociation between the signal-to-noise properties of the evidence that would support a level of choice accuracy versus the level associated with a degree of confidence (e.g., meta-d′; see Fleming and Lau, 2014; Maniscalco and Lau, 2012).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Fleming and Lau, 2014 source
Not checked in the free run.
- cites Maniscalco and Lau, 2012
Not checked in the free run.
The authors state this as an assumption.
Subjects adopt a consistent criterion on degree of belief (log odds) to decide in favor of high or low confidence.
“For simplicity, we assumed that subjects adopt a consistent criterion on “degree of belief” (log odds, Figure 3b) to decide in favor of high or low confidence.”
The authors state this as an assumption.
The same information was used to reach an initial choice about direction and confidence.
“The critical assumptions are (i) the same information was used to reach an initial choice about direction and confidence, and (ii) there is additional information available to the decision maker after committing to a choice and confidence category, despite the disappearance of the random dot motion upon response initiation.”
The authors state this as an assumption.
There is additional information available to the decision maker after committing to a choice and confidence category, despite the disappearance of the random dot motion upon response initiation.
“The critical assumptions are (i) the same information was used to reach an initial choice about direction and confidence, and (ii) there is additional information available to the decision maker after committing to a choice and confidence category, despite the disappearance of the random dot motion upon response initiation.”
The authors state this as an assumption.
Evidence was accumulated past the point of the initial decision for a fixed amount of time (tpip) that is less than the non-decision time.
“We assumed that evidence was accumulated past the point of the initial decision for a fixed amount of time (a free parameter, tpip) that is less than the non-decision time.”
The authors state this as an assumption.
The state of the belief at tpip determines the final confidence and choice.
“The state of the belief at tpip determines the final confidence and choice (see Materials and methods).”
The authors state this as an assumption.
Subjects might not apply identical criteria for confidence and choice in the pre- and post-initiation epoch (δθ parameters).
“we allowed for the possibility that subjects might not apply the identical criteria for confidence and choice in the pre- and post-initiation epoch (the δθ parameters; see Materials and methods).”
The authors state this as an assumption.
Choice and confidence are related but processed as if separate attributes, via a correspondence between decision variable and belief.
“To explain our findings, we assumed that choice and confidence are related but processed as if separate attributes, via a correspondence between decision variable and belief.”
The authors state this as an assumption.
The model parameterization requires the pink and red regions to be same for both initial high and low confidence decisions.
“Note that the model parameterization requires the pink and red regions to be same for both initial high and low confidence decisions.”
The paper relies on this without support.
Non-decision time typically exceeds 300 ms even in relatively fast perceptual decisions.
“Even in relatively fast perceptual decisions, this so-called non-decision time typically exceeds 300 ms.”
The paper relies on this without support.
The temporal stream of evidence in random dot kinematograms is effectively a sequence of independent, statistically stationary samples.
“the temporal stream of evidence is effectively a sequence of independent, statistically stationary samples.”
The paper relies on this without support.
There are processing delays between the arrival of information from the environment and updating its representation in working memory as a decision variable, and there are delays between commitment to a decision and communicating this decision through the motor system.
“Specifically, there are processing delays between the arrival of information from the environment and updating its representation in working memory as a decision variable bearing on a proposition, and there are delays between commitment to a decision and communicating this decision through the motor system.”
The authors state this as an assumption.
About half the noise is shared between the two accumulators (arises from the stimulus).
“This choice of covariance assumes that about half the noise is shared between the two accumulators (i.e. arises from the stimulus) and is within the normal range of fitted covariances (Kiani et al., 2014a).”
The authors state this as an assumption.
Once the initial decision has been made, the accumulators continue to integrate information that was not accessible at the time of the initial decision for a further post-initiation period tpip.
“To model changes of mind, we assumed that once the initial decision had been made, the accumulators continued to integrate information that was not accessible at the time of the initial decision (due to latencies in the sensory and motor system) for a further post-initiation period tpip (constrained to be less than tnd).”
The authors state this as an assumption.
A change in confidence or choice occurs if at the end of the post-initiation period the log odds crosses a confidence or choice threshold.
“A change in confidence or choice would occur if at the end of this period the log odds crossed a confidence or choice threshold, respectively (Figure 3b).”
The paper relies on this without support.
There are motor costs involved in changes of mind.
“Since there are motor costs involved in changes of mind, we included additional parameters that could move the confidence and choice thresholds away from their initial values in the post-initiation stage by δθ1, δθ2 and δθ3 (Figure 3b).”
The paper relies on this without support.
Under sequential sampling models, meta-d′ ≠ d′ even when choice and confidence are explained by the same decision variable.
“Under sequential sampling (e.g., bounded drift diffusion), meta-d′ ≠ d′, even when the choice and confidence are explained by the same decision variable.”
The authors state this as an assumption.
Momentary evidence is drawn from a bivariate Gaussian with mean (κC, -κC).
“Momentary evidence was modeled as draws from a bivariate Gaussian with a mean that depends on the coherence (C) such that drift rate (/s) is κC for the rightward and -κC for the leftward accumulator, giving a mean of (κC,-κC).”
The authors state this as an assumption.
The two accumulators share noise with negative covariance ρ.
“The bivariate Gaussian has a negative covariance ρ which determines the extent to which the two accumulators share noise (for example arising from fluctuations in the stimulus).”
The authors state this as an assumption.
A decision is made when one of the two races crosses a decision bound B.
“A decision is made when one of the two races crosses a decision bound B.”
The authors state this as an assumption.
Reaction time is determined by the time to reach the decision bound plus a non-decision time modeled as a normal distribution with mean μtnd and standard deviation σtnd.
“The reaction time is determined by the time to reach the decision bound and an additional non-decision time (e.g., due to sensory and motor latencies) which was modeled as a normal distribution with mean μtnd and standard deviation σtnd.”
The authors state this as an assumption.
Confidence is modeled with a log-odds threshold θ separating high from low confidence judgments.
“To model confidence, we included a log-odds threshold θ which separated high from low confidence judgments.”
The authors state this as an assumption.
The confidence threshold is time-invariant.
“For simplicity, we chose a time-invariant threshold.”
The paper relies on this without support.
Fluctuations have a stronger impact when motion is weak.
“Because fluctuations have a stronger impact when motion is weak, only the lowest motion strengths (≤6.4% coh) were included in these analyses.”
The paper relies on this without support.
Due to the stochastic nature of the motion stimuli, the strength of motion varies from trial to trial and within a trial.
“Due to the stochastic nature of the motion stimuli, the strength of motion will vary from trial to trial, and even within a trial.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Variation in post-decision confidence predicts how likely the decision is to be revised if an opportunity arises.
“Variation in post-decision confidence predicts how likely the decision is to be revised if an opportunity arises,9”
From Two prefrontal-subcortical circuits estimate controllability and reflect its impact on learning (2026) · cites this paper as [11]
The Results (Changes of mind; page unavailable) link post-decision evidence to both confidence changes and reversals of choice. They do not show that variation in post-decision confidence itself predicts the likelihood of revising a decision; the reported probability of changing direction is conditioned on whether the initial choice was correct, not on post-decision confidence. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Cites
The Neural Basis of Decision Making
Gold, J. I., & Shadlen, M. N. (2007) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.29.051605.113038 · Cited by 3,764 · see citing works
Self-citationSpatiotemporal energy models for the perception of motion
Adelson, E. H., & Bergen, J. R. (1985) · Journal of the Optical Society of America A · doi:10.1364/josaa.2.000284 · Cited by 3,442 · see citing works
The time course of perceptual choice: The leaky, competing accumulator model.
Usher, M., & McClelland, J. L. (2001) · Psychological Review · doi:10.1037/0033-295x.108.3.550 · Cited by 2,152 · see citing works
(1986) · Cited by 2,022 · see citing works
Calibration of probabilities: the state of the art to 1980
Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982) · Judgment under Uncertainty · doi:10.1017/cbo9780511809477.023 · Cited by 1,967 · see citing works
The analysis of visual motion: a comparison of neuronal and psychophysical performance
Britten, K., Shadlen, M., Newsome, W., & Movshon, J. (1992) · The Journal of Neuroscience · doi:10.1523/jneurosci.12-12-04745.1992 · Cited by 1,958 · see citing works
Self-citationRoitman, J. D., & Shadlen, M. N. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-21-09475.2002 · Cited by 1,589 · see citing works
Self-citationModeling Response Times for Two-Choice Decisions
Ratcliff, R., & Rouder, J. N. (1998) · Psychological Science · doi:10.1111/1467-9280.00067 · Cited by 1,478 · see citing works
Fleming, S. M., & Lau, H. C. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00443 · Cited by 1,386 · cited 2× · see citing works
“Nor do we know how a criterion is applied to this mapping to render the categorical 354 choices in our study, or in post-decision wagering decisions (Kiani and Shadlen, 2009), or in 355 confidence ratings (Maniscalco and Lau, 2012; Fleming and Lau, 2014; Kiani et al., 2014a).”
BackgroundMethodRepresentation of Confidence Associated with a Decision by Neurons in the Parietal Cortex
Kiani, R., & Shadlen, M. N. (2009) · Science · doi:10.1126/science.1169405 · Cited by 1,027 · see citing works
Self-citationInformation theory of choice-reaction times
Laming, D. (1968) · Medical Entomology and Zoology · Cited by 952 · see citing works
Maniscalco, B., & Lau, H. (2012) · Consciousness and Cognition · doi:10.1016/j.concog.2011.09.021 · Cited by 883 · see citing works
Neural correlates, computation and behavioural impact of decision confidence
Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008) · Nature · doi:10.1038/nature07200 · Cited by 809 · see citing works
The neural basis of metacognitive ability
Fleming, S. M., & Dolan, R. J. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2011.0417 · Cited by 747 · cited 1× · see citing works
“DOI: 10.7554/eLife.12192.001”
InfluentialA general mechanism for perceptual decision-making in the human brain
Heekeren, H. R., Marrett, S., Bandettini, P. A., & Ungerleider, L. G. (2004) · Nature · doi:10.1038/nature02966 · Cited by 741 · see citing works
Calibration of Probabilities: The State of the Art
Lichtenstein, S., Fischhoff, B., & Phillips, L. (1977) · Springer eBooks · doi:10.1007/978-94-010-1276-8_19 · Cited by 736 · see citing works
Two-stage dynamic signal detection: A theory of choice, decision time, and confidence.
Pleskac, T. J., & Busemeyer, J. R. (2010) · Psychological Review · doi:10.1037/a0019737 · Cited by 722 · see citing works
Models for choice-reaction time
Stone, M. (1960) · Psychometrika · doi:10.1007/bf02289729 · Cited by 696 · see citing works
Changes of mind in decision-making
Resulaj, A., Kiani, R., Wolpert, D. M., & Shadlen, M. N. (2009) · Nature · doi:10.1038/nature08275 · Cited by 678 · see citing works
Self-citationDecision-making with multiple alternatives
Churchland, A. K., Kiani, R., & Shadlen, M. N. (2008) · Nature Neuroscience · doi:10.1038/nn.2123 · Cited by 671 · see citing works
Self-citationDecision Making as a Window on Cognition
Shadlen, M. N., & Kiani, R. (2013) · Neuron · doi:10.1016/j.neuron.2013.10.047 · Cited by 660 · see citing works
Self-citationSimultaneous over- and underconfidence: the role of error in judgment processes
Erev, I., Wallsten, T. S., & Budescu, D. V. (1994) · Psychological Review · doi:10.1037/0033-295x.101.3.519 · Cited by 644 · see citing works
A role for neural integrators in perceptual decision making
Mazurek, M. E., Roitman, J. D., Ditterich, J., & Shadlen, M. N. (2003) · Cerebral Cortex · doi:10.1093/cercor/bhg097 · Cited by 620 · see citing works
Self-citationRepresentation of a perceptual decision in developing oculomotor commands
Gold, J. I., & Shadlen, M. N. (2000) · Nature · doi:10.1038/35006062 · Cited by 616 · see citing works
Self-citationRats and Humans Can Optimally Accumulate Evidence for Decision-Making
Brunton, B. W., Botvinick, M., & Brody, C. D. (2013) · Science · doi:10.1126/science.1233912 · Cited by 607 · see citing works
The Cost of Accumulating Evidence in Perceptual Decision Making
Drugowitsch, J., Moreno-Bote, R., Churchland, A. K., Shadlen, M. N., & Pouget, A. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.4010-11.2012 · Cited by 577 · see citing works
Self-citationConfidence in value-based choice
De Martino, B., Fleming, S. M., Garrett, N., & Dolan, R. J. (2013) · Nature Neuroscience · doi:10.1038/nn.3279 · Cited by 534 · see citing works
Decision Processes in Visual Perception
(1979) · Elsevier eBooks · doi:10.1016/c2013-0-11654-6 · Cited by 511 · see citing works
Kiani, R., Hanks, T. D., & Shadlen, M. N. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4761-07.2008 · Cited by 500 · see citing works
Self-citationDistinct relationships of parietal and prefrontal cortices to evidence accumulation
Hanks, T. D., Kopec, C. D., Brunton, B. W., Duan, C. A., Erlich, J. C., & Brody, C. D. (2015) · Nature · doi:10.1038/nature14066 · Cited by 486 · see citing works
Choice Certainty Is Informed by Both Evidence and Decision Time
Kiani, R., Corthell, L., & Shadlen, M. N. (2014) · Neuron · doi:10.1016/j.neuron.2014.12.015 · Cited by 470 · see citing works
Self-citationInternal and External Influences on the Rate of Sensory Evidence Accumulation in the Human Brain
Kelly, S. P., & O'Connell, R. G. (2013) · The Journal of Neuroscience · doi:10.1523/jneurosci.3355-13.2013 · Cited by 455 · see citing works
Galvin, S. J., Podd, J. V., Drga, V., & Whitmore, J. (2003) · Psychonomic Bulletin & Review · doi:10.3758/bf03196546 · Cited by 405 · see citing works
Elapsed Decision Time Affects the Weighting of Prior Probability in a Perceptual Decision Task
Hanks, T. D., Mazurek, M. E., Kiani, R., Hopp, E., & Shadlen, M. N. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.5613-10.2011 · Cited by 378 · see citing works
Self-citationNeural Mechanisms of Speed-Accuracy Tradeoff
Heitz, R. P., & Schall, J. D. (2012) · Neuron · doi:10.1016/j.neuron.2012.08.030 · Cited by 373 · see citing works
The calibration and resolution of confidence in perceptual judgments
Baranski, J. V., & Petrusic, W. M. (1994) · Perception & Psychophysics · doi:10.3758/bf03205299 · Cited by 308 · see citing works
A computational framework for the study of confidence in humans and animals
Kepecs, A., & Mainen, Z. F. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2012.0037 · Cited by 305 · see citing works
The construction of confidence in a perceptual decision
Zylberberg, A., Barttfeld, P., & Sigman, M. (2012) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2012.00079 · Cited by 297 · see citing works
Self-citationConfidence and accuracy of near-threshold discrimination responses
Kunimoto, C., Miller, J., & Pashler, H. (2001) · Consciousness and Cognition · doi:10.1006/ccog.2000.0494 · Cited by 291 · see citing works
Ploran, E. J., Nelson, S. M., Velanova, K., Donaldson, D. I., Petersen, S. E., & Wheeler, M. E. (2007) · The Journal of Neuroscience · doi:10.1523/jneurosci.3522-07.2007 · Cited by 282 · see citing works
Decision making by urgency gating: theory and experimental support
Thura, D., Beauregard-Racine, J., Fradet, C.-W., & Cisek, P. (2012) · Journal of Neurophysiology · doi:10.1152/jn.01071.2011 · Cited by 267 · see citing works
The relative judgment theory of two choice response time
Link, S. W. (1975) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(75)90053-x · Cited by 251 · see citing works
Concurrent processing of saccades in visual search
McPeek, R. M., Skavenski, A. A., & Nakayama, K. (2000) · Vision Research · doi:10.1016/s0042-6989(00)00102-4 · Cited by 234 · see citing works
A neural mechanism of speed-accuracy tradeoff in macaque area LIP
Hanks, T., Kiani, R., & Shadlen, M. N. (2014) · eLife · doi:10.7554/elife.02260 · Cited by 222 · see citing works
Self-citationError Correction Time without External Error Signals
RABBITT, P. M. A. (1966) · Nature · doi:10.1038/212438a0 · Cited by 213 · see citing works
A modular planar robotic manipulandum with end-point torque control
Howard, I. S., Ingram, J. N., & Wolpert, D. M. (2009) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2009.05.005 · Cited by 211 · see citing works
Self-citationDeliberation in the Motor System: Reflex Gains Track Evolving Evidence Leading to a Decision
Selen, L. P. J., Shadlen, M. N., & Wolpert, D. M. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.5273-11.2012 · Cited by 204 · see citing works
Self-citationMoran, R., Teodorescu, A. R., & Usher, M. (2015) · Cognitive Psychology · doi:10.1016/j.cogpsych.2015.01.002 · Cited by 185 · see citing works
Effects of cortical microstimulation on confidence in a perceptual decision
Fetsch, C. R., Kiani, R., Newsome, W. T., & Shadlen, M. N. (2014) · Neuron · doi:10.1016/j.neuron.2014.07.011 · Cited by 185 · see citing works
Self-citationGood, I. J. (1979) · Biometrika · doi:10.1093/biomet/66.2.393 · Cited by 184 · see citing works
Two Types of ROC Curves and Definitions of Parameters
Clarke, F. R., Birdsall, T. G., & Tanner, W. P. (1959) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907764 · Cited by 183 · see citing works
Individual consistency in the accuracy and distribution of confidence judgments
Ais, J., Zylberberg, A., Barttfeld, P., & Sigman, M. (2016) · Cognition · doi:10.1016/j.cognition.2015.10.006 · Cited by 174 · see citing works
Self-citationRatcliff, R., & Starns, J. J. (2013) · Psychological Review · doi:10.1037/a0033152 · Cited by 160 · see citing works
Kiani, R., Cueva, C. J., Reppas, J. B., & Newsome, W. T. (2014) · Current Biology · doi:10.1016/j.cub.2014.05.049 · Cited by 158 · see citing works
Self-citationMotor effort alters changes of mind in sensorimotor decision making
Burk, D., Ingram, J. N., Franklin, D. W., Shadlen, M. N., & Wolpert, D. M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0092681 · Cited by 117 · see citing works
Self-citationThe time course of visual information accrual guiding eye movement decisions
Caspi, A., Beutter, B. R., & Eckstein, M. P. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0305329101 · Cited by 116 · see citing works
Calibration of Probabilities: The State of the Art to 1980
Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1981) · doi:10.21236/ada101986 · Cited by 106 · see citing works
Dynamics of postdecisional processing of confidence.
Yu, S., Pleskac, T. J., & Zeigenfuse, M. D. (2015) · Journal of Experimental Psychology General · doi:10.1037/xge0000062 · Cited by 103 · see citing works
Processing a Display Even after you Make a Response to it. How Perceptual Errors can be Corrected
Rabbitt, P., & Vyas, S. (1981) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/14640748108400790 · Cited by 97 · see citing works
Moreno‐Bote, R. (2010) · Neural Computation · doi:10.1162/neco.2010.12-08-930 · Cited by 89 · see citing works
Relation between belief and performance in perceptual decision making
Drugowitsch, J., Moreno-Bote, R., & Pouget, A. (2014) · PLoS ONE · doi:10.1371/journal.pone.0096511 · Cited by 72 · see citing works
Response Reversals in Recognition Memory.
Zandt, T. V., & Maldonado‐Molina, M. M. (2004) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.30.6.1147 · Cited by 62 · see citing works
Perceptual Criteria in the Human Brain
White, C. N., Mumford, J. A., & Poldrack, R. A. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.1744-12.2012 · Cited by 60 · see citing works
Exploring the cortical evidence of a sensory-discrimination process
Romo, R., Hernández, A., Zainos, A., Brody, C., & Salinas, E. (2002) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.2002.1100 · Cited by 58 · see citing works
Perceptual decision processes flexibly adapt to avoid change-of-mind motor costs
Moher, J., & Song, J. (2014) · Journal of Vision · doi:10.1167/14.8.1 · Cited by 57 · see citing works
Confidence estimation as a stochastic process in a neurodynamical system of decision making
Wei, Z., & Wang, X. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00793.2014 · Cited by 31 · see citing works
Concurrent processing of saccades
McPeek, R. M., Keller, E. L., & Nakayama, K. (1999) · Behavioral and Brain Sciences · doi:10.1017/s0140525x99402158 · Cited by 12 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Wald, 1947, Sequential Analysis
Cited by
Sorted by citations
Self-Evaluation of Decision-Making: A General Bayesian Framework for Metacognitive Computation
Fleming, S. M., & Daw, N. D. (2017) · Psychological Review · doi:10.1037/rev0000045 · Cited by 489 · see citing works
Rouault, M., Seow, T. X. F., Gillan, C. M., & Fleming, S. M. (2018) · Biological Psychiatry · doi:10.1016/j.biopsych.2017.12.017 · Cited by 312 · see citing works
Decision-making in sensorimotor control
Gallivan, J. P., Chapman, C. S., Wolpert, D. M., & Flanagan, J. R. (2018) · Nature reviews. Neuroscience · doi:10.1038/s41583-018-0045-9 · Cited by 305 · see citing works
Self-citationSubjective Confidence Predicts Information Seeking in Decision Making
Desender, K., Boldt, A., & Yeung, N. (2018) · Psychological Science · doi:10.1177/0956797617744771 · Cited by 296 · see citing works
Metacognition and Confidence: A Review and Synthesis
Fleming, S. M. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-022423-032425 · Cited by 256 · see citing works
“Pleskac & Busemeyer (2010) additionally proposed that this evidence accumulation process may continue after a decision has been made, to inform estimates of decision confidence and potentially leading to changes of mind (van den Berg et al., 2016a, Resulaj et al., 2009).”
Perceptual confidence neglects decision-incongruent evidence in the brain
Peters, M. A. K., Thesen, T., Ko, Y., Maniscalco, B., Carlson, C., Davidson, M., Doyle, W., Kuzniecky, R., Devinsky, O., Halgren, E., et al. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0139 · Cited by 250 · see citing works
Confidence drives a neural confirmation bias
Rollwage, M., Loosen, A. M., Hauser, T., Moran, R., Dolan, R., & Fleming, S. (2020) · Nature Communications · doi:10.1038/s41467-020-16278-6 · Cited by 165 · see citing works
Metacognitive Failure as a Feature of Those Holding Radical Beliefs
Rollwage, M., Dolan, R. J., & Fleming, S. M. (2018) · Current Biology · doi:10.1016/j.cub.2018.10.053 · Cited by 193 · see citing works
Advancing theorizing about fast-and-slow thinking
Neys, W. D. (2022) · Behavioral and Brain Sciences · doi:10.1017/s0140525x2200142x · Cited by 191 · see citing works
Human Metacognition Across Domains: Insights from Individual Differences and Neuroimaging
Rouault, M., McWilliams, A., Allen, M. G., & Fleming, S. M. (2018) · Personality Neuroscience · doi:10.1017/pen.2018.16 · Cited by 176 · see citing works
Neural mediators of changes of mind about perceptual decisions
Fleming, S. M., van der Putten, E. J., & Daw, N. D. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0104-6 · Cited by 155 · see citing works
Comparing Bayesian and non-Bayesian accounts of human confidence reports
Adler, W. T., & Ma, W. J. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006572 · Cited by 118 · see citing works
Tagliabue, C. F., Veniero, D., Benwell, C., Cecere, R., Savazzi, S., & Thut, G. (2019) · Scientific Reports · doi:10.1038/s41598-019-41024-4 · Cited by 150 · see citing works
Explicit representation of confidence informs future value-based decisions
Folke, T., Jacobsen, C., Fleming, S. M., & Martino, B. D. (2016) · Nature Human Behaviour · doi:10.1038/s41562-016-0002 · Cited by 155 · see citing works
Mamassian, P. (2016) · Annual Review of Vision Science · doi:10.1146/annurev-vision-111815-114630 · Cited by 139 · see citing works
Forming global estimates of self-performance from local confidence
Rouault, M., Dayan, P., & Fleming, S. M. (2019) · Nature Communications · doi:10.1038/s41467-019-09075-3 · Cited by 134 · see citing works
Zylberberg, A., Fetsch, C. R., & Shadlen, M. N. (2016) · eLife · doi:10.7554/elife.17688 · Cited by 140 · see citing works
Self-citationConfidence Is the Bridge between Multi-stage Decisions
van den Berg, R., Zylberberg, A., Kiani, R., Shadlen, M. N., & Wolpert, D. M. (2016) · Current Biology · doi:10.1016/j.cub.2016.10.021 · Cited by 125 · see citing works
Self-citationThe neural system of metacognition accompanying decision-making in the prefrontal cortex
Qiu, L., Su, J., Ni, Y., Bai, Y., Zhang, X., Li, X., & Wan, X. (2018) · PLOS Biology · doi:10.1371/journal.pbio.2004037 · Cited by 125 · see citing works
Metacognition in Multisensory Perception
Deroy, O. D., Spence, C. J., & Noppeney, U. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.08.006 · Cited by 127 · see citing works
Decoding and perturbing decision states in real time
Peixoto, D., Verhein, J. R., Kiani, R., Kao, J. C., Nuyujukian, P., Chandrasekaran, C., Brown, J., Fong, S., Ryu, S. I., Shenoy, K. V., et al. (2021) · Nature · doi:10.1038/s41586-020-03181-9 · Cited by 120 · see citing works
Self-citationThe idiosyncratic nature of confidence
Navajas, J., Hindocha, C., Foda, H., Keramati, M., Latham, P. E., & Bahrami, B. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0215-1 · Cited by 125 · see citing works
Models, movements, and minds: bridging the gap between decision making and action
Wispinski, N. J., Gallivan, J. P., & Chapman, C. S. (2018) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13973 · Cited by 105 · see citing works
Human VMPFC encodes early signatures of confidence in perceptual decisions
Gherman, S., & Philiastides, M. G. (2018) · eLife · doi:10.7554/elife.38293 · Cited by 107 · see citing works
The Psychology of Reaching: Action Selection, Movement Implementation, and Sensorimotor Learning
Kim, H. E., Avraham, G., & Ivry, R. B. (2020) · Annual Review of Psychology · doi:10.1146/annurev-psych-010419-051053 · Cited by 105 · see citing works
Tsutsui‐Kimura, I., Matsumoto, H., Akiti, K., Yamada, M. M., Uchida, N., & Watabe‐Uchida, M. (2020) · eLife · doi:10.7554/elife.62390 · Cited by 91 · see citing works
Post-decisional accounts of biases in confidence
Navajas, J., Bahrami, B., & Latham, P. E. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.05.005 · Cited by 97 · see citing works
On-line confidence monitoring during decision making
Dotan, D., Meyniel, F., & Dehaene, S. (2017) · Cognition · doi:10.1016/j.cognition.2017.11.001 · Cited by 98 · see citing works
Stolyarova, A., Rakhshan, M., Hart, E. E., O’Dell, T. J., Peters, M. A. K., Lau, H. C., Soltani, A., & Izquierdo, A. (2019) · Nature Communications · doi:10.1038/s41467-019-12725-1 · Cited by 90 · see citing works
Piercing of Consciousness as a Threshold-Crossing Operation
Kang, Y. H., Petzschner, F. H., Wolpert, D. M., & Shadlen, M. N. (2017) · Current Biology · doi:10.1016/j.cub.2017.06.047 · Cited by 87 · see citing works
Self-citationReciprocity of social influence
Mahmoodi, A., Bahrami, B., & Mehring, C. (2018) · Nature Communications · doi:10.1038/s41467-018-04925-y · Cited by 92 · see citing works
Fechner’s law in metacognition: A quantitative model of visual working memory confidence.
van den Berg, R., Yoo, A. H., & Ma, W. J. (2017) · Psychological Review · doi:10.1037/rev0000060 · Cited by 79 · see citing works
Evidence accumulation relates to perceptual consciousness and monitoring
Pereira, M., Mégevand, P., Tan, M. X., Chang, W., Wang, S., Rezai, A. R., Seeck, M., Corniola, M. V., Momjian, S., Bernasconi, F., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-23540-y · Cited by 102 · see citing works
Two sides of the same coin: Monetary incentives concurrently improve and bias confidence judgments
Lebreton, M., Langdon, S., Slieker, M. J., Nooitgedacht, J. S., Goudriaan, A., Denys, D., van Holst, R. V., & Luigjes, J. (2017) · Science Advances · doi:10.1126/sciadv.aaq0668 · Cited by 65 · see citing works
Metacognitive impairments extend perceptual decision making weaknesses in compulsivity
Hauser, T. U., Allen, M., Bullmore, E. T., Goodyer, I. M., Fonagy, P., Jones, P. B., Fearon, P. R. M., Prabhu, G., Moutoussis, M., Clair, M. S., et al. (2017) · Scientific Reports · doi:10.1038/s41598-017-06116-z · Cited by 81 · see citing works
Psychophysical reverse correlation reflects both sensory and decision-making processes
Okazawa, G., Sha, L., Purcell, B. A., & Kiani, R. (2018) · Nature Communications · doi:10.1038/s41467-018-05797-y · Cited by 83 · see citing works
Self-citationDisentangling the origins of confidence in speeded perceptual judgments through multimodal imaging
Pereira, M., Faivre, N., Iturrate, I., Wirthlin, M., Serafini, L., Martin, S., Desvachez, A., Blanke, O., Van De Ville, D., & Millán, J. D. R. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1918335117 · Cited by 72 · see citing works
Understanding neural signals of post-decisional performance monitoring: An integrative review
Desender, K., Ridderinkhof, K. R., & Murphy, P. R. (2021) · eLife · doi:10.7554/elife.67556 · Cited by 77 · see citing works
Fetsch, C. R., Odean, N. N., Jeurissen, D., El-Shamayleh, Y., Horwitz, G. D., & Shadlen, M. N. (2018) · eLife · doi:10.7554/elife.36523 · Cited by 74 · see citing works
Self-citationServant, M., White, C. N., Montagnini, A., & Burle, B. (2016) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_00989 · Cited by 75 · see citing works
Carruthers, P., & Williams, D. M. (2022) · Cognition · doi:10.1016/j.cognition.2022.105117 · Cited by 71 · see citing works
A neural circuit model of decision uncertainty and change-of-mind
Atiya, N. A. A., Rañó, I., Prasad, G., & Wong‐Lin, K. (2019) · Nature Communications · doi:10.1038/s41467-019-10316-8 · Cited by 69 · see citing works
Track It to Crack It: Dissecting Processing Stages with Finger Tracking
Dotan, D., Pinheiro‐Chagas, P., Roumi, F. A., & Dehaene, S. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.10.002 · Cited by 70 · see citing works
Confidence in masked orientation judgments is informed by both evidence and visibility
Rausch, M., Hellmann, S., & Zehetleitner, M. (2017) · Attention Perception & Psychophysics · doi:10.3758/s13414-017-1431-5 · Cited by 69 · see citing works
Beyond decision! Motor contribution to speed–accuracy trade-off in decision-making
Spieser, L., Servant, M., Hasbroucq, T., & Burle, B. (2016) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1172-9 · Cited by 66 · see citing works
Dynamic expressions of confidence within an evidence accumulation framework
Desender, K., Donner, T. H., & Verguts, T. (2020) · Cognition · doi:10.1016/j.cognition.2020.104522 · Cited by 61 · see citing works
Khalvati, K., Kiani, R., & Rao, R. P. N. (2021) · Nature Communications · doi:10.1038/s41467-021-25419-4 · Cited by 64 · see citing works
Self-citationPost-decision biases reveal a self-consistency principle in perceptual inference
Luu, L., & Stocker, A. A. (2018) · eLife · doi:10.7554/elife.33334 · Cited by 59 · see citing works
Response-based outcome predictions and confidence regulate feedback processing and learning
Frömer, R., Nassar, M. R., Bruckner, R., Stürmer, B., Sommer, W., & Yeung, N. (2021) · eLife · doi:10.7554/elife.62825 · Cited by 62 · see citing works
Visual metacognition: Measures, models, and neural correlates.
Rahnev, D. (2021) · American Psychologist · doi:10.1037/amp0000937 · Cited by 62 · see citing works
Dynamic influences on static measures of metacognition
Desender, K., Vermeylen, L., & Verguts, T. (2022) · Nature Communications · doi:10.1038/s41467-022-31727-0 · Cited by 62 · see citing works
On second thoughts: changes of mind in decision-making
Stone, C., Mattingley, J. B., & Rangelov, D. (2022) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2022.02.004 · Cited by 57 · see citing works
Sweis, B. M., Thomas, M. J., & Redish, A. D. (2018) · PLoS Biology · doi:10.1371/journal.pbio.2005853 · Cited by 56 · see citing works
Confirmation bias is adaptive when coupled with efficient metacognition
Rollwage, M., & Fleming, S. M. (2021) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2020.0131 · Cited by 54 · see citing works
Frömer, R., & Shenhav, A. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.12.006 · Cited by 50 · see citing works
Subjective experience of difficulty depends on multiple cues
Desender, K., Filip Van Opstal, A., & den Bussche, E. V. (2017) · Scientific Reports · doi:10.1038/srep44222 · Cited by 49 · see citing works
Direct observation of the neural computations underlying a single decisionPreprint
Steinemann, N. A., Stine, G. M., Trautmann, E. M., Zylberberg, A., Wolpert, D. M., & Shadlen, M. N. (2023) · eLife · doi:10.7554/elife.90859.3 · Cited by 82 · see citing works
Self-citationEvidence or Confidence: What Is Really Monitored during a Decision?
Lee, D. G., Jean Daunizeau, A., & Pezzulo, G. (2023) · Psychonomic Bulletin & Review · doi:10.3758/s13423-023-02255-9 · Cited by 49 · see citing works
Subjective value then confidence in human ventromedial prefrontal cortex
Shapiro, A. D., & Grafton, S. T. (2020) · PLoS ONE · doi:10.1371/journal.pone.0225617 · Cited by 46 · see citing works
Rausch, M., Zehetleitner, M., Steinhauser, M., & Maier, M. E. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116963 · Cited by 44 · see citing works
Three challenges for connecting model to mechanism in decision-making
Churchland, A. K., & Kiani, R. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.06.008 · Cited by 41 · see citing works
Self-citationConfidence is higher in touch than in vision in cases of perceptual ambiguity
Fairhurst, M. T., Travers, E., Vincent Hayward, B., & Deroy, O. D. (2018) · Scientific Reports · doi:10.1038/s41598-018-34052-z · Cited by 42 · see citing works
Strength and weight: The determinants of choice and confidence
Kvam, P. D., & Pleskac, T. J. (2016) · Cognition · doi:10.1016/j.cognition.2016.04.008 · Cited by 40 · see citing works
Evidence for metacognitive bias in perception of voluntary action
Charles, L., Chardin, C., & Haggard, P. (2019) · Cognition · doi:10.1016/j.cognition.2019.104041 · Cited by 38 · see citing works
Neurocomputational mechanisms of confidence in self and others
Bang, D., Moran, R., Daw, N. D., & Fleming, S. M. (2022) · Nature Communications · doi:10.1038/s41467-022-31674-w · Cited by 39 · see citing works
Consistent patterns of distractor effects during decision making
Chau, B. K. H., Law, C.-K., Lopez‐Persem, A., Klein-Flügge, M. C., & Rushworth, M. F. S. (2020) · eLife · doi:10.7554/elife.53850 · Cited by 37 · see citing works
I know better! Emerging metacognition allows adolescents to ignore false advice
Moses-Payne, M. E., Habicht, J., Bowler, A., Steinbeis, N., & Hauser, T. U. (2021) · Developmental Science · doi:10.1111/desc.13101 · Cited by 40 · see citing works
Sources of confidence in value-based choice
Brus, J., Aebersold, H., Grueschow, M., & Polanía, R. H. (2021) · Nature Communications · doi:10.1038/s41467-021-27618-5 · Cited by 38 · see citing works
Flexible categorization in perceptual decision making
Prat-Ortega, G., Wimmer, K., Roxin, A., & de la Rocha, J. (2021) · Nature Communications · doi:10.1038/s41467-021-21501-z · Cited by 37 · see citing works
Abandoning and modifying one action plan for alternatives
Song, J. (2017) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2016.0195 · Cited by 34 · see citing works
Towards a common conceptual space for metacognition in perception and memory
Mazancieux, A., Pereira, M., Faivre, N., Mamassian, P., Moulin, C. J. A., & Souchay, C. (2023) · Nature Reviews Psychology · doi:10.1038/s44159-023-00245-1 · Cited by 36 · see citing works
Verdonck, S., Loossens, T., & Philiastides, M. G. (2020) · Psychological Review · doi:10.1037/rev0000258 · Cited by 33 · see citing works
Changes-of-mind in the absence of new post-decision evidence
Atiya, N. A. A., Zgonnikov, A., Schoemann, M., Scherbaum, S., O’Hora, D. P., & Wong‐Lin, K. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007149 · Cited by 32 · see citing works
Dynamic sources of evidence supporting confidence judgments and error detection.
Charles, L., & Yeung, N. (2018) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/xhp0000583 · Cited by 38 · see citing works
Movement Improves the Quality of Temporal Perception and Decision-Making
Wiener, M., Zhou, W., Bader, F., & Joiner, W. M. (2019) · eNeuro · doi:10.1523/eneuro.0042-19.2019 · Cited by 33 · see citing works
Post-decision wagering after perceptual judgments reveals bi-directional certainty readouts
Moreira, C. M., Rollwage, M., Kaduk, K., Wilke, M., & Kagan, I. (2018) · Cognition · doi:10.1016/j.cognition.2018.02.026 · Cited by 35 · see citing works
Confidence Representation of Perceptual Decision by EEG and Eye Data in a Random Dot Motion Task
Shooshtari, S. V., Esmaily, J., Azizi, Z. R., & Ebrahimpour, R. (2019) · Neuroscience · doi:10.1016/j.neuroscience.2019.03.031 · Cited by 30 · see citing works
The folded X-pattern is not necessarily a statistical signature of decision confidence
Rausch, M., & Zehetleitner, M. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007456 · Cited by 34 · see citing works
Postdecision Evidence Integration and Depressive Symptoms
Moses-Payne, M. E., Rollwage, M., Fleming, S. M., & Roiser, J. P. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00639 · Cited by 30 · see citing works
Human VMPFC encodes early signatures of confidence in perceptual decisionsPreprint
Gherman, S., & Philiastides, M. G. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/224337 · Cited by 30 · see citing works
Metacognition and confidence: A review and synthesisPreprint
Fleming, S. M. (2023) · doi:10.31234/osf.io/ge7tz · Cited by 27 · see citing works
Salinas, E., & Stanford, T. R. (2018) · Scientific Reports · doi:10.1038/s41598-018-32224-5 · Cited by 24 · see citing works
Reaching Consensus in Polarized Moral Debates
Navajas, J., Heduan, F. Á., Garrido, J. M., Gonzalez, P. A., Garbulsky, G., Ariely, D., & Sigman, M. (2019) · Current Biology · doi:10.1016/j.cub.2019.10.018 · Cited by 25 · see citing works
Prior information differentially affects discrimination decisions and subjective confidence reports
Constant, M., Pereira, M., Faivre, N., & Filevich, E. (2023) · Nature Communications · doi:10.1038/s41467-023-41112-0 · Cited by 27 · see citing works
Barany, D. A., Gómez-Granados, A., Schrayer, M., Cutts, S. A., & Singh, T. (2020) · Journal of Neurophysiology · doi:10.1152/jn.00098.2020 · Cited by 23 · see citing works
Confidence in visual motion discrimination is preserved in individuals with schizophrenia
Faivre, N., Roger, M., Pereira, M., de Gardelle, V., Vergnaud, J., Passerieux, C., & Roux, P. (2021) · Journal of Psychiatry and Neuroscience · doi:10.1503/jpn.200022 · Cited by 24 · see citing works
Changing our minds about changes of mind
Fleming, S. M. (2016) · eLife · doi:10.7554/elife.14790 · Cited by 22 · see citing works
Feuerriegel, D., Murphy, M., Konski, A., Mepani, V., Sun, J., Hester, R., & Bode, S. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119447 · Cited by 24 · see citing works
Enachescu, V., Schrater, P., Schaal, S., & Christopoulos, V. (2021) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1009429 · Cited by 21 · see citing works
A population representation of the confidence in a decision in the parietal cortex
Zylberberg, A., & Shadlen, M. N. (2025) · Cell Reports · doi:10.1016/j.celrep.2025.115526 · Cited by 20 · see citing works
Self-citationDifferentiating between Models of Perceptual Decision Making Using Pupil Size Inferred Confidence
Kawaguchi, K., Clery, S., Pourriahi, P., Seillier, L., Haefner, R. M., & Nienborg, H. (2018) · Journal of Neuroscience · doi:10.1523/jneurosci.0735-18.2018 · Cited by 20 · see citing works
Perceptual change-of-mind decisions are sensitive to absolute evidence magnitude.
Turner, W., Feuerriegel, D., Andrejević, M., Hester, R., & Bode, S. (2020) · Cognitive Psychology · doi:10.1016/j.cogpsych.2020.101358 · Cited by 18 · see citing works
Analyzing dynamic decision-making models using Chapman-Kolmogorov equations
Barendregt, N. W., Josić́, K., & Kilpatrick, Z. P. (2019) · Journal of Computational Neuroscience · doi:10.1007/s10827-019-00733-5 · Cited by 19 · see citing works
Flexibility of Timescales of Evidence Evaluation for Decision Making
Ganupuru, P., Goldring, A. B., Harun, R., & Hanks, T. D. (2019) · Current Biology · doi:10.1016/j.cub.2019.05.037 · Cited by 22 · see citing works
Cai, Y., Jin, Z., Zhai, C., Wang, H., Wang, J., Tang, Y., & Kwok, S. C. (2022) · Communications Biology · doi:10.1038/s42003-022-03762-6 · Cited by 21 · see citing works
Controllability boosts neural and cognitive signatures of changes-of-mind in uncertain environments
Rouault, M., Weiss, A., Lee, J. K., Drugowitsch, J., Chambon, V., & Wyart, V. (2022) · eLife · doi:10.7554/elife.75038 · Cited by 18 · see citing works
Expressions for Bayesian confidence of drift diffusion observers in fluctuating stimuli tasks
Calder-Travis, J., Bogacz, R., & Yeung, N. (2023) · Journal of Mathematical Psychology · doi:10.1101/2020.02.25.965384 · Cited by 18 · see citing works
Choices change the temporal weighting of decision evidence
Talluri, B. C., Urai, A. E., Bronfman, Z. Z., Brezis, N., Tsetsos, K., Usher, M., & Donner, T. H. (2021) · Journal of Neurophysiology · doi:10.1152/jn.00462.2020 · Cited by 18 · see citing works
Modelling Speed-Accuracy Tradeoffs in the Stopping Rule for Confidence JudgmentsPreprint
Herregods, S., Denmat, P. L., Vermeylen, L., & Desender, K. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.02.27.530208 · Cited by 17 · see citing works
The validity and consistency of continuous joystick response in perceptual decision-making
Szul, M. J., Bompas, A., Sumner, P., & Zhang, J. (2019) · Behavior Research Methods · doi:10.3758/s13428-019-01269-3 · Cited by 15 · see citing works
- Authors
- van den Berg, R., Anandalingam, K., Zylberberg, A., Kiani, R., Shadlen, M. N., & Wolpert, D. M.
- Affiliations
- Computational and Biological Learning Laboratory, Department of Engineering, Cambridge University, Cambridge, United Kingdom
- University of Cambridge (GB)
- Howard Hughes Medical Institute (US)
- New York University (US)
- Corresponding author
- Wolpert, Daniel M
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2016-02-01
- Volume
- 5
- Pages
- e12192-e12192
- ISSN
- 2050-084X (ISSN-L)
- PMC
- PMC4798971
- OpenAlex
- W2261997804
- Semantic Scholar
- 8a3457c13959cfb9435daf0710bf5eac7eacbc9b
- Cited by
- 270
- References
- 67
- Citations by year
- 2026: 19 · 2025: 20 · 2024: 23 · 2023: 38 · 2022: 24 · 2021: 26 · 2020: 32 · 2019: 26 · 2018: 31 · 2017: 16 · 2016: 14
- FWCI
- 25.07
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- eLife (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- Funders
- Wellcome Trust
- Howard Hughes Medical Institute
- Royal Society
- Human Frontier Science Program
- National Eye Institute
- Simons Foundation
- Alfred P. Sloan Foundation
- Topics
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Keywords
- decision confidence, sequential sampling, vacillation, change of mind, perceptual decision making, motion direction discrimination, reaction time, processing delay
- MeSH terms
- Choice Behavior, Decision Making, Humans, Models, Neurological, Reaction Time, Healthy Volunteers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
