CoChat

Measures of metacognition on signal-detection theoretic models

Open access
Barrett, Adam B., Dienes, Zoltan, Seth, Anil K. · 2013 · Psychological Methods

The DOI matches its Crossref record; 54 of the 55 references it cites were found in a registry or an index.

Abstract

Analyzing metacognition, specifically knowledge of accuracy of internal perceptual, memorial, or other knowledge states, is vital for many strands of psychology, including determining the accuracy of feelings of knowing and discriminating conscious from unconscious cognition. Quantifying metacognitive sensitivity is however more challenging than quantifying basic stimulus sensitivity. Under popular signal-detection theory (SDT) models for stimulus classification tasks, approaches based on Type II receiver-operating characteristic (ROC) curves or Type II d-prime risk confounding metacognition with response biases in either the Type I (classification) or Type II (metacognitive) tasks. A new approach introduces meta-d': The Type I d-prime that would have led to the observed Type II data had the subject used all the Type I information. Here, we (a) further establish the inconsistency of the Type II d-prime and ROC approaches with new explicit analyses of the standard SDT model and (b) analyze, for the first time, the behavior of meta-d' under nontrivial scenarios, such as when metacognitive judgments utilize enhanced or degraded versions of the Type I evidence. Analytically, meta-d' values typically reflect the underlying model well and are stable under changes in decision criteria; however, in relatively extreme cases, meta-d' can become unstable. We explore bias and variance of in-sample measurements of meta-d' and supply MATLAB code for estimation in general cases. Our results support meta-d' as a useful measure of metacognition and provide rigorous methodology for its application. Our recommendations are useful for any researchers interested in assessing metacognitive accuracy.

Overview

Integrity signals

Registry

Crossref · 10.1037/a0033268

Cites

54 of 54 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 54 references

Cited by

See all 156

Metadata

Authors
Barrett, A. B., Dienes, Z., & Seth, A. K.
Affiliations
University of Sussex (GB)
Venue
Psychological Methods
Published
2013
Volume
18
Issue
4
Pages
535-552
ISSN
  • 1082-989X (ISSN-L)
  • 1939-1463
OpenAlex
W2006647339
Cited by
155
References
55
Citations by year
2026: 2 · 2025: 13 · 2024: 6 · 2023: 16 · 2022: 18 · 2021: 16 · 2020: 13 · 2019: 11 · 2018: 12 · 2017: 13 · 2016: 12 · 2015: 9 · 2014: 11 · 2013: 3
FWCI
6.37
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Psychological Methods (publishedVersion)
  • PubMed (publishedVersion)
  • Sussex Research Online (University of Sussex) (submittedVersion)
  • Figshare (submittedVersion, cc-by)
Funders
  • Dr. Mortimer and Dame Theresa Sackler Foundation
  • Engineering and Physical Sciences Research Council (EP/G007543/1)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Keywords
metacognition, signal detection theory, metacognitive sensitivity, response bias, metacognitive judgments, metacognitive accuracy, metacognitive confidence, metacognitive monitoring
MeSH terms
Cognition, Data Interpretation, Statistical, Humans, Models, Psychological, Signal Detection, Psychological

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.