Measures of metacognition on signal-detection theoretic models
Open accessThe DOI matches its Crossref record; 54 of the 55 references it cites were found in a registry or an index.
Abstract
Analyzing metacognition, specifically knowledge of accuracy of internal perceptual, memorial, or other knowledge states, is vital for many strands of psychology, including determining the accuracy of feelings of knowing and discriminating conscious from unconscious cognition. Quantifying metacognitive sensitivity is however more challenging than quantifying basic stimulus sensitivity. Under popular signal-detection theory (SDT) models for stimulus classification tasks, approaches based on Type II receiver-operating characteristic (ROC) curves or Type II d-prime risk confounding metacognition with response biases in either the Type I (classification) or Type II (metacognitive) tasks. A new approach introduces meta-d': The Type I d-prime that would have led to the observed Type II data had the subject used all the Type I information. Here, we (a) further establish the inconsistency of the Type II d-prime and ROC approaches with new explicit analyses of the standard SDT model and (b) analyze, for the first time, the behavior of meta-d' under nontrivial scenarios, such as when metacognitive judgments utilize enhanced or degraded versions of the Type I evidence. Analytically, meta-d' values typically reflect the underlying model well and are stable under changes in decision criteria; however, in relatively extreme cases, meta-d' can become unstable. We explore bias and variance of in-sample measurements of meta-d' and supply MATLAB code for estimation in general cases. Our results support meta-d' as a useful measure of metacognition and provide rigorous methodology for its application. Our recommendations are useful for any researchers interested in assessing metacognitive accuracy.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
The free-energy principle: a unified brain theory?
Friston, K. (2010) · Nature Reviews Neuroscience · doi:10.1038/nrn2787 · Cited by 8,120 · cited 2×
“An important future challenge lies in integrating SDT with the increasingly influential framework of 'predictive coding' or the so-called 'Bayesian Brain' hypothesis (Rao & Ballard, 1999; Bubic, Cramon, & Schubotz, 2010; Friston, 2010; Clark, in press).”
BackgroundWhatever next? Predictive brains, situated agents, and the future of cognitive science
Clark, A. (2013) · Behavioral and Brain Sciences · doi:10.1017/s0140525x12000477 · Cited by 5,924
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · cited 1×
“An important future challenge lies in integrating SDT with the increasingly influential framework of 'predictive coding' or the so-called 'Bayesian Brain' hypothesis (Rao & Ballard, 1999; Bubic, Cramon, & Schubotz, 2010; Friston, 2010; Clark, in press).”
BackgroundApproximate is Better than “Exact” for Interval Estimation of Binomial Proportions
Agresti, A., & Coull, B. A. (1998) · The American Statistician · doi:10.1080/00031305.1998.10480550 · Cited by 3,223 · cited 1×
“6 Such a prior, though vague, adds some information and can increase the accuracy of estimates when data are limited (Agresti & Coull, 1998; Greenland, 2006).”
BackgroundPragmatics of measuring recognition memory: Applications to dementia and amnesia.
Snodgrass, J. G., & Corwin, J. (1988) · Journal of Experimental Psychology General · doi:10.1037//0096-3445.117.1.34 · Cited by 3,089 · cited 1×
“An alternative way of dealing with outlying subjects is to add 0.5, i.e. a 'flattening constant', to every data cell for every subject (Snodgrass & Corwin, 1988).”
BackgroundInfluentialDetection theory: A user's guide
Lories, G. (1993) · Behavioural Processes · doi:10.1016/0376-6357(93)90083-4 · Cited by 2,835
A Reference Bayesian Test for Nested Hypotheses and its Relationship to the Schwarz Criterion
Kass, R. E., & Wasserman, L. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476592 · Cited by 1,253 · cited 1×
“…one observation for each of the hit and false alarm rates and corresponding miss and correct rejection rates, i.e. it is a unit information prior (Kass & Wasserman, 1995), corresponding to the belief that with approximately 95% probability all of the hit and false alarm rates lie between 0.05…”
BackgroundInfluentialRosenthal, D. M. (2005) · doi:10.1093/oso/9780198236979.001.0001 · Cited by 993
A comparison of current measures of the accuracy of feeling-of-knowing predictions
Nelson, T. O. (1984) · Psychological Bulletin · doi:10.1037/0033-2909.95.1.109 · Cited by 976
Approximate Is Better than "Exact" for Interval Estimation of Binomial Proportions
Agresti, A., & Coull, B. A. (1998) · The American Statistician · doi:10.2307/2685469 · Cited by 972
Cited by
Items 101–156 of 156
Neural Correlates of Metacognition Impairment in Opioid Addiction
Moeller, S. J., Abeykoon, S. K., Dhayagude, P., Varnas, B., Weinstein, J. J., Perlman, G., Gil, R. B., Fleming, S. M., & Abi‐Dargham, A. (2024) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2024.07.014 · Cited by 13
Subliminal understanding of active versus passive sentences.
Armstrong, A.-M., & Dienes, Z. P. (2014) · Psychology of Consciousness Theory Research and Practice · doi:10.1037/css0000002 · Cited by 11
Self-citationBor, D., Schwartzman, D. J., Barrett, A. B., & Seth, A. K. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/058032 · Cited by 8
Self-citationOn why we lack confidence in some signal-detection-based analyses of confidence
Arnold, D. H., Johnston, A., Adie, J. M., & Yarrow, K. (2023) · Consciousness and Cognition · doi:10.1016/j.concog.2023.103532 · Cited by 10
Herman, A. M., Olszewska, A. M., Gaca, M., Droździel, D., & Marchewka, A. (2023) · Psychophysiology · doi:10.1111/psyp.14402 · Cited by 7
Seban, P., Šikl, R., & Urban, K. (2025) · International Journal of Educational Research · doi:10.1016/j.ijer.2025.102578 · Cited by 11
Beyond counting the correct responses: Metacognitive monitoring and score estimations in mathematics
Başokçu, T. O., & Güzel, M. A. (2022) · Psychology in the Schools · doi:10.1002/pits.22665 · Cited by 7
d$$^{\prime }_{o}$$: Sensitivity at the optimal criterion location
Aujla, H. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01913-5 · Cited by 6
Evaluating false positive rates of standard and hierarchical measures of metacognitive accuracy
Rausch, M., & Zehetleitner, M. (2023) · Metacognition and Learning · doi:10.1007/s11409-023-09353-y · Cited by 9
“…studies have examined whether measures of metacognitive accuracy are contaminated by task performance, task criteria, and confidence criteria (Barrett et al., 2013; Guggenmos, 2021; Masson & Rotello, 2009; Rahnev, 2023; Rausch & Zehetleitner, 2017; Rausch et al., 2023; Shekhar & Rahnev,…”
BackgroundSustained perceptual invisibility of solid shapes following contour adaptation to partial outlines
Cox, M. A., Lowe, K. A., Blake, R., & Maier, A. (2014) · Consciousness and Cognition · doi:10.1016/j.concog.2014.02.007 · Cited by 6
Faivre, N., Filevich, E., Solovey, G., Kühn, S., & Blanke, O. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/095950 · Cited by 7
Bor, D., Barrett, A. B., Schwartzman, D. J., & Seth, A. K. (2018) · Consciousness and Cognition · doi:10.1016/j.concog.2018.07.011 · Cited by 7
“In addition, the analysis of Barrett et al. (2013) suggests that including extreme data points should weaken statistical inference since, as we have already noted, small random fluctuations in hit and miss rates at extreme ends of the spectrum will translate into large fluctuations in measured values of meta-d’.”
InfluentialSelf-citationSaccenti, D., Moro, A. S., Salvetti, G., Sassaroli, S., Malgaroli, A., Lamanna, J., & Ferro, M. (2025) · Biomedicines · doi:10.3390/biomedicines13071522 · Cited by 6
Serotonergic Effects on InteroceptionPreprint
Livermore, J. J. A., Holmes, C. L., Moga, G., Adamatzky, K., Critchley, H., Garfinkel, S. N., & Campbell-Meiklejohn, D. K. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.08.28.262550 · Cited by 5
“…poor fit for signal detection approaches to analysis (rather than the AUC used here) that assume that metacognitive insight arises from the same information and processes as the perceptual decision, necessitating above chance accuracy to interpret above chance metacognition (Barrett et al., 2013).”
Common computations for metacognition and meta-metacognitionPreprint
Zheng, Y., Recht, S., & Rahnev, D. (2023) · doi:10.31234/osf.io/g4h65 · Cited by 5
The influence of tDCS on the speed-accuracy tradeoff and metacognitive decision making
Sabio, J., Ballard, T., Filmer, H. L., & Dux, P. E. (2025) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2025.109130 · Cited by 4
“…also reported trial-by-trial confidence, which was used to compute meta-d ’ , a formal measure of metacognition representing how sensitive the subject is to their own performance (Barrett et al., 2013; Maniscalco and Lau, 2012); and m-ratio , representing the subject ’ s metacognitive efficiency.”
Hughes, G. I., & Thomas, A. (2022) · Journal of Intelligence · doi:10.3390/jintelligence10040101 · Cited by 5
“This illustration exemplifies the proposal that the relative accuracy of JOLs is logically independent of absolute levels of long-term memory performance (Benjamin and Diaz 2008; Mazzoni and Nelson 1995; Nelson 1984; Gonzalez and Nelson 1996; Spellman et al. 2008).”
BackgroundChen, B., Mundy, M. E., & Tsuchiya, N. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.01712 · Cited by 7
Lehmann, M., Neumann, C., Wasserthal, S., Delis, A., Schultz, J., Hurlemann, R., & Ettinger, U. (2022) · Behavioural Brain Research · doi:10.1016/j.bbr.2022.113925 · Cited by 5
I know that I know. But do I know that I do not know?
Polyanskaya, L. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1128200 · Cited by 3
“The signal detection modeling approach offers a clear advantage when comparing metacognition across tasks, domains, and modalities that vary in terms of task performance and metacognitive bias, which affect alternative measures of metacognition based on retrospective confidence (Masson and Rotello, 2009; Maniscalco an…”
BackgroundYoshida, K., Sawamura, D., Ogawa, K., Mototani, T., Ikoma, K., & Sakai, S. (2023) · Journal of Cognitive Neuroscience · doi:10.1162/jocn_a_02064 · Cited by 4
“…self-awareness and performance monitoring (Dromer, Kheloufi, & Azouvi, 2021a; Al Banna, Redha, Abdulla, Nair, & Donnellan, 2016; Ham et al., 2014; Barrett, Dienes, & Seth, 2013; Bach & David, 2006; Schmitz, Rowley, Kawahara, & Johnson, 2006), leading patients to underestimate their disability or…”
Kelso, L. E., Dobolyi, D. G., Grabman, J. H., & Dodson, C. S. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2503971122 · Cited by 7
Suprathreshold perceptual decisions constrain models of confidencePreprint
Locke, S. M., Landy, M. S., & Mamassian, P. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/p6395 · Cited by 3
GGSDT: A unified signal detection framework for confidence data analysisPreprint
Miyoshi, K., & Nishida, S. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.10.28.514329 · Cited by 3
Bilinguals' metacognition is affected by the language used as a medium of education
Ordin, M., & Polyanskaya, L. (2025) · Review of Education · doi:10.1002/rev3.70088 · Cited by 2
How much can children see and report about their experience of a brief glance at a natural scene?
Watanabe, R., Tsuchiya, N., Qianchen, L., Myowa, M., & Moriguchi, Y. (2025) · Neuroscience of Consciousness · doi:10.1093/nc/niaf019 · Cited by 4
“Extreme values of Type 1 responses (e.g. all responses in signal present trials to be yes) are known to produce invalid Type 2 AUC estimates (Galvin et al. 2003, Barrett et al. 2013).”
Detecting and identifying offset gaze
Balsdon, T., & Clifford, C. W. G. (2017) · Attention Perception & Psychophysics · doi:10.3758/s13414-017-1347-0 · Cited by 3
“One way to test this assumption is to use metacognitive ratings (Timmermans and Cleeremans, 2015; Barrett et al., 2013; but see also Jachs et al., 2015).”
MethodRuby, E., Maniscalco, B., Lau, H. C., & Peters, M. A. K. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/198218 · Cited by 3
Confiance, métacognition et perception
Massoni, S. (2017) · L Actualité économique · doi:10.7202/1039885ar · Cited by 3
Metacognition analysis of five grade students in elementary school on numbers
Triwahyuningtyas, D., & Sesanti, N. R. (2023) · International Journal of Evaluation and Research in Education (IJERE) · doi:10.11591/ijere.v12i1.23233 · Cited by 8
Matsuyoshi, D., Isato, A., & Yamada, M. (2024) · BMC Psychology · doi:10.1186/s40359-024-01625-9 · Cited by 1
“The size of the fixation point is exaggerated for illustration purposes ability [1, 40], hierarchical Bayesian estimation allows for accurately estimating metacognitive measures even when low sensitivity is expected because of illusory percepts [41].”
BackgroundThe confidence-noise confidence-boost (CNCB) model of confidence rating data
Mamassian, P., & de Gardelle, V. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012451 · Cited by 2
Dunstone, J., Atkinson, M., Grainger, C., Renner, E., & Caldwell, C. A. (2021) · PLoS ONE · doi:10.1371/journal.pone.0247183 · Cited by 3
“Although the meta-d’ method of assessing metacognitive efficiency is designed to be robust against type-II response bias, the measure can be affected by very high or low values of type-II hit and false alarm rates [25].”
MethodAccessing the mind: relating metacognition to attention, memory, and cognitive function
Matthews, J. (2020) · Figshare · doi:10.26180/5c5a1486c5372 · Cited by 1
“…processes and uniting it with prevailing, signal-detection theoretic models of metacognition that assume second-order information is constituted by first-order information (i.e., bottom-up signals carry content) (Barrett et al., 2013) .”
Flexible social learning strategies are harder than the sum of their partsPreprint
Dunstone, J., Atkinson, M., Grainger, C., Renner, E., & Caldwell, C. A. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/xcyu9 · Cited by 1
Seth, A. K., Barrett, A. B., Schwartzman, D. J., & Bor, D. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/rhvtq · Cited by 1
Self-citationDo people know when they are good at spotting liars? – Metacognitive efficiency in Lie DetectionPreprint
Said, N., Volz, S., Reinhard, M., Müller, P. A., & Huff, M. (2022) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/v6nbd · Cited by 1
The Pragmatics of Confidence in Perceptual and Value-based Choice
Folke, N. E. T. (2018) · Apollo (University of Cambridge) · doi:10.17863/cam.22112 · Cited by 1
“Meta-d’ can be estimated via maximum likelihood methods or via other forms of analytic approximation (Barrett et al., 2013; Maniscalco & Lau, 2014).”
BackgroundMethodInfluentialNovel regression methods for metacognitionPreprint
Kristensen, S. B., Sandberg, K., & Bibby, B. M. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/423947
“In Barrett et al., 2013 and Maniscalco and Lau, 2014 θ in the type 2 task, called ˜ θ , is allowed to vary from that in the type 1 task.”
BackgroundMethodInfluentialRys, W. (2020) · Arrow@dit (Dublin Institute of Technology)
Dunstone, J., Atkinson, M., Renner, E., & Caldwell, C. A. (2022) · Entropy · doi:10.3390/e24030325
“The impact of the dual-task on participants’ metacognition was analysed using the metaSDT package in R [28].”
MethodInfluentialDiscrete processing in visual perception
Green, M. L. (2021) · doi:10.54718/vpvq5299
Emotional states and self-confidence independently fluctuate at different time scalesPreprint
da Fonseca, M., Maffei, G., Moreno‐Bote, R., & Hyafil, A. (2021) · doi:10.31234/osf.io/swpcf
d'o: Sensitivity at the optimal criterion locationPreprint
Aujla, H. (2022) · doi:10.31234/osf.io/ygkjn
Matsuyoshi, D., Isato, A., & Yamada, M. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.10.24.513462
“Although a perceptually-demanding task (i.e., visual illusion) often prevents us from estimating reliable metacognitive ability (1, 40), hierarchical Bayesian estimation allows for accurately estimating metacognitive measures even when expecting low perceptual sensitivity (41).”
BackgroundBasic Methods and Theoretical Development of Signal Detection Theory
SHIMAZU, N., & Okada, K. (2022) · Kodo Keiryogaku (The Japanese Journal of Behaviormetrics) · doi:10.2333/jbhmk.49.63 · Cited by 1
Bayesian Generalized Linear Mixed Effects Models for Deception Detection AnalysesPreprint
Zloteanu, M., & Vuorre, M. (2023) · doi:10.31234/osf.io/fdh5b
Single point estimation of a decision space
Aujla, H. (2025) · Behavior Research Methods · doi:10.3758/s13428-025-02768-2
Schmid, G. B. (2025) · Springer eBooks · doi:10.1007/978-3-662-70831-6_2
Information-Theoretic Measures of Metacognition: Bounds and Relation to Group Performance
Meyen, S., Göppert, F., Schrenk, C., von Luxburg, U., & Franz, V. H. (2025) · Open Mind · doi:10.1162/opmi.a.40 · Cited by 5
“The measures that normalize relative to the normal noise case, M − ratio and meta − I r 1 , achieve some degree of independence from accuracies (Barrett et al., 2013; Guggenmos, 2021, 2022) but transfer the problematic confound to the uniform noise case.”
On metacognition and the dynamics of selective attention
Recht, S. (2019) · theses.fr (ABES) · doi:10.70675/9091ccaez90a4z489fz8603z56a8a1538875
Towards a unified view of metacognition : Insights from metamemory
Mazancieux, A. (2020) · theses.fr (ABES) · doi:10.70675/f37dcebbz4e30z4da4za014z74039ef3666f · Cited by 1
Li, G. Y., Chen, L., Tang, C., Ma, B., Jiang, Y., Deguchi, D., Yamashita, T., & Shimada, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29755-6_2 · Cited by 1
Sociolinguistic Context Constrains Bilingual Advantages in Metacognition
Polyanskaya, L., & Ordin, M. (2026) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.70350
“To estimate the metacognitive sensitivity measures, we used a signal detection approach [ 18, 19, 87, 88 ] based on the conceptualization of correct answers with high confidence considered as meta-hits, correct answers with low confidence as meta-misses, wrong answers with high confidence as…”
Metadata
- Authors
- Barrett, A. B., Dienes, Z., & Seth, A. K.
- Affiliations
- University of Sussex (GB)
- Venue
- Psychological Methods
- Published
- 2013
- Volume
- 18
- Issue
- 4
- Pages
- 535-552
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2006647339
- Cited by
- 155
- References
- 55
- Citations by year
- 2026: 2 · 2025: 13 · 2024: 6 · 2023: 16 · 2022: 18 · 2021: 16 · 2020: 13 · 2019: 11 · 2018: 12 · 2017: 13 · 2016: 12 · 2015: 9 · 2014: 11 · 2013: 3
- FWCI
- 6.37
- Type
- journal article
- Language
- en
- Open access
- green
- figshare.com
- Licence
- cc-by
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Sussex Research Online (University of Sussex) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Funders
- Dr. Mortimer and Dame Theresa Sackler Foundation
- Engineering and Physical Sciences Research Council (EP/G007543/1)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- metacognition, signal detection theory, metacognitive sensitivity, response bias, metacognitive judgments, metacognitive accuracy, metacognitive confidence, metacognitive monitoring
- MeSH terms
- Cognition, Data Interpretation, Statistical, Humans, Models, Psychological, Signal Detection, Psychological
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.