The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
HMeta-d: hierarchical Bayesian estimation of metacognitive efficiency from confidence ratings
Open accessAbstract
Metacognition refers to the ability to reflect on and monitor one’s cognitive processes, such as perception, memory and decision-making. Metacognition is often assessed in the lab by whether an observer’s confidence ratings are predictive of objective success, but simple correlations between performance and confidence are susceptible to undesirable influences such as response biases. Recently an alternative approach to measuring metacognition has been developed (Maniscalco & Lau, 2012) that characterises metacognitive sensitivity (meta- d ′) by assuming a generative model of confidence within the framework of signal detection theory. However, current estimation routines require an abundance of confidence rating data to recover robust parameters, and only provide point estimates of meta- d ’. In contrast, hierarchical Bayesian estimation methods provide opportunities to enhance statistical power, incorporate uncertainty in group-level parameter estimates and avoid edge-correction confounds. Here I introduce such a method for estimating metacognitive efficiency (meta- d ’/ d ’) from confidence ratings and demonstrate its application for assessing group differences. A tutorial is provided on both the meta- d ’ model and the preparation of behavioural data for model fitting. Through numerical simulations I show that a hierarchical approach outperforms alternative fitting methods in situations where limited data are available, such as when quantifying metacognition in patient populations. In addition, the model may be flexibly expanded to estimate parameters encoding other influences on metacognitive efficiency. MATLAB software and documentation for implementing hierarchical meta- d ’ estimation (HMeta-d) can be downloaded at https://github.com/smfleming/HMeta-d .
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Inference from iterative simulation using multiple sequences
Gelman, A., & Rubin, D. B. (1992) · Statistical Science · doi:10.1214/ss/1177011136 · Cited by 16,627
Data Analysis Using Regression and Multilevel/Hierarchical Models
Gelman, A., & Hill, J. (2006) · doi:10.1017/cbo9780511790942 · Cited by 13,597
Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry
Flavell, J. H. (1979) · American Psychologist · doi:10.1037/0003-066x.34.10.906 · Cited by 11,282 · cited 1×
“Metacognition is defined as ‘knowledge of one’s own cognitive processes’ (Flavell 1979).”
BackgroundStatistical Methods for Psychology.
Gudgeon, A. C., & Howell, D. C. (1994) · The Statistician · doi:10.2307/2348956 · Cited by 8,278
Judgment Under Uncertainty: Heuristics and Biases.
Shafer, G. R., Kahnerman, D., Slovic, P., & Tversky, A. (1984) · Journal of the American Statistical Association · doi:10.2307/2288362 · Cited by 5,657
Detection Theory: A User's Guide
Macmillan, N. A., & Creelman, C. D. (1991) · Cited by 5,415
JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling
Plummer, M. (2003) · Cited by 4,720
Judgment under uncertainty: Heuristics and biases
Tversky, A., & Kahneman, D. (1982) · Judgment under Uncertainty · doi:10.1017/cbo9780511809477.002 · Cited by 3,643
Detection theory: A user's guide
Lories, G. (1993) · Behavioural Processes · doi:10.1016/0376-6357(93)90083-4 · Cited by 2,835
Calibration of probabilities: the state of the art to 1980
Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982) · Judgment under Uncertainty · doi:10.1017/cbo9780511809477.023 · Cited by 1,967
Cited by
Towards a comprehensive assessment of interoception in a multi-dimensional framework
Suksasilp, C., & Garfinkel, S. N. (2022) · Biological Psychology · doi:10.1016/j.biopsycho.2022.108262 · Cited by 288
Morales, J., Lau, H., & Fleming, S. M. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.2360-17.2018 · Cited by 274
Self-citationMetacognitive Failure as a Feature of Those Holding Radical Beliefs
Rollwage, M., Dolan, R. J., & Fleming, S. M. (2018) · Current Biology · doi:10.1016/j.cub.2018.10.053 · Cited by 193
Self-citationHuman Metacognition Across Domains: Insights from Individual Differences and Neuroimaging
Rouault, M., McWilliams, A., Allen, M. G., & Fleming, S. M. (2018) · Personality Neuroscience · doi:10.1017/pen.2018.16 · Cited by 176
Self-citationAbnormalities of confidence in psychiatry: an overview and future perspectives
Hoven, M., Lebreton, M., Engelmann, J. B., Denys, D., Luigjes, J., & van Holst, R. J. (2019) · Translational Psychiatry · doi:10.1038/s41398-019-0602-7 · Cited by 171
Interoception of breathing and its relationship with anxiety
Harrison, O. K., Köchli, L., Marino, S., Luechinger, R., Hennel, F., Brand, K., Hess, A. J., Frässle, S., Iglesias, S., Vinckier, F., et al. (2021) · Neuron · doi:10.1016/j.neuron.2021.09.045 · Cited by 163
How Local and Global Metacognition Shape Mental Health
Seow, T. X. F., Rouault, M., Gillan, C. M., & Fleming, S. M. (2021) · Biological Psychiatry · doi:10.1016/j.biopsych.2021.05.013 · Cited by 159
Self-citationPitts, M. A., Lutsyshyna, L. A., & Hillyard, S. A. (2018) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2017.0348 · Cited by 135
Forming global estimates of self-performance from local confidence
Rouault, M., Dayan, P., & Fleming, S. M. (2019) · Nature Communications · doi:10.1038/s41467-019-09075-3 · Cited by 134
Self-citationMazancieux, A., Fleming, S. M., Souchay, C., & Moulin, C. J. A. (2020) · Journal of Experimental Psychology: General · doi:10.1037/xge0000746 · Cited by 131
“By estimating meta-d= in a Bayesian hierarchical framework (Fleming, 2017), it is possible to directly estimate covariance in metacognitive efficiencies across domains.”
BackgroundMethodSelf-citationRespiration, Heartbeat, and Conscious Tactile Perception
Grund, M., Al, E., Pabst, M., Dabbagh, A., Stephani, T., Nierhaus, T., Gaebler, M., & Villringer, A. (2022) · The Journal of Neuroscience · doi:10.1523/jneurosci.0592-21.2021 · Cited by 121
If perception is probabilistic, why does it not seem probabilistic?
Block, N. (2018) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2017.0341 · Cited by 91
The relationship between heartbeat counting and heartbeat discrimination: A meta-analysis
Hickman, L., Seyedsalehi, A., Cook, J. L., Bird, G., & Murphy, J. (2020) · Biological Psychology · doi:10.1016/j.biopsycho.2020.107949 · Cited by 90
The accuracy of German citizens’ confidence in their climate change knowledge
Fischer, H., Amelung, D., & Said, N. (2019) · Nature Climate Change · doi:10.1038/s41558-019-0563-0 · Cited by 78
Formation of global self-beliefs in the human brain
Rouault, M., & Fleming, S. M. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2003094117 · Cited by 76
Self-citationModeling perceptual confidence and the confidence forced-choice paradigm.
Mamassian, P., & de Gardelle, V. (2022) · Psychological Review · doi:10.1037/rev0000312 · Cited by 70
Measuring metacognitive performance: type 1 performance dependence and test-retest reliability
Guggenmos, M. (2021) · Neuroscience of Consciousness · doi:10.1093/nc/niab040 · Cited by 66
The future of metacognition research: Balancing construct breadth with measurement rigor
Katyal, S., & Fleming, S. M. (2024) · Cortex · doi:10.1016/j.cortex.2023.11.002 · Cited by 63
“In what follows, we suggest approaches to redressing this balance.”
BackgroundMethodSelf-citationDynamic influences on static measures of metacognition
Desender, K., Vermeylen, L., & Verguts, T. (2022) · Nature Communications · doi:10.1038/s41467-022-31727-0 · Cited by 62
Identifying cultural differences in metacognition.
van der Plas, E., Zhang, S., Dong, K., Bang, D., Li, J., Wright, N. D., & Fleming, S. M. (2022) · Journal of Experimental Psychology: General · doi:10.1037/xge0001209 · Cited by 32
Self-citationHoven, M., Rouault, M., van Holst, R., & Luigjes, J. (2023) · Psychological Medicine · doi:10.1017/s003329172300209x · Cited by 28
Blaming luck, claiming skill: Self-attribution bias in error assignment
Okamoto, N., Taylor, M., Kubo, T., Ishii, S., De Martino, B., & Cortese, A. (2025) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1013787 · Cited by 2
Metacognition in anxious-depression is state-dependent: an observational treatment study
Fox, C. A., Lee, C. T., Hanlon, A. K., Seow, T. X., Lynch, K., Harty, S., Richards, D., Palacios, J., O’Keane, V., Stephan, K. E., et al. (2023) · doi:10.7554/elife.87193.2 · Cited by 1
Metadata
- Authors
- Fleming, S. M.
- Venue
- Neuroscience of Consciousness
- Publisher
- Oxford University Press (OUP)
- Published
- 2017-01-01
- Volume
- 2017
- Issue
- 1
- Pages
- nix007-nix007
- ISSN
- 2057-2107 (ISSN-L)
- OpenAlex
- W2953219509
- Cited by
- 309
- References
- 58
- FWCI
- 8.45
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.