CoChat
Sign Up
Cited by 63Memory Processes and InfluencesNeural and Behavioral Psychology StudiesEpistemology, Ethics, and Metaphysics

The future of metacognition research: Balancing construct breadth with measurement rigor

Open access
Katyal, Sucharit, Fleming, Stephen M.·2024·Cortex
DOI 10.1016/j.cortex.2023.11.002PMC PMC11139654

Published version of a preprint

Open this paper in CoChat — free account

Abstract

Foundational work in the psychology of metacognition identified a distinction between metacognitive knowledge (stable beliefs about one's capacities) and metacognitive experiences (local evaluations of performance). More recently, the field has focused on developing tasks and metrics that seek to identify metacognitive capacities from momentary estimates of confidence in performance, and providing precise computational accounts of metacognitive failure. However, this notable progress in formalising models of metacognitive judgments may come at a cost of ignoring broader elements of the psychology of metacognition - such as how stable meta-knowledge is formed, how social cognition and metacognition interact, and how we evaluate affective states that do not have an obvious ground truth. We propose that construct breadth in metacognition research can be restored while maintaining rigour in measurement, and highlight promising avenues for expanding the scope of metacognition research. Such a research programme is well placed to recapture qualitative features of metacognitive knowledge and experience while maintaining the psychophysical rigor that characterises modern research on confidence and performance monitoring.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.cortex.2023.11.002

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The quality of generated content and its uses may be more difficult to evaluate objectively.

    “but the quality of generated content and its uses may be more difficult to evaluate objectively (e.g., consider how one would objectively evaluate the quality of an LLM-generated email) [87,99].”

    From The Metacognitive Demands and Opportunities of Generative AI (2024) · cites this paper as [87]

    The Affective metacognition section (page unavailable) says that some subjective evaluations lack an objective ground truth, supporting the general difficulty of objective assessment. It does not address the quality or uses of generated content, including LLM-generated emails. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

References 137 · Cited by 63

Research works printed but not linked: 84

  1. Aitchison L. Lengyel M. With or without you: Predictive coding and Bayesian inference in the brain Current Opinion in Neurobiology 46 2017 219 227 10.1016/j.conb.2017.08.010 28942084 PMC5836998 · DOI · Paper page
  2. Allen M. Frank D. Schwarzkopf D.S. Fardo F. Winston J.S. Hauser T.U. Unexpected arousal modulates the influence of sensory noise on confidence Elife 5 2016 e18103 PMC5079750 27776633
  3. Arbuzova P. Peters C. Röd L. Koß C. Maurer H. Maurer L.K. Measuring metacognition of direct and indirect parameters of voluntary movement Journal of Experimental Psychology: General 150 11 2021 2208 2229 10.1037/xge0000892 33900097 · DOI
  4. Bandura A. Self-efficacy: Toward a unifying theory of behavioral change Psychological Review 84 2 1977 191 847061
  5. Barrett L.F. Simmons W.K. Interoceptive predictions in the brain Nature Reviews Neuroscience 16 7 2015 7 10.1038/nrn3950 PMC4731102 26016744 · DOI · Paper page
  6. Beck B. Peña-Vivas V. Fleming S. Haggard P. Metacognition across sensory modalities: Vision, warmth, and nociceptive pain Cognition 186 2019 32 41 10.1016/j.cognition.2019.01.018 30739057 PMC6411924 · DOI
  7. Bocanegra B.R. Poletiek F.H. Ftitache B. Clark A. Intelligent problem-solvers externalize cognitive operations Nature Human Behaviour 3 2 2019 2 10.1038/s41562-018-0509-y 30944451 · DOI
  8. Boorman E.D. O'Doherty J.P. Adolphs R. Rangel A. The behavioral and neural mechanisms underlying the tracking of expertise Neuron 80 6 2013 1558 1571 10.1016/j.neuron.2013.10.024 24360551 PMC3878380 · DOI
  9. Bornstein A.M. Khaw M.W. Shohamy D. Daw N.D. Reminders of past choices bias decisions for reward in humans Nature Communications 8 1 2017 1 10.1038/ncomms15958 PMC5490260 28653668 · DOI
  10. Boundy-Singer Z.M. Ziemba C.M. Goris R.L.T. Confidence reflects a noisy decision reliability estimate Nature Human Behaviour 2022 1 13 10.1038/s41562-022-01464-x 36344656 · DOI · Paper page
  11. Bulley A. Schacter D.L. Deliberating trade-offs with the future Nature Human Behaviour 4 3 2020 3 10.1038/s41562-020-0834-9 PMC7147875 32184495 · DOI
  12. Carruthers P. How we know our own minds: The relationship between mindreading and metacognition Behavioral and Brain Sciences 32 2 2009 121 19386144
  13. Carruthers P. The opacity of mind: An integrative theory of self-knowledge 2011 OUP Oxford
  14. Cavalan Q. Vergnaud J.-C. de Gardelle V. From local to global estimations of confidence in perceptual decisions Journal of Experimental Psychology: General 152 9 2023 2544 2558 10.1037/xge0001411 37155285 · DOI
  15. Clarke F.R. Birdsall T.G. Tanner W.P. Jr. Two types of ROC curves and definitions of parameters The Journal of the Acoustical Society of America 31 5 1959 629 630
  16. De Martino B. Fleming S.M. Garrett N. Dolan R.J. Confidence in value-based choice Nature Neuroscience 16 1 2013 105 110 10.1038/nn.3279 23222911 PMC3786394 · DOI · Paper page
  17. Denison R.N. Adler W.T. Carrasco M. Ma W.J. Humans incorporate attention-dependent uncertainty into perceptual decisions and confidence Proceedings of the National Academy of Sciences 115 43 2018 11090 11095 10.1073/pnas.1717720115 PMC6205425 30297430 · DOI · Paper page
  18. Desender K. Vermeylen L. Verguts T. Dynamic influences on static measures of metacognition Nature Communications 13 1 2022 1 10.1038/s41467-022-31727-0 PMC9301893 35864100 · DOI · Paper page
  19. Dixon R.A. Hultsch D.F. Hertzog C. The metamemory in adulthood (MIA) questionnaire Psychopharmacology Bulletin 24 4 1988 671 688 3249770
  20. Donoso M. Collins A.G.E. Koechlin E. Foundations of human reasoning in the prefrontal cortex Science 344 6191 2014 1481 1486 10.1126/science.1252254 24876345 · DOI
  21. Embon I. Cukier S. Iorio A. Barttfeld P. Solovey G. Is visual metacognition associated with ASD traits? A regression analysis shows no link between visual metacognition and AQ scores PsyArXiv 2022 10.31234/osf.io/nb78j 36934669 · DOI
  22. Fandakova Y. Selmeczy D. Leckey S. Grimm K.J. Wendelken C. Bunge S.A. Changes in ventromedial prefrontal and insular cortex support the development of metamemory from childhood into adolescence Proceedings of the National Academy of Sciences 114 29 2017 7582 7587 10.1073/pnas.1703079114 PMC5530674 28673976 · DOI
  23. Fitzgerald L.M. Arvaneh M. Dockree P.M. Domain-specific and domain-general processes underlying metacognitive judgments Consciousness and Cognition 49 2017 264 277 10.1016/j.concog.2017.01.011 28222381 · DOI
  24. Fleming S.M. HMeta-d: Hierarchical Bayesian estimation of metacognitive efficiency from confidence ratings Neuroscience of Consciousness 2017 1 2017 10.1093/nc/nix007 PMC5858026 29877507 · DOI
  25. Fleming S.M. Daw N.D. Self-evaluation of decision-making: A general Bayesian framework for metacognitive computation Psychological Review 124 1 2017 91 28004960 PMC5178868
  26. Fleming S.M. Ryu J. Golfinos J.G. Blackmon K.E. Domain-specific impairment in metacognitive accuracy following anterior prefrontal lesions Brain: a Journal of Neurology 137 10 2014 2811 2822 10.1093/brain/awu221 25100039 PMC4163038 · DOI
  27. Fleming S.M. Weil R.S. Nagy Z. Dolan R.J. Rees G. Relating Introspective accuracy to individual differences in brain Structure Science 329 5998 2010 1541 1543 10.1126/science.1191883 20847276 PMC3173849 · DOI · Paper page
  28. Fleur D.S. Bredeweg B. van den Bos W. Metacognition: Ideas and insights from neuro- and educational sciences Npj Science of Learning 6 1 2021 1 10.1038/s41539-021-00089-5 PMC8187395 34103531 · DOI
  29. Galvin S.J. Podd J.V. Drga V. Whitmore J. Type 2 tasks in the theory of signal detectability: Discrimination between correct and incorrect decisions Psychonomic Bulletin & Review 10 4 2003 843 876 10.3758/BF03196546 15000533 · DOI · Paper page
  30. Garfinkel S.N. Critchley H.D. Threat and the body: How the heart supports fear processing Trends in Cognitive Sciences 20 1 2016 34 46 10.1016/j.tics.2015.10.005 26628111 · DOI · Paper page
  31. Gilewski M.J. Zelinski E.M. Schaie K.W. The Memory Functioning Questionnaire for assessment of memory complaints in adulthood and old age Psychology and Aging 5 1990 482 490 10.1037/0882-7974.5.4.482 2278670 · DOI · Paper page
  32. Goodman L.A. Kruskal W.H. Measures of association for Cross Classifications Goodman L.A. Kruskal W.H. Measures of association for Cross Classifications 1979 Springer 2 34 10.1007/978-1-4612-9995-0_1 · DOI
  33. Guggenmos M. Measuring metacognitive performance: Type 1 performance dependence and test-retest reliability Neuroscience of Consciousness 2021 1 2021 niab040 10.1093/nc/niab040 34858637 PMC8633424 · DOI · Paper page
  34. Guggenmos M. Reverse engineering of metacognition eLife 11 2022 e75420 10.7554/eLife.75420 PMC9477496 36107147 · DOI
  35. Harrison O.K. Garfinkel S.N. Marlow L. Finnegan S.L. Marino S. Köchli L. The Filter Detection Task for measurement of breathing-related interoception and metacognition Biological Psychology 165 2021 108185 10.1016/j.biopsycho.2021.108185 34487805 · DOI
  36. Hart J.T. Memory and the feeling-of-knowing experience Journal of Educational Psychology 56 1965 208 216 10.1037/h0022263 5825050 · DOI
  37. Hembacher E. Ghetti S. Don't look at my answer: Subjective uncertainty underlies preschoolers' Exclusion of their least accurate memories Psychological Science 25 9 2014 1768 1776 10.1177/0956797614542273 25015686 · DOI · Paper page
  38. Hoven M. Luigjes J. Denys D. Rouault M. van Holst R.J. How do confidence and self-beliefs relate in psychopathology: A transdiagnostic approach Nature Mental Health 1 5 2023 5 10.1038/s44220-023-00062-8 · DOI
  39. Ickes W. Empathic accuracy Journal of Personality 61 4 1993 587 610
  40. Janowsky J.S. Shimamura A.P. Squire L.R. Memory and metamemory: Comparisons between patients with frontal lobe lesions and amnesic patients Psychobiology 17 1 1989 3 11 10.3758/BF03337811 · DOI
  41. Jansen R.A. Rafferty A.N. Griffiths T.L. A rational model of the Dunning–Kruger effect supports insensitivity to evidence in low performers Nature Human Behaviour 5 6 2021 6 10.1038/s41562-021-01057-0 33633375 · DOI · Paper page
  42. Johnstone A. Friston K. Rees G. Lawson R.P. Metacognitive and noradrenergic differences in autistic adults PsyArXiv 2022 10.31234/osf.io/d2f68 · DOI
  43. Jönsson F.U. Olsson M.J. Olfactory metacognition Chemical Senses 28 7 2003 651 658 10.1093/chemse/bjg058 14578127 · DOI
  44. Katyal S. Huys Q. Dolan R. Fleming S. How underconfidence is maintained in anxiety and depression PsyArXiv 2023 10.31234/osf.io/qcg92 · DOI
  45. Kelly K.J. Metcalfe J. Metacognition of emotional face recognition Emotion 11 2011 896 906 10.1037/a0023746 21859205 · DOI
  46. Kepecs A. Mainen Z.F. A computational framework for the study of confidence in humans and animals Philosophical Transactions of the Royal Society B: Biological Sciences 367 1594 2012 1322 1337 10.1098/rstb.2012.0037 PMC3318772 22492750 · DOI · Paper page
  47. Kepecs A. Uchida N. Zariwala H.A. Mainen Z.F. Neural correlates, computation and behavioural impact of decision confidence Nature 455 7210 2008 7210 10.1038/nature07200 18690210 · DOI · Paper page
  48. Koriat A. The self-consistency model of subjective confidence Psychological Review 119 2012 80 113 10.1037/a0025648 22022833 · DOI
  49. Kruger J. Dunning D. Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments Journal of Personality and Social Psychology 77 6 1999 1121 10626367
  50. Lebreton M. Abitbol R. Daunizeau J. Pessiglione M. Automatic integration of confidence in the brain valuation signal Nature Neuroscience 18 8 2015 8 10.1038/nn.4064 26192748 · DOI · Paper page
  51. Li H.-H. Ma W.J. Confidence reports in decision-making with multiple alternatives violate the Bayesian confidence hypothesis Nature Communications 11 1 2020 1 10.1038/s41467-020-15581-6 PMC7181620 32332712 · DOI
  52. Lockl K. Schneider W. Knowledge about the mind: Links between theory of mind and later metamemory Child Development 78 1 2007 148 167 10.1111/j.1467-8624.2007.00990.x 17328698 · DOI
  53. Mamassian P. Visual confidence Annual Review of Vision Science 2 1 2016 459 481 10.1146/annurev-vision-111815-114630 28532359 · DOI
  54. Maniscalco B. Lau H. A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings Consciousness and Cognition 21 1 2012 422 430 10.1016/j.concog.2011.09.021 22071269 · DOI · Paper page
  55. Marcke H.V. Denmat P.L. Verguts T. Desender K. Manipulating prior beliefs causally induces under- and overconfidence (p. 2022.03.01.482511) bioRxiv 2022 10.1101/2022.03.01.482511 38427319 · DOI · Paper page
  56. McIntosh R.D. Moore A.B. Liu Y. Della Sala S. Skill and self-knowledge: Empirical refutation of the dual-burden account of the Dunning–Kruger effect Royal Society Open Science 9 12 2022 191727 10.1098/rsos.191727 36483762 PMC9727674 · DOI
  57. Metcalfe B. D. of P. J Metcalfe J. Shimamura A.P. Metacognition: Knowing about knowing 1994 MIT Press
  58. Meyniel F. Sigman M. Mainen Z.F. Confidence as Bayesian probability: From neural origins to behavior Neuron 88 1 2015 78 92 10.1016/j.neuron.2015.09.039 26447574 · DOI · Paper page
  59. Morales J. Lau H. Fleming S.M. Domain-general and domain-specific patterns of activity supporting metacognition in human prefrontal cortex Journal of Neuroscience 38 14 2018 3534 3546 10.1523/JNEUROSCI.2360-17.2018 29519851 PMC5895040 · DOI · Paper page
  60. Nelson T. Narens L. Metamemory: A theoretical framework and new findings Psychology of learning and motivation 26 1990 Academic Press 125 173 10.1016/S0079-7421(08)60053-5 · DOI · Paper page
  61. Patel D. Fleming S.M. Kilner J.M. Inferring subjective states through the observation of actions Proceedings of the Royal Society B: Biological Sciences 279 1748 2012 4853 4860 10.1098/rspb.2012.1847 PMC3497093 23034708 · DOI
  62. Pearson J. Rademaker R.L. Tong F. Evaluating the mind's Eye: The metacognition of visual imagery Psychological Science 22 12 2011 1535 1542 10.1177/0956797611417134 22058106 · DOI
  63. Peters M.A.K. Towards characterizing the canonical computations generating phenomenal experience Neuroscience and Biobehavioral Reviews 142 2022 104903 10.1016/j.neubiorev.2022.104903 36202256 · DOI
  64. Pouget A. Drugowitsch J. Kepecs A. Confidence and certainty: Distinct probabilistic quantities for different goals Nature Neuroscience 19 3 2016 366 374 10.1038/nn.4240 26906503 PMC5378479 · DOI · Paper page
  65. Rahnev D. Fleming S.M. How experimental procedures influence estimates of metacognitive ability Neuroscience of Consciousness 2019 1 2019 10.1093/nc/niz009 PMC6556214 31198586 · DOI
  66. Rahnev D. Koizumi A. McCurdy L.Y. D'Esposito M. Lau H. Confidence leak in perceptual decision making Psychological Science 26 11 2015 1664 1680 10.1177/0956797615595037 26408037 PMC4636919 · DOI
  67. Rosenbaum G.M. Grassie H.L. Hartley C.A. Valence biases in reinforcement learning shift across adolescence and modulate subsequent memory eLife 11 2022 e64620 10.7554/eLife.64620 PMC8786311 35072624 · DOI
  68. Rouault M. Fleming S.M. Formation of global self-beliefs in the human brain Proceedings of the National Academy of Sciences 117 44 2020 27268 27276 10.1073/pnas.2003094117 PMC7959580 33060292 · DOI · Paper page
  69. Rouault M. Seow T. Gillan C.M. Fleming S.M. Psychiatric symptom dimensions are associated with dissociable shifts in metacognition but not task performance Biological Psychiatry 84 6 2018 443 451 10.1016/j.biopsych.2017.12.017 29458997 PMC6117452 · DOI
  70. Rouault M. Will G.-J. Fleming S.M. Dolan R.J. Low self-esteem and the formation of global self-performance estimates in emerging adulthood Translational Psychiatry 12 1 2022 1 10.1038/s41398-022-02031-8 PMC9276660 35821225 · DOI
  71. Rouy M. de Gardelle V. Reyes G. Sackur J. Vergnaud J.C. Filevich E. Metacognitive improvement: Disentangling adaptive training from experimental confounds Journal of experimental psychology: General, No pagination Specified-No pagination Specified 2022 10.1037/xge0001185 35157481 · DOI
  72. Shekhar M. Rahnev D. Distinguishing the roles of dorsolateral and anterior PFC in visual metacognition Journal of Neuroscience 38 22 2018 5078 5087 10.1523/JNEUROSCI.3484-17.2018 29720553 PMC6705938 · DOI · Paper page
  73. Shekhar M. Rahnev D. Sources of metacognitive inefficiency Trends in Cognitive Sciences 25 1 2021 12 23 10.1016/j.tics.2020.10.007 33214066 PMC8610081 · DOI
  74. Shekhar M. Rahnev D. The nature of metacognitive inefficiency in perceptual decision making Psychological Review 128 1 2021 45 70 10.1037/rev0000249 32673034 PMC7883626 · DOI
  75. Skov M. Nadal M. A farewell to art: Aesthetics as a topic in psychology and neuroscience Perspectives on Psychological Science 15 3 2020 630 642 10.1177/1745691619897963 32027577 · DOI
  76. Trudel N. Rushworth M.F. Wittmann M.K. Neural activity tracking identity and confidence in social information (p. 2021.07.06.449597) bioRxiv 2021 10.1101/2021.07.06.449597 PMC9917428 36763582 · DOI
  77. Vaccaro A.G. Fleming S.M. Thinking about thinking: A coordinate-based meta-analysis of neuroimaging studies of metacognitive judgements Brain and Neuroscience Advances 2 2018 10.1177/2398212818810591 2398212818810591 PMC6238228 30542659 · DOI · Paper page
  78. van der Plas E. Mason D. Livingston L.A. Craigie J. Happe F. Fleming S. Computations of confidence are modulated by mentalizing ability PsyArXiv 2021 10.31234/osf.io/c4pzj · DOI
  79. Vickers D. Decision processes in visual perception 1979 Academic Press
  80. Wittmann M.K. Kolling N. Faber N.S. Scholl J. Nelissen N. Rushworth M.F.S. Self-other mergence in the frontal cortex during Cooperation and Competition Neuron 91 2 2016 482 493 10.1016/j.neuron.2016.06.022 27477020 PMC4961240 · DOI · Paper page
  81. Yeung N. Summerfield C. Metacognition in human decision-making: Confidence and error monitoring Philosophical Transactions of the Royal Society B: Biological Sciences 367 1594 2012 1310 1321 10.1098/rstb.2011.0416 PMC3318764 22492749 · DOI · Paper page
  82. Ye Q. Zou F. Lau H. Hu Y. Kwok S.C. Causal evidence for mnemonic metacognition in human precuneus The Journal of Neuroscience: the Official Journal of the Society for Neuroscience 38 28 2018 6379 6387 10.1523/JNEUROSCI.0660-18.2018 29921714 PMC6041789 · DOI
  83. Yon D. Frith C.D. Precision and the Bayesian brain Current Biology 31 17 2021 R1026 R1032 10.1016/j.cub.2021.07.044 34520708 · DOI
  84. Zheng Y. Wang D. Ye Q. Zou F. Li Y. Kwok S.C. Diffusion property and functional connectivity of superior longitudinal fasciculus underpin human metacognition Neuropsychologia 156 2021 107847 10.1016/j.neuropsychologia.2021.107847 33812946 · DOI

Authors
Katyal, S., & Fleming, S. M.
Affiliations
Wellcome Centre for Human Neuroimaging (GB)
Corresponding author
  • Katyal, Sucharit
  • Fleming, Stephen M.
ORCID
Venue
Cortex
Publisher
Elsevier BV
Published
2024-02
Volume
171
Pages
223-234
ISSN
0010-9452 (ISSN-L)
DOI
10.1016/j.cortex.2023.11.002
PMC
PMC11139654
OpenAlex
W4388451761
Cited by
63
References
137
Citations by year
2026: 28 · 2025: 24 · 2024: 8 · 2023: 3
FWCI
9.43
Type
journal article
Language
en
Open access
Licence
Locations
  • Cortex (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Research at the University of Copenhagen (University of Copenhagen) (publishedVersion, cc-by)
  • UCL Discovery (University College London) (submittedVersion, cc-by)
  • PsyArXiv (OSF Preprints) (submittedVersion)
  • PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
Funders
  • Canadian Institute for Advanced Research
  • Wellcome Trust (203147/Z/16/Z, Z/17/Z, 203147, 206648/Z/17/Z, 203147/Z/16/)
  • Leverhulme Trust
  • Max-Planck-Gesellschaft (203147/Z/16/Z, 206648/Z/17/Z)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
  • Social Sciences › Arts and Humanities › Philosophy › Epistemology, Ethics, and Metaphysics
Keywords
metacognition, metacognitive knowledge, metacognitive experience, metacognitive confidence, social cognition, affective evaluation
MeSH terms
Metacognition, Humans, Judgment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review