Metacognitive resources for adaptive learning⋆
Open accessThe DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Abstract
Biological organisms display remarkably flexible behaviours. This is an area of active investigation, in particular in the fields of artificial intelligence, computational and cognitive neuroscience. While inductive biases and broader cognitive functions are undoubtedly important, the ability to monitor and evaluate one's performance or oneself -- metacognition -- strikes as a powerful resource for efficient learning. Often measured as decision confidence in neuroscience and psychology experiments, metacognition appears to reflect a broad range of abstraction levels and downstream behavioural effects. Within this context, the formal investigation of how metacognition interacts with learning processes is a recent endeavour. Of special interest are the neural and computational underpinnings of confidence and reinforcement learning modules. This review discusses a general hierarchy of confidence functions and their neuro-computational relevance for adaptive behaviours. It then introduces novel ways to study the formation and use of meta-representations and nonconscious mental representations related to learning and confidence, and concludes with a discussion on outstanding questions and wider perspectives.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817 · cited 2×
“Machines can reach and periodically surpass human performance in specific contexts [e.g., Atari games, or chess (Lake et al., 2015; Sorokin et al., 2015)].”
How to grow a mind: Statistics, structure, and abstraction
Tenenbaum, J. B., Kemp, C., Griffiths, T. L., & Goodman, N. D. (2011) · Science · doi:10.1126/science.1192788 · Cited by 1,815
The importance of mixed selectivity in complex cognitive tasks
Rigotti, M., Barak, O., Warden, M. R., Wang, X.-J., Daw, N. D., Miller, E. K., & Fusi, S. (2013) · Nature · doi:10.1038/nature12160 · Cited by 1,639 · cited 2×
“, as in mixed selectivity) (Fusi et al., 2016; Rigotti et al., 2013), (3) different variables encoded by different neurons within localized circuits.”
BackgroundNeuronal Coding of Prediction Errors
Schultz, W., & Dickinson, A. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.473 · Cited by 1,427 · cited 1×
“Yet somewhere, most likely the ventral striatum in the basal ganglia, neurons code for general prediction errors [as an abstract signal, arising from any dimension (Schultz and Dickinson, 2000)].”
Fleming, S. M., & Lau, H. C. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00443 · Cited by 1,408 · cited 2×
“Human studies have, instead, tended to incorporate broader views, focusing on both normative definitions of confidence (Sanders et al., 2016) as well as on psychological, cognitive aspects grounded in signal detection theory (Fleming and Lau, 2014; Rounis et al., 2010).”
Multiplexed echo planar imaging for sub-second whole brain FMRI and fast diffusion imaging
Feinberg, D. A., Moeller, S., Smith, S. M., Auerbach, E., Ramanna, S., Glasser, M. F., Miller, K. L., Ugurbil, K., & Yacoub, E. (2010) · PLoS ONE · doi:10.1371/journal.pone.0015710 · Cited by 1,296 · cited 1×
“…context multivoxel neural reinforcement (also known as decoded neurofeedback) is an attractive causal approach (Shibata et al., 2011; Taschereau-Dumouchel et al., 2020), which has evolved with recent developments in neuro-imaging (Feinberg et al., 2010; Haxby et al., 2014, 2011; Xu et al., 2013).”
Representation of Confidence Associated with a Decision by Neurons in the Parietal Cortex
Kiani, R., & Shadlen, M. N. (2009) · Science · doi:10.1126/science.1169405 · Cited by 1,027 · cited 3×
“Studies in monkeys tend to use opt-out tasks, whereby the animal can choose a safe option instead of a more rewarding but riskier discrimination (Kiani and Shadlen, 2009; Komura et al., 2013).”
Empirical support for higher-order theories of conscious awareness
Lau, H., & Rosenthal, D. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2011.05.009 · Cited by 869 · cited 1×
“It could take part in generating meta -representations – summarised re -repre- sentations of ongoing sensory or memory representations (Brown et al., 2019; Dehaene et al., 2017; Lau and Rosenthal, 2011).”
What is consciousness, and could machines have it?
Dehaene, S., Lau, H., & Kouider, S. (2017) · Science · doi:10.1126/science.aan8871 · Cited by 855 · cited 2×
“It could take part in generating meta -representations – summarised re -repre- sentations of ongoing sensory or memory representations (Brown et al., 2019; Dehaene et al., 2017; Lau and Rosenthal, 2011).”
Relating Introspective Accuracy to Individual Differences in Brain Structure
Fleming, S. M., Weil, R. S., Nagy, Z., Dolan, R. J., & Rees, G. (2010) · Science · doi:10.1126/science.1191883 · Cited by 809
Neural correlates, computation and behavioural impact of decision confidence
Kepecs, A., Uchida, N., Zariwala, H. A., & Mainen, Z. F. (2008) · Nature · doi:10.1038/nature07200 · Cited by 809 · cited 3×
“In rodents, the ACC has been found critical for confidence in visual modality (Stolyarova et al., 2019), while the orbitofrontal cortex (OFC) in olfactory modality (Kepecs et al., 2008; Lak et al., 2014).”
Why neurons mix: high dimensionality for higher cognition
Fusi, S., Miller, E. K., & Rigotti, M. (2016) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2016.01.010 · Cited by 766 · cited 1×
“…implements (1) single learning variables encoded in parallel across multiple regions, (2) a diversity of variables encoded by single neurons [i.e., as in mixed selectivity (Fusi et al., 2016; Rigotti et al., 2013)], (3) different variables encoded by different neurons within localized circuits.”
Rational regulation of learning dynamics by pupil-linked arousal systems
Nassar, M. R., Rumsey, K. M., Wilson, R. C., Parikh, K., Heasly, B., & Gold, J. I. (2012) · Nature Neuroscience · doi:10.1038/nn.3130 · Cited by 722 · cited 1×
“Confidence can control the learning rate - low confidence would engage higher learning rates because little is known about the environment or the current situation (Nassar et al., 2012, 2010).”
Decoding Neural Representational Spaces Using Multivariate Pattern Analysis
Haxby, J. V., Connolly, A. C., & Guntupalli, J. S. (2014) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-062012-170325 · Cited by 717 · cited 1×
“…context multivoxel neural reinforcement (also known as decoded neurofeedback) is an attractive causal approach (Shibata et al., 2011; Taschereau-Dumouchel et al., 2020), which has evolved with recent developments in neuro-imaging (Feinberg et al., 2010; Haxby et al., 2014, 2011; Xu et al., 2013).”
Metalearning and neuromodulation
Doya, K. (2002) · Neural Networks · doi:10.1016/s0893-6080(02)00044-8 · Cited by 704
Reinforcement learning, fast and slow
Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., & Hassabis, D. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.02.006 · Cited by 667
Prefrontal cortex as a meta-reinforcement learning system
Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., Hassabis, D., & Botvinick, M. (2018) · Nature Neuroscience · doi:10.1038/s41593-018-0147-8 · Cited by 596 · cited 2×
“This feature seems inherently important in a functional architecture of hierarchical learning systems such as the brain (Kawato et al., 1987; Wang et al., 2018), since it provides direct substrate for computations at high abstraction levels.”
Understanding the Higher-Order Approach to Consciousness
Brown, R., Lau, H., & LeDoux, J. E. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.06.009 · Cited by 573 · cited 2×
“Crucially, these representations should be found in the prefrontal cortex, as a relevant higher-order representation that directly stems from an inner monitoring system (Brown et al., 2019).”
Confidence in value-based choice
De Martino, B., Fleming, S. M., Garrett, N., & Dolan, R. J. (2013) · Nature Neuroscience · doi:10.1038/nn.3279 · Cited by 534 · cited 1×
“…several recent studies have found confidence to interact with reinforcement learning (Cortese et al., 2020; Lak et al., 2019, 2017), as well as guiding future choices (De Martino et al., 2012; Folke et al., 2016), and operating on reasoning / credit assignment (Sarafyazd and Jazayeri, 2019).”
Confidence and certainty: Distinct probabilistic quantities for different goals
Pouget, A., Drugowitsch, J., & Kepecs, A. (2016) · Nature Neuroscience · doi:10.1038/nn.4240 · Cited by 528 · cited 2×
“In terms of the terminology used – e.g., certainty vs confidence (Dehaene et al., 2017; Pouget et al., 2016), the computational approaches used to determine the basis of confidence judgements (Maniscalco et al., 2021; Meyniel et al., 2015; Peters and Lau, 2015; Pouget et al., 2016), and on the…”
Relative blindsight in normal observers and the neural correlate of visual consciousness
Lau, H. C., & Passingham, R. E. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0607716103 · Cited by 518 · cited 1×
“In humans, the PFC – and particularly the ros-trolateral and dorsolateral PFC (rlPFC/DLPFC) – is a central component of the explicit metacognitive / confidence circuitry (Cortese et al., 2016; Fleming et al., 2010; Lau and Passingham, 2006; Rounis et al., 2010).”
Forward frontal fields: phylogeny and fundamental function
Wise, S. P. (2008) · Trends in Neurosciences · doi:10.1016/j.tins.2008.08.008 · Cited by 478 · cited 1×
“This may be particularly true between rodents and non-human primates or humans (Wise, 2008), as the latter two display high anatomical and functional homology in terms of brain areas related to reward-guided learning and decisions (Neubert et al., 2015).”
Choice Certainty Is Informed by Both Evidence and Decision Time
Kiani, R., Corthell, L., & Shadlen, M. N. (2014) · Neuron · doi:10.1016/j.neuron.2014.12.015 · Cited by 470 · cited 1×
“In monkeys, neurons in the area LIP code for implicit confidence signals (Kiani et al., 2014; Kiani and Shadlen, 2009).”
A rational analysis of rule-based concept learning
Goodman, N. D., Tenenbaum, J. B., Feldman, J., & Griffiths, T. L. (2008) · Cognitive Science · doi:10.1080/03640210701802071 · Cited by 469
Post-decision wagering objectively measures awareness
Persaud, N., McLeod, P., & Cowey, A. (2007) · Nature Neuroscience · doi:10.1038/nn1840 · Cited by 454
Rounis, E., Maniscalco, B., Rothwell, J. C., Passingham, R. E., & Lau, H. (2010) · Cognitive Neuroscience · doi:10.1080/17588921003632529 · Cited by 407 · cited 2×
“In humans, the PFC – and particularly the ros-trolateral and dorsolateral PFC (rlPFC/DLPFC) – is a central component of the explicit metacognitive / confidence circuitry (Cortese et al., 2016; Fleming et al., 2010; Lau and Passingham, 2006; Rounis et al., 2010).”
Learning task-state representations
Niv, Y. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0470-8 · Cited by 356
Metacognition: computation, biology and function
Fleming, S. M., Dolan, R. J., & Frith, C. D. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2012.0021 · Cited by 344 · cited 1×
“Metacognition, or cognition about cognition, is the ability to reflect upon and report one ’ s own mental states (Fleming et al., 2012; Metcalfe and Son, 2012).”
A computational framework for the study of confidence in humans and animals
Kepecs, A., & Mainen, Z. F. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2012.0037 · Cited by 305
The construction of confidence in a perceptual decision
Zylberberg, A., Barttfeld, P., & Sigman, M. (2012) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2012.00079 · Cited by 297 · cited 1×
“Here the link develops over two levels: confidence itself relying more on the information that supports the decision made (Michel and Peters, 2020; Zylberberg et al., 2012), or the latest choices (Talluri et al., 2018), as well as a more general tendency of individuals with alterations in…”
Morales, J., Lau, H., & Fleming, S. M. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.2360-17.2018 · Cited by 274 · cited 1×
“Research has identified neural correlates of confidence operating in a single domain (e.g., perceptual only vs. memory only), as well as more abstract (e.g., both perception and memory) (McCurdy et al., 2013; Miyamoto et al., 2017; Morales et al., 2018).”
Automatic integration of confidence in the brain valuation signal
Lebreton, M., Abitbol, R., Daunizeau, J., & Pessiglione, M. (2015) · Nature Neuroscience · doi:10.1038/nn.4064 · Cited by 265
Signatures of a Statistical Computation in the Human Sense of Confidence
Sanders, J. I., Hangya, B., & Kepecs, A. (2016) · Neuron · doi:10.1016/j.neuron.2016.03.025 · Cited by 253 · cited 1×
“Human studies have, instead, tended to incorporate broader views, focusing on both normative definitions of confidence (Sanders et al., 2016) as well as on psychological, cognitive aspects grounded in signal detection theory (Fleming and Lau, 2014; Rounis et al., 2010).”
Responses of pulvinar neurons reflect a subject's confidence in visual categorization
Komura, Y., Nikkuni, A., Hirashima, N., Uetake, T., & Miyamoto, A. (2013) · Nature Neuroscience · doi:10.1038/nn.3393 · Cited by 226
Orbitofrontal Cortex Is Required for Optimal Waiting Based on Decision Confidence
Lak, A., Costa, G. M., Romberg, E., Koulakov, A. A., Mainen, Z. F., & Kepecs, A. (2014) · Neuron · doi:10.1016/j.neuron.2014.08.039 · Cited by 209
Distinct encoding of decision confidence in human medial prefrontal cortex
Bang, D., & Fleming, S. M. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1800795115 · Cited by 207 · cited 1×
“DLPFC activity correlated with the reported scalar confidence, while the pregenual anterior cingulate cortex (pgACC) contributed by forming an estimate of the probability of making a correct choice (Bang and Fleming, 2018; Mo-rales et al., 2018).”
Peters, M. A. K., & Lau, H. (2015) · eLife · doi:10.7554/elife.09651 · Cited by 197 · cited 1×
“…et al., 2016), the computational approaches used to determine the basis of confidence judgements (Maniscalco et al., 2021; Meyniel et al., 2015; Peters and Lau, 2015; Pouget et al., 2016), and on the crucial yet overlooked issue of confounding variables (Fleming and Lau, 2014; Maniscalco and…”
Metacognitive Failure as a Feature of Those Holding Radical Beliefs
Rollwage, M., Dolan, R. J., & Fleming, S. M. (2018) · Current Biology · doi:10.1016/j.cub.2018.10.053 · Cited by 193 · cited 1×
“…more on the information that supports the decision made (Michel and Peters, 2020; Zylberberg et al., 2012), or the latest choices (Talluri et al., 2018), as well as a more general tendency of individuals with alterations in metacognition to display high confirmation bias (Rollwage et al., 2018).”
Hierarchical reasoning by neural circuits in the frontal cortex
Sarafyazd, M., & Jazayeri, M. (2019) · Science · doi:10.1126/science.aav8911 · Cited by 191 · cited 2×
“When the reliability of the current world model – i.e., how well it predicts events in the world, or choice outcomes – goes down (Donoso et al., 2014), confidence could trigger changes in the course of action (Sarafyazd and Jazayeri, 2019).”
Midbrain dopamine neurons signal belief in choice accuracy during a perceptual decision
Lak, A., Nomoto, K., Keramati, M., Sakagami, M., & Kepecs, A. (2017) · Current Biology · doi:10.1016/j.cub.2017.02.026 · Cited by 170
Encoding of sensory prediction errors in the human cerebellum
Schlerf, J., Ivry, R. B., & Diedrichsen, J. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.4504-11.2012 · Cited by 168 · cited 1×
“Prediction errors offer a great example: correlates have been found in almost every region of the brain, from the cerebellum (Heffley and Hull, 2019; Schlerf et al., 2012) to the frontopolar cortex (Boorman et al., 2011).”
Multivoxel neurofeedback selectively modulates confidence without changing perceptual performance
Cortese, A., Amano, K., Koizumi, A., Kawato, M., & Lau, H. (2016) · Nature Communications · doi:10.1038/ncomms13669 · Cited by 157 · cited 1×
“In humans, the PFC – and particularly the ros-trolateral and dorsolateral PFC (rlPFC/DLPFC) – is a central component of the explicit metacognitive / confidence circuitry (Cortese et al., 2016; Fleming et al., 2010; Lau and Passingham, 2006; Rounis et al., 2010).”
Self-citationBengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1709.08568 · Cited by 155
Towards an unconscious neural reinforcement intervention for common fears
Taschereau-Dumouchel, V., Cortese, A., Chiba, T., Knotts, J. D., Kawato, M., & Lau, H. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1721572115 · Cited by 153
Self-citationRahnev, D., Desender, K., Lee, A. L. F., Adler, W. T., Aguilar-Lleyda, D., Akdoğan, B., Arbuzova, P., Atlas, L. Y., Balcı, F., Bang, J. W., et al. (2020) · Nature Human Behaviour · doi:10.1038/s41562-019-0813-1 · Cited by 151
Confidence controls perceptual evidence accumulation
Balsdon, T., Wyart, V., & Mamassian, P. (2020) · Nature Communications · doi:10.1038/s41467-020-15561-w · Cited by 137 · cited 1×
“Confidence has been shown to play a key function in controlling evidence accumulation online (Balsdon et al., 2020; Brosnan et al., 2020; Lim et al., 2020).”
Koizumi, A., Amano, K., Cortese, A., Shibata, K., Yoshida, W., Seymour, B., Kawato, M., & Lau, H. (2016) · Nature Human Behaviour · doi:10.1038/s41562-016-0006 · Cited by 133 · cited 1×
“This way, one can induce perceptual learning (Amano et al., 2016; Shibata et al., 2011), change confidence (Cortese et al., 2017, 2016; Koizumi et al., 2017), reduce fears (Koizumi et al., 2016; Taschereau-Dumouchel et al., 2018), and – presumably – elicit more complex secondary associations.”
Self-citationThe signal processing architecture underlying subjective reports of sensory awareness
Maniscalco, B., & Lau, H. (2016) · Neuroscience of Consciousness · doi:10.1093/nc/niw002 · Cited by 133
Behavior- and Modality-General Representation of Confidence in Orbitofrontal Cortex
Masset, P., Ott, T., Lak, A., Hirokawa, J., & Kepecs, A. (2020) · Cell · doi:10.1016/j.cell.2020.05.022 · Cited by 109 · cited 1×
“More recent work has shown that single OFC neurons compute abstract confidence, across sensory (olfactory and auditory) modalities (Masset et al., 2020).”
Causal neural network of metamemory for retrospection in primates
Miyamoto, K., Osada, T., Setsuie, R., Takeda, M., Tamura, K., Adachi, Y., & Miyashita, Y. (2017) · Science · doi:10.1126/science.aal0162 · Cited by 106 · cited 1×
“Research has identified neural correlates of confidence operating in a single domain (e.g., perceptual only vs. memory only), as well as more abstract (e.g., both perception and memory) (McCurdy et al., 2013; Miyamoto et al., 2017; Morales et al., 2018).”
Maniscalco, B., Odegaard, B., Grimaldi, P., Cho, S. H., Basso, M. A., Lau, H., & Peters, M. A. K. (2021) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1008779 · Cited by 44 · cited 1×
“…vs confidence (Dehaene et al., 2017; Pouget et al., 2016), the computational approaches used to determine the basis of confidence judgements (Maniscalco et al., 2021; Meyniel et al., 2015; Peters and Lau, 2015; Pouget et al., 2016), and on the crucial yet overlooked issue of confounding…”
Cited by
Benvenuti, M., Cangelosi, A., Weinberger, A., Mazzoni, E., Benassi, M., Barbaresi, M., & Orsoni, M. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2023.107903 · Cited by 223
HS Kumar, A. (2023) · Biology, Engineering, Medicine and Science Reports · doi:10.5530/bems.9.1.5 · Cited by 165
Hoven, M., Rouault, M., van Holst, R., & Luigjes, J. (2023) · Psychological Medicine · doi:10.1017/s003329172300209x · Cited by 28
Blaming luck, claiming skill: Self-attribution bias in error assignment
Okamoto, N., Taylor, M., Kubo, T., Ishii, S., De Martino, B., & Cortese, A. (2025) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1013787 · Cited by 2
Self-citation
Metadata
- Authors
- Cortese, A.
- Venue
- Neuroscience Research
- Publisher
- Elsevier BV
- Published
- 2022-05
- Volume
- 178
- Pages
- 10-19
- ISSN
- 0168-0102 (ISSN-L)
- OpenAlex
- W3200759090
- Cited by
- 33
- References
- 51
- FWCI
- 2.93
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.