The influence of delaying judgments of learning on metacognitive accuracy: a meta-analytic review
The DOI matches its Crossref record; 82 of the 83 references it cites were found in a registry or an index.
Abstract
Many studies have examined the accuracy of predictions of future memory performance solicited through judgments of learning (JOLs). Among the most robust findings in this literature is that delaying predictions serves to substantially increase the relative accuracy of JOLs compared with soliciting JOLs immediately after study, a finding termed the delayed JOL effect. The meta-analyses reported in the current study examined the predominant theoretical accounts as well as potential moderators of the delayed JOL effect. The first meta-analysis examined the relative accuracy of delayed compared with immediate JOLs across 4,554 participants (112 effect sizes) through gamma correlations between JOLs and memory accuracy. Those data showed that delaying JOLs leads to robust benefits to relative accuracy (g = 0.93). The second meta-analysis examined memory performance for delayed compared with immediate JOLs across 3,807 participants (98 effect sizes). Those data showed that delayed JOLs result in a modest but reliable benefit for memory performance relative to immediate JOLs (g = 0.08). Findings from these meta-analyses are well accommodated by theories suggesting that delayed JOL accuracy reflects access to more diagnostic information from long-term memory rather than being a by-product of a retrieval opportunity. However, these data also suggest that theories proposing that the delayed JOL effect results from a memorial benefit or the match between the cues available for JOLs and those available at test may also provide viable explanatory mechanisms necessary for a comprehensive account.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“many of these “classic” findings in the metacognition rely on measures such as G (Rhodes and Tauber, 2011) that may be confounded by bias and performance effects (although see Jang et al., 2012).”
From How to measure metacognition (2014) · cites this paper as Rhodes and Tauber, 2011
In the citing paper
“For example, the delayed JOL effect was found to be most robust when studying word pairs (Rhodes and Tauber, 2011) but was not found for studying expository text (e.g., Maki 1998) or learning to solve problems (e.g., Baars et al. 2018).”
From The Relation Between Students’ Effort and Monitoring Judgments During Learning: A Meta-analysis (2020) · cites this paper as Rhodes and Tauber, 2011
Citations
Cites
Statistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695 · cited 2×
“The product of J and Cohen’s d thus provides an unbiased estimate of the effect size that is often referred to as Hedges’s g (Hedges, 1981):”
BackgroundMethodThe Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,642
Metamemory: A Theoretical Framework and New Findings
Nelson, T. O. (1990) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60053-5 · Cited by 2,583 · cited 3×
“Theories of metacognition presume that monitoring serves to influence control processes during learning (e.g., Nelson, 1996; Nelson & Narens, 1990).”
BackgroundModels of Working Memory: Mechanisms of Active Maintenance and Executive Control
Miyake, A., & Shah, P. (1999) · Cited by 2,455
Distributed practice in verbal recall tasks: A review and quantitative synthesis.
Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006) · Psychological Bulletin · doi:10.1037/0033-2909.132.3.354 · Cited by 1,888
Metacognition: Knowing about Knowing
(1994) · The MIT Press eBooks · doi:10.7551/mitpress/4561.001.0001 · Cited by 1,720
Meta-analysis of experiments with matched groups or repeated measures designs
Dunlap, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.170 · Cited by 1,693
Monitoring one’s own knowledge during study: A cue-utilization approach to judgments of learning
Koriat, A. (1997) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.126.4.349 · Cited by 1,363
Cited by
Items 201–232 of 232
The Influence of Divided Attention on Learners' Metacognition
Peng, Y. (2019) · UA Campus Repository (The University of Arizona) · Cited by 2
The Influence of Feedback on Predictions of Future Memory Performance
Sitzman, D. M., & Rhodes, M. G. (2013) · PsycEXTRA Dataset · doi:10.1037/e633262013-768
Self-citationAttualità di Giuseppe Parini: poesia e impegno civile
Millefiorini, F. (2000)
Standardized database of 400 complex abstract fractalsPreprint
Ovalle-Fresa, R., Pietro, S. V. D., Reber, T. P., Balbi, E., & Rothen, N. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/uy9hk · Cited by 1
Higham, P. A., Griffiths, L. R., & Coria, K. A. (2020) · Springer eBooks · doi:10.1007/978-3-319-24612-3_536
Higham, P. A., Griffiths, L. R., & Coria, K. A. (2017) · Springer eBooks · doi:10.1007/978-3-319-28099-8_536-1
Metacognitive Monitoring and Control Processes Underlying Generation-Potentiated LearningPreprint
Alakbarova, D., Peper, P., & Ball, H. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/bjzhu
Jurchiș, R., Preda, A., Costea, A. R., & Skora, L. I. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/9g3xz
The disconnect between metamemory and memory for emotional images
Dodson, S. A., & Westerman, D. L. (2024) · Cognition & Emotion · doi:10.1080/02699931.2024.2373321
“Such “reac-tivity” effects seem to depend on the nature of the stimuli (e.g. relatedness of word pairs and list composition) and are generally observed with delayed JOLs (e.g. Dunlosky & Nelson, 1997; Kelemen & Weaver, 1997; Rhodes & Tauber, 2011).”
Abualrish, M. A., Shater, A., Yousef, J., Naser, K. M., & Abu-Attiyeh, J. H. (2024) · International Conference on Intelligent Data Science Technologies and Applications (IDSTA) · doi:10.1109/idsta62194.2024.10746996
Kortenhoeven, A., & Serra, M. J. (2025) · Metacognition and Learning · doi:10.1007/s11409-025-09417-1 · Cited by 1
“They found that delayed JOLs were not only more accurate than immediate JOLs (as commonly occurs, see Rhodes & Tauber, 2011), but also that delayed JOLs showed greater association with activity in memory regions than did immediate JOLs.”
Le, N. D. (2025) · Roczniki Psychologiczne · doi:10.18290/rpsych2024.0017
“…primarily focused on delayed JOLs, provided some time after initially studying an item, and those delayed judgements have been consistently shown to relatively accurately predict test performance (Nelson & Dunlosky, 1991) and cause positive reactivity under some conditions (Rhodes & Tauber, 2011).”
Judgments of effort and associated cues are influenced by stimulus context
Ashburner, M., & Risko, E. F. (2025) · Consciousness and Cognition · doi:10.1016/j.concog.2025.103873
“Important for the present work, a number of investigations have demonstrated metacognitive judgments to be sensitive to the context in which they are made (e.g., Dunn et al. 2017; Rhodes & Tauber, 2011; Schwarz, 2010; Tatz & Peynircioglu, 2020; Zawadzka, Simkiss, & Hanczakowski, 2018).”
Rivers, M. L., Northern, P. E., & Tauber, S. K. (2025) · Educational Psychology Review · doi:10.1007/s10648-025-10040-4 · Cited by 1
“…(i.e., a cue-only JOL, as in chateau-), they at least partially attempt to covertly retrieve the target word ( castle ), which improves memory for the target on a later cued-recall test (Nelson & Dunlosky, 1991; Son & Metcalfe, 2005; Spellman & Bjork, 1992; for a review, see Rhodes & Tauber, 2011).”
Self-citationPieschl, S., & Dignath, C. (2025) · Springer eBooks · doi:10.1007/978-3-662-71375-4_10
An inquiry about students' naïve knowledge of metacognitive strategies and the delayed JOL effect
Todorov, I. (2011) · PsycEXTRA Dataset · doi:10.1037/e530212013-001
“By deliberately delaying the JOLs, Nelson and Dunlosky (1991) discovered one of the most robust phenomena in metacognitive literature and labeled it the delayed JOL effect (see Sikström & Jönsson, 2005; Rhodes & Tauber, 2011, for reviews).”
BackgroundInfluentialYue, L., Liu, Z., & Gao, X. (2026) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2026.0487
Janson, M. P., Wissel, S., Ingendahl, F., & Undorf, M. (2026) · Learning and Instruction · doi:10.1016/j.learninstruc.2025.102293 · Cited by 2
Calvert, J., Maxwell, N. P., & Smith, K. A. (2026) · Quarterly Journal of Experimental Psychology · doi:10.1177/17470218261425967
“While several metacognitive judgment tasks have been developed, a common approach is to have participants predict their future memory performance (e.g., Judgments of Learning [JOLs]; Arbuckle & Cuddy, 1969; Koriat, 1997; see Rhodes & Tauber, 2011).”
The Cognitive Mechanisms of the Positivity Reactivity Effect on Word Recognition Memory
Li, B., & Yang, C. (2026) · Journal of Intelligence · doi:10.3390/jintelligence14030047
“Traditionally, JOLs have been viewed as passive reflections of an individual’s metamemory state, serving to guide subsequent study decisions about what, when, and how to learn (Rhodes & Tauber, 2011).”
Le fractionnement de la métamémoire dans la maladie d'Alzheimer
Bertrand, J. (2017) · theses.fr (ABES) · doi:10.70675/d7c308d7z62dbz4e27zaa10zae95309a6df8
Jurchiș, R., Costea, A. R., Preda, A., & Skora, L. I. (2026) · PsyArXiv (OSF Preprints)
Judgments of learning modify memory even when made covertly or using a non-numerical scale
Halamish, V., Meer, Y., & Undorf, M. (2026) · Metacognition and Learning · doi:10.1007/s11409-026-09471-3
"Ich weiß, dass ich nichts weiß" — ist Nichtwissen schlimm?
Friederichs, H. (2026) · doi:10.59350/025n3-83c34
"Ich weiß, dass ich nichts weiß" — ist Nichtwissen schlimm?
Friederichs, H. (2026) · doi:10.59350/tp89g-ypk65
Korst, B., van Dijk-Allen, M., & Smilek, D. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6902596
Ingendahl, F., & Undorf, M. (2026) · Metacognition and Learning · doi:10.1007/s11409-026-09479-9
“Research on the delayed-JOL effect (e.g., Rhodes & Tauber, 2011 ; Tekin & Roediger, 2021) and the forward effect of immedi - ate JOLs (Kubik et al., 2022 ; Ritter et al., 2025 ) suggests that assessing JOLs based on the cue rather than the intact cue-target pair induces retrieval processes and thus…”
Jurchiș, R., Preda, A., Costea, A. R., & Skora, L. I. (2026) · Cortex · doi:10.1016/j.cortex.2026.06.025
Sastoque, L. G., Gilet, A.-L., & Bouazzaoui, B. (2026) · Memory · doi:10.1080/09658211.2026.2711903
“Nevertheless, a meta-analysis conducted by Rhodes and Tauber (2011) revealed that delayed JOLs contribute to improved learning only when both the stimulus and the target were presented during the JOLs.”
Identifying and enhancing children’s metacognitive monitoring and control skills
Oudman, S., & van Loon, M. H. (2026) · Theory Into Practice · doi:10.1080/00405841.2026.2714727
Metacognition in Older Adulthood
Tauber, S. K., & Witherby, A. E. (2017) · Springer eBooks · doi:10.1007/978-981-287-082-7_215
Self-citation
Metadata
- Authors
- Rhodes, M. G., & Tauber, S. K.
- Affiliations
- Colorado State University (US)
- Kent State University (US)
- ORCID
- Tauber, Sarah K.
- Venue
- Psychological Bulletin
- Published
- 2011
- Volume
- 137
- Issue
- 1
- Pages
- 131-148
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2061235287
- Cited by
- 230
- References
- 83
- Citations by year
- 2026: 14 · 2025: 13 · 2024: 15 · 2023: 23 · 2022: 11 · 2021: 22 · 2020: 14 · 2019: 14 · 2018: 12 · 2017: 16 · 2016: 12 · 2015: 10 · 2014: 18 · 2013: 9 · 2012: 17
- FWCI
- 8.90
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Functions and Memory
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Keywords
- judgments of learning, metacognitive accuracy, metacognition, memory prediction, memory performance, long-term memory, retrieval cues, metacognitive confidence
- MeSH terms
- Cognition, Cues, Humans, Judgment, Learning, Memory, Short-Term, Models, Psychological, Time Factors, Memory, Long-Term
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.