The influence of delaying judgments of learning on metacognitive accuracy: a meta-analytic review
The DOI matches its Crossref record; 82 of the 83 references it cites were found in a registry or an index.
Abstract
Many studies have examined the accuracy of predictions of future memory performance solicited through judgments of learning (JOLs). Among the most robust findings in this literature is that delaying predictions serves to substantially increase the relative accuracy of JOLs compared with soliciting JOLs immediately after study, a finding termed the delayed JOL effect. The meta-analyses reported in the current study examined the predominant theoretical accounts as well as potential moderators of the delayed JOL effect. The first meta-analysis examined the relative accuracy of delayed compared with immediate JOLs across 4,554 participants (112 effect sizes) through gamma correlations between JOLs and memory accuracy. Those data showed that delaying JOLs leads to robust benefits to relative accuracy (g = 0.93). The second meta-analysis examined memory performance for delayed compared with immediate JOLs across 3,807 participants (98 effect sizes). Those data showed that delayed JOLs result in a modest but reliable benefit for memory performance relative to immediate JOLs (g = 0.08). Findings from these meta-analyses are well accommodated by theories suggesting that delayed JOL accuracy reflects access to more diagnostic information from long-term memory rather than being a by-product of a retrieval opportunity. However, these data also suggest that theories proposing that the delayed JOL effect results from a memorial benefit or the match between the cues available for JOLs and those available at test may also provide viable explanatory mechanisms necessary for a comprehensive account.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“many of these “classic” findings in the metacognition rely on measures such as G (Rhodes and Tauber, 2011) that may be confounded by bias and performance effects (although see Jang et al., 2012).”
From How to measure metacognition (2014) · cites this paper as Rhodes and Tauber, 2011
In the citing paper
“For example, the delayed JOL effect was found to be most robust when studying word pairs (Rhodes and Tauber, 2011) but was not found for studying expository text (e.g., Maki 1998) or learning to solve problems (e.g., Baars et al. 2018).”
From The Relation Between Students’ Effort and Monitoring Judgments During Learning: A Meta-analysis (2020) · cites this paper as Rhodes and Tauber, 2011
Citations
Cites
Statistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695 · cited 2×
“The product of J and Cohen’s d thus provides an unbiased estimate of the effect size that is often referred to as Hedges’s g (Hedges, 1981):”
BackgroundMethodThe Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,642
Metamemory: A Theoretical Framework and New Findings
Nelson, T. O. (1990) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60053-5 · Cited by 2,583 · cited 3×
“Theories of metacognition presume that monitoring serves to influence control processes during learning (e.g., Nelson, 1996; Nelson & Narens, 1990).”
BackgroundModels of Working Memory: Mechanisms of Active Maintenance and Executive Control
Miyake, A., & Shah, P. (1999) · Cited by 2,455
Distributed practice in verbal recall tasks: A review and quantitative synthesis.
Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006) · Psychological Bulletin · doi:10.1037/0033-2909.132.3.354 · Cited by 1,888
Metacognition: Knowing about Knowing
(1994) · The MIT Press eBooks · doi:10.7551/mitpress/4561.001.0001 · Cited by 1,720
Meta-analysis of experiments with matched groups or repeated measures designs
Dunlap, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.170 · Cited by 1,693
Monitoring one’s own knowledge during study: A cue-utilization approach to judgments of learning
Koriat, A. (1997) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.126.4.349 · Cited by 1,363
Buchbesprechungen / Book Reviews
Förstl, H. (2006) · Zeitschrift für Psychiatrie, Psychologie und Psychotherapie · doi:10.1024/1661-4747.54.1.68a · Cited by 1,300
Access to information in working memory: Exploring the focus of attention.
Oberauer, K. (2002) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037//0278-7393.28.3.411 · Cited by 1,161
A comparison of current measures of the accuracy of feeling-of-knowing predictions
Nelson, T. O. (1984) · Psychological Bulletin · doi:10.1037/0033-2909.95.1.109 · Cited by 976
Consciousness and metacognition.
Nelson, T. O. (1996) · American Psychologist · doi:10.1037//0003-066x.51.2.102 · Cited by 858
Accuracy of metacognitive monitoring affects learning of texts.
Thiede, K. W., Anderson, M. C. M., & Therriault, D. (2003) · Journal of Educational Psychology · doi:10.1037//0022-0663.95.1.66 · Cited by 826
Why Investigate Metacognition?
Nelson, T. O., & Narens, L. (1994) · Metacognition · doi:10.7551/mitpress/4561.003.0003 · Cited by 750
(2009) · Cited by 670
Metacognition and consciousness
Koriat, A. (2007) · The Cambridge Handbook of Consciousness · doi:10.1017/cbo9780511816789.012 · Cited by 554 · cited 2×
“Metacognition has frequently been examined by soliciting predictions of memory performance (for reviews, see Koriat, 2007; Metcalfe, 2000), often through judgments of learning (JOLs) made immediately after the presentation of an item or following a delay (e.g., Arbuckle & Cuddy, 1969; Koriat, 1997;…”
BackgroundAssociative encoding and retrieval: Weak and strong cues.
Thomson, D. M., & Tulving, E. (1970) · Journal of Experimental Psychology · doi:10.1037/h0029997 · Cited by 520
Memory predictions are influenced by perceptual information: Evidence for metacognitive illusions.
Rhodes, M. G., & Castel, A. D. (2008) · Journal of Experimental Psychology: General · doi:10.1037/a0013684 · Cited by 459
Self-citationKoriat, A., Ma'ayan, H., & Nussinson, R. (2006) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.135.1.36 · Cited by 422
Evidence that judgments of learning are causally related to study choice
METCALFE, J., & FINN, B. (2008) · Psychonomic Bulletin & Review · doi:10.3758/pbr.15.1.174 · Cited by 409
Hedges, L. V. (1994) · Cited by 397
A Region of Proximal Learning model of study time allocation
Metcalfe, J., & Kornell, N. (2005) · Journal of Memory and Language · doi:10.1016/j.jml.2004.12.001 · Cited by 327
Memory predictions are based on ease of processing
Begg, I., Duft, S., Lalonde, P., Melnick, R., & Sanvito, J. (1989) · Journal of Memory and Language · doi:10.1016/0749-596x(89)90016-8 · Cited by 322
The Cambridge Handbook of Consciousness
(2012) · The Cambridge Handbook of Consciousness · doi:10.1017/cbo9780511816789.001 · Cited by 321
Importance of the kind of cue for judgments of learning (JOL) and the delayed-JOL effect
Dunlosky, J., & Nelson, T. O. (1992) · Memory & Cognition · doi:10.3758/bf03210921 · Cited by 308
Handbook of Metamemory and Memory
Dunlosky, J. T., & Bjork, R. A. (2013) · Cited by 290
Individual differences in metacognition: Evidence against a general metacognitive ability
Kelemen, W. L., Frost, P. J., & Weaver, C. A. (2000) · Memory & Cognition · doi:10.3758/bf03211579 · Cited by 252
Age-related differences in absolute but not relative metamemory accuracy.
Connor, L. T., Dunlosky, J., & Hertzog, C. (1997) · Psychology and Aging · doi:10.1037//0882-7974.12.1.50 · Cited by 244
The effects of encoding fluency and retrieval fluency on judgments of learning
Koriat, A., & Ma’ayan, H. (2005) · Journal of Memory and Language · doi:10.1016/j.jml.2005.01.001 · Cited by 235
Dunlosky, J., & Nelson, T. O. (1994) · Journal of Memory and Language · doi:10.1006/jmla.1994.1026 · Cited by 225
Metacognitive illusions for auditory information: Effects on monitoring and control
Rhodes, M. G., & Castel, A. D. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.3.550 · Cited by 179
Self-citationMasson, M. E. J., & Rotello, C. M. (2009) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/a0014876 · Cited by 173
Discrimination of item strength at time of presentation.
Arbuckle, T. Y., & Cuddy, L. L. (1969) · Journal of Experimental Psychology · doi:10.1037/h0027455 · Cited by 166
When Predictions Create Reality: Judgments of Learning May Alter What They Are Intended to Assess
Spellman, B. A., & Bjork, R. A. (1992) · Psychological Science · doi:10.1111/j.1467-9280.1992.tb00680.x · Cited by 166
The role of memory for past test in the underconfidence with practice effect.
Finn, B., & Metcalfe, J. (2007) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.33.1.238 · Cited by 158
Control of the Contents of Working Memory--A Comparison of Two Paradigms and Two Age Groups.
Oberauer, K. (2005) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.31.4.714 · Cited by 145
Delaying judgments of learning affects memory, not metamemory
Kimball, D. R., & Metcalfe, J. (2003) · Memory & Cognition · doi:10.3758/bf03196445 · Cited by 128
Koriat, A., & Bjork, R. A. (2006) · Memory & Cognition · doi:10.3758/bf03193244 · Cited by 123
A Revised Methodology for Research on Metamemory: Pre-judgment Recall And Monitoring (PRAM).
Nelson, T. O., Narens, L., & Dunlosky, J. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.1.53 · Cited by 122
Judgments of learning: Evidence for a two-stage process
Son, L. K., & Metcalfe, J. (2005) · Memory & Cognition · doi:10.3758/bf03193217 · Cited by 122
Metacognition: Process, Function and Use
Chambres, P., Izaute, M., & Marescaux, P.-J. (2002) · Springer eBooks · doi:10.1007/978-1-4615-1099-4 · Cited by 120
Developmental trends in children's memory monitoring
Schneider, W., Visé, M., Lockl, K., & Nelson, T. O. (2000) · Cognitive Development · doi:10.1016/s0885-2014(00)00024-1 · Cited by 120
Why do delayed summaries improve metacomprehension accuracy?
Anderson, M. C. M., & Thiede, K. W. (2007) · Acta Psychologica · doi:10.1016/j.actpsy.2007.10.006 · Cited by 120
Understanding the Delayed-Keyword Effect on Metacomprehension Accuracy.
Thiede, K. W., Dunlosky, J., Griffin, T. D., & Wiley, J. (2005) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.31.6.1267 · Cited by 116
Metcalfe, J. (2000) · The Oxford Handbook of Memory · doi:10.1093/oso/9780195122657.003.0013 · Cited by 108
Enhanced memory at delays: Why do judgments of learning improve over time?
Kelemen, W. L., & Weaver, C. A., III (1997) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037//0278-7393.23.6.1394 · Cited by 100
Dunlosky, J. T., & Nelson, T. O. (1997) · Journal of Memory and Language · doi:10.1006/jmla.1996.2476 · Cited by 96
Framing effects on metacognitive monitoring and control
FINN, B. (2008) · Memory & Cognition · doi:10.3758/mc.36.4.813 · Cited by 90
Second-Order Judgments About Judgments of Learning
Dunlosky, J., Serra, M. J., Matvey, G., & Rawson, K. A. (2005) · The Journal of General Psychology · doi:10.3200/genp.132.4.335-346 · Cited by 88
Familiarity and retrieval processes in delayed judgments of learning.
Metcalfe, J., & Finn, B. (2008) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/a0012580 · Cited by 83
Measuring ordinal association in situations that contain tied scores.
Gonzalez, R., & Nelson, T. O. (1996) · Psychological Bulletin · doi:10.1037//0033-2909.119.1.159 · Cited by 79
Children’s metamemorial judgments in an event recall task
Roebers, C. M., von der Linden, N., Schneider, W., & Howie, P. M. (2007) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2006.12.006 · Cited by 79
Thiede, K. W., & Dunlosky, J. (1994) · Journal of Educational Psychology · doi:10.1037//0022-0663.86.2.290 · Cited by 77
Carroll, M., Nelson, T. O., & Kirwan, A. V. (1997) · Acta Psychologica · doi:10.1016/s0001-6918(96)00040-6 · Cited by 71
Thomas, A. K., & McDaniel, M. A. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194054 · Cited by 53
Exploring a mnemonic debiasing account of the underconfidence-with-practice effect.
Koriat, A., Ma'ayan, H., Sheffer, L., & Bjork, R. A. (2006) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.32.3.595 · Cited by 51
Multiple study trials and judgments of learning
Meeter, M., & Nelson, T. O. (2003) · Acta Psychologica · doi:10.1016/s0001-6918(03)00023-4 · Cited by 51
Metamemory cues and monitoring accuracy: Judging what you know and what you will know.
Kelemen, W. L. (2000) · Journal of Educational Psychology · doi:10.1037//0022-0663.92.4.800 · Cited by 51
Anchoring effects in the absolute accuracy of immediate versus delayed judgments of learning
Scheck, P., Meeter, M., & Nelson, T. O. (2004) · Journal of Memory and Language · doi:10.1016/j.jml.2004.03.004 · Cited by 50
Does Retrieval Fluency Contribute to the Underconfidence-With-Practice Effect?
Serra, M. J., & Dunlosky, J. (2005) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.31.6.1258 · Cited by 48
Judgments of Learning at Delays: Shifts in Response Patterns or Increased Metamemory Accuracy?
Weaver, C. A., & Kelemen, W. L. (1997) · Psychological Science · doi:10.1111/j.1467-9280.1997.tb00445.x · Cited by 44
Do Older Adults Show Less Confidence in Their Monitoring of Learning?
Serra, M. J., Dunlosky, J. T., & Hertzog, C. K. (2008) · Experimental Aging Research · doi:10.1080/03610730802271898 · Cited by 44
Dunlosky, J., Rawson, K. A., & McDonald, S. L. (2002) · Applied Metacognition · doi:10.1017/cbo9780511489976.005 · Cited by 42 · cited 2×
“Recognition (Dunlosky & Nelson, 1997; Thiede & Dunlosky, 1994) and free recall (Dunlosky et al., 2002; Matvey et al., 2006) tests have also been used.”
BackgroundMethodJang, Y., & Nelson, T. O. (2005) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.134.3.308 · Cited by 36
A Model for Stochastic Drift in Memory Strength to Account for Judgments of Learning.
Sikström, S., & Jönsson, F. U. (2005) · Psychological Review · doi:10.1037/0033-295x.112.4.932 · Cited by 36
Overschelde, J. P. V., & Nelson, T. O. (2006) · Memory & Cognition · doi:10.3758/bf03195916 · Cited by 36
Processing similarity does not improve metamemory: Evidence against transfer-appropriate monitoring.
Weaver, C. A., & Kelemen, W. L. (2003) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.29.6.1058 · Cited by 32
Does the amount of material to be remembered influence judgements of learning (JOLs)?
Tauber, S. K., & Rhodes, M. G. (2010) · Memory · doi:10.1080/09658211003662755 · Cited by 31
Self-citationThe effects of categorical relatedness on judgements of learning (JOLs)
Matvey, G., Dunlosky, J. T., & Schwartz, B. L. (2006) · Memory · doi:10.1080/09658210500216844 · Cited by 31
Age Differences in the Underconfidence-With-Practice Effect
Rast, P., & Zimprich, D. (2009) · Experimental Aging Research · doi:10.1080/03610730903175782 · Cited by 30
Predictions of recall and study strategy decisions after diffuse brain injury
Kennedy, M. R. T., Carney, E. J., & Peters, S. M. (2003) · Brain Injury · doi:10.1080/0269905031000110436 · Cited by 26
Drugs, memory, and metamemory: A dose-effect study with lorazepam and scopolamine.
Mintzer, M. Z., & Griffiths, R. R. (2005) · Experimental and Clinical Psychopharmacology · doi:10.1037/1064-1297.13.4.336 · Cited by 21
Dunlosky, J. T., Domoto, P. K., Wang, M., Ishikawa, T., Roberson, I., Nelson, T. O., & Ramsay, D. S. (1998) · Experimental and Clinical Psychopharmacology · doi:10.1037//1064-1297.6.1.77 · Cited by 21
Does ADHD in Adults Affect the Relative Accuracy of Metamemory Judgments?
Knouse, L. E., Paradise, M. J., & Dunlosky, J. (2006) · Journal of Attention Disorders · doi:10.1177/1087054706288116 · Cited by 20
Self-Monitoring Recall During Two Tasks After Traumatic Brain Injury: A Preliminary Study
Kennedy, M. R. T. (2004) · American Journal of Speech-Language Pathology · doi:10.1044/1058-0360(2004/015 · Cited by 18
Spellman, B. A., Bloomfield, A., & Bjork, R. A. (2008) · Cited by 17
Immediate judgments of learning are insensitive to implicit interference effects at retrieval
Eakin, D. K., & Hertzog, C. K. (2011) · Memory & Cognition · doi:10.3758/s13421-011-0138-4 · Cited by 10
Memory Monitoring and the Delayed JOL Effect
Narens, L., Nelson, T. O., & Scheck, P. (2015) · Routledge eBooks · doi:10.4324/9780203805503.ch8 · Cited by 7
THE EFFECT OF PRACTICE WITH TEST ON THE RELATIVE ACCURACY OF JUDGMENTS OF LEARNING
Jang, Y. (2006) · University Libraries (University of Maryland) · Cited by 2
HOW DO JUDGMENTS OF INTENTION (JOIS) DIFFER FROM JUDGMENTS OF LEARNING (JOLS)
Scheck, P. (2006) · University Libraries (University of Maryland) · Cited by 1
Cited by
Self-Regulated Learning: Beliefs, Techniques, and Illusions
Bjork, R. A., Dunlosky, J., & Kornell, N. (2013) · Annual Review of Psychology · doi:10.1146/annurev-psych-113011-143823 · Cited by 1,468
Fleming, S. M., & Lau, H. C. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00443 · Cited by 1,408
Dunlosky, J., & Rawson, K. A. (2012) · Learning and Instruction · doi:10.1016/j.learninstruc.2011.08.003 · Cited by 713
Executive function and metacognition: Towards a unifying framework of cognitive self-regulation
Roebers, C. M. (2017) · Developmental Review · doi:10.1016/j.dr.2017.04.001 · Cited by 592
“Empirical evidence in adults has also suggested only low correspondence of different monitoring indicators (Boduroglu, Tekcan, & Kapucu, 2014; Rhodes & Tauber, 2011).”
BackgroundCollege Students’ Time Management: a Self-Regulated Learning Perspective
Wolters, C. A., & Brady, A. C. (2021) · Educational Psychology Review · doi:10.1007/s10648-020-09519-z · Cited by 317
Metadata
- Authors
- Rhodes, M. G., & Tauber, S. K.
- Affiliations
- Colorado State University (US)
- Kent State University (US)
- ORCID
- Tauber, Sarah K.
- Venue
- Psychological Bulletin
- Published
- 2011
- Volume
- 137
- Issue
- 1
- Pages
- 131-148
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2061235287
- Cited by
- 230
- References
- 83
- Citations by year
- 2026: 14 · 2025: 13 · 2024: 15 · 2023: 23 · 2022: 11 · 2021: 22 · 2020: 14 · 2019: 14 · 2018: 12 · 2017: 16 · 2016: 12 · 2015: 10 · 2014: 18 · 2013: 9 · 2012: 17
- FWCI
- 8.90
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Functions and Memory
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Keywords
- judgments of learning, metacognitive accuracy, metacognition, memory prediction, memory performance, long-term memory, retrieval cues, metacognitive confidence
- MeSH terms
- Cognition, Cues, Humans, Judgment, Learning, Memory, Short-Term, Models, Psychological, Time Factors, Memory, Long-Term
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.