A meta-analysis and systematic review of reactivity to judgements of learning
Open accessThe DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Abstract
Judgements of learning (JoL) are often used in memory research as a means for assessing an individual's metacognitive beliefs about their learning. JoL have been shown to reliably predict performance as well as learning behaviours and decisions . Participants may, however, modify their behaviour in response to performing JoL. There has, however, been little consensus as to the reliability and direction of the effect. We report on a meta-analyses that assesses the evidence that memory performance is reactive to JoL. The results indicate that overall providing JoL does not have a significant effect on memory performance (g = 0.054, 95% CI -0.027 to 0.135). However, sub-groups analysis showed that this effect depends on the nature of the stimuli to be recalled, with moderate positive reactivity observed for related word pairs (g = 0.323, 95% CI 0.083 to 0.563) and word lists (g = 0.384, 95% CI 0.146 to 0.622) but no reactivity when pairs were unrelated or a mixture of related and unrelated pairs. These results indicate that researchers should be aware that eliciting JoL may well influence participants' underlying encoding processes, especially when using related word pairs or word lists.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recent studies established that making concurrent judgments of learning (JOLs) can significantly alter (typically enhance) memory itself - a reactivity effect.”
From When judging what you know changes what you really know: Soliciting metamemory judgments reactively enhances children’s learning (2022) · cites this paper as Double, Birney, & Walker, 2018
In the citing paper
“For instance, research has observed positive reactivity effects on learning of word lists (Double et al., 2018), related word pairs (Soderstrom et al., 2015), and general knowledge facts (Yang, Huang, et al., 2021), whereas no reactivity is found on learning of texts (Ariel, Karpicke, Witherby, & Tauber, 2021), unrelated word pairs (Soderstrom et al., 2015; Dougherty, Robey, & Buttaccio, 2018), or”
From When judging what you know changes what you really know: Soliciting metamemory judgments reactively enhances children’s learning (2022) · cites this paper as Double et al., 2018
Citations
Cites
Conducting Meta-Analyses in R with the metafor Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
Protocol Analysis: Verbal Reports as Data
Venkatesan, M., Ericsson, K. A., & Simon, H. A. (1986) · Journal of Marketing Research · doi:10.2307/3151491 · Cited by 4,347
Protocol Analysis: Verbal Reports as Data
Ericsson, K. A., & Simon, H. A. (1993) · doi:10.7551/mitpress/5657.001.0001 · Cited by 3,537
Morris, S. B., & DeShon, R. P. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.1.105 · Cited by 2,502
Flawed Self-Assessment: Implications for Health, Education, and the Workplace
Dunning, D., Heath, C., & Suls, J. M. (2004) · Psychological Science in the Public Interest · doi:10.1111/j.1529-1006.2004.00018.x · Cited by 1,865
The Hawthorne Effect: a randomised, controlled trial
McCarney, R., Warner, J., Iliffe, S., van Haselen, R., Griffin, M., & Fisher, P. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-30 · Cited by 1,715
Schwarzer, G., Carpenter, J. R., & Rücker, G. (2015) · Use R! · doi:10.1007/978-3-319-21416-0 · Cited by 1,230
Why People Fail to Recognize Their Own Incompetence
Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003) · Current Directions in Psychological Science · doi:10.1111/1467-8721.01235 · Cited by 1,224
Research works printed but not linked: 6
- 10.1037/0096-1523.22.3.615 · doi:10.1037/0096-1523.22.3.615
- 10.1037/0278-7393.23.6.1394 · doi:10.1037/0278-7393.23.6.1394
- Koriat, A. & Helstrup, T. (2007). Metacognitive aspects of memory everyday memory (pp. 251–274). New York, NY: Psychology Press.
- Narens, L., Nelson, T. O. & Scheck, P. (2008). Memory monitoring and the delayed JOL effect handbook of metamemory and memory (pp. 137–153). New York, NY: Psychology Press.
- Rhodes M. G., 2016, Oxford handbook of metacognition, 65
- Yan Y., 2015, Chinese Journal of Clinical Psychology, 23, 867
Cited by
Chang, D. H., Lin, M. P.-C., Hajian, S., & Wang, Q. Q. (2023) · Sustainability · doi:10.3390/su151712921 · Cited by 370
“Additionally, JOL can influence an individual’s confidence, which in turn can affect their overall approach to learning [11].”
BackgroundA role for metamemory in cognitive offloading
Hu, X., Luo, L., & Fleming, S. M. (2019) · Cognition · doi:10.1016/j.cognition.2019.104012 · Cited by 121
Muncer, G., Higham, P. A., Gosling, C. J., Cortese, S., Wood‐Downie, H., & Hadwin, J. A. (2021) · Educational Psychology Review · doi:10.1007/s10648-021-09620-x · Cited by 119
Reactivity to Measures of Metacognition
Double, K. S., & Birney, D. P. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02755 · Cited by 118
Self-citationThe influence of making judgments of learning on memory performance: Positive, negative, or both?
Janes, J. L., Rivers, M. L., & Dunlosky, J. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-018-1463-4 · Cited by 88
Metadata
- Authors
- Double, K. S., Birney, D. P., & Walker, S. A.
- Affiliations
- The University of Sydney (AU)
- Corresponding author
- Double, Kit S.
- Venue
- Memory
- Publisher
- Informa UK Limited
- Published
- 2017-11-21
- Volume
- 26
- Issue
- 6
- Pages
- 741-750
- ISSN
- 0965-8211 (ISSN-L)
- 1464-0686
- OpenAlex
- W2768752593
- Cited by
- 120
- References
- 54
- Citations by year
- 2026: 16 · 2025: 18 · 2024: 19 · 2023: 23 · 2022: 9 · 2021: 16 · 2020: 9 · 2019: 6 · 2018: 3
- FWCI
- 0.55
- Type
- journal article
- Language
- en
- Open access
- Licence
- cc-by
- Locations
- Memory (publishedVersion)
- PubMed (publishedVersion)
- Durham Research Online (Durham University) (submittedVersion, cc-by)
- OSF Preprints (OSF Preprints) (submittedVersion)
- Funders
- Australian Research Council (DP140101147)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Functions and Memory
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Keywords
- judgments of learning, reactivity, metacognition, memory performance, encoding processes, word lists, metacognitive beliefs
- MeSH terms
- Metacognition, Humans, Judgment, Learning, Memory, Mental Recall
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A meta-analysis and systematic review of reactivity to judgements of learning. https://app.cochat.ai/papers/10.1080/09658211.2017.1404111
- The paper
- Double, K. S., Birney, D. P., & Walker, S. A. (2018). A meta-analysis and systematic review of reactivity to judgements of learning. Memory. https://doi.org/10.1080/09658211.2017.1404111
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.