When asking the question changes the ultimate answer: Metamemory judgments change memory
The DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index.
Abstract
Self-report measurements are ubiquitous in psychology, but they carry the potential of altering processes they are meant to measure. We assessed whether a common metamemory measure, judgments of learning, can change the ongoing process of memorizing and subsequent memory performance. Judgments of learning are a form of metamemory monitoring described as conscious reflection on one's own memory performance or encoding activities for the purpose of exerting strategic control over one's study and retrieval activities (T. O. Nelson & Narens, 1990). Much of the work examining the conscious monitoring of encoding relies heavily on a paradigm in which participants are asked to estimate the probability that they will recall a given item in a judgment of learning. In 5 experiments, we find effects of measuring judgments of learning on how people allocate their study time to difficult versus easy items, and on what they will recall. These results suggest that judgments of learning are partially constructed in response to the measurement question. The tendency of judgments of learning to alter performance places them in the company of other reactive verbal reporting methods, counseling researchers to consider incorporating control groups, creating alternative scales, and exploring other verbal reporting methods. Less directive methods of accessing participants' metacognition and other judgments should be considered as an alternative to response scales.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“All previous reactivity studies presented stimuli in a unimodal form (e.g., word pairs presented visually; Mitchum et al., 2016; Yang, Huang, et al., 2021), which is somewhat unrepresentative of materials in real educational settings.”
From When judging what you know changes what you really know: Soliciting metamemory judgments reactively enhances children’s learning (2022) · cites this paper as Mitchum et al., 2016
In the citing paper
“Some recent studies have documented that, in some situations, the reactivity effect can be negative (that is, making JOLs impairs retention).”
From When judging what you know changes what you really know: Soliciting metamemory judgments reactively enhances children’s learning (2022) · cites this paper as Mitchum, Kelley, and Fox (2016)
In the citing paper
“The positive reactivity effect on children's learning may be accounted for by the positivereactivity theory (Mitchum et al., 2016; Rivers, 2018; Dougherty, Scheck, Nelson, & Narens, 2005), which proposes that positive reactivity is primarily caused by the fact that making item-by-item JOLs encourages learners to adopt more effective encoding strategies and exert greater study effort.”
From When judging what you know changes what you really know: Soliciting metamemory judgments reactively enhances children’s learning (2022) · cites this paper as Mitchum et al., 2016
Citations
Cites
James, W. (1913) · Henry Holt and Company eBooks · doi:10.1037/11059-000 · Cited by 25,226
Grice, H. P. (1975) · Speech Acts · doi:10.1163/9789004368811_003 · Cited by 23,221
Telling more than we can know: Verbal reports on mental processes.
Nisbett, R. E., & Wilson, T. D. (1977) · Psychological Review · doi:10.1037/0033-295x.84.3.231 · Cited by 11,317
An introduction to latent semantic analysis
Landauer, T. K., Foltz, P. W., & Laham, D. (1998) · Discourse Processes · doi:10.1080/01638539809545028 · Cited by 4,880
Ericsson (1993) · doi:10.1037/0033-295x.87.3.215 · Cited by 4,539
Protocol Analysis: Verbal Reports as Data
Ericsson, K. A., & Simon, H. A. (1993) · doi:10.7551/mitpress/5657.001.0001 · Cited by 3,537
Self-reports: How the questions shape the answers.
Schwarz, N. (1999) · American Psychologist · doi:10.1037/0003-066x.54.2.93 · Cited by 2,846
The MRC Psycholinguistic Database
Coltheart, M. (1981) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/14640748108400805 · Cited by 2,437
On the Rate of Gain of Information
Hick, W. E. (1952) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470215208416600 · Cited by 2,279
The construction of preference.
Slovic, P. (1995) · American Psychologist · doi:10.1037/0003-066x.50.5.364 · Cited by 2,009
Cited by
A meta-analysis and systematic review of reactivity to judgements of learning
Double, K. S., Birney, D. P., & Walker, S. A. (2018) · Memory · doi:10.1080/09658211.2017.1404111 · Cited by 120
“…(Dougherty, Scheck, Nelson, & Narens, 2005; Soderstrom et al., 2015; Zechmeister & Shaughnessy, 1980), whereas other have found no reactivity (Benjamin, Bjork, & Schwartz, 1998; Kelemen & Weaver, 1997; Tauber & Rhodes, 2012), and still others have found negative reactivity (Mitchum et al., 2016).”
BackgroundInfluentialEfklides, A., & Schwartz, B. L. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09896-9 · Cited by 129
“Indeed, in cognitive literature, this phenomenon is called negative reactivity, and it has caused some researchers to question the value of prompting metacognitive judgments (Mitchum et al., 2016).”
Reactivity to Measures of Metacognition
Double, K. S., & Birney, D. P. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02755 · Cited by 118
Providing different types of group awareness information to guide collaborative learning
Schnaubert, L., & Bodemer, D. (2019) · International Journal of Computer-Supported Collaborative Learning · doi:10.1007/s11412-018-9293-y · Cited by 106
The influence of making judgments of learning on memory performance: Positive, negative, or both?
Janes, J. L., Rivers, M. L., & Dunlosky, J. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-018-1463-4 · Cited by 88
“…Mitchum et al. (2016) reported that the increased relatedness effect was largely driven by negative reactivity, wherein recall for unrelated pairs was lower for the JOL group; some trends toward positive reactivity for related pairs occurred in Mitchum et al. (2016), but they were not significant.”
BackgroundMethodResultInfluential
Metadata
- Authors
- Mitchum, A. L., Kelley, C. M., & Fox, M. C.
- Affiliations
- Florida State University (US)
- Venue
- Journal of Experimental Psychology: General
- Published
- 2016
- Volume
- 145
- Issue
- 2
- Pages
- 200-219
- ISSN
- 0096-3445 (ISSN-L)
- 1939-2222
- OpenAlex
- W2479323233
- Cited by
- 148
- References
- 85
- Citations by year
- 2026: 16 · 2025: 18 · 2024: 19 · 2023: 21 · 2022: 11 · 2021: 16 · 2020: 15 · 2019: 9 · 2018: 7 · 2017: 12 · 2016: 3
- FWCI
- 6.59
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Experimental Psychology General (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Functions and Memory
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Keywords
- judgments of learning, metamemory, metacognitive monitoring, self-report bias, measurement effects, study time allocation, memory performance, metacognition, response scales, metacognitive regulation
- MeSH terms
- Metacognition, Adult, Humans, Judgment, Learning, Mental Recall, Word Association Tests, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: When asking the question changes the ultimate answer: Metamemory judgments change memory. https://app.cochat.ai/papers/10.1037/a0039923
- The paper
- Mitchum, A. L., Kelley, C. M., & Fox, M. C. (2016). When asking the question changes the ultimate answer: Metamemory judgments change memory. Journal of Experimental Psychology: General. https://doi.org/10.1037/a0039923
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.