Testing the retrieval effort hypothesis: Does greater difficulty correctly recalling information lead to higher levels of memory?

Pyc, Mary A.; Rawson, Katherine A.

doi:10.1016/j.jml.2009.01.004

Cited by 551Memory Processes and InfluencesCognitive Functions and MemoryMemory and Neural Mechanisms

Testing the retrieval effort hypothesis: Does greater difficulty correctly recalling information lead to higher levels of memory?

Pyc, Mary A., Rawson, Katherine A.·2009·Journal of Memory and Language

DOI 10.1016/j.jml.2009.01.004

Open this paper in CoChat — free account

Integrity signals

Registry

Crossref · 10.1016/j.jml.2009.01.004

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Tests that are harder and require more retrieval effort produce greater benefits for later memory.
“Moreover, it has been found that the harder the test, and the greater the effort required for retrieval, the greater the benefit to subsequent memory (e.g., Carpenter & DeLosh, 2006;Pyc & Rawson, 2009).”
From The benefit of generating errors during learning (2014) · cites this paper as Pyc & Rawson, 2009
Several studies have directly tested the Retrieval Effort Hypothesis, each finding evidence in support.
“There are several studies that have directly tested the REH, each finding evidence in support of this notion (Carpenter & DeLosh, 2006; Karpicke & Roediger, 2007b; Pyc & Rawson, 2009).”
From Does difficulty moderate learning? A comparative analysis of the desirable difficulties framework and cognitive load theory (2025) · cites this paper as Pyc & Rawson, 2009
Studies related to REH typically involve simple, low-element-interactivity material, such as paired associates and word lists.
“Studies related to REH typically involve simple, low-element-interactivity material, such as paired associates (Pyc & Rawson, 2009; Vaughn et al., 2013) and word lists (Karpicke & Roediger, 2007b; Lehman et al., 2014).”
From Does difficulty moderate learning? A comparative analysis of the desirable difficulties framework and cognitive load theory (2025) · cites this paper as Pyc & Rawson, 2009
The level of retrieval effort cannot be defined independent of its eventual effect on retrievability.
“A major challenge, though, is that the level of retrieval effort cannot be defined independent of its eventual effect on retrievability (cf.Pyc & Rawson 2009); likewise, it is not clear ahead of time which difficulties are desirable and which are undesirable and whether the answer to that question differs from person to person.”
From Practicing Retrieval Facilitates Learning (2021) · cites this paper as Pyc & Rawson 2009

The power of testing memory: Basic research and implications for educational practice
Roediger, H. L., & Karpicke, J. D. (2006) · Perspectives on Psychological Science · doi:10.1111/j.1745-6916.2006.00012.x · Cited by 2,037 · see citing works
When Does Feedback Facilitate Learning of Words?
Pashler, H., Cepeda, N. J., Wixted, J. T., & Rohrer, D. (2005) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.31.1.3 · Cited by 445 · see citing works
Expanding retrieval practice promotes short-term retention, but equally spaced retrieval enhances long-term retention
Karpicke, J. D., & Roediger, H. L. (2007) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.33.4.704 · Cited by 421 · cited 6× · see citing works
“Experiment 2 also extended beyond the results of Karpicke and Roediger (2007a) by evaluating latencies for longer ISIs and higher-criterion levels.”
BackgroundResultInfluential
Norms of paired-associate recall during multitrial learning of Swahili-English translation equivalents
Nelson, T. O., & Dunlosky, J. (1994) · Memory · doi:10.1080/09658219408258951 · Cited by 140 · see citing works

Flagged first, then by citations

Sorted by citations

Improving Students’ Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013) · Psychological Science in the Public Interest · doi:10.1177/1529100612453266 · Cited by 3,138 · see citing works
Self-citation
The critical role of retrieval practice in long-term retention
Roediger, H. L., & Butler, A. C. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.09.003 · Cited by 1,796 · see citing works
Self-Regulated Learning: Beliefs, Techniques, and Illusions
Bjork, R. A., Dunlosky, J., & Kornell, N. (2013) · Annual Review of Psychology · doi:10.1146/annurev-psych-113011-143823 · Cited by 1,468 · see citing works
The effect of testing versus restudy on retention: A meta-analytic review of the testing effect.
Rowland, C. A. (2014) · Psychological Bulletin · doi:10.1037/a0037559 · Cited by 1,066 · see citing works
Retrieval Practice Produces More Learning than Elaborative Studying with Concept Mapping
Karpicke, J. D., & Blunt, J. R. (2011) · Science · doi:10.1126/science.1199327 · Cited by 1,019 · see citing works
Rethinking the Use of Tests: A Meta-Analysis of Practice Testing
Adesope, O. O., Trevisan, D. A., & Sundararajan, N. (2017) · Review of Educational Research · doi:10.3102/0034654316689306 · Cited by 544 · see citing works
Repeated testing produces superior transfer of learning relative to repeated studying.
Butler, A. C. (2010) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/a0019902 · Cited by 492 · see citing works
Spacing and Testing Effects
(2010) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(10)53003-2 · Cited by 255 · see citing works
Retrieval-Based Learning: A Decade of Progress
Karpicke, J. D. (2017) · Learning and Memory: A Comprehensive Reference · doi:10.1016/b978-0-12-809324-5.21055-9 · Cited by 223 · see citing works
Tests enhance the transfer of learning
Rohrer, D., Taylor, K., & Sholar, B. (2010) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/a0017678 · Cited by 217 · see citing works
When and why do retrieval attempts enhance subsequent encoding?
Grimaldi, P. J., & Karpicke, J. D. (2012) · Memory & Cognition · doi:10.3758/s13421-011-0174-0 · Cited by 200 · see citing works
Quizzing in Middle‐School Science: Successful Transfer Performance on Classroom Exams
McDaniel, M. A., Thomas, R. C., Agarwal, P. K., McDermott, K. B., & Roediger, H. L. (2013) · Applied Cognitive Psychology · doi:10.1002/acp.2914 · Cited by 171 · see citing works
The benefit of generating errors during learning
Potts, R., & Shanks, D. R. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033194 · Cited by 166 · see citing works
Computer-assisted second language vocabulary learning in a paired-associate paradigm: a critical investigation of flashcard software
Nakata, T. (2011) · Computer Assisted Language Learning · doi:10.1080/09588221.2010.520675 · Cited by 167 · see citing works
Not New, but Nearly Forgotten: the Testing Effect Decreases or even Disappears as the Complexity of Learning Materials Increases
van Gog, T., & Sweller, J. (2015) · Educational Psychology Review · doi:10.1007/s10648-015-9310-x · Cited by 164 · see citing works
The Power of Successive Relearning: Improving Performance on Course Exams and Long-Term Retention
Rawson, K. A., Dunlosky, J., & Sciartelli, S. M. (2013) · Educational Psychology Review · doi:10.1007/s10648-013-9240-4 · Cited by 148 · see citing works
Self-citation
When generating a prediction boosts learning: The element of surprise
Brod, G., Hasselhorn, M., & Bunge, S. A. (2018) · Learning and Instruction · doi:10.1016/j.learninstruc.2018.01.013 · Cited by 131 · see citing works
Is expanding retrieval a superior method for learning text materials?
Karpicke, J. D., & Roediger, H. L. (2010) · Memory & Cognition · doi:10.3758/mc.38.1.116 · Cited by 137 · see citing works
Practicing Retrieval Facilitates Learning
McDermott, K. B. (2021) · Annual review of psychology · doi:10.1146/annurev-psych-010419-051019 · Cited by 113 · see citing works
Towards a Theory of Learning for Naming Rehabilitation: Retrieval Practice and Spacing Effects
Middleton, E. L., Schwartz, M. F., Rawson, K. A., Traut, H., & Verkuilen, J. (2016) · Journal of Speech, Language, and Hearing Research · doi:10.1044/2016_jslhr-l-15-0303 · Cited by 125 · see citing works
Self-citation
The Effects of Spaced Practice on Second Language Learning: A Meta‐Analysis
Kim, S. K., & Webb, S. (2022) · Language Learning · doi:10.1111/lang.12479 · Cited by 119 · see citing works
Individual differences in the effects of retrieval from long-term memory
Brewer, G. A., & Unsworth, N. (2012) · Journal of Memory and Language · doi:10.1016/j.jml.2011.12.009 · Cited by 114 · see citing works
The effects of interleaving versus blocking on foreign language pronunciation learning
Carpenter, S. K., & Mueller, F. E. (2013) · Memory & Cognition · doi:10.3758/s13421-012-0291-4 · Cited by 103 · see citing works
The Testing Effect in the Psychology Classroom: A Meta-Analytic Perspective
Schwieren, J., Barenberg, J., & Dutke, S. (2017) · Psychology Learning & Teaching · doi:10.1177/1475725717695149 · Cited by 100 · see citing works
How the amount and spacing of retrieval practice affect the short- and long-term retention of mathematics knowledge
Lyle, K. B., Bego, C. R., Hopkins, R. F., Hieb, J. L., & Ralston, P. A. S. (2020) · Educational Psychology Review · doi:10.1007/s10648-019-09489-x · Cited by 103 · see citing works
Retrieval practice over the long term: Should spacing be expanding or equal-interval?
Kang, S. H. K., Lindsey, R. V., Mozer, M. C., & Pashler, H. (2014) · Psychonomic Bulletin & Review · doi:10.3758/s13423-014-0636-z · Cited by 90 · see citing works
Neural correlates of testing effects in vocabulary learning
van den Broek, G. S. E., Takashima, A., Segers, E., Fernández, G., & Verhoeven, L. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.03.071 · Cited by 85 · see citing works
Retrieval-Based Learning: Positive Effects of Retrieval Practice in Elementary School Children
Karpicke, J. D., Blunt, J. R., & Smith, M. A. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00350 · Cited by 82 · see citing works
Lesser neural pattern similarity across repeated tests is associated with better long-term memory retention.
Karlsson Wirebring, L., Wiklund-Hornqvist, C., Eriksson, J., Andersson, M., Jonsson, B., & Nyberg, L. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.3550-14.2015 · Cited by 65 · see citing works
Smoking Cessation Smartphone App Use Over Time: Predicting 12-Month Cessation Outcomes in a 2-Arm Randomized Trial
Bricker, J. B., Mull, K. E., Santiago-Torres, M., Miao, Z., Perski, O., & Di, C. (2022) · Journal of Medical Internet Research · doi:10.2196/39208 · Cited by 44 · see citing works
Distributed Practice and Retrieval Practice in Primary School Vocabulary Learning: A Multi‐classroom Study
Goossens, N. A. M. C., Camp, G., Verkoeijen, P. P. J. L., Tabbers, H. K., Bouwmeester, S., & Zwaan, R. A. (2016) · Applied Cognitive Psychology · doi:10.1002/acp.3245 · Cited by 40 · see citing works
Testing enhances learning: A review of the literature
Binks, S. (2018) · Journal of Professional Nursing · doi:10.1016/j.profnurs.2017.08.008 · Cited by 38 · see citing works
Mixing topics while studying does not enhance learning.
Hausman, H., & Kornell, N. (2014) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2014.03.003 · Cited by 33 · see citing works
Spaced Retrieval Practice Imposes Desirable Difficulty in Calculus Learning
Lyle, K. B., Bego, C. R., Ralston, P. A. S., & Immekus, J. C. (2022) · Educational Psychology Review · doi:10.1007/s10648-022-09677-2 · Cited by 31 · see citing works
“This is distinct from the idea that spacing makes retrieval more effortful (Bjork, 1994; Maddox & Balota, 2015; Maddox et al., 2018; Pyc & Rawson, 2009).”
Background
Learning the hard way: need for cognition influences attitudes toward and self-reported use of desirable difficulties.
Weissgerber, S. C., Reinhard, M.-A., & Schindler, S. (2018) · Educational Psychology · doi:10.1080/01443410.2017.1387644 · Cited by 30 · see citing works
Retrieval practice and spacing effects in multi-session treatment of naming impairment in aphasia
Middleton, E. L., Rawson, K. A., & Verkuilen, J. (2019) · Cortex · doi:10.1016/j.cortex.2019.07.003 · Cited by 29 · see citing works
Self-citation
Effects of retrieval formats on second language vocabulary learning
Nakata, T. (2016) · International Review of Applied Linguistics in Language Teaching · doi:10.1515/iral-2015-0022 · Cited by 28 · see citing works
Effects of successive relearning on recall: Does relearning override the effects of initial learning criterion?
Vaughn, K. E., Dunlosky, J., & Rawson, K. A. (2016) · Memory & Cognition · doi:10.3758/s13421-016-0606-y · Cited by 28 · see citing works
Self-citation
The testing effect under divided attention.
Buchin, Z. L., & Mulligan, N. W. (2017) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/xlm0000427 · Cited by 27 · see citing works
The beneficial effect of testing: an event-related potential study
Bai, C.-H., Bridger, E. K., Zimmer, H. D., & Mecklinger, A. (2015) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2015.00248 · Cited by 25 · see citing works
The benefits of successive relearning on multiple learning outcomes.
Higham, P. A., Zengel, B., Bartlett, L. K., & Hadwin, J. A. (2022) · Journal of Educational Psychology · doi:10.1037/edu0000693 · Cited by 23 · see citing works
Learning by explaining after pauses in video lectures: Are provided visuals a scaffold or a crutch?
Fiorella, L. (2022) · Applied Cognitive Psychology · doi:10.1002/acp.3994 · Cited by 19 · see citing works
Does difficulty moderate learning? A comparative analysis of the desirable difficulties framework and cognitive load theory
Pyke, W., Lunau, J., & Javadi, A.-H. (2025) · Quarterly Journal of Experimental Psychology · doi:10.1177/17470218241308143 · Cited by 15 · see citing works
“There are several studies that have directly tested the REH, each finding evidence in support of this notion (Carpenter & DeLosh, 2006; Karpicke & Roediger, 2007b; Pyc & Rawson, 2009).”
Influential
Is Covert Retrieval an Effective Learning Strategy? Is It as Effective as Overt Retrieval? Answers from a Meta-Analytic Review
Yu, Y., Zhao, W., Li, A., Shanks, D. R., Hu, X., Luo, L., & Yang, C. (2025) · Educational Psychology Review · doi:10.1007/s10648-025-10024-4 · Cited by 2 · see citing works
A Teacher’s Guide to Creating Materials That Support Efficient Learning: From Scientific Background to AI-based content creation
McCabe, N., van der Velde, M., Braam, M., & Wilschut, T. (2024) · doi:10.31234/osf.io/qz64y · Cited by 1 · see citing works
Testing the testing effect with advanced materials while accounting for individual differences
Redifer, J. L., Myers, S. J., Bae, C. L., Naas, A., & Scott, A. (2026) · Instructional Science · doi:10.1007/s11251-025-09758-z · Cited by 2 · see citing works
Toward mastering foreign-language translations: transfer between productive and receptive learning
Bernardi, E., Vaughn, K. E., Dunlosky, J., & Rawson, K. A. (2024) · Memory · doi:10.1080/09658211.2024.2397043
Self-citation

Authors: Pyc, M. A., & Rawson, K. A.
Affiliations: Kent State University (US)
Corresponding author: Pyc, Mary A.
ORCID: Rawson, Katherine A.
Venue: Journal of Memory and Language
Publisher: Elsevier BV
Published: 2009-05
Volume: 60
Issue: 4
Pages: 437-447
ISSN: 0749-596X (ISSN-L)
DOI: 10.1016/j.jml.2009.01.004
OpenAlex: W2168292591
Cited by: 551
References: 4
Citations by year: 2026: 16 · 2025: 33 · 2024: 41 · 2023: 35 · 2022: 82 · 2021: 42 · 2020: 43 · 2019: 35 · 2018: 33 · 2017: 30 · 2016: 23 · 2015: 25 · 2014: 28 · 2013: 28 · 2012: 16
FWCI: 9.04
Type: journal article
Language: en
Open access: closed
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations: Journal of Memory and Language (publishedVersion)
Funders: Kent State University
Institute of Education Sciences
Topics: Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Functions and Memory
Life Sciences › Neuroscience › Cognitive Neuroscience › Memory and Neural Mechanisms
Keywords: retrieval practice, desirable difficulties, memory consolidation, interstimulus interval, test performance, metacognition

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Testing the retrieval effort hypothesis: Does greater difficulty correctly recalling information lead to higher levels of memory?

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Record

About these checks