Not New, but Nearly Forgotten: the Testing Effect Decreases or even Disappears as the Complexity of Learning Materials Increases
Open accessAbstract
The testing effect is a finding from cognitive psychology with relevance for education. It shows that after an initial study period, taking a practice test improves long-term retention compared to not taking a test and—more interestingly—compared to restudying the learning material. Boundary conditions of the effect that have received attention include the test format, retrieval success on the initial test, the retention interval, or the spacing of tests. Another potential boundary condition concerns the complexity of learning materials, that is, the number of interacting information elements a learning task contains. This insight is not new, as research from a century ago already had indicated that the testing effect decreases as the complexity of learning materials increases, but that finding seems to have been nearly forgotten. Studies presented in this special issue suggest that the effect may even disappear when the complexity of learning material is very high. Since many learning tasks in schools are high in element interactivity, a failure to find the effect under these conditions is relevant for education. Therefore, this special issue hopes to put this potential boundary condition back on the radar and provide a starting point for discussion and future research on this topic.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The generation effect is reduced in novice learners because there are no previous schemas to work with.
“Furthermore, the generation effect is reduced in novice learners, as there are no previous schemas to work with (Leahy et al., 2015; van Gog & Sweller, 2015).”
From Does difficulty moderate learning? A comparative analysis of the desirable difficulties framework and cognitive load theory (2025) · cites this paper as van Gog & Sweller, 2015
Cites
Sweller, J., Ayres, P., & Kalyuga, S. (2011) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_446 · Cited by 5,538 · see citing works
Self-citationThe Psychology of Learning and Motivation
(2021) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/s0079-7421(21)x0002-3 · Cited by 5,081 · see citing works
Journal of Experimental Psychology: Human Learning and Memory
Bourne, L. E. (1977) · Cited by 4,328 · see citing works
Acquisition of cognitive skill
Anderson, J. R. (1982) · Psychological Review · doi:10.1037/0033-295x.89.4.369 · Cited by 3,663 · see citing works
Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention
Roediger, H. L., & Karpicke, J. D. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01693.x · Cited by 2,867 · see citing works
The power of testing memory: Basic research and implications for educational practice
Roediger, H. L., & Karpicke, J. D. (2006) · Perspectives on Psychological Science · doi:10.1111/j.1745-6916.2006.00012.x · Cited by 2,037 · see citing works
Element Interactivity and Intrinsic, Extraneous, and Germane Cognitive Load
Sweller, J. (2010) · Educational Psychology Review · doi:10.1007/s10648-010-9128-5 · Cited by 2,020 · see citing works
Self-citationThe Critical Importance of Retrieval for Learning
Karpicke, J. D., & Roediger, H. L. (2008) · Science · doi:10.1126/science.1152408 · Cited by 1,791 · see citing works
The effect of testing versus restudy on retention: A meta-analytic review of the testing effect.
Rowland, C. A. (2014) · Psychological Bulletin · doi:10.1037/a0037559 · Cited by 1,066 · cited 2× · see citing works
“According to a Google Scholar search (December 2014), his work has only been cited in nine other publications.”
Karpicke, J. D., Butler, A. C., & Roediger III, H. L. (2009) · Memory · doi:10.1080/09658210802647009 · Cited by 745 · see citing works
Metacognition and affect: What can metacognitive experiences tell us about the learning process?
Efklides, A. (2006) · Educational Research Review · doi:10.1016/j.edurev.2005.11.001 · Cited by 724 · see citing works
Exploring the Assistance Dilemma in Experiments with Cognitive Tutors
Koedinger, K. R., & Aleven, V. (2007) · Educational Psychology Review · doi:10.1007/s10648-007-9049-0 · Cited by 632 · see citing works
Repeated retrieval during learning is the key to long-term retention☆
Karpicke, J. D., & ROEDIGERIII, H. (2006) · Journal of Memory and Language · doi:10.1016/j.jml.2006.09.004 · Cited by 599 · see citing works
Testing the testing effect in the classroom
McDaniel, M. A., Anderson, J. L., Derbish, M. H., & Morrisette, N. (2007) · European Journal of Cognitive Psychology · doi:10.1080/09541440701326154 · Cited by 563 · see citing works
Pyc, M. A., & Rawson, K. A. (2009) · Journal of Memory and Language · doi:10.1016/j.jml.2009.01.004 · Cited by 551 · see citing works
Effects of pairs of problems and examples on task performance and different types of cognitive load
Leppink, J., Paas, F., van Gog, T., van der Vleuten, C. P. M., & van Merriënboer, J. J. G. (2014) · Learning and Instruction · doi:10.1016/j.learninstruc.2013.12.001 · Cited by 520 · see citing works
Test format and corrective feedback modify the effect of testing on long-term retention
Kang, S. H. K., McDermott, K. B., & Roediger, H. L. (2007) · European Journal of Cognitive Psychology · doi:10.1080/09541440601056620 · Cited by 513 · see citing works
Cue strength as a moderator of the testing effect: The benefits of elaborative retrieval.
Carpenter, S. K. (2009) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/a0017021 · Cited by 503 · see citing works
Repeated testing produces superior transfer of learning relative to repeated studying.
Butler, A. C. (2010) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/a0019902 · Cited by 492 · cited 1× · see citing works
“…transfer (e.g., recalling a word in a sentence that differed from the word that had to be recalled from that same sentence on an initial test) or contained cues that were also present in the original text (i.e., corresponding words or terms) that would invite the correct answer (e.g., Butler 2010).”
Unsuccessful retrieval attempts enhance subsequent learning.
Kornell, N., Hays, M. J., & Bjork, R. A. (2009) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/a0015729 · Cited by 484 · see citing works
Ten Benefits of Testing and Their Applications to Educational Practice
Roediger, H. L., III, Putnam, A. L., & Smith, M. A. (2011) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/b978-0-12-387691-1.00001-6 · Cited by 453 · see citing works
When Does Feedback Facilitate Learning of Words?
Pashler, H., Cepeda, N. J., Wixted, J. T., & Rohrer, D. (2005) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.31.1.3 · Cited by 445 · see citing works
Testing improves long-term retention in a simulated classroom setting
Butler, A. C., & Roediger, H. L. (2007) · European Journal of Cognitive Psychology · doi:10.1080/09541440701326097 · Cited by 434 · see citing works
The "testing" phenomenon: Not gone but nearly forgotten.
Glover, J. A. (1989) · Journal of Educational Psychology · doi:10.1037/0022-0663.81.3.392 · Cited by 392 · cited 2× · see citing works
“Interestingly, this insight is not new, although, as was once true of the testing effect itself (Glover 1989), it seems to have been nearly forgotten.”
BackgroundTest-enhanced learning in the classroom: Long-term improvements from quizzing.
Roediger, H. L., III, Agarwal, P. K., McDaniel, M. A., & McDermott, K. B. (2011) · Journal of Experimental Psychology Applied · doi:10.1037/a0026252 · Cited by 390 · see citing works
Learning as a Generative Activity: Eight Learning Strategies that Promote Understanding
Fiorella, L., & Mayer, R. E. (2015) · doi:10.1017/cbo9781107707085 · Cited by 388 · see citing works
Carpenter, S. K., & Delosh, E. L. (2006) · Memory & Cognition · doi:10.3758/bf03193405 · Cited by 386 · see citing works
Larsen, D. P., Butler, A. C., & Roediger III, H. L. (2009) · Medical Education · doi:10.1111/j.1365-2923.2009.03518.x · Cited by 383 · see citing works
Why Testing Improves Memory: Mediator Effectiveness Hypothesis
Pyc, M. A., & Rawson, K. A. (2010) · Science · doi:10.1126/science.1191465 · Cited by 371 · see citing works
McDaniel, M. A., Agarwal, P. K., Huelser, B. J., McDermott, K. B., & Roediger, H. L. (2011) · Journal of Educational Psychology · doi:10.1037/a0021782 · Cited by 360 · cited 1× · see citing works
“…condition, because even though the term testing effect is also used to refer to beneficial effects of testing as an additional activity (e.g., McDaniel et al. 2011; Smith and Karpicke 2014), any beneficial effects of testing as an additional activity could be due to increased time on task…”
Examining the testing effect with open‐ and closed‐book tests
Agarwal, P. K., Karpicke, J. D., Kang, S. H. K., Roediger, H. L., III, & McDermott, K. B. (2007) · Applied Cognitive Psychology · doi:10.1002/acp.1391 · Cited by 343 · see citing works
Recitation As a Factor in Memorizing
Gates, A. I. (2010) · Medical Entomology and Zoology · Cited by 317 · see citing works
Roediger, H. L., III, & Pyc, M. A. (2012) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2012.09.002 · Cited by 310 · see citing works
Spitzer, H. F. (1939) · Journal of Educational Psychology · doi:10.1037/h0063404 · Cited by 306 · see citing works
The pretesting effect: do unsuccessful retrieval attempts enhance learning?
Richland, L. E., Kornell, N., & Kao, L. S. (2009) · Journal of Experimental Psychology: Applied · doi:10.1037/a0016496 · Cited by 302 · see citing works
Effects of worked examples, example-problem, and problem-example pairs on novices’ learning
van Gog, T., Kester, L., & Paas, F. (2011) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2010.10.004 · Cited by 283 · see citing works
The effects of tests on learning and forgetting.
Carpenter, S. K., Pashler, H., Wixted, J. T., & Vul, E. (2008) · Memory & Cognition · doi:10.3758/mc.36.2.438 · Cited by 281 · see citing works
Retrieval-Based Learning: An Episodic Context Account
Karpicke, J. D., Lehman, M. S., & Aue, W. R. (2012) · Current Directions in Psychological Science · Cited by 276 · see citing works
Using tests to enhance 8th grade students' retention of U.S. history facts
Carpenter, S. K., Pashler, H., & Cepeda, N. J. (2009) · Applied Cognitive Psychology · doi:10.1002/acp.1507 · Cited by 254 · see citing works
Subjective organization and effects of repetition in multi-trial free-recall learning
Tulving, E. (1966) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(66)80016-6 · Cited by 249 · see citing works
Different rates of forgetting following study versus test trials
Wheeler, M., Ewers, M., & Buonanno, J. F. (2003) · Memory · doi:10.1080/09658210244000414 · Cited by 249 · see citing works
Testing Enhances the Transfer of Learning
Carpenter, S. K. (2012) · Current Directions in Psychological Science · doi:10.1177/0963721412452728 · Cited by 230 · see citing works
Retrieval-Based Learning: A Perspective for Enhancing Meaningful Learning
Karpicke, J. D., & Grimaldi, P. J. (2012) · Educational Psychology Review · doi:10.1007/s10648-012-9202-2 · Cited by 226 · see citing works
A testing effect with multimedia learning.
Johnson, C. I., & Mayer, R. E. (2009) · Journal of Educational Psychology · doi:10.1037/a0015183 · Cited by 221 · see citing works
Test-potentiated learning: Distinguishing between direct and indirect effects of tests.
Arnold, K. M., & McDermott, K. B. (2012) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/a0029199 · Cited by 216 · see citing works
The testing effect in free recall is associated with enhanced organizational processes
Zaromb, F. M., & Roediger, H. L. (2010) · Memory & Cognition · doi:10.3758/mc.38.8.995 · Cited by 206 · see citing works
McDermott, K. B., Agarwal, P. K., D'Antonio, L., Roediger, H. L., III, & McDaniel, M. A. (2013) · Journal of Experimental Psychology Applied · doi:10.1037/xap0000004 · Cited by 199 · see citing works
Learners’ choices and beliefs about self-testing
Kornell, N., & Son, L. K. (2009) · Memory · doi:10.1080/09658210902832915 · Cited by 190 · see citing works
The dynamics of learning and allocation of study time to a region of proximal learning
Metcalfe, J., & Kornell, N. (2003) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.132.4.530 · Cited by 186 · see citing works
Karpicke, J. D., Lehman, M., & Aue, W. R. (2014) · Psychology of Learning and Motivation · doi:10.1016/b978-0-12-800283-4.00007-1 · Cited by 185 · see citing works
McDaniel, M. A., Wildman, K. M., & Anderson, J. L. (2012) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2011.10.001 · Cited by 177 · see citing works
What types of learning are enhanced by a cued recall test?
Carpenter, S. K., Pashler, H., & Vul, E. (2006) · Psychonomic Bulletin & Review · doi:10.3758/bf03194004 · Cited by 172 · see citing works
Learning with retrieval-based concept mapping.
Blunt, J. R., & Karpicke, J. D. (2014) · Journal of Educational Psychology · doi:10.1037/a0035934 · Cited by 171 · see citing works
The Testing Effect and the Retention Interval: Questions and Answers
Toppino, T. C., & Cohen, M. S. (2009) · Experimental Psychology · doi:10.1027/1618-3169.56.4.252 · Cited by 160 · see citing works
Retrieval practice with short-answer, multiple-choice, and hybrid tests
Smith, M. A., & Karpicke, J. D. (2013) · Memory · doi:10.1080/09658211.2013.831454 · Cited by 143 · see citing works
Agarwal, P. K., Bain, P. M., & Chamberlain, R. W. (2012) · Educational Psychology Review · doi:10.1007/s10648-012-9210-2 · Cited by 132 · see citing works
Covert retrieval practice benefits retention as much as overt retrieval practice.
Smith, M. A., Roediger, H. L., III, & Karpicke, J. D. (2013) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/a0033569 · Cited by 124 · see citing works
Learning as a Generative Activity: Eight Learning Strategies that Promote Understanding
Fiorella, L., & Mayer, R. E. (2015) · Cited by 109 · see citing works
Why is test–restudy practice beneficial for memory? An evaluation of the mediator shift hypothesis.
Pyc, M. A., & Rawson, K. A. (2011) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/a0026166 · Cited by 98 · see citing works
Material-appropriate processing: A contextualist approach to reading and studying strategies
McDaniel, M. A., & Einstein, G. O. (1989) · Educational Psychology Review · doi:10.1007/bf01326639 · Cited by 95 · see citing works
A Test of the Testing Effect: Acquiring Problem‐Solving Skills From Worked Examples
van Gog, T., & Liesbeth Kester, A. (2012) · Cognitive Science · doi:10.1111/cogs.12002 · Cited by 95 · see citing works
THE PSYCHOLOGY OF LEARNING AND MOTIVATION: Cognition in Education
(2011) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/b978-0-12-387691-1.00011-9 · Cited by 91 · see citing works
Retrieval attempts enhance learning, but retrieval success (versus failure) does not matter.
Kornell, N., Klein, P. J., & Rawson, K. A. (2014) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/a0037850 · Cited by 90 · see citing works
Retrieval-based learning: The need for guided retrieval in elementary school children
Karpicke, J. D., Blunt, J. R., Smith, M. A., & Karpicke, S. (2014) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2014.07.008 · Cited by 89 · see citing works
Does response mode affect amount recalled or the magnitude of the testing effect?
Putnam, A. L., & Roediger, H. L. (2013) · Memory & Cognition · doi:10.3758/s13421-012-0245-x · Cited by 86 · see citing works
Weinstein, Y., McDermott, K. B., & Roediger, H. L., III (2010) · Journal of Experimental Psychology Applied · doi:10.1037/a0020992 · Cited by 83 · see citing works
Tse, C.-S., Balota, D. A., & Roediger, H. L. (2010) · Psychology and Aging · doi:10.1037/a0019933 · Cited by 80 · see citing works
Cranney, J., Ahn, M., McKinnon, R., Morris, S., & Watts, K. (2009) · European Journal of Cognitive Psychology · doi:10.1080/09541440802413505 · Cited by 80 · see citing works
Enhancing visuospatial learning: The benefit of retrieval practice
Kang, S. H. K. (2010) · Memory & Cognition · doi:10.3758/mc.38.8.1009 · Cited by 77 · see citing works
Leahy, W., Hanham, J., & Sweller, J. (2015) · Educational Psychology Review · doi:10.1007/s10648-015-9296-4 · Cited by 77 · see citing works
Self-citationThe Effect of Retrieval Practice in Primary School Vocabulary Learning
Goossens, N. A. M. C., Camp, G., Verkoeijen, P. P. J. L., & Tabbers, H. K. (2013) · Applied Cognitive Psychology · doi:10.1002/acp.2956 · Cited by 72 · see citing works
Bouwmeester, S., & Verkoeijen, P. (2011) · Journal of Memory and Language · doi:10.1016/j.jml.2011.02.005 · Cited by 69 · see citing works
Tests enhance retention and transfer of spatial learning
Carpenter, S. K., & Kelly, J. W. (2012) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0221-2 · Cited by 69 · see citing works
van Gog, T., Liesbeth Kester, A., Dirkx, K., Hoogerheide, V., Boerboom, J., & Verkoeijen, P. P. J. L. (2015) · Educational Psychology Review · doi:10.1007/s10648-015-9297-3 · Cited by 66 · see citing works
Benefits of testing memory: Best practices and boundary conditions
Roediger, H. L., III (2009) · Psychology Press eBooks · doi:10.4324/9780203869611-10 · Cited by 65 · see citing works
Effects of testing on learning of functions
Kang, S. H. K., McDaniel, M. A., & Pashler, H. (2011) · Psychonomic Bulletin & Review · doi:10.3758/s13423-011-0113-x · Cited by 57 · see citing works
Free recall enhances subsequent learning
Arnold, K. M., & McDermott, K. B. (2013) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0370-3 · Cited by 55 · see citing works
Learning Adinkra symbols: The effect of testing
Coppens, L. C., Verkoeijen, P. P. J. L., & Rikers, R. M. J. P. (2011) · Journal of Cognitive Psychology · doi:10.1080/20445911.2011.507188 · Cited by 53 · see citing works
Is the benefit of retrieval practice modulated by motivation?
Kang, S. H. K., & Pashler, H. (2014) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2014.05.006 · Cited by 50 · see citing works
Retrieval practice and elaborative encoding benefit memory in younger and older adults.
Coane, J. H. (2013) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2013.04.001 · Cited by 47 · see citing works
Guided retrieval practice of educational materials using automated scoring.
Grimaldi, P. J., & Karpicke, J. D. (2013) · Journal of Educational Psychology · doi:10.1037/a0033208 · Cited by 47 · see citing works
Jang, Y., Wixted, J. T., Pecher, D., Zeelenberg, R., & Huber, D. E. (2012) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2011.638079 · Cited by 46 · see citing works
Retrieval practice: the lack of transfer to deductive inferences
Tran, R., Rohrer, D., & Pashler, H. (2014) · Psychonomic Bulletin & Review · doi:10.3758/s13423-014-0646-x · Cited by 39 · see citing works
The role of organizational processes in long-term retention.
Masson, M. E. J., & McDaniel, M. A. (1981) · Journal of Experimental Psychology Human Learning & Memory · doi:10.1037/0278-7393.7.2.100 · Cited by 34 · see citing works
Carpenter, S. K., Sachs, R. E., Martin, B., Schmidt, K., & Looft, R. (2011) · Psychonomic Bulletin & Review · doi:10.3758/s13423-011-0185-7 · Cited by 28 · see citing works
A Short-Term Testing Effect in Cross-Language Recognition
Verkoeijen, P. P. J. L., Bouwmeester, S., & Camp, G. (2012) · Psychological Science · doi:10.1177/0956797611435132 · Cited by 25 · see citing works
The Effect of Testing on the Retention of Coherent and Incoherent Text Material
de Jonge, M., Tabbers, H. K., & Rikers, R. M. J. P. (2015) · Educational Psychology Review · doi:10.1007/s10648-015-9300-z · Cited by 20 · see citing works
Applying cognitive psychology to education: Complexities and prospects.
Roediger, H. L., III, & Pyc, M. A. (2012) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2012.10.006 · Cited by 9 · see citing works
Is recitation an effective tool for adult learners?
McDermott, K. B., & Naaz, F. (2014) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2014.06.006 · Cited by 9 · see citing works
The Relative Benefits of Studying and Testing on Long-Term Retention
Roediger, H. L., III, & Nestojko, J. F. (2015) · doi:10.4324/9781315885582-12 · Cited by 9 · see citing works
Organizational Processes Contribute to the Testing Effect in Free Recall
Zaromb, F. M. (2010) · Open Scholarship Institutional Repository (Washington University in St. Louis) · doi:10.7936/k7vh5kzn · Cited by 5 · see citing works
Flagged first, then by citations
2 of the 93 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Eight Ways to Promote Generative Learning
Fiorella, L., & Mayer, R. E. (2016) · Educational Psychology Review · doi:10.1007/s10648-015-9348-9 · Cited by 832 · see citing works
Transfer of test- enhanced learning: meta-analytic review and synthesis
Pan, S. C., & Rickard, T. C. (2018) · Psychological Bulletin · doi:10.1037/bul0000151 · Cited by 286 · see citing works
Teaching the science of learning
Weinstein, Y., Madan, C., & Sumeracki, M. A. (2018) · Cognitive Research · doi:10.1186/s41235-017-0087-y · Cited by 204 · see citing works
Retrieval-Based Learning: A Decade of Progress
Karpicke, J. D. (2017) · Learning and Memory: A Comprehensive Reference · doi:10.1016/b978-0-12-809324-5.21055-9 · Cited by 223 · see citing works
The science of effective learning with spacing and retrieval practice
Carpenter, S. K., Pan, S. C., & Butler, A. C. (2022) · Nature Reviews Psychology · doi:10.1038/s44159-022-00089-1 · Cited by 184 · see citing works
Retrieval practice & Bloom’s taxonomy: Do students need fact knowledge before higher order learning?
Agarwal, P. K. (2018) · Journal of Educational Psychology · doi:10.1037/edu0000282 · Cited by 178 · see citing works
Kirk-Johnson, A., Galla, B. M., & Fraundorf, S. H. (2019) · Cognitive Psychology · doi:10.1016/j.cogpsych.2019.101237 · Cited by 156 · see citing works
The Testing Effect Is Alive and Well with Complex Materials
Karpicke, J. D., & Aue, W. R. (2015) · Educational Psychology Review · doi:10.1007/s10648-015-9309-3 · Cited by 140 · see citing works
McLaren, B. M., van Gog, T., Ganoe, C. H., Karabinos, M., & Yaron, D. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.08.038 · Cited by 110 · see citing works
Ross, B., Chase, A., Robbie, D., Oates, G., & Absalom, Y. (2018) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-018-0113-2 · Cited by 95 · see citing works
Enhancing learning by retrieval: Enriching free recall with elaborative prompting
Endres, T., Carpenter, S. K., Martin, A., & Renkl, A. (2016) · Learning and Instruction · doi:10.1016/j.learninstruc.2016.11.010 · Cited by 91 · see citing works
Relations between the worked example and generation effects on immediate and delayed tests
Chen, O., Kalyuga, S., & Sweller, J. (2016) · Learning and Instruction · doi:10.1016/j.learninstruc.2016.06.007 · Cited by 82 · see citing works
Self-citationEffects of incorporating retrieval into learning tasks: The complexity of the tasks matters
Roelle, J., & Berthold, K. (2017) · Learning and Instruction · doi:10.1016/j.learninstruc.2017.01.008 · Cited by 71 · see citing works
Carpenter, S. K., Endres, T., & Hui, L. (2020) · Educational Psychology Review · doi:10.1007/s10648-020-09562-w · Cited by 75 · see citing works
Roelle, J., & Nückles, M. (2019) · Journal of Educational Psychology · doi:10.1037/edu0000345 · Cited by 74 · see citing works
Cognitive Load Theory, Resource Depletion and the Delayed Testing Effect
Leahy, W., & Sweller, J. (2019) · Educational Psychology Review · doi:10.1007/s10648-019-09476-2 · Cited by 59 · see citing works
Self-citationEndres, T., Lovell, O., Morkunas, D., Rieß, W., & Renkl, A. (2023) · British Journal of Educational Psychology · doi:10.1111/bjep.12563 · Cited by 57 · see citing works
Stenlund, T., Jönsson, F. U., & Jonsson, B. (2016) · Educational Psychology · doi:10.1080/01443410.2016.1143087 · Cited by 55 · see citing works
Yeo, D. J. S., & Fazio, L. K. (2018) · Journal of Educational Psychology · doi:10.1037/edu0000268 · Cited by 55 · see citing works
Extending the testing effect to self-regulated learning
Fernandez, J., & Jamet, É. (2016) · Metacognition and Learning · doi:10.1007/s11409-016-9163-9 · Cited by 51 · see citing works
Investigating the testing effect: Retrieval as a characteristic of effective study strategies
Bae, C. L. L., Therriault, D. J., & Redifer, J. L. (2018) · Learning and Instruction · doi:10.1016/j.learninstruc.2017.12.008 · Cited by 49 · see citing works
Retrieval Practice Benefits Deductive Inference
Eglington, L. G., & Kang, S. H. K. (2016) · Educational Psychology Review · doi:10.1007/s10648-016-9386-y · Cited by 47 · see citing works
Ebersbach, M., Feierabend, M., & Nazari, K. B. B. (2020) · Applied Cognitive Psychology · doi:10.1002/acp.3639 · Cited by 45 · see citing works
Complexity affects performance, cognitive load, and awareness
Zeitlhofer, I., Zumbach, J., & Schweppe, J. (2024) · Learning and Instruction · doi:10.1016/j.learninstruc.2024.102001 · Cited by 58 · see citing works
The Status of the Testing Effect for Complex Materials: Still a Winner
Rawson, K. A. (2015) · Educational Psychology Review · doi:10.1007/s10648-015-9308-4 · Cited by 43 · see citing works
When Instructional Guidance is Needed
Chen, O., Kalyuga, S., & Sweller, J. (2016) · The Educational and Developmental Psychologist · doi:10.1017/edp.2016.16 · Cited by 42 · see citing works
Self-citationMorphew, J., da Silva, M. R., Herman, G. L., & West, M. (2019) · Applied Cognitive Psychology · doi:10.1002/acp.3605 · Cited by 43 · see citing works
Hoch, E., Sidi, Y., Ackerman, R., Hoogerheide, V., & Scheiter, K. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09779-5 · Cited by 39 · see citing works
Tests and academic cheating: do learning tasks influence cheating by way of negative evaluations?
Wenzel, K., & Reinhard, M. (2020) · Social Psychology of Education · doi:10.1007/s11218-020-09556-0 · Cited by 44 · see citing works
Cognitive Load Theory, Element Interactivity, and the Testing and Reverse Testing Effects
Hanham, J., Leahy, W., & Sweller, J. (2017) · Applied Cognitive Psychology · doi:10.1002/acp.3324 · Cited by 36 · see citing works
Self-citationUndesirable Difficulty Effects in the Learning of High-Element Interactivity Materials
Chen, O., Castro-Alonso, J. C., PAAS, F. G. W. C., & Sweller, J. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01483 · Cited by 32 · see citing works
Self-citationTesting Is More Desirable When It Is Adaptive and Still Desirable When Compared to Note-Taking
Heitmann, S., Grund, A., Berthold, K., Fries, S., & Roelle, J. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02596 · Cited by 27 · see citing works
Does Providing Prompts During Retrieval Practice Improve Learning?
Smith, M. A., Blunt, J. R., Whiffen, J. W., & Karpicke, J. D. (2016) · Applied Cognitive Psychology · doi:10.1002/acp.3227 · Cited by 27 · see citing works
Facilitating transfer through incorrect examples and explanatory feedback
Corral, D., & Carpenter, S. K. (2020) · Quarterly Journal of Experimental Psychology · doi:10.1177/1747021820909454 · Cited by 26 · see citing works
Belda-Medina, J., & Kokošková, V. (2024) · Innovation in Language Learning and Teaching · doi:10.1080/17501229.2024.2435900 · Cited by 30 · see citing works
Ebersbach, M., & Nazari, K. B. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.00811 · Cited by 22 · see citing works
Teplitski, M., Irani, T. A., Krediet, C., Cesare, M. D., & Marvasi, M. (2018) · Journal of Food Science Education · doi:10.1111/1541-4329.12129 · Cited by 24 · see citing works
A database of general knowledge question performance in older adults
Coane, J. H., & Umanath, S. (2021) · Behavior Research Methods · doi:10.3758/s13428-020-01493-2 · Cited by 20 · see citing works
Sepp, S., Wong, M., Hoogerheide, V., & Castro-Alonso, J. C. (2022) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12715 · Cited by 30 · see citing works
Emerging and Future Directions in Test-Enhanced Learning Research
Pan, S. C., Dunlosky, J., Xu, K. M., & Ouwehand, K. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09857-2 · Cited by 18 · see citing works
Retrieval Practice Enhances Analogical Problem Solving
Wong, S. S. H., Ng, G. J. P., Tempel, T., & Lim, S. W. H. (2017) · The Journal of Experimental Education · doi:10.1080/00220973.2017.1409185 · Cited by 19 · see citing works
Nip, T., Gunter, E. L., Herman, G. L., Morphew, J., & West, M. (2018) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3159450.3159500 · Cited by 17 · see citing works
Yan, V. X., Sana, F., & Carvalho, P. F. (2023) · Policy Insights from the Behavioral and Brain Sciences · doi:10.1177/23727322231218906 · Cited by 21 · see citing works
Wenzel, K., & Reinhard, M. (2019) · Intelligence · doi:10.1016/j.intell.2019.101405 · Cited by 20 · see citing works
Remote Interview Methods in Chemical Education Research
Trate, J. M., Teichert, M. A., Murphy, K. L., Srinivasan, S., Luxford, C. J., & Schneider, J. L. (2020) · Journal of Chemical Education · doi:10.1021/acs.jchemed.0c00680 · Cited by 16 · see citing works
Pyke, W., Lunau, J., & Javadi, A.-H. (2025) · Quarterly Journal of Experimental Psychology · doi:10.1177/17470218241308143 · Cited by 15 · see citing works
“…beneficial for such material (Chen et al., 2016b), research indicates that the generation effect diminishes or even reverses when dealing with high-element-interactivity tasks, such as complex problem-solving (Leahy et al., 2015; van Gog et al., 2015), for a review, see van Gog and Sweller (2015).”
Revisiting cognitive load theory: second thoughts and unaddressed questions
Leppink, J. (2020) · Scientia Medica · doi:10.15448/1980-6108.2020.1.36918 · Cited by 17 · see citing works
The testing effect and analogical problem-solving
Peterson, D. J., & Wissman, K. T. (2018) · Memory · doi:10.1080/09658211.2018.1491603 · Cited by 15 · see citing works
Greving, S., Lenhard, W., & Richter, T. (2022) · Teaching of Psychology · doi:10.1177/00986283211061204 · Cited by 14 · see citing works
Wenzel, K., & Reinhard, M. (2021) · Learning and Motivation · doi:10.1016/j.lmot.2020.101706 · Cited by 14 · see citing works
Rummer, R., & Schweppe, J. (2021) · Unterrichtswissenschaft · doi:10.1007/s42010-021-00137-4 · Cited by 14 · see citing works
The Testing Effect in the Lecture Hall: Does it Depend on Learner Prerequisites?
Glaser, J., & Richter, T. (2022) · Psychology Learning & Teaching · doi:10.1177/14757257221136660 · Cited by 15 · see citing works
Weißgerber, S. C., & Rummer, R. (2022) · Learning and Instruction · doi:10.1016/j.learninstruc.2022.101679 · Cited by 14 · see citing works
Endres, T., & Eitel, A. (2023) · Applied Cognitive Psychology · doi:10.1002/acp.4160 · Cited by 14 · see citing works
Slomer, A., & Chenkin, J. (2017) · AEM Education and Training · doi:10.1002/aet2.10044 · Cited by 12 · see citing works
Chikha, H. B., Zoudji, B., & Khacharem, A. (2023) · Psychological Research · doi:10.1007/s00426-023-01806-y · Cited by 13 · see citing works
Wong, M., Castro-Alonso, J. C., Ayres, P., & PAAS, F. G. W. C. (2019) · Routledge eBooks · doi:10.4324/9780429283895-7 · Cited by 13 · see citing works
The Use of Retrieval Practice in the Health Professions: A State-of-the-Art Review
Serra, M. J., Kaminske, A. N., Nebel, C. L., & Coppola, K. M. (2025) · Behavioral Sciences · doi:10.3390/bs15070974 · Cited by 16 · see citing works
Specific questions during retrieval practice are better for texts containing seductive details
Eitel, A., Endres, T., & Renkl, A. (2022) · Applied Cognitive Psychology · doi:10.1002/acp.3984 · Cited by 12 · see citing works
A Case Study of Using Test-Enhanced Learning as a Formative Assessment in High School Mathematics
Szeibert, J., Muzsnay, A., Szabó, C., & Zámbó, C. (2022) · International Journal of Science and Mathematics Education · doi:10.1007/s10763-022-10264-8 · Cited by 16 · see citing works
Pilot Study on the Relationship of Test Anxiety to Utilizing Self-Testing in Self-Regulated Learning
Weißgerber, S. C., & Reinhard, M. (2018) · International Journal of Psychological Studies · doi:10.5539/ijps.v10n4p95 · Cited by 10 · see citing works
Degree of learning and linear forgetting
Fisher, J. S., & Radvansky, G. A. (2021) · Quarterly Journal of Experimental Psychology · doi:10.1177/17470218211056464 · Cited by 9 · see citing works
Frank, B., & Kluge, A. (2018) · Zeitschrift für Arbeitswissenschaft · doi:10.1007/s41449-018-00136-9 · Cited by 9 · see citing works
Indirekte Effekte von Abrufübungen – Intuitiv und doch häufig unterschätzt
Endres, T., & Renkl, A. (2021) · Unterrichtswissenschaft · doi:10.1007/s42010-021-00140-9 · Cited by 9 · see citing works
Explanations and Interactives Improve Subjective Experiences in Online Courseware
Thomas, M. P., Türkay, S., & Parker, M. J. (2017) · The International Review of Research in Open and Distributed Learning · doi:10.19173/irrodl.v18i7.3076 · Cited by 9 · see citing works
Ruitenburg, S. K., Ackermans, K., Kirschner, P. A., Jarodzka, H., & Camp, G. (2024) · Learning and Instruction · doi:10.1016/j.learninstruc.2024.102027 · Cited by 11 · see citing works
Effects of making errors in learning a foreign language
Guzmán-Muñoz, F. J. (2020) · Journal of Cognitive Psychology · doi:10.1080/20445911.2020.1711766 · Cited by 6 · see citing works
Girard, O., Gaoua, N., Grantham, J., Knez, W. L., Walsh, A., & Racinais, S. (2021) · Temperature · doi:10.1080/23328940.2021.1959289 · Cited by 8 · see citing works
Kang, Y., Ha, H., & Lee, H. S. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09772-y · Cited by 7 · see citing works
Fraundorf, S. H., Caddick, Z. A., Nokes‐Malach, T. J., & Rottman, B. M. (2023) · Cognitive Research Principles and Implications · doi:10.1186/s41235-023-00508-8 · Cited by 10 · see citing works
Der Testungseffekt beim Lernen mit Texten
Rummer, R. (2021) · Psychologische Rundschau · doi:10.1026/0033-3042/a000518 · Cited by 8 · see citing works
Jakobsson, A., Loberg, J. M., & Kjörk, M. (2023) · International Journal of Science Education · doi:10.1080/09500693.2023.2283906 · Cited by 10 · see citing works
Hartman, J. D., & Eichler, J. F. (2024) · Education sciences · doi:10.3390/educsci14111224 · Cited by 6 · see citing works
Abrufeffekte im Gedächtnis: Ein Überblick zur aktuellen Grundlagenforschung
Tempel, T., & Pastötter, B. (2021) · Psychologische Rundschau · doi:10.1026/0033-3042/a000517 · Cited by 7 · see citing works
Retrieval practice can improve classroom review despite low practice test performance
Leggett, J. M. I., Burt, J. S., & Carroll, A. (2019) · Applied Cognitive Psychology · doi:10.1002/acp.3517 · Cited by 6 · see citing works
Achieving Testing Effects in an Authentic College Classroom
Shobe, E. R. (2021) · Teaching of Psychology · doi:10.1177/00986283211015669 · Cited by 7 · see citing works
Berger, R., Kulgemeyer, C., & Lensing, P. (2019) · Zeitschrift für Didaktik der Naturwissenschaften · doi:10.1007/s40573-019-00099-2 · Cited by 6 · see citing works
Howe, P. D. L., McKague, M., Lodge, J. M., Blunden, A. G., & Saw, G. (2018) · Psychology Learning & Teaching · doi:10.1177/1475725718764181 · Cited by 6 · see citing works
The testing effect under divided attention: Educational application.
Buchin, Z. L., & Mulligan, N. W. (2019) · Journal of Experimental Psychology Applied · doi:10.1037/xap0000230 · Cited by 6 · see citing works
May, B. M. (2021) · International Journal of Mathematical Education in Science and Technology · doi:10.1080/0020739x.2021.1961034 · Cited by 7 · see citing works
Active Retrieval Improves Procedural Learning in Orthopedic Surgery
Ndoja, S., Dion, C.-A. B., Pirshahid, A. A., Charron, B. P., Durocher, A. M., McCarton, A., & LeBel, M. (2022) · Journal of surgical education · doi:10.1016/j.jsurg.2022.05.001 · Cited by 5 · see citing works
Sumeracki, M. A., Nebel, C. L., Kaminske, A. N., & Kuepper-Tetzel, C. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09854-5 · Cited by 4 · see citing works
Impact of different practice testing methods on learning outcomes
Akbulut, Y. (2024) · European Journal of Education · doi:10.1111/ejed.12626 · Cited by 6 · see citing works
Norman, G. R. (2016) · Advances in Health Sciences Education · doi:10.1007/s10459-016-9671-z · Cited by 5 · see citing works
Rachatasumrit, N., Koedinger, K. R., & Carvalho, P. F. (2025) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-025-00511-8 · Cited by 5 · see citing works
Wang, M., Yang, M., & Kyle, W. C. (2023) · Disciplinary and Interdisciplinary Science Education Research · doi:10.1186/s43031-023-00083-4 · Cited by 6 · see citing works
Do worked examples boost the spacing effect on lasting learning?
Krauspe, J., Ebersbach, M., Ludwig, A., & Scharf, F. (2025) · Learning and Instruction · doi:10.1016/j.learninstruc.2025.102103 · Cited by 5 · see citing works
Wie können Abrufübungen (nicht) für das Lernen in Schule und Hochschule relevant gemacht werden?
Roelle, J., Endres, T., & Renkl, A. (2021) · Unterrichtswissenschaft · doi:10.1007/s42010-021-00139-2 · Cited by 4 · see citing works
Rachatasumrit, N., Carvalho, P. F., Li, S., & Koedinger, K. R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-36272-9_5 · Cited by 4 · see citing works
Test-Based Learning: Inconsistent Effects Between Higher- and Lower-Level Test Questions
Roelle, J., Roelle, D., & Berthold, K. (2018) · The Journal of Experimental Education · doi:10.1080/00220973.2018.1434756 · Cited by 4 · see citing works
Dechamps, T., & Skulmowski, A. (2025) · Trends in Neuroscience and Education · doi:10.1016/j.tine.2025.100256 · Cited by 3 · see citing works
Wang, J., & Huo, C. (2025) · The Electronic Library · doi:10.1108/el-01-2025-0010 · Cited by 3 · see citing works
Retrieval Practice: A Tool for Teaching the Control-of-Variables Strategy in Science Classrooms?
Kranz, J., Möller, A., Kaufmann, K., & Tempel, T. (2024) · The Journal of Experimental Education · doi:10.1080/00220973.2024.2392684 · Cited by 7 · see citing works
Supporting comprehension: The advantages of multiple–choice over true—false practice tests
Hildenbrand, L., & Wiley, J. (2025) · Memory & Cognition · doi:10.3758/s13421-025-01726-9 · Cited by 3 · see citing works
Wilkinson, A. M., Hall, A. C. G., & Hogan, E. E. (2019) · Memory · doi:10.1080/09658211.2019.1632349 · Cited by 3 · see citing works
The Effect of Retrieval Practice on Transitive Inference.
Picklesimer, M. E., Buchin, Z. L., & Mulligan, N. W. (2019) · Experimental Psychology · doi:10.1027/1618-3169/a000467 · Cited by 3 · see citing works
Cadaret, C. N., & Yates, D. T. (2021) · Journal of Animal Science · doi:10.1093/jas/skab246 · Cited by 3 · see citing works
Sheykhmohammadi, N., Atashpendar, A., & Zampuniéris, D. (2024) · International Conference on Computer Supported Education · doi:10.5220/0012682500003693 · Cited by 3 · see citing works
Tordet, C., Fernandez, J., & Jamet, É. (2024) · Journal of Computer Assisted Learning · doi:10.1111/jcal.13083 · Cited by 7 · see citing works
Behavioral Analytics for Distributed Practices in Programming Problem-Solving
Alzaid, M., & Hsiao, I. (2019) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie43999.2019.9028583 · Cited by 3 · see citing works
- Authors
- van Gog, T., & Sweller, J.
- Affiliations
- UNSW Sydney (AU)
- ORCID
- Sweller, John
- Venue
- Educational Psychology Review
- Published
- 2015-05-16
- Volume
- 27
- Issue
- 2
- Pages
- 247-264
- ISSN
- 1040-726X (ISSN-L)
- 1573-336X
- OpenAlex
- W2128915986
- Cited by
- 164
- References
- 93
- Citations by year
- 2026: 11 · 2025: 18 · 2024: 22 · 2023: 21 · 2022: 10 · 2021: 16 · 2020: 13 · 2019: 15 · 2018: 17 · 2017: 7 · 2016: 10 · 2015: 4
- FWCI
- 7.61
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Educational Psychology Review (publishedVersion, cc-by)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion, other-oa)
- EUR Research Repository (Erasmus University Rotterdam) (submittedVersion, other-oa)
- RePub (Erasmus University Rotterdam) (submittedVersion, other-oa)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Utrecht University Repository (Utrecht University) (submittedVersion)
- Funders
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (452‐11‐006)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Keywords
- testing effect, learning complexity, element interactivity, long-term retention, practice tests, restudying, boundary conditions, retrieval practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
