Four semesters investigating frequency of testing, the testing effect, and transfer of training.
Abstract
We carried out 4 semester-long studies of student performance in a college research methods course (total N = 588). Two sections of it were taught each semester with systematic and controlled differences between them. Key manipulations were repeated (with some variation) across the 4 terms, allowing assessment of replicability of effects. Variables studied included frequency of tests (e.g., 2 vs. 8 in-class exams), the repetition of some and not other exam items (i.e., the testing effect), and variation of test items between the in-class exams and the final exam (e.g., identical items vs. controlled changes in items). Some studies also manipulated presence or absence of low-stakes quizzes. The repetition of test items generally led to better performance. However, we did not observe consistent superiority for items that were repeated exactly over those that were repeated in modified form; the reverse was more often the case. The effect of the low-stakes quizzes was minimal at best. Results are discussed in terms of memory and transfer of training models. (PsycINFO Database Record (c) 2017 APA, all rights reserved)
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The benefit of quizzing accrued even when the type of question changed from the quiz to the exam.
“When quizzing (with feedback) was incorporated, exam performance was enhanced for that material (relative to the nonquizzed materials), and this benefit accrued even when the type of question changed from the quiz to the exam (cf.Foss & Pirozzolo 2017).”
From Practicing Retrieval Facilitates Learning (2021) · cites this paper as Foss & Pirozzolo 2017
Cites
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013) · Psychological Science in the Public Interest · doi:10.1177/1529100612453266 · Cited by 3,138 · see citing works
Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention
Roediger, H. L., & Karpicke, J. D. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01693.x · Cited by 2,867 · see citing works
When and where do we apply what we learn?: A taxonomy for far transfer
Barnett, S. M., & Ceci, S. J. (2002) · Psychological Bulletin · doi:10.1037/0033-2909.128.4.612 · Cited by 2,076 · cited 1× · see citing works
“Stepping back, there is an extensive literature on transfer of training, one of the oldest topics in experimental psychology (e.g., Barnett & Ceci, 2002; Detterman & Sternberg, 1993; Harrison, Shipstead, & Engle, 2015; Mayer & Wittrock, 1996; Singley & Anderson, 1989; Taatgen, 2013).”
BackgroundThe power of testing memory: Basic research and implications for educational practice
Roediger, H. L., & Karpicke, J. D. (2006) · Perspectives on Psychological Science · doi:10.1111/j.1745-6916.2006.00012.x · Cited by 2,037 · see citing works
Distributed practice in verbal recall tasks: A review and quantitative synthesis.
Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006) · Psychological Bulletin · doi:10.1037/0033-2909.132.3.354 · Cited by 1,888 · see citing works
The effect of testing versus restudy on retention: A meta-analytic review of the testing effect.
Rowland, C. A. (2014) · Psychological Bulletin · doi:10.1037/a0037559 · Cited by 1,066 · cited 2× · see citing works
“While its prominence has sometimes waned, as evidenced by the title of Glover’s (1989) paper: “The ‘Testing’ Phenomenon: Not Gone But Nearly Forgotten,” the evidence for it is substantial and has continued to grow (e.g., Dunlosky, Rawson, Marsh, Nathan, & Willingham, 2013; Kang, McDaniel, & Pashler, 2011; Rawson & Dun…”
BackgroundA meta-analytic review of the distribution of practice effect: Now you see it, now you don’t
Donovan, J. J., & Radosevich, D. J. (1999) · Journal of Applied Psychology · doi:10.1037/0021-9010.84.5.795 · Cited by 696 · see citing works
Spacing Effects in Learning: A Temporal Ridgeline of Optimal Retention
Cepeda, N. J., Vul, E., Rohrer, D., Wixted, J. T., & Pashler, H. (2008) · Psychological Science · doi:10.1111/j.1467-9280.2008.02209.x · Cited by 631 · see citing works
Repeated testing produces superior transfer of learning relative to repeated studying.
Butler, A. C. (2010) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/a0019902 · Cited by 492 · cited 1× · see citing works
“This finding is consistent with data from Butler (2010) and replicates observations made by Bjork et al. (2014), and by McDermott et al. (2014).”
ResultMcDaniel, M. A., Agarwal, P. K., Huelser, B. J., McDermott, K. B., & Roediger, H. L. (2011) · Journal of Educational Psychology · doi:10.1037/a0021782 · Cited by 360 · see citing works
Maintenance of Foreign Language Vocabulary and the Spacing Effect
Bahrick, H. P., Bahrick, L. E., Bahrick, A. S., & Bahrick, P. E. (1993) · Psychological Science · doi:10.1111/j.1467-9280.1993.tb00571.x · Cited by 334 · see citing works
Generalizing test-enhanced learning from the laboratory to the classroom
Mcdaniel, M. A., Roediger, H. L., & Mcdermott, K. B. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194052 · Cited by 326 · see citing works
(2010) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(10)53003-2 · Cited by 255 · see citing works
Using tests to enhance 8th grade students' retention of U.S. history facts
Carpenter, S. K., Pashler, H., & Cepeda, N. J. (2009) · Applied Cognitive Psychology · doi:10.1002/acp.1507 · Cited by 254 · see citing works
The Exam-A-Day Procedure Improves Performance in Psychology Classes
Leeming, F. C. (2002) · Teaching of Psychology · doi:10.1207/s15328023top2903_06 · Cited by 222 · see citing works
Effects of testing on learning of functions
Kang, S. H. K., McDaniel, M. A., & Pashler, H. (2011) · Psychonomic Bulletin & Review · doi:10.3758/s13423-011-0113-x · Cited by 57 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Testing (quizzing) boosts classroom learning: a systematic and meta-analytic review
Yang, C., Luo, L., Vadillo, M. A., Yu, R., & Shanks, D. R. (2021) · Psychological Bulletin · doi:10.1037/bul0000309 · Cited by 333 · see citing works
Agarwal, P. K., Nunes, L. D., & Blunt, J. R. (2021) · Educational Psychology Review · doi:10.1007/s10648-021-09595-9 · Cited by 211 · see citing works
“We included experiments by Foss and Pirozzolo (2017), in which retrieval practice in the form of 4 – 8 exams was compared to learning after 2 – 3 exams.”
MethodThe science of effective learning with spacing and retrieval practice
Carpenter, S. K., Pan, S. C., & Butler, A. C. (2022) · Nature Reviews Psychology · doi:10.1038/s44159-022-00089-1 · Cited by 184 · see citing works
Practicing Retrieval Facilitates Learning
McDermott, K. B. (2021) · Annual review of psychology · doi:10.1146/annurev-psych-010419-051019 · Cited by 113 · see citing works
“When quizzing (with feedback) was incorporated, exam performance was enhanced for that material (relative to the nonquizzed materials), and this benefit accrued even when the type of question changed from the quiz to the exam (cf. Foss & Pirozzolo 2017).”
BackgroundExamining the Testing Effect in University Teaching: Retrievability and Question Format Matter
Greving, S., & Richter, T. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02412 · Cited by 70 · see citing works
Morphew, J., da Silva, M. R., Herman, G. L., & West, M. (2019) · Applied Cognitive Psychology · doi:10.1002/acp.3605 · Cited by 43 · see citing works
“More recently, Foss and Pirozzolo (2017) found that across four semesters, students assessed more frequently scored higher on a final exam with a small effect size (d = 0.16).”
BackgroundInfluentialGurung, R. A. R., & Burns, K. (2019) · Applied Cognitive Psychology · doi:10.1002/acp.3507 · Cited by 42 · see citing works
Emerging and Future Directions in Test-Enhanced Learning Research
Pan, S. C., Dunlosky, J., Xu, K. M., & Ouwehand, K. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09857-2 · Cited by 18 · see citing works
“Further, in line with an educational emphasis, some researchers have transitioned from examining practice testing in laboratory settings to conducting studies in classrooms and other authentic educational contexts (e.g., Foss & Pirozzolo, 2017; McDaniel et al., 2011; Rawson et al., 2013).”
Greving, S., & Richter, T. (2022) · Journal of Cognitive Psychology · doi:10.1080/20445911.2022.2085281 · Cited by 14 · see citing works
“…activities that are known to promote retention such as restudying (e.g. Batsell et al., 2017; Bell et al., 2015; Downs, 2015; Bjork et al., 2014; Foss & Pirozzolo, 2017; Glass et al., 2008; Johnson & Kiviniemi, 2009; Khanna, 2015; Lyle & Crawford, 2011; Marsh et al., 2012; Mayer et al., 2009;…”
BackgroundComparing Two Models of Collaborative Testing for Teaching Statistics
Eastridge, J., & Benson, W. L. (2019) · Teaching of Psychology · doi:10.1177/0098628319888113 · Cited by 12 · see citing works
“Repeated knowledge retrieval makes learned information more accessible for future use and has both short- and longterm benefits (Batsell et al., 2017; Foss & Pirozzolo, 2017; Racsmány, Szollosi, & Bencze, 2018; Yang & Shanks, 2018).”
BackgroundInvestigating the Effects of Testing Frequency on Programming Performance and Students' Behavior
Smith, D. H., Emeka, C., Fowler, M., West, M., & Zilles, C. B. (2023) · Technical Symposium on Computer Science Education · doi:10.1145/3545945.3569821 · Cited by 10 · see citing works
“As noted by Foss and Pirozzolo [9], this may be partially due to those under the frequent testing condition being tested on material closer to when they learned it and the smaller size of the more frequent assessments.”
BackgroundThomas, A., Smith, A. M., Kamal, K., & Gordon, L. T. (2020) · doi:10.1016/j.jarmac.2019.12.005 · Cited by 10
Personal Reflections on Science Communication and Sharing Retrieval Practice Research with Teachers
Agarwal, P. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09839-w · Cited by 9 · see citing works
“For example, Jones et al. (2016) found benefits from retrieval practice during spelling lessons in elementary school, Foss and Pirozzolo (2017) found benefits from retrieval practice in semester-long undergraduate psychology courses, and Kromann et al. (2010) found benefits from retrieval practice…”
BackgroundCan tests improve learning in real university classrooms?
Lamotte, M., Izaute, M., & Darnon, C. (2021) · Journal of Cognitive Psychology · doi:10.1080/20445911.2021.1956939 · Cited by 7 · see citing works
Achieving Testing Effects in an Authentic College Classroom
Shobe, E. R. (2021) · Teaching of Psychology · doi:10.1177/00986283211015669 · Cited by 7 · see citing works
“…restudy or nothing) of between-groups designs gives a learning advantage to some students over others (e.g., Bangert-Drowns et al., 1991; Foss & Pirozzolo, 2017) and within-subjects designs place all students at a learning disadvantage for some material (e.g., Bjork et al., 2014; McDaniel et…”
BackgroundThe Effects of Frequent Quizzing on Exam Scores in a Baccalaureate Nursing Course
Sartain, A. F., & Wright, V. (2020) · Nursing Education Perspectives · doi:10.1097/01.nep.0000000000000623 · Cited by 5 · see citing works
Foss, D. J., Pirozzolo, J. W., & Kulesz, P. A. (2023) · Learning and Instruction · doi:10.1016/j.learninstruc.2023.101807 · Cited by 3 · see citing works
Self-citationDo Domain Knowledge and Retrieval Practice Predict Students’ Study Order Decisions?
Babineau, A. L., Witherby, A. E., Ariel, R., Pelch, M. A., & Tauber, S. K. (2022) · Journal of Intelligence · doi:10.3390/jintelligence10040122 · Cited by 3 · see citing works
Eastridge, J. (2019) · Nurse Education Today · doi:10.1016/j.nedt.2019.06.001 · Cited by 3 · see citing works
Differential Effects of Additional Formative Assessments on Student Learning Behaviors and Outcomes
Borter, N. (2024) · Studia paedagogica · doi:10.5817/sp2023-3-1 · Cited by 6 · see citing works
“…to those measuring learning success (Carpenter, 2011; Eriksson, et al., 2011; Karpicke & Roediger, 2007) and when non-identical exercises covering the same material are employed (Batsell et al., 2017; Foss & Pirozzolo, 2017; Francis et al., 2020; Jensen et al., 2020; McDaniel et al., 2013).”
BackgroundGlodowski, K. R., Thompson, R. H., & Asunción, E. (2019) · Journal of Behavioral Education · doi:10.1007/s10864-019-09330-z
True-False Testing on Trial: Guilty as Charged or Falsely Accused?
Brabec, J. A., Pan, S. C., Bjork, E., & Bjork, R. (2020) · Educational Psychology Review · doi:10.1007/s10648-020-09546-w · Cited by 9 · see citing works
Napolitano, R., Solnosky, R. L., & Reinhart, W. F. (2025) · Journal of Civil Engineering Education · doi:10.1061/jceecd.eieng-2072
The Promoting Effect of Retrieval Practice Effect on Students’ Learning
詹, 静. (2024) · Advances in Social Sciences · doi:10.12677/ass.2024.138708
- Authors
- Foss, D. J., & Pirozzolo, J. W.
- ORCID
- Foss, Donald J.
- Venue
- Journal of Educational Psychology
- Published
- 2017-11
- Volume
- 109
- Issue
- 8
- Pages
- 1067-1083
- ISSN
- 0022-0663 (ISSN-L)
- 1939-2176
- OpenAlex
- W2600659263
- Cited by
- 30
- References
- 0
- Citations by year
- 2025: 3 · 2024: 5 · 2023: 3 · 2022: 3 · 2021: 3 · 2020: 4 · 2019: 5 · 2018: 3 · 2013: 1
- FWCI
- 1.89
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Educational Psychology (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Social Sciences › Education › Evaluation of Teaching Practices
- Social Sciences › Education › Student Assessment and Feedback
- Keywords
- testing effect, transfer of training, student performance, research methods course, memory models, final examinations
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
