Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing
Open accessAbstract
Multiple-choice tests are used frequently in higher education without much consideration of the impact this form of assessment has on learning. Multiple-choice testing enhances retention of the material tested (the testing effect); however, unlike other tests, multiple-choice can also be detrimental because it exposes students to misinformation in the form of lures. The selection of lures can lead students to acquire false knowledge (Roediger & Marsh, 2005). The present research investigated whether feedback could be used to boost the positive effects and reduce the negative effects of multiple-choice testing. Subjects studied passages and then received a multiple-choice test with immediate feedback, delayed feedback, or no feedback. In comparison with the no-feedback condition, both immediate and delayed feedback increased the proportion of correct responses and reduced the proportion of intrusions (i.e., lure responses from the initial multiple-choice test) on a delayed cued recall test. Educators should provide feedback when using multiple-choice tests.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Feedback can enhance the benefits of testing.
“However, there is evidence that feedback may enhance the benefits of tests (Butler & Roediger, 2008;Pashler, Cepeda, Wixted & Rohrer, 2005).”
From The benefit of generating errors during learning (2014) · cites this paper as Butler & Roediger, 2008
Practice testing with feedback is more effective than testing alone.
“Practice testing with feedback has been shown to be more effective than testing alone (Butler & Roediger, 2008).”
From Predicting as a learning strategy (2021) · cites this paper as Butler & Roediger, 2008
Feedback enhances the mnemonic benefits of testing by helping students correct errors and confirm correct answers.
“The mnemonic benefits of testing are further enhanced by feedback, which helps students to correct errors and confirm correct answers.”
From Test-enhanced learning in medical education (2008) · cites this paper as [27]
Delayed feedback produces superior retention relative to immediate feedback, at least in some cases.
“recent research shows that, at least in some cases, delayed feedback produces superior retention relative to immediate feedback.27,31”
From Test-enhanced learning in medical education (2008) · cites this paper as [27]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Psychology of Learning and Motivation: Advances in Research and Theory
(2010) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/c2009-0-62209-1 · Cited by 6,035 · see citing works
Journal on Educational Psychology
Winston, J., Mandernach, J., Winsor, D. L., Sinnott, J. D., & Gujjar, A. A. (2014) · Cited by 2,947 · see citing works
Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention
Roediger, H. L., & Karpicke, J. D. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01693.x · Cited by 2,867 · see citing works
Self-citationThe power of testing memory: Basic research and implications for educational practice
Roediger, H. L., & Karpicke, J. D. (2006) · Perspectives on Psychological Science · doi:10.1111/j.1745-6916.2006.00012.x · Cited by 2,037 · see citing works
Self-citationDistributed practice in verbal recall tasks: A review and quantitative synthesis.
Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006) · Psychological Bulletin · doi:10.1037/0033-2909.132.3.354 · Cited by 1,888 · see citing works
Teaching Tips: Strategies, Research, and Theory for College and University Teachers
Kleinsasser, R. C., & McKeachie, W. J. (1994) · Modern Language Journal · doi:10.2307/328598 · Cited by 1,592 · see citing works
Semantic integration of verbal information into a visual memory
Loftus, E. F., Miller, D. G., & Burns, H. J. (1978) · Journal of Experimental Psychology: Human Learning and Memory · doi:10.1037/0278-7393.4.1.19 · Cited by 1,511 · see citing works
The instructional effect of feedback in test-like events.
Bangert-Drowns, R. L., Kulik, C.-L. C., Kulik, J. A., & Morgan, M. (1991) · Review of Educational Research · doi:10.3102/00346543061002213 · Cited by 1,313 · see citing works
McKeachie's Teaching Tips: Strategies, Research, and Theory for College and University Teachers
McKeachie, W. J., Svinivki, M., & Nicol, D. (2010) · Cited by 1,227 · see citing works
The Science of Learning and the Art of Teaching.
Skinner, B. F. (2007) · Appleton-Century-Crofts eBooks · doi:10.1037/11324-010 · Cited by 1,140 · see citing works
An Extension of Box's Results on the Use of the $F$ Distribution in Multivariate Analysis
Geisser, S., & Greenhouse, S. W. (1958) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177706545 · Cited by 1,098 · see citing works
Monitoring and control processes in the strategic regulation of memory accuracy.
Koriat, A., & Goldsmith, M. (1996) · Psychological Review · doi:10.1037/0033-295x.103.3.490 · Cited by 1,062 · see citing works
(1965) · AV communication review · doi:10.1007/bf02770787 · Cited by 939 · see citing works
Timing of feedback and verbal learning.
Kulik, J. A., & Kulik, C.-L. C. (1988) · Review of Educational Research · doi:10.3102/00346543058001079 · Cited by 703 · see citing works
The influence of retrieval on retention
Carrier, L. M., & Pashler, H. (1992) · Memory & Cognition · doi:10.3758/bf03202713 · Cited by 580 · see citing works
Feedback in written instruction: The place of response certitude
Kulhavy, R. W., & Stock, W. (1989) · Educational Psychology Review · doi:10.1007/bf01320096 · Cited by 579 · see citing works
Feedback in Written Instruction
Kulhavy, R. W. (1977) · Review of Educational Research · doi:10.3102/00346543047002211 · Cited by 571 · cited 1× · see citing works
“…have contended that feedback is functionally different from reinforcement and that delayed feedback is more effective because it gives errors a chance to dissipate, making the process of learning the correct response easier (e.g., Kulhavy , 1977; Kulhavy & Anderson, 1972; Kulhavy & Stock, 1989).”
BackgroundTesting the testing effect in the classroom
McDaniel, M. A., Anderson, J. L., Derbish, M. H., & Morrisette, N. (2007) · European Journal of Cognitive Psychology · doi:10.1080/09541440701326154 · Cited by 563 · see citing works
The real test bias: Influences of testing on teaching and learning.
Frederiksen, N. (1984) · American Psychologist · doi:10.1037/0003-066x.39.3.193 · Cited by 536 · see citing works
Forgetting and the law of disuse.
McGeoch, J. A. (1932) · Psychological Review · doi:10.1037/h0069819 · Cited by 494 · see citing works
When Does Feedback Facilitate Learning of Words?
Pashler, H., Cepeda, N. J., Wixted, J. T., & Rohrer, D. (2005) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.31.1.3 · Cited by 445 · see citing works
An Alternative to Null-Hypothesis Significance Tests
Killeen, P. R. (2005) · Psychological Science · doi:10.1111/j.0956-7976.2005.01538.x · Cited by 443 · see citing works
Testing improves long-term retention in a simulated classroom setting
Butler, A. C., & Roediger, H. L. (2007) · European Journal of Cognitive Psychology · doi:10.1080/09541440701326097 · Cited by 434 · see citing works
Self-citationThe effects of presentation and recall of material in free-recall learning
Tulving, E. (1967) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(67)80092-6 · Cited by 408 · see citing works
The "testing" phenomenon: Not gone but nearly forgotten.
Glover, J. A. (1989) · Journal of Educational Psychology · doi:10.1037/0022-0663.81.3.392 · Cited by 392 · see citing works
The Positive and Negative Consequences of Multiple-Choice Testing.
Roediger, H. L., III, & Marsh, E. J. (2005) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.31.5.1155 · Cited by 386 · see citing works
Examining the testing effect with open‐ and closed‐book tests
Agarwal, P. K., Karpicke, J. D., Kang, S. H. K., Roediger, H. L., III, & McDermott, K. B. (2007) · Applied Cognitive Psychology · doi:10.1002/acp.1391 · Cited by 343 · see citing works
Correcting a metacognitive error: Feedback increases retention of low-confidence correct responses.
Butler, A. C., Karpicke, J. D., & Roediger, H. L., III (2008) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.34.4.918 · Cited by 332 · see citing works
Self-citationThe effect of type and timing of feedback on learning from multiple-choice tests.
Butler, A. C., Karpicke, J. D., & Roediger, H. L. (2007) · Journal of Experimental Psychology: Applied · doi:10.1037/1076-898x.13.4.273 · Cited by 321 · see citing works
Self-citationEffects of Frequent Classroom Testing
Bangert‐Drowns, R. L., Kulik, J. A., & Kulik, C. C. (1991) · The Journal of Educational Research · doi:10.1080/00220671.1991.10702818 · Cited by 320 · see citing works
Koriat, A., & Goldsmith, M. (1994) · Journal of Experimental Psychology General · doi:10.1037//0096-3445.123.3.297 · Cited by 307 · see citing works
Marsh, E. J. (2003) · Journal of Memory and Language · doi:10.1016/s0749-596x(03)00092-5 · Cited by 276 · see citing works
Disparate Effects of Repeated Testing: Reconciling Ballard's (1913) and Bartlett's (1932) Results
Wheeler, M. A., & Roediger, H. L. (1992) · Psychological Science · doi:10.1111/j.1467-9280.1992.tb00036.x · Cited by 254 · see citing works
Self-citationSpacing effects and their implications for theory and practice
Dempster, F. N. (1989) · Educational Psychology Review · doi:10.1007/bf01320097 · Cited by 244 · see citing works
Koriat, A., & Goldsmith, M. (1994) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.123.3.297 · Cited by 235 · see citing works
Altering memory representations through retrieval.
McDaniel, M. A., & Masson, M. E. J. (1985) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.11.2.371 · Cited by 229 · see citing works
Errors committed with high confidence are hypercorrected.
Butterfield, B., & Metcalfe, J. (2001) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.27.6.1491 · Cited by 223 · see citing works
Delay-retention effect with multiple-choice tests.
Kulhavy, R. W., & Anderson, R. C. (1972) · Journal of Educational Psychology · doi:10.1037/h0033243 · Cited by 209 · see citing works
The memorial consequences of multiple-choice testing
Marsh, E. J., Roediger, H. L., III, Bjork, R. A., & Bjork, E. L. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194051 · Cited by 161 · see citing works
Misinformation Effects in Recall: Creating False Memories through Repeated Retrieval
Roediger, H. L., III, Jacoby, J. D., & McDermott, K. B. (1996) · Journal of Memory and Language · doi:10.1006/jmla.1996.0017 · Cited by 157 · see citing works
Tests and test feedback as learning sources
McDaniel, M. A., & Fisher, R. P. (1991) · Contemporary Educational Psychology · doi:10.1016/0361-476x(91)90037-l · Cited by 138 · see citing works
Roediger, H. L., III, & Payne, D. G. (1985) · Memory & Cognition · doi:10.3758/bf03198437 · Cited by 131 · see citing works
Effects of delay of information feedback and task complexity on the identification of concepts.
Bourne, L. E. (1957) · Journal of Experimental Psychology · doi:10.1037/h0044684 · Cited by 83 · see citing works
Comparison of several feedback methods for correcting errors by computer-assisted instruction.
Gilman, D. A. (1969) · Journal of Educational Psychology · doi:10.1037/h0028501 · Cited by 79 · see citing works
When additional multiple-choice lures aid versus hinder later memory
Butler, A. C., Marsh, E. J., Goode, M. K., & Roediger, H. L., III (2006) · Applied Cognitive Psychology · doi:10.1002/acp.1239 · Cited by 79 · see citing works
Self-citationDelay-retention effect in natural classroom settings.
Surber, J. R., & Anderson, R. C. (1975) · Journal of Educational Psychology · doi:10.1037/h0077003 · Cited by 78 · see citing works
Remembering, knowing, and reconstructing the past
Roediger, H., Wheeler, M., & Rajaram, S. (1993) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/s0079-7421(08)60295-9 · Cited by 69 · see citing works
Self-citationSome deleterious consequences of the act of recollection
Schooler, J. W., Foster, R. A., & Loftus, E. F. (1988) · Memory & Cognition · doi:10.3758/bf03197757 · Cited by 65 · see citing works
Delay-improved retention of a difficult task.
Brackbill, Y., Bravos, A., & Starr, R. H. (1962) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0041561 · Cited by 64 · see citing works
Using tests as learning opportunities.
Foos, P. W., & Fisher, R. P. (1988) · Journal of Educational Psychology · doi:10.1037/0022-0663.80.2.179 · Cited by 64 · cited 1× · see citing works
“Because of the mnemonic benefit conferred by retrieval, many researchers have argued that tests should be used as learning tools in the classroom (e.g., Bangert-Drowns, Kulik, & Kulik, 1991; Foos & Fisher, 1988; Glover, 1989; Jones, 1923–1924; Roediger & Karpicke, 2006b).”
BackgroundEncountering misspellings and spelling performance: Why wrong isn't right.
Brown, A. S. (1988) · Journal of Educational Psychology · doi:10.1037/0022-0663.80.4.488 · Cited by 60 · see citing works
FEEDBACK PROCEDURES IN COMPUTER‐ASSISTED ARITHMETIC INSTRUCTION
Tait, K., Hartley, J. R., & Anderson, R. C. (1973) · British Journal of Educational Psychology · doi:10.1111/j.2044-8279.1973.tb00752.x · Cited by 58 · see citing works
Jacoby, L. L., & Hollingshead, A. (1990) · Canadian Journal of Psychology/Revue Canadienne de Psychologie · doi:10.1037/h0084259 · Cited by 55 · see citing works
The Effect of Immediate Knowledge of Quiz Results on Final Examination Scores in Freshman Chemistry
Angell, G. W. (1949) · The Journal of Educational Research · doi:10.1080/00220671.1949.10881704 · Cited by 55 · see citing works
Verbal retention as a function of the informativeness and delay of informative feedback.
Sturges, P. T. (1969) · Journal of Educational Psychology · doi:10.1037/h0026638 · Cited by 51 · see citing works
Testing Effects Measured with Alternate Test Forms
Duchastel, P. C., & Nungester, R. J. (1982) · The Journal of Educational Research · doi:10.1080/00220671.1982.10885400 · Cited by 47 · see citing works
The negative suggestion effect: Pondering incorrect alternatives may be hazardous to your knowledge.
Brown, A. S., Schilling, H. E. H., & Hockensmith, M. L. (1999) · Journal of Educational Psychology · doi:10.1037/0022-0663.91.4.756 · Cited by 42 · see citing works
Generality of the Negative Suggestion Effect in Objective Tests
Toppino, T. C., & Luipersbeck, S. M. (1993) · The Journal of Educational Research · doi:10.1080/00220671.1993.9941229 · Cited by 38 · see citing works
Learning from tests: Facilitation of delayed recall by initial recognition alternatives.
Whitten, W. B., & Leonard, J. M. (1980) · Journal of Experimental Psychology Human Learning & Memory · doi:10.1037/0278-7393.6.2.127 · Cited by 29 · see citing works
Effects of Systematic Variations in Reinforcement Contingencies on Learner Performance
Sullivan, H. J., Schutz, R. E., & Robert, L. B. (1971) · American Educational Research Journal · doi:10.3102/00028312008001135 · Cited by 23 · see citing works
Paige, D. D. (1966) · The Journal of Educational Research · doi:10.1080/00220671.1966.10883355 · Cited by 17 · see citing works
Does guessing on a multiple-choice test affect later cued recall?
Huelser, B. J., & Marsh, E. J. (2006) · PsycEXTRA Dataset · doi:10.1037/e527352012-767 · Cited by 5 · see citing works
Effects of Informative Feedback and Active Training upon Concept Attainment
Schroth, M. L. (1977) · Psychological Reports · doi:10.2466/pr0.1977.40.2.647 · Cited by 4 · see citing works
Differential Effects of Immediate and Delayed Feedback on Test Performance
McConnell, M. D., Hunt, R. R., & Smith, R. E. (2006) · PsycEXTRA Dataset · doi:10.1037/e586172007-001 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Loftus, E. F., Miller, D. G., & Burns, H. J. (1978). Semantic integra- tion of verbal material into a visual memory. Journal of Experimental Psychology: Human Learning & Memory, 4, 19-31.
- Brown, A. S., Schilling, H. E. H., & Hockensmith, M. L. (1999). inforcement contingencies. American Educational Research Journal, 8, 135-141.
Cited by
Sorted by citations
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013) · Psychological Science in the Public Interest · doi:10.1177/1529100612453266 · Cited by 3,138 · see citing works
The critical role of retrieval practice in long-term retention
Roediger, H. L., & Butler, A. C. (2011) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.09.003 · Cited by 1,796 · see citing works
Self-citationTest-enhanced learning in medical education
Larsen, D. P., Butler, A. C., & Roediger III, H. L. (2008) · Medical Education · doi:10.1111/j.1365-2923.2008.03124.x · Cited by 610 · see citing works
Self-citationRethinking the Use of Tests: A Meta-Analysis of Practice Testing
Adesope, O. O., Trevisan, D. A., & Sundararajan, N. (2017) · Review of Educational Research · doi:10.3102/0034654316689306 · Cited by 544 · see citing works
Repeated testing produces superior transfer of learning relative to repeated studying.
Butler, A. C. (2010) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/a0019902 · Cited by 492 · see citing works
Self-citationMcDaniel, M. A., Agarwal, P. K., Huelser, B. J., McDermott, K. B., & Roediger, H. L. (2011) · Journal of Educational Psychology · doi:10.1037/a0021782 · Cited by 360 · see citing works
Retrieval-Based Learning: A Decade of Progress
Karpicke, J. D. (2017) · Learning and Memory: A Comprehensive Reference · doi:10.1016/b978-0-12-809324-5.21055-9 · Cited by 223 · see citing works
Quizzing in Middle‐School Science: Successful Transfer Performance on Classroom Exams
McDaniel, M. A., Thomas, R. C., Agarwal, P. K., McDermott, K. B., & Roediger, H. L. (2013) · Applied Cognitive Psychology · doi:10.1002/acp.2914 · Cited by 171 · see citing works
The benefit of generating errors during learning
Potts, R., & Shanks, D. R. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033194 · Cited by 166 · see citing works
Attali, Y., & van der Kleij, F. (2017) · Computers & Education · doi:10.1016/j.compedu.2017.03.012 · Cited by 119 · see citing works
Kornell, N. (2014) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/a0033699 · Cited by 88 · see citing works
Does response mode affect amount recalled or the magnitude of the testing effect?
Putnam, A. L., & Roediger, H. L. (2013) · Memory & Cognition · doi:10.3758/s13421-012-0245-x · Cited by 86 · see citing works
Predicting as a learning strategy
Brod, G. (2021) · Psychonomic Bulletin & Review · doi:10.3758/s13423-021-01904-1 · Cited by 52 · see citing works
Ebersbach, M., Feierabend, M., & Nazari, K. B. B. (2020) · Applied Cognitive Psychology · doi:10.1002/acp.3639 · Cited by 45 · see citing works
Testing enhances learning: A review of the literature
Binks, S. (2018) · Journal of Professional Nursing · doi:10.1016/j.profnurs.2017.08.008 · Cited by 38 · see citing works
Mathematics practice without feedback: A desirable difficulty in a classroom setting
Fyfe, E. R., & Rittle-Johnson, B. (2017) · Instructional Science · doi:10.1007/s11251-016-9401-1 · Cited by 37 · see citing works
Retrieval practice and spacing effects in multi-session treatment of naming impairment in aphasia
Middleton, E. L., Rawson, K. A., & Verkuilen, J. (2019) · Cortex · doi:10.1016/j.cortex.2019.07.003 · Cited by 29 · see citing works
From Obstacle to Catalyst: A Narrative Review of Learning from Errors in Science Education
Fabre-Mitjans, N., Jiménez-Valverde, G., Viciana, S., & Crespo i Torres, F. (2026) · Encyclopedia · doi:10.3390/encyclopedia6090192
- Authors
- Butler, A. C., & Roediger, H. L.
- Affiliations
- Washington University in St. Louis (US)
- Corresponding author
- Butler, Andrew C.
- ORCID
- Butler, Andrew C.
- Venue
- Memory & Cognition
- Published
- 2008-04
- Volume
- 36
- Issue
- 3
- Pages
- 604-616
- ISSN
- 0090-502X (ISSN-L)
- 1532-5946
- OpenAlex
- W2143805697
- Cited by
- 457
- References
- 64
- Citations by year
- 2026: 25 · 2025: 29 · 2024: 23 · 2023: 24 · 2022: 27 · 2021: 39 · 2020: 31 · 2019: 31 · 2018: 29 · 2017: 28 · 2016: 18 · 2015: 30 · 2014: 34 · 2013: 22 · 2012: 23
- FWCI
- 7.41
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Memory & Cognition (publishedVersion)
- PubMed (publishedVersion)
- Funders
- James S. McDonnell Foundation
- Institute of Education Sciences (R305H030339)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Social Sciences › Education › Student Assessment and Feedback
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
- Keywords
- multiple-choice tests, feedback, testing effect, false knowledge, misinformation, retention, delayed feedback, immediate feedback, cued recall, higher education assessment
- MeSH terms
- Choice Behavior, Cues, Feedback, Humans, Learning, Psychological Tests, Mental Recall, Retention, Psychology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
