The DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index.
Effects of Feedback in a Computer-Based Learning Environment on Students’ Learning Outcomes: A Meta-Analysis
Abstract
In this meta-analysis, we investigated the effects of methods for providing item-based feedback in a computer-based environment on students’ learning outcomes. From 40 studies, 70 effect sizes were computed, which ranged from −0.78 to 2.29. A mixed model was used for the data analysis. The results show that elaborated feedback (EF; e.g., providing an explanation) produced larger effect sizes (0.49) than feedback regarding the correctness of the answer (KR; 0.05) or providing the correct answer (KCR; 0.32). EF was particularly more effective than KR and KCR for higher order learning outcomes. Effect sizes were positively affected by EF feedback, and larger effect sizes were found for mathematics compared with social sciences, science, and languages. Effect sizes were negatively affected by delayed feedback timing and by primary and high school. Although the results suggested that immediate feedback was more effective for lower order learning than delayed feedback and vice versa, no significant interaction was found.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“On the one hand, there is robust evidence that elaborate feedback with correct answers facilitates learning (Heckler and Mikula, 2016; Van der Kleij et al., 2015; Wisniewski et al., 2020).”
From Assessing the Impact and Underlying Pathways of Sequenced AI feedback on Student Learning (2026) · cites this paper as Van der Kleij et al., 2015
In the citing paper
“However, non-layered feedback can be effective without layering (e.g., (Heckler and Mikula, 2016; Van der Kleij et al., 2015; Wisniewski et al., 2020)).”
From Assessing the Impact and Underlying Pathways of Sequenced AI feedback on Student Learning (2026) · cites this paper as Van der Kleij et al., 2015
In the citing paper
“A particularly effective form of feedback provides the correct answer with an explanation of why it is the correct answer (Heckler and Mikula, 2016; Van der Kleij et al., 2015; Butler et al., 2013; Wisniewski et al., 2020).”
From Assessing the Impact and Underlying Pathways of Sequenced AI feedback on Student Learning (2026) · cites this paper as Van der Kleij et al., 2015
In the citing paper
“These conclusions are further supported by meta-analytic evidence: a synthesis of 40 studies comprising 70 effect sizes revealed that elaborated feedback (e.g., providing explanations) produced a substantially larger mean effect size than feedback limited to correctness judgments or feedback providing only the correct answer (Van der Kleij et al., 2015).”
From Assessing the Impact and Underlying Pathways of Sequenced AI feedback on Student Learning (2026) · cites this paper as Van der Kleij et al., 2015
Citations
Cites
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905
(2000) · Cited by 9,198
Assessment and Classroom Learning
Black, P., & Wiliam, D. (1998) · Assessment in Education: Principles, Policy & Practice · doi:10.1080/0969595980050102 · Cited by 8,152
Visible learning: a synthesis of over 800 meta‐analyses relating to achievement
Carter, M. (2009) · Educational Psychology · doi:10.1080/01443410903415150 · Cited by 7,696
Kluger, A. N., & DeNisi, A. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.2.254 · Cited by 5,882
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695
Shute, V. J. (2008) · Review of Educational Research · doi:10.3102/0034654307313795 · Cited by 4,631
Formative assessment and the design of instructional systems
Sadler, D. R. (1989) · Instructional Science · doi:10.1007/bf00117714 · Cited by 4,285
Inside the Black Box: Raising Standards through Classroom Assessment
Black, P., & Wiliam, D. (2010) · Phi Delta Kappan · doi:10.1177/003172171009200119 · Cited by 4,231
The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring
BLOOM, B. S. (1984) · Educational Researcher · doi:10.3102/0013189x013006004 · Cited by 2,734 · cited 2×
“According to Bloom (1984), one-to-one tutoring is the most effective form of instruction.”
BackgroundA Fail-Safe N for Effect Size in Meta-Analysis
Orwin, R. G. (1983) · Journal of Educational Statistics · doi:10.2307/1164923 · Cited by 2,064 · cited 1×
“This issue was addressed with the fail-safe N method (Orwin, 1983), which estimates how many studies reporting nonsignificant effects sizes would be need to be combined with the studies in the meta-analysis in order to arrive at a specific result.”
MethodHandbook of research on educational communications and technology
(2004) · British Journal of Educational Technology · doi:10.1111/j.1467-8535.2004.00409_15.x · Cited by 1,854
Handbook of Research on Educational Communications and Technology
Jonassen, D. H., Spector, M. J., Driscoll, M., Merrill, M. D., van Merriënboer, J. J. G., & Driscoll, M. P. (2008) · Routledge eBooks · doi:10.4324/9780203880869 · Cited by 1,692
Formative assessment: a critical review
Bennett, R. E. (2011) · Assessment in Education: Principles, Policy & Practice · doi:10.1080/0969594x.2010.513678 · Cited by 1,498
Language learning and technology
Healey, D. (2016) · doi:10.4324/9781315657899.ch1 · Cited by 1,420
The instructional effect of feedback in test-like events.
Bangert-Drowns, R. L., Kulik, C.-L. C., Kulik, J. A., & Morgan, M. (1991) · Review of Educational Research · doi:10.3102/00346543061002213 · Cited by 1,313
What is assessment for learning
Wiliam, D. (2011) · Studies in Educational Evaluation · doi:10.1016/j.stueduc.2011.03.001 · Cited by 1,282
Estimation of effect size from a series of independent experiments.
Hedges, L. (1982) · Psychological Bulletin · doi:10.1037/0033-2909.92.2.490 · Cited by 1,030
Estimation of a Single Effect Size: Parametric and Nonparametric Methods
Hedges, L. V., & Olkin, I. (1985) · Statistical Methods for Meta-Analysis · doi:10.1016/b978-0-08-057065-5.50010-5 · Cited by 983
Handbook of Research on Learning and Instruction
Mayer, R. E., & Alexander, P. A. (2011) · Routledge eBooks · doi:10.4324/9780203839089 · Cited by 969
The Handbook of Research Synthesis
Ziegel, E. R., Cooper, H., & Hedges, L. V. (1995) · Technometrics · doi:10.2307/1269651 · Cited by 936
Wainer, H., Dorans, N. J., Flaugher, R. L., Green, B. F., & Mislevy, R. J. (2000) · Routledge eBooks · doi:10.4324/9781410605931 · Cited by 659
Moreno, R. (2004) · Instructional Science · doi:10.1023/b:truc.0000021811.66966.1d · Cited by 615
Feedback in written instruction: The place of response certitude
Kulhavy, R. W., & Stock, W. (1989) · Educational Psychology Review · doi:10.1007/bf01320096 · Cited by 579
Feedback in Written Instruction
Kulhavy, R. W. (1977) · Review of Educational Research · doi:10.3102/00346543047002211 · Cited by 571 · cited 1×
“With EF, however, the distinction between feedback (in terms of corrections) and instruction fades (Hattie & Timperley, 2007) because “the process itself takes on the forms of new instruction, rather than informing the student solely about correctness” (Kulhavy, 1977, p. 212).”
BackgroundStobart, G. (2008) · doi:10.4324/9780203930502 · Cited by 479
Formative Assessment: A Meta-Analysis and a Call for Research
Kingston, N. M., & Nash, B. (2011) · Educational Measurement Issues and Practice · doi:10.1111/j.1745-3992.2011.00220.x · Cited by 443
A Meta-Analysis of the Effects of Feedback in Computer-Based Instruction
Azevedo, R., & Bernard, R. M. (1995) · Journal of Educational Computing Research · doi:10.2190/9lmd-3u28-3a0g-ftqt · Cited by 308
Moreno, R., & Valdez, A. (2005) · Educational Technology Research and Development · doi:10.1007/bf02504796 · Cited by 287
International Encyclopaedia of Education
Osborn, M. (1994) · Cited by 269
Effects of feedback in a computer-based assessment for learning
der Kleij, F. M. V., Eggen, T. J. H. M., Timmers, C. F., & Veldkamp, B. P. (2011) · Computers & Education · doi:10.1016/j.compedu.2011.07.020 · Cited by 245
Self-citationSanz, C., & Morgan‐Short, K. (2004) · Language Learning · doi:10.1111/j.1467-9922.2004.00248.x · Cited by 244
Intelligent Computer Feedback for Second Language Instruction
NAGATA, N. (1993) · Modern Language Journal · doi:10.1111/j.1540-4781.1993.tb01980.x · Cited by 213
Effects of differential feedback on students’ examination performance.
Lipnevich, A. A., & Smith, J. K. (2009) · Journal of Experimental Psychology: Applied · doi:10.1037/a0017841 · Cited by 199
Improving self-regulation, learning strategy use, and achievement with metacognitive feedback
Lee, H. W., Lim, K. Y., & Grabowski, B. L. (2010) · Educational Technology Research and Development · doi:10.1007/s11423-010-9153-6 · Cited by 182
Narciss, S., & Huth, K. (2006) · Learning and Instruction · doi:10.1016/j.learninstruc.2006.07.003 · Cited by 162
Wauters, K., Desmet, P., & Noortgate, W. V. D. (2010) · Journal of Computer Assisted Learning · doi:10.1111/j.1365-2729.2010.00368.x · Cited by 137
Studies of the Effect of Formative Assessment on Student Achievement: So Much More is Needed
McMillan, J. H., Venable, J. C., & Varier, D. (2020) · doi:10.7275/tmwm-7792 · Cited by 129
Kopp, V., Stark, R., & Fischer, M. R. (2008) · Medical Education · doi:10.1111/j.1365-2923.2008.03122.x · Cited by 123
The Role of Classroom Assessment in Student Performance on TIMSS
Rodriguez, M. C. (2004) · Applied Measurement in Education · doi:10.1207/s15324818ame1701_1 · Cited by 119
Timmers, C. F., & Veldkamp, B. P. (2010) · Computers & Education · doi:10.1016/j.compedu.2010.11.007 · Cited by 113
Dynamic task selection: Effects of feedback and learner control on efficiency and motivation
Corbalan, G., Kester, L., & van Merriënboer, J. J. G. (2008) · Learning and Instruction · doi:10.1016/j.learninstruc.2008.07.002 · Cited by 109
Smits, M. H. S. B., Boon, J., Sluijsmans, D. M. A., & van Gog, T. (2008) · Interactive Learning Environments · doi:10.1080/10494820701365952 · Cited by 108
Computerized Task‐Based Exposure, Explicitness, Type of Feedback, and Spanish L2 Development
Rosa, E. M., & Leow, R. P. (2004) · Modern Language Journal · doi:10.1111/j.0026-7902.2004.00225.x · Cited by 105
Neri, A., Cucchiarini, C., & Strik, H. (2008) · ReCALL · doi:10.1017/s0958344008000724 · Cited by 103
Cameron, B. H., & Dwyer, F. M. (2005) · The Journal of Interactive Learning Research · Cited by 102
The Effects of Feedback and Incentives on Achievement in Computer-Based Instruction
Morrison, G. R., Ross, S. M., Jayasinghe, M. G., & Casey, J. P. (1995) · Contemporary Educational Psychology · doi:10.1006/ceps.1995.1002 · Cited by 92
Using Technology to Enhance Mathematical Reasoning: Effects of Feedback and Self-Regulation Learning
Kramarski, B., & Zeichner, O. (2001) · Educational Media International · doi:10.1080/09523980110041458 · Cited by 88
Nagata, N., & Swisher, M. V. (1995) · Foreign Language Annals · doi:10.1111/j.1944-9720.1995.tb00803.x · Cited by 85
Feedback Types in Programmed Instruction: A Systematic Review
Jaehnig, W., & Miller, M. L. (2007) · The Psychological Record · doi:10.1007/bf03395573 · Cited by 82
Meta‐Analytic Methodology and Inferences About the Efficacy of Formative Assessment
Briggs, D. C., Ruiz‐Primo, M. A., Furtak, E. M., Shepard, L. A., & Yin, Y. M. (2012) · Educational Measurement Issues and Practice · doi:10.1111/j.1745-3992.2012.00251.x · Cited by 81
Control of feedback in computer-assisted instruction
Pridemore, D., & Klein, J. D. (1991) · Educational Technology Research and Development · doi:10.1007/bf02296569 · Cited by 75
Control of Practice and Level of Feedback in Computer-Based Instruction
Pridemore, D. R., & Klein, J. D. (1995) · Contemporary Educational Psychology · doi:10.1006/ceps.1995.1030 · Cited by 74
Computer-based feedback in linear algebra: Effects on transfer performance and motivation
Corbalan, G., PAAS, F. G. W. C., & Cuypers, H. (2010) · Computers & Education · doi:10.1016/j.compedu.2010.03.002 · Cited by 73
Reading comprehension exercises online: The effects of feedback, proficiency and interaction
Murphy, P. (2007) · Language learning & technology · doi:10.64152/10125/44120 · Cited by 66
Clariana, R. B., & Lee, D. (2001) · Educational Technology Research and Development · doi:10.1007/bf02504913 · Cited by 64
Collins, M., Carnine, D. W., & Gersten, R. M. (1987) · Exceptional Children · doi:10.1177/001440298705400308 · Cited by 60
Bridging the Gap between Expert-Novice Differences
Ifenthaler, D. (2010) · Journal of Research on Technology in Education · doi:10.1080/15391523.2010.10782564 · Cited by 50
Levels of questioning and forms of feedback: instructional factoars in courseware design
MERRILL, J. E. (1987) · The Journal of Computer Based Instruction · Cited by 49
der Kleij, F. M. V., Timmers, C. F., & Eggen, T. J. H. M. (2011) · CADMO · doi:10.3280/cad2011-001004 · Cited by 46
Self-citationEffects of complex feedback on computer-assisted modular instruction
Gordijn, J., & Nijhof, W. (2002) · Comput. Educ. · doi:10.1016/s0360-1315(02)00025-8 · Cited by 40
The effects of immediate online feedback upon diagnostic performance
Papa, F. J., Aldrich, D. G., & Schumacker, R. E. (1999) · Academic Medicine · doi:10.1097/00001888-199910000-00027 · Cited by 28
Vispoel, W. P. (1998) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1998.tb00532.x · Cited by 27
The Role of Item Feedback in Self-Adapted Testing
Roos, L. L., Wise, S. L., & Plake, B. S. (1997) · Educational and Psychological Measurement · doi:10.1177/0013164497057001005 · Cited by 19
Computerized Adaptive Testing in Computer Assisted Learning?
Veldkamp, B. P., Matteucci, M., & Eggen, T. J. H. M. (2011) · Communications in computer and information science · doi:10.1007/978-3-642-20074-8_3 · Cited by 14
Self-citationThe effects of type of task and type of feedback on L2 development in CALL
Moreno, N. (2007) · Cited by 10
Gradient- and Full-Response Feedback in Computer-Assisted Instruction
Hall, K. A., Adams, M. J., & Js, T. (1968) · The Journal of Educational Research · doi:10.1080/00220671.1968.10883643 · Cited by 7
O'Neil, H., & Huang, T. (2008) · Cited by 7
Encouraging mindful feedback processing: computer-based instruction in descriptive statistics
Moreno, R., & Valdez, A. (2008) · Cited by 5
Miller, S. T. (2009) · QSpace (Queen's University Library) · Cited by 4
Epstein, J. I. (1997) · Cited by 3
Dwyer, F. M., & Lin, H. (2006) · Cited by 2
Dunlap, J. C., & Mazingo, D. E. (2006) · Cited by 2
Munyofu, M. (2008) · ProQuest LLC eBooks · Cited by 1
Xu, M. (2011) · Huskie Commons (Northern Illinois University) · Cited by 1
Research works printed but not linked: 15
- Black P., 1998, Phi Delta Kappan, 80, 139
- *Bowles M. A., 2005, Effects of verbalization condition and type of feedback on L2 development in a CALL task
- *Cameron B., 2005, Journal of Interactive Learning Research, 16, 243
- Cohen J., 1988, Statistical power analysis for the behavioural sciences
- 10.3102/10769986006002107 · DOI · Paper page
- *Kim J. Y. L., 1991, Journal of Computer-Based Instruction, 18, 14
- *Merril J., 1987, Journal of Computer-Based Instruction, 14, 18
- Narciss S., 2004, Instructional design for multimedia learning, 181
- Nyquist J. B. (2003). The benefits of reconstruing feedback as a larger system of formative assessment: A meta-analysis (Unpublished master’s thesis). Vanderbilt University, Nashville, TN.
- 10.3102/10769986008002157 · DOI · Paper page
- Resing W. C. M. (1990). Intelligentie en leerpotentieel. Een onderzoek naar het leerpotentieel van jonge leerlingen uit het basis-en speciaal onderwijs [Intelligence and learning potential: A study on the learning potential of young children in regular and special education] (Unpublished doctoral dissertation). Vrije Universiteit Amsterdam, Netherlands.
- Shadish W. R., 1994, The handbook of research synthesis, 261
- Smith P. L., 2005, Instructional design
- *Wager S. U. (1983). The effect of immediacy and type of informative feedback on retention in a computer-assisted task (Unpublished doctoral dissertation). The Florida State University, Tallahassee.
- White H. D., 1994, The handbook of research synthesis, 41
Cited by
Sorted by citations
Sung, Y.-T., Chang, K.-E., & Liu, T.-C. (2016) · Computers & Education · doi:10.1016/j.compedu.2015.11.008 · Cited by 1,574
The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback Research
Wisniewski, B., Zierer, K., & Hattie, J. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.03087 · Cited by 1,199
Hillmayr, D., Ziernwald, L., Reinhold, F., Hofer, S. I., & Reiss, K. M. (2020) · Computers & Education · doi:10.1016/j.compedu.2020.103897 · Cited by 574
Virtual Reality and Its Applications in Education: Survey
Kamińska, D., Sapiński, T., Wiak, S., Tikk, T., Haamer, R. E., Avots, E., Helmi, A., Ozcinar, C., & Anbarjafari, G. (2019) · Information · doi:10.3390/info10100318 · Cited by 506
Differentiated Instruction in Secondary Education: A Systematic Review of Research Evidence
Smale-Jacobse, A. E., Meijer, A., Helms‐Lorenz, M., & Maulana, R. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02366 · Cited by 560
“Review studies show that such adaptive ICT applications may considerably improve student achievement (Ma et al., 2014; Van der Kleij et al., 2015; Kulik and Fletcher, 2016; Shute and Rahimi, 2017).”
Background
Metadata
- Authors
- Van der Kleij, F. M., Feskens, R. C. W., & Eggen, T. J. H. M.
- Affiliations
- Cito Institute for Educational Measurement and University of Twente
- Cito Institute for Educational Measurement
- University of Twente (NL)
- Venue
- Review of Educational Research
- Published
- 2015-12
- Volume
- 85
- Issue
- 4
- Pages
- 475-511
- ISSN
- 0034-6543 (ISSN-L)
- 1935-1046
- OpenAlex
- W2110661236
- Cited by
- 714
- References
- 76
- Citations by year
- 2026: 91 · 2025: 96 · 2024: 89 · 2023: 73 · 2022: 71 · 2021: 74 · 2020: 64 · 2019: 61 · 2018: 39 · 2017: 31 · 2016: 19 · 2015: 4
- FWCI
- 155.08
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Review of Educational Research (publishedVersion)
- University of Twente Research Information (publishedVersion)
- University of Twente Research Information (submittedVersion)
- Topics
- Social Sciences › Education › Student Assessment and Feedback
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Keywords
- elaborated feedback, computer-assisted instruction, learning outcomes, feedback timing, immediate feedback, delayed feedback, higher-order learning, mathematics learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
