Testing After Worked Example Study Does Not Enhance Delayed Problem-Solving Performance Compared to Restudy
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
Four experiments investigated whether the testing effect also applies to the acquisition of problem-solving skills from worked examples. Experiment 1 ( n = 120) showed no beneficial effects of testing consisting of isomorphic problem solving or example recall on final test performance, which consisted of isomorphic problem solving, compared to continued study of isomorphic examples. Experiment 2 ( n = 124) showed no beneficial effects of testing consisting of identical problem solving compared to restudying an identical example. Interestingly, participants who took both an immediate and a delayed final test outperformed those taking only a delayed test. This finding suggested that testing might become beneficial for retention but only after a certain level of schema acquisition has taken place through restudying several examples. However, experiment 2 had no control condition restudying examples instead of taking the immediate test. Experiment 3 ( n = 129) included such a restudy condition, and there was no evidence that testing after studying four examples was more effective for final delayed test performance than restudying, regardless of whether restudied/tested problems were isomorphic or identical. Experiment 4 ( n = 75) used a similar design as experiment 3 (i.e., testing/restudy after four examples), but with examples on a different topic and with a different participant population. Again, no evidence of a testing effect was found. Thus, across four experiments, with different types of initial tests, different problem-solving domains, and different participant populations, we found no evidence that testing enhanced delayed test performance compared to restudy. These findings suggest that the testing effect might not apply to acquiring problem-solving skills from worked examples.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cognitive Architecture and Instructional Design
Sweller, J., van Merrienboer, J. J. G., & Paas, F. G. W. C. (1998) · Educational Psychology Review · doi:10.1023/a:1022193728205 · Cited by 6,023 · cited 2×
“Research has shown that for novices, who have little if any prior knowledge of a task, studying worked examples (often alternated with problem-solving practice) is more effective and more efficient for learning than mere problem-solving practice (i.e., the Bworked example effect^; for reviews, see Atkinson et al. 2000…”
BackgroundTest-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention
Roediger, H. L., & Karpicke, J. D. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01693.x · Cited by 2,867
Paas, F. G. W. C. (1992) · Journal of Educational Psychology · doi:10.1037/0022-0663.84.4.429 · Cited by 2,150
The power of testing memory: Basic research and implications for educational practice
Roediger, H. L., & Karpicke, J. D. (2006) · Perspectives on Psychological Science · doi:10.1111/j.1745-6916.2006.00012.x · Cited by 2,037
Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_4 · Cited by 1,927
The Use of Worked Examples as a Substitute for Problem Solving in Learning Algebra
Sweller, J., & Cooper, G. A. (1985) · Cognition and Instruction · doi:10.1207/s1532690xci0201_3 · Cited by 1,214
Paas, F. G. W. C., & Van Merriënboer, J. J. G. (1994) · Journal of Educational Psychology · doi:10.1037/0022-0663.86.1.122 · Cited by 1,198
Memory and Metamemory Considerations in the Training of Human Beings
Bjork, R. A. (1994) · Metacognition · doi:10.7551/mitpress/4561.003.0011 · Cited by 1,084
Expertise Reversal Effect and Its Implications for Learner-Tailored Instruction
Kalyuga, S. (2007) · Educational Psychology Review · doi:10.1007/s10648-007-9054-3 · Cited by 1,056 · cited 1×
“That is, it is known that example study loses its effectiveness for learning once a certain level of knowledge is reached (Kalyuga et al. 2001; see also Kalyuga 2007; Kalyuga et al. 2003).”
BackgroundEffects of schema acquisition and rule automation on mathematical problem-solving transfer.
Cooper, G., & Sweller, J. (1987) · Journal of Educational Psychology · doi:10.1037/0022-0663.79.4.347 · Cited by 688 · cited 1×
“…practice with example–problem pairs, in which a worked example is immediately followed by an isomorphic problem to solve (e.g., Carroll 1994; Cooper and Sweller 1987; Kalyuga et al. 2001; Sweller and Cooper 1985), other studies compared problem-solving practice to example study only (e.g.,…”
ResultExploring the Assistance Dilemma in Experiments with Cognitive Tutors
Koedinger, K. R., & Aleven, V. (2007) · Educational Psychology Review · doi:10.1007/s10648-007-9049-0 · Cited by 632 · cited 1×
“This is a stronger control condition than conventional, untutored problem solving because it provides more instructional guidance (i.e., scaffolding, feedback; Koedinger and Aleven 2007) to learners.”
BackgroundWhen problem solving is superior to studying worked examples.
Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001) · Journal of Educational Psychology · doi:10.1037/0022-0663.93.3.579 · Cited by 548 · cited 2×
“That is, it is known that example study loses its effectiveness for learning once a certain level of knowledge is reached (Kalyuga et al. 2001; see also Kalyuga 2007; Kalyuga et al. 2003).”
BackgroundResultRenkl, A., & Atkinson, R. K. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_3 · Cited by 540 · cited 1×
“Asking learners to complete solution steps (Paas 1992) or increasingly more solution steps (i.e., a fading strategy; see Renkl and Atkinson 2003) could be regarded as a form of testing at a part-task level and is known to be an effective strategy for acquiring problem-solving skills from worked…”
BackgroundEffects of pairs of problems and examples on task performance and different types of cognitive load
Leppink, J., Paas, F., van Gog, T., van der Vleuten, C. P. M., & van Merriënboer, J. J. G. (2014) · Learning and Instruction · doi:10.1016/j.learninstruc.2013.12.001 · Cited by 520 · cited 2×
“…be worthwhile to investigate this motivational component in future research, because seen solely from the perspective of learning outcomes, the results from this study as well as others (Leppink et al. 2014; Van Gog et al. 2011) suggest that there is no need to alternate examples and problems.”
BackgroundResultTest format and corrective feedback modify the effect of testing on long-term retention
Kang, S. H. K., McDermott, K. B., & Roediger, H. L. (2007) · European Journal of Cognitive Psychology · doi:10.1080/09541440601056620 · Cited by 513
Toward an Instructionally Oriented Theory of Example‐Based Learning
Renkl, A. (2014) · Cognitive Science · doi:10.1111/cogs.12086 · Cited by 509
Instructional Efficiency: Revisiting the Original Construct in Educational Research
van Gog, T., & Paas, F. (2008) · Educational Psychologist · doi:10.1080/00461520701756248 · Cited by 473 · cited 1×
“This suggests that in terms of instructional efficiency, (re)studying examples is more efficient than testing, in the sense that the same level of final test performance was attained with less investment of mental effort during acquisition (Hoffman and Schraw 2010; Van Gog and Paas 2008).”
ResultFeedback enhances the positive effects and reduces the negative effects of multiple-choice testing
Butler, A. C., & Roediger, H. L. (2008) · Memory & Cognition · doi:10.3758/mc.36.3.604 · Cited by 457 · cited 1×
“…effect on memory in the sense that incorrect answers selected on a multiple choice practice test may persist and be given as answers on a final cued recall test (e.g., Roediger and Marsh 2005), unless feedback is provided after a multiple-choice practice test (e.g., Butler and Roediger 2008).”
BackgroundTesting improves long-term retention in a simulated classroom setting
Butler, A. C., & Roediger, H. L. (2007) · European Journal of Cognitive Psychology · doi:10.1080/09541440701326097 · Cited by 434
Learning from Worked-Out Examples: The Effects of Example Variability and Elicited Self-Explanations
Renkl, A., Stark, R., Gruber, H., & Mandl, H. (1998) · Contemporary Educational Psychology · doi:10.1006/ceps.1997.0959 · Cited by 374
Generalizing test-enhanced learning from the laboratory to the classroom
Mcdaniel, M. A., Roediger, H. L., & Mcdermott, K. B. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194052 · Cited by 326 · cited 2×
“Given the relevance of the testing effect for education (see, e.g., Agarwal et al. 2012; Karpicke and Grimaldi 2012;McDaniel et al. 2007; Roediger and Karpicke 2006a), it is important to establish whether the testing effect would also apply to the acquisition of problem-solving skills from worked…”
BackgroundEffects of worked examples, example-problem, and problem-example pairs on novices’ learning
van Gog, T., Kester, L., & Paas, F. (2011) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2010.10.004 · Cited by 283 · cited 9×
“…1 and 2, mental effort investment—although it seemed somewhat lower in the restudy (examples only) condition—did not differ significantly between the restudy and testing (example–problem pair) conditions (a finding that replicates effort data from these conditions in Van Gog et al. 2011).”
BackgroundMethodResultSelf-citationThe effects of tests on learning and forgetting.
Carpenter, S. K., Pashler, H., Wixted, J. T., & Vul, E. (2008) · Memory & Cognition · doi:10.3758/mc.36.2.438 · Cited by 281 · cited 3×
“For instance, in many studies on the testing effect, final test moment (immediate vs. delayed) is a between-subjects (e.g., Coppens et al. 2011) or between-items (e.g., Carpenter et al. 2008) factor instead of a repeated measures within-subjects factor as in the Van Gog and Kester (2012) study.”
BackgroundUsing worked examples as an instructional support in the algebra classroom.
Carroll, W. M. (1994) · Journal of Educational Psychology · doi:10.1037/0022-0663.86.3.360 · Cited by 209 · cited 1×
“…problem-solving practice with example–problem pairs, in which a worked example is immediately followed by an isomorphic problem to solve (e.g., Carroll 1994; Cooper and Sweller 1987; Kalyuga et al. 2001; Sweller and Cooper 1985), other studies compared problem-solving practice to…”
ResultCognitive load theory and aging: effects of worked examples on training efficiency
Van Gerven, P. W. M., Paas, F. G. W. C., Van Merriënboer, J. J. G., & Schmidt, H. G. (2002) · Learning and Instruction · doi:10.1016/s0959-4752(01)00017-2 · Cited by 205 · cited 2×
“…is immediately followed by an isomorphic problem to solve (e.g., Carroll 1994; Cooper and Sweller 1987; Kalyuga et al. 2001; Sweller and Cooper 1985), other studies compared problem-solving practice to example study only (e.g., Nievelstein et al. 2013; Van Gerven et al. 2002; Van Gog et al. 2006).”
ResultTesting beyond words: using tests to enhance visuospatial map learning
Carpenter, S. K., & Pashler, H. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194092 · Cited by 133
Accounting for Beneficial Effects of Worked Examples in Tutored Problem Solving
Salden, R. J. C. M., Koedinger, K. R., Renkl, A., Aleven, V., & McLaren, B. M. (2010) · Educational Psychology Review · doi:10.1007/s10648-010-9143-6 · Cited by 96 · cited 1×
“…et al. 2008), worked examples vs. tutored problems (McLaren et al. 2014), and faded examples with increasingly more steps for the learner to complete with tutor support vs. tutored problem solving only (Schwonke et al. 2009; for a review of examples in tutoring systems, see Salden et al. 2010).”
BackgroundLeahy, W., Hanham, J., & Sweller, J. (2015) · Educational Psychology Review · doi:10.1007/s10648-015-9296-4 · Cited by 77 · cited 1×
“To conclude, the four experiments from this study along with the study by Van Gog and Kester (2012) and Leahy et al. (2015) suggest that the testing effect may not apply to the acquisition of complex problem-solving skills from worked examples.”
ResultEffects of testing on learning of functions
Kang, S. H. K., McDaniel, M. A., & Pashler, H. (2011) · Psychonomic Bulletin & Review · doi:10.3758/s13423-011-0113-x · Cited by 57 · cited 1×
“2011), videotaped lectures (Butler and Roediger 2007), visuospatial materials such as maps (Carpenter and Pashler 2007), numerical materials such as functions (Kang et al. 2011), and multimedia materials such as animations (Johnson and Mayer 2009).”
BackgroundExploring the Assistance Dilemma: Comparing Instructional Support in Examples and Problems
McLaren, B. M., van Gog, T., Ganoe, C., Yaron, D., & Karabinos, M. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-07221-0_44 · Cited by 34 · cited 2×
“A recent study suggests that feedback after problem solving in the form of worked examples might only increase learning time but not learning outcomes compared to studying examples only (McLaren et al. 2014).”
Background
Cited by
Eight Ways to Promote Generative Learning
Fiorella, L., & Mayer, R. E. (2016) · Educational Psychology Review · doi:10.1007/s10648-015-9348-9 · Cited by 832
Transfer of test- enhanced learning: meta-analytic review and synthesis
Pan, S. C., & Rickard, T. C. (2018) · Psychological Bulletin · doi:10.1037/bul0000151 · Cited by 286
Retrieval-Based Learning: A Decade of Progress
Karpicke, J. D. (2017) · Learning and Memory: A Comprehensive Reference · doi:10.1016/b978-0-12-809324-5.21055-9 · Cited by 223
“Second, none of the three studies with worked example materials (Leahy et al., 2015; van Gog and Kester, 2012; van Gog et al., 2015) manipulated the complexity of materials.”
Backgroundvan Gog, T., & Sweller, J. (2015) · Educational Psychology Review · doi:10.1007/s10648-015-9310-x · Cited by 164
“…of the studies presented in this issue may have been different if feedback or restudy opportunities after testing had been provided (although van Gog et al. 2015 did increase the level of success on the initial practice test through a longer study phase in experiments 3 and 4, yet still…”
InfluentialCognitive Load Theory, Resource Depletion and the Delayed Testing Effect
Leahy, W., & Sweller, J. (2019) · Educational Psychology Review · doi:10.1007/s10648-019-09476-2 · Cited by 59
Metadata
- Authors
- van Gog, T., Kester, L., Dirkx, K., Hoogerheide, V., Boerboom, J., & Verkoeijen, P. P. J. L.
- Venue
- Educational Psychology Review
- Published
- 2015-02-24
- Volume
- 27
- Issue
- 2
- Pages
- 265-289
- ISSN
- 1040-726X (ISSN-L)
- 1573-336X
- OpenAlex
- W2082988750
- Cited by
- 66
- References
- 30
- FWCI
- 4.79
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.