When Problem Solving Followed by Instruction Works: Evidence for Productive Failure
Open accessThe DOI matches its Crossref record; 79 of the 79 references it cites were found in a registry or an index; 8 of the 35 sources its claims cite were checked for support.
Abstract
When learning a new concept, should students engage in problem solving followed by instruction (PS-I) or instruction followed by problem solving (I-PS)? Noting that there is a passionate debate about the design of initial learning, we report evidence from a meta-analysis of 53 studies with 166 comparisons that compared PS-I with I-PS design. Our results showed a significant, moderate effect in favor of PS-I (Hedge’s g 0.36 [95% confidence interval 0.20; 0.51]). The effects were even stronger (Hedge’s g ranging between 0.37 and 0.58) when PS-I was implemented with high fidelity to the principles of Productive Failure (PF), a subset variant of PS-I design. Students’ grade level, intervention time span, and its (quasi-)experimental nature contributed to the efficacy of PS-I over I-PS designs. Contrasting trends were, however, observed for younger age learners (second to fifth graders) and for the learning of domain-general skills, for which effect sizes favored I-PS. Overall, an estimation of true effect sizes after accounting for publication bias suggested a strong effect size favoring PS-I (Hedge’s g 0.87).
Overview
CoChat Summary
TL;DR
This meta-analysis of 53 studies with 166 comparisons found that problem solving followed by instruction (PS-I) produced a significant, moderate advantage over instruction followed by problem solving (I-PS) for conceptual knowledge and transfer (Hedge’s g = 0.36). Effects were stronger when PS-I was implemented with high fidelity to Productive Failure principles, and after accounting for publication bias the estimated true effect was Hedge’s g = 0.87.
Key findings
- Overall, PS-I outperformed I-PS for conceptual knowledge and transfer: Hedge’s g = 0.36, 95% CI [0.20, 0.51], p < .0001. For procedural knowledge, the effect was nonsignificant: Hedge’s g = −0.03, 95% CI [−0.20, 0.15].
- Higher fidelity to Productive Failure (PF) design criteria was associated with larger effects favoring PS-I; the overall PF fidelity score significantly predicted effect size (β = 0.0065, p < .001, 95% CI [0.0044, 0.0087]). Effects ranged between Hedge’s g = 0.37 and 0.58 when PS-I was implemented with high PF fidelity.
- Subgroup differences: younger learners (2nd–5th graders) showed an effect favoring I-PS (Hedge’s g = −0.09, 95% CI [−0.92, −0.16]), and domain-general skills also favored I-PS (Hedge’s g = −0.17, 95% CI [−1.11, −0.02]). Quasi-experimental comparisons (g = 0.46) and interventions spanning a few days (g = 0.41) descriptively favored PS-I more than experimental comparisons and shorter interventions, though these subgroup differences were not significant.
- Publication-bias checks: funnel plot and Egger’s test were not significant (intercept = 0.63, p = .219), but p-curve analysis indicated evidential value and estimated a true effect size of Hedge’s g = 0.87 favoring PS-I; heterogeneity was moderate (I² = 42.01%).
Methods
- Meta-analysis of 53 studies comprising 166 experimental comparisons that compared PS-I with I-PS; inclusion required experimental or quasi-experimental comparison and assessment of conceptual knowledge or transfer, with procedural knowledge optional.
- Searches used Google Scholar citations to seminal PF articles and key follow-up PF articles; 45 articles met criteria after screening and duplicate removal.
- Pooled effect sizes used a three-level multilevel meta-analysis model (participants nested within studies, studies nested within articles), Hedge’s g with 95% CIs, Cochran’s Q, and I². Subgroup and meta-regression analyses examined seven PF fidelity criteria, student characteristics, and intervention characteristics; publication bias was assessed with funnel plots, Egger’s test, trim-and-fill, and p-curve analysis.
Limitations
- The paper states that nearly 75% of included comparisons targeted math and physics, with skewed subtopics; more studies are needed in other STEM and non-STEM domains and with postgraduates/professionals before generalizing.
- The paper states that individual differences were not available, few studies reported actual failure rates, multiple RSM generation is a weak proxy for failure, and only one study explicitly assessed negative knowledge. It also notes that PF fidelity criteria and other characteristics were dichotomized, and that experimental versus quasi-experimental designs may not be directly comparable because learning objectives can differ.
- The paper states that the identified list of influencing factors is important but not exhaustive, and that the current meta-analysis lacks studies involving students with learning disabilities or difficulties.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
7 Valid · 4 Not valid · 21 Not checked · 106 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“With a moderate ( < 50%) level of heterogeneity in included comparisons ( I 2 = 42.01%), we expect these estimates to be robust (van Aert etal., 2016).”
What the sources say
- Disputedcites van Aert et al., 2016 matched to “Conducting Meta-Analyses Based on p Values: Reservations and Recommendations for Applying p -Uniform and p -Curve” (2016) sourceExperimental — accuracy not yet measured
In the section “If Heterogeneity Is Moderate to Large, p-Curve and p-Uniform Overestimate Effect Size” (page unavailable), the paper reports that effect-size estimates are biased at its moderate-heterogeneity condition (I² ≈ 0.5). It does not assess I² = 42.01%, so that exact threshold is not tested; its finding does not support the cited expectation that moderate heterogeneity makes the estimates robust. (LLM, conf=0.84)
Checked by a reasoning model — may be wrong.
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Research on goal specificity (e.g., Miller etal., 1999; Vollmeyer etal., 1996), mechanisms of errorful generation (Cyr & Anderson, 2015), as well as prior PS-I work (e.g., Kapur, 2015; Schwartz etal., 2011) has suggested that the benefits of prior knowledge activation such as noticing inconsistencies across multiple problem instances, encoding critical features from instruction, and so on, are”
- Also stated in: § Students' Incoming Characteristics We found that effects favoring PS-I over I-PS were higher if students belonged to a higher grade level. Younger age students (2nd to 5th graders), however, seemed to benefit from I-PS interventions, although the average effect size favoring I-PS was low (Hedge's g of -0.09 [95% CI -0.92, -0.16]. This is despite 72% ( n = 18/25) of the comparisons involving 2nd to 5th graders providing evidence for high prior knowledge activation (reflected in the PF fidelity criteria of evidence for multiple RSM generation ). However, the extent to which the activated prior knowledge is relevant for learning the targeted concepts can substantially affect students' performance. Research on goal specificity (e.g., Miller etal., 1999; Vollmeyer etal., 1996), mechanisms of errorful generation (Cyr & Anderson, 2015), as well as prior PS-I work (e.g., Kapur, 2015; Schwartz etal., 2011) has suggested that the benefits of prior knowledge activation such as noticing inconsistencies across multiple problem instances, encoding critical features from instruction, and so on, are contingent on the relevance of the activation. One explanation for the negative effect sizes disfavoring PS-I for young learners might therefore be the low relevance (albeit, high activation ) of prior knowledge. Factors stemming from students' unawareness of productive metacognitive learning strategies (e.g., planning and monitoring solution generation efficacy, self-explaining, debugging, and error-evaluation) might alternatively contribute to this contrasting trend. Finally, across the comparisons involving 2nd to 5th graders, the low overall PF fidelity score ( M = 56.86%, SD = 27.84%) might also be responsible for negative effects for PS-I. For these younger students then, scaffolding the initial problem-solving phase of PS-I to compensate for their insufficient prior knowledge about cognitive and metacognitive learning strategies might be critical. Richland and Simms (2015), more generally, have documented the importance of scaffolding exploratory problem solving through a series of studies on induction within (non-) STEM domains. They emphasize explicit support in noticing the relevance of relational thinking, providing adequate processing resources to mentally hold and manipulate relations, and facilitating recognition of both similarities and differences when drawing analogies. We further found that comparisons where students were pretested on concepts targeted in the intervention showed a higher effect size favoring PS-I (Hedge's g 0.39 [95% CI -0.26, 0.42]), relative to interventions where no pretest was held (Hedge's g 0.31 [95% CI -0.44, 0.30]), or the pretest targeted only prerequisite concepts (Hedge's g 0.30 [95% CI -0.63, 0.50]). Therefore, including a pretest targeting similar learning materials as the PS-I intervention itself may serve to enhance the differential advantages over I-PS (see Yang etal., 2018, for the facilitatory effects of forward testing), instead of diminishing PS-I's effectiveness due to potential overlap with the triggered learning mechanisms (Newman & DeCaro, 2019). · plain
“contingent on the relevance of the activation.”
What the sources say
- Partly supportedcites Miller et al., 1999 matched to “Goals and Learning in Microworlds” (1999) sourceExperimental — accuracy not yet measured
In “Implications for Learning Theory: Knowledge Dependencies” (page unavailable), the paper says that learning transfers when goal-dependent relationships coincide with pedagogically relevant relationships. This supports the importance of relevance, but the passage concerns knowledge acquired through microworld interaction, not the benefits of activating prior knowledge or the stated examples of noticing inconsistencies and encoding features from instruction. (LLM, conf=0.82)
- Not checked — no full text of the sourcecites Vollmeyer et al., 1996 matched to “The Impact of Goal Specificity on Strategy Use and the Acquisition of Problem Structure” (1996) source
- Not matchedcites Cyr & Anderson, 2015
An author–year citation the paper’s reference list doesn’t settle. We don’t search these by name.
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Students' Incoming Characteristics We found that effects favoring PS-I over I-PS were higher if students belonged to a higher grade level. Younger age students (2nd to 5th graders), however, seemed to benefit from I-PS interventions, although the average effect size favoring I-PS was low (Hedge's g of -0.09 [95% CI -0.92, -0.16]. This is despite 72% ( n = 18/25) of the comparisons involving 2nd to 5th graders providing evidence for high prior knowledge activation (reflected in the PF fidelity criteria of evidence for multiple RSM generation ). However, the extent to which the activated prior knowledge is relevant for learning the targeted concepts can substantially affect students' performance. Research on goal specificity (e.g., Miller etal., 1999; Vollmeyer etal., 1996), mechanisms of errorful generation (Cyr & Anderson, 2015), as well as prior PS-I work (e.g., Kapur, 2015; Schwartz etal., 2011) has suggested that the benefits of prior knowledge activation such as noticing inconsistencies across multiple problem instances, encoding critical features from instruction, and so on, are contingent on the relevance of the activation. One explanation for the negative effect sizes disfavoring PS-I for young learners might therefore be the low relevance (albeit, high activation ) of prior knowledge. Factors stemming from students' unawareness of productive metacognitive learning strategies (e.g., planning and monitoring solution generation efficacy, self-explaining, debugging, and error-evaluation) might alternatively contribute to this contrasting trend. Finally, across the comparisons involving 2nd to 5th graders, the low overall PF fidelity score ( M = 56.86%, SD = 27.84%) might also be responsible for negative effects for PS-I. For these younger students then, scaffolding the initial problem-solving phase of PS-I to compensate for their insufficient prior knowledge about cognitive and metacognitive learning strategies might be critical. Richland and Simms (2015), more generally, have documented the importance of scaffolding exploratory problem solving through a series of studies on induction within (non-) STEM domains. They emphasize explicit support in noticing the relevance of relational thinking, providing adequate processing resources to mentally hold and manipulate relations, and facilitating recognition of both similarities and differences when drawing analogies. We further found that comparisons where students were pretested on concepts targeted in the intervention showed a higher effect size favoring PS-I (Hedge's g 0.39 [95% CI -0.26, 0.42]), relative to interventions where no pretest was held (Hedge's g 0.31 [95% CI -0.44, 0.30]), or the pretest targeted only prerequisite concepts (Hedge's g 0.30 [95% CI -0.63, 0.50]). Therefore, including a pretest targeting similar learning materials as the PS-I intervention itself may serve to enhance the differential advantages over I-PS (see Yang etal., 2018, for the facilitatory effects of forward testing), instead of diminishing PS-I's effectiveness due to potential overlap with the triggered learning mechanisms (Newman & DeCaro, 2019). · plain
Every source the paper cites for this claim was checked and supports it.
In the paper
“To the best of our knowledge, this is the first integrative empirical counterevidence against the claim that problem solving as a sensemaking activity for novices is not an effective instructional approach (cf. Kirschner etal., 2006, and follow-up work).”
What the sources say
- Supportedcites Kirschner et al., 2006 matched to “Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching” (2006) source
The cited paper’s Current Research Supporting Direct Guidance section says that problem solving becomes relatively effective only when learners have sufficient experience and that novices need extensive guidance to avoid unproductive problem-solving search. Its Conclusions likewise favor strong guidance over minimally guided instruction for novice to intermediate learners. The citing authors’ assertion that their meta-analysis is the first counterevidence is their own claim, not one attributed to Kirschner et al. (LLM, conf=0.87)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“12 Results from a complementary Bayesian multilevel meta-analysis model (Harrer etal., 2019; Williams etal., 2018) with weakly informative priors for true pooled effect size (µ ~ N (0, 1)) and between-study heterogeneity ( τ ~ HC (0, 0.5)) aligned with these pooled effect size estimates: Hedge's g = 0.36, 95% CI [0.19, 0.52].”
What the sources say
- Not checked — no full text of the sourcecites Harrer et al., 2019 matched to “Doing Meta-Analysis in R” (2019) source
- Supportedcites Williams et al., 2018 matched to “Bayesian Meta-Analysis with Weakly Informative Prior Distributions” (2018) source
The Introduction and Random-Effects Model section describe Bayesian random-effects meta-analysis using weakly informative priors, supporting the citation as a reference for the method. The Hedges’ g estimate and interval are presented as the citing authors’ results, not as findings attributed to Williams et al. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“13 Previous reviews regarding the forward testing effect (Yang et al., 2018) suggest inconclusive evidence for the extent to which it might generalize to younger children (and older adults).”
What the sources say
- Supportedcites Yang et al., 2018 matched to “Enhancing learning and retrieval of new information: a review of the forward testing effect” (2018) source
The Individual differences section reports no forward testing effect in younger children learning single items, but says this does not rule out an effect with complex materials. The Future research directions section explicitly asks whether the effect generalizes to older adults. Page numbers are not supplied in our copy. (LLM, conf=0.92)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
75 more claims
Create a free account to see all 138 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 36
- It may state two separate claims as one. 33
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While PF works across learning subjects, this work focuses on high school math, as it has been most explored in PF research (Sinha and Kapur, 2021).”
From Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failure (2025) · cites this paper as Sinha and Kapur, 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“which is more effective for human learning than other classical teaching methods (Sinha and Kapur, 2021).”
From Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failure (2025) · cites this paper as Sinha and Kapur, 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While PF works across learning subjects, this work focuses on high school math, as it has been most explored in PF research (Sinha and Kapur, 2021).”
From Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failure (2025) · cites this paper as Sinha and Kapur, 2021
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“As a case study, we create a prototype tutor for high school math following Productive Failure (PF), an advanced and effective learning design.”
From Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failure (2025) · cites this paper as Sinha and Kapur, 2021
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- ConsistentF(3, 162) = 6.485reported p = .0004 · recomputed p < .001
“F (3, 162) = 6.485, p = .0004”
- Consistentχ²(1) = 33.15reported p < .001 · recomputed p < .001
“χ 2 (1) = 33.15, p < .001”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Intervention Characteristics
“conceptual knowledge (Hedge's g = 0.33”
§ Intervention Characteristics
“95% CI [-0.30, 0.22]”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Productive Failure Fidelity for the PS-I Design
“Undergraduates (0.28”
§ Productive Failure Fidelity for the PS-I Design
“[-0.46, 0.24])”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Students' Incoming Characteristics
“Hedge's g of -0.09”
§ Students' Incoming Characteristics
“[95% CI -0.92, -0.16]”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Intervention Characteristics
“transfer (Hedge's g = 0.40”
§ Intervention Characteristics
“95% CI [-0.19, 0.32]”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Students' Incoming Characteristics
“these estimates increased (or became more positive) with the age range.”
§ Productive Failure Fidelity for the PS-I Design
“6th to 10th graders (0.50 [-0.04, 0.58]) Undergraduates (0.28 [-0.46, 0.24])”
2 more findings
Create a free account to see every finding of this check.
Citations
Cites
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
Data Mining: Practical Machine Learning Tools and Techniques
(2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,437
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · cited 1×
“Higgins and Green (2011) was used as a reference text to guide all analyses.”
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099 · cited 1×
“Additionally, we also used inferential statistics (Egger’s test) for evaluating presence of asymmetry in the funnel plot and trim-and-fill procedures (Duval & Tweedie, 2000) if applicable.”
Active learning increases student performance in science, engineering, and mathematics
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1319030111 · Cited by 9,787 · cited 1×
“Meta-analysis has been shown to be an effective technique in evaluating broadscale pedagogical changes (Freeman et al., 2014).”
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006) · Educational Psychologist · doi:10.1207/s15326985ep4102_1 · Cited by 7,776 · cited 4×
“…explicitly via cognitive scaffolds, worked examples, and so on, our view is that the first stage of PF does not resemble (direct) instruction, a learning experience where students are introduced to formal concepts of a domain in the most unambiguous (eloquent) way possible (Kirschner et al., 2006).”
InfluentialCochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,178
Colman (2022) · doi:10.1073/pnas · Cited by 4,370
Advances in Instructional Psychology
(2019) · Routledge eBooks · doi:10.4324/9781315864341 · Cited by 2,532
Proceedings for the Annual Meeting of the Cognitive Science Society
Dziemianko, M., Clarke, A. D. F., & Keller, F. (2013) · Cited by 1,726
Model Selection: An Integral Part of Inference
Buckland, S. T., Burnham, K. P., & Augustin, N. H. (1997) · Biometrics · doi:10.2307/2533961 · Cited by 1,717 · cited 1×
“We use the sum of Akaike weights for each explanatory (independent) variable in the subset of regression models it appeared as a measure of its importance (Buckland et al., 1997).”
Motivating the Academically Unmotivated: A Critical Issue for the 21st Century
Hidi, S., & Harackiewicz, J. M. (2000) · Review of Educational Research · doi:10.3102/00346543070002151 · Cited by 1,689 · cited 1×
“Affective draw of the problem is an interrelated fidelity criterion that functions to pique students’ situational interest (Hidi & Harackiewicz, 2000).”
Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors
Gelman, A., & Carlin, J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614551642 · Cited by 1,343
Fitting three-level meta-analytic models in R: A step-by-step tutorial
Assink, M., & Wibbelink, C. J. M. (2016) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.12.3.p154 · Cited by 1,330 · cited 1×
“A multilevel meta-analysis model was used to pool the effect sizes (Assink & Wibbelink, 2016).”
Interpreting Effect Sizes of Education Interventions
Kraft, M. (2019) · Annenberg Institute for School Reform at Brown University · doi:10.26300/8pjp-2z74 · Cited by 1,286
Interpreting effect sizes of education interventions
Kraft, M. A. (2020) · Educational Researcher · doi:10.3102/0013189x20912798 · Cited by 1,281
Schwartz, D. L., & Bransford, J. D. (1998) · Cognition and Instruction · doi:10.1207/s1532690xci1604_4 · Cited by 1,037
Meta-Analysis of Inquiry-Based Learning: Effects of Guidance
Lazonder, A. W., & Harmsen, R. (2016) · Review of Educational Research · doi:10.3102/0034654315627366 · Cited by 967 · cited 1×
“…during the instruction phase via the use of conversational and social interactive skills (such as critiquing, arguing, engaging students in scientific inquiry, etc.) may be posited to be more effective than an expository style of using one-way instructive presentations (Lazonder & Harmsen, 2016).”
Empirical Benchmarks for Interpreting Effect Sizes in Research
Hill, C. J., Bloom, H. S., Black, A. R., & Lipsey, M. W. (2008) · Child Development Perspectives · doi:10.1111/j.1750-8606.2008.00061.x · Cited by 950 · cited 2×
“8 large), the practical importance of these estimates should be judged by the nature of the intervention being evaluated, its target population, and the outcome measure or measures being used (Hill et al., 2008).”
InfluentialKapur, M. (2008) · Cognition and Instruction · doi:10.1080/07370000802212669 · Cited by 867 · cited 5×
“cited either of the two seminal PF articles (Kapur, 2008; Kapur & Bielaczyc, 2012) and/or other key follow-up PF articles (Kapur, 2014, 2015, 2016),2 2.”
InfluentialSelf-citationSchwartz, D. L., & Martin, T. (2004) · Cognition and Instruction · doi:10.1207/s1532690xci2202_1 · Cited by 763 · cited 3×
“This is because in the majority of the cases (e.g., Kapur & Bielaczyc, 2012; Loibl & Rummel, 2014a; Mazziotti et al., 2015; Schwartz & Martin, 2004), the PS phase following instruction was practice-oriented (rather than being generative in nature).”
The double-edged sword of pedagogy: Instruction limits spontaneous exploration and discovery
Bonawitz, E., Shafto, P., Gweon, H., Goodman, N. D., Spelke, E., & Schulz, L. (2011) · Cognition · doi:10.1016/j.cognition.2010.10.001 · Cited by 730 · cited 1×
“The fact that instruction limits spontaneous exploration and discovery is not unsurprising (Bonawitz et al., 2011).”
Gusenbauer, M. (2019) · Scientometrics · doi:10.1007/s11192-018-2958-5 · Cited by 687 · cited 2×
“1 Google Scholar was chosen because it is currently the most comprehensive academic search engine with 389 million records (Gusenbauer, 2019).”
Designing for Productive Failure
Kapur, M., & Bielaczyc, K. (2012) · Journal of the Learning Sciences · doi:10.1080/10508406.2011.591717 · Cited by 674 · cited 23×
“Students’ demographic and/or incoming characteristics (κ = 1.0, 1.0, and 0.84) a. Age range (as assessed by their grade level): 2nd to 5th graders, 6th to 10th graders, undergraduates, others (postgraduates, professionals) b. Geographical distribution: Europe, North America, Asia, Australia c. Nature of pretesting7: n…”
InfluentialSelf-citationKapur, M. (2016) · Educational Psychologist · doi:10.1080/00461520.2016.1155457 · Cited by 523 · cited 11×
“Finally, ascertaining what prior knowledge (formal or intuitive) students bring into a learning activity is key to designing appropriate guidance to build upon it (Kapur, 2016).”
InfluentialSelf-citationSchwartz, D. L., Chase, C. C., Oppezzo, M. A., & Chin, D. B. (2011) · Journal of Educational Psychology · doi:10.1037/a0025140 · Cited by 464 · cited 2×
“, 1996), mechanisms of errorful generation (Cyr & Anderson, 2015), as well as prior PS-I work (e.g., Kapur, 2015; Schwartz et al., 2011) has suggested that the benefits of prior knowledge activation such as noticing inconsistencies across multiple problem instances, encoding critical features from instruction, and so…”
Productive Failure in Learning Math
Kapur, M. (2014) · Cognitive Science · doi:10.1111/cogs.12107 · Cited by 453 · cited 3×
“Evidence comes not only from quasi-experimental studies conducted in the real ecologies of classrooms (e.g., Hofer et al., 2018; Kapur, 2012; Loibl & Rummel, 2014b) but also from controlled experimental studies (e.g., Kapur, 2014; Newman & DeCaro, 2019; B. Schneider & Blikstein, 2018).”
Self-citationIncreasing Situational Interest in the Classroom
Schraw, G., Flowerday, T., & Lehman, S. (2001) · Educational Psychology Review · doi:10.1023/a:1016619705184 · Cited by 393 · cited 1×
“It can be instantiated by creating intuitive hooks that engage students in design via contrasting cases, authentic storylines, and situating the problem within interactive learning environments (Schraw et al., 2001).”
Parents’ Views of Failure Predict Children’s Fixed and Growth Intelligence Mind-Sets
Haimovitz, K., & Dweck, C. S. (2016) · Psychological Science · doi:10.1177/0956797616639727 · Cited by 383
The Effectiveness of Direct Instruction Curricula: A Meta-Analysis of a Half Century of Research
Stockard, J., Wood, T. W., Coughlin, C., & Rasplica Khoury, C. (2018) · Review of Educational Research · doi:10.3102/0034654317751919 · Cited by 371 · cited 2×
“…in direct instruction, therefore, is that once targeted domain concepts have been formally introduced and worked examples presented to support solution schema construction, only then are students prepared to be subjected to (un)-guided problem solving in a subsequent phase (Stockard et al., 2018).”
Simonsohn, U., Simmons, J. P., & Nelson, L. D. (2015) · Journal of Experimental Psychology General · doi:10.1037/xge0000104 · Cited by 343 · cited 3×
“04s) or an equal probability of different significance levels (Simonsohn et al., 2015).”
Influential(2009) · Routledge eBooks · doi:10.4324/9780203878842 · Cited by 328
When Is It Better to Learn Together? Insights from Research on Collaborative Learning
Nokes-Malach, T. J., Richey, J. E., & Gadgil, S. (2015) · Educational Psychology Review · doi:10.1007/s10648-015-9312-8 · Cited by 308 · cited 1×
“This is because it affords students the opportunity to cue each other’s prior knowledge, build on the complementary expertise of a group member, and use the increased memory and problem-solving resources to detect and correct errors when developing multiple RSMs (Nokes-Malach et al., 2015).”
Towards a Theory of When and How Problem Solving Followed by Instruction Supports Learning
Loibl, K., Roll, I., & Rummel, N. (2017) · Educational Psychology Review · doi:10.1007/s10648-016-9379-x · Cited by 281 · cited 19×
“The past decade has seen a growing body of evidence for the efficacy of PF in facilitating conceptual knowledge and transfer (Kapur, 2016; Loibl et al., 2017).”
InfluentialRethinking the Boundaries of Cognitive Load Theory in Complex Learning
Kalyuga, S., & Singh, A.-M. (2016) · Educational Psychology Review · doi:10.1007/s10648-015-9352-0 · Cited by 241
Meta-representation: an introduction
diSessa, A. A., & Sherin, B. L. (2000) · The Journal of Mathematical Behavior · doi:10.1016/s0732-3123(01)00051-7 · Cited by 238 · cited 1×
“By enforcing appropriate socio-mathematical norms and focusing on motivational scaffolds to keep students engaged in RSM generation, the negative influence of threats to students’ status and respect is mitigated (Cobb, 1995; Sherin, 2000; Thomas & Brown, 2007).”
Productive failure in learning the concept of variance
Kapur, M. (2012) · Instructional Science · doi:10.1007/s11251-012-9209-6 · Cited by 228 · cited 13×
“Students’ demographic and/or incoming characteristics (κ = 1.0, 1.0, and 0.84) a. Age range (as assessed by their grade level): 2nd to 5th graders, 6th to 10th graders, undergraduates, others (postgraduates, professionals) b. Geographical distribution: Europe, North America, Asia, Australia c. Nature of pretesting7: n…”
InfluentialSelf-citationThe Impact of Goal Specificity on Strategy Use and the Acquisition of Problem Structure
Vollmeyer, R., Burns, B. D., & Holyoak, K. J. (1996) · Cognitive Science · doi:10.1207/s15516709cog2001_3 · Cited by 224 · cited 2×
“Research on goal specificity (e.g., Miller et al., 1999; Vollmeyer et al., 1996), mechanisms of errorful generation (Cyr & Anderson, 2015), as well as prior PS-I work (e.”
Developing High‐Quality Teachers: teacher evaluation for improvement
Looney, J. (2011) · European Journal of Education · doi:10.1111/j.1465-3435.2011.01492.x · Cited by 222 · cited 2×
“789 and building on student-generated solutions are two teacher characteristics we could reliably code, credentials such as the teacher’s intellectual ability, knowledge of subject-area, and classroom management skills, if reported in future studies, are very important to consider (Looney, 2011) when evaluating PS-I’s…”
van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616650874 · Cited by 219 · cited 1×
“%) level of heterogeneity in included comparisons (I2 = 42.01%), we expect these estimates to be robust (van Aert et al., 2016).”
InfluentialAnalogy, higher order thinking, and education
Richland, L. E., & Simms, N. (2015) · WIREs Cognitive Science · doi:10.1002/wcs.1336 · Cited by 211 · cited 1×
“Richland and Simms (2015), more generally, have documented the importance of Productive Failure in Preparatory Problem Solving scaffolding exploratory problem solving through a series of studies on induction within (non-) STEM domains.”
Negative Knowledge: Understanding Professional Learning and Expertise
Gartmeier, M., Bauer, J., Gruber, H., & Heid, H. (2008) · Vocations and Learning · doi:10.1007/s12186-008-9006-1 · Cited by 198 · cited 1×
“By discussion of tradeoffs of different approaches to a problem, teacher’s emphasis on negative knowledge (Gartmeier et al., 2008) can facilitate reflection and increase students’ certainty and efficiency of future actions.”
Motivation and Transfer: The Role of Mastery-Approach Goals in Preparation for Future Learning
Belenky, D. M., & Nokes-Malach, T. J. (2012) · Journal of the Learning Sciences · doi:10.1080/10508406.2011.651232 · Cited by 195 · cited 8×
“All coded comparisons with subgroup categories and implementation-level details for PS-I and I-PS can be found in the online supplementary materials.”
InfluentialKnowing what you don't know makes failure productive
Loibl, K., & Rummel, N. (2014) · Learning and Instruction · doi:10.1016/j.learninstruc.2014.08.004 · Cited by 189 · cited 7×
“Empirical results within PF, and more generally PS-I, suggest that such an activity design holds high potential to positively impact curiosity and affect (e.g., Glogger-Frey et al., 2015; Lamnina & Chase, 2019; Loibl & Rummel, 2014a; Sinha et al., 2021; Sinha & Kapur, 2021).”
InfluentialHomanMatthew, D., & GelmanAndrew (2014) · Journal of Machine Learning Research · doi:10.5555/2627435.2638586 · Cited by 184 · cited 1×
“Based on recommendations in Harrer et al. (2019), the overall Bayesian methodology involved 4,000 iterations of the Markov Chain Monte Carlo–based NuTS sampling procedure (Hoffman & Gelman, 2014).”
Loibl, K., & Rummel, N. (2013) · Instructional Science · doi:10.1007/s11251-013-9282-5 · Cited by 169 · cited 7×
“Empirical results within PF, and more generally PS-I, suggest that such an activity design holds high potential to positively impact curiosity and affect (e.g., Glogger-Frey et al., 2015; Lamnina & Chase, 2019; Loibl & Rummel, 2014a; Sinha et al., 2021; Sinha & Kapur, 2021).”
InfluentialGoals and Learning in Microworlds
Miller, C. S., Lehman, J. F., & Koedinger, K. R. (1999) · Cognitive Science · doi:10.1207/s15516709cog2303_2 · Cited by 160 · cited 1×
“Research on goal specificity (e.g., Miller et al., 1999; Vollmeyer et al., 1996), mechanisms of errorful generation (Cyr & Anderson, 2015), as well as prior PS-I work (e.g., Kapur, 2015; Schwartz et al., 2011) has suggested that the benefits of prior knowledge activation such as noticing…”
Glogger-Frey, I., Fleischer, C., Grüny, L., Kappich, J., & Renkl, A. (2015) · Learning and Instruction · doi:10.1016/j.learninstruc.2015.05.001 · Cited by 151 · cited 2×
“Empirical results within PF, and more generally PS-I, suggest that such an activity design holds high potential to positively impact curiosity and affect (e.g., Glogger-Frey et al., 2015; Lamnina & Chase, 2019; Loibl & Rummel, 2014a; Sinha et al., 2021; Sinha & Kapur, 2021).”
Rule-Learning Events in the Acquisition of a Complex Skill: An Evaluation of Cascade
VanLehn, K. (1999) · Journal of the Learning Sciences · doi:10.1207/s15327809jls0801_3 · Cited by 150 · cited 1×
“Indeed, theoretical (and empirically tested) accounts of problem solving endorse such an explanatory account (Chi, 2000; VanLehn, 1999).”
Song, Y. (2018) · Educational Technology Research and Development · doi:10.1007/s11423-018-9600-3 · Cited by 137 · cited 2×
“…2 Examples of PS-I designs with varying design fidelity to PF Study characteristics Loehr et al. (2014) (Study 1) Loibl and Rummel (2014a) (Study 1) Song (2018) Problems affording multiple RSMs Yes Yes Yes Evidence for multiple RSM generation No No Yes Affective draw of the problem No Yes…”
Lamnina, M., & Chase, C. C. (2019) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2019.101785 · Cited by 128 · cited 1×
“Empirical results within PF, and more generally PS-I, suggest that such an activity design holds high potential to positively impact curiosity and affect (e.g., Glogger-Frey et al., 2015; Lamnina & Chase, 2019; Loibl & Rummel, 2014a; Sinha et al., 2021; Sinha & Kapur, 2021).”
Harrer, M., Cuijpers, P., & Ebert, D. (2019) · Zenodo · doi:10.5281/zenodo.2551803 · Cited by 126
Bayesian Meta-Analysis with Weakly Informative Prior Distributions
Williams, D. R., Rast, P., & Bürkner, P. C. (2018) · doi:10.31234/osf.io/7tbrm · Cited by 124 · cited 1×
“12 Results from a complementary Bayesian multilevel meta-analysis model (Harrer et al., 2019; Williams et al., 2018) with weakly informative priors for true pooled effect size (µ ~N(0, 1)) and between-study heterogeneity (τ ~ HC(0, 0.5)) aligned with these pooled effect size estimates: Hedge’s g =…”
InfluentialEnhancing learning and retrieval of new information: a review of the forward testing effect
Yang, C., Potts, R., & Shanks, D. R. (2018) · npj Science of Learning · doi:10.1038/s41539-018-0024-y · Cited by 119 · cited 4×
“13 Previous reviews regarding the forward testing effect (Yang et al., 2018) suggest inconclusive evidence for the extent to which it might generalize to younger children (and older adults).”
InfluentialHofer, S. I., Schumacher, R., Rubin, H., & Stern, E. (2018) · Journal of Educational Psychology · doi:10.1037/edu0000266 · Cited by 113 · cited 3×
“Additionally, several articles involving quasi-experimental comparisons (e.g., Fukaya et al., 2018; Hofer et al., 2018) stated that the I phase of I-PS was designed to provide opportunities for reflection and/or group discussion, that is, students were asked to engage with the presented worked examples, ask questions,…”
Problem-solving or Explicit Instruction: Which Should Go First When Element Interactivity Is High?
Ashman, G., Kalyuga, S., & Sweller, J. (2020) · Educational Psychology Review · doi:10.1007/s10648-019-09500-5 · Cited by 103 · cited 1×
“The complexity of such learning materials (often proxied using ill-defined constructs such as element interactivity) might be a critical boundary condition, especially since a recent PS-I work (Ashman et al., 2020) suggests that advantages of PS-I over I-PS may diminish with increasing complexity.”
How to make failure productive: Fostering learning from errors through elaboration prompts
Loibl, K., & Leuders, T. (2019) · Learning and Instruction · doi:10.1016/j.learninstruc.2019.03.002 · Cited by 102 · cited 1×
“Within PS-I studies, Loibl and Leuders (2018) and Loibl and Leuders (2019), for instance, have found that explicitly prompting students to elaborate on their errors during follow-up instruction improves students’ conceptual knowledge (more so than an experimental condition with monologue-dominant…”
Learning from failure: a meta-analysis of the empirical studies
Darabi, A., Arrington, T. L., & Sayilir, E. (2018) · Educational Technology Research and Development · doi:10.1007/s11423-018-9579-9 · Cited by 99 · cited 2×
“Finally, there is one quantitative meta-analytic review of PS-I versus I-PS comparisons (Darabi et al., 2018).”
Matlen, B. J., & Klahr, D. (2013) · Instructional Science · doi:10.1007/s11251-012-9248-z · Cited by 96 · cited 1×
“Students might not be in a position to use their awareness of knowledge gaps to consolidate information during the instruction phase (Matlen & Klahr, 2013).”
The Play of Imagination: Extending the Literary Mind
Thomas, D., & Brown, J. S. (2009) · Springer eBooks · doi:10.1007/978-1-4020-9992-2_6 · Cited by 93
Cultural Tools and Mathematical Learning: A Case Study
Cobb, P. (1995) · Journal for Research in Mathematics Education · doi:10.2307/749480 · Cited by 82
Thomas, D., & Brown, J. S. (2007) · Games and Culture · doi:10.1177/1555412007299458 · Cited by 82
The Nature of Learning: Using Research to Inspire Practice
OECD (2010) · Educational Research and Innovation · doi:10.1787/9789264086487-5-en · Cited by 61 · cited 1×
“The generation of multiple RSMs in the problem-solving phase can be considered a proxy for prior knowledge activation, a key cognitive mechanism (M. Schneider & Stern, 2010) explaining the differential benefit of preparatory learning activities (Loibl et al., 2017).”
Invention Versus Direct Instruction: For Some Content, It’s a Tie
Chase, C. C., & Klahr, D. (2017) · Journal of Science Education and Technology · doi:10.1007/s10956-017-9700-6 · Cited by 52 · cited 2×
“In comparisons where students in the I-PS condition were provided the same invention problem as PS-I post-instruction (e.g., Chase & Klahr, 2017; Jarosz et al., 2017; TablE 2 Examples of PS-I designs with varying design fidelity to PF Study characteristics Loehr et al. (2014) (Study 1) Loibl and…”
The preparatory effects of problem solving versus problem posing on learning from instruction
Kapur, M. (2015) · Learning and Instruction · doi:10.1016/j.learninstruc.2015.05.004 · Cited by 50 · cited 2×
“…(e.g., Miller et al., 1999; Vollmeyer et al., 1996), mechanisms of errorful generation (Cyr & Anderson, 2015), as well as prior PS-I work (e.g., Kapur, 2015; Schwartz et al., 2011) has suggested that the benefits of prior knowledge activation such as noticing inconsistencies across multiple…”
Self-citationLearning by exploring: How much guidance is optimal?
Newman, P. M., & DeCaro, M. S. (2019) · Learning and Instruction · doi:10.1016/j.learninstruc.2019.05.005 · Cited by 48 · cited 3×
“Evidence comes not only from quasi-experimental studies conducted in the real ecologies of classrooms (e.g., Hofer et al., 2018; Kapur, 2012; Loibl & Rummel, 2014b) but also from controlled experimental studies (e.g., Kapur, 2014; Newman & DeCaro, 2019; B. Schneider & Blikstein, 2018).”
Sinha, T., & Kapur, M. (2021) · Learning and Instruction · doi:10.1016/j.learninstruc.2021.101488 · Cited by 48 · cited 1×
“Empirical results within PF, and more generally PS-I, suggest that such an activity design holds high potential to positively impact curiosity and affect (e.g., Glogger-Frey et al., 2015; Lamnina & Chase, 2019; Loibl & Rummel, 2014a; Sinha et al., 2021; Sinha & Kapur, 2021).”
Self-citationMistakes as stepping stones: Effects of errors on episodic memory among younger and older adults.
Cyr, A.-A., & Anderson, N. D. (2015) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/xlm0000073 · Cited by 47 · cited 1×
“Research on goal specificity (e.g., Miller et al., 1999; Vollmeyer et al., 1996), mechanisms of errorful generation (Cyr & Anderson, 2015), as well as prior PS-I work (e.g., Kapur, 2015; Schwartz et al., 2011) has suggested that the benefits of prior knowledge activation such as noticing…”
Sinha, T., Kapur, M., West, R., Catasta, M., Hauswirth, M., & Trninić, D. (2020) · Journal of Educational Psychology · doi:10.1037/edu0000483 · Cited by 45 · cited 1×
“Empirical results within PF, and more generally PS-I, suggest that such an activity design holds high potential to positively impact curiosity and affect (e.g., Glogger-Frey et al., 2015; Lamnina & Chase, 2019; Loibl & Rummel, 2014a; Sinha et al., 2021; Sinha & Kapur, 2021).”
Self-citationWhen should guidance be presented in physics instruction?
Hsu, C.-Y., Kalyuga, S., & Sweller, J. (2015) · Archives of Scientific Psychology · doi:10.1037/arc0000012 · Cited by 44 · cited 1×
“Prior empirical work (Glogger-Frey et al., 2015; Hsu et al., 2015) has used some exemplar studies to generalize the claim about lack of (truly) experimental work in the PS-I literature, and hence frequent noncomparability of PS-I and PS intervention conditions.”
Wait for it . . . Delaying Instruction Improves Mathematics Problem Solving: A Classroom Study
Loehr, A. M., Fyfe, E. R., & Rittle‐Johnson, B. (2014) · The Journal of Problem Solving · doi:10.7771/1932-6246.1166 · Cited by 41 · cited 2×
“Table 2 showcases some examples of comparisons with varying PF design fidelity (Loehr et al., 2014; Loibl & Rummel, 2014a; Song, 2018), along with the coded subgroup category for those comparisons.”
Chen, O., & Kalyuga, S. (2020) · European Journal of Psychology of Education · doi:10.1007/s10212-019-00445-5 · Cited by 37 · cited 2×
“For procedural knowledge, the results align with prior discussions in the PS-I literature (Chen & Kalyuga, 2020; Loibl et al., 2017) suggesting that despite not being better than I-PS, PS-I to say the least, does not hurt or compromise on students’ knowledge of procedures.”
Loibl, K., & Leuders, T. (2018) · Journal für Mathematik-Didaktik · doi:10.1007/s13138-018-0130-7 · Cited by 34 · cited 2×
“Surprisingly, only one study in the meta-analysis (Loibl & Leuders, 2018) explicitly emphasized the assessment of negative knowledge and found differential benefits compared to conceptual knowledge outcomes.”
Mazziotti, C., Rummel, N., Deiglmayr, A., & Loibl, K. (2019) · npj Science of Learning · doi:10.1038/s41539-019-0041-5 · Cited by 28 · cited 1×
“been exemplified in prior PS-I work comparing individual and collaborative PS-I (e.g., Mazziotti et al., 2015; Mazziotti et al., 2019; Sears, 2006), and makes a strong case to assess the differential impact of group work, when implemented within PS-I and compared with a flipped I-PS sequence.”
Tangible User Interfaces and Contrasting Cases as a Preparation for Future Learning
Schneider, B., & Blikstein, P. (2018) · Journal of Science Education and Technology · doi:10.1007/s10956-018-9730-8 · Cited by 22 · cited 1×
“Evidence comes not only from quasi-experimental studies conducted in the real ecologies of classrooms (e.g., Hofer et al., 2018; Kapur, 2012; Loibl & Rummel, 2014b) but also from controlled experimental studies (e.g., Kapur, 2014; Newman & DeCaro, 2019; B. Schneider & Blikstein, 2018).”
Achievement motivation and knowledge development during exploratory learning
DeCaro, D. A., DeCaro, M. S., & Rittle-Johnson, B. (2015) · Learning and Individual Differences · doi:10.1016/j.lindif.2014.10.015 · Cited by 20 · cited 7×
“All coded comparisons with subgroup categories and implementation-level details for PS-I and I-PS can be found in the online supplementary materials.”
InfluentialCycles of Exploration, Reflection, and Consolidation in Model-Based Learning of Genetics
Kim, B., Pathak, S. A., Jacobson, M. J., Zhang, B., & Gobert, J. D. (2015) · Journal of Science Education and Technology · doi:10.1007/s10956-015-9564-6 · Cited by 19 · cited 1×
“Some previous PS-I work (Kim et al., 2015), however, implicates that repeated exposures to PS-I via multiple smaller cycles of problem solving and instruction happening closely together (resulting in lengthier interventions overall) may, in fact, be beneficial.”
Learning by Invention: Small Group Discussion Activities that Support Learning in Statistics
Jarosz, A. F., Goldenberg, O., & Wiley, J. (2016) · Discourse Processes · doi:10.1080/0163853x.2015.1129593 · Cited by 19 · cited 1×
“In comparisons where students in the I-PS condition were provided the same invention problem as PS-I post-instruction (e.g., Chase & Klahr, 2017; Jarosz et al., 2017; TablE 2 Examples of PS-I designs with varying design fidelity to PF Study characteristics Loehr et al. (2014) (Study 1) Loibl and…”
(2015) · Computer-supported collaborative learning/The Computer-Supported Collaborative Learning Conference · doi:10.22318/cscl2015.363 · Cited by 9
Research works printed but not linked: 3
- Fukaya, T., Uesaka, Y., & Ichikawa, S. (2019). Investigating the effects of thinking after instruction approach. Educational Technology Research , 41 (1), 1-11. https:// doi.org/10.15077/etr.42105 · doi:10.15077/etr.42105
- Hall, M. A. (1999). Correlation-based feature selection for machine learning [Doctoral dissertation, The University of Waikato Hamilton]. https://www.cs.waikato.ac. nz/~mhall/thesis.pdf
- Sears, D. A. (2006). Effects of innovation versus efficiency tasks on collaboration and learning [Doctoral dissertation, Stanford University]. https://iase-web.org/documents/dissertations/06.Sears.pdf
Cited by
Making Sense of Generative Learning
Fiorella, L. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09769-7 · Cited by 135
“The benefits of problem-solving-first approaches appear strongest when learners attempt to solve rich problems that afford generating multiple potential solutions (Kapur, 2016; Sinha & Kapur, 2021).”
BackgroundCognitive Load Theory in Computing Education Research: A Review
Duran, R., Zavgorodniaia, A., & Sorva, J. (2022) · ACM Transactions on Computing Education · doi:10.1145/3483843 · Cited by 92
“Lately, such discussions have frequently featured the pedagogical pattern of productive failure [39, 92], in which a problem-solving activity that fails by design precedes other activities that teach a solution.”
BackgroundLichtenfeld, S., Pekrun, R. H., Marsh, H. W., Nett, U. E., & Reiss, K. M. (2023) · Journal of Educational Psychology · doi:10.1037/edu0000748 · Cited by 122
“However, a recent meta-analysis (Sinha & Kapur, 2021) has suggested that failures in complex learning prior to a memory consolidation phase can be detrimental rather than beneficial for young students’ learning (2nd to 5th graders).”
BackgroundMastery experiences in immersive virtual reality promote pro-environmental waste-sorting behavior
Stenberdt, V. A., & Makransky, G. (2023) · Computers & Education · doi:10.1016/j.compedu.2023.104760 · Cited by 98
Kapur, M., Hattie, J., Grossman, I., & Sinha, T. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.956416 · Cited by 114
“However, to the extent they can generate and explore multiple solutions to the problem, even if they are suboptimal or incorrect, this failure prepares them to learn from subsequent instruction (Sinha and Kapur, 2021a).”
Self-citation
Metadata
- Authors
- Sinha, T., & Kapur, M.
- Affiliations
- ETH Zurich (CH)
- Venue
- Review of Educational Research
- Published
- 2021-06-18
- Volume
- 91
- Issue
- 5
- Pages
- 761-798
- ISSN
- 0034-6543 (ISSN-L)
- 1935-1046
- OpenAlex
- W3174759423
- Cited by
- 212
- References
- 77
- Citations by year
- 2026: 49 · 2025: 57 · 2024: 30 · 2023: 44 · 2022: 29 · 2021: 3
- FWCI
- 25.61
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Review of Educational Research (publishedVersion, cc-by)
- Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
- ETH Zürich Research Collection (cc-by)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Social Sciences › Education › Science Education and Pedagogy
- Keywords
- productive failure, conceptual learning, publication bias, grade level effects, domain-general skills
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.