Do growth mindset interventions impact students’ academic achievement? A systematic review and meta-analysis with recommendations for best practices.

Blinding teachers to students' conditions reduces the chance that teachers' beliefs about condition effectiveness and knowledge of student assignment will influence their behavior toward the student or their evaluation of student performance.

“Finally, blinding teachers to students' conditions reduces the chance that teachers' beliefs about condition effectiveness and knowledge of student assignment will influence their behavior toward the student or their evaluation of student performance (e.g., Ainsworth et al., 2015;Rosenthal & Jacobson, 1968).”

• cites Ainsworth et al., 2015 sourcePartly supported

  The Background to Every Child Counts evaluation (page not provided) says that teachers’ knowledge of assignment and beliefs about the programme could lead them to award higher marks, supporting the claim about evaluation of performance. The paper addresses bias in outcome assessment, not whether blinding changes teachers’ behavior toward students; it also notes that blinding teachers to allocation is extremely difficult. (LLM, conf=0.88)

  Experimental — accuracy not yet measured
• cites Rosenthal & Jacobson, 1968 sourceNot checked — no full text of the source

Given the weak links between mindset and motivational processes and between these motivational processes and achievement, one cannot infer the effect of mindset on academic achievement based on mindset's relationship to a motivational process.

“However, given the weak links between mindset and motivational processes (e.g., Burgoyne et al., 2020;Burnette et al., 2013;Payne et a., 2007) and between these motivational processes and achievement (e.g., Burnette et al., 2013), one cannot infer the effect of mindset on academic achievement based on mindset's relationship to a motivational process.”

• cites Burgoyne et al., 2020 sourceNot checked — no full text of the source
• cites Burnette et al., 2013 sourcePartly supported

  Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  Experimental — accuracy not yet measured
• cites Payne et a., 2007 sourceNot checked — no full text of the source

Recent research (Burgoyne et al., 2018; Limeri et al., 2020) has questioned the assumption that measures of mindset collected before and after an intervention are valid and reliable.

“Recent research (Burgoyne et al., 2018;Limeri et al., 2020) has questioned this assumption.”

• cites Burgoyne et al., 2018 sourceNot checked — no full text of the source
• cites Limeri et al., 2020 sourcePartly supported

  The Introduction (page unavailable) questions the validity of the mindset scale for undergraduates, and the Discussion (page unavailable) suggests that differing interpretations of its items may introduce measurement error. The paper does not specifically assess measures collected before and after an intervention or establish that those measures are unreliable; it notes that some studies have reported test–retest reliability. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

Among the highest-quality evidence (6 studies, N = 13,571), the effect on academic achievement was non-significant (d = 0.02, 95% CI [-0.06, 0.10]).

“When examining the highest-quality evidence (6 studies, N = 13,571), the effect was non-significant: 𝑑 ̅ = 0.02, 95% CI = [-0.06,0.10].”

Authors with a financial incentive to report positive findings published significantly larger effects than authors without this incentive.

“Authors with a financial incentive to report positive findings published significantly larger effects than authors without this incentive.”

The overall effect was non-significant after correcting for potential publication bias.

“which was non-significant after correcting for potential publication bias.”

Among studies demonstrating the intervention influenced students' mindsets as intended (13 studies, N = 18,355), the effect on academic achievement was non-significant (d = 0.04, 95% CI [-0.01, 0.10]).

“When examining only studies demonstrating the intervention influenced students' mindsets as intended (13 studies, N = 18,355), the effect was non-significant: 𝑑 ̅ = 0.04, 95% CI = [-0.01,0.10].”

• AssumptionThe authors state this as an assumption: The second meta-analysis model assumes that measures of mindset are valid and reliable, and that other factors are not the key mechanisms driving effects.

No theoretically-meaningful moderators were significant.

“No theoretically-meaningful moderators were significant.”

Six studies met at least 60% of the best practices criteria.

“This process resulted in six studies meeting at least 60% of the best practices criteria.”

Across all 63 studies (N = 97,672), the review found major shortcomings in study design, analysis, and reporting.

“When examining all studies (63 studies, N = 97,672), we found major shortcomings in study design, analysis, and reporting”

Across all studies, the overall effect of growth mindset interventions on academic achievement was small (d = 0.05, 95% CI [0.02, 0.09]).

“Across all studies, we observed a small overall effect: 𝑑 ̅ = 0.05, 95% CI = [0.02,0.09]”

No studies met all best practices criteria.

“No studies met all best practices criteria.”

Across all studies, there were suggestions of researcher and publication bias.

“and suggestions of researcher and publication bias”

No studies included in our systematic review and meta-analyses artificially dichotomized continuous measures.

“No studies included in our systematic review and meta-analyses artificially dichotomized continuous measures.”

The published and unpublished average effects were not significantly different from one another, Q(1) = 0.96, p = .328.

“They were not significantly different from one another, Q(1) = 0.96, p = .328.”

The Publication Status × Financial Incentive Status interaction is significant, F(1, 75) = 6.35, p = .014.

“The Publication Status × Financial Incentive Status interaction is significant, F(1, 75) = 6.35, p = .014.”

About 10% of the null effects in this literature were described by the study authors as though they were significant.

“About 10% of the null effects in this literature were described by the study authors as though they were significant.”

In the authors' search, the majority of published studies for which an unpublished version was found excluded participants from the published study.

“In our search, the majority of published studies for which we found an unpublished version excluded participants from the published study.”

Yeager et al.'s (2014) Study 3 manipulation check failed: they found no measurable evidence for the impact of the growth mindset intervention on students' mindsets.

“Relatedly, Yeager et al.'s (2014) Study 3 manipulation check failed: they found no measurable evidence for the impact of the growth mindset intervention on students' mindsets.”

• cites Yeager et al.'s (2014)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Adherence to best practices varied considerably, with the majority (70%) of the independent samples meeting 50% or fewer best practices criteria.

“Adherence to best practices varied considerably, with the majority (70%) of the independent samples meeting 50% or fewer best practices criteria (see Figure 4).”

• AssumptionRests on an assumption: Studies without the best-practice characteristics increase the risk to internal validity in individual study results.

The between-study variability in effect sizes due to heterogeneity rather than random error was moderate, I2 = 39.14 (τ2 = .005).

“The between-study variability in effect sizes due to heterogeneity rather than random error was moderate, I 2 = 39.14 (τ 2 = .005).”

The authors found no compelling evidence that mindset skeptics conducted any of the growth mindset interventions included in the current meta-analysis.

“We found no compelling evidence that mindset skeptics conducted any of the growth mindset interventions included in the current meta-analysis.”

In five of the seven cases, participants' data included in the unpublished version were excluded from the published version.

“In five of the seven cases, participants' data included in the unpublished version were excluded from the published version.”

Across all studies regardless of quality, the overall meta-analytic average standardized mean difference in academic achievement between students receiving a growth mindset intervention and students in a control group was d = 0.05, 95% CI = [0.02,0.09], p = .004.

“Across all studies regardless of quality, the overall meta-analytic average standardized mean difference in academic achievement between students receiving a growth mindset intervention and students in a control group was 𝑑 ̅ = 0.05, 95% CI = [0.02,0.09], p = .004.”

Less than one third of the documents (19 of 61) reported a statistically significant difference between treatment and control groups on academic achievement in the hypothesized direction.

“Less than one third of the documents (19 of 61) reported a statistically significant difference between treatment and control groups on academic achievement in the hypothesized direction.”

Of the articles that reported a significant effect, about half attributed the effect to the intervention influencing students' mindsets without evidence the intervention had any effect on students' mindsets.

“Of the articles that reported a significant effect, about half attributed the effect to the intervention influencing students' mindsets without evidence the intervention had any effect on students' mindsets.”

Articles with one or more authors with a financial incentive to report positive effects (30% of documents) were more than two and a half times as likely (56% vs. 21%) to find a significant difference between treatment and control groups on academic achievement in the hypothesized direction than articles where no authors had a financial incentive, χ2(1, N = 61) = 7.09, p = .008.

“Articles with one or more authors with a financial incentive to report positive effects (30% of documents) were more than two and a half times as likely (56% vs. 21%) to find a significant difference between treatment and control groups on academic achievement in the hypothesized direction than articles where no authors had a financial incentive, χ 2 (1, N = 61) = 7.09, p = .008.”

All authors either appeared neutral or expectant of positive effects.

“All authors either appeared neutral or expectant of positive effects.”

Among studies meeting the minimal standard of evidence (13 studies, N = 18,355), the effect of growth mindset interventions on academic achievement was non-significant: d = 0.04, 95% CI [-0.01, 0.10], p = .146.

“When examining these cases (13 studies, N = 18,355), the effect of growth mindset interventions on academic achievement was non-significant: 𝑑 ̅ = 0.04, 95% CI = [-0.01,0.10], p = .146.”

Only 26% of the effect sizes came from studies that met the minimal standard of evidence that the intervention influenced students to have more of a growth mindset.

“Only 26% of the effect sizes came from studies that met this minimal standard of evidence.”

Our moderator analysis testing whether the treatment effect varied by the length of time between the intervention and the measure of academic achievement was non-significant.

“Our moderator analysis testing whether the treatment effect varied by the length of time between the intervention and the measure of academic achievement was non-significant.”

Meta-Analysis 3 had substantial heterogeneity, I2 = 62.71% (τ2 = .006).

“This meta-analysis had substantial heterogeneity, I 2 = 62.71% (τ 2 = .006).”

The majority (five of seven) of unpublished reports of studies that were later published excluded participants prior to publication.

“The majority of them (five of seven) excluded participants prior to publication.”

We found that 58% of the samples-37% of the total students-included in the meta-analysis reported equivalent academic achievement between the treatment and control groups at baseline.

“We found that 58% of the samples-37% of the total studentsincluded in the meta-analysis reported equivalent academic achievement between the treatment and control groups at baseline.”

For studies in which the intervention altered students' mindsets as intended, the overall meta-analytic average standardized mean difference between growth mindset treatment and control groups in academic achievement was non-significant, d = 0.04, 95% CI [-0.01, 0.10], p = .146.

“For studies in which the intervention altered students' mindsets as intended, the overall meta-analytic average standardized mean difference between growth mindset treatment and control groups in academic achievement was non-significant, 𝑑 ̅ = 0.04, 95% CI = [-0.01,0.10], p = .146.”

After correcting for publication bias, there were no significant models that adhered to any number of any combination of best practices criteria.

“After correcting for publication bias, there were no significant models that adhered to any number of any combination of best practices criteria.”

We observed heterogeneity in each of our meta-analyses.

“We observed heterogeneity-differences in effects across studies-in each of our metaanalyses.”

Significant effects were most likely to be reported by authors who had financial incentives to report significant effects.

“significant effects were most likely to be reported by authors who had financial incentives to report significant effects”

About 10% of the null effects in the meta-analysis were described by study authors as though they were significant.

“This was the case for about 10% of the null effects in our meta-analysis.”

Studies with one or more authors with a financial incentive to report positive effects were more than two and a half times as likely to report significant effects than studies with no perceived financial conflict of interest.

“We found that, relative to authors with no perceived financial conflict of interest, studies with one or more authors with a financial incentive to report positive effects were more than two and half times as likely to report significant effects.”

As the number of best practices met increased, the number of significant models decreased.

“As the number of best practices met increased, the number of significant models decreased.”

Many studies conducted in countries where the dominant language is not English nevertheless write the study report in English in an effort to publish in English-language journals (11% of the studies in the current meta-analysis).

“Many studies conducted in countries where the dominant language is not English nevertheless write the study report in English in an effort to publish in Englishlanguage journals (11% of the studies in the current meta-analysis).”

The number of participants removed prior to publication was as high as 6,222 students, half the total sample.

“The number of participants removed was as high as 6,222 students, half the total sample.”

Egger's regression suggests multiple studies may have been conducted but are missing from the available literature and are not included in the present meta-analysis (1-tailed p = .034; 2-tailed p = .067).

“For this meta-analysis, Egger's regression suggests multiple studies may have been conducted but are missing from the available literature and are not included in the present meta-analysis (recommended 1-tailed p = .034;Borenstein et al., 2005; 2-tailed p = .067),see Table 8.”

• cites Borenstein et al., 2005Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Authors with financial incentives were no more likely to find evidence that their growth mindset interventions changed students' mindsets than authors without financial incentives.

“Yet, authors with financial incentives were no more likely to find evidence that their growth mindset interventions changed students' mindsets than authors without”

No systematic review evaluating the quality and quantity of growth mindset interventions has previously been conducted.

“Despite mindset's popularity, no systematic review evaluating the quality and quantity of growth mindset interventions has previously been conducted.”

The average effect size of studies where an author had a financial incentive to find positive results was not significantly different from the average effect size of studies where no authors had perceived financial conflicts of interest.

“We found that the average effect size of studies where an author had a financial incentive to find positive results was not significantly different from the average effect size of studies where no authors had perceived financial conflicts of interest, see Table 8.”

After correcting for publication bias, no meta-analytic models were significant.

“Further, after correcting for publication bias, no meta-analytic models were significant.”

We found that even when considering the best evidence available (Meta-Analysis 3), we had to considerably lower quality standards to have enough studies to meta-analyze.

“We found that even when considering the best evidence available (Meta-Analysis 3), we had to considerably lower quality standards to have enough studies to metaanalyze.”

Approximately 1% of the total searched records were relevant to the present meta-analysis.

“Approximately 1% of the total searched records were relevant to the present meta-analysis.”

Almost all growth mindset interventions allow multiple factors to covary with experimental condition.

“First, almost all growth mindset interventions allow multiple factors to covary with experimental condition.”

When no quality control was imposed, the meta-analytic model was significant.

“When no quality control was imposed, the metaanalytic model was significant.”

Among cases with evidence of mindset change and meeting more than half the best practices criteria (10 effects from 6 studies, N = 13,571), the effect of growth mindset interventions on academic achievement was non-significant: d = 0.02, 95% CI [-0.06, 0.10], p = .666.

“When examining these cases (10 effects from 6 studies, N = 13,571), the effect of growth mindset interventions on academic achievement was non-significant: 𝑑 ̅ = 0.02, 95% CI = [-0.06,0.10], p = .666.”

Our moderator analysis testing whether the treatment effect varied by level of challenge was non-significant.

“Our moderator analysis testing whether the treatment effect varied by level of challenge was non-significant.”

Only 25 of the 96 effect sizes were from studies that provided evidence that the growth mindset intervention influenced treatment students to have more of a growth mindset.

“Only 25 of the 96 effect sizes were from studies that provided evidence that the growth mindset intervention influenced treatment students to have more of a growth mindset.”

We know of no evidence to support the claim that growth mindset interventions produce long-term benefits.

“We know of no evidence to support this claim.”

When examining the best available evidence, the effect of growth mindset interventions on academic achievement was non-significant, d = 0.02, 95% CI [-0.06, 0.10], p = .666.

“When examining the best available evidence, the effect of growth mindset interventions on academic achievement was non-significant: 𝑑 ̅ = 0.02, 95% CI = [-0.06,0.10], p = .666.”

Authors with financial incentives to report positive effects appeared less likely to publish weaker effects than authors without such a financial incentive.

“Authors with financial incentives to report positive effects appeared less likely to publish weaker effects than authors without such a financial incentive (significant Publication status × Financial incentive status interaction).”

In Meta-Analysis 2 (minimal standard of evidence), for published studies where no authors had a financial incentive to find positive results, the effect was significant but negative, d = -0.37, 95% CI [-0.63, -0.10], p = .007.

“No authors with a financial incentive 𝑑 ̅ = -0.37 95% CI [-0.63, -0.10] p = .007”

• Also stated in: § Additional information · hedged

  “For published studies where no authors had a financial incentive to find positive results, the effect was significant, but negative, suggesting growth mindset interventions that changed students' mindsets harmed students' academic achievement, see Table 14.”

The PET estimate of the true effect after correcting for publication bias and other small-study effects was non-significant, d = 0.01.

“Similar to the results of the trim-and-fill, the PET estimate of the true effect after correcting for publication bias and other small-study effects was non-significant, 𝑑 ̅ = 0.01, see Table 8.”

For low-SES students, the effect of growth mindset intervention on academic achievement was significant, d = 0.15, 95% CI [0.002, 0.30], p = .047.

“Low 𝑑 ̅ = 0.15 95% CI [0.002, 0.30] p = .047”

Only 10% of the effects came from studies that both provided evidence the intervention changed students' mindsets and met more than half the best practices criteria.

“Only 10% of the effects came from studies that both provided evidence the intervention changed students' mindsets and met more than half the best practices criteria.”

Methodological, analytical, reporting, and conflict of interest standards are relatively lax in the mindset literature.

“We observed that methodological, analytical, reporting, and conflict of interest standards are relatively lax in the mindset literature; this flexibility in standards might be the source of most significant growth mindset intervention effects that are reported.”

When correcting for publication bias in the first meta-analytic model, the effect was non-significant, d = 0.03, 95% CI [-0.01, 0.07], p = .097.

“Critically, when correcting for publication bias, the effect was non-significant, 𝑑 ̅ = 0.03, 95% CI [-0.01, 0.07], p = .097,or 𝑑 ̅”

71 of the 96 effect sizes included in Meta-Analysis 1 were from interventions with no measurable impact on students' mindsets.

“Critically, 71 of the 96 effect sizes included in Meta-Analysis 1 were from interventions with no measurable impact on students' mindsets.”

No model, regardless of the requirement that included studies demonstrated a successful manipulation check, revealed a significant growth mindset intervention effect after correcting for likely publication bias.

“No model, regardless of this requirement, revealed a significant growth mindset intervention effect after correcting for likely publication bias.”

The PET analysis revealed a statistically significant slope, b = .42, 95% CI [.07, .76], SE = .18, p = .019, indicating the presence of publication bias.

“The PET analysis revealed a statistically significant slope, b = .42,95% CI [.07, .76],SE = .18,p = .019,indicating the presence of publication bias.”

If the estimated 10 studies missing from the meta-analysis were available and included, Duval and Tweedie's analysis estimated the overall effect would be non-significant.

“If the estimated 10 studies missing from our meta-analysis were available and included, Duval and Tweedie's analysis estimated the overall effect would be non-significant, see Table 8.”

For high academic challenge students, the effect of growth mindset intervention on academic achievement was significant, d = 0.07, 95% CI [0.01, 0.13], p = .019.

“High challenge level (e.g., low grades) 𝑑 ̅ = 0.07 95% CI [0.01, 0.13] p = .019”

Significant meta-analytic results were no longer significant after adjusting for publication bias.

“Instead, significant meta-analytic results only occurred when quality control was lacking, and these results were no longer significant after adjusting for publication bias.”

The effects across levels of academic challenge did not differ significantly, Q(2) = 0.28, p = .870.

“Academic Challenge Status a Q(2) = 0.28, p = .870”

In Meta-Analysis 2, no publication bias analyses suggested that the model's effect was impacted by publication bias.

“No publication bias analyses suggested that this model's effect was impacted by publication bias, see Table 14.”

Our meta-analysis included one very small sample (Fleurizard & Young, 2018, N = 10).

“Our meta-analysis included one very small sample (Fleurizard & Young, 2018, N = 10).”

• cites Fleurizard & Young, 2018Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

No studies adhered to all of the best practices criteria.

“No studies adhered to all of the best practices criteria.”

Significant meta-analytic results only occurred when quality control was lacking.

“Instead, significant meta-analytic results only occurred when quality control was lacking, and these results were no longer significant after adjusting for publication bias.”

Trim-and-fill estimated that 10 studies with smaller-than-average effect sizes were missing from the available literature.

“estimated that 10 studies with smaller-than-average effect sizes were missing from the available literature.”

The average growth mindset treatment effect size of studies where one or more authors had a financial incentive to find positive results was not significantly different than the average effect size where no authors had a perceived financial conflict of interest.

“The average growth mindset treatment effect size of studies where one or more authors had a financial incentive to find positive results was not significantly different than the average growth mindset treatment effect size where no authors had a perceived financial conflict of interest, see Table 14.”

• Also stated in: § Additional information · plain

  “financial incentives, χ 2 (1, N = 61) = 0.13, p = .911.”

No publication bias analyses suggested that Meta-Analysis 3's effect was impacted by publication bias.

“No publication bias analyses suggested that this model's effect was impacted by publication bias.”

Fewer than one in four interventions attempted to reduce expectancy bias by keeping students, study administrators, and teachers blind to condition.

“Fewer than one in four interventions attempted to reduce expectancy bias by keeping students, study administrators, and teachers blind to condition.”

As the number of best practices met increased, the number of significant models quickly decreased, such that no models where studies adhered to any combination of three or more best practices were significant.

“As the number of best practices met increased, the number of significant models quickly decreased, such that no models where studies adhered to any combination of three or more best practices were significant, see Table 15.”

Some highly-cited studies interpreted non-significant effects as though they were significant (e.g., Aronson et al., 2002; Good et al., 2003).

“some highly-cited studies interpreted non-significant effects as though they were significant (e.g., Aronson et al., 2002;Good et al., 2003)”

• cites Aronson et al., 2002Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Good et al., 2003 sourceNot checked — no full text of the source

When imposing quality control by examining interventions that changed students' mindsets as intended or examining the highest-quality evidence, significant effects were conspicuously absent.

“However, when imposing quality control by examining interventions that changed students' mindsets as intended or examining the highest-quality evidence, significant effects were conspicuously absent.”

There were no studies that met all the best practices criteria.

“There were no studies that met all the best practices criteria.”

There were not enough published studies by authors with financial incentives in this meta-analysis to conduct the moderator analysis.

“there were not enough published studies by authors with financial incentives in this meta-analysis to conduct this moderator analysis.”

Our trim-and-fill analysis estimated we were missing ten studies with smaller-than-average effect sizes.

“our trim-and-fill analysis estimated we were missing ten studies with smaller-than-average effect sizes.”

Assuming construct measures are valid and reliable, manipulation checks are critical for drawing accurate conclusions about the effect of the independent variable on the dependent variable when the independent variable can only be manipulated indirectly, as is the case with growth mindset interventions.

“Assuming construct measures are valid and reliable, manipulation checks are critical for drawing accurate conclusions about the effect of the independent variable on the dependent variable when the independent variable can only be manipulated indirectly (Hoewe, 2017), as is the case with growth mindset interventions.”

• cites Hoewe, 2017 sourceNot checked — no full text of the source

• AssumptionThe authors state this as an assumption: Construct measures are valid and reliable.

Only when the manipulation check succeeds is there evidence that an effect may be attributed to the hypothesized mechanism.

“Only when the manipulation check succeeds is there evidence that an effect may be attributed to the hypothesized mechanism.”

When increased effort and challenge seeking are combined with growth mindset intervention, changes in effort and challenge seeking following the intervention cannot be attributed to holding more of a growth mindset.

“When these factors are combined with the growth mindset intervention, changes in effort and challenge seeking following the intervention cannot be attributed to holding more of a growth mindset-it may be from directly encouraging students to engage in these behaviors.”

Sisk et al. (2018) meta-analysis of 38 independent samples (N = 57,155) found a small overall standardized mean difference favoring growth mindset interventions (d = 0.08, 95% CI [0.02, 0.14]).

“Overall, Sisk and colleagues (2018) found a small meta-analytic standardized mean difference between intervention and control groups' academic achievement, favoring the interventions: 𝑑 ̅ = 0.08, 95% CI [0.02, 0.14].”

• cites Sisk et al. (2018) sourceNot checked — no full text of the source

If manipulation checks are not included or the manipulation check fails, there is insufficient evidence to attribute intervention effects to the hypothesized mechanism.

“If manipulation checks are not included or the manipulation check fails, there is insufficient evidence to attribute intervention effects to the hypothesized mechanism.”

Outes and colleagues conducted a large-scale growth mindset intervention in Peru.

“For example, Outes and colleagues conducted a large-scale growth mindset intervention in Peru.”

• cites Outes et al. (2017)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

At least some of the observed differences between treatment and control groups in the mindset intervention literature might be driven by encouraging students to work harder.

“These results suggest that at least some of the observed differences between treatment and control groups found in the mindset intervention literature might be driven by encouraging students to work harder.”

• cites Li & Bates (2019) sourceNot checked — no full text of the source

Without a control protocol that is otherwise identical to the treatment, one cannot determine whether teaching a growth mindset is necessary.

“Without a control protocol that is otherwise identical to the treatment we cannot determine whether teaching a growth mindset is necessary.”

Without blinding, subject-expectancy effects might occur, especially when participants are aware or intuit that their assigned condition is designed to be beneficial or not beneficial.

“Without blinding, subject-expectancy effects might occur, especially in cases where participants are aware or intuit that their assigned condition is designed to be beneficial (the treatment condition) or is not designed to be beneficial (the control condition) (Boot et al., 2013).”

• cites Boot et al., 2013Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Per-protocol analyses can introduce systematic bias by removing those who do not comply.

“However, per-protocol analyses can introduce systematic bias by removing those who do not comply.”

Of the 400 schools intended to administer the intervention, 161 schools did not receive the materials or did not administer the intervention.

“Of the 400 schools intended to administer the intervention, 161 schools did not receive the materials or did not administer the intervention.”

• cites Outes et al. (2017)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Flexibility in intent-to-treat analysis decisions can bias results.

“Though intent-to-treat analyses are often viewed as a solution to the bias that can come with per protocol analyses, flexibility in intent-to-treat analysis decisions can also bias results.”

Apparent effects of growth mindset interventions on academic achievement are likely attributable to inadequate study design, reporting flaws, and bias.

“We conclude that apparent effects of growth mindset interventions on academic achievement are likely attributable to inadequate study design, reporting flaws, and bias.”

• Also stated in: § Abstract · hedged

  “positive results are rare and possibly spurious due to inadequately designed interventions, reporting flaws, and bias.”

We should be concerned when failure to comply with treatment increases the reported effect of treatment.

“We should therefore be concerned when failure to comply with treatment increases the reported effect of treatment.”

Blinding to condition assignment does not ensure that expectations are equated because the intervention itself can influence students' expectations about improvement.

“Unfortunately, blinding to condition assignment does not ensure that expectations are equated: The intervention itself can influence students' expectations about improvement.”

Multiple differences between treatment and control protocols in growth mindset interventions make it unclear whether differences are due to changing mindsets, mindset training augmented with effort encouragement, or effort encouragement alone.

“Multiple differences between treatment and control protocols make it unclear whether (a) differences between groups are due to changing students' mindsets, (b) mindset training must be augmented with encouragement to work harder to be effective, or (c) differences are simply due to encouragement to work harder.”

The true explanatory effect of an intervention cannot be determined when researchers have such flexibility in applying estimates and adjustments.

“The true explanatory effect of an intervention cannot be determined when researchers have such flexibility in applying estimates and adjustments.”

Non-significant results described as significant likely cause confusion about the impact of growth mindset interventions.

“Non-significant results described as significant likely causes confusion about the impact of growth mindset interventions.”

The result that articles with financially incentivized authors were more likely to report significant effects suggests that financial incentives may be the source of this difference.

“This result suggests that financial incentives may be the source of this difference.”

Studies that assigned students to condition by group without correction violate the assumption of independence and underestimate sampling error.

“These studies violate the assumption of independence and underestimate sampling error, as students within a classroom or school will be more similar to each other and more different from other groups due to factors (e.g., shared teachers, shared schedule) other than the experimental condition.”

An ANCOVA controlling for pre-intervention differences would indicate a benefit of the intervention for the treatment group despite no improvement from the intervention (Lord's paradox).

“An ANCOVA controlling for pre-intervention differences would indicate a benefit of the intervention for the treatment group despite no improvement from the intervention (i.e., Lord's paradox, Lord, 1967; see also Wright, 2006).”

• cites Lord, 1967 sourceNot checked — no full text of the source
• cites Wright, 2006 sourceNot checked — no full text of the source

The null result for the difference between published and unpublished average effects appears to be due to different publishing patterns between researchers with a financial incentive to find positive effects versus researchers without such a financial incentive.

“This null result appears to be due to different publishing patterns between researchers with a financial incentive to find positive effects versus researchers without such a financial incentive.”

Statistically controlling for pre-intervention differences does not necessarily solve the problem of baseline differences.

“Statistically controlling for pre-intervention differences does not necessarily solve this problem.”

The adjustment yielded effect sizes 830% to 1,800% of the effect sizes for students who received the treatment.

“Thus, an adjustment ostensibly intended to better reflect effects had students received the treatment, yielded effect sizes 830% to 1,800% of the effect sizes for students who received the treatment.”

Failing to implement the treatment substantially increased the apparent effect of the treatment.

“Thus, failing to implement the treatment substantially increased the apparent effect of the treatment.”

Financial incentives to report particular outcomes may influence study design, analyses, how findings are interpreted, and which results are reported and published.

“Financial incentives to report particular outcomes may influence study design, analyses, how findings are interpreted, and which results are reported and published (Roseman et al., 2011).”

• cites Roseman et al., 2011Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Paunesku and colleagues combined all three intervention groups into one and compared this single combined intervention group to the control group for one of their two outcomes.

“However, Paunesku and colleagues combined all three intervention groups into one (the mindset intervention, the sense-of-purpose intervention, and the combined mindset and sense-of-purpose intervention) and compared this single combined intervention group to the control group for one of their two outcomes.”

• cites Paunesku et al. (2015)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

When assignment is by classroom, differences in grades cannot be solely attributed to the treatment manipulation.

“Differences in grades cannot be solely attributed to the treatment manipulation: students sharing a classroom have commonalities in their educational environment other than the treatment and therefore some degree of relatedness.”

Mindset theory skeptics appear less likely to conduct growth mindset interventions than mindset theory proponents.

“Mindset theory skeptics appear less likely to conduct growth mindset interventions than mindset theory proponents.”

Publication suppression leads to a skewed representation of results in the available literature.

“Publication suppression leads to a skewed representation of results in the available literature.”

Per-protocol analyses yielded ds = 0.03 and 0.01 for two academic achievement measures.

“Per-protocol analyses-analyses that only included the 239 schools that administered the treatment as intended-yielded ds = 0.03 and 0.01 for two academic achievement measures (see Sisk et al., 2018).”

• cites Sisk et al., 2018 sourceNot checked — no full text of the source

Paunesku et al. (2015) conducted a study where students were randomly assigned to a control condition, a growth mindset intervention, a sense-of-purpose intervention, or a growth-mindset-and-sense-of-purpose combined intervention.

“For example, Paunesku et al. (2015) conducted a study where students were randomly assigned to a control condition, a growth mindset intervention, a senseof-purpose intervention, or a growth-mindset-and-sense-of-purpose combined intervention.”

• cites Paunesku et al. (2015)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Examining whether a growth mindset intervention predicts a predictor of academic achievement does not provide strong evidence that the intervention directly predicts that outcome.

“Examining whether a growth mindset intervention predicts a predictor of academic achievement does not provide strong evidence that the intervention directly predicts that outcome.”

Evidence from fields like medicine raises doubts about the ability to remain neutral when financial incentives are aligned with one outcome.

“Although researchers no doubt view their own work as objective and untainted by corporate influences, evidence from fields like medicine raise doubts about the ability to remain neutral when financial incentives are aligned with one outcome (e.g., see Bekelman et al., 2003;Garg et al., 2005;Perlis et al., 2005).”

• cites Bekelman et al., 2003Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Garg et al., 2005 sourceNot checked — no full text of the source
• cites Perlis et al., 2005 sourceNot checked — no full text of the source

When the manipulation check fails, it is unclear whether the latent construct of growth mindset is responsible for the causal effect of the intervention.

“However, when the manipulation check fails, it is unclear whether the latent construct of growth mindset is responsible for the causal effect of the intervention.”

Some mindset researchers profit from selling self-help books that tout the benefits of growth mindsets.

“Some mindset researchers profit from selling self-help books that tout the benefits of growth mindsets.”

If the treatment and control groups differ on the outcome variable measure at baseline, the effect of the intervention may be ambiguous.

“If the treatment and control groups differ on the outcome variable measure at baseline, the effect of the intervention may be ambiguous.”

With 21 significance tests and an alpha level of .05, we should expect that one outcome will be significant by chance even if the null hypothesis is true.

“With 21 significance tests and an alpha level of .05,we should expect that one outcome will be significant by chance even if the null hypothesis is true, because the typical alpha level (.05) permits a false positive rate of 5%.”

By designing the study to combine multiple interventions, the effects of the growth mindset intervention cannot be isolated experimentally.

“By designing the study to combine multiple interventions, the effects of the growth mindset intervention cannot be isolated experimentally.”

Skewed representation can lead to misunderstandings regarding the strength or reliability of effects.

“Skewed representation can lead to misunderstandings regarding the strength or reliability of effects.”

All three publication bias tests indicated that the mindset intervention literature contains significant publication bias.

“All three tests indicated that the mindset intervention literature contains significant publication bias.”

Findings unsupportive of growth mindset hypotheses are suppressed from publication.

“This result suggests that findings unsupportive of growth mindset hypotheses are suppressed from publication.”

Growth mindset interventions that changed students' mindsets harmed students' academic achievement (in published studies with no financial incentive).

“suggesting growth mindset interventions that changed students' mindsets harmed students' academic achievement”

Publication bias appeared to be a factor in the meta-analysis.

“Publication bias appeared to be a factor in our meta-analysis as well.”

Apparent relationships between mindsets and motivations and behaviors are due to measurement problems, inadequate study design, reporting flaws, and/or bias, as appears to be the case in the growth mindset intervention literature.

“By examining both the quantity and quality of evidence, researchers should be able to assess whether (a) true relationships exist between mindsets and motivations and behaviors, or whether (b) apparent relationships are due to measurement problems, inadequate study design, reporting flaws, and/or bias, as appears to be the case in the growth mindset intervention literature.”

When considering the publication bias-corrected estimates, d = 0.03 and d = 0.01, there is a 50.8/49.2 to 50.3/49.7 chance that a randomly selected student from the treatment group will have a higher score on the measure of academic achievement than a randomly selected student from the control group.

“When considering the publication bias-corrected estimates, 𝑑 ̅ = 0.03 and 𝑑 ̅ = 0.01, there is a 50.8/49.2to 50.3/49.7chance that a randomly selected student from the treatment group will have a higher score on the measure of academic achievement than a randomly selected student from the control group (Magnusson, 2020).”

• cites Magnusson, 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The claim that changing students' mindsets will impact their academic achievement is undermined by the null result for interventions that measurably changed students' mindsets.

“Specifically, the claim that changing students' mindsets will impact their academic achievement is undermined by the null result for interventions that measurably changed students' mindsets.”

Apparent effects of growth mindset interventions may be spurious and due to inadequate study design, inappropriate interpretation of results, and bias.

“Thus, apparent effects of growth mindset interventions may be spurious and due to inadequate study design, inappropriate interpretation of results, and bias.”

Overall effect sizes we observed are slightly upwardly biased.

“Thus, overall effect sizes we observed are slightly upwardly biased.”

When multiple characteristics of the protocol differ between groups, the mechanism of any effect is undeterminable.

“When multiple characteristics of the protocol differ between groups, the mechanism of any effect is undeterminable.”

The strongest association was counter to mindset theory.

“The strongest association was counter to mindset theory.”

• cites Burgoyne et al., 2020 sourceNot checked — no full text of the source

When studies assign classes or schools to condition rather than individual students, the Type I error rate is inflated.

“One such issue occurs when studies assign classes or schools to condition rather than individual students, which inflates the Type I error rate (Hox, 1998;McCoach & Adelson, 2010).”

• cites Hox, 1998 sourceNot checked — no full text of the source
• cites McCoach & Adelson, 2010 sourceNot checked — no full text of the source

The argument that growth mindsets lead to meaningful changes in motivation and behavior is not well supported.

“That said, the argument that growth mindsets lead to meaningful changes in motivation and behavior is not well supported.”

The evidence for the claim that mindset interacts with level of challenge such that interventions are most effective for struggling students is limited.

“The evidence for this claim is limited.”

This flexibility in standards might be the source of most significant growth mindset intervention effects that are reported.

“We observed that methodological, analytical, reporting, and conflict of interest standards are relatively lax in the mindset literature; this flexibility in standards might be the source of most significant growth mindset intervention effects that are reported.”

Treating all studies as having the same quality and ability to interpret mechanisms, and not accounting for bias, may lead to misleading conclusions.

“Our research indicates that treating all studies as having the same quality and ability to interpret mechanisms, and not accounting for bias, may lead to misleading conclusions.”

We cannot conclude that growth mindset interventions are more important for low-SES students than not-low SES students.

“We therefore cannot conclude that growth mindset interventions are more important for low-SES students than not-low SES students.”

The theoretical underpinnings of mindset remain largely unsubstantiated after nearly forty years of research with millions of dollars in funding.

“It is perhaps surprising that after nearly forty years of mindset research (Dweck first introduced the theory in 1983; see Dweck & Bempechat, 1983) with millions of dollars in funding, the theoretical underpinnings of mindset remain largely unsubstantiated.”

Financial incentives may influence which effects are published.

“Thus, financial incentives may influence which effects are published.”

In a meta-analysis of measured mindsets and motivations, Burnette and colleagues (2013) generally found weak associations.

“In a meta-analysis of measured mindsets and motivations, Burnette and colleagues (2013) generally found weak associations.”

• cites Burnette and colleagues (2013)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

This publication suppression biases the available evidence to appear more supportive of growth mindset intervention effects on students' academic achievement than may be warranted.

“This publication suppression biases the available evidence to appear more supportive of growth mindset intervention effects on students' academic achievement than may be warranted.”

The What Works Clearinghouse (2020) uses a default intraclass correlation of ρ = .20 for achievement outcomes if no intraclass correlation is provided by the study.

“the What Works Clearinghouse (2020) under the Department of Education uses a default intraclass correlation of ρ = .20for achievement outcomes if no intraclass correlation is provided by the study.”

• cites What Works Clearinghouse (2020)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Excluding data and removing subsamples prior to publication leaves a skewed view of interventions' effects in the published literature.

“Excluding data and removing subsamples prior to publication leaves a skewed view of interventions' effects in the published literature.”

Taken together along with Meta-Analysis 3, the results suggest that apparent effects of growth mindset interventions may be due to inadequate study designs, reporting flaws, and/or bias.

“Taken together along with Meta-Analysis 3, our results suggest that apparent effects of growth mindset interventions may be due to inadequate study designs, reporting flaws, and/or bias.”

In a preregistered, large-sample, direct test, Burgoyne et al. (2020) found weak and null relationships between mindset and several motivations and behaviors assumed to originate from mindset.

“Likewise, in a preregistered, large-sample, direct test of mindset's associations with several motivations and behaviors assumed to originate from mindset (e.g., goal orientation, challenge seeking, resilience), we (Burgoyne et al., 2020) found weak and null relationships.”

• cites Burgoyne et al., 2020 sourceNot checked — no full text of the source

These results undermine the theoretical claims of mindset theory.

“These results undermine the theoretical claims of mindset theory.”

Post-test differences between treatment and control groups do not provide evidence that the growth mindset intervention influenced students' mindsets, because the groups could have differed at baseline or the control group could have become more fixed.

“Post-test differences between treatment and control groups do not provide evidence that the growth mindset intervention influenced students' mindsets, because the groups could have differed at baseline or the control group could have become more fixed.”

When considering the publication bias-corrected estimates, d = 0.03 and d = 0.01, there is 99-100% overlap between the treatment and control groups' academic achievement distributions.

“When considering the publication bias-corrected estimates, 𝑑 ̅ = 0.03 and 𝑑 ̅ = 0.01, there is 99-100% overlap between the treatment and control groups' academic achievement distributions.”

The current evidence does not support claims that growth mindsets are beneficial for students' academic achievement.

“In sum, despite the popularity of growth mindset interventions, the current evidence does not support claims that growth mindsets are beneficial for students' academic achievement.”

Taken together, the overall uncorrected effect size found in the present meta-analysis is likely overestimated.

“Taken together, the overall uncorrected effect size found in the present meta-analysis is likely overestimated.”

Studies adhering to best practices are unlikely to demonstrate that growth mindset interventions benefit students' academic achievement.

“Taken together, our findings indicate that studies adhering to best practices are unlikely to demonstrate that growth mindset interventions benefit students' academic achievement.”

We cannot conclude that students' level of academic challenge is an important factor for determining the efficacy of a growth mindset intervention.

“Therefore, we cannot conclude that students' level of academic challenge is an important factor for determining the efficacy of a growth mindset intervention.”

Apparent effects of growth mindset interventions on academic achievement are likely spurious and due to inadequate study design, flawed reporting, and bias.

“This pattern suggests that apparent effects of growth mindset interventions on academic achievement are likely spurious and due to inadequate study design, flawed reporting, and bias.”