A systematic review and meta-analysis of effects of early life non-cognitive skills on academic, psychosocial, cognitive and health outcomes

May combine two claims.

Analysis of funnel plots demonstrates asymmetry of effect size and potential small study bias.

“Analysis of funnel plots clearly demonstrate asymmetry of effect size and the potential of small study bias43.”

• cites [43] Egger, M. & Smith, G. D. Misleading meta-analysis. BMJ 310, 752–754 (1995). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Prediction intervals for observational academic achievement were consistent with negative, null, and positive effects, ranging from -0.01 to 0.33 for literacy and -0.07 to 0.52 for school readiness.

“Prediction intervals were consistent with negative, null and positive effects, ranging from -0.01 to 0.33 for literacy and -0.07 to 0.52 for school readiness.”

May combine two claims.

The 95% prediction interval for the 11 literacy EQI studies was -0.13 to 0.79, consistent with negative, null, and positive effects.

“The 95% prediction interval for the 11 literacy studies available for meta-analysis was consistent with negative, null and positive effects (-0.13 to 0.79).”

May combine two claims.

For observational academic achievement studies, effect sizes ranged from 0.16 for literacy to 0.22 for academic achievement and school readiness.

“For observational studies, Figure 2 (panel B) shows effect sizes ranged from 0.16 (95%CI 0.12 to 0.20) for literacy and 0.22 (95%CI 0.14 to 0.31) for academic achievement and school readiness.”

May combine two claims.

Of 554 eligible studies, 222 (40%) were rated better evidence, 21.5% weak, and 38.5% poor with effectively no attempt to control confounding.

“Table 1 shows that of the 554 eligible studies, only 40% (n=222) were rated as “better” evidence, 21.5% classified as weak and 38.5% as poor, where there was effectively no attempt to control confounding.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The systematic search identified 9553 articles from electronic and hand-searched sources.

“The systematic search identified 9553 articles from electronic and hand-searched sources.”

May combine two claims.

Funnel plots for both experimental and observational studies were positively skewed and consistent with smaller studies having larger effects.

“Funnel plots for both experimental and observational studies were positively skewed and consistent with smaller studies having larger effects.”

May combine two claims.

Only four studies could be included in the fade out analysis, with no clear pattern supporting fade out effects.

“There were only four studies that could be included in this analysis, so interpretive caution is warranted with no clear pattern to support evidence of fade out effects.”

May combine two claims.

Forest plots and 95% prediction intervals show large heterogeneity of reported effect sizes generally including the null.

“Additionally, forest plots and 95% prediction intervals show large heterogeneity of reported effect sizes generally including the null.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Associations between non-cognitive skills in childhood and later outcomes have been widely investigated.

“Associations between “non-cognitive” skills in childhood, such as attention, self-regulation, and perseverance, and later outcomes have been widely investigated.”

Probably not a substantial claim: may not pass S3 (load-bearing).

After removing duplicates and assessing eligibility, 554 articles were included.

“After removing duplicates and assessing eligibility, 554 articles were included and presented in a PRISMA36 flowchart (Figure 1).”

May combine two claims.

In the meta-analysis, separating experimental/quasi-experimental interventions (EQIs) from observational publications showed larger effects for EQIs than for higher-quality observational studies, whose effects ranged from about 0.06 to 0.22 SD.

“Presenting the analysis in Figure 2 by separating EQIs (Panel A) and observational publications (Panel B) shows larger effects from EQIs than found in higher quality observational studies which ranged from approximately 0.06 to 0.22SD.”

May combine two claims.

One quasi-experimental health study reported an effect of 0.79SD, but this effect is difficult to interpret because of inadequate description of the control group and outcome.

“One quasi-experimental study reported an effect of 0.79SD, but this effect is difficult to interpret because of an inadequate description of the control group and the outcome.”

May combine two claims.

One higher-quality observational publication that examined six cohorts with 5.5-year follow-up reported effects from 0 to 0.2 SD, consistent with the review's observational findings.

“This is consistent with one higher quality observational publication11 which examined six different cohorts with longer follow-up of 5.5 years and reported effects from 0 to 0.2SD.”

• cites [11] Duncan, G. J. et al. School readiness and later achievement. Dev. Psychol. 43, 1428–1446 (2007). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Most studies of self-regulation used the HTKS task and showed some effects on vocabulary.

“Most studies of self-regulation used the HTKS task and showed some effects on vocabulary.”

May combine two claims.

Intervention studies of non-cognitive skills at around age 4 with median follow-up under one year were generally consistent with short-term effects on academic achievement of 0.2 to 0.4 SD, but effects were heterogeneous, with 95% prediction intervals including negative, null, and positive effects.

“These studies were generally consistent with 0.2 to 0.4SD short-term effects on academic achievement but effects were heterogeneous with 95% prediction intervals including negative, null and positive effects.”

May combine two claims.

Observational studies on academic achievement generally showed effects around 0.2 SD, but all 95% prediction intervals included the null.

“Observational studies on academic achievement generally showed effects around 0.2SD but all 95% prediction intervals included the null.”

May combine two claims.

For executive function, effects ranged from -0.36 to 0.52SD, but the evidence is predominantly from convenience samples.

“For executive function effects ranged from a detrimental -0.36 to 0.52SD, but the evidence is predominantly from convenience samples.”

May combine two claims.

Three reasonably sized cluster RCTs provided the best estimate of the effect of non-cognitive skills on language and cognitive outcomes, finding effects of about 0.1 to 0.2 SD.

“Three reasonably-sized cluster RCTs provided the best estimate of the effect of non-cognitive skills on language and cognitive outcomes25,49. They found effects of ~0.1 to 0.2SD.”

• cites [25] Raver, C. C. et al. CSRP’s Impact on low-income preschoolers’ preacademic skills: self-regulation as a mediating mechanism. Child Dev. 82, 362–378 (2011). sourceNot checked yet

  Not checked in the free run.

• cites [49] Barnett, W. S. et al. Educational effects of the Tools of the Mind curriculum: a randomized trial. Early Child. Res. Q. 23, 299–313 (2008). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

In this review, there is little evidence to decide which mode of delivery is best, and no evidence of attempts at purposive testing of which way to intervene.

“In our review, there is little evidence to decide which mode of delivery is best and we can find no evidence of attempts at purposive testing of which way to intervene (e.g. teacher, student, parent or various combinations).”

May combine two claims.

Observational studies provide little evidence that effects are likely to be bigger than about 0.1 SD, with seven of nine longitudinal studies showing few differences and cross-sectional studies reporting mixed effects (-0.38 to 0.56 SD).

“The observational studies provide little evidence that the effects are likely to be bigger than ~0.1SD, with seven of nine longitudinal studies showing few differences and cross-sectional studies reporting mixed effects (-0.38 to 0.56SD).”

May combine two claims.

Average age at intervention was around 4 years, with median follow-up under one year.

“Average age at the time of intervention was around 4 years with median follow up time under one year.”

May combine two claims.

Higher-quality RCTs examining externalising outcomes reported positive effects, and the largest RCT reported null effects.

“Higher quality RCTs examining externalising outcomes, reported positive5152 and null effects in the largest of the RCTs37.”

• cites [5152]Not checked yet

  Not checked in the free run.

• cites [37] Webster‐Stratton, C., Jamila Reid, M. & Stoolmiller, M. Preventing conduct problems and improving school readiness: evaluation of the incredible years teacher and child training programs in high‐risk schools. J. Child Psychol. Psychiatry 49, 471–488 (2008). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

All these studies were small (n=12 to 64) and reported effects that attenuated over time or were inconsistent at different ages.

“However, all these studies were small (ranging from n=12 to 64) and reported effects that attenuated over time or were inconsistent at different ages.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Research on non-cognitive skills in childhood has spanned many disciplines and traditions, leading to many constructs and tasks that are sometimes similar in definition and operationalisation.

“Research that has examined non-cognitive skills in childhood has spanned many disciplines and research traditions, leading to a large number of constructs and tasks being investigated that are sometimes similar in their definition and operationalisation69,70.”

• cites [69] Duckworth, A. L. & Kern, M. L. A meta-analysis of the convergent validity of self-control measures. J. Res. Pers. 45, 259–268 (2011). sourceNot checked yet

  Not checked in the free run.

• cites [70] Zhou, Q., Chen, S. H. & Main, A. Commonalities and differences in the research on children’s effortful control and executive function: a call for an integrated model of self-regulation. Child Dev. Perspect. 6, 112–121 (2012). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Other studies imply that non-specific interventions seem to generate better generalised outcomes.

“Other studies imply that non-specific interventions seem to generate better generalised outcomes31, which may suggest that more holistic programs including multi-dimensional content, may better support overall child development and broad-based benefits.”

• cites [31] Heckman, J. J., Pinto, R. & Savelyev, P. Understanding the mechanisms through which an early childhood program boosted adult outcomes. Am. Econ. Rev. 103, 2052–2086 (2013). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Exceptions were Barnett et al. and Blair and Raver, who examined effects of the Tools of the Mind intervention in cluster RCTs and both found small effects of about 0.1 SD for vocabulary.

“Exceptions were: Barnett et al.49 and Blair and Raver79 who examined effects of Tools of the Mind intervention in cluster RCTs and both found small effects of ~0.1SD for vocabulary.”

• cites [49] Barnett, W. S. et al. Educational effects of the Tools of the Mind curriculum: a randomized trial. Early Child. Res. Q. 23, 299–313 (2008). sourceNot checked yet

  Not checked in the free run.

• cites [79] Blair, C. & Raver, C. C. Closing the achievement gap through modification of neurocognitive and neuroendocrine function: results from a cluster randomized controlled trial of an innovative approach to the education of children in kindergarten. PLoS ONE 9, e112393 (2014). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The Continuous Performance Task is used in executive functioning research as a measure of sustained attention and inhibitory control, but also as a measure of effortful control.

“For example the Continuous Performance Task (aka “Go/No Go” task) is used in executive functioning research as a measure of sustained attention and inhibitory control, but it is also used as a measure of effortful control70.”

• cites [70] Zhou, Q., Chen, S. H. & Main, A. Commonalities and differences in the research on children’s effortful control and executive function: a call for an integrated model of self-regulation. Child Dev. Perspect. 6, 112–121 (2012). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The Head Toes Knees Shoulders Task has been used to measure both behavioural self-regulation and executive functioning.

“Similarly, the “Head Toes Knees Shoulders Task” has been used to measure both behavioural self-regulation73 and executive functioning74.”

• cites [73] Ponitz, C. C., McClelland, M. M., Matthews, J. & Morrison, F. J. A structured observation of behavioral self-regulation and its contribution to kindergarten outcomes. Dev. Psychol. 45, 605–619 (2009). sourceNot checked yet

  Not checked in the free run.

• cites [74] Cameron, C. E. et al. Fine motor skills and executive function both contribute to kindergarten achievement. Child Dev. 83, 1229–1244 (2012). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Outcomes across the 28 better-quality observational studies were diverse, ranging from anthropometry to injury to physiological characteristics, and were consistent with effects ranging from 0.06 to 0.14 SD.

“Outcomes reported across the 28 better quality observational studies were diverse ranging from anthropometry, to injury to physiological characteristics and were consistent with effects ranging from 0.06 to 0.14SD.”

May combine two claims.

The longitudinal studies were dominated by non-cognitive skills measured using the HTKS and the WJ Picture Vocabulary as the outcome, and despite the popularity of these tools, the results indicate no effects on vocabulary outcomes.

“The longitudinal studies were dominated by non-cognitive skills measured using the HTKS and the WJ Picture Vocabulary as the outcome, and despite the popularity of these measurement tools, the results indicate no effects on vocabulary outcomes.”

May combine two claims.

The largest effect sizes came from a well-designed regression discontinuity study (0.44 SD), a non-randomised intervention (0.55-0.73 SD), and a small, low-quality randomised trial.

“The largest effect sizes were from a well-designed regression discontinuity study (0.44SD)58, a non-randomised intervention (0.55-0.73SD)59 and a small, low quality randomised trial60.”

• cites [58] Weiland, C. & Yoshikawa, H. Impacts of a pre kindergarten program on children’s mathematics, language, literacy, executive function, and emotional skills. Child Dev. 84, 2112–2130 (2013). sourceNot checked yet

  Not checked in the free run.

• cites [59] Bradley, R. T., Galvin, P., Atkinson, M. & Tomasino, D. Efficacy of an emotion self-regulation program for promoting development in preschool children. Glob. Adv. Health Med. 1, 36–50 (2012). sourceNot checked yet

  Not checked in the free run.

• cites [60] Ford, R. M., McDougall, S. J. & Evans, D. Parent-delivered compensatory education for children at risk of educational failure: improving the academic and self-regulatory skills of a Sure Start preschool sample. Br. J. Psychol. 100, 773–797 (2009). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Another RCT showed that effects on literacy depended on the component of literacy measured, and effects on some outcomes faded after one year.

“Similarly, another RCT showed that effects on literacy depended on the component of literacy that was measured40,41 and effects on some outcomes faded after one year42.”

• cites [40] Nix, R. L., Bierman, K. L., Domitrovich, C. E. & Gill, S. Promoting children’s social-emotional skills in preschool can enhance academic and behavioral functioning in kindergarten: findings from Head Start REDI. Early Educ. Dev. 24, 1000–1019 (2013). sourceNot checked yet

  Not checked in the free run.

• cites [41] Bierman, K. L. et al. Promoting academic and social‐emotional school readiness: the Head Start REDI program. Child Dev. 79, 1802–1817 (2008). sourceNot checked yet

  Not checked in the free run.

• cites [42] Bierman, K. L. et al. Effects of Head Start REDI on children’s outcomes 1 year later in different kindergarten contexts. Child Dev. 85, 140–159 (2014). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Such an intervention has generated about 13% greater improvement than the natural growth in reading during that time.

“Such an intervention has generated about 13% greater improvement than the natural growth in reading during that time.”

May combine two claims.

No RCTs had a formal pre-registered protocol, and two-thirds did not explicitly identify primary outcomes.

“No RCTs had a formal pre-registered protocol and two thirds did not explicitly identify primary outcomes (See Supplementary Table 6 on Risk of Bias Tool83).”

• cites [83] Higgins, J. P. T. et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 343, d5928 (2011). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

It was common for cluster RCTs to have too few clusters to achieve balance between intervention and control groups, and in some it was unclear whether clustering was adequately dealt with in the analysis.

“It was common for cluster RCTs to have too few clusters to achieve balance between intervention and control groups and in some it was unclear whether clustering was adequately dealt with in the analysis85.”

• cites [85] Leyrat, C., Morgan, K., Leurent, B. & Kahan, B. Cluster randomized trials with a small number of clusters: which analyses should be used? Int. J. Epidemiol. 47, 321–331 (2018). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The effect estimates show underlying heterogeneity, with 95% prediction intervals generally consistent with negative, null, and positive effects.

“We urge caution in interpreting this overall finding as unequivocally positive, given the potential for small study (publication) bias that may over-estimate the true effects, and the underlying heterogeneity of effect estimates as shown in 95% prediction intervals that were generally consistent with negative, null and positive effects.”

May combine two claims.

This review clearly demonstrated large between-study heterogeneity, with 95% prediction intervals consistent with negative, null, and positive effects among sub-domains such as literacy and numeracy.

“This review clearly demonstrated large between-study heterogeneity from 95% prediction intervals that were consistent with negative, null and positive effects among sub-domains such as literacy and numeracy.”

May combine two claims.

The quality of RCTs was not ideal, and reporting of some details was poor or even absent.

“The quality of RCTs was not ideal and reporting of some details was poor or even absent.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The review sought to examine whether studies using more representative population-based samples tended to generate smaller effect estimates, but the number of population-based samples in this field is rather small.

“For instance we sought to examine whether studies that used more representative population-based samples tended to generate smaller effect estimates, but the number of population-based samples in this field is actually rather small.”

May combine two claims.

Ninety-two percent of RCTs did not adequately report randomisation procedures, 81% did not report concealment of allocation processes and participant flow, and most failed to address missing data.

“Ninety-two percent of RCTs did not adequately report randomisation procedures, 81% did not report concealment of allocation processes and participant flow, and most failed to address missing data.”

May combine two claims.

Studies typically began at age 4-5, with median follow-up of about one year, and very few studies had follow-up beyond age 10, so there is very little evidence addressing medium- to longer-term outcomes.

“Studies typically began at age 4-5, with median follow-up of about one year, and with very few studies with follow-up beyond age 10, there is very little evidence addressing effects on medium to longer-term outcomes.”

May combine two claims.

There were only four studies in which the review could assess evidence for fade-out, and results were inconclusive.

“There were only four studies in which we could assess evidence for fade out and results were inconclusive.”

Classified as probably not a substantial claim.

Poor reporting made it difficult to fully assess study quality.

“Poor reporting made it difficult to fully assess study quality and we strongly encourage researchers, journal editors and reviewers to use tools such as the CONSORT statement (http://www.consort-statement.org/) for reporting, and for RCTs to be pre-registered.”

May combine two claims.

The overall meta-analysed effects from EQIs may be over-estimates that include a null effect.

“This suggests the overall meta-analysed effects from EQIs reported here may be over-estimates that include a null effect.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The review assessed risk of bias in EQIs using a Risk of Bias tool.

“some of the EQIs included in better evidence did not receive high quality ratings according to the Risk of Bias tool (Supplementary Table 6).”

Classified as probably not a substantial claim.

Effect sizes from EQIs and observational studies would only be comparable if the EQI induced a SD change in the particular non-cognitive skill.

“Furthermore, (as pointed out by a reviewer) effect sizes from EQIs and observational studies would only be comparable if the EQI induced a SD change in the particular non-cognitive skill.”

May combine two claims.

Interventions to improve non-cognitive skills may directly improve outcomes or indirectly through cognitive ability or other mechanisms.

“Interventions to improve non-cognitive skills may directly improve outcomes7,15, or indirectly, through cognitive ability or other mechanisms.”

• cites [7] Heckman, J. J. & Kautz, T. Hard evidence on soft skills. Labour Econ. 19, 451–464 (2012). sourceNot checked yet

  Not checked in the free run.

• cites [15] Heckman, J. J. & Kautz, T. Fostering and Measuring Skills: Interventions that Improve Character and Cognition (National Bureau of Economic Research, 2013). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Heckman et al. reported that while the Perry Preschool intervention initially increased IQ, these increases were not maintained to age 7-8 years.

“In analysing the long term outcomes of the trial, Heckman et al.31, reported that while initially the intervention increased IQ these increases were not maintained to age 7-8 years.”

• cites [31] Heckman, J. J., Pinto, R. & Savelyev, P. Understanding the mechanisms through which an early childhood program boosted adult outcomes. Am. Econ. Rev. 103, 2052–2086 (2013). sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

If non-cognitive skills are not developed early in life, compensating later is extremely difficult and expensive, reducing returns on later investments.

“It is argued that if these skills are not developed early in life, then it can be extremely difficult and expensive to compensate later in life, and this reduces returns on later investments6.”

• cites [6] Heckman, J. J. Skill formation and the economics of investing in disadvantaged children. Science 312, 1900–1902 (2006). sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The review classified observational studies as better evidence only if they made reasonable attempts to control for confounding (endogeneity) bias.

“observational studies that made reasonable attempts to control for confounding (endogeneity) bias.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Governments in many countries invest significant resources to bolster early life capabilities to sustain future economic productivity and social inclusion, especially for disadvantaged children.

“It is also of great policy interest to governments in many countries3,4, who wish to sustain future economic productivity and social inclusion, by investing significant resources to bolster the development of human capabilities in early life5, especially for disadvantaged children.”

• cites [3] Skills for Social Progress: The Power of Social and Emotional Skills (OECD, Paris, 2015).Not checked yet

  Not checked in the free run.

• cites [4] Institute of Education The Impact of Non-cognitive Skills for Young People (UK Cabinet Office, 2013).Not checked yet

  Not checked in the free run.

• cites [5] Allen, G. Early Intervention: the Next Steps. An Independent Report to Her Majesty’s Government (UK Cabinet Office, 2011).Not checked yet

  Not checked in the free run.

May combine two claims.

That meta-analysis did not exclude poor-quality studies and did not explore potential small-study bias.

“However, this meta-analysis did not exclude poor quality studies, and did not explore potential for small study bias.”

• cites [47] Allan, N. P., Hume, L. E., Allan, D. M., Farrington, A. L. & Lonigan, C. J. Relations between inhibitory control and the development of academic skills in preschool and kindergarten: a meta-analysis. Dev. Psychol. 50, 2368–2379 (2014). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Heckman and colleagues argued that the Perry Preschool intervention resulted in better outcomes not by increasing intelligence, but through fostering non-intelligence based socio-emotional personality skills.

“While we can find no evidence that the Perry Preschool Program deliberately set out to influence non-cognitive abilities, Heckman and colleagues argued that the intervention resulted in better outcomes for the participants not as a result of increasing their intelligence, but through fostering non-intelligence based socio-emotional “personality” skills31 (p. 2503).”

• cites [31] Heckman, J. J., Pinto, R. & Savelyev, P. Understanding the mechanisms through which an early childhood program boosted adult outcomes. Am. Econ. Rev. 103, 2052–2086 (2013). sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The HighScope Perry Preschool Program suggests other mechanisms may be involved.

“However, the HighScope Perry Preschool Program started in 1962 suggests other mechanisms may be involved30,31.”

• cites [30] Schweinhart, L. J. et al. Lifetime Effects: The High/Scope Perry Preschool Study through Age 40 (High/Scope Press, Ypsilanti, 2005).Not checked yet

  Not checked in the free run.

• cites [31] Heckman, J. J., Pinto, R. & Savelyev, P. Understanding the mechanisms through which an early childhood program boosted adult outcomes. Am. Econ. Rev. 103, 2052–2086 (2013). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Heckman has argued that interventions to develop non-cognitive skills, especially in disadvantaged young children, have the potential for high rates of return due to positive effects in multiple life domains.

“Heckman has argued that interventions to develop these skills, especially in disadvantaged young children have the potential for high rates of return due to their positive effects in multiple life domains6.”

• cites [6] Heckman, J. J. Skill formation and the economics of investing in disadvantaged children. Science 312, 1900–1902 (2006). sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Non-cognitive traits are thought to be especially malleable in early life.

“They have been the target of interventions, especially in early life when these traits are thought to be especially malleable16, and for disadvantaged children, who may benefit most6.”

• cites [16] Diamond, A. & Lee, K. Interventions shown to aid executive function development in children 4 to 12 years old. Science 333, 959–964 (2011). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Our own longitudinal analyses in three large population-based cohorts in the UK and Australia showed both cognitive abilities and non-cognitive skills were important in explaining socioeconomic inequalities in academic achievement early in life, and non-cognitive skills were only weakly associated with cognitive ability.

“For instance, our own longitudinal analyses in three large population-based cohorts in the UK and Australia showed both cognitive abilities and non-cognitive skills were important in explaining socioeconomic inequalities in academic achievement early in life and that non-cognitive skills were only weakly associated with cognitive ability17.”

• cites [17] Pearce, A. et al. Do early life cognitive ability and self-regulation skills explain socio-economic inequalities in academic achievement? An effect decomposition analysis in UK and Australian cohorts. Soc. Sci. Med. 165, 108–118 (2016). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Disadvantaged children may benefit most from interventions to improve non-cognitive skills.

“They have been the target of interventions, especially in early life when these traits are thought to be especially malleable16, and for disadvantaged children, who may benefit most6.”

• cites [6] Heckman, J. J. Skill formation and the economics of investing in disadvantaged children. Science 312, 1900–1902 (2006). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Non-cognitive abilities have been associated with mental health, physical health, school readiness and academic achievement, crime, employment and income, and mortality.

“Non-cognitive abilities have been associated with a number of shorter and longer-term outcomes including mental health18,19, physical health20, school readiness and academic achievement21,22, crime23, employment and income10, and mortality24.”

• cites [18] Eisenberg, N. et al. Relations among maternal socialization, effortful control, and maladjustment in early childhood. Dev. Psychopathol. 22, 507–525 (2010). sourceNot checked yet

  Not checked in the free run.

• cites [19] Fergusson, D. M., Boden, J. M. & Horwood, L. Childhood self-control and adult outcomes: results from a 30-year longitudinal study. J. Am. Acad. Child Adolesc. Psychiatry 52, 709–717.e1 (2013). sourceNot checked yet

  Not checked in the free run.

• cites [20] Evans, G. W., Fuller-Rowell, T. E. & Doan, S. N. Childhood cumulative risk and obesity: the mediating role of self-regulatory ability. Pediatrics 129, e68–e73 (2012). sourceNot checked yet

  Not checked in the free run.

• cites [21] Blair, C. & Razza, R. P. Relating effortful control, executive function, and false belief understanding to emerging math and literacy ability in kindergarten. Child Dev. 78, 647–663 (2007). sourceNot checked yet

  Not checked in the free run.

• cites [22] Mischel, W., Shoda, Y. & Peake, P. K. The nature of adolescent competencies predicted by preschool delay of gratification. J. Pers. Soc. Psychol. 54, 687–696 (1988). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Because the review deliberately selected higher-quality observational studies with more stringent confounder controls, the true effects of non-cognitive skills may be closer to those from higher-quality observational studies, which may include a null effect.

“We deliberately selected higher quality observational studies with more stringent controls for confounding, so it is possible that true effects of non-cognitive skills are actually closer to those from higher quality observational studies that may include a null effect.”

May combine two claims.

Children who received the Perry Preschool intervention went on to have more successful lives in adulthood including greater labour market success, reduced crime involvement and better health.

“Despite this, children who received the intervention went on to enjoy more successful lives in adulthood including greater labour market success, reduced crime involvement and better health30,33,34.”

• cites [30] Schweinhart, L. J. et al. Lifetime Effects: The High/Scope Perry Preschool Study through Age 40 (High/Scope Press, Ypsilanti, 2005).Not checked yet

  Not checked in the free run.

• cites [33] Schweinhart, L. J., Weikart D. P. & Barnes, H. V. Significant Benefits: The High/Scope Perry Preschool Study Through Age 27 (Monographs of the High/Scope Educational Research Foundation) (High/Scope Press, Ypsilanti, 1993).Not checked yet

  Not checked in the free run.

• cites [34] Heckman, J., Moon, S. H., Pinto, R., Savelyev, P. & Yavitz, A. Analyzing social experiments as implemented: a reexamination of the evidence from the HighScope Perry Preschool Program. Quant. Econom. 1, 1–46 (2010). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Observational studies often over-estimate effects found in large, well-designed RCTs.

“This is the opposite of what is often seen where observational studies over-estimate effects found in large, well designed RCTs.”

• cites [45] Fewell, Z., Davey Smith, G. & Sterne, J. A. The impact of residual and unmeasured confounding in epidemiologic studies: a simulation study. Am. J. Epidemiol. 166, 646–655 (2007). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Children's cognitive ability (intelligence or IQ) associates with academic achievement and later success in adulthood.

“It is widely accepted that children’s cognitive ability (i.e., intelligence or IQ) associates with academic achievement and later success in adulthood26–29.”

• cites [26] Deary, I. J., Whiteman, M. C., Starr, J. M., Whalley, L. J. & Fox, H. C. The impact of childhood intelligence on later life: following up the Scottish mental surveys of 1932 and 1947. J. Pers. Soc. Psychol. 86, 130–147 (2004). sourceNot checked yet

  Not checked in the free run.

• cites [27] Fergusson, D. M., John Horwood, L. & Ridder, E. M. Show me the child at seven II: childhood intelligence and later outcomes in adolescence and young adulthood. J. Child Psychol. Psychiatry 46, 850–858 (2005). sourceNot checked yet

  Not checked in the free run.

• cites [28] Kuh, D., Richards, M., Hardy, R., Butterworth, S. & Wadsworth, M. E. Childhood cognitive ability and deaths up until middle age: a post-war birth cohort study. Int. J. Epidemiol. 33, 408–413 (2004). sourceNot checked yet

  Not checked in the free run.

• cites [29] Whalley, L. J. & Deary, I. J. Longitudinal cohort study of childhood IQ and survival up to age 76. BMJ 322, 819–822 (2001). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Bowles and Gintis argued that schooling does not make children more intelligent, but socializes them into and rewards characteristics and behaviours valued in the labour market.

“They argued that schooling does not make children more intelligent, rather, it socializes them into, and rewards, certain characteristics and behaviours that are valued in the labour market.”

• cites [1] Bowles, S. & Gintis, H. Schooling in Capitalist America: Educational Reform and the Contradictions of Economic Life (Basic Books, New York, 1976).Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

This review covers the entire published interdisciplinary research field describing intentional efforts and observational analogues of interventions to improve development of non-cognitive skills, although most evidence comes from rich countries, especially the USA.

“This review covers the entire published inter-disciplinary research field describing intentional efforts and observational analogues of interventions to improve development of non-cognitive skills, albeit with most evidence coming from rich countries, especially the USA.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

Each approach points to different conceptualisations of where psychosocial problems arise and how, where, and with whom to intervene.

“Each approach points to different conceptualisations of where psychosocial problems arise and for how, where and whom to intervene (e.g. teachers, psychologists, community nurses or social workers).”

Probably not a substantial claim: may not pass S2 (checkable).

The scope of the review should minimise cherry-picking of results to bolster a particular concept, theory, or intervention.

“The scope of the review should minimise ‘cherry picking’ of results to bolster a particular concept, theory or intervention.”

Classified as probably not a substantial claim.

The authors a priori created criteria for bias based on well-established procedures, including quality appraisal tools, evidence hierarchies, directed acyclic graphs, and content knowledge about potential sources of confounding and selection bias.

“We a priori created criteria for bias based on well-established procedures including quality appraisal tools, evidence hierarchies, directed acyclic graphs and content knowledge about potential sources of confounding and selection bias.”

Probably not a substantial claim: may not pass S3 (load-bearing).

This diversity reflected different approaches to improving children's psychosocial outcomes, such as supporting parents or helping teachers manage classroom behaviour.

“This diversity reflected different approaches to improving children’s psychosocial outcomes, such as supporting parents or helping teachers to manage classroom behaviour.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The compilation of 554 publications was systematic, but the assessment of evidence quality was based on the authors' judgement of potential bias.

“The compilation of 554 publications was systematic but our assessment of the quality of the evidence is based on our judgement of the potential for bias.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Although the quality assessment involves subjective judgement, the authors are confident that any other reasonable assessment of evidence quality would not change the overall conclusions.

“While this involves an element of subjective judgement, we are confident that any other reasonable assessment of the quality of evidence would not change the overall conclusions presented here.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

This exception highlights the potential value of these comparisons.

“This exception highlights the potential value of these comparisons.”

Probably not a substantial claim: may not pass S3 (load-bearing).

It is possible that some relevant articles were not included, even though the review undertook an extensive search including multiple databases, numerous search terms, contacting authors, and hand searching reference lists.

“It is possible that some relevant articles were not included in this review, even though we undertook an extensive search that included multiple databases, numerous search terms, contacting authors of potentially-relevant papers, and hand searching reference lists of published papers.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The variable effects on externalising problems could be due to differences in intervention focus, mode of delivery, or implementation fidelity problems in larger trials.

“These variable effects could be due to differences in the focus of intervention, mode of delivery (parent, teacher or both), or problems with implementation fidelity in larger trials.”

May combine two claims.

Previous reviews of some aspects of non-cognitive skills have not been systematic in covering the entire literature or included screening for evidence quality.

“While there have been reviews of some aspects of non-cognitive skills,3,4,14,15,47,63 none have been systematic in covering the entire literature, or included screening for evidence quality.”

• cites [3] Skills for Social Progress: The Power of Social and Emotional Skills (OECD, Paris, 2015).Not checked yet

  Not checked in the free run.

• cites [4] Institute of Education The Impact of Non-cognitive Skills for Young People (UK Cabinet Office, 2013).Not checked yet

  Not checked in the free run.

• cites [14] Borghans, L., Duckworth, A. L., Heckman, J. J. & Ter Weel, B. The economics and psychology of personality traits. J. Hum. Resour. 43, 972–1059 (2008).Not checked yet

  Not checked in the free run.

• cites [15] Heckman, J. J. & Kautz, T. Fostering and Measuring Skills: Interventions that Improve Character and Cognition (National Bureau of Economic Research, 2013). sourceNot checked yet

  Not checked in the free run.

• cites [47] Allan, N. P., Hume, L. E., Allan, D. M., Farrington, A. L. & Lonigan, C. J. Relations between inhibitory control and the development of academic skills in preschool and kindergarten: a meta-analysis. Dev. Psychol. 50, 2368–2379 (2014). sourceNot checked yet

  Not checked in the free run.

• cites [63] Diamond, A. Executive functions. Annu. Rev. Psychol. 64, 135–168 (2013). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

This may suggest that more holistic programs including multi-dimensional content may better support overall child development and broad-based benefits.

“which may suggest that more holistic programs including multi-dimensional content, may better support overall child development and broad-based benefits.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Small p values on the Egger regression coefficient suggest the presence of small study bias that may produce larger effects.

“Small p values on the Egger regression coefficient suggest the presence of small study bias that may produce larger effects.”

• AssumptionThe authors state this as an assumption: The y intercept of Egger regression is expected to be zero if there is an even spatial spread of studies within the funnel.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

When eligibility could not be determined from the title or abstract, the full text was reviewed, and unclear eligibility was resolved by group consensus.

“Where eligibility was not able to be determined by the title or abstract the full text was reviewed, and when eligibility was unclear this was resolved by group consensus.”

Classified as probably not a substantial claim.

Their assessment of better quality evidence relied on a subjective judgment of the risk of bias from confounding.

“Thus, our assessment of better quality evidence relied on a subjective judgment of the risk of bias from confounding.”

• AssumptionThe authors state this as an assumption: Conditional exchangeability between exposed and unexposed is assumed for interpreting causal effects from observational data.

Probably not a substantial claim: may not pass S3 (load-bearing).

Cohen cautioned that generic application of effect sizes to all research fields was an operation fraught with many dangers.

“He cautioned that such generic application of sizes of effects to all research fields was”

• cites [92] Cohen, J. Statistical Power Analysis for the Behavioral Sciences 2nd edn (Lawrence Erlbaum Associates, Hillsdale, 1988).Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Deciding whether an intervention is worth implementing will depend not only on its benefits but also on its costs, discount rate, scalability, and other considerations.

“Deciding whether an intervention is worth implementing will depend not only on its benefits, but also its costs, discount rate, scalability and a range of other potential considerations.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Even though these metrics are widely used as reference points, Cohen did not intend them to be used as absolutes.

“Even though these metrics are widely, often ritualistically, used as reference points, Cohen did not intend them to be used as absolutes.”

• cites [92] Cohen, J. Statistical Power Analysis for the Behavioral Sciences 2nd edn (Lawrence Erlbaum Associates, Hillsdale, 1988).Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The impression is that new RCTs were not being explicitly justified on the basis of what was already known from existing RCTs.

“so the impression is that new RCTs were not being explicitly justified on the basis of what was already known from existing RCTs.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The first 300 references were screened as a group by all authors, and subsequent references were screened independently, with Kappa values for agreement above 0.80.

“To ensure consistency of searching, the first 300 references were searched as a group by all authors and subsequent references were searched independently (Kappa values for agreement were >0.80).”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Lipsey et al. present a sophisticated understanding of interpreting effect sizes.

“Lipsey, et. al. present a sophisticated understanding of interpreting effect sizes96.”

• cites [96] Lipsey, M. W. et al. Translating the Statistical Representation of the Effects of Education Interventions into More Readily Interpretable Forms (US Department of Education, 2012).Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Interventions that have small effects on average across the population and are cheap could be very cost-effective, particularly if they influence long-term outcomes in adulthood.

“Interventions that have small effects on average across the population and that are cheap could be very cost effective, particularly if they influence long term outcomes in adulthood.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

For example, if the best interventions available to improve a particular outcome found reliable effects of 0.2 SD when trialled in large population-based samples, then a novel intervention finding the same effect might be considered large.

“For example, if the best interventions available to improve a particular outcome, found reliable effects of 0.2SD when trialled in large population based samples, then a novel intervention finding the same effect might be considered large.”

Classified as probably not a substantial claim. May combine two claims.

The authors JL, LS and AS reviewed all eligible studies and rated their evidence quality as better, weak, or poor based on study design and confounding adjustment.

“The authors JL, LS and AS reviewed all eligible studies and rated their evidence quality as ‘better, weak, or poor’ on the basis of study design and confounding adjustment (Table 1).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

ND independently reviewed data extraction for 15% of all studies, including all intervention studies, and consensus was reached for discrepancies.

“LS, JL, CC, AS, TG and TN extracted data from articles. ND independently (i.e., blinded to assessments of other authors) reviewed the data extraction for 15% of all studies, including all intervention studies, and consensus was reached for the very small number of discrepancies.”

Classified as probably not a substantial claim.

To overcome the problem of missing variance or exact p values, they made assumptions about p values to calculate confidence intervals.

“To overcome this problem for conducting meta-analyses using inverse variance weighting we were forced to make assumptions about p values to calculate confidence intervals.”

• AssumptionThe authors state this as an assumption: When p was reported as less than a specific value, p was assumed to equal that value (e.g., p<0.01 assumed p=0.01) for calculating confidence intervals.

May combine two claims.

Larger effects observed in smaller RCTs may be due to several factors, including publication bias favouring positive results, true heterogeneity due to differing baseline risks, implementation difficulties in maintaining intensity and fidelity in larger community settings, and poorer methodological design of smaller studies.

“Larger effects observed in smaller RCTs may be due to several factors including publication bias favouring positive results, true heterogeneity of effects due to differing baseline risks in different intervention populations, implementation difficulties in maintaining intervention intensity and fidelity in larger community settings, and poorer methodological design of smaller studies81.”

• cites [81] Egger, M., Davey Smith, G., Schneider, M. & Minder, C. Bias in meta-analysis detected by a simple, graphical test. BMJ 315, 629–634 (1997). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Use of standardized effect measures has been criticized in several disciplines.

“Use of such standardized effect measures has been criticized in several disciplines.”

• cites [93] Greenland, S., Maclure, M., Schlesselman, J. J., Poole, C. & Morgenstern, H. Standardized regression coefficients: a further critique and review of some alternatives. Epidemiology 2, 387–392 (1991). sourceNot checked yet

  Not checked in the free run.

• cites [94] King, G. How not to lie with statistics: avoiding common mistakes in quantitative political science. Am. J. Polit. Sci. 30, 666–687 (1986). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In political science, King argued that if apples and oranges cannot be meaningfully compared on the original outcome measurement scale, this lack of comparability is not improved by comparing standardized fruit.

“In political science, King argued that if apples and oranges cannot be meaningfully compared on the original outcome measurement scale then this lack of comparability is not improved by comparing standardized fruit94.”

• cites [94] King, G. How not to lie with statistics: avoiding common mistakes in quantitative political science. Am. J. Polit. Sci. 30, 666–687 (1986). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

For RCTs, risk of bias was assessed using the Cochrane Collaboration Risk of Bias Assessment Tool.

“For RCTs, the risk of bias was assessed using the Cochrane Collaboration Risk of Bias Assessment Tool83 (Supplementary Table 6).”

• cites [83] Higgins, J. P. T. et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 343, d5928 (2011). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Data extraction used a standardised form created by the authors.

“The following information was systematically extracted from each article using a standardised form created by the authors.”

Classified as probably not a substantial claim.

Only results from studies that met the definition of 'Better evidence' are reported.

“Here we only report results from studies that met the definition of ‘Better evidence’.”

• AssumptionThe authors state this as an assumption: Conditional exchangeability between exposed and unexposed is assumed for interpreting causal effects from observational data.

Classified as probably not a substantial claim. May combine two claims.

Publications that made no attempt to statistically control for common causes of exposure and outcome were rated as poor because likelihood of confounding bias was high, and so could not inform assessment of likely causal effects.

“Publications that made no attempt to statistically control for common causes of exposure and outcome were rated as ‘poor’ because the likelihood of confounding (endogeneity) bias was high, and so these publications could not inform any assessment of likely causal effects of non-cognitive skills on outcomes.”

• AssumptionThe authors state this as an assumption: Conditional exchangeability between exposed and unexposed is assumed for interpreting causal effects from observational data.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Therefore, the traditional labelling of an intervention as having small effects (~0.2 SD) is inappropriate because it fails to consider the research, policy, and practice context within which the intervention is situated.

“Therefore, the traditional labelling of an intervention as having “small” effects (~0.2SD) is inappropriate because it fails to consider the research, policy and practice context within which the intervention is situated.”

Classified as probably not a substantial claim. May combine two claims.

Causal interpretation of an exposure effect from observational data depends on several assumptions, one being conditional exchangeability between exposed and unexposed.

“We adopted a “potential outcomes approach” to conceptualizing confounding where the interpretation of a ‘causal’ effect of an exposure estimated from observational data relies on several assumptions101.”

• cites [101] Little, R. J. & Rubin, D. B. Causal effects in clinical and epidemiological studies via potential outcomes: concepts and analytical approaches. Annu. Rev. Public Health 21, 121–145 (2000). sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

To reduce bias from studies reporting multiple measures of the same outcome, they obtained an overall estimate across all reported measures.

“To reduce bias that may have arisen from studies reporting multiple measures of the same outcome, we obtained an overall estimate across all of the reported measures.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In epidemiology, Greenland et al. have argued that standardizing effects, rather than making them more comparable across studies, serves to confound comparison by the observed standard deviation, which is often an artefact of the study sample, particularly for homogeneous convenience samples.

“In epidemiology, Greenland et al.93, have argued that the process of standardizing effects, rather than making them more comparable across studies, simply serves to confound that comparison by the observed standard deviation, which is often an artefact of the study sample, particularly for homogenous convenience samples.”

• cites [93] Greenland, S., Maclure, M., Schlesselman, J. J., Poole, C. & Morgenstern, H. Standardized regression coefficients: a further critique and review of some alternatives. Epidemiology 2, 387–392 (1991). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Another way of norming effect size may be to consider the size of intervention effects against secular change in an outcome over time.

“Another way of norming effect size may be to consider the size of intervention effects against secular change in an outcome over time.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The systematic review protocol was preregistered with PROSPERO in December 2013.

“The systematic review protocol was preregistered with the International Prospective Register for Systematic Reviews (PROSPERO, CRD42013006566) in December 2013 and is available at: http://www.crd.york.ac.uk/PROSPERO/.”

Probably not a substantial claim: may not pass S3 (load-bearing).

They examined asymmetry of published evidence using funnel plots and Egger regression.

“We examined asymmetry of the published evidence by generating funnel plots of effect size against inverse of study size separately for EQIs and observational studies (Supplementary Figures 20a-23b) and calculated the summary Egger regression coefficient and p value indicating the degree of asymmetry81.”

• cites [81] Egger, M., Davey Smith, G., Schneider, M. & Minder, C. Bias in meta-analysis detected by a simple, graphical test. BMJ 315, 629–634 (1997). sourceNot checked yet

  Not checked in the free run.

• AssumptionThe authors state this as an assumption: The y intercept of Egger regression is expected to be zero if there is an even spatial spread of studies within the funnel.

Probably not a substantial claim: may not pass S3 (load-bearing).

Searches were not restricted by language.

“Searches were not restricted by language.”

Probably not a substantial claim: may not pass S1 (asserted), S2 (checkable), S3 (load-bearing).

It is frequently argued that non-cognitive skills developed in childhood have major impacts on long-term adult outcomes.

“However, it is frequently argued that non-cognitive skills developed in childhood have major impacts on long-term adult outcomes6.”

• cites [6] Heckman, J. J. Skill formation and the economics of investing in disadvantaged children. Science 312, 1900–1902 (2006). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Thus, interventions with short-term effects but few detectable long-term effects are unlikely to be cost-effective.

“Thus, interventions which have short term effects, but few detectable long-term effects are unlikely to be cost-effective.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

For example, they show that the secular growth in reading from kindergarten to grade one in the US is estimated to be about 1.5 SD, and by grades 4-5 this growth has declined to about 0.4 SD per year.

“For example, they show that the secular growth in reading from kindergarten to grade one in the US is estimated to be about 1.5 SD. By grades 4-5 this growth has declined to about 0.4 SD per year.”

• cites [96] Lipsey, M. W. et al. Translating the Statistical Representation of the Effects of Education Interventions into More Readily Interpretable Forms (US Department of Education, 2012).Not checked yet

  Not checked in the free run.

May combine two claims.

Overall, there is some evidence supporting a role for non-cognitive skills in better academic achievement, psychosocial, cognitive, and language outcomes, but these effects are highly heterogeneous as they relate to the shorter-term outcomes examined in this review.

“Overall, yes, there is some evidence supporting a role for non-cognitive skills in better academic achievement, psychosocial, cognitive and language, but these effects are highly heterogeneous as they relate to the shorter-term outcomes examined in this review.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors avoided labelling effect sizes as small (~0.2 SD), medium (~0.5 SD), or large (~0.8 SD) according to Cohen's suggestions.

“We have avoided labelling effect sizes as “small (~0.2SD)”, “medium (~0.5SD)”, or “large (~0.8SD)” according to Cohen’s suggestions92.”

• cites [92] Cohen, J. Statistical Power Analysis for the Behavioral Sciences 2nd edn (Lawrence Erlbaum Associates, Hillsdale, 1988).Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Study outcomes were not included as search terms, to capture all published outcomes associated with non-cognitive abilities.

“Study outcomes were not included as search terms to capture all published outcomes associated with non-cognitive abilities.”