Internet interventions for mental health in university students: A systematic review and meta‐analysis

May combine two claims.

For all outcomes, effects were significantly higher when interventions were compared with passive controls than with active controls (all p < 0.01); passive-control effects were depression g = 0.27 (95% CI 0.17-0.36, NNT = 6.58), anxiety g = 0.39 (95% CI 0.25-0.52, NNT = 4.59), and stress g = 0.33 (95% CI 0.15-0.51, NNT = 5.43), while active-control effects were depression g = -0.06 (95% CI -0.23-

“For all outcomes, effects were significantly higher (all p < 0.01) when interventions were compared with passive controls (depression: g = 0.27, 95% CI [0.17, 0.36], NNT = 6.58; anxiety: g = 0.39, 95% CI [0.25, 0.52], NNT = 4.59; stress: g = 0.33, 95% CI [0.15, 0.51], NNT = 5.43)”

• Also stated in: § 4 | CONCLUSIONS · plain

  “than active control groups (depression: g = -0.06,95% CI [-0.23, 0.11], NNH = 29.41;anxiety: g = 0.02, 95% CI [-0.25, 0.29], NNT = 83.33;stress: g = -0.19,95% CI [-0.37, 0.01], NNH = 9.43).”

May combine two claims.

The authors found no indication that guidance, risk of bias, or use of convenience samples was significantly related to effect size (all p ≥ 0.05).

“We found no indication that guidance, risk of bias, or employment of convenience samples were significantly related to effect size (all p ≥ 0.05).”

May combine two claims.

Across 31 comparisons, Internet interventions for depression produced a pooled effect of g = 0.18 (95% CI 0.08 to 0.27), with an NNT of 9.80.

“We could compare the effect of Internet interventions on symptoms of depression with control groups in 31 comparisons.The overall effect size was g = 0.18 (95% CI [0.08, 0.27]), which corresponds to an NNT of 9.80.”

May combine two claims.

Across four comparisons, Internet interventions had a pooled effect on well-being of g = 0.15 (95% CI -0.20 to 0.50), corresponding to NNT = 11.90, and this effect was not statistically significant (p = 0.259).

“A pooled effect of g = 0.15 (95% CI [-0.20, 0.50]) was found for the effect of Internet interventions on well-being (four comparisons).This corresponds with an NNT of 11.90, but the effect was not statistically significant (p = 0.259).”

May combine two claims.

Two studies evaluated Internet interventions for insomnia against controls; they were not pooled in meta-analysis, and both had low risk-of-bias ratings.

“two studies evaluated the effect of Internet interventions for insomnia relative to controls.These studies were not pooled in meta-analysis.Both interventions had a low risk of bias rating.”

May combine two claims.

Effects were lowest when participants were recruited through a study subject pool: depression g = 0.04 (95% CI -0.10-0.17, NNT = 45.45) and stress g = -0.22 (95% CI -0.70-0.27, NNH = 8.06); these effects were not significant (both p > 0.05).

“Effects were lowest when participants were recruited through a study subject pool (g = 0.04, 95% CI [-0.10, 0.17], NNT = 45.45;stress: g = -0.22,95% CI [-0.70, 0.27], NNH = 8.06).These effects were not significant (both p > 0.05).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Eleven trials (22.92%) used active control conditions (12 comparisons).

“whereas 11 (22.92%)used active control conditions (12 comparisons).”

May combine two claims.

In an influence analysis that left out one study, the eating disorder symptom effect was slightly higher at g = 0.61 (95% CI 0.35-0.68, NNT = 2.99), and heterogeneity became non-significant (I2 = 39%, 95% CI 0-73, p = 0.120).

“A slightly higher effect of g = 0.61 (95% CI [0.35, 0.68], NNT = 2.99) was found when leaving out one study in the influence analysis, and heterogeneity became non-significant (I 2 = 39%; 95% CI [0-73], p = 0.120).”

May combine two claims.

The prediction interval for the functioning effect included only positive values and ranged from g = 0.02 to 0.81.

“The prediction interval only included positive values and ranged from g = 0.02 to 0.81.”

May combine two claims.

For depression, effects were highest for interventions lasting 4-8 weeks (g = 0.31, 95% CI 0.13-0.49, NNT = 5.75) compared with shorter (g = 0.09, 95% CI -0.02-0.21, NNT = 20) or longer (g = 0.13, 95% CI -0.43-0.69, NNT = 13.51) programs (p = 0.027).

“For depression, effects were highest for interventions between 4 and 8 weeks in length (g = 0.31, 95% CI [0.13, 0.49], NNT = 5.75) compared with shorter (g = 0.09, 95% CI [-0.02, 0.21], NNT = 20) or longer (g = 0.13, 95% CI [-0.43, 0.69], NNT = 13.51)programs (p = 0.027).”

May combine two claims.

Effects were higher for web-based recruitment: depression g = 0.30 (95% CI 0.25-0.57, NNT = 5.95) and stress g = 0.63 (95% CI -0.05-1.31, NNT = 2.91).

“Effects were higher for web-based recruitment (g = 0.30, 95% CI [0.25, 0.57], NNT = 5.95; stress: g = 0.63, 95% CI [-0.05, 1.31], NNT = 2.91).”

May combine two claims.

In an influence analysis that left out one study, the functioning effect was similar at g = 0.45 (95% CI 0.10-0.81, NNT = 4), but heterogeneity became non-significant (I2 = 31%, 95% CI 0-69, p = 0.180).

“A similar effect emerged when leaving one study out in the influence analysis (g = 0.45; 95% CI [0.10, 0.81], NNT = 4), but heterogeneity became non-significant (I 2 = 31%; 95% CI [0-69], p = 0.180).”

May combine two claims. The quote only partly states this claim.

The pooled effect for functioning outcomes across nine comparisons was g = 0.41 (95% CI 0.26-0.56), corresponding to NNT = 4.39.

“g = 0.41 (95% CI [0.26, 0.56]), which corresponds with an NNT of 4.39.”

May combine two claims.

For anxiety, the pooled effect across 27 comparisons was g = 0.27 (95% CI 0.13 to 0.40), corresponding to an NNT of 6.58.

“The overall effect size for anxiety (27 comparisons) was g = 0.27 (95% CI [0.13, 0.40]), which corresponds with an NNT of 6.58.”

May combine two claims.

For depression and anxiety, effects were higher for interventions based on CBT principles: depression g = 0.28 (95% CI 0.15-0.40, NNT = 6.41) and anxiety g = 0.36 (95% CI 0.23-0.50, NNT = 5).

“For both target outcomes, effects were higher for interventions based on CBT principles (depression: g = 0.28, 95% CI [0.15, 0.40], NNT = 6.41; anxiety: g = 0.36, 95% CI [0.23, 0.50], NNT = 5).”

May combine two claims.

The pooled effect for eating disorder symptoms was g = 0.52 (95% CI 0.22-0.83), equal to NNT = 3.50.

“The pooled effect was g = 0.52 (95% CI [0.22, 0.83]).This equals an NNT of 3.50.”

May combine two claims.

For depression, the p curve analysis was inconclusive: the right-skewness test showed no evidential value, the flatness test was non-significant, and the corrected average effect estimate was d = 0.09.

“The results of the p curve analysis were inconclusive.The test for right-skewness did not indicate the presence of evidential value (p Full = 0.269, p Half = 0.427, k = 8; see Table S5).However, the test for flatness was not significant (p Full = 0.149, p Half = 0.981, and p Binomial = 0.423); p curve's estimate of the average true effect size was Cohen's d = 0.09.”

• AssumptionThe authors state this as an assumption: The p curve method assumes that publication bias arises because only statistically significant results are considered for publication.

Probably not a substantial claim: may not pass S3 (load-bearing).

Participant mean ages spanned 18.37 to 29 years.

“The mean age ranged from 18.37 to 29.”

The quote only partly states this claim.

For functioning outcomes, the p curve indicated the presence of evidential value.

“The p curve indicated the presence of evidential value.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Sample sizes ranged from 38 to 2,638 participants.

“Sample sizes ranged from N = 38 to 2,638.”

May combine two claims.

Effects were lower and non-significant for skill trainings (depression g = 0.04, 95% CI -0.23-0.30, NNT = 45.45; anxiety g = -0.06, 95% CI -0.46-0.35, NNH = 29.41) and other techniques (depression g = 0.10, 95% CI -0.01-0.21, NNT = 17.86; anxiety g = 0.03, 95% CI -0.43-0.50, NNT = 62.5).

“Effects were lower and non-significant (all p > 0.05) for skill trainings (depression: g = 0.04, 95% CI [-0.23, 0.30], NNT = 45.45;anxiety: g = -0.06,95% CI [-0.46, 0.35], NNH = 29.41)and other techniques (depression: g = 0.10, 95% CI [-0.01, 0.21], NNT = 17.86; anxiety: g = 0.03, 95% CI [-0.43, 0.50], NNT = 62.5).”

May combine two claims.

In an influence analysis that left out one study, the well-being effect became significant at g = 0.25 (95% CI 0.11-0.39, NNT = 7.14, p = 0.016), with between-study heterogeneity remaining low (I2 = 0%, 95% CI 0-0).

“A significant effect (g = 0.25; 95% CI [0.11, 0.39], NNT = 7.14; p = 0.016) was found when leaving out one study in the influence analysis, and the between-study heterogeneity remained low (I 2 = 0%; 95% CI [0-0]).”

May combine two claims.

Pooled effects of interventions compared with active controls were not significant for depression, anxiety, and stress (all p > 0.05).

“The pooled effects of interventions compared with active controls were not significant (all p > 0.05).”

May combine two claims.

The most frequent primary or target outcome was depression (26 studies), then anxiety (24), stress (16), disordered eating (9), well-being (4), and sleep (2).

“Depression was the target or primary outcome of 26 studies, followed by anxiety (k = 24), stress (k = 16), disordered eating (k = 9), well-being (k = 4), and sleep (k = 2).”

May combine two claims.

When only low risk-of-bias studies were pooled, the functioning effect was similar at g = 0.41 (95% CI 0.22-0.60, NNT = 4.39), with non-significant between-study heterogeneity (I2 = 54%, 95% CI 0-83, p = 0.07).

“We also found a similar effect (g = 0.41; 95% CI [0.22, 0.60], NNT = 4.39) with non-significant between-study heterogeneity (I 2 = 54%; 95% CI [0-83], p = 0.07) when we only pooled the effects of studies with a low risk of bias.”

May combine two claims.

Depression effects were higher in studies where no compensation was provided (g = 0.31, 95% CI 0.18-0.45, NNT = 5.75) than in studies that compensated participants (g = 0.08, 95% CI -0.05-0.20, NNT = 21.74).

“was provided (g = 0.31, 95% CI [0.18, 0.45], NNT = 5.75) compared with studies that compensated participants (g = 0.08, 95% CI [-0.05, 0.20], NNT = 21.74).”

The quote only partly states this claim.

Quality across the included randomized trials was generally suboptimal.

“The overall quality was suboptimal for some of the included RCTs.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The database search yielded 44,839 records.

“The database search yielded 44,839 records.”

May combine two claims.

Most interventions were delivered through a website (43 studies, 89.58%); three studies (6.25%) used e-mail, five (10.42%) used mobile-based components, and three (6.25%) used mobile apps.

“In 43 studies (89.58%), interventions were delivered through a website.In three studies (6.25%), intervention content was provided via e-mail.Mobile-based components were used in five studies (10.42%), of which three (6.25%)employed mobile apps.”

The quote only partly states this claim.

For functioning outcomes, no indications of publication bias were found.

“We found no indications of publication bias.”

May combine two claims.

For depression, the pooled effect in unselected samples was g = 0.09 (95% CI -0.05-0.23, NNT = 20) and did not attain statistical significance (p = 0.182).

“than in unselected samples (g = 0.09, 95% CI [-0.05, 0.23], NNT = 20).The pooled effect for interventions in unselected samples did not attain statistical significance (p = 0.182).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Seventy-four per cent of participants were female.

“Seventy-four per cent (n = 7,831) of all participants were female.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Student functioning outcomes were assessed in nine studies.

“Student functioning outcomes (e.g., work productivity and clinical impairment) were assessed in nine studies.”

May combine two claims.

Intervention technique significantly moderated effects on depression (p = 0.027) and anxiety (p = 0.018).

“Intervention technique moderated effects on depression (p = 0.027) and anxiety (p = 0.018).”

May combine two claims.

The depression result remained similar in sensitivity checks that removed outliers, varied how multiple comparisons were handled, limited to low-risk-of-bias trials, and assessed single-study influence.

“Similar effects emerged in all sensitivity analyses, including analyses in which outliers were removed, when only the highest, lowest, and combined effect of multiple comparisons was considered and when only studies with a low risk of bias rating were included, and in the influence analysis.This supports the robustness of this finding.”

May combine two claims.

Among passive controlled studies, most used wait lists (27 studies), followed by no intervention controls (6) and psychoeducational material without behaviour-change instructions (4).

“Among passive controlled studies, most employed wait lists (k = 27, 56.25%), followed by no intervention controls (k = 6, 12.5%) and provision of psychoeducational material without instructions for behaviour change (k = 4, 8.33%).”

May combine two claims.

Higher effects were also found when participants were not given any compensation and were not recruited through a study subject pool.

“Higher effects were also found when participants were not given any compensation and were not recruited through a study subject pool.”

May combine two claims.

Long-term effect data were reported for only a small proportion of included studies, and follow-up periods varied considerably, so such outcomes could not be pooled.

“Furthermore, as long-term effect data were only reported for a small proportion of the included studies, and follow-up periods varied considerably, we were not able to pool such outcomes.”

May combine two claims.

Subgroup analyses indicate that effects are higher for interventions of moderate length (1-2 months), consistent with previous research.

“Results from our subgroup analyses indicate that effects are higher for interventions of moderate length (1-2 months), which is in line with previous research (Heber et al., 2017;Richards & Richardson, 2012).”

• cites Heber et al., 2017Not checked yet

  Not checked in the free run.

• cites Richards & Richardson, 2012Not checked yet

  Not checked in the free run.

May combine two claims.

Heterogeneity was substantial in some analyses and remained moderate even after outliers were removed.

“Heterogeneity was substantial in some analyses and remained moderate even after outliers were removed.”

May combine two claims.

The small depression effect and the moderate eating disorder and student functioning effects from the main analysis also emerged when accounting for potential publication bias and when only low risk-of-bias studies were included.

“The small effect on depression as well as the moderate effects on eating disorder symptoms and student functioning found in the main analysis also emerged when accounting for potential publication bias and when only studies with a low risk of bias were included.”

May combine two claims.

Subgroup analyses found higher effects in samples preselected through symptom cut-offs or risk factors, and for interventions of medium length (4-8 weeks), based on CBT principles, and compared with passive control groups.

“In subgroup analyses, we found that effects were higher in samples that were preselected through symptom cut-offs or risk factors, as well as for interventions that were of medium length (4-8 weeks), based on CBT principles, and were compared with passive control groups.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A substantial proportion of university students experience mental health disorders each year, with estimates ranging from 12% to 46%.

“It is estimated that 12-46% of all university students are affected by mental health disorders in any given year (Auerbach et al., 2016;Auerbach et al., 2018;Blanco et al., 2008;Eisenberg, Hunt, & Speer, 2013;Verger, Guagliardo, Gilbert, Rouillon, & Kovess-Masfety, 2009).”

• cites Auerbach et al., 2016Not checked yet

  Not checked in the free run.

• cites Auerbach et al., 2018Not checked yet

  Not checked in the free run.

• cites Blanco et al., 2008Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Two reviewers assessed the methodological quality of included trials with the Cochrane Risk of Bias framework.

“The validity of included studies was evaluated by two researchers (M.H. and S. A.). Assessment followed the approach described by Furlan, Pennick, Bombardier, and van Tulder (2009), using the domains of the”

• cites Furlan, Pennick, Bombardier, and van Tulder (2009)Not checked yet

  Not checked in the free run.

• cites Higgins & Green, 2011Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Both the broader WMH-ICS meta-analysis project and this review were preregistered in PROSPERO.

“The WHM-ICS's meta-analysis initiative and this specific study have both been registered with PROSPERO (CRD42017068758; CRD42018090259).”

Probably not a substantial claim: may not pass S3 (load-bearing).

The average duration of untreated mental disorders is between 4 and 23 years.

“The average duration of untreated mental disorders stretches between 4 to 23 years (Wang et al., 2005)”

• cites Wang et al., 2005Not checked yet

  Not checked in the free run.

May combine two claims.

Mental disorders in young adults are linked to poorer academic performance, leaving college, and reduced functioning later in life.

“are associated with long-standing negative outcomes for both the individual and society, including lowered academic achievement (Eisenberg, Golberstein, & Hunt, 2009;Hysenbegasi, Hass, & Rowland, 2005), college dropout (Ishii et al., 2018;Kessler, Foster, Saunders, & Stang, 1995), and worse functioning in later life (Goldman-Mellor et al., 2014;Niederkrotenthaler et al., 2014).”

• cites Eisenberg, Golberstein, & Hunt, 2009Not checked yet

  Not checked in the free run.

• cites Ishii et al., 2018Not checked yet

  Not checked in the free run.

• cites Goldman-Mellor et al., 2014Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

This systematic review followed PRISMA reporting guidance.

“The procedures and results of this systematic review are outlined in accordance to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Liberati et al., 2009).”

• cites Liberati et al., 2009Not checked yet

  Not checked in the free run.

May combine two claims.

Only about 20% of students with mental illness receive minimally adequate care, indicating a large treatment gap.

“research documents a substantial treatment gap in university students suffering from mental illness, with only one in five receiving minimally adequate treatment (Auerbach et al., 2016).”

• cites Auerbach et al., 2016Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Two researchers independently screened and assessed studies under senior supervision; disagreements were discussed and, if unresolved, referred to two senior researchers.

“Both steps were performed independently by two researchers (M.H. and S. A.) supervised by a senior researcher (D.E.).Discussion between researchers was initiated in case of assessor disagreement; two senior researchers (D.E. and H. B.) were consulted when disagreement could not be resolved.”

May combine two claims.

Internet content can be easily and anonymously accessed.

“Contents can be easily and anonymously accessed through the Internet, and Internetdelivered programs provide high cost-effectiveness and scalability (Ebert et al., 2018).”

• cites Ebert et al., 2018Not checked yet

  Not checked in the free run.

May combine two claims.

Internet-delivered programs provide high cost-effectiveness and scalability.

“Internetdelivered programs provide high cost-effectiveness and scalability (Ebert et al., 2018).”

• cites Ebert et al., 2018Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Publication bias was assessed using contour-enhanced funnel plots and Egger's test, and when indicated, the Duval and Tweedie trim and fill procedure was used to adjust for possible bias.

“First, we inspected contour-enhanced funnel plots (Peters, Sutton, Jones, Abrams, & Rushton, 2008) and performed Egger's test of the intercept (Egger, Smith, Schneider, & Minder, 1997) to assess funnel plot asymmetry.When we found indications of publication bias, we used the Duval and Tweedie trim and fill procedure (Duval & Tweedie, 2000) to adjust for possible bias.”

• cites Peters, Sutton, Jones, Abrams, & Rushton, 2008Not checked yet

  Not checked in the free run.

• cites Egger, Smith, Schneider, & Minder, 1997 sourceNot checked yet

  Not checked in the free run.

• cites Duval & Tweedie, 2000Not checked yet

  Not checked in the free run.

• AssumptionThe authors state this as an assumption: The trim-and-fill method assumes that publication bias operates primarily through effect size.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A 2008 systematic review covered Internet-based prevention of eating disorders but did not include a meta-analysis.

“A systematic review on Internet-based eating disorder prevention was published in 2008 (Yager & O'Dea, 2008), with no meta-analysis performed.”

• cites Yager & O'Dea, 2008Not checked yet

  Not checked in the free run.

May combine two claims.

A p curve method was applied to identify selective reporting and to estimate a corrected average effect.

“We therefore additionally used p curve, a novel method to detect selective reporting bias in meta-analysis (Simonsohn et al., 2014a), which assumes that bias is caused because only statistically significant results (p < 0.05) are considered for publication.”

• cites Simonsohn et al., 2014aNot checked yet

  Not checked in the free run.

• AssumptionThe authors state this as an assumption: The p curve method assumes that publication bias arises because only statistically significant results are considered for publication.

Classified as probably not a substantial claim.

That meta-analysis did not find Internet-based programs to outperform active control conditions.

“but no superiority of these interventions compared with active controls.”

• cites Davies, Morriss, and Glazebrook (2014)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Previous research suggests that guided Internet interventions have higher effect sizes than unguided interventions.

“Although previous research suggests that guided Internet interventions have higher effect sizes than unguided interventions (Baumeister, Reichler, Munzinger, & Lin, 2014;Cowpertwait & Clarke, 2013), we did not find that guidance significantly moderated intervention efficacy.”

• cites Baumeister, Reichler, Munzinger, & Lin, 2014Not checked yet

  Not checked in the free run.

• cites Cowpertwait & Clarke, 2013Not checked yet

  Not checked in the free run.

May combine two claims.

Despite the limitations, Internet-based mental health interventions for university students can be a potentially effective means for a range of conditions and can have a beneficial impact on students' functioning.

“Despite these limitations, we conclude that Internet-based mental health interventions for university students can be a potentially effective mean for a range of conditions and can have a beneficial impact on university students' functioning.”

Probably not a substantial claim: may not pass S3 (load-bearing).

There are no iron-clad criteria to assess the importance of an effect.

“Although these guidelines are commonly used in psychological research, it should be noted that there are no iron-clad criteria to assess the importance of an effect (Durlak, 2009).”

• cites Durlak, 2009Not checked yet

  Not checked in the free run.