The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index; none of the 27 sources its claims cite is checked for support yet.
Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis
Open accessAbstract
Digital mental health interventions (DMHIs) present a promising way to address gaps in mental health service provision. However, the relationship between user engagement and outcomes in the context of these interventions has not been established. This study addressed the current state of evidence on the relationship between engagement with DMHIs and mental health outcomes. MEDLINE, PsycINFO, and EmBASE databases were searched from inception to August 1, 2021. Original or secondary analyses of randomized controlled trials (RCTs) were included if they examined the relationship between DMHI engagement and post-intervention outcome(s). Thirty-five studies were eligible for inclusion in the narrative review and 25 studies had sufficient data for meta-analysis. Random-effects meta-analyses indicated that greater engagement was significantly associated with post-intervention mental health improvements, regardless of whether this relationship was explored using correlational [ r = 0.24, 95% CI (0.17, 0.32), Z = 6.29, p < 0.001] or between-groups designs [Hedges' g = 0.40, 95% CI (0.097, 0.705), p = 0.010]. This association was also consistent regardless of intervention type (unguided/guided), diagnostic status, or mental health condition targeted. This is the first review providing empirical evidence that engagement with DMHIs is associated with therapeutic gains. Implications and future directions are discussed. Systematic Review Registration: PROSPERO, identifier: CRD 42020184706.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This systematic review and meta-analysis found that greater engagement with digital mental health interventions was modestly but significantly associated with post-intervention mental health improvements. It is reported as the first meta-analytic evidence quantifying this engagement–outcome relationship across randomized controlled trial analyses.
Key findings
- In the main correlational meta-analysis, greater engagement was associated with better post-intervention mental health outcomes: 20 studies, N = 1,808, r = 0.25, 95% CI (0.17, 0.32), Z = 6.29, p < 0.001; the abstract reports r = 0.24 for this association.
- In between-groups designs, higher content access was associated with moderate improvements: 5 studies, Hedges’ g = 0.40, SE = 0.16, 95% CI (0.097, 0.705), Z = 2.59, p = 0.010.
- Subgroup meta-analyses found significant positive associations for unguided interventions (r = 0.23), guided interventions (r = 0.30), self-report symptom screening (r = 0.24), diagnosed samples (r = 0.28), anxiety outcomes (r = 0.33), and depressive outcomes (r = 0.33).
- Narrative synthesis of 35 studies was mixed: 14 studies (40%) reported a significant positive association, 5 (17.1%) reported mixed findings, and 16 (45.7%) found no significant association.
Methods
- Systematic search of MEDLINE, PsycINFO, and EmBASE from inception to August 1, 2021; eligible studies were original or secondary analyses of RCTs of digital mental health interventions that quantitatively examined engagement and post-intervention mental health outcomes.
- Thirty-five studies were included in the narrative review; 25 had sufficient data for meta-analysis. Engagement was defined as objective intervention use, with number of modules/activities/sessions completed used as the primary engagement measure.
- Random-effects meta-analyses were conducted separately for correlational and between-groups designs, with planned subgroup analyses by guided/unguided intervention, diagnostic status, and anxiety/depression target; heterogeneity was assessed with I², publication bias with funnel plots and Egger’s test, and risk of bias with Cochrane RoB 2.
Limitations
- The authors state that included studies differed in target population, digital intervention, and mental health condition, preventing specific analyses of a particular engagement measure on a particular outcome.
- The authors state that studies used different statistical approaches, so repeated-measures and between-subject designs had to be analyzed separately; Pearson’s r also had to be estimated for some correlational studies.
- The authors note that bivariate correlations do not account for other variables that may influence the engagement–outcome association; the text also makes evident risk-of-bias concerns, including selective reporting in 28 studies (80.0%), outcome measurement bias in 19 studies (54.3%), and lack of participant blinding in 16 studies (45.7%).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
48 Own result · 23 Own method · 17 Not checked yet · 5 Own interpretation · 1 No basis given
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Fourteen studies examined DMHIs that were guided (20, 50–62). Among these studies, six reported a positive association between engagement and mental health outcomes”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Narrative Synthesis of the Effect of Engagement on Mental Health Outcomes
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Sixteen studies (45.7%) did not find any significant association between engagement and post-intervention mental health outcomes”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Narrative Synthesis of the Effect of Engagement on Mental Health Outcomes
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“while the other 11 did not find a significant association”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Narrative Synthesis of the Effect of Engagement on Mental Health Outcomes
plainn = 23- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This association was also consistent regardless of intervention type (unguided/guided), diagnostic status, or mental health condition targeted.”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Abstract
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The included studies' engagement measures and mental health outcomes are sufficiently comparable to pool in meta-analysis.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the remaining 23 studies, participants were not formally diagnosed with a mental health condition but were screened for symptoms indicative of mental disorders. Of these studies, 12 reported a positive association between engagement and the primary mental health outcome”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Narrative Synthesis of the Effect of Engagement on Mental Health Outcomes
plaincorrelationaln = 23- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Random-effects meta-analyses indicated that greater engagement was significantly associated with post-intervention mental health improvements, regardless of whether this relationship was explored using correlational [r = 0.24, 95% CI (0.17, 0.32), Z = 6.29, p < 0.001] or between-groups designs [Hedges' g = 0.40, 95% CI (0.097, 0.705), p = 0.010].”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Abstract
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The included studies' engagement measures and mental health outcomes are sufficiently comparable to pool in meta-analysis.
- Rests on an assumption: The association between engagement and mental health outcomes can be interpreted as therapeutic gains from engagement, rather than reverse causation or confounding.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Random-effects meta-analyses indicated that greater engagement was significantly associated with post-intervention mental health improvements, regardless of whether this relationship was explored using correlational [r = 0.24, 95% CI (0.17, 0.32), Z = 6.29, p < 0.001] or between-groups designs [Hedges' g = 0.40, 95% CI (0.097, 0.705), p = 0.010].”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Abstract
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The included studies' engagement measures and mental health outcomes are sufficiently comparable to pool in meta-analysis.
- Rests on an assumption: The association between engagement and mental health outcomes can be interpreted as therapeutic gains from engagement, rather than reverse causation or confounding.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of these studies, six reported a positive association between engagement and the primary mental health outcome (34, 39, 51, 54, 55, 66) while the other six did not find a significant association”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Narrative Synthesis of the Effect of Engagement on Mental Health Outcomes
plainn = 12- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of these studies, six reported a positive association between engagement and the primary mental health outcome (34, 39, 51, 54, 55, 66) while the other six did not find a significant association”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Narrative Synthesis of the Effect of Engagement on Mental Health Outcomes
plaincorrelationaln = 12- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“while five did not find any significant association”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Narrative Synthesis of the Effect of Engagement on Mental Health Outcomes
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 20
- Probably not a substantial claim. 10
- The quote only partly states this claim. 1
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The remaining four studies included participants who used both unguided and guided versions of the same digital intervention (63–66). Of these studies, three reported a positive association between engagement and mental health outcomes”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Narrative Synthesis of the Effect of Engagement on Mental Health Outcomes
plaincorrelationaln = 4- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“while one did not find a significant association”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Narrative Synthesis of the Effect of Engagement on Mental Health Outcomes
plaincorrelationaln = 4- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Nine studies (35, 36, 42, 44, 46, 50, 52, 59, 64) investigated the relationship between engagement and psychological distress or psychological recovery. Of these, three reported a positive association between engagement and post-intervention symptoms (36, 50, 52) while six did not find any significant association (35, 42, 44, 46, 59, 64).”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Narrative Synthesis of the Effect of Engagement on Mental Health Outcomes
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Omission of Oromendia et al. (66) had the largest effect reduction, increasing the Hedges' g value from 0.40 to 0.25.”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Meta-Analysis of the Effect of Engagement on Mental Health Outcomes
plaindirectn = 5- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Nine studies (33, 38, 40, 48, 49, 57, 58, 63, 65) investigated the relationship between engagement and depression-related symptoms. Of these, six reported a positive association between engagement and post-intervention symptoms (33, 40, 49, 57, 63, 65) while three did not find any significant association (38, 48, 58).”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Narrative Synthesis of the Effect of Engagement on Mental Health Outcomes
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of the 25 studies included in the meta-analysis, 20 studies (20, 33–35, 37, 40, 41, 43, 47, 48, 51, 54–57, 59, 61, 63–65) employed correlational designs to investigate the association between engagement and mental health outcomes. The other five studies (36, 38, 50, 52, 66) used between-group mixed model comparisons to identify any differences in post-intervention outcomes between users who”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Meta-Analysis of the Effect of Engagement on Mental Health Outcomes
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All studies were included in the narrative synthesis, and 25 of the 35 were included in the meta-analysis.”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There was no heterogeneity in the distribution of individual study effect sizes (I2 = 0.0%).”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Meta-Analysis of the Effect of Engagement on Mental Health Outcomes
plainn = 423- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This search strategy was tested in MEDLINE, where it achieved 100% sensitivity against the initial test set”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Methods > Search Strategy
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: The search strategy adapted from MEDLINE to other databases retains adequate sensitivity.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Inter-rater consensus was acceptable for both title/abstract (κ = 0.86, p < 0.001) and full-text screening (κ = 0.80, p < 0.001).”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Methods > Search Strategy
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“This study is registered with PROSPERO (CRD 42020184706) and adheres to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guidelines (23).”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Methods > Search Strategy
What the sources say
- Not checked yetcites [23] Liberati, 2009, The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration, J Clin Epidemiol. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Ratings were independently conducted by two pairs of coders (40% by DZQG and LM, and 60% by DZQG and JH). Discrepancies were resolved through mutual discussion. An overall agreement rate of 93.3% was reached.”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Methods > Data Extraction and Analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Selective reporting was identified to be the largest source of bias, with 28 studies (80.0%) not reporting sufficient information—such as a prospectively published trial protocol—to rule out bias in this domain.”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Risk of Bias
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Inter-rater consensus was acceptable for both title/abstract (κ = 0.86, p < 0.001) and full-text screening (κ = 0.80, p < 0.001).”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Methods > Search Strategy
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“To date, only one prior systematic review (18) has explored the effect of engagement on the effectiveness of digital interventions. This review was limited to a narrative synthesis, citing heterogeneity in how engagement is operationalized”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Introduction
What the sources say
- Not checked yetcites [18] Donkin, 2011, Systematic review of the impact of adherence on the effectiveness of e-therapies, J Med Internet Res. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Risk of outcome measurement bias was the second most common source of potential bias, with 19 studies (54.3%) reporting that outcome assessors—usually study participants themselves—were aware of the intervention received by study participants.”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Risk of Bias
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Risk of bias was assessed using Version 2 of the Cochrane risk-of-bias tool for randomized trials (RoB 2) (32).”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Methods > Data Extraction and Analysis
What the sources say
- Not checked yetcites [32] Sterne, 2019, RoB 2: a revised tool for assessing risk of bias in randomised trials, BMJ. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“That is, the results unequivocally support that greater engagement with digital interventions is modestly but significantly associated with improvements in mental health (effect size range: r = 0.23 to Hedges' g = −0.40) regardless of the level of guidance provided, mental health symptom severity of users, or type of mental health condition(s) targeted by the intervention.”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Discussion
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Third, Pearson's r had to be estimated for some of the studies which employed correlational designs. While estimating r from other indices may not be ideal, it is preferred to omitting studies without this data so as to maximize objectivity and minimize selection bias (26, 29).”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Discussion
What the sources say
- Not checked yetcites [26] Wilson, 2001
- Not checked yetcites [29] Borenstein, 2009, Effect sizes for continuous data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Controlling for factors linked with engagement may be necessary for verifying the robustness of its relationship with outcomes.”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Discussion
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
In the paper
“As effect sizes from repeated measures and between-subject designs are not comparable (26), data provided by both types of studies had to be analyzed separately.”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Discussion
What the sources say
- Not checked yetcites [26] Wilson, 2001
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While the narrative synthesis showed mixed support for a positive engagement—outcome relationship, the meta-analyses (main and subgroup) consistently supported our main a priori hypothesis.”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“A review of empirical studies has found collectively low rates of DMHI completion, with over 70% of users failing to complete all treatment modules, and more than 50% disengaging before completing half of all treatment modules (15).”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Introduction
What the sources say
- Not checked yetcites [15] Karyotaki, 2015, Predictors of treatment dropout in self-guided web-based interventions for depression: an “individual patient data” meta-analysis, Psychol Med. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“a review of efficacy studies of smartphone apps for depression and anxiety found that apps which incorporated more elements aimed at promoting user engagement had larger effect sizes (17).”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Introduction
What the sources say
- Not checked yetcites [17] Wu, 2021, Smartphone apps for depression and anxiety: a systematic review and meta-analysis of techniques to increase engagement, NPJ Digit Med. source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“Low engagement—defined as suboptimal levels of user access and/or adherence to an intervention (12)—is touted as one of the main reasons why the potential benefits of these interventions remain unrealized in the real world (13, 14).”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Introduction
What the sources say
- Not checked yetcites [12] Torous, 2018, Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements, Evid Based Mental Health. source
- Not checked yetcites [13] Bidargaddi, 2018, To prompt or not to prompt? A microrandomized trial of time-varying push notifications to increase proximal engagement with a mobile health app, JMIR mHealth uHealth. source
- Not checked yetcites [14] Yardley, 2016, Understanding and promoting effective engagement with digital behavior change interventions, Am J Prev Med. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“First, the included studies differed in many ways, such as by target population, DMHIs employed, and the types of mental health conditions examined. Thus, specific analyses of the effect of a specific measure of engagement on a particular mental health outcome could not be conducted.”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“For example, one of the included studies reported that sessions completed and time spent were correlated with reduced depression scores at post-intervention; however, these associations were non-significant after email support was accounted for (63).”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Discussion
What the sources say
- Not checked yetcites [63] Fuhr, 2018, The association between adherence and outcome in an Internet intervention for depression, J Affect Disord. source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Secondly, our findings suggest that module completion may be one of the more acceptable measures of engagement to evaluate.”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Discussion
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
In the paper
“Our findings validate the qualitative findings reported in the systematic review by Donkin and colleagues (18), who reported that improvements in mental health-related outcomes appeared to be associated with the number of modules accessed, but not with other engagement indicators (e.g., time spent, logins, and online interactions).”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Discussion
What the sources say
- Not checked yetcites Donkin and colleagues (18)
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper relies on this without support.
The included studies' engagement measures and mental health outcomes are sufficiently comparable to pool in meta-analysis.
“Studies which utilized similar measures of engagement and which employed similar analytical techniques to test the association between engagement and outcome(s) were pooled together for meta-analysis.”
The paper relies on this without support.
The association between engagement and mental health outcomes can be interpreted as therapeutic gains from engagement, rather than reverse causation or confounding.
“This is the first review providing empirical evidence that engagement with DMHIs is associated with therapeutic gains.”
The paper relies on this without support.
The search strategy adapted from MEDLINE to other databases retains adequate sensitivity.
“This search strategy was tested in MEDLINE, where it achieved 100% sensitivity against the initial test set, and was subsequently adapted for use in the other databases.”
The paper relies on this without support.
The increasing number of individual studies examining the engagement–mental health outcome association now enables a review and quantification of the literature.
“The increasing number of individual studies examining the engagement—mental health outcome association published in recent years, now enables a review and quantification of the literature.”
The paper relies on this without support.
The relationship between user engagement and outcomes in digital mental health interventions has not been established.
“However, the relationship between user engagement and outcomes in the context of these interventions has not been established.”
The paper relies on this without support.
This is the first systematic review and meta-analysis to quantitatively examine whether the level of user engagement with a digital intervention is associated with change in mental health outcomes after the intervention period.
“To our knowledge, this is the first systematic review and meta-analysis to quantitatively examine whether the level of user engagement with a digital intervention was associated with change in mental health outcomes after the intervention period.”
The paper relies on this without support.
Robust empirical evidence to support or validate the hypothesis that engagement with digital interventions is positively associated with improvements in mental health is scant.
“Although it is widely accepted that the extent of engagement with digital interventions will be positively associated with improvements in mental health, robust empirical evidence to support or validate this hypothesis is scant.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This finding is consistent with previous research that has highlighted the role of continuous engagement in the effectiveness of digital health interventions25–27.”
From Impact of digital health interventions on glycemic control and weight management. (2025) · cites this paper as [25]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“issues of poor engagement with digital health interventions are well documented, with an average of 70% of users not completing all treatment modules [28]”
From The Effectiveness of a Smartphone Intervention Targeting Suicidal Ideation in Young Adults: Randomized Controlled Trial Examining the Influence of Loneliness (2023) · cites this paper as [28]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Greater engagement can be associated with post-intervention improvement in mental health [10].”
From Young People’s Barriers and Facilitators of Engagement with Web-Based Mental Health Interventions for Anxiety and Depression: A Qualitative Study (2024) · cites this paper as [10]
The cited source says what the claim attributes to it.
In the citing paper
“there is also evidence that the level of engagement with digital health interventions is a significant positive moderator of clinical benefit [28,29].”
The cited source says what the claim attributes to it.
In the citing paper
“Pooled findings from prior research estimate that treatment dropout is 74% for unsupervised digital therapeutics [37], which is problematic as low engagement has been shown to reduce the efficacy [38].”
From The effect of a therapeutic smartphone application on suicidal ideation in young adults: Findings from a randomized controlled trial in Australia. (2022) · cites this paper as [38]
In the citing paper
“especially given that user engagement is associated with better treatment outcomes [15,53] and more cost-effective treatment [54,55].”
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Consistentz = 6.29reported p < .001 · recomputed p < .001
“Z = 6.29, p < .001”
- Consistentz = 2.59reported p = .01 · recomputed p = .010
“Z = 2.59, p = .010”
- Consistentz = 6.07reported p < .001 · recomputed p < .001
“Z = 6.07, p < .001”
- Consistentz = 6.13reported p < .001 · recomputed p < .001
“Z = 6.13, p < .001”
- Consistentz = 6.03reported p < .001 · recomputed p < .001
“Z = 6.03, p < .001”
- Consistentz = 6.76reported p < .001 · recomputed p < .001
“Z = 6.76, p < .001”
- Consistentz = 3.12reported p = .002 · recomputed p = .002
“Z = 3.12, p = .002”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“Five studies (17.1%) reported mixed findings”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Narrative Synthesis of the Effect of Engagement on Mental Health Outcomes
Reported 17.1 · recomputed 14.285714
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Study Characteristics
“Mean participant ages ranged from 11.0 (SD 2.57) to 58.4 (SD 9.0) years.”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Study Characteristics
“AdultsMage = 63.6 (13.9)60% female”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Meta-Analysis of the Effect of Engagement on Mental Health Outcomes
“increasing the Hedges' g value from 0.40 to 0.25.”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Meta-Analysis of the Effect of Engagement on Mental Health Outcomes
“Omission of Oromendia et al. (66) had the largest effect reduction”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Discussion
“effect size range: r = 0.23 to Hedges' g = −0.40”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Meta-Analysis of the Effect of Engagement on Mental Health Outcomes
“Hedges' g = 0.40, SE =0.16, 95% CI (0.097, 0.705), Z = 2.59, p = 0.010”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Narrative Synthesis of the Effect of Engagement on Mental Health Outcomes
“In the remaining 23 studies, participants were not formally diagnosed with a mental health condition but were screened for symptoms indicative of mental disorders. Of these studies, 12 reported a positive association between engagement and the primary mental health outcome (33, 36, 40, 41, 43, 47, 49, 50, 52, 57, 63, 65)”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Study Characteristics
“Fulfill diagnostic criteria for schizophrenia-related, bipolar, or depressive disorder.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Meta-Analysis of the Effect of Engagement on Mental Health Outcomes
“r = 0.24, 95% CI [0.17, 0.32], Z = 6.29, p < 0.001”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Meta-Analysis of the Effect of Engagement on Mental Health Outcomes
“Removal of Mira et al. (65) had the largest effect influence on the model, reducing the overall r from 0.27 to 0.17.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Narrative Synthesis of the Effect of Engagement on Mental Health Outcomes
“Five studies (17.1%) reported mixed findings, where improvements in the primary mental health outcome were associated with some, but not all, measures of engagement (37, 43, 47, 49, 57).”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Study Characteristics
“Pain self-efficacy | Activities completed (+)”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Narrative Synthesis of the Effect of Engagement on Mental Health Outcomes
“this was consistent across all engagement measures used if multiple measures were used in a single study (33, 34, 36, 39, 40, 50–52, 54, 55, 61, 63, 65, 66)”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Study Characteristics
“Exercises completed (+)Emails sent (ns)Mean time logged in (ns)Total time logged in (ns)No. of logins (ns)”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Meta-Analysis of the Effect of Engagement on Mental Health Outcomes
“Data for five of the 10 studies which investigated the relationship between engagement and anxiety-related symptoms, and for six of the 12 studies involving participants who met diagnostic criteria for a mental health condition were combined and examined in two separate meta-analyses.”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Meta-Analysis of the Effect of Engagement on Mental Health Outcomes
“The overall mean pooled correlation for the meta-analysis of 6 studies with depressive symptoms as the primary outcome”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Narrative Synthesis of the Effect of Engagement on Mental Health Outcomes
“Sixteen studies (45.7%) did not find any significant association between engagement and post-intervention mental health outcomes (20, 35, 38, 41, 42, 44–46, 48, 53, 56, 58–60, 62, 64).”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Study Characteristics
“Activities completed (+)”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“Most of the interventions were either fully or partially based on cognitive-behavioral therapy (27 studies, 80%).”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Study Characteristics
Reported 80 · recomputed 77.142857
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Abstract
“correlational [r = 0.24, 95% CI (0.17, 0.32), Z = 6.29, p < 0.001]”
§ Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis > Results > Meta-Analysis of the Effect of Engagement on Mental Health Outcomes
“20 studies, N = 1,808 participants; Figure 2) showed a small, significant positive association [r = 0.25, 95% CI (0.17, 0.32), Z = 6.29, p < 0.001]”
Citations
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139 · cited 1×
“Study heterogeneity was evaluated for each meta-analysis using the I 2 statistic, with thresholds of 25, 50, and 75% denoting low, moderate, and high levels of heterogeneity, respectively (28).”
MethodBias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722 · cited 1×
“For meta-analyses with a sufficient number of included studies (i.e., > 10 studies), publication bias was assessed using funnel plots and Egger’s regression test (30).”
MethodRoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787 · cited 1×
“Riskof bias was assessed using Version 2 of the Cochrane risk-of-bias tool for randomized trials (RoB 2) (32).”
MethodLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578
(2000) · Cited by 9,198
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,840
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
Murray, C. J. L., Barber, R. M., Foreman, K. J., Özgören, A. A., Abd-Allah, F., Abera, S. F., Aboyans, V., Abraham, J. P., Abubakar, I., Abu-Raddad, L. J., et al. (2015) · The Lancet · doi:10.1016/s0140-6736(15)61340-x · Cited by 1,966
On the Use of Beta Coefficients in Meta-Analysis.
Peterson, R. A., & Brown, S. P. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.1.175 · Cited by 1,643
Research works printed but not linked: 1
- Borenstein, 2013, Comprehensive Meta-Analysis (version 3.0)
Cited by
Sorted by citations
Items 101–200 of 305
Xiang, X., Kayser, J., Turner, S., & Zheng, C. (2023) · Research on Social Work Practice · doi:10.1177/10497315231184143 · Cited by 7
“Low user engagement with DMHIs is a known issue and a barrier to realizing the bene fi ts of these interventions in real-world settings (Gan et al., 2021).”
McGillivray, L., Keng-Meng Hui, N., Wong, Q. J. J., Han, J., Qian, J., & Torok, M. (2023) · JMIR Mental Health · doi:10.2196/44862 · Cited by 7
“However, issues of poor engagement with digital health interventions are well documented, with an average of 70% of users not completing all treatment modules [28], yet reasons for poor engagement are not fully understood.”
Self-citationHabicht, J., Dina, L. M., Stylianou, M., Harper, R., Hauser, T. U., & Rollwage, M. (2024) · doi:10.31234/osf.io/mj46k · Cited by 7
Keyan, D., Hall, J., Jordan, S., Watts, S., Au, T. M., Dawson, K. S., Sway, R. A., Crawford, J., Sorsdahl, K. R., Luitel, N. P., et al. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1528580 · Cited by 12
Hatoum, A. H., Coutts‐Bain, D., Hunnisett, N., Baffsky, R., O’Dea, B., & Torok, M. H. (2025) · EClinicalMedicine · doi:10.1016/j.eclinm.2025.103601 · Cited by 6
Self-citationSustained engagement with a digital youth mental health platform: A mixed-methods study
Valentine, L., Nicholas, J., Sorenson, R. P., Chen, N. A., McEnery, C., Louis, S., Cross, S. P. M., Mangelsdorf, S. N., O’Sullivan, S., Wren, T. W., et al. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100899 · Cited by 6
“A recent meta-analysis by Gan et al. (2021) found that higher levels of engagement with DMHIs were modestly but significantly associated with improvements in mental health outcomes across a range of populations, conditions, and intervention types.”
Rheinberger, D., Baffsky, R., McGillivray, L., Gan, D. Z. Q., Larsen, M. E., & Torok, M. H. (2024) · Digital Health · doi:10.1177/20552076241230072 · Cited by 4
“The engagement challenges posed by digital therapeutics 19,20 have led to recent growth in ‘ design thinking ’ research which aims to identify enablers and barriers of use.”
BackgroundSelf-citationMapping the Process of Engagement With Digital Health Interventions: A Cross-Case Synthesis
Milne‐Ives, M., Homer, S. R., Andrade, J., & Meinert, E. (2025) · Mayo Clinic Proceedings Innovations Quality & Outcomes · doi:10.1016/j.mayocpiqo.2025.100625 · Cited by 5
Kornfield, R., Popowski, S. A., Tack, E. B., Svoboda, J., Herrera, M. A., Nguyen, T., Knapp, A. A., Mohr, D. C., & Meyerhoff, J. (2025) · JMIR Formative Research · doi:10.2196/68950 · Cited by 8
“…unmet mental health needs in the United States across demographic groups, many DMHIs are designed to be suitable across various individuals in need [39-41]; however, it is increasingly recognized that programs may be more appealing and potentially more effective when designed for narrower segments…”
Bufton, K., Bates, M. T., Fuller‐Tyszkiewicz, M., Hamid, J., & Westrupp, E. M. (2025) · Health Expectations · doi:10.1111/hex.70385 · Cited by 9
“Addressing these limitations may enhance the effectiveness of user collaboration and support digital programs in realising their full potential to widen the reach and engagement of evidence ‐ based mental health treatments [1, 17, 20].”
Das, P. J., Kyndait, P., Jamir, S. I., Baishya, D., Kalita, J. K., & Komu, L. T. (2024) · International Journal of Scientific Research in Science and Technology · doi:10.32628/ijsrst24116186 · Cited by 5
Davis, C. A., Miller, M. L., & McLean, C. P. (2024) · JMIR mhealth and uhealth · doi:10.2196/49393 · Cited by 5
“Our finding that overall app use was unrelated to PTSD symptom change is consistent with previous research on PTSD Coach [6] but inconsistent with meta-analytic findings showing that engagement with mHealth apps in general is associated with symptom change [24].”
ResultKleinau, E., Lamba, T., Jaskiewicz, W., Gorentz, K., Hungerbuehler, I., Rahimi, D., Kokota, D., Maliwichi, L., Jamu, E. S., Zumazuma, A., et al. (2023) · medRxiv · doi:10.1101/2023.01.24.23284959 · Cited by 5
Jackson, H. M., Batterham, P. J., Ohan, J. L., Calear, A. L., & Farrer, L. M. (2024) · BMC Psychiatry · doi:10.1186/s12888-024-05953-3 · Cited by 8
“This is due in part to levels of user engagement [9], which have frequently been found to be suboptimal in research settings [10] and tend to be even lower among free-range users in the community [11].”
BackgroundHo, T. Q. A., Engel, L., Le, L. K., Melvin, G. A., Ride, J., Le, H., & Mihalopoulos, C. (2025) · Patient · doi:10.1007/s40271-025-00739-5 · Cited by 4
Wu, Z., Luo, F., Wang, S., hu, X., & Chen, M. (2025) · JMIR Cancer · doi:10.2196/64754 · Cited by 5
“The anonymity, immediate access, and flexibility afforded by DMHIs effectively overcome common barriers to accessing care, such as cost, time constraints, or stigma [16].”
E-Health Intervention for Fear of Cancer Recurrence
Lyhne, J. D., Smith, A. B., Timm, S., Klein, B., Thewes, B., Girgis, A., Bamgboje‐Ayodele, A., Beatty, L., Fardell, J. E., Fink, P., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.42112 · Cited by 7
Fouyaxis, J., Bidargaddi, N. P., Du, W., Looi, J. C., & Lipschitz, J. M. (2024) · Digital Health · doi:10.1177/20552076241306782 · Cited by 6
Hopman, K., Richards, D., & Norberg, M. M. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2477757 · Cited by 4
“This finding contrasts to findings of the previously discussed systematic review that identified higher levels of engagement were significantly associated with improved mental health outcomes (Gan et al. 2021).”
InfluentialTsirozidis, G., Kirk, U. B., & Zaggl, M. A. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2485395 · Cited by 4
“In the domain of mHealth solutions, engagement is seen as a key to the realisation of the potential health outcomes which health apps can provide (Gan et al. 2021; Jakob et al. 2022; Meyerowitz-Katz et al. 2020; Ser-rano et al. 2017; Taki et al. 2017).”
Jenny, G. J., & Bauer, G. F. (2023) · Scandinavian Journal of Work and Organizational Psychology · doi:10.16993/sjwop.185 · Cited by 5
Glenn, S. D., Turrisi, R., Mallett, K. A., Waldron, K. A., & Lenker, L. K. (2023) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2023.09.010 · Cited by 4
Firet, L., Teunissen, T. A. M., Kool, R. B., Akkermans, R. P., Lagro‐Janssen, A. L. M., van der Vaart, C. H., & Assendelft, W. J. (2024) · BMC Family Practice · doi:10.1186/s12875-024-02325-4 · Cited by 4
Øverup, C., & Hald, G. M. (2024) · Current Psychology · doi:10.1007/s12144-024-06351-4 · Cited by 4
“Digital interventions have been found to be effective across a range of mental health concerns and (patient) populations (e.g., Doss et al., 2016; Gan et al., 2021; Hald et al., 2020; Shaygan et al., 2023).”
Zarski, A., Sextl-Plötz, T., Schmidt-Hantke, J., Hess, N. S., & Buntrock, C. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1626428 · Cited by 5
Kikas, K., O'Moore, K., Miller, R. K., Matheson, J.-A. T., Li, S., Varghese, K., Baldwin, P., Cockayne, N., Whitton, A. E., & Newby, J. M. (2025) · JMIR Human Factors · doi:10.2196/69499 · Cited by 3
“…of adults with lived and living experience by including relevant therapeutic content and sustaining their interest with appealing design features and functionality, especially given that user engagement is associated with better treatment outcomes [15,53] and more cost-effective treatment [54,55].”
Lange, L., Willemer, H., Banse, M., Ernst, A., Daubmann, A., Holz, A., Bleich, C., Weg‐Remers, S., & Schulz, H. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1455143 · Cited by 3
Golmohamadi, S., Murray, K., & Farrer, L. M. (2025) · Body Image · doi:10.1016/j.bodyim.2025.101936 · Cited by 3
Kim, G. S., Baek, S., Kim, L., & Kwon, S. (2025) · Journal of Medical Internet Research · doi:10.2196/78810 · Cited by 3
“Higher levels of engagement also show a significant correlation with improved postintervention mental health outcomes [7].”
Lin, T., Godorecci, K., Zhang, C., Mackey, B., Mattoni, M., Garner, L. E., Brown, L. A., Guzick, A. G., & Tyler, J. (2026) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2026.2630174 · Cited by 2
Ferguson, I., George, G. C., Narine, K., Turner, A., McGhee, Z., Bajwa, H., Hart, F. G., Carter, S. E., & Beard, C. (2024) · JMIR Mental Health · doi:10.2196/56758 · Cited by 3
“DMHIs are not reaching people who have typically not accessed mental health treatment; most data to date come from White women—the same demographic that is already best served by existing treatments [10,11].”
Backgroundvan Mierlo, T., Fournier, R., Yeung, S. K., & Lahutina, S. (2025) · JMIR Research Protocols · doi:10.2196/73773 · Cited by 4
The journey of playing: a qualitative evaluation of ‘Maze Out,’ a serious game for eating disorders
Guala, M. M., Nielsen, H. N., Bul, K., Clinton, D. N., Scarpelli, M., Christiansen, R., Skårderud, F., & Nielsen, A. S. (2025) · Journal of Eating Disorders · doi:10.1186/s40337-025-01306-x · Cited by 3
“These aspects are important as greater adherence and engagement are significantly associated with post-intervention mental health improvements [28, 29].”
Kiselev, N., Schaffner, R. M., Csepregi, Z., Wenger, A., Castro, R. P., & Haug, S. (2025) · JMIR mhealth and uhealth · doi:10.2196/64861 · Cited by 3
Abundo, M. F., & Sav, A. (2025) · Healthcare · doi:10.3390/healthcare13172113 · Cited by 3
“Evidence from the digital mental health field supports this finding; for example, Gan et al. [32] found that engagement was significantly linked to positive outcomes in digital mental health interventions, especially when support tools like reminders and interactive content were included.”
Opie, J. E., Vuong, A. T., McIntosh, J., & Kuntsche, S. (2025) · Mental Health and Digital Technologies · doi:10.1108/mhdt-07-2025-0050 · Cited by 2
Balcombe, L., & Leo, D. D. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.964251 · Cited by 3
Milbourn, B. T., Johannes, A., Moreton, A., Thomas, L., Clarke, N., Zimmermann, F., Kacic, V., Girdler, S. J., McGarry, S., & Black, M. H. (2023) · Mental Health Science · doi:10.1002/mhs2.33 · Cited by 3
Alfaro, A. J., Wielgosz, J., Kuhn, E. R., Carlson, C., & Gould, C. E. (2023) · Journal of Clinical Psychology · doi:10.1002/jclp.23636 · Cited by 4
Digital interventions for mental health: challenges and opportunities
Kim, M., Lee, S., Jeong, I., Jeon, M., & Jung, D. (2024) · Journal of Korean Medical Association · doi:10.5124/jkma.2024.67.12.752 · Cited by 3
Bernstein, E. E., Daniel, K. E., Miyares, P. E., Hoeppner, S. S., Bentley, K. H., Snorrason, Í., Fisher, L. B., Greenberg, J. L., Weingarden, H. M., Harrison, O., et al. (2025) · JMIR Mental Health · doi:10.2196/63497 · Cited by 4
Govindasamy, K., Elayaraja, M., Abderrahman, A. B., Parpa, K., Katanić, B., & Granacher, U. (2025) · Healthcare · doi:10.3390/healthcare13050575 · Cited by 3
“The use of digital interventions to promote physical activity have garnered interest because of their mental health benefits; however, there is a paucity of trials available to strengthen the existing evidence on the effect of digitally directed PA promotions on mental health outcomes [38,39].”
Gamification of an Automated Internet Intervention to Enhance Engagement and Adherence
Curland, R. A., Muñoz, R. F., & Leykin, Y. (2025) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-025-00528-9 · Cited by 3
“Though greater engagement in the intervention has been repeatedly linked to better mental health outcomes (Gan et al., 2021), the association may not hold true for every person or every intervention.”
Achieving Inclusivity by Design: Social and Contextual Information in Medical Knowledge
Hastings, J. (2022) · Yearbook of Medical Informatics · doi:10.1055/s-0042-1742509 · Cited by 4
Zhang, X. (2025) · Interactive Learning Environments · doi:10.1080/10494820.2025.2454444 · Cited by 3
“Similar studies have indicated that mobile applications designed to support mental health can lead to improvements in well-being and mental health literacy (Gan et al., 2021).”
Development of a Digital Companion for Employee Well-Being: A Mixed Methods Approach
Duggirala, M., Naik, V., Mohan, A., Patel, S., & Athavale, S. (2025) · Springer eBooks · doi:10.1007/978-981-96-0108-0_12 · Cited by 3
Meuldijk, D., Deady, M., Collins, D., Williams, D. O., Bryant, R. A., & Harvey, S. B. (2025) · JMIR Formative Research · doi:10.2196/50995 · Cited by 2
“This feedback is consistent with observations from reviews of efficacy studies on smartphone interventions, in which apps incorporating more elements aimed at promoting user engagement have shown larger effect sizes on several MH outcomes [42].”
Zheng, Y., Dou, Z., Guo, T., Wang, Y., & Zeng, X. (2025) · Mindfulness · doi:10.1007/s12671-025-02548-1 · Cited by 2
“The potential decline in engagement, in turn, may have diluted the effectiveness of both the mindfulness and non-mindfulness components (Gan et al., 2021).”
Borgnolo, L. J., McKenna, S., Hickie, I. B., Varidel, M., Turner, A., Gorban, C., LaMonica, H. M., Chong, M. K., Capon, W., Dimitropoulos, G., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/67597 · Cited by 3
“Randomized controlled trials have demonstrated the efficacy of digital mental health technologies (DMHTs) [1,3,4], yet their translation into real-world settings remains problematic [5,6].”
Adapting an Anxiety Sensitivity Intervention for Perinatal Mental Health
Zimmermann, M., Yonkers, K. A., Tulu, B., Ford, L., Peacock‐Chambers, E., Clare, C. A., Boudreaux, E. D., Lemon, S. C., & Byatt, N. (2025) · The Journal of Nervous and Mental Disease · doi:10.1097/nmd.0000000000001841 · Cited by 2
Hassan, A., Breare, H., Rollo, M. E., Mullan, B. A., Pollard, C. M., Kerr, D. A., Dhaliwal, S. S., & Begley, A. M. (2025) · Obesity Reviews · doi:10.1111/obr.70035 · Cited by 2
Murillo, S., Philipsheck, A. P., Dementyev, F., Bello-Soto, B., Bhatt, E., Wilson, H., Madison, K., Schuck, L., Wiafe, S., Brown, C. L., et al. (2025) · Frontiers in Sports and Active Living · doi:10.3389/fspor.2025.1646805 · Cited by 3
Frameworks for Developing Digital Mental Health Interventions
Neto, D. D., Sebastião, R., Baptista, T. M., & Petkari, E. (2025) · European Psychologist · doi:10.1027/1016-9040/a000571 · Cited by 2
García-Huércano, C., Cerón, S. C., Martínez-Vispo, C., Bellón, J. Á., Morejón, A. R., Tamayo‐Morales, O., & Moreno-Peral, P. (2025) · Psychological Medicine · doi:10.1017/s0033291725102262 · Cited by 2
Varker, T., Metcalf, O., Mok, Y. F., Lamb, K. E., Fredrickson, J., Arjmand, H., Howlett, P., Gibson, K., Pham, L., Everett, K., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02092-1 · Cited by 2
Zainal, N. H., & Newman, M. G. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.120963 · Cited by 2
Bastoni, S., Doğan, K., & Kelders, S. M. (2026) · Behaviour and Information Technology · doi:10.1080/0144929x.2026.2619650 · Cited by 2
“Furthermore, digital mental health interventions are often characterised by poor engagement (Gan et al. 2021), that is, the way people think and feel about the technology, and their behaviour towards the technology is not optimal, negatively impacting outcomes.”
Tyler, N., Croke, S., Low, C.-N., Cassidy, N., Gkintoni, E., McMillan, B., & Panagioti, M. (2026) · Health Expectations · doi:10.1111/hex.70598 · Cited by 2
“…optimisation techniques such as ensuring well‐integrated and meaningful gamification, and tailoring of programmes to the user improve engagement [20–22], suggesting that the solutions that arose organically from this workshop focusing specifically on the PTSD Hub and this clinical population…”
Flaucher, M., Ott, T., Nissen, M., Dabrock, P., Huebner, H., Beckmann, M. W., Fasching, P. A., Leutheuser, H., Eskofier, B. M., & Mocker, A. (2026) · JMIR Cancer · doi:10.2196/77750 · Cited by 3
Jackson, H. M., Farrer, L. M., Werner‐Seidler, A., Perry, Y., Christensen, H. M., Ohan, J. L., Calear, A. L., & Batterham, P. J. (2023) · Journal of Mood and Anxiety Disorders · doi:10.1016/j.xjmad.2023.100027 · Cited by 2
Self-citationCorthésy-Blondin, L., Lemyre, A., Poitras, M., & Guay, S. (2023) · Technology Mind and Behavior · doi:10.1037/tmb0000121 · Cited by 2
Abouzeid, N., & Lal, S. (2024) · PLoS ONE · doi:10.1371/journal.pone.0301886 · Cited by 2
“Research has shown that a greater level of engagement with DMHI is significantly associated with improved mental health outcomes [17].”
BackgroundSchouten, M. J. E., Goudriaan, A. E., Schaub, M. P., Dekker, J., & Blankers, M. (2024) · Psychological Medicine · doi:10.1017/s0033291724000953 · Cited by 2
Williams, M. P., Manjourides, J., Smith, L. H., Rainer, C., Hightow‐Weidman, L., & Haley, D. F. (2024) · Journal of Medical Internet Research · doi:10.2196/57619 · Cited by 2
“…highlighted the impact of DHIs on improving mental health outcomes, to our knowledge, this is the first study that directly measures the impact of baseline mental health symptoms on effective engagement in a study focused on a nonmental health–related outcome using a digital intervention [110,111].”
Engagement Patterns and User Feedback from a Dialectical Behavior Therapy App for Eating Disorders
Anderson, C., Fuller‐Tyszkiewicz, M., Messer, M., & Linardon, J. (2025) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-025-00511-4 · Cited by 3
Bucci, S., Zhang, X., Dabrowska, K., Larkin, A., Quayle, E., Schwannauer, M., Varese, F., & Whelan, P. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100858 · Cited by 2
“Engagement with DHIs is associated with therapeutic gains (Gan et al., 2021) but has been noted as a significant challenge in DHIs.”
González-Robles, A., Romero-Ferreiro, V., Roca, P., Rodríguez‐Jiménez, T., & Díaz‐García, A. (2026) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2026.03.008 · Cited by 2
Grzeszczyk, M. K., Adamczyk, P., Marek, S., Pręcikowski, R., Kuś, M., Lelujko, M. P., Blanco, R., Trzcinski, T. P., Sitek, A., Malawski, M., et al. (2023) · PubMed Central · doi:10.48550/arxiv.2302.03616 · Cited by 2
“This is a problem when health intervention outcome is associated with effective engagement with the application [4].”
BackgroundJackson, H. M., Batterham, P. J., Calear, A. L., Ohan, J. L., & Farrer, L. M. (2024) · JMIR Mental Health · doi:10.2196/53794 · Cited by 1
“These indicators provide relatively objective information on use patterns and have been associated with small to moderate improvements in mental health outcomes in some instances [25,26].”
BackgroundVázquez, F. L., Torres, Á., Blanco, V., Bouza, Q., Otero, P., Andrade, E., Simón, M. A., Bueno, A. M., Arrojo, M., Páramo, M., et al. (2024) · BMC Psychiatry · doi:10.1186/s12888-024-06076-5 · Cited by 1
“Additionally, given the importance of user engagement to achieve therapeutic outcomes in this format [74], strategies to improve adherence are proposed, such as support, personalized feedback, or reminders [75, 76].”
BackgroundKoh, D. M.-Y., De-Min, A. A., Jones, L., Chandrasekara, D., Wu, L., Cardamone-Breen, M. C., Olivier, P. L., & Yap, M. B. H. (2025) · Children and Youth Services Review · doi:10.1016/j.childyouth.2025.108260 · Cited by 1
Senyard, E., Rowe, A., Krishnamoorthy, G., Spence, S. H., Donovan, C. L., & March, S. (2025) · JMIR Pediatrics and Parenting · doi:10.2196/60523 · Cited by 1
“These findings are consistent with those of a recent systematic review and meta-analysis of 35 predominantly adult dMH interventions (both therapist-supported and self-help), which found that greater engagement was associated with stronger treatment gains [8].”
InfluentialGuertler, D., Baumann, S., Möhring, A., Krause, K., Freyer‐Adam, J., Ulbricht, S., Bischof, G., Batra, A., Rumpf, H., Wurm, S., et al. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.04.095 · Cited by 2
Enhancing wellbeing in cancer care: Engagement in smart-messaging programmes for symptom management
Boutry, C., Mays, C., Kochar, A., Moghaddam, N. G., Patel, P., Watson, E. J., Dicks, M., Gibbons, F., Malins, S., & Rathbone, J. (2025) · Digital Health · doi:10.1177/20552076251330383 · Cited by 1
“43 This underscores the signi fi cance of prioritising engagement when implementing such interventions.”
O’Sullivan, C., Coppinger, T. C., Dhanapala, I., & Merrotsy, A. (2025) · Mental Health and Digital Technologies · doi:10.1108/mhdt-03-2025-0018 · Cited by 2
Simpson, K. M., McHardy, R. J. W., Zhou, J. W., Levasseur-Puhach, S., Côté, C. M., Braun, M., Clement, F. M., MacKinnon, A., Nickel, N. C., Afifi, T. O., et al. (2025) · BMC Psychiatry · doi:10.1186/s12888-025-06964-4 · Cited by 4
Bastien, L., Cho, S., Petrovic, J., Khoury, B., Bloom, E. L., & Heath, N. L. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100868 · Cited by 1
“…health interventions (both guided and unguided) for general adult populations, a meta-analysis found that improvements in outcomes such as mental health, self-compassion, and psychological distress were modestly associated with greater engagement with digital interventions (Gan et al., 2021).”
Arias-Murcia, S. E., Sánchez-Castro, J. C., Hrynyschyn, R., & Stock, C. (2025) · Exploration of Digital Health Technologies · doi:10.37349/edht.2025.101161 · Cited by 1
Tello, A. L., Pérez-Gómez, A., Toledo-Lozano, C. G., Callejas-Gómez, M. D. P., Durón-Figueroa, R., Moreno-Coutiño, A. B., Ramirez-Treviño, A., Nava, S., de Velasco, D. A. O.-G., Zárate, E., et al. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1659374 · Cited by 1
Ramia, J. A., Khoury, S., Armani, N., Cuijpers, P. J., Carswell, K., Hof, E. V., Zoghbi, E., Sijbrandij, M., & Chammay, R. E. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1665093 · Cited by 1
Varsi, C., Børøsund, E., Ehlers, S. L., Clark, M. M., Andrykowski, M. A., & Nes, L. S. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1703540 · Cited by 1
Doğan, K., ten Klooster, I., Kip, H., Haeri, M. A., & Kelders, S. M. (2026) · Behaviour and Information Technology · doi:10.1080/0144929x.2026.2619659 · Cited by 1
“It has been proposed as an important factor in the context of DHIs as it is associated with increased adherence and lower drop-out rates, potentially increasing DHI effectiveness (Delaney et al. 2021; Gan et al. 2021; Mclaughlin et al. 2021).”
Bernier, L., Murray, G. E., Kratzer, M. P. L., & Palfai, T. P. (2026) · Journal of Clinical Psychology in Medical Settings · doi:10.1007/s10880-026-10127-2 · Cited by 1
“This gap is not unique to chronic pain research; across digital mental health and other health behavior interventions, limited work has examined which forms of engagement contribute to symptom improvement or behavior change (Gan et al., 2021).”
Naumann, E. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.100952 · Cited by 2
Baum, G., Hawrilenko, M., Cascalheira, C. J., Ward, E. J., Graupensperger, S. A., Brown, M. D., & Chekroud, A. M. (2026) · Population Health Management · doi:10.1177/19427891261420041 · Cited by 1
Kronthaler, S. M., Kosman, E., Tissen‐Diabaté, T., Mühle, E., Weber-Schallauer, L., Pross, T., Karsten, M. M., Beier, K. M., & Hatzler, D. L. (2026) · Journal of Medical Internet Research · doi:10.2196/86042 · Cited by 1
“This aligns with meta-analytic evidence showing that engagement modestly predicts improved outcomes, with specific indicators such as module completion being most predictive [83].”
Mirzadegan, I. A., Lewis, E. M., Cole, S. L., & Meyer, A. M. (2026) · JMIR Pediatrics and Parenting · doi:10.2196/79898 · Cited by 1
“Meta-analytic work has shown that engagement in a DHP for mental health is positively associated with clinical improvement, regardless of the intervention focus or whether the intervention is guided or unguided [7].”
Identifying what works in mental health apps through meta-regression analyses of 169 trials
Kraiss, J. T., Fiß, F., Chakhssi, F., Aktas, F. B., Koelen, J. A., & Simões, J. P. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02466-z · Cited by 1
McAlpine, R. G., Jaglinska, M., Jedlovszky, K., Kuc, J., Castro, A., Piot, A., Timmermann, C., Skipper, J. I., Sacchet, M. D., & Kamboj, S. K. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-107512 · Cited by 1
Dynamic Personalized Optimization: An AI Functionality Framework for Digital Therapeutics
Rim, D. (2026) · JMIR Medical Informatics · doi:10.2196/75256 · Cited by 1
Koller, C. N., Blanchard, M., Mettler, J., Prétat, T., Azevedo, P. M., & Hügle, T. (2026) · JMIR mhealth and uhealth · doi:10.2196/78827 · Cited by 1
“It was found that increased user engagement with digital interventions is linked to greater improvements in outcomes [4].”
The use of telemedicine in palliative care in low- and middle-income countries: A scoping review
Goodman, W., Bhattarai, M., Shahzad, H., Chen, S., Burke, S. M., & Allsop, M. J. (2026) · Palliative Medicine · doi:10.1177/02692163261441432 · Cited by 1
“51,52 Therefore, understanding the level of engagement with telemedicine approaches is important for their implementation in healthcare systems in LMICs and future research should ensure this is measured.”
Luo, Y., Inoue, K., Tajika, A., Toyomoto, R., Sakata, M., Akechi, T., Horikoshi, M., Noma, H., & Furukawa, T. A. (2026) · Psychological Medicine · doi:10.1017/s0033291726104279 · Cited by 1
“Unlike face-to-face CBT, smartphone-based programs lack direct therapist interaction, which may result in low engagement and diminished treatment effects (Gan et al., 2021; Torous et al., 2018).”
Rudnicki, K., Joris, G., Meyvis, F., Verschraegen, M., & Poels, K. (2026) · Behavioral Sciences · doi:10.3390/bs16050778 · Cited by 1
Kusier, A. O., Dalsgaard, C., Hoffmann, S. H., Thygesen, L. C., & Folker, A. P. (2026) · JMIR Human Factors · doi:10.2196/91017 · Cited by 1
“Evidence suggests that the level of engagement with digital interventions is critical for achieving positive mental health outcomes [13].”
Akdağ, R., van der Voort, A., Tessensohn, L. M., Versteeg, J., & Vogelaar, S. (2026) · JMIR Formative Research · doi:10.2196/94578 · Cited by 1
Digitally enabled asynchronous remote medical management of anxiety and depression: A cohort study
Perlman, A., Pickman, Y., Dreyfuss, M., Manes, I., Bak, P., Souroujon, D., Paz, E., Ebbert, J. O., & Zeltzer, D. (2024) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x241233788 · Cited by 1
Wu, J.-C., Ram, N., Marks, J. G., Streeper, N. M., & Conroy, D. E. (2024) · Chinese Political Science Review · doi:10.1007/s41111-024-00255-1 · Cited by 1
“…e’ engagement can drive behavioral change and clinical outcomes, a recent study suggested that summary measures of engagement with digital interventions (e.g., number of logins or activities completed) might predict clinical outcomes seen over longer periods, like months or year (Gan et al. 2021).”
Men’s experiences of HeadsUpGuys: A strength-based case study
Ogrodniczuk, J. S., Beharry, J., Storey, Q. K., & Oliffe, J. L. (2024) · Health Education Journal · doi:10.1177/00178969241279649 · Cited by 1
“…with an improvement in all of the visitor impression variables, which is consistent with the conclusions from a recent systematic review and meta-analysis that found that the more users engaged with digital interventions, the greater their improvements in mental health outcomes (Gan et al., 2021).”
Pivoting: Responding to the Mental Health Needs of Youth of Color With Technology
Anderson, R. E., McCall, M. P., & Otaka, N. (2024) · Current Directions in Psychological Science · doi:10.1177/09637214241280614 · Cited by 1
“Despite the promise of digital interventions to improve scale and uptake, their proposed benefits can be hampered by low engagement or adherence (Gan et al., 2021).”
Metadata
- Authors
- Gan, D. Z. Q., McGillivray, L., Han, J., Christensen, H., & Torok, M.
- Affiliations
- Black Dog Institute (AU)
- Corresponding author
- Gan, Daniel Z. Q.
- Venue
- Frontiers in Digital Health
- Publisher
- Frontiers Media SA
- Published
- 2021-11-04
- Volume
- 3
- Pages
- 764079-764079
- ISSN
- 2673-253X (ISSN-L)
- PMC
- PMC8599127
- OpenAlex
- W3213740945
- Semantic Scholar
- e547b3c221a962dd1b4e073942362ba3e9bc08fa
- Cited by
- 305
- References
- 74
- Citations by year
- 2026: 82 · 2025: 95 · 2024: 71 · 2023: 42 · 2022: 14 · 2021: 1
- FWCI
- 5.14
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Digital Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Australian Government
- National Health and Medical Research Council
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Medicine › Psychiatry and Mental health › Mental Health Treatment and Access
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Keywords
- digital mental health interventions, user engagement, mental health outcomes, mental health treatment gap, therapeutic gain
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
