Mobile phone-based interventions for mental health: A systematic meta-review of 14 meta-analyses of randomized controlled trials
Open accessThe DOI matches its Crossref record; 70 of the 73 references it cites were found in a registry or an index; 7 of the 81 sources its claims cite were checked for support.
Abstract
Mobile phone-based interventions have been proposed as a means for reducing the burden of disease associated with mental illness. While numerous randomized controlled trials and meta-analyses have investigated this possibility, evidence remains unclear. We conducted a systematic meta-review of meta-analyses examining mobile phone-based interventions tested in randomized controlled trials. We synthesized results from 14 meta-analyses representing 145 randomized controlled trials and 47,940 participants. We identified 34 effect sizes representing unique pairings of participants, intervention, comparisons, and outcome (PICO) and graded the strength of the evidence as using umbrella review methodology. We failed to find convincing evidence of efficacy (i.e., n > 1000, p < 10−6, I2 < 50%, absence of publication bias); publication bias was rarely assessed for the representative effect sizes. Eight effect sizes provided highly suggestive evidence (i.e., n > 1000, p < 10−6), including smartphone interventions outperforming inactive controls on measures of psychological symptoms and quality of life (d s = 0.32 to 0.47) and text message-based interventions outperforming non-specific controls and active controls for smoking cessation (d s = 0.31 and 0.19, respectively). The magnitude of effects and strength of evidence tended to diminish as comparison conditions became more rigorous (i.e., inactive to active, non-specific to specific). Four effect sizes provided suggestive evidence, 14 effect sizes provided weak evidence, and eight effect sizes were non-significant. Despite substantial heterogeneity, no moderators were identified. Adverse effects were not reported. Taken together, results support the potential of mobile phone-based interventions and highlight key directions to guide providers, policy makers, clinical trialists, and meta-analysts working in this area.
Overview
CoChat Summary
TL;DR
This meta-review of 14 meta-analyses representing 145 randomized controlled trials and 47,940 participants found no convincing evidence for mobile phone-based mental health interventions. Eight of 34 representative effect sizes provided highly suggestive evidence, but effects were generally small and weakened as comparison conditions became more rigorous.
Key findings
- No effect size met criteria for convincing evidence (n > 1000, p < 10−6, I2 < 50%, absence of publication bias); publication bias was tested for only 2 of 34 eligible effect sizes.
- Eight effect sizes provided highly suggestive evidence: smartphone interventions outperformed inactive controls for anxiety (ds = 0.32 and 0.45), depression (d = 0.32), stress (d = 0.47), and quality of life (d = 0.35); mobile phone and text message-based interventions outperformed non-specific controls for smoking cessation (ds = 0.30 and 0.31); text message-based interventions outperformed active controls for smoking cessation (d = 0.19).
- Four effect sizes provided suggestive evidence, 14 provided weak evidence, and 8 were non-significant; effects generally diminished from inactive to active and specific active comparisons.
- No moderators were identified, and adverse events were not reported; only one meta-analysis evaluated adverse events, finding they were not reported in the seven included trials.
Methods
- Preregistered systematic meta-review following PRISMA; searched six databases through October 31, 2020, for meta-analyses of mobile phone-based interventions on mental health outcomes, requiring at least four RCTs, standardized effect sizes with 95% CIs, and comparison conditions categorized as inactive, active, non-specific, specific, or adjunct.
- Included 14 meta-analyses representing 145 unique primary RCTs and 47,940 participants; identified 34 representative effect sizes based on the largest sample per PICO pairing and graded certainty using umbrella review criteria.
- Meta-analysis quality was assessed with the NIH tool (average 7.07/8); risk of bias, publication bias, and moderator tests were extracted when available.
Limitations
- The paper states that findings are limited by the available meta-analytic and primary study literature, possible underestimation of evidence strength due to lack of publication bias assessment, substantial heterogeneity within and between meta-analyses, rare valid moderator tests, and limited standardized assessment of intervention quality.
- The paper also notes diminished confidence due to risk of bias from lack of blinding of personnel/participants and incomplete outcome data, and it states that adverse events were almost entirely absent from the meta-analytic literature.
- Evident from the text: only one meta-analysis tested moderators within an eligible effect size, publication bias was assessed for only 2 of 34 representative effect sizes, and no eligible effect sizes focused exclusively on youth or adolescent samples.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
6 Valid · 1 Not valid · 66 Not checked · 64 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Compared to specific active controls, meditation apps and smartphone interventions were not significantly superior in their effects on anxiety (ds = 0.26 and 0.09, respectively [21, 39]).”
§ Mobile phone-based interventions for mental health: A systematic meta-review of 14 meta-analyses of randomized controlled trials > Results > Synthesis of results
What the sources say
- Supportedcites [21] J Linardon, 2019, The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials., World Psychiatry source
The ‘Smartphone interventions vs. active comparisons’ section reports a pooled effect of g=0.09 for four comparisons, with a 95% confidence interval of −0.21 to 0.39. Table 1 identifies these as comparisons for generalized anxiety symptoms, supporting the claim that the effect was not statistically significant. The paper labels the effect size g rather than d. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Smartphone interventions did not differ significantly from specific active controls (d = 0.13 [21]) or when tested as an adjunct to treatment (d = 0.26 [21]).”
§ Mobile phone-based interventions for mental health: A systematic meta-review of 14 meta-analyses of randomized controlled trials > Results > Synthesis of results
What the sources say
- Supportedcites [21] J Linardon, 2019, The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials., World Psychiatry source
The “Smartphone interventions vs. active comparisons” section reports an effect on depressive symptoms of g=0.13 (95% CI: −0.07 to 0.34), which is not statistically significant. Table 1 also reports g=0.26 (95% CI: −0.09 to 0.61) for smartphones used as an adjunct intervention. The source labels these effect sizes g, rather than d, but supports the cited findings. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Compared to inactive controls, smartphone interventions showed highly suggestive evidence of small magnitude effects on stress (d = 0.47 [21]).”
§ Mobile phone-based interventions for mental health: A systematic meta-review of 14 meta-analyses of randomized controlled trials > Results > Synthesis of results
What the sources say
- Partly supportedcites [21] J Linardon, 2019, The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials., World Psychiatry sourceExperimental — accuracy not yet measured
Table 2 (page not available in our copy) reports a small effect on stress versus waitlist controls of Hedges’ g=0.47, supporting the numerical result for that inactive-control subgroup. It reports g=0.35 for all control conditions combined, however, and does not classify the evidence as “highly suggestive” or report the subgroup estimate as d. (LLM, conf=0.88)
boostedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Apps did not differ from non-specific controls on a combination of smoking and drinking outcomes (d = 0.18 [20]).”
§ Mobile phone-based interventions for mental health: A systematic meta-review of 14 meta-analyses of randomized controlled trials > Results > Synthesis of results
What the sources say
- Partly supportedcites [20] KK Weisel, 2019, Standalone smartphone apps for mental health—a systematic review and meta-analysis., Npj Digit Med sourceExperimental — accuracy not yet measured
The Results > Substance use section reports no significant difference for pooled smoking and drinking outcomes, with an effect size of 0.18. It reports Hedges’ g, not d, and describes the comparison as against controls rather than a specifically defined group of non-specific controls; the Results table lists several inactive control types. Page numbers are unavailable in our copy. (LLM, conf=0.83)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Apps did not differ from non-specific controls (d = 0.14 [20]).”
§ Mobile phone-based interventions for mental health: A systematic meta-review of 14 meta-analyses of randomized controlled trials > Results > Synthesis of results
What the sources say
- Partly supportedcites [20] KK Weisel, 2019, Standalone smartphone apps for mental health—a systematic review and meta-analysis., Npj Digit Med sourceExperimental — accuracy not yet measured
The Results section 'Self-injurious thoughts and behaviors' supports the qualitative claim: suicidal ideation was assessed in four comparisons (n = 286, three using the same intervention) and the pooled effect was non-significant (g = −0.14, 95%CI −0.37 to 0.1, P = 0.246), i.e. apps did not differ from controls; the Abstract likewise lists suicidal ideation among the outcomes with no significant effect (g = −0.14 to 0.18). The comparator characterisation is consistent with the review, whose 19 trials all used non-specific/inactive controls (k = 2 no-treatment, k = 9 wait-list, k = 3 information-only, k = 4 sham, k = 1 mood chart; e.g. Tighe 2017 'Inactive: WLC'). However, the number attributed is not what the source reports: it gives Hedges' g = −0.14 (point estimate marginally favouring control), not a positive Cohen's d = 0.14, so the sign/direction and the effect-size metric in the citing sentence are unsupported, even though the magnitude and null result match. (LLM, conf=0.70)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
74 more claims · 1 with their cited sources checked
Create a free account to see all 137 claims and to verify the 74 references still open.
- It may state two separate claims as one. 40
- Probably not a substantial claim. 28
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“There is strong evidence that psychological interventions can be effectively delivered through mHealth115–117.”
From A common factors perspective on mindfulness-based interventions (2022) · cites this paper as [116]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, the evidence for the effectiveness of DMHIs has been limited, with a meta-analysis of randomised controlled trials (RCTs) finding only small effect sizes, potential publication bias, and a lack of active controls8–10.”
From “It happened to be the perfect thing”: experiences of generative AI chatbots for mental health (2024) · cites this paper as [8]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“At the same time, a growing body of research shows that interactions with chatbots can contribute to improving their users’ mental health and quality of life, especially if the specific needs of the user match with chatbot capabilities (22, 24, 27, 31, 34, 40, 49–56).”
From Waiting for a digital therapist: three challenges on the path to psychotherapy delivered by artificial intelligence (2023) · cites this paper as [50]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Further, when intervention effects have been compared with those of rigorous control conditions, the effects can be weaker [17].”
In the citing paper
“Another factor affecting the inclusion of DMHIs in routine clinical care is that many of these interventions have been designed as “self-help” or “stand-alone” products [8,43,44].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Mobile phone-based interventions for mental health: A systematic meta-review of 14 meta-analyses of randomized controlled trials > Results > Synthesis of results
“and as weak for 14 effect sizes (38.2%).”
§ Mobile phone-based interventions for mental health: A systematic meta-review of 14 meta-analyses of randomized controlled trials > Discussion
“Thirteen of the 34 effect sizes provided weak (p < .050) evidence for mobile phone-based interventions.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“and as weak for 14 effect sizes (38.2%).”
§ Mobile phone-based interventions for mental health: A systematic meta-review of 14 meta-analyses of randomized controlled trials > Results > Synthesis of results
Reported 38.2 · recomputed 41.176471
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Mobile phone-based interventions for mental health: A systematic meta-review of 14 meta-analyses of randomized controlled trials > Results > Study characteristics
“Meta-analyses were published between 2015 and 2020.”
§ Mobile phone-based interventions for mental health: A systematic meta-review of 14 meta-analyses of randomized controlled trials > Results > Study characteristics
“| Gál (2021) [39] | adults | n/a | meditation apps | anxiety, depression, stress, wellbeing | 34 | Cochrane | 7 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“Eight effect sizes (26.5%) were non-significant.”
§ Mobile phone-based interventions for mental health: A systematic meta-review of 14 meta-analyses of randomized controlled trials > Results > Synthesis of results
Reported 26.5 · recomputed 23.529412
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“25 (73.5%) were significant at p < .050.”
§ Mobile phone-based interventions for mental health: A systematic meta-review of 14 meta-analyses of randomized controlled trials > Results > Synthesis of results
Reported 25 · recomputed 26
Citations
Cites
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2013) · doi:10.1176/appi.books.9780890425596 · Cited by 116,369
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · cited 1×
“io/ s2t67/) and followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA [31]) guidelines.”
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Dermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
Wang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,657
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225 · cited 1×
“, odds ratio, hazard ratio) were converted into Cohen’s d using standard methods [44].”
MethodCicchetti, D. V. (1994) · Psychological Assessment · doi:10.1037/1040-3590.6.4.284 · Cited by 9,349
Mental health and the COVID-19 pandemic
Pfefferbaum, B., & North, C. S. (2020) · New England Journal of Medicine · doi:10.1056/nejmp2008017 · Cited by 4,378 · cited 1×
“high income countries there is rising concern around unmet mental health needs, particularly during the COVID-19 pandemic [3].”
BackgroundThe Great Psychotherapy Debate: The Evidence for What Makes Psychotherapy Work
Wampold, B. E. (2015) · doi:10.4324/9780203582015 · Cited by 1,773
How important are the common factors in psychotherapy? An update
Wampold, B. E. (2015) · World Psychiatry · doi:10.1002/wps.20238 · Cited by 1,728
Psychological testing and psychological assessment: A review of evidence and issues.
Meyer, G. J., Finn, S. E., Eyde, L. D., Kay, G. G., Moreland, K. L., Dies, R. R., Eisman, E. J., Kubiszyn, T. W., & Reed, G. M. (2001) · American Psychologist · doi:10.1037//0003-066x.56.2.128 · Cited by 1,469 · cited 1×
“Scaled at a population level, even small effects may meaningfully impact public health [54].”
BackgroundFirth, J., Solmi, M., Wootton, R. E., Vancampfort, D., Schuch, F. B., Hoare, E., Gilbody, S., Torous, J., Teasdale, S. B., Jackson, S. E., et al. (2020) · World Psychiatry · doi:10.1002/wps.20773 · Cited by 1,242 · cited 1×
“We coded the quality of each meta-analysis using the National Institutes of Health (NIH) Quality Assessment of Systematic Reviews and Meta-Analyses Tool [41] and interpreted scores in line with previous meta-reviews [42] where 7 or 8 indicates “good,” 4 to 6 “fair,” and 0 to 3 “poor” quality.”
MethodSelf-citationBorenstein, M., Hedges, L. V., Higgins, J. P. T., Rothstein, H. R., & Ben Van Den, A. (2020) · International Coaching Psychology Review · doi:10.53841/bpsicpr.2020.15.2.121 · Cited by 1,215
Behavioral activation treatments of depression: a meta-analysis.
Cuijpers, P., van Straten, A., & Warmerdam, L. (2007) · Clinical Psychology Review · doi:10.1016/j.cpr.2006.11.001 · Cited by 1,144 · cited 1×
“Unlike the psychotherapy literature in which treatments are often time-limited and meta-analyses can cleanly examine effects at post-treatment versus follow-up [68], mobile phone-based interventions particularly those without guidance can be easily accessed ongoingly thus making demarcation of “post-treatment” more am…”
BackgroundFirth, J., Torous, J., Nicholas, J., Carney, R., Pratap, A., Rosenbaum, S., & Sarris, J. (2017) · World Psychiatry · doi:10.1002/wps.20472 · Cited by 1,097
Self-citationWampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., & Ahn, H.-N. (1997) · Psychological Bulletin · doi:10.1037/0033-2909.122.3.203 · Cited by 1,091
Mobile phone-based interventions for smoking cessation
Whittaker, R., McRobbie, H., Bullen, C., Rodgers, A., & Gu, Y. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006611.pub4 · Cited by 1,079
Objective user engagement with mental health apps: systematic search and panel-based usage analysis
Baumel, A., Muench, F., Edan, S., & Kane, J. M. (2019) · Journal of Medical Internet Research · doi:10.2196/14567 · Cited by 965 · cited 1×
“, health apps generally and mental health apps specifically [56, 57]).”
BackgroundLinardon, J., Cuijpers, P., Carlbring, P., Messer, M., & Fuller‐Tyszkiewicz, M. (2019) · World Psychiatry · doi:10.1002/wps.20673 · Cited by 860 · cited 9×
“32 [21]) and meditation apps showed weak evidence of small magnitude effects (d = 0.”
BackgroundInfluentialKuyken, W., Warren, F. C., Taylor, R. S., Whalley, B., Crane, C., Bondolfi, G., Hayes, R., Huijbers, M., Ma, H., Schweizer, S., et al. (2016) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2016.0076 · Cited by 821
Mobile Text Messaging for Health: A Systematic Review of Reviews
Hall, A. K., Cole-Lewis, H., & Bernhardt, J. M. (2015) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031914-122855 · Cited by 815 · cited 1×
“, text message-based interventions) have been studied even longer [8].”
BackgroundFu, R., Gartlehner, G., Grant, M., Shamliyan, T., Sedrakyan, A., Wilt, T. J., Griffith, L., Oremus, M., Raina, P., Ismaila, A., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.08.010 · Cited by 730 · cited 1×
“, psychiatric symptoms, stress, quality of life, addictive behaviors that are included in the DSM-5 [33] such as alcohol and tobacco use) (3) based on a minimum of four randomized controlled trials (RCTs) [34] (4) using comparison conditions that could be categorized as inactive, active, non-specific, specific, or adj…”
BackgroundTen simple rules for conducting umbrella reviews
Fusar-Poli, P., & Radua, J. (2018) · Evidence Based Mental Health · doi:10.1136/ebmental-2018-300014 · Cited by 724 · cited 5×
“We evaluated the certainty of the evidence using criteria drawn from umbrella review methodology [22, 24].”
BackgroundMethodInfluentialFirth, J., Torous, J., Nicholas, J., Carney, R., Rosenbaum, S., & Sarris, J. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.04.046 · Cited by 675 · cited 2×
“Firth (2017a) [38] adults mental health concerns smartphone intervention anxiety 9 Cochrane 6”
BackgroundSelf-citation(2019) · Cited by 633
Goldberg, S. B., Riordan, K. M., Sun, S., & Davidson, R. J. (2022) · Perspectives on Psychological Science · doi:10.1177/1745691620968771 · Cited by 520 · cited 1×
“, interventions tested using more rigorous specific active control conditions appear less effective [35]).”
BackgroundSelf-citationCan Mobile Health Technologies Transform Health Care?
Steinhubl, S. R., Muse, E. D., & Topol, E. J. (2013) · JAMA · doi:10.1001/jama.2013.281078 · Cited by 496 · cited 1×
“Psychological interventions delivered via mobile technology have been proposed as a means for reducing the global burden of disease associated with mental illness [53].”
BackgroundMobile phone text messaging and app-based interventions for smoking cessation
Whittaker, R., McRobbie, H., Bullen, C., Rodgers, A., Gu, Y., & Dobson, R. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006611.pub5 · Cited by 452 · cited 5×
“Whittaker (2019) [27] adults/ adolescents smoking text messaging / smartphone app smoking cessation 17 Cochrane, GRADE 8”
BackgroundInfluentialStandalone smartphone apps for mental health—a systematic review and meta-analysis
Weisel, K. K., Fuhrmann, L. M., Berking, M., Baumeister, H., Cuijpers, P., & Ebert, D. D. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0188-8 · Cited by 447 · cited 6×
“While our review included several meta-analyses investigating text messaging for smoking cessation, only one effect size included assessment of other addictive behaviors (smoking and drinking combined [20]).”
BackgroundInfluentialPratap, A., Neto, E. C., Snyder, P., Stepnowsky, C., Elhadad, N., Grant, D., Mohebbi, M. H., Mooney, S., Suver, C., Wilbanks, J., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0224-8 · Cited by 410 · cited 1×
“, health apps generally and mental health apps specifically [56, 57]).”
How to obtain the confidence interval from a P value
Altman, D. G., & Bland, J. M. (2011) · BMJ · doi:10.1136/bmj.d2090 · Cited by 402 · cited 1×
“P-values were calculated from confidence intervals using standard methods [47].”
MethodTorous, J., & Roberts, L. W. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.0262 · Cited by 362 · cited 2×
“, by the United States Food and Drug Administration) may help guide consumers and providers [10].”
BackgroundSelf-citationBarbui, C., Purgato, M., Abdulmalik, J., Acarturk, C., Eaton, J., Gastaldon, C., Gureje, O., Hanlon, C., Jordans, M., Lund, C., et al. (2020) · The Lancet Psychiatry · doi:10.1016/s2215-0366(19)30511-5 · Cited by 349
Mobile Apps for Mental Health Issues: Meta-Review of Meta-Analyses
Lecomte, T., Potvin, S., Corbière, M., Guay, S., Samson, C., Cloutier, B., Francoeur, A., Pennou, A., & Khazaal, Y. (2020) · JMIR mHealth and uHealth · doi:10.2196/17458 · Cited by 336 · cited 3×
“[25] conducted the only meta-review investigating the effects of mobile phone-based interventions on mental health.”
BackgroundInfluentialMobile Health Interventions for Improving Health Outcomes in Youth: A Meta-analysis
Fedele, D. A., Cushing, C. C., Fritz, A., Amaro, C. M., & Ortega, A. (2017) · JAMA Pediatrics · doi:10.1001/jamapediatrics.2017.0042 · Cited by 316 · cited 1×
“Future primary studies with youth and adolescent samples may be warranted, particularly given evidence of clinical need, acceptability, and potential efficacy of mobile health interventions for this group [66].”
BackgroundUser engagement in mental health apps: A review of measurement, reporting, and validity
Ng, M. M., Firth, J., Minen, M., & Torous, J. (2019) · Psychiatric Services · doi:10.1176/appi.ps.201800519 · Cited by 293 · cited 1×
“While the first generation of these studies focused on feasibility and acceptability, the accumulating evidence clearly indicates that people suffering from all mental health conditions, including even schizophrenia (which may be associated with severe disability that in theory could interfere with feasibility and acc…”
BackgroundSelf-citationGál, É., Ștefan, S., & Cristea, I. A. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.09.134 · Cited by 289 · cited 3×
“However, instead of separating comparison conditions based on whether they were active or inactive, some meta-analyses [39] separated comparison conditions based on whether they were intended to be therapeutic [32].”
BackgroundRizvi, S. L., Dimeff, L. A., Skutch, J., Carroll, D., & Linehan, M. M. (2011) · Behavior Therapy · doi:10.1016/j.beth.2011.01.003 · Cited by 271
Text Messaging-Based Interventions for Smoking Cessation: A Systematic Review and Meta-Analysis
Scott-Sheldon, L. A. J., Lantini, R., Jennings, E. G., Thind, H., Rosen, R. K., Salmoirago-Blotcher, E., & Bock, B. C. (2016) · JMIR mHealth and uHealth · doi:10.2196/mhealth.5436 · Cited by 231 · cited 2×
“19 [51]) and weak evidence of small magnitude effects when tested as an adjunct to other support on smoking cessation (d = 0.”
Champion, L., Economides, M., & Chandler, C. (2018) · PLOS ONE · doi:10.1371/journal.pone.0209482 · Cited by 225 · cited 2×
“The included primary studies tested a variety of interventions, some of which are widely used commercial products (e.g., Headspace [60, 61]) while others are interventions designed in collaboration with researchers (e.g., Wildflowers [62]).”
BackgroundDeady, M., Choi, I., Calvo, R. A., Glozier, N., Christensen, H., & Harvey, S. B. (2017) · BMC Psychiatry · doi:10.1186/s12888-017-1473-1 · Cited by 222
Association of Antidepressant Use With Adverse Health Outcomes: A Systematic Umbrella Review
Dragioti, E., Solmi, M., Favaro, A., Fusar-Poli, P., Dazzan, P., Thompson, T., Stubbs, B., Firth, J., Fornaro, M., Tsartsalis, D., et al. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.2859 · Cited by 215 · cited 3×
“Effect sizes were evaluated based on the certainty of the evidence using previously employed metrics [22,24].”
MethodThe digital placebo effect: mobile mental health meets clinical psychiatry
Torous, J., & Firth, J. (2016) · The Lancet Psychiatry · doi:10.1016/s2215-0366(15)00565-9 · Cited by 200 · cited 1×
“This literature would be strengthened through more studies including active and ideally specific active controls which are capable of identifying key intervention ingredients and disentangling intervention-specific elements from the effect of expectancy and other non-specific factors alone (although these non-specific…”
BackgroundSelf-citationMobile phone-based interventions for smoking cessation
Whittaker, R., Borland, R., Bullen, C., Lin, R. B., McRobbie, H., & Rodgers, A. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006611.pub2 · Cited by 196 · cited 3×
“Whittaker (2016) [37] adults/ adolescents smoking text messaging / smartphone app smoking cessation 12 Cochrane, GRADE 7”
Jonsson, U., Alaie, I., Parling, T., & Arnberg, F. K. (2014) · Contemporary Clinical Trials · doi:10.1016/j.cct.2014.02.005 · Cited by 175 · cited 2×
“While limited assessment of adverse events is an issue in the broader psychotherapy literature [58], it may be particularly problematic to ignore within the context of mobile phone-based interventions which often include less support than traditional treatments.”
BackgroundBy the numbers: ratings and utilization of behavioral health mobile applications
Carlo, A. D., Hosseini Ghomi, R., Renn, B. N., & Areán, P. A. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0129-6 · Cited by 164 · cited 1×
“The landscape has expanded to such an extent that professional societies have created evaluation frameworks [11] and healthcare regulators around the world are exploring new ways to categorize and regulate this burgeoning space [12–14].”
BackgroundLinardon, J. (2020) · Behavior Therapy · doi:10.1016/j.beth.2019.10.002 · Cited by 133 · cited 1×
“Linardon (2020) [71] adults/ adolescents n/a smartphone app self-compassion, mindfulness/acceptance, depression/distress 33 Cochrane 7”
Walsh, K. M., Saab, B. J., & Farb, N. A. (2019) · JMIR Mental Health · doi:10.2196/10844 · Cited by 123
Loo Gee, B., Griffiths, K. M., & Gulliver, A. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv043 · Cited by 92 · cited 2×
“Gee (2016) [49] adults/ adolescents n/a ecological momentary interventions anxiety 6 Cochrane 8”
BackgroundWasil, A. R., Gillespie, S., Patel, R., Petre, A., Venturo-Conerly, K. E., Shingleton, R. M., Weisz, J. R., & DeRubeis, R. J. (2020) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000604 · Cited by 86
Preventing depression using a smartphone app: a randomized controlled trial
Deady, M., Glozier, N., Calvo, R., Johnston, D., Mackinnon, A., Milne, D., Choi, I., Gayed, A., Peters, D., Bryant, R., et al. (2022) · Psychological Medicine · doi:10.1017/s0033291720002081 · Cited by 85
Rodriguez-Villa, E., & Torous, J. B. (2019) · BMC Medicine · doi:10.1186/s12916-019-1447-x · Cited by 80
Self-citationAlliance with an unguided smartphone app: validation of the digital working alliance inventory
Goldberg, S. B., Baldwin, S. A., Riordan, K. M., Torous, J., Dahl, C. J., Davidson, R. J., & Hirshberg, M. J. (2022) · Assessment · doi:10.1177/10731911211015310 · Cited by 74
Self-citationWhich eHealth interventions are most effective for smoking cessation? A systematic review
Do, H., Tran, B., Pham, Q., Nguyen, L., Tran, T., Latkin, C., Dunne, M. P., & Baker, P. (2018) · Patient Preference and Adherence · doi:10.2147/ppa.s169397 · Cited by 64 · cited 1×
“Do (2018) [70] adults/ adolescents smoking text messaging smoking cessation 6 Cochrane 8”
BackgroundLee, T. T., & Kesselheim, A. S. (2018) · Annals of Internal Medicine · doi:10.7326/m17-2715 · Cited by 62
Text Messaging for Addiction: A Review
Keoleian, V., Polcin, D., & Galloway, G. P. (2015) · Journal of Psychoactive Drugs · doi:10.1080/02791072.2015.1009200 · Cited by 59 · cited 1×
“One potentially rich area may be the application of text message-based interventions for addictive behaviors [63].”
BackgroundHow Kaiser Permanente Created a Mental Health and Wellness Digital Ecosystem
Mordecai, D., Histon, T., Neuwirth, E., Heisler, W. S., Kraft, A., Bang, Y., Franchino, K., Taillac, C., & Nixon, J. P. (2021) · NEJM Catalyst · doi:10.1056/cat.20.0295 · Cited by 56 · cited 1×
“Large healthcare systems like Kaiser Permanente that had already integrated mobile health apps into care before COVID-19 doubled the number of app referrals in the first months of the pandemic (40,000 in May 2020 [15]), representing the continued expansion of mobile mental health interventions.”
BackgroundInkster, B., & Digital Mental Health Data Insights Group (DMHDIG) (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2020.578902 · Cited by 54
Senanayake, B., Wickramasinghe, S. I., Chatfield, M. D., Hansen, J., Edirippulige, S., & Smith, A. C. (2019) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x19875852 · Cited by 46 · cited 1×
“Senanayake (2019) [72] adults/ adolescents depression text messaging depression 7 Joanna Briggs Institute 7”
BackgroundParker, L. M., Bero, L. A., Gillies, D. M., Raven, M. K., & Grundy, Q. (2018) · International Journal of Health Policy and Management · doi:10.15171/ijhpm.2018.117 · Cited by 42
Hrynyschyn, R., & Dockweiler, C. (2021) · JMIR mHealth and uHealth · doi:10.2196/24703 · Cited by 35 · cited 1×
“Yet surprisingly, this research has not yielded clear answers with recent 2021 systematic reviews of app interventions reporting outcomes ranging from “inconsistent results” [18] to “proven effectiveness” [19].”
BackgroundConnected Mental Health: Systematic Mapping Study
(2020) · doi:10.2196/19950 · Cited by 33 · cited 1×
“From fewer than five studies per year in 2011 [16] to now hundreds per year–there exist thousands of research studies on mobile health interventions.”
BackgroundSpohr, S. A., Taxman, F. S., & Walters, S. T. (2015) · Addictive Behaviors · doi:10.1016/j.addbeh.2015.07.005 · Cited by 27 · cited 3×
“[52] tested nine moderators as predictors of smoking cessation in text message-based interventions.”
BackgroundInfluentialAre Mental Health Apps Adequately Equipped to Handle Users in Crisis?
Parrish, E. M., Filip, T. F., Torous, J. B., Nebeker, C., Moore, R. C., & Depp, C. A. (2021) · Crisis · doi:10.1027/0227-5910/a000785 · Cited by 24
Self-citationPlanas, R., & Yuguero, O. (2021) · Informatics for Health and Social Care · doi:10.1080/17538157.2021.1887196 · Cited by 19 · cited 1×
“Yet surprisingly, this research has not yielded clear answers with recent 2021 systematic reviews of app interventions reporting outcomes ranging from “inconsistent results” [18] to “proven effectiveness” [19].”
BackgroundCox, K. L., Allida, S. M., & Hackett, M. L. (2021) · Health Education Journal · doi:10.1177/0017896920959368 · Cited by 14 · cited 4×
“When multiple effect sizes within a given PICO subcategory were reported (e.g., effects of text messaging on depression symptoms, effects of text messaging on Beck Depression Inventory [48]), we selected the effect size based on the larger sample.”
BackgroundMethodInfluential
Cited by
“It happened to be the perfect thing”: experiences of generative AI chatbots for mental health
Siddals, S., Torous, J., & Coxon, A. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00097-4 · Cited by 241
“However,theevidencefortheeffectivenessofDMHIshasbeenlimited,with ameta-analysisofrandomisedcontrolledtrials(RCTs) fi nding only small effect sizes, potential publication bias, and a lack of active controls 8 – 10 .”
Self-citationLinardon, J., Torous, J., Firth, J., Cuijpers, P., Messer, M., & Fuller‐Tyszkiewicz, M. (2024) · World Psychiatry · doi:10.1002/wps.21183 · Cited by 207
“…complex machine learning algorithms also enable treatment content to be regularly updated and personalized to the needs of the user based on data collected passively (e.g., global positioning system coordinates to infer social determinants of health) and actively (e.g., symptom tracking) 13,14 .”
BackgroundSelf-citationProviding Human Support for the Use of Digital Mental Health Interventions: Systematic Meta-review
Werntz, A., Amado, S., Jasman, M., Ervin, A., & Rhodes, J. E. (2023) · Journal of Medical Internet Research · doi:10.2196/42864 · Cited by 158
“Although there have been some meta-reviews investigating the effects of DMHIs on mental health [8,51], only 1 scoping review has examined the role of human support in DMHI use [39].”
Assessment of Mental Health Services Available Through Smartphone Apps
Camacho, E., Cohen, A., & Torous, J. B. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.48784 · Cited by 112
Self-citationGrodniewicz, J. P., & Hohol, M. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1190084 · Cited by 100
Metadata
- Authors
- Goldberg, S. B., Lam, S. U., Simonsson, O., Torous, J., & Sun, S.
- Affiliations
- University of Wisconsin–Madison (US)
- Beth Israel Deaconess Medical Center (US)
- Brown University (US)
- Corresponding author
- Goldberg, Simon B.
- Venue
- PLOS Digital Health
- Publisher
- Public Library of Science (PLoS)
- Published
- 2022-01-18
- Volume
- 1
- Issue
- 1
- Pages
- e0000002
- ISSN
- 2767-3170 (ISSN-L)
- PMC
- PMC8881800
- OpenAlex
- W4206302417
- Cited by
- 237
- References
- 73
- Citations by year
- 2026: 31 · 2025: 61 · 2024: 62 · 2023: 55 · 2022: 28
- FWCI
- 45.48
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLOS Digital Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- OSF Preprints (OSF Preprints) (submittedVersion, cc-by)
- OSF Preprints (OSF Preprints) (submittedVersion)
- Funders
- National Center for Complementary and Alternative Medicine (K23AT010879, K23AT011173)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Keywords
- mobile phone intervention, smartphone-based intervention, text messaging intervention, mental health, psychological symptoms, quality of life, smoking cessation, intervention efficacy, publication bias, umbrella review, meta-review
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.