The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Smartphone apps for depression and anxiety: a systematic review and meta-analysis of techniques to increase engagement
Open accessAbstract
Meta-analyses have shown that digital mental health apps can be efficacious in reducing symptoms of depression and anxiety. However, real-world usage of apps is typically not sustained over time, and no studies systematically examine which features increase sustained engagement with apps or the relationship between engagement features and clinical efficacy. We conducted a systematic search of the literature to identify empirical studies that (1) investigate standalone apps for depression and/or anxiety in symptomatic participants and (2) report at least one measure of engagement. Features intended to increase engagement were categorized using the persuasive system design (PSD) framework and principles of behavioral economics. Twenty-five studies with 4159 participants were included in the analysis. PSD features were commonly used, whereas behavioral economics techniques were not. Smartphone apps were efficacious in treating symptoms of anxiety and depression in randomized controlled trials, with overall small-to-medium effects (g = 0.2888, SE = 0.0999, z(15) = 2.89, p = 0.0119, Q(df = 14) = 41.93, p < 0.0001, I2 = 66.6%), and apps that employed a greater number of engagement features as compared to the control condition had larger effect sizes (β = 0.0450, SE = 0.0164, t(15) = 2.7344, p = 0.0161). We observed an unexpected negative association between PSD features and engagement, as measured by completion rate (β = −0.0293, SE = 0.0121, t(17) = 02.4142, p = 0.0281). Overall, PSD features show promise for augmenting app efficacy, though engagement, as reflected in study completion, may not be the primary factor driving this association. The results suggest that expanding the use of PSD features in mental health apps may increase clinical benefits and that other techniques, such as those informed by behavioral economics, are employed infrequently.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
How to perform a meta-analysis with R: a practical tutorial
Balduzzi, S., Rücker, G., & Schwarzer, G. (2019) · Evidence Based Mental Health · doi:10.1136/ebmental-2019-300117 · Cited by 5,827
Maps of Bounded Rationality: Psychology for Behavioral Economics
Kahneman, D. (2003) · American Economic Review · doi:10.1257/000282803322655392 · Cited by 5,377
Persuasive Systems Design: Key Issues, Process Model, and System Features
Oinas-Kukkonen, H., & Harjumaa, M. (2009) · Communications of the Association for Information Systems · doi:10.17705/1cais.02428 · Cited by 1,663
Wang, J., Mann, F., Lloyd-Evans, B., Ma, R., & Johnson, S. (2018) · BMC Psychiatry · doi:10.1186/s12888-018-1736-5 · Cited by 1,494
Persuasive System Design Does Matter: a Systematic Review of Adherence to Web-based Interventions
Kelders, S. M., Kok, R. N., Ossebaard, H. C., & Van Gemert-Pijnen, J. E. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2104 · Cited by 1,308
Firth, J., Torous, J., Nicholas, J., Carney, R., Pratap, A., Rosenbaum, S., & Sarris, J. (2017) · World Psychiatry · doi:10.1002/wps.20472 · Cited by 1,097
Inkster, B., Sarda, S., & Subramanian, V. (2018) · doi:10.2196/preprints.12106 · Cited by 1,063
The impact of guidance on Internet-based mental health interventions — A systematic review
Baumeister, H., Reichler, L., Munzinger, M., & Lin, J. (2014) · Internet Interventions · doi:10.1016/j.invent.2014.08.003 · Cited by 1,037
Objective user engagement with mental health apps: systematic search and panel-based usage analysis
Baumel, A., Muench, F., Edan, S., & Kane, J. M. (2019) · Journal of Medical Internet Research · doi:10.2196/14567 · Cited by 965
Research works printed but not linked: 21
- Patel, M. S., Asch, D. A. & Volpp, K. G. Framing financial incentives to increase physical activity among overweight and obese adults. Ann. Intern. Med. 165, 600 (2016). · DOI
- Patel, M. S. et al. A randomized, controlled trial of lottery-based financial incentives to increase physical activity among overweight and obese adults. Am. J. Health Promot 32, 1568–1575 (2018). · DOI
- Huotari, K. & Hamari, J. Defining gamification: a service marketing perspective. in Proceeding of the 16th International Academic MindTrek Conference on - MindTrek ’12 17 (ACM Press, 2012). https://doi.org/10.1145/2393132.2393137. · DOI · Paper page
- Kelders, S. M., Sommers-Spijkerman, M. & Goldberg, J. Investigating the direct impact of a gamified versus nongamified well-being intervention: an exploratory experiment. J. Med. Internet Res. 20, e247 (2018). · DOI
- Pramana, G. et al. Using mobile health gamification to facilitate cognitive behavioral therapy skills practice in child anxiety treatment: open clinical trial. JMIR Serious Games 6, e9 (2018). · DOI
- Bakker, D. & Rickard, N. Engagement with a cognitive behavioural therapy mobile phone app predicts changes in mental health and wellbeing: MoodMission. Aust. Psychol. 54, 245–260 (2019). · DOI
- Caplan, S., Sosa Lovera, A. & Reyna Liberato, P. A feasibility study of a mental health mobile app in the Dominican Republic: the untold story. Int. J. Ment. Health 47, 311–345 (2018). · DOI
- Enock, P. M., Hofmann, S. G. & McNally, R. J. Attention bias modification training via smartphone to reduce social anxiety: a randomized, controlled multi-session experiment. Cogn. Ther. Res. 38, 200–216 (2014). · DOI · Paper page
- Mehrotra, S., Sudhir, P., Rao, G., Thirthalli, J. & Srikanth, T. K. Development and pilot testing of an internet-based self-help intervention for depression for indian users. Behav. Sci. 8, 36 (2018). · DOI
- Pratap, A. et al. Using mobile apps to assess and treat depression in hispanic and latino populations: fully remote randomized clinical trial. J. Med. Internet Res. 20, e10130 (2018). · DOI
- Lim, M. H. et al. A pilot digital intervention targeting loneliness in youth mental health. Front. Psychiatry 10, 604 (2019). · DOI
- Kahneman, D. A psychological perspective on economics. Am. Econ. Rev. 93, 162–168 (2003). · DOI · Paper page
- Vlaev, I., King, D., Darzi, A. & Dolan, P. Changing health behaviors using financial incentives: a review from behavioral economics. BMC Public Health 19, 1059 (2019). · DOI · Paper page
- Torous, J., Michalak, E. E. & O’Brien, H. L. Digital health and engagement-looking behind the measures and methods. JAMA Netw. Open 3, e2010918 (2020). · DOI
- Patel, M. S. et al. Effectiveness of behaviorally designed gamification interventions with social incentives for increasing physical activity among overweight and obese adults across the united states: the STEP UP randomized clinical trial. JAMA Intern. Med. 1–9, https://doi.org/10.1001/jamainternmed.2019.3505 (2019). · DOI · Paper page
- Moher, D., Liberati, A., Tetzlaff, J. & Altman, D. G. PRISMA Group Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 339, b2535 (2009). · DOI · Paper page
- Qu, C., Sas, C., Daudén Roquet, C. & Doherty, G. Functionality of top-rated mobile apps for depression: systematic search and evaluation. JMIR Ment. Health 7, e15321 (2020). · DOI
- Chan, S., Torous, J., Hinton, L. & Yellowlees, P. Towards a framework for evaluating mobile mental health apps. Telemed. J. E Health 21, 1038–1041 (2015). · DOI · Paper page
- Higgins, J. P. T., Thompson, S. G., Deeks, J. J. & Altman, D. G. Measuring inconsistency in meta-analyses. BMJ 327, 557–560 (2003). · DOI · Paper page
- Duval, S. & Tweedie, R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 56, 455–463 (2000). · DOI · Paper page
- Viechtbauer, W. Conducting meta-analyses in R with the metafor package. J Stat Softw 36, 1–48 (2010). · DOI · Paper page
Cited by
Artificially intelligent chatbots in digital mental health interventions: a review
Boucher, E. M., Harake, N. R., Ward, H. E., Stoeckl, S. E., Vargas, J., Minkel, J., Parks, A. C., & Zilca, R. (2021) · Expert Review of Medical Devices · doi:10.1080/17434440.2021.2013200 · Cited by 440
“engagement features [31], though other meta-analyses found significant pooled effects for depression but not for other psychological outcomes, including anxiety [32].”
BackgroundGan, D. Z. Q., McGillivray, L., Han, J., Christensen, H., & Torok, M. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.764079 · Cited by 305
“In addition, a review of efficacy studies of smartphone apps for depression and anxiety found that apps which incorporated more elements aimed at promoting user engagement had larger effect sizes (17).”
BackgroundDeniz-Garcia, A., Fabelo, H., Rodriguez-Almeida, A. J., Zamora-Zamorano, G., Castro-Fernandez, M., Alberiche Ruano, M. D. P., Solvoll, T., Granja, C., Schopf, T. R., Callico, G. M., et al. (2023) · Journal of Medical Internet Research · doi:10.2196/44030 · Cited by 268
Hartman, C. A., Larsson, H., Vos, M., Bellato, A., Libutzki, B., Solberg, B. S., Chen, Q., Du Rietz, E., Mostert, J. C., Kittel-Schneider, S., et al. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105209 · Cited by 104
Serrano-Ripoll, M. J., Zamanillo-Campos, R., Fiol-DeRoque, M. A., Castro, A., & Ricci-Cabello, I. (2022) · JMIR mHealth and uHealth · doi:10.2196/29621 · Cited by 96
Metadata
- Authors
- Wu, A., Scult, M. A., Barnes, E. D., Betancourt, J. A., Falk, A., & Gunning, F. M.
- Venue
- npj Digital Medicine
- Published
- 2021-02-11
- Volume
- 4
- Issue
- 1
- Pages
- 20-20
- ISSN
- 2398-6352 (ISSN-L)
- OpenAlex
- W3128708259
- Cited by
- 161
- References
- 43
- FWCI
- 5.09
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Funders
- Weill Cornell Medicine Children’s Health Council Investigator Fund
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Smartphone apps for depression and anxiety: a systematic review and meta-analysis of techniques to increase engagement. https://app.cochat.ai/papers/10.1038/s41746-021-00386-8
- The paper
- Wu, A., Scult, M. A., Barnes, E. D., Betancourt, J. A., Falk, A., & Gunning, F. M. (2021). Smartphone apps for depression and anxiety: a systematic review and meta-analysis of techniques to increase engagement. npj Digital Medicine. https://doi.org/10.1038/s41746-021-00386-8
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.