Assessing Patient Adherence to and Engagement With Digital Interventions for Depression in Clinical Trials: Systematic Literature Review
Open accessThe DOI matches its Crossref record; 99 of the 99 references it cites were found in a registry or an index.
Abstract
Background: New approaches to the treatment of depression are necessary for patients who do not respond to current treatments or lack access to them because of barriers such as cost, stigma, and provider shortage. Digital interventions for depression are promising; however, low patient engagement could limit their effectiveness. Objective: This systematic literature review (SLR) assessed how participant adherence to and engagement with digital interventions for depression have been measured in the published literature, what levels of adherence and engagement have been reported, and whether higher adherence and increased engagement are linked to increased efficacy. Methods: We focused on a participant population of adults (aged ≥18 years) with depression or major depressive disorder as the primary diagnosis and included clinical trials, feasibility studies, and pilot studies of digital interventions for treating depression, such as digital therapeutics. We screened 756 unique records from Ovid MEDLINE, Embase, and Cochrane published between January 1, 2000, and April 15, 2022; extracted data from and appraised the 94 studies meeting the inclusion criteria; and performed a primarily descriptive analysis. Otsuka Pharmaceutical Development & Commercialization, Inc (Princeton, New Jersey, United States) funded this study. Results: This SLR encompassed results from 20,111 participants in studies using 47 unique web-based interventions (an additional 10 web-based interventions were not described by name), 15 mobile app interventions, 5 app-based interventions that are also accessible via the web, and 1 CD-ROM. Adherence was most often measured as the percentage of participants who completed all available modules. Less than half (44.2%) of the participants completed all the modules; however, the average dose received was 60.7% of the available modules. Although engagement with digital interventions was measured differently in different studies, it was most commonly measured as the number of modules completed, the mean of which was 6.4 (means ranged from 1.0 to 19.7) modules. The mean amount of time participants engaged with the interventions was 3.9 (means ranged from 0.7 to 8.4) hours. Most studies of web-based (34/45, 76%) and app-based (8/9, 89%) interventions found that the intervention group had substantially greater improvement for at least 1 outcome than the control group (eg, care as usual, waitlist, or active control). Of the 14 studies that investigated the relationship between engagement and efficacy, 9 (64%) found that increased engagement with digital interventions was significantly associated with improved participant outcomes. The limitations of this SLR include publication bias, which may overstate engagement and efficacy, and low participant diversity, which reduces the generalizability. Conclusions: Patient adherence to and engagement with digital interventions for depression have been reported in the literature using various metrics. Arriving at more standardized ways of reporting adherence and engagement would enable more effective comparisons across different digital interventions, studies, and populations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rush, A. J., Trivedi, M. H., Wisniewski, S. R., Nierenberg, A. A., Stewart, J. W., Warden, D., Niederehe, G., Thase, M. E., Lavori, P. W., Lebowitz, B. D., et al. (2006) · American Journal of Psychiatry · doi:10.1176/ajp.2006.163.11.1905 · Cited by 5,643
Mobile app rating scale: a new tool for assessing the quality of health mobile apps
Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D., & Mani, M. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3422 · Cited by 2,233 · cited 1×
“…App Advisor [180], One Mind’s PsyberGuide [181], Organisation for the Review of Care and Health Apps’s App Library [182], Mobile App Rating Scale [183], mobile health app trustworthiness checklist [184], Framework to Assist Stakeholders in Technology Evaluation for Recovery [185], and Mhealth…”
Prevalence of depression symptoms in US adults before and during the COVID-19 pandemic
Ettman, C. K., Abdalla, S. M., Cohen, G. H., Sampson, L., Vivier, P. M., & Galea, S. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.19686 · Cited by 2,138 · cited 1×
“Depression prevalence in the United States increased more than 3-fold during the COVID-19 pandemic; individuals with lower income and greater exposure to stress have had an even higher risk of depression during the pandemic [11].”
Epidemiology of Adult DSM-5 Major Depressive Disorder and Its Specifiers in the United States
Hasin, D. S., Sarvet, A. L., Meyers, J. L., Saha, T. D., Ruan, W. J., Stohl, M., & Grant, B. F. (2018) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.4602 · Cited by 1,737 · cited 1×
“Approximately 1 in 3 adults with MDD in the United States does not receive treatment [15].”
Appraising the Evidence: Reviewing Disparate Data Systematically
Hawker, S., Payne, S., Kerr, C., Hardey, M., & Powell, J. (2002) · Qualitative Health Research · doi:10.1177/1049732302238251 · Cited by 1,216
Torous, J., Bucci, S., Bell, I. H., Kessing, L. V., Faurholt‐Jepsen, M., Whelan, P., Carvalho, A. F., Keshavan, M., Linardon, J., & Firth, J. (2021) · World Psychiatry · doi:10.1002/wps.20883 · Cited by 1,199 · cited 5×
“Digital health technologies may help address many of the treatment needs of patients with depression [32,33].”
InfluentialPerski, O., Blandford, A., West, R., & Michie, S. (2017) · Translational Behavioral Medicine · doi:10.1007/s13142-016-0453-1 · Cited by 1,193
Adherence in Internet Interventions for Anxiety and Depression
Christensen, H., Griffiths, K. M., & Farrer, L. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1194 · Cited by 1,133 · cited 4×
“However, rather than engagement, Christensen et al [57] defined adherence as “the extent to which individuals experience the content of the Internet intervention.””
InfluentialBorghouts, J., Eikey, E., Mark, G., De Leon, C., Schueller, S. M., Schneider, M., Stadnick, N., Zheng, K., Mukamel, D., & Sorkin, D. H. (2021) · Journal of Medical Internet Research · doi:10.2196/24387 · Cited by 1,069 · cited 3×
“Extroversion, fatigue, and more severe depression symptoms were barriers to engagement [187].”
InfluentialObjective user engagement with mental health apps: systematic search and panel-based usage analysis
Baumel, A., Muench, F., Edan, S., & Kane, J. M. (2019) · Journal of Medical Internet Research · doi:10.2196/14567 · Cited by 965 · cited 1×
“Another real-world study of 93 Android mental health apps found that only 3.9% of the users engaged with the apps 15 days later, and only 3.3% of the users sustained engagement on day 30 [44].”
InfluentialAndrews, G., Basu, A., Cuijpers, P., Craske, M. G., McEvoy, P., English, C. L., & Newby, J. M. (2018) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2018.01.001 · Cited by 922 · cited 2×
“A meta-analysis of internet-delivered cognitive behavioral therapy studies revealed that interventions had higher effect sizes when compared with a waitlist control versus a CAU group [166].”
Karyotaki, E., Efthimiou, O., Miguel, C., Maas genannt Bermpohl, F., Furukawa, T. A., Cuijpers, P., Individual Patient Data Meta-Analyses for Depression (IPDMA-DE) Collaboration, Riper, H., Patel, V., Mira, A., et al. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.4364 · Cited by 866
Fleming, T., Bavin, L., Lucassen, M., Stasiak, K., Hopkins, S., & Merry, S. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9275 · Cited by 670 · cited 4×
“However, there is no universally accepted definition for either term; both are used interchangeably in the literature, and each word has been used when defining the other [46,52-55].”
InfluentialHarnessing context sensing to develop a mobile intervention for depression
Burns, M. N., Begale, M., Duffecy, J., Gergle, D., Karr, C. J., Giangrande, E., & Mohr, D. C. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1838 · Cited by 646 · cited 2×
“RenderX ensured equal access by giving participants phones with phone plans for the duration of the study [116].”
Semahegn, A., Torpey, K., Manu, A., Assefa, N., Tesfaye, G., & Ankomah, A. (2020) · Systematic Reviews · doi:10.1186/s13643-020-1274-3 · Cited by 555
Standalone smartphone apps for mental health—a systematic review and meta-analysis
Weisel, K. K., Fuhrmann, L. M., Berking, M., Baumeister, H., Cuijpers, P., & Ebert, D. D. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0188-8 · Cited by 447 · cited 1×
“However, there is no consensus in the field yet on whether digital interventions are most useful before beginning psychotherapy or best when used as stand-alone treatments [54,68,86,168,169].”
Geographic Variation in the Supply of Selected Behavioral Health Providers
Andrilla, C. H. A., Patterson, D. G., Garberson, L. A., Coulthard, C., & Larson, E. H. (2018) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2018.01.004 · Cited by 419 · cited 1×
“Many obstacles prevent people from receiving treatment, including cost [16], a shortage of mental health care providers [16], an uneven distribution of providers [16,17], long wait times for appointments [18], side effects of medication [19], and stigma about mental illness [16].”
Gilbody, S., Littlewood, E., Hewitt, C., Brierley, G., Tharmanathan, P., Araya, R., Barkham, M., Bower, P., Cooper, C., Gask, L., et al. (2015) · BMJ · doi:10.1136/bmj.h5627 · Cited by 409
Titov, N., Andrews, G., Davies, M., McIntyre, K., Robinson, E., & Solley, K. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010939 · Cited by 406
Garrido, S., Millington, C., Cheers, D., Boydell, K., Schubert, E., Meade, T., & Nguyen, Q. V. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00759 · Cited by 400 · cited 1×
“User engagement data from various studies indicate challenges with initial and sustained engagement [43-45].”
Digital interventions for the treatment of depression: a meta-analytic review
Moshe, I., Terhorst, Y., Philippi, P., Domhardt, M., Cuijpers, P., Cristea, I., Pulkki-Råback, L., Baumeister, H., & Sander, L. B. (2021) · Psychological Bulletin · doi:10.1037/bul0000334 · Cited by 390 · cited 1×
“…have shown that interventions delivered with support have larger effect sizes than unguided interventions, it has been suggested that newer unguided digital interventions with features such as engagement reminders may have similar efficacy to interventions delivered with clinician support [173].”
Berger, T., Hämmerli, K., Gubser, N., Andersson, G., & Caspar, F. (2011) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2011.616531 · Cited by 383
Mongelli, F., Georgakopoulos, P., & Pato, M. T. (2020) · Focus · doi:10.1176/appi.focus.20190028 · Cited by 331 · cited 1×
“Many obstacles prevent people from receiving treatment, including cost [16], a shortage of mental health care providers [16], an uneven distribution of providers [16,17], long wait times for appointments [18], side effects of medication [19], and stigma about mental illness [16].”
Sieverink, F., Kelders, S. M., & van Gemert-Pijnen, J. E. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8578 · Cited by 331 · cited 9×
“Therefore, it may be useful to investigate >1 metric of engagement when measuring a dose-response relationship because the metrics alone may impact the ability to detect such a relationship [55].”
InfluentialCBT for depression: a pilot RCT comparing mobile phone vs. computer
Watts, S., Mackenzie, A., Thomas, C., Griskaitis, A., Mewton, L., Williams, A., & Andrews, G. (2013) · BMC Psychiatry · doi:10.1186/1471-244x-13-49 · Cited by 329 · cited 1×
“%) or a difference in efficacy in terms of preintervention to postintervention scores on the PHQ-9, Beck Depression Inventory-II, or Kessler Psychological Distress Scale [156].”
Wagner, B., Horn, A. B., & Maercker, A. (2014) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.06.032 · Cited by 311 · cited 1×
“Many studies provided coach feedback to participants after each lesson [72,75,77,99,106,108,111,122,124,130,135,141,144,145,154,155,157,158].”
IntelliCare: An Eclectic, Skills-Based App Suite for the Treatment of Depression and Anxiety
Mohr, D. C., Tomasino, K. N., Lattie, E. G., Palac, H. L., Kwasny, M. J., Weingardt, K., Karr, C. J., Kaiser, S. M., Rossom, R. C., Bardsley, L. R., et al. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6645 · Cited by 301
Page, M. J., Sterne, J. A. C., Higgins, J. P. T., & Egger, M. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1468 · Cited by 294
User engagement in mental health apps: A review of measurement, reporting, and validity
Ng, M. M., Firth, J., Minen, M., & Torous, J. (2019) · Psychiatric Services · doi:10.1176/appi.ps.201800519 · Cited by 293 · cited 1×
“The search yielded 1181 records, and an additional 590 records were identified by manually searching the reference lists of relevant review articles [57,65-68”
Clinician-Assisted Internet-Based Treatment is Effective for Panic: A Randomized Controlled Trial
Wims, E., Titov, N., Andrews, G., & Choi, I. (2010) · Australian & New Zealand Journal of Psychiatry · doi:10.3109/00048671003614171 · Cited by 290
Johansson, R., Sjöberg, E., Sjögren, M., Johnsson, E., Carlbring, P., Andersson, T., Rousseau, A., & Andersson, G. (2012) · PLoS ONE · doi:10.1371/journal.pone.0036905 · Cited by 268 · cited 2×
“Many studies provided coach feedback to participants after each lesson [72,75,77,99,106,108,111,122,124,130,135,141,144,145,154,155,157,158].”
Roepke, A. M., Jaffee, S. R., Riffle, O. M., McGonigal, J., Broome, R., & Maxwell, B. (2015) · Games for Health Journal · doi:10.1089/g4h.2014.0046 · Cited by 265
Internet-Based Treatment for Adults with Depressive Symptoms: Randomized Controlled Trial
Warmerdam, L., van Straten, A., Twisk, J., Riper, H., & Cuijpers, P. (2008) · Journal of Medical Internet Research · doi:10.2196/jmir.1094 · Cited by 255
State of the Field of Mental Health Apps
Neary, M., & Schueller, S. M. (2018) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2018.01.002 · Cited by 247 · cited 1×
“…and tools to help evaluate digital interventions, including the American Psychiatric Association’s App Advisor [180], One Mind’s PsyberGuide [181], Organisation for the Review of Care and Health Apps’s App Library [182], Mobile App Rating Scale [183], mobile health app trustworthiness…”
Goldberg, S. B., Lam, S. U., Simonsson, O., Torous, J., & Sun, S. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000002 · Cited by 237 · cited 2×
“Similarly, in a meta-review of meta-analyses examining the outcomes of RCTs of app-based interventions, Goldberg et al [167] reported that digital interventions had a small but significant effect on improving depression symptoms compared with inactive controls but not compared with active controls.”
Clinician-Assisted Internet-Based Treatment is Effective for Depression: Randomized Controlled Trial
Perini, S., Titov, N., & Andrews, G. (2009) · Australian & New Zealand Journal of Psychiatry · doi:10.1080/00048670902873722 · Cited by 231 · cited 2×
“Many studies provided coach feedback to participants after each lesson [72,75,77,99,106,108,111,122,124,130,135,141,144,145,154,155,157,158].”
Vernmark, K., Lenndin, J., Bjärehed, J., Carlsson, M., Karlsson, J., Öberg, J., Carlbring, P., Eriksson, T., & Andersson, G. (2010) · Behaviour Research and Therapy · doi:10.1016/j.brat.2010.01.005 · Cited by 226
Steffen, A., Nübel, J., Jacobi, F., Bätzing, J., & Holstiege, J. (2020) · BMC Psychiatry · doi:10.1186/s12888-020-02546-8 · Cited by 212 · cited 1×
“Depression has a substantial global humanistic burden [1] and is associated with lower academic performance [2,3], reduced adherence to treatments for other medical conditions [4], impaired quality of life [5,6], mental and somatic comorbidities [7], and premature death [1].”
Urgent need for improved mental health care and a more collaborative model of care
Lake, J., & Turner, M. S. (2017) · The Permanente Journal · doi:10.7812/tpp/17-024 · Cited by 201 · cited 1×
“Given the barriers to access and the limited response to existing treatments, there is an unmet need for new and more personalized treatments for depression [29-31].”
Virtual reality for supporting the treatment of depression and anxiety: scoping review
Baghaei, N., Chitale, V., Hlasnik, A., Stemmet, L., Liang, H.-N., & Porter, R. (2021) · JMIR Mental Health · doi:10.2196/29681 · Cited by 197 · cited 1×
“Digital technologies such as chatbots, telehealth, smartphone apps, and virtual reality are now more widely used in mental health care than before the COVID-19 pandemic [32,34,39].”
Andersson, G., Hesser, H., Veilord, A., Svedling, L., Andersson, F., Sleman, O., Mauritzson, L., Sarkohi, A., Claesson, E., Zetterqvist, V., et al. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.08.022 · Cited by 195
Donkin, L., Hickie, I. B., Christensen, H., Naismith, S. L., Neal, B., Cockayne, N. L., & Glozier, N. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2771 · Cited by 193
de Graaf, L. E., Gerhards, S. A. H., Arntz, A., Riper, H., Metsemakers, J. F. M., Evers, S. M. A. A., Severens, J. L., Widdershoven, G., & Huibers, M. J. H. (2009) · British Journal of Psychiatry · doi:10.1192/bjp.bp.108.054429 · Cited by 190
Meyer, B., Bierbrodt, J., Schröder, J., Berger, T., Beevers, C. G., Weiss, M., Jacob, G., Späth, C., Andersson, G., Lutz, W., et al. (2015) · Internet Interventions · doi:10.1016/j.invent.2014.12.003 · Cited by 188 · cited 1×
“%) found the digital intervention to be effective for at least 1 outcome compared with the control, without finding efficacy to be correlated with engagement or adherence [63,64,87,102].”
Ruwaard, J., Schrieken, B., Schrijver, M., Broeksteeg, J., Dekker, J., Vermeulen, H., & Lange, A. (2009) · Cognitive Behaviour Therapy · doi:10.1080/16506070802408086 · Cited by 183
Amick, H. R., Gartlehner, G., Gaynes, B. N., Forneris, C., Asher, G. N., Morgan, L. C., Coker-Schwimmer, E., Boland, E., Lux, L. J., Gaylord, S., et al. (2015) · BMJ · doi:10.1136/bmj.h6019 · Cited by 181
Choi, I., Zou, J., Titov, N., Dear, B. F., Li, S., Johnston, L., Andrews, G., & Hunt, C. (2012) · Journal of Affective Disorders · doi:10.1016/j.jad.2011.11.003 · Cited by 180 · cited 1×
“Common support measures offered were weekly phone calls with a coach [62,64,73,74,80,81,84,86,94,103,109,116,119,132, 134,139,148,159,161] and adherence reminders [61,63,95,107,112,122,129,144,153,158].”
Anguera, J. A., Jordan, J. T., Castaneda, D., Gazzaley, A., & Areán, P. A. (2016) · BMJ Innovations · doi:10.1136/bmjinnov-2015-000098 · Cited by 172
Graham, A. K., Greene, C. J., Kwasny, M. J., Kaiser, S. M., Lieponis, P., Powell, T., & Mohr, D. C. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.1011 · Cited by 162
Birney, A. J., Gunn, R., Russell, J. K., & Ary, D. V. (2016) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4231 · Cited by 159
Moberg, C., Niles, A., & Beermann, D. (2019) · Journal of Medical Internet Research · doi:10.2196/12556 · Cited by 157 · cited 1×
“Moberg et al [88] found that the participants who used the digital intervention to record more of their thoughts in the app had greater reductions in anxiety and the Depression Anxiety and Stress Scales-21 stress subscale score from the posttreatment time point to follow-up; however, an initial…”
Høifødt, R. S., Lillevoll, K. R., Griffiths, K. M., Wilsgaard, T., Eisemann, M., Waterloo, K., & Kolstrup, N. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2714 · Cited by 154 · cited 2×
“A few studies integrated the digital intervention with face-to-face meetings, either with a therapist [104,112] or in a group counseling session [85,125].”
Characteristics and challenges of the clinical pipeline of digital therapeutics
Patel, N. A., & Butte, A. J. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00370-8 · Cited by 143 · cited 3×
“These tools offer increased access to treatment (including asynchronous access outside the clinic) [34-36], can reduce stigma concerns [35], and have the potential to be cost-effective [34,37,38].”
A randomized controlled trial of internet-based therapy in depression
Moritz, S., Schilling, L., Hauschildt, M., Schröder, J., & Treszl, A. (2012) · Behaviour Research and Therapy · doi:10.1016/j.brat.2012.04.006 · Cited by 140
Renn, B. N., Hoeft, T. J., Lee, H. S., Bauer, A. M., & Areán, P. A. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0077-1 · Cited by 140
Pham, Q., Graham, G., Carrion, C., Morita, P. P., Seto, E., Stinson, J. N., & Cafazzo, J. A. (2019) · JMIR mHealth and uHealth · doi:10.2196/11941 · Cited by 131 · cited 2×
“However, there is no universally accepted definition for either term; both are used interchangeably in the literature, and each word has been used when defining the other [46,52-55].”
Improving the Efficiency of Psychotherapy for Depression: Computer-Assisted Versus Standard CBT
Thase, M. E., Wright, J. H., Eells, T. D., Barrett, M. S., Wisniewski, S. R., Balasubramani, G. K., McCrone, P., & Brown, G. K. (2018) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2017.17010089 · Cited by 131
Personalized machine learning of depressed mood using wearables
Shah, R. V., Grennan, G., Zafar-Khan, M., Alim, F., Dey, S., Ramanathan, D., & Mishra, J. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01445-0 · Cited by 123
Bolier, L., Haverman, M., Kramer, J., Westerhof, G. J., Riper, H., Walburg, J. A., Boon, B., & Bohlmeijer, E. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2603 · Cited by 116 · cited 1×
“Moreover, Bolier et al [113] indicated that participants who completed >1 lesson had significantly greater improvement in their Hospital Anxiety and Depression Scale anxiety subscale scores at 2-month follow-up than those who completed ≤ 1 lesson.”
Diversity in randomized clinical trials of depression: A 36-year review
Polo, A. J., Makol, B. A., Castro, A. S., Colón-Quintana, N., Wagstaff, A. E., & Guo, S. (2019) · Clinical Psychology Review · doi:10.1016/j.cpr.2018.09.004 · Cited by 114
Uptake and usage of IntelliCare: A publicly available suite of mental health and well-being apps
Lattie, E. G., Schueller, S. M., Sargent, E., Stiles-Shields, C., Tomasino, K. N., Corden, M. E., Begale, M., Karr, C. J., & Mohr, D. C. (2016) · Internet Interventions · doi:10.1016/j.invent.2016.06.003 · Cited by 113 · cited 2×
“User engagement data from various studies indicate challenges with initial and sustained engagement [43-45].”
Chien, I., Enrique, A., Palacios, J., Regan, T., Keegan, D., Carter, D., Tschiatschek, S., Nori, A., Thieme, A., Richards, D., et al. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.10791 · Cited by 111 · cited 2×
“…learning and a data set of >54,000 adults using an internet-based cognitive behavioral therapy platform for depression and anxiety, Chien et al [175] identified 5 discrete subtypes of users based on engagement and concluded that the level of engagement was not always proportional to clinical…”
InfluentialRichards, D., Timulak, L., O'Brien, E., Hayes, C., Vigano, N., Sharry, J., & Doherty, G. (2015) · Behaviour Research and Therapy · doi:10.1016/j.brat.2015.10.005 · Cited by 109
Evaluating evaluation frameworks: a scoping review of frameworks for assessing health apps
Lagan, S., Sandler, L., & Torous, J. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-047001 · Cited by 108
Harnessing Peer Support in an Online Intervention for Older Adults with Depression
Tomasino, K. N., Lattie, E. G., Ho, J., Palac, H. L., Kaiser, S. M., & Mohr, D. C. (2017) · The American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2017.04.015 · Cited by 107 · cited 1×
“Common support measures offered were weekly phone calls with a coach [62,64,73,74,80,81,84,86,94,103,109,116,119,132, 134,139,148,159,161] and adherence reminders [61,63,95,107,112,122,129,144,153,158].”
Kelders, S. M., Bohlmeijer, E. T., Pots, W. T. M., & van Gemert-Pijnen, J. E. W. C. (2015) · Behaviour Research and Therapy · doi:10.1016/j.brat.2015.06.014 · Cited by 105 · cited 1×
“A subsequent study by Kelders et al [105] discovered that the completion of all modules and lesson reached were associated with improvement in CES-D and Hospital Anxiety and Depression Scale anxiety subscale scores after intervention and at follow-up.”
Digital therapeutics for mental health: Is attrition the Achilles heel?
Nwosu, A., Boardman, S., Husain, M. M., & Doraiswamy, P. M. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.900615 · Cited by 101
Keyloun, K. R., Hansen, R. N., Hepp, Z., Gillard, P., Thase, M. E., & Devine, E. B. (2017) · CNS Drugs · doi:10.1007/s40263-017-0417-0 · Cited by 99
The association between adherence and outcome in an Internet intervention for depression
Fuhr, K., Schröder, J., Berger, T., Moritz, S., Meyer, B., Lutz, W., Hohagen, F., Hautzinger, M., & Klein, J. P. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.12.028 · Cited by 97 · cited 1×
“%) study differentiated support depending on baseline participant characteristics by offering email support from a therapist to participants with a PHQ-9 score >10 [143].”
Kivi, M., Eriksson, M. C. M., Hange, D., Petersson, E.-L., Vernmark, K., Johansson, B., & Björkelund, C. (2014) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2014.921834 · Cited by 92
Geraedts, A. S., Kleiboer, A. M., Twisk, J., Wiezer, N. M., van Mechelen, W., & Cuijpers, P. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3539 · Cited by 89
Kelders, S. M., Kip, H., & Greeff, J. (2020) · Journal of Medical Internet Research · doi:10.2196/17757 · Cited by 88
Kelders, S. M., Bohlmeijer, E. T., & Van Gemert-Pijnen, J. E. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2258 · Cited by 88
Examining the Reach of Smartphone Apps for Depression and Anxiety
Wasil, A. R., Gillespie, S., Shingleton, R., Wilks, C. R., & Weisz, J. R. (2020) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2019.19090905 · Cited by 87 · cited 3×
“The most popular depression and anxiety apps average 10,000 downloads each month, with thousands of daily users [40].”
Salamanca-Sanabria, A., Richards, D., Timulak, L., Connell, S., Mojica Perilla, M., Parra-Villa, Y., & Castro-Camacho, L. (2020) · JMIR Mental Health · doi:10.2196/13392 · Cited by 83 · cited 1×
“Common support measures offered were weekly phone calls with a coach [62,64,73,74,80,81,84,86,94,103,109,116,119,132, 134,139,148,159,161] and adherence reminders [61,63,95,107,112,122,129,144,153,158].”
Multimodal e-mental health treatment for depression: a feasibility trial
Mohr, D. C., Duffecy, J., Jin, L., Ludman, E. J., Lewis, A., Begale, M., & McCarthy Jr, M. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1370 · Cited by 82 · cited 1×
“Common support measures offered were weekly phone calls with a coach [62,64,73,74,80,81,84,86,94,103,109,116,119,132, 134,139,148,159,161] and adherence reminders [61,63,95,107,112,122,129,144,153,158].”
Using the Internet to enhance the treatment of depression
Robertson, L., Smith, M., Castle, D., & Tannenbaum, D. (2006) · Australasian Psychiatry · doi:10.1080/j.1440-1665.2006.02315.x · Cited by 80
Montero-Marín, J., Araya, R., Pérez-Yus, M. C., Mayoral, F., Gili, M., Botella, C., Baños, R., Castro, A., Romero-Sanchiz, P., López-Del-Hoyo, Y., et al. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5695 · Cited by 75
Löbner, M., Pabst, A., Stein, J., Dorow, M., Matschinger, H., Luppa, M., Maroß, A., Kersting, A., König, H.-H., & Riedel-Heller, S. G. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2018.06.008 · Cited by 72
Schneider, J., Sarrami Foroushani, P., Grime, P., & Thornicroft, G. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.2871 · Cited by 67 · cited 1×
“Common support measures offered were weekly phone calls with a coach [62,64,73,74,80,81,84,86,94,103,109,116,119,132, 134,139,148,159,161] and adherence reminders [61,63,95,107,112,122,129,144,153,158].”
What is needed to eradicate the depression epidemic, and why
Ormel, J., Cuijpers, P., Jorm, A., & Schoevers, R. A. (2020) · Mental Health & Prevention · doi:10.1016/j.mhp.2019.200177 · Cited by 67
Levin, M. E., Hicks, E. T., & Krafft, J. (2022) · Journal of American College Health · doi:10.1080/07448481.2020.1728281 · Cited by 65
Dahne, J., Collado, A., Lejuez, C. W., Risco, C. M., Diaz, V. A., Coles, L., Kustanowitz, J., Zvolensky, M. J., & Carpenter, M. J. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.03.009 · Cited by 62
Heim, E., Ramia, J. A., Hana, R. A., Burchert, S., Carswell, K., Cornelisz, I., Cuijpers, P., El Chammay, R., Noun, P., van Klaveren, C., et al. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100380 · Cited by 61
Karyotaki, E., Klein, A. M., Ciharova, M., Bolinski, F., Krijnen, L., de Koning, L., de Wit, L., van der Heijde, C. M., Ebert, D. D., Riper, H., et al. (2022) · Behaviour Research and Therapy · doi:10.1016/j.brat.2021.104028 · Cited by 60 · cited 1×
“Many studies provided coach feedback to participants after each lesson [72,75,77,99,106,108,111,122,124,130,135,141,144,145,154,155,157,158].”
Stiles-Shields, C., Montague, E., Kwasny, M. J., & Mohr, D. C. (2019) · Psychological Services · doi:10.1037/ser0000261 · Cited by 59 · cited 3×
“Some studies found a dose-response relationship (where users with higher engagement with the technology achieved better outcomes) [61,62], whereas others either found no such relationship [53,63,64] or indicated that users can be impacted by interventions that they do not fully complete [55].”
Richards, D., Duffy, D., Blackburn, B., Earley, C., Enrique, A., Palacios, J., Franklin, M., Clarke, G., Sollesse, S., Connell, S., et al. (2018) · BMC Psychiatry · doi:10.1186/s12888-018-1639-5 · Cited by 52
A Gamified Smartphone-Based Intervention for Depression: Randomized Controlled Pilot Trial
Lukas, C. A., Eskofier, B., & Berking, M. (2021) · JMIR Mental Health · doi:10.2196/16643 · Cited by 49
Duffy, D., Enrique, A., Connell, S., Connolly, C., & Richards, D. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00902 · Cited by 49 · cited 1×
“Although some studies have indicated that digital interventions can decrease symptoms and benefit waitlisted patients before starting therapy [168,170], others have found no benefit [86,171].”
Smith, J., Newby, J. M., Burston, N., Murphy, M. J., Michael, S., Mackenzie, A., Kiln, F., Loughnan, S. A., O'Moore, K. A., Allard, B. J., et al. (2017) · Internet Interventions · doi:10.1016/j.invent.2017.05.001 · Cited by 45
Rahmadiana, M., Karyotaki, E., Schulte, M., Ebert, D. D., Passchier, J., Cuijpers, P., Berger, T., van Ballegooijen, W., Wimbarti, S., & Riper, H. (2021) · JMIR Mental Health · doi:10.2196/20036 · Cited by 44 · cited 1×
“Many studies provided coach feedback to participants after each lesson [72,75,77,99,106,108,111,122,124,130,135,141,144,145,154,155,157,158].”
Predictors of recurrence of major depressive disorder
Lye, M.-S., Tey, Y.-Y., Tor, Y.-S., Shahabudin, A. F., Ibrahim, N., Ling, K.-H., Stanslas, J., Loh, S.-P., Rosli, R., Lokman, K. A., et al. (2020) · PLOS ONE · doi:10.1371/journal.pone.0230363 · Cited by 43
Wallin, E., Norlund, F., Olsson, E. M. G., Burell, G., Held, C., & Carlsson, T. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9690 · Cited by 36 · cited 1×
“Depression has a substantial global humanistic burden [1] and is associated with lower academic performance [2,3], reduced adherence to treatments for other medical conditions [4], impaired quality of life [5,6], mental and somatic comorbidities [7], and premature death [1].”
Economic burden of relapse/recurrence in patients with major depressive disorder
Gauthier, G., Mucha, L., Shi, S., & Guerin, A. (2019) · Journal of Drug Assessment · doi:10.1080/21556660.2019.1612410 · Cited by 36
Richards, D., Timulak, L., Doherty, G., Sharry, J., Colla, A., Joyce, C., & Hayes, C. (2014) · BMC Psychiatry · doi:10.1186/1471-244x-14-147 · Cited by 35 · cited 1×
“Many studies provided coach feedback to participants after each lesson [72,75,77,99,106,108,111,122,124,130,135,141,144,145,154,155,157,158].”
Mira, A., Soler, C., Alda, M., Baños, R., Castilla, D., Castro, A., García-Campayo, J., García-Palacios, A., Gili, M., Hurtado, M., et al. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00325 · Cited by 29
Karyotaki, E., Klein, A. M., Riper, H., Wit, L. D., Krijnen, L., Bol, E., Bolinski, F., Burger, S., Ebert, D. D., Auerbach, R. P., et al. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-028739 · Cited by 28
Feasibility of a worker-directed web-based intervention for employees with depressive symptoms
Geraedts, A. S., Kleiboer, A. M., Wiezer, N. M., Cuijpers, P., van Mechelen, W., & Anema, J. R. (2014) · Internet Interventions · doi:10.1016/j.invent.2014.07.001 · Cited by 17 · cited 1×
“Many studies provided coach feedback to participants after each lesson [72,75,77,99,106,108,111,122,124,130,135,141,144,145,154,155,157,158].”
Cited by
Madrid‐Cagigal, A., Kealy, C., Potts, C., Mulvenna, M. D., Byrne, M., Barry, M. M., & Donohoe, G. (2025) · Early Intervention in Psychiatry · doi:10.1111/eip.70017 · Cited by 78
Acceptability of Digital Mental Health Interventions for Depression and Anxiety: Systematic Review
Lau, C. K. Y., Saad, A., Camara, B., Rahman, D., & Bolea-Alamanac, B. (2024) · Journal of Medical Internet Research · doi:10.2196/52609 · Cited by 51
“There remains a need to establish consensus approaches to analyzing attrition, adherence, and engagement [170], and their relationship with acceptability.”
Drüge, M., Guthardt, L., Haller, E., Michalak, J., & Apolinário-Hagen, J. (2024) · Advances in Experimental Medicine and Biology · doi:10.1007/978-981-97-4402-2_11 · Cited by 11
Metadata
- Authors
- Forbes, A., Keleher, M. R., Venditto, M., & DiBiasi, F.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2023-08-11
- Volume
- 25
- Pages
- e43727
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W4382361283
- Cited by
- 94
- References
- 99
- FWCI
- 3.26
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.