The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Conducting a fully mobile and randomised clinical trial for depression: access, engagement and expense
Open accessAbstract
Importance: Advances in mobile technology have resulted in federal and industry-level initiatives to facilitate large-scale clinical research using smart devices. Although the benefits of technology to expand data collection are obvious, assumptions about the reach of mobile research methods (access), participant willingness to engage in mobile research protocols (engagement), and the cost of this research (cost) remain untested. Objective: To assess the feasibility of a fully mobile randomised controlled trial using assessments and treatments delivered entirely through mobile devices to depressed individuals. Design: Using a web-based research portal, adult participants with depression who also owned a smart device were screened, consented and randomised to 1 of 3 mental health apps for treatment. Assessments of self-reported mood and cognitive function were conducted at baseline, 4, 8 and 12 weeks. Physical and social activity was monitored daily using passively collected phone use data. All treatment and assessment tools were housed on each participant's smart phone or tablet. Interventions: A cognitive training application, an application based on problem-solving therapy, and a mobile-sensing application promoting daily activities. Results: Access: We screened 2923 people and enrolled 1098 participants in 5 months. The sample characteristics were comparable to the 2013 US census data. Recruitment via Craigslist.org yielded the largest sample. Engagement: Study engagement was high during the first 2 weeks of treatment, falling to 44% adherence by the 4th week. Cost: The total amount spent on for this project, including staff costs and β testing, was $314 264 over 2 years. Conclusions and relevance: These findings suggest that mobile randomised control trials can recruit large numbers of participants in a short period of time and with minimal cost, but study engagement remains challenging. Trial registration number: NCT00540865.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In depression, mobile apps have tracked changes in patient-reported mood [27-29] and have been used as part of randomized controlled trials to evaluate treatment efficacy [30].”
From Wearable technology for high-frequency cognitive and mood assessment in major depressive disorder: longitudinal observational study (2019) · cites this paper as [30]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
The Patient Health Questionnaire-2: Validity of a Two-Item Depression Screener
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2003) · Medical Care · doi:10.1097/01.mlr.0000093487.78664.3c · Cited by 6,359
Ferrari, A. J., Charlson, F. J., Norman, R. E., Patten, S. B., Freedman, G., Murray, C. J. L., Vos, T., & Whiteford, H. A. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001547 · Cited by 3,115 · cited 1×
“15 It is the leading cause of disability world-wide, 16 17 and the enrolment of depressed individuals into clinical trials is difficult.”
Sheehan, D. V., Harnett-Sheehan, K., & Raj, B. A. (1996) · International Clinical Psychopharmacology · doi:10.1097/00004850-199606003-00015 · Cited by 1,440
Assessing Psychiatric Impairment in Primary Care with the Sheehan Disability Scale
Leon, A. C., Olfson, M., Portera, L., Farber, L., & Sheehan, D. V. (1997) · The International Journal of Psychiatry in Medicine · doi:10.2190/t8em-c8yh-373n-1uwd · Cited by 987 · cited 1×
“27 28 We also asked participants to rate their health on a scale of excellent to poor.”
BackgroundmHealth: A Mechanism to Deliver More Accessible, More Effective Mental Health Care
Price, M., Yuen, E. K., Goetter, E. M., Herbert, J. D., Forman, E. M., Acierno, R., & Ruggiero, K. J. (2014) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.1855 · Cited by 488
Measuring the global burden of disease and epidemiological transitions: 2002–2030
López, A. D., & Mathers, C. D. (2006) · Annals of Tropical Medicine and Parasitology · doi:10.1179/136485906x97417 · Cited by 462 · cited 1×
“15 It is the leading cause of disability world-wide, 16 17 and the enrolment of depressed individuals into clinical trials is difficult.”
Increasing recruitment to randomised trials: a review of randomised controlled trials
Watson, J. M., & Torgerson, D. J. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-34 · Cited by 333
Strategies for Increasing Recruitment to Randomised Controlled Trials: Systematic Review
Caldwell, P. H. Y., Hamilton, S., Tan, A., & Craig, J. C. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000368 · Cited by 307 · cited 1×
“12 13 However, retention issues are particularly problematic for internet studies, with one recent internet-based trial reporting a drop-out rate of over 90% in a sample of 3000 individuals.”
BackgroundImproving Primary Care for Depression in Late Life: The Design of a Multicenter Randomized Trial
Unützer, J., Katon, W. J., Williams, J. W., Callahan, C. M., Harpole, L. H., Hunkeler, E. M., Hoffing, M., Areán, P. A., Hegel, M. T., Schoenbaum, M., et al. (2001) · Medical Care · doi:10.1097/00005650-200108000-00005 · Cited by 288 · cited 1×
“…information on mental health status using the PHQ-9 24 for depression, the generalized anxiety disorder (GAD)-7, 22 for generalised anxiety, a four-item mania and psychosis-screening instrument 25 and the four-item National Institute on Alcohol Abuse and Alcoholism (NIAAA) Alcohol Screening Test.”
MethodSelf-citation
Cited by
Sorted by citations
Firth, J., Torous, J., Nicholas, J., Carney, R., Pratap, A., Rosenbaum, S., & Sarris, J. (2017) · World Psychiatry · doi:10.1002/wps.20472 · Cited by 1,097
The Use and Effectiveness of Mobile Apps for Depression: Results From a Fully Remote Clinical Trial
Arean, P. A., Hallgren, K. A., Jordan, J. T., Gazzaley, A., Atkins, D. C., Heagerty, P. J., & Anguera, J. A. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.6482 · Cited by 376
“We have already published the data on the methodological issues in the recruitment and retention of participants into a large-scale, remote RCT [10].”
Self-citationPratap, A., Neto, E. C., Snyder, P., Stepnowsky, C., Elhadad, N., Grant, D., Mohebbi, M. H., Mooney, S., Suver, C., Wilbanks, J., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0224-8 · Cited by 410
“These studies are able to recruit at-scale because participants can be identified and consented(28) to participate in the study without ever having stepped foot in a research lab, with significantly lower costs than conventional clinical trials.(23,24) Mobile technologies also allow investigators an opportunity to col…”
Self-citationThe WPA-Lancet psychiatry commission on the future of psychiatry
Bhugra, D., Tasman, A., Pathare, S., Priebe, S., Smith, S., Torous, J., Arbuckle, M. R., Langford, A., Alarcón, R. D., Chiu, H. F. K., et al. (2017) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30333-4 · Cited by 392
Torous, J. B., Onnela, J., & Keshavan, M. S. (2017) · Translational Psychiatry · doi:10.1038/tp.2017.25 · Cited by 349
“22 It is possible to imagine how the ubiquity of smartphones will enable studies to capture rare digital phenotypes from the general population, as well as ones we today do not yet recognize; digital phenotyping may also identify which subjects to conduct more extensive genetic, molecular, cellular…”
BackgroundState of the Field of Mental Health Apps
Neary, M., & Schueller, S. M. (2018) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2018.01.002 · Cited by 247
Leveraging Data Science to Combat COVID-19: A Comprehensive Review
Latif, S., Usman, M., Manzoor, S., Iqbal, W., Qadir, J., Gareth Tyson, A., Castro, I., Razi, A., Boulos, M. N. K., Weller, A., et al. (2020) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2020.3020521 · Cited by 273
“Data-driven solutions are available to rapidly identify eligible participants [117], [212] and data collection platforms already exist to monitor symptoms remotely [213].”
BackgroundSmartphone Apps for the Treatment of Mental Disorders: Systematic Review
Miralles, I., Granell, C., Díaz-Sanahuja, L., Van Woensel, W., Bretón-López, J., Mira, A., Castilla, D., & Casteleyn, S. (2020) · JMIR mHealth and uHealth · doi:10.2196/14897 · Cited by 188
“7Cups [160], Be Good to Yourself [18], BlueWatch [161], Dcombat [162], Get Happy Program [163], HeadGear [164], iCare Prevent [165], MedLink [166], Mobile Sensing and Support [167], MoodHacker [168], Moodivate [169], MyGamePlan [170], PRIME-D [171], Push-D [172], SocioEmpathy [173], SPSRS [174], SuperBetter [175], The…”
BackgroundMoberg, C., Niles, A., & Beermann, D. (2019) · Journal of Medical Internet Research · doi:10.2196/12556 · Cited by 157
“Though it is not the first all-virtual RCT of a mental health support app [29,30], this study was an effort to advance the literature on the real-world use of technology-enabled CBT.”
Dogan, E., Sander, C., Wagner, X., Hegerl, U., & Kohls, E. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7006 · Cited by 177
Online Patient Recruitment in Clinical Trials: Systematic Review and Meta-Analysis
Brøgger‐Mikkelsen, M., Ali, Z., Zibert, J. R., Andersen, A. D., & Thomsen, S. F. (2020) · Journal of Medical Internet Research · doi:10.2196/22179 · Cited by 167
Video Games for Neuro-Cognitive Optimization
Mishra, J., Anguera, J. A., & Gazzaley, A. H. (2016) · Neuron · doi:10.1016/j.neuron.2016.04.010 · Cited by 164
Self-citationWu, A., Scult, M. A., Barnes, E. D., Betancourt, J. A., Falk, A. E., & Gunning, F. M. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00386-8 · Cited by 174
Speich, B., von Niederhäusern, B., Schur, N., Hemkens, L. G., Fürst, T., Bhatnagar, N., Alturki, R., Agarwal, A., Kasenda, B., Pauli-Magnus, C., et al. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.12.018 · Cited by 146
Watson, N. L., Mull, K. E., Heffner, J. L., McClure, J. B., & Bricker, J. B. (2018) · Journal of Medical Internet Research · doi:10.2196/10351 · Cited by 118
“Although some studies have successfully recruited samples comprised entirely of males for eHealth research through targeted advertisement campaigns [27,49,54], it is quite common among eHealth intervention trials to have a greater proportion of women compared to men [3,9,24,25,30,55-58].”
BackgroundResultInfluentialRemote smartphone monitoring of Parkinson’s disease and individual response to therapy
Omberg, L., Neto, E. C., Perumal, T. M., Pratap, A., Tediarjo, A., Adams, J., Bloem, B. R., Bot, B. M., Elson, M. J., Goldman, S. M., et al. (2021) · Nature Biotechnology · doi:10.1038/s41587-021-00974-9 · Cited by 141
Hornstein, S., Zantvoort, K., Lueken, U., Funk, B., & Hilbert, K. (2023) · Frontiers in Digital Health · doi:10.3389/fdgth.2023.1170002 · Cited by 134
The accuracy of passive phone sensors in predicting daily mood
Pratap, A., Atkins, D. C., Renn, B. N., Tanana, M. J., Mooney, S. D., Anguera, J. A., & Areán, P. A. (2019) · Depression and Anxiety · doi:10.1002/da.22822 · Cited by 119
Self-citationSimblett, S., Matcham, F., Siddi, S., Bulgari, V., Barattieri di San Pietro, C., Hortas López, J., Ferrão, J., Polhemus, A., Haro, J. M., de Girolamo, G., et al. (2019) · JMIR mHealth and uHealth · doi:10.2196/11325 · Cited by 120
“Clinical trials of mobile technologies for individuals with depression have highlighted engagement as a specific challenge [12].”
BackgroundPratap, A., Renn, B. N., Volponi, J. J., Mooney, S. D., Gazzaley, A. H., Areán, P. A., & Anguera, J. A. (2018) · Journal of Medical Internet Research · doi:10.2196/10130 · Cited by 116
“We attempted to address each of these issues by selectively targeting an underrepresented Hispanic/Latino population and using accessible, Spanish translated versions of the evidence-based intervention apps used in the initial study [8].”
BackgroundMethodInfluentialSelf-citationMohr, D. C., Schueller, S. M., Tomasino, K. N., Kaiser, S. M., Alam, N., Karr, C., Vergara, J. L., Gray, E. L., Kwasny, M. J., & Lattie, E. G. (2019) · Journal of Medical Internet Research · doi:10.2196/13609 · Cited by 116
Sequeira, L., Perrotta, S., LaGrassa, J., Merikangas, K. R., Kreindler, D. M., Kundur, D., Courtney, D., Szatmári, P., Battaglia, M., & Strauss, J. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.11.156 · Cited by 113
“While mobile apps allow for ease of recruitment and minimal costs in disseminating interventions, studies with active monitoring by participants require much participant feed-back and time (Anguera et al., 2016), and can be challenging in certain populations, such as youth with depression.”
BackgroundSocial media recruitment for mental health research: A systematic review
Sanchez, C., Grzenda, A. L., Varias, A. D., Widge, A. S., Carpenter, L. L., McDonald, W. M., Nemeroff, C. B., Kalin, N. H., Martin, G., Tohen, M. F., et al. (2020) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2020.152197 · Cited by 108
Understanding People’s Use of and Perspectives on Mood-Tracking Apps: Interview Study
Schueller, S. M., Neary, M., Lai, J., & Epstein, D. A. (2021) · JMIR Mental Health · doi:10.2196/29368 · Cited by 102
Forbes, A., Keleher, M. R., Venditto, M., & DiBiasi, F. (2023) · Journal of Medical Internet Research · doi:10.2196/43727 · Cited by 102
Hidalgo‐Mazzei, D., Reinares, M., Mateu, A., Nikolova, V. L., Bonnín, C. D. M., Samalin, L., García-Estela, A., Pérez, V., Young, A. H., Strejilevich, S., et al. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2018.08.048 · Cited by 92
What Works Best to Engage Participants in Mobile App Interventions and e-Health: A Scoping Review
Oakley-Girvan, I., Yunis, R., Longmire, M., & Ouillon, J. S. (2022) · Telemedicine and e-Health · doi:10.1089/tmj.2021.0176 · Cited by 92
Immediate Mood Scaler: Tracking Symptoms of Depression and Anxiety Using a Novel Mobile Mood Scale
Nahum, M., Vleet, T. V., Sohal, V. S., Mirzabekov, J. J., Rao, V. R., Wallace, D. L., Lee, M. B., Dawes, H. E., Stark‐Inbar, A., Jordan, J. T., et al. (2017) · JMIR mhealth and uhealth · doi:10.2196/mhealth.6544 · Cited by 97
Self-citationCrouthamel, M., Quattrocchi, E., Watts, S., Wang, S., Berry, P., Garcia-Gancedo, L., Hamy, V., & Williams, R. E. (2018) · JMIR mHealth and uHealth · doi:10.2196/mhealth.9656 · Cited by 76
“Previous studies have explored various forms of incentivization to boost engagement, most notably the ability to visualize a medical benefit and payment [32,33].”
BackgroundSensing Apps and Public Data Sets for Digital Phenotyping of Mental Health: Systematic Review
Mendes, J. P. M., de Moura, I. R., van de Ven, P. W. J., Viana, D., Silva, F., Coutinho, L., Teixeira, S. S., Rodrigues, J. J. P. C., & Teles, A. S. (2021) · Journal of Medical Internet Research · doi:10.2196/28735 · Cited by 85
“…Positioning, inertial, virtual, and ambient Beiwe [15] Raw data collection It does not infer information Positioning, inertial, and virtual EVO [37] Mental state prediction Sleep, sociability, mobility, physical activity, device usage Positioning, inertial, virtual, and ambient CrossCheck [38]…”
BackgroundAcceptability of mHealth augmentation of Collaborative Care: A mixed methods pilot study
Bauer, A. M., Iles-Shih, M., Ghomi, R. H., Rue, T. C., Grover, T., Kincler, N., Miller, M., & Katon, W. J. (2017) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2017.11.010 · Cited by 84
“The health tips were selected from tips used in a recent trial of depression apps [8,21] and included suggestions for managing depressed mood such as self-care activities (e.g., healthy eating, pleasant activities) or managing challenges (e.g., meditation, fi nding balance).”
BackgroundMethodSyrjala, K. L., Yi, J. C., Artherholt, S. B., Romano, J. M., Crouch, M.-L., Fiscalini, A. S., Hegel, M. T., Flowers, M. E. D., Martin, P. J., & Leisenring, W. M. (2018) · Journal of Cancer Survivorship · doi:10.1007/s11764-018-0693-9 · Cited by 76
“Challenges include recruiting those with more severe chronic symptoms to enroll [67, 68] and maintaining engagement for those who enroll [69].”
Cormack, F., McCue, M., Taptiklis, N., Skirrow, C., Glazer, E., Panagopoulos, E., van Schaik, T. A., Fehnert, B., King, J., & Barnett, J. H. (2019) · JMIR Mental Health · doi:10.2196/12814 · Cited by 79
“In depression, mobile apps have tracked changes in patient-reported mood [27-29] and have been used as part of randomized controlled trials to evaluate treatment efficacy [30].”
Contribution of sustained attention abilities to real-world academic skills in children
Gallen, C. L., Schaerlaeken, S., Younger, J. W., Younger, J. W., O’Laughlin, K. D., Anguera, J. A., Bunge, S. A., Ferrer, E. E., Hoeft, F., McCandliss, B. D., et al. (2023) · Scientific Reports · doi:10.1038/s41598-023-29427-w · Cited by 105
Self-citationSpahrkäs, S. S., Looijmans, A., Sanderman, R., & Hagedoorn, M. (2020) · Psycho-Oncology · doi:10.1002/pon.5492 · Cited by 70
Danieli, M., Ciulli, T., Mousavi, S. M., & Riccardi, G. (2021) · JMIR Formative Research · doi:10.2196/30053 · Cited by 64
“Their functionalities range from (1) delivering questionnaires for mood self-monitoring [3,4] and (2) providing recommendations for emotion regulation [5] to (3) engaging users in rule-based interactions [6], sometimes with the support of web-based scripted dialogs [7].”
Transforming Psychiatry into Data-Driven Medicine with Digital Measurement Tools
Hsin, H., Fromer, M., Peterson, B. E., Walter, C., Fleck, M. S., Campbell, A. T., Varghese, P., & Califf, R. M. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0046-0 · Cited by 65
Ben‐Zeev, D., Chander, A., Tauscher, J. S., Buck, B., Nepal, S. K., Campbell, A. T., & Doron, G. (2021) · Journal of Medical Internet Research · doi:10.2196/29201 · Cited by 65
“…condition does not require in-person contact with the research team (eg, no intervention control, waitlist control, or digital attention–control alternative), recruitment, treatment, and assessment of outcomes can all be conducted fully remotely and potentially on a single device [25-30].”
Marciniak, M. A., Shanahan, L., Rohde, J., Schulz, A., Wackerhagen, C., Kobylińska, D., Tuescher, O., Binder, H., Walter, H., Kalisch, R., et al. (2020) · JMIR mhealth and uhealth · doi:10.2196/19836 · Cited by 59
Cohen, K. A., & Schleider, J. L. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100496 · Cited by 71
Steinglass (2017) · doi:10.1002/da.22624 · Cited by 54
Self-citationCarpenter, J. M., Crutchley, P., Zilca, R., Schwartz, H. A., Smith, L. K., Cobb, A. M., & Parks, A. C. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5725 · Cited by 58
“Whereas many RCTs aim to standardize or maximize engagement (ideally, every participant would show 100% engagement) [8], multilevel models can harness that variation, creating a dose-response relationship between behavior and outcome.”
BackgroundSmartphone apps for psychological health: A brief state of the science review.
Bush, N. E., Armstrong, C. M., & Hoyt, T. (2018) · Psychological Services · doi:10.1037/ser0000286 · Cited by 65
Daniore, P., Nittas, V., & von Wyl, V. (2022) · Journal of Medical Internet Research · doi:10.2196/39910 · Cited by 64
“Remote digital health studies promise significant cost, time, and scalability advantages when compared with traditional studies, by allowing key steps of the study investigations to be conducted in real-time and without in-person presence [3,4].”
Nickels, S., Edwards, M. D., Poole, S. F., Winter, D., Gronsbell, J. L., Rozenkrants, B., Miller, D. P., Fleck, M. S., McLean, A., Peterson, B. E., et al. (2021) · JMIR Mental Health · doi:10.2196/27589 · Cited by 50
“For example, in a randomized controlled trial on digital sensing and cognitive behavioral therapy on 1098 depressed participants, adherence dropped to 41% by the 12-week mark [19].”
BackgroundSelf-citationDigital Phenotyping and Feature Extraction on Smartphone Data for Depression Detection
Yang, M., Ngai, E. C. ‐., Hu, X., Hu, B., Liu, J., Erol Gelenbe, A., & Leung, V. C. M. (2024) · Proceedings of the IEEE · doi:10.1109/jproc.2025.3542324 · Cited by 56
“Although subjects in mobile applications can be easily recruited, this study requires participants to provide a lot of feedback and time [163].”
Leveraging technology to personalize cognitive enhancement methods in aging
Ziegler, D. A., Anguera, J. A., Gallen, C. L., Hsu, W.-Y., Wais, P. E., & Gazzaley, A. (2022) · Nature Aging · doi:10.1038/s43587-022-00237-5 · Cited by 50
“Indeed, it is now clear that mobile technology can be especially powerful in improving research-participant access, especially to those living in rural areas or members of minority ethnic groups, while simultaneously reducing the expense and time course of such trials 124 .”
BackgroundSelf-citationUnderstanding users’ perspectives on mobile apps for anxiety management
Balaskas, A., Schueller, S. M., Cox, A. L., & Doherty, G. J. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.854263 · Cited by 56
Stergiopoulos, S., Michaels, D. L., Kunz, B. L., & Getz, K. A. (2019) · Therapeutic Innovation & Regulatory Science · doi:10.1177/2168479018817517 · Cited by 50
The New Digital Divide For Digital Biomarkers
Torous, J. B., Rodriguez, J. A., & Powell, A. C. (2017) · Digital Biomarkers · doi:10.1159/000477382 · Cited by 47
“mental health apps undergo research evaluation [21, 22], the vast majority does not and little is known about their safety or efficacy.”
BackgroundHensel, J. M., Shaw, J., Ivers, N. M., Desveaux, L., Vigod, S. N., Cohen, A., Onabajo, N., Agarwal, P., Mukerji, G., Yang, R., et al. (2019) · Journal of Medical Internet Research · doi:10.2196/10838 · Cited by 39
“High drop out of up to 50% in trials of Web-based mental health interventions has been reported [10] and retention in digitized trials has specifically been discussed as a challenge [30].”
Anand, T., Nitpolprasert, C., Kerr, S. J., Muessig, K. E., Promthong, S., Chomchey, N., Hightow‐Weidman, L., Chaiyahong, P., Phanuphak, P., Ananworanich, J., et al. (2017) · AIDS Care · doi:10.1080/09540121.2017.1286288 · Cited by 39
Patient and physician perspectives of a smartphone application for depression: a qualitative study
Patoz, M.-C., Hidalgo‐Mazzei, D., Blanc, O., Verdolini, N., Pacchiarotti, I., Murru, A., Zukerwar, L., Vieta, E., Llorca, P., & Samalin, L. (2021) · BMC Psychiatry · doi:10.1186/s12888-021-03064-x · Cited by 36
“For this particular point, studies showed inconsistent results, where some highlighted a quick drop out rate of self-monitoring apps [56, 57], while others noted, conversely, a fairly high retention rate for these tools [52, 54, 58].”
ResultKeefe, R. S. E., Cañadas, E., Farlow, D., & Etkin, A. (2022) · American Journal of Psychiatry · doi:10.1176/appi.ajp.21020125 · Cited by 36
Recommendations to promote equity, diversity and inclusion in decentralized clinical trials
Aiyegbusi, O. L., Rivera, S. C., Kamudoni, P., Anderson, N. E., Collis, P., Denniston, A. K., Harding, R., Hughes, S., Khunti, K., Kotecha, D., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-03323-w · Cited by 39
Pratap, A., Allred, R., Duffy, J., Rivera, D., Lee, H. S., Renn, B. N., & Areán, P. A. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.15717 · Cited by 33
Self-citationKlein, A., Clucas, J. C., Krishnakumar, A., Ghosh, S., Auken, W. V., Thonet, B., Sabram, I., Acuna, N., Keshavan, A., Rossiter, H., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/22369 · Cited by 36
“…MindLogger, with updated content (particularly with regard to sleep, positive and negative thoughts, food and drink, internet, and social media), enhanced with clarification of the content, inclusion of colorful images, and formats adapted for children and young adults to encourage engagement [75].”
BackgroundHrynyschyn, R., & Dockweiler, C. (2021) · JMIR mHealth and uHealth · doi:10.2196/24703 · Cited by 35
“Patients can also receive therapy whenever they need it most without taking an appointment with their therapist, which often results in long waiting times with possible aggravation of symptoms [23].”
BackgroundToward dynamic phenotypes and the scalable measurement of human behavior
Germine, L. T., Strong, R. W., Singh, S. M., & Sliwinski, M. J. (2020) · Neuropsychopharmacology · doi:10.1038/s41386-020-0757-1 · Cited by 30
“Anticipated burden reduces enrollment and enthusiasm for participating in research [8, 16, 17], whereas actual and perceived burden increases attrition [18, 19], reduces data quality [20], and is among the largest cost drivers for longitudinal research [21, 22].”
BackgroundClosing the Research-To-Practice Gap in Digital Psychiatry
Lipschitz, J. M., Hogan, T. P., Bauer, M. S., & Mohr, D. C. (2019) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.18com12659 · Cited by 33
Smartphone-based passive assessment of mobility in depression: Challenges and opportunities
Renn, B. N., Pratap, A., Atkins, D. C., Mooney, S. D., & Areán, P. A. (2018) · Mental health and physical activity · doi:10.1016/j.mhpa.2018.04.003 · Cited by 34
“Case Example: The BRIGHTEN Study To illustrate the challenges and potential applications of capturing mobility using mobile phones, we present secondary data analysis of the BRIGHTEN study, which has been described in detail elsewhere (Anguera et al., 2016; Areán et al., 2016).”
MethodSelf-citationGentile, A. J., Lima, C. N. L., Flygare, O., Enander, J., Wilhelm, S., Mataix‐Cols, D., & Rück, C. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-024693 · Cited by 32
Review and Implementation of Self-Help and Automated Tools in Mental Health Care
Chan, S. R., Li, L., Torous, J. B., Gratzer, D., & Yellowlees, P. (2019) · Psychiatric Clinics of North America · doi:10.1016/j.psc.2019.07.001 · Cited by 29
Younger, J. W., O’Laughlin, K. D., Anguera, J. A., Bunge, S. A., Ferrer, E. E., Hoeft, F., McCandliss, B. D., Mishra, J., Rosenberg‐Lee, M., Gazzaley, A. H., et al. (2023) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2023.1195013 · Cited by 31
Self-citationWhite, K. M., Williamson, C., Bergou, N., Oetzmann, C., de Angel, V., Matcham, F., Henderson, C., & Hotopf, M. H. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00624-7 · Cited by 31
Creating an Empirical Dermatology Dataset Through Crowdsourcing With Web Search Advertisements
Ward, A., Li, J., Wang, J., Lakshminarasimhan, S., Carrick, A., Campana, B. J. L., Hartford, J., Sreenivasaiah, P. K., Tiyasirisokchai, T., Virmani, S., et al. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.46615 · Cited by 45
An Introduction to Core Competencies for the Use of Mobile Apps in Cognitive and Behavioral Practice
Schueller, S. M., Armstrong, C. M., Neary, M., & Ciulla, R. (2020) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2020.11.002 · Cited by 29
Recruitment and Retention in Remote Research: Learnings From a Large, Decentralized Real-world Study
Li, S. X., Halabi, R., Selvarajan, R., Woerner, M., Fillipo, I. G., Banerjee, S., Mosser, B., Jain, F. A., Areán, P. A., & Pratap, A. (2022) · JMIR Formative Research · doi:10.2196/40765 · Cited by 30
“Although these recruitment channels have shown the potential to reach and recruit large populations remotely [34-36], the long-term and uniform retention of remote participants has been challenging.”
BackgroundSelf-citationAssessing Cognitive Function in Multiple Sclerosis With Digital Tools: Observational Study
Hsu, W., Rowles, W. M., Anguera, J. A., Anderson, A., Younger, J. W., Friedman, S., Gazzaley, A. H., & Bove, R. M. (2021) · Journal of Medical Internet Research · doi:10.2196/25748 · Cited by 27
“With advanced technology, remote, computerized platforms for cognitive assessment and treatment, using personalizing features, including adaptive staircase algorithms for populations with cognitive deficits [7-14], have been developed.”
Self-citationFillipo, I. G., Pullmann, M. D., Hull, T. D., Zech, J. M., Wu, J., Litvin, B., Chen, S., & Areán, P. A. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.963741 · Cited by 29
“Data suggests that lack of synchronous human connection results in lower motivation to continue research participation (5, 6).”
BackgroundMethodInfluentialSelf-citationHsu, W., Rowles, W. M., Anguera, J. A., Zhao, C., Anderson, A., Alexander, A. M., Sacco, S., Henry, R. G., Gazzaley, A. H., & Bove, R. M. (2021) · Journal of Medical Internet Research · doi:10.2196/24356 · Cited by 26
Self-citationComparing Approaches to Mobile Depression Assessment for Measurement-Based Care: Prospective Study
Bauer, A. M., Baldwin, S. A., Anguera, J. A., Areán, P. A., & Atkins, D. C. (2018) · Journal of Medical Internet Research · doi:10.2196/10001 · Cited by 23
“Ethical approval was granted by the University of California San Francisco Committee for Human Research, and details of the trial, including methods for obtaining participants’ informed consent, have been previously published [14,15].”
MethodSelf-citationSchriewer, K., & Bulaj, G. W. (2016) · Frontiers in Public Health · doi:10.3389/fpubh.2016.00217 · Cited by 28
“When developing music-streaming medical software, protection of patient’s electronic health records, cybersecurity, long-term patient engagement (18), as well as unhealthy and maladaptive effects of music shall be taken into account (45, 46).”
BackgroundNavigating the ethics of internet‐guided self‐help interventions.
Borgueta, A. M., Purvis, C. K., & Newman, M. G. (2018) · Clinical Psychology Science and Practice · doi:10.1111/cpsp.12235 · Cited by 22
Briffault, X., Morgiève, M., & Courtet, P. (2018) · Brain Sciences · doi:10.3390/brainsci8060098 · Cited by 21
“In order to avoid the frequently observed decrease in the rate of device use and compliance [70,71], the proposed EMA/EMI applications are designed with particular attention to user commitment and motivation, in particular through the use of gamification [72].”
MethodDesign for mobile mental health: Exploring the informed participation approach
Aryana, B., & Brewster, L. (2019) · Health Informatics Journal · doi:10.1177/1460458219873540 · Cited by 22
Eng, C. M., Tsegai-Moore, A., & Fisher, A. V. (2024) · Brain Sciences · doi:10.3390/brainsci14050451 · Cited by 26
“There is recent evidence that suggests GAs offer high test sensitivity for evaluating EF in school-age children [31].”
BackgroundPark, C., Zhu, J., Ho, R., Rosenblat, J. D., Iacobucci, M., Gill, H., Mansur, R. B., & McIntyre, R. S. (2020) · Annals of Clinical Psychiatry · doi:10.1177/104012372003200109 · Cited by 21
“34 If shorter intervention periods produce more favorable effects (ie, reduce depressive symptoms and are capable of producing sustained responses), this could inform patient care (ie, user engagement and motivation can be sustained more easily over shorter periods).”
Verna, E. C., Serper, M., Chu, J., Corey, K. E., Fix, O. K., Hoyt, K., Page, K. A., Loomba, R. S., Li, M., Everson, G. T., et al. (2020) · Hepatology · doi:10.1002/hep.31491 · Cited by 20
“(52) Mobile technologies and Web-based clinical trial design can recruit large numbers of participants in a short time span with minimal costs.(53) Similarly, such technology may address long-standing barriers to clinical research participation, such as the recruitment of subjects from rural and low-income populations.”
BackgroundParks, A. M., Duffecy, J. L., McCabe, J. E., Breman, R. B., Milgrom, J., Hirshler, Y., Gemmill, A. W., Segre, L. S., Felder, J. N., & Uscher‐Pines, L. (2022) · JMIR Pediatrics and Parenting · doi:10.2196/35320 · Cited by 21
“With regard to costs, digital trials are also likely to be more efficient because they require smaller teams of investigators [2] and avoid power reduction due to clustering, which is an issue when recruiting from multiple sites [13].”
Relationships Between Executive Functions and Computational Thinking
Liu, T. (2024) · Journal of Educational Computing Research · doi:10.1177/07356331241242435 · Cited by 23
“Recently, the University of California, San Francisco (UCSF) introduced a novel mobile assessment battery referred to as Adaptive Cognitive Evaluation (ACE) to measure students ’ EFs (Anguera et al., 2016).”
Pratap, A., Anguera, J. A., Renn, B. N., Neto, E. C., Volponi, J. J., Mooney, S. D., & Areán, P. A. (2017) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3123024.3127877 · Cited by 20
Self-citationSpahrkäs, S. S., Looijmans, A., Sanderman, R., & Hagedoorn, M. (2020) · JMIR Research Protocols · doi:10.2196/15969 · Cited by 20
“On the basis of previous studies testing health-related apps targeting behavior [34-39], we estimated the percentage of participants that would lose interest in the app or study assessments, and, therefore, would be lost to follow-up (eg, the 12-weeks measurement), at 60%.”
MethodSmartphone-based serious games for mental health: a scoping review
Gómez-Cambronero, Á., Mann, A.-L., Mira, A., Doherty, G. J., & Casteleyn, S. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-18971-w · Cited by 20
“…Randomized Controlled Trials (RCTs) or pilot RCTs (i.e., Relax and Race, The Loom [68], Happify Breather [69], Raw Hand [70], PuzzleWalk [71], I Love Breast [72], Drug Defence [73], POD Adventures [74], NM Vara et al. 2022 [75], LINA [51], ACE, and EVO Project [76] (see Section 3.11.4 for details).”
InfluentialmHealth technologies in clinical trials: Opportunities and challenges
Kakkar, A. K., Sarma, P., & Medhi, B. (2018) · Indian Journal of Pharmacology · doi:10.4103/ijp.ijp_391_18 · Cited by 19
“[15] Similarly, Volkova et al . developed a food label trial smartphone app to conduct a fully automated RCT to assess the effects of nutrition labels on food purchases.”
BackgroundPratap, A., Homiar, A., Waninger, L., Herd, C., Suver, C., Volponi, J. J., Anguera, J. A., & Areán, P. A. (2022) · Scientific Data · doi:10.1038/s41597-022-01633-7 · Cited by 22
“The two randomized clinical trials, Brighten (Bridging Research Innovations for Greater Health in Technology, Emotion, and Neuroscience) V1 study 29 , (henceforth V1) and Brighten V2 7 , (henceforth V2), have been used to investigate: 1) feasibility of conducting fully remote decentralized clinical…”
BackgroundMethodSelf-citationJesuthasan, J., Low, M., & Ong, T. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.846375 · Cited by 21
“However, despite the recent growth in digital healthcare solutions and their clinical potential, completion rates for digital health programs tend to be low, and limited user engagement and adherence is one of the biggest challenges in maximizing their impact (15, 16).”
BackgroundFlynn, R. M., Colon-Acosta, N., Zhou, J., & Bower, J. D. (2019) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-019-03881-w · Cited by 17
“…have potential to provide objective measures to supplement clinicians’ interviews and self-report survey measures, which require extensive time to administer, are open to subjective interpretation, and lack precision (Anguera et al. 2016; Butcher 2006; Gamberini et al. 2008; Piotrowski 1999).”
BackgroundMethodInfluentialAccelerating Research With Technology: Rapid Recruitment for a Large-Scale Web-Based Sleep Study
Deering, S., Grade, M. M., Uppal, J., Foschini, L., Juusola, J. L., Amdur, A., & Stepnowsky, C. J. (2019) · JMIR Research Protocols · doi:10.2196/10974 · Cited by 16
“For instance, Web-based mental health studies with a similar number of participants reached targets in 5 months [29], 3 months [30], and 4 days [31].”
BackgroundMorriss, R. K., Kaylor‐Hughes, C., Rawsthorne, M., Coulson, N., Simpson, S., Guo, B., James, M., Lathe, J., Moran, P., Tata, L. J., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/23487 · Cited by 17
“Automated trials have sometimes failed to engage some populations [18,19], whereas others offering otherwise difficult to obtain structured psychological treatments have been more successful [20-22].”
Planas, R., & Yuguero, O. (2021) · Informatics for Health and Social Care · doi:10.1080/17538157.2021.1887196 · Cited by 17
Hategan, A., Giroux, C., & Bourgeois, J. A. (2019) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-019-00086-x · Cited by 17
“Anguera et al. (2016) reported that while mobile technology can be a successful way to recruit research participants, reach a wide section of the population, and is cost-effective, the user ’ s willingness to participate in mobile research protocols (engagement) has remained challenging; study…”
BackgroundArmstrong, C. M., Hoyt, T., Kinn, J. T., Ciulla, R., & Bush, N. E. (2017) · Best practices in mental health · doi:10.70256/314845ciwvqz · Cited by 16
Ben‐Zeev, D., Buck, B., Chander, A., Brian, R., Wang, W., Atkins, D. C., Brenner, C. J., Cohen, T., Campbell, A. T., & MUNSON, J. A. (2020) · Schizophrenia Bulletin Open · doi:10.1093/schizbullopen/sgaa060 · Cited by 16
Beiwinkel, T., Hey, S., Bock, O., & Rössler, W. (2017) · Frontiers in Public Health · doi:10.3389/fpubh.2017.00249 · Cited by 15
Wang, J., Rogge, A. A., Armour, M., Smith, C. A., D’Adamo, C. R., Pischke, C. R., Yen, H., Wu, M., Moré, A. O. O., Witt, C. M., et al. (2019) · JMIR mhealth and uhealth · doi:10.2196/14661 · Cited by 15
“For the assessment of access of app studies, Anguera et al [40] reported the recruitment number, whereas Zens et al [14] reported the consent/download rate.”
BackgroundMethodThe Digital Platform and Its Emerging Role in Decentralized Clinical Trials
Copland, R. R., Hanke, S., Rogers, A., Mpaltadoros, L., Lazarou, I., Zeltsi, A., Nikolopoulos, S., MacDonald, T. M., & Mackenzie, I. S. (2024) · Journal of Medical Internet Research · doi:10.2196/47882 · Cited by 19
“Meanwhile, remote monitoring gives researchers greater insight into the day-to-day variability of disease activity and its wider impact [71,72].”
Global sensitivity analysis of clinical trials with missing patient-reported outcomes
Scharfstein, D. O., & McDermott, A. (2018) · Statistical Methods in Medical Research · doi:10.1177/0962280218759565 · Cited by 14
“[5] Joaquin A Anguera, Joshua T Jordan, Diego Castaneda, Adam Gazzaley, and Patricia A Areán.”
BackgroundOmylinska‐Thurston, J., Aithal, S., Liverpool, S., Clark, R. D., Moula, Z., Wood, J., Viliardos, L., Rodríguez-Dorans, E., Farish-Edwards, F., Parsons, A. S., et al. (2024) · JMIR Mental Health · doi:10.2196/55500 · Cited by 15
Management of Depression and Anxiety in Perioperative Medicine
Rolfzen, M. L., Nägele, P., Conway, C. R., Gibbons, R. D., & Bartels, K. (2024) · Anesthesiology · doi:10.1097/aln.0000000000005076 · Cited by 13
“126 Digital mental health treatments have proven helpful in all depression severity subtypes.”
Metadata
- Authors
- Anguera, J. A., Jordan, J. T., Castaneda, D., Gazzaley, A., & Areán, P. A.
- Affiliations
- University of California, San Francisco (US)
- University of Washington (US)
- Corresponding author
- Areán, Patricia A
- Venue
- BMJ Innovations
- Publisher
- BMJ
- Published
- 2016-01
- Volume
- 2
- Issue
- 1
- Pages
- 14-21
- ISSN
- 2055-642X (ISSN-L)
- 2055-8074
- PMC
- PMC4789688
- OpenAlex
- W2261303017
- Cited by
- 172
- References
- 34
- Citations by year
- 2026: 7 · 2025: 10 · 2024: 19 · 2023: 13 · 2022: 19 · 2021: 22 · 2020: 21 · 2019: 22 · 2018: 17 · 2017: 16 · 2016: 6
- FWCI
- 31.18
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc
- Locations
- BMJ Innovations (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by-nc)
- Funders
- University of California, San Francisco
- National Institute of Mental Health (MH100466, R34MH100466, K24MH074717, T32MH018261)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Keywords
- depression, mobile health apps, participant recruitment, participant engagement, trial costs, cognitive training, problem-solving therapy, mobile sensing, self-reported mood, cognitive function
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
