Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements
Open accessAbstract
The potential of smartphone apps to improve quality and increase access to mental health care is increasingly clear. Yet even in the current global mental health crisis, real-world uptake of smartphone apps by clinics or consumers remains low. To understand this dichotomy, this paper reviews current challenges surrounding user engagement with mental health smartphone apps. While smartphone engagement metrics and reporting remains heterogeneous in the literature, focusing on themes offers a framework to identify underlying trends. These themes suggest that apps are not designed with service users in mind, do not solve problems users care most about, do not respect privacy, are not seen as trustworthy and are unhelpful in emergencies. Respecting these current issues surrounding mental health app engagement, we propose several solutions and highlight successful examples of mental health apps with high engagement. Further research is necessary to better characterise engagement with mental health apps and identify best practices for design, testing and implementation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
This multi-modal approach allows for personal engagement and increases the utilization of digitally delivered interventions, which have been demonstrated to have a steep drop off in utilization when delivered alone.
“This multi-modal approach allows for personal engagement and also increases the utilization of digitally delivered interventions, which have been demonstrated to have a steep drop off in utilization when delivered alone [54].”
From Skills to Enhance Positivity in adolescents at risk for suicide: Protocol for a randomized controlled trial (2023) · cites this paper as [54]
The Improving App Engagement section says clinician or peer involvement and integration into care settings can enhance app engagement; it also describes a highly adhered-to app that was not a stand-alone service. The Methods section reports a sharp drop-off in use of some mental health apps. But the review frames the benefit of added support as emerging evidence or potential, rather than demonstrating that a multi-modal approach increases utilization compared with delivery alone. (LLM, conf=0.84)
Experimental — accuracy not yet measuredPatient adherence may be improved by providing evidence-based solutions such as 24/7 access to educational information and resources, personalized support, and targeted interventions and skills training via technology.
“Patient adherence may also be improved by providing evidence-based solutions such as 24/7 access to educational information and resources, personalized support, and targeted interventions and skills training via technology (eg, apps or websites) [21,35] in the comfort of their own homes [20,30].”
From A Digital Mental Health Support Program for Depression and Anxiety in Populations With Attention-Deficit/Hyperactivity Disorder: Feasibility and Usability Study (2023) · cites this paper as [30]
The “Improving app engagement” section says health education and clinician or peer support may increase engagement, and that embedding apps in users’ settings can enhance adherence (page not provided). It does not establish that 24/7 access, personalized support, or targeted skills training as a package improves patient adherence. (LLM, conf=0.84)
Experimental — accuracy not yet measuredDigital interventions have the potential to evoke changes unavailable in traditional methods because of the ability to design personalized treatment models and measure engagement.
“Digital interventions have the potential to evoke changes unavailable in traditional methods because of the ability to design personalized treatment models and measure engagement [28-30].”
From A Digital Mental Health Support Program for Depression and Anxiety in Populations With Attention-Deficit/Hyperactivity Disorder: Feasibility and Usability Study (2023) · cites this paper as [30]
The Additional information section (page unavailable) says smartphone apps may transform psychiatric care, and Poor usability describes a customised symptom tracker. The Methods section reports app-usage data as a way to assess engagement, but also says such data are scarce. The paper does not establish that these capabilities produce changes unavailable through traditional methods. (LLM, conf=0.84)
Experimental — accuracy not yet measuredEven when the app is downloaded, some users never use it, and those who engage with the app are unlikely to do so more than a few times, showing a decrease in use over time.
“Even when the app is downloaded, studies show that some users never use it, and those who engage with the app are unlikely to do so more than a few times, showing a decrease in use over time [49,50].”
From App-Based Mindfulness Meditation for People of Color Who Experience Race-Related Stress: Protocol for a Randomized Controlled Trial (2022) · cites this paper as [50]
The Methods section reports that only 14% of PTSD Coach downloaders used it the following day and that use of a schizophrenia symptom-tracking app dropped off, with users unlikely to continue more than a few times. It does not establish that some people who downloaded an app never used it; its separate example of people who never downloaded a study app does not support that part of the claim. Page numbers are unavailable in our copy. (LLM, conf=0.88)
Experimental — accuracy not yet measuredThere is evidence of low treatment adherence in self-directed app-based mindfulness meditation.
“Given the evidence of low treatment adherence in self-directed app-based mindfulness meditation [49-51], we will use a combination of app and text messaging reminders.”
From App-Based Mindfulness Meditation for People of Color Who Experience Race-Related Stress: Protocol for a Randomized Controlled Trial (2022) · cites this paper as [50]
The Methods section reports rapid drop-off in use of mental health apps, and Improving App Engagement describes adherence as a challenge for digital interventions. The review does not establish low treatment adherence specifically for self-directed app-based mindfulness meditation; its mention of mindfulness apps concerns their clinical quality, not adherence. Page numbers were not available in our copy. (LLM, conf=0.85)
Experimental — accuracy not yet measured
There is extensive evidence of low enrollment, poor use, and high attrition rates in self-directed app-based interventions.
“Issues of treatment adherence are significant considering extensive evidence of low enrollment, poor use, and high attrition rates in self-directed app-based interventions [49-51].”
From App-Based Mindfulness Meditation for People of Color Who Experience Race-Related Stress: Protocol for a Randomized Controlled Trial (2022) · cites this paper as [50]
The Methods/background passage reports low uptake and poor continued use of mental health apps, including a study in which 57.9% of participants never downloaded the app and another in which only 14% used an app the day after downloading. But it also says engagement data for mental health apps are scarce; it does not establish extensive evidence specifically about enrollment and attrition in self-directed app-based interventions. Page numbers are unavailable in our copy. (LLM, conf=0.84)
Experimental — accuracy not yet measuredDigital mental health interventions have limitations, mainly low engagement by users and low completion rates.
“although some limitations have been found, mainly regarding low engagement by users and low completion rates [15,16].”
From Engagement and effectiveness of a healthy-coping intervention via chatbot for university students during the COVID-19 pandemic: Mixed methods proof-of-concept study (2021) · cites this paper as [15]
The Introduction and Methods describe low user engagement as a key limitation of mental health smartphone apps, including rapid drop-off in use. The paper does not establish low intervention completion rates as a separate finding, and its focus on smartphone apps is narrower than digital mental health interventions generally. (LLM, conf=0.86)
Experimental — accuracy not yet measuredOffering a challenge may improve engagement over short periods, thereby allowing some therapeutic benefit and the potential for developing it into a habit.
“Others suggest offering a challenge may improve engagement over short periods, thereby allowing some therapeutic benefit and the potential for developing it into a habit [58].”
From Feasibility and Acceptability of the Aboriginal and Islander Mental Health Initiative for Youth App: Nonrandomized Pilot With First Nations Young People (2023) · cites this paper as [58]
Issues with intervention engagement are not restricted to digital interventions, and similar challenges can occur with face-to-face delivery.
“Issues with intervention engagement are not restricted to digital interventions, and similar challenges can occur with face-to-face delivery [58].”
From Feasibility and Acceptability of the Aboriginal and Islander Mental Health Initiative for Youth App: Nonrandomized Pilot With First Nations Young People (2023) · cites this paper as [58]
If a digital tool is not used for a recommended period, it is unlikely that it will have the intended effect.
“if the tool is not used for a recommended period, it is unlikely that it will have the intended effect [17,18]”
From Optimization of a Transdiagnostic Mobile Emotion Regulation Intervention for University Students: Protocol for a Microrandomized Trial (2023) · cites this paper as [17]
A range of factors can influence engagement with DMHIs, such as the relevance of information to the user, a lack of user motivation to persist with a self-guided intervention, and poor user experience with the technology.
“A range of factors can influence engagement with DMHIs, such as the relevance of information to the user provided by a digital intervention [13], a lack of user motivation to persist with a self-guided intervention [14], and poor user experience with the technology [15].”
From Barriers to and facilitators of user engagement with digital mental health interventions: Systematic review (2021) · cites this paper as [15]
A previous review of user engagement with mental health apps theorized that one reason for low engagement is that these apps do not consider user privacy.
“A previous review of user engagement with mental health apps theorized that one reason for low engagement is that these apps do not consider user privacy [15].”
From Barriers to and facilitators of user engagement with digital mental health interventions: Systematic review (2021) · cites this paper as [15]
There is little consensus on the usability and effectiveness of mental health apps.
“However, there is little consensus on the usability and effectiveness of such systems [13].”
From A conversational artificial intelligence agent for a mental health care app: Evaluation study of its participatory design (2021) · cites this paper as [13]
Low engagement is a barrier to both optimal efficacy and adoption of digital mental health interventions into health settings and other translational pathways.
“Low engagement—defined as suboptimal levels of user access and/or adherence to an intervention (12)—is touted as one of the main reasons why the potential benefits of these interventions remain unrealized in the real world (13, 14).”
From Effect of Engagement With Digital Interventions on Mental Health Outcomes: A Systematic Review and Meta-Analysis (2021) · cites this paper as [12]
Most general mental health care apps face difficulty sustaining engagement with high dropout.
“most general mental health care apps face difficulty sustaining engagement with high dropout”
From A Therapeutic Relational Agent for Reducing Problematic Substance Use (Woebot): Development and Usability Study (2021) · cites this paper as [14]
Several factors contribute to low DMHI engagement, including technical issues, motivation issues, and a lack of user-driven design.
“Several factors contribute to low DMHI engagement, including technical issues, motivation issues, and a lack of user-driven design [11,25,31,32].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Research summary on the therapeutic relationship and psychotherapy outcome.
Lambert, M. J., & Barley, D. E. (2001) · Psychotherapy: Theory, Research, Practice, Training · doi:10.1037/0033-3204.38.4.357 · Cited by 1,621 · see citing works
Health App Use Among US Mobile Phone Owners: A National Survey
Krebs, P., & Duncan, D. T. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4924 · Cited by 1,274 · see citing works
Firth, J., Torous, J., Nicholas, J., Carney, R., Pratap, A., Rosenbaum, S., & Sarris, J. (2017) · World Psychiatry · doi:10.1002/wps.20472 · Cited by 1,097 · see citing works
Self-citationMental Health Smartphone Apps: Review and Evidence-Based Recommendations for Future Developments
Bakker, D., Kazantzis, N., Rickwood, D., & Rickard, N. (2016) · JMIR Mental Health · doi:10.2196/mental.4984 · Cited by 886 · see citing works
Firth, J., Torous, J., Nicholas, J., Carney, R., Rosenbaum, S., & Sarris, J. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.04.046 · Cited by 675 · see citing works
Self-citationInfluence of Pokémon Go on Physical Activity: Study and Implications
Althoff, T., White, R. W., & Horvitz, E. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.6759 · Cited by 522 · see citing works
Health Literacy and Health Information Technology Adoption: The Potential for a New Digital Divide
Mackert, M., Mabry-Flynn, A., Champlin, S., Donovan, E. E., & Pounders, K. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.6349 · Cited by 503 · see citing works
Review and evaluation of mindfulness-based iPhone apps
Mani, M., Kavanagh, D. J., Hides, L., & Stoyanov, S. R. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4328 · Cited by 495 · see citing works
Mobile Apps for Bipolar Disorder: A Systematic Review of Features and Content Quality
Nicholas, J., Larsen, M. E., Proudfoot, J., & Christensen, H. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4581 · Cited by 438 · see citing works
Self-citationGilbody, S., Littlewood, E., Hewitt, C., Brierley, G., Tharmanathan, P., Araya, R., Barkham, M., Bower, P., Cooper, C., Gask, L., et al. (2015) · BMJ · doi:10.1136/bmj.h5627 · Cited by 409 · see citing works
The Use and Effectiveness of Mobile Apps for Depression: Results From a Fully Remote Clinical Trial
Arean, P. A., Hallgren, K. A., Jordan, J. T., Gazzaley, A., Atkins, D. C., Heagerty, P. J., & Anguera, J. A. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.6482 · Cited by 376 · see citing works
Miner, A. S., Milstein, A., Schueller, S., Hegde, R., Mangurian, C., & Linos, E. (2016) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.0400 · Cited by 368 · see citing works
Torous, J., & Roberts, L. W. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.0262 · Cited by 362 · see citing works
Self-citationAvailability and quality of mobile health app privacy policies
Sunyaev, A., Dehling, T., Taylor, P. L., & Mandl, K. D. (2015) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002605 · Cited by 360 · see citing works
Ben‐Zeev, D., Kaiser, S. M., Brenner, C. J., Begale, M. J., Duffecy, J. L., & Mohr, D. C. (2013) · Psychiatric Rehabilitation Journal · doi:10.1037/prj0000019 · Cited by 306 · see citing works
A systematic assessment of smartphone tools for suicide prevention
Larsen, M. E., Nicholas, J., & Christensen, H. (2016) · PLoS ONE · doi:10.1371/journal.pone.0152285 · Cited by 289 · see citing works
Self-citationAncker, J. S., Witteman, H. O., Hafeez, B., Provencher, T., de Graaf, M. V., & Wei, E. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4209 · Cited by 277 · see citing works
The Hidden Cost of Personal Quantification
Etkin, J. (2016) · Journal of Consumer Research · doi:10.1093/jcr/ucv095 · Cited by 274 · see citing works
Usability of Commercially Available Mobile Applications for Diverse Patients
Sarkar, U., Gourley, G. I., Lyles, C. R., Tieu, L., Clarity, C., Newmark, L., Singh, K., & Bates, D. W. (2016) · Journal of General Internal Medicine · doi:10.1007/s11606-016-3771-6 · Cited by 257 · see citing works
Tighe, J., Shand, F., Ridani, R., Mackinnon, A., De La Mata, N., & Christensen, H. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-013518 · Cited by 227 · see citing works
Self-citationFranklin, J. C., Fox, K. R., Franklin, C. R., Kleiman, E. M., Ribeiro, J. D., Jaroszewski, A. C., Hooley, J. M., & Nock, M. K. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000093 · Cited by 217 · see citing works
mHealth in the Wild: Using Novel Data to Examine the Reach, Use, and Impact of PTSD Coach
Owen, J. E., Jaworski, B. K., Kuhn, E., Makin-Byrd, K. N., Ramsey, K. M., & Hoffman, J. E. (2015) · JMIR Mental Health · doi:10.2196/mental.3935 · Cited by 216 · see citing works
Digital phenotyping of suicidal thoughts
Kleiman, E. M., Turner, B. J., Fedor, S., Beale, E. E., Picard, R. W., Huffman, J. C., & Nock, M. K. (2018) · Depression and Anxiety · doi:10.1002/da.22730 · Cited by 210 · see citing works
The Asthma Mobile Health Study, a large-scale clinical observational study using ResearchKit
Chan, Y.-F. Y., Wang, P., Rogers, L., Tignor, N., Zweig, M., Hershman, S. G., Genes, N., Scott, E. R., Krock, E., Badgeley, M., et al. (2017) · Nature Biotechnology · doi:10.1038/nbt.3826 · Cited by 204 · see citing works
Torous, J. B., Chan, S. R., Gipson, S. Y.-M. T., Kim, J. W., Nguyen, T.-Q., Luo, J., & Wang, P. (2018) · Psychiatric Services · doi:10.1176/appi.ps.201700423 · Cited by 184 · see citing works
Self-citationBiagianti, B., Quraishi, S. H., & Schlosser, D. A. (2018) · Psychiatric Services · doi:10.1176/appi.ps.201700283 · Cited by 90 · see citing works
Development of a mental health smartphone app: perspectives of mental health service users
Goodwin, J., Cummins, J. S., Behan, L., & O’Brien, S. M. (2016) · Journal of Mental Health · doi:10.3109/09638237.2015.1124392 · Cited by 77 · see citing works
Mobile health devices: will patients actually use them?
Shaw, R. J., Steinberg, D., Bonnet, J. P., Modarai, F., George, A. E., Cunningham, T., Mason, M., Shahsahebi, M., Grambow, S. C., Bennett, G. G., et al. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv186 · Cited by 72 · see citing works
Characterizing Smartphone Engagement for Schizophrenia: Results of a Naturalist Mobile Health Study
Torous, J., Staples, P., Slaters, L., Adams, J., Sandoval, L., Onnela, J., & Keshavan, M. (2017) · Clinical Schizophrenia & Related Psychoses · doi:10.3371/csrp.jtps.071317 · Cited by 44 · see citing works
Self-citationUnderstanding consumers’ decisions to adopt technology-enabled transformative services
Schuster, L., Drennan, J., & Lings, I. (2015) · The Service Industries Journal · doi:10.1080/02642069.2015.1090979 · Cited by 42 · see citing works
Beyond symptom monitoring: Consumer needs for bipolar disorder self-management using smartphones
Nicholas, J., Boydell, K., & Christensen, H. (2017) · European Psychiatry · doi:10.1016/j.eurpsy.2017.05.023 · Cited by 41 · see citing works
Self-citationMackie, C., Dunn, N., MacLean, S., Testa, V., Heisel, M., & Hatcher, S. (2017) · Evidence Based Mental Health · doi:10.1136/eb-2017-102764 · Cited by 41 · see citing works
Promise and perils of digital psychiatry
Torous, J. B., Keshavan, M. S., & Gutheil, T. G. (2014) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2014.06.006 · Cited by 41 · see citing works
Self-citationTorous, J. B., & Roux, S. (2017) · JMIR Mental Health · doi:10.2196/mental.7911 · Cited by 36 · see citing works
Self-citationDetecting and Explaining Crisis
Kshirsagar, R., Morris, R. R., & Bowman, S. (2017) · doi:10.18653/v1/w17-3108 · Cited by 35 · see citing works
Predictors of Treatment Change and Engagement in Cognitive-Behavioral Group Therapy for Depression
Westra, H. A., Dozois, D. J. A., & Boardman, C. (2002) · Journal of Cognitive Psychotherapy · doi:10.1891/jcop.16.2.227.63996 · Cited by 34 · see citing works
It feels different from real life
de Alva, F. E. M., Wadley, G., & Lederman, R. (2015) · Australian Conference on Computer-Human Interaction (OzCHI) · doi:10.1145/2838739.2838806 · Cited by 30 · see citing works
Alfonsson, S., Olsson, E. M. G., Linderman, S., Winnerhed, S., & Hursti, T. J. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.01.035 · Cited by 25 · see citing works
Digital technology: coming of age?
Marzano, L., Hollis, C. P., Cipriani, A., & Malhi, G. S. (2017) · Evidence-Based Mental Health · doi:10.1136/eb-2017-102821 · Cited by 14 · see citing works
Flagged first, then by citations
1 of the 41 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Borghouts, J., Eikey, E., Mark, G., De Leon, C., Schueller, S. M., Schneider, M., Stadnick, N., Zheng, K., Mukamel, D., & Sorkin, D. H. (2021) · Journal of Medical Internet Research · doi:10.2196/24387 · Cited by 1,069 · see citing works
Lattie, E. G., Adkins, E. C., Winquist, N., Stiles-Shields, C., Wafford, Q. E., & Graham, A. K. (2019) · Journal of Medical Internet Research · doi:10.2196/12869 · Cited by 843 · see citing works
Marx, W., Penninx, B. W. J. H., Solmi, M., Furukawa, T. A., Firth, J., Carvalho, A. F., & Berk, M. (2023) · Nature Reviews Disease Primers · doi:10.1038/s41572-023-00454-1 · Cited by 850 · see citing works
Self-citationTorous, J., Lipschitz, J., Ng, M., & Firth, J. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.11.167 · Cited by 546 · see citing works
Self-citationDesigning and scaling up integrated youth mental health care
McGorry, P. D., Mei, C., Chanen, A., Hodges, C., Alvarez‐Jimenez, M., & Killackey, E. (2022) · World Psychiatry · doi:10.1002/wps.20938 · Cited by 546 · see citing works
Remote patient monitoring using artificial intelligence: Current state, applications, and challenges
Shaik, T., Tao, X., Higgins, N., Li, L., Gururajan, R., Zhou, X., & Acharya, U. R. (2023) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1485 · Cited by 580 · see citing works
An overview of and recommendations for more accessible digital mental health services
Lattie, E. G., Stiles-Shields, C., & Graham, A. K. (2022) · Nature Reviews Psychology · doi:10.1038/s44159-021-00003-1 · Cited by 446 · see citing works
Toward clinical digital phenotyping: a timely opportunity to consider purpose, quality, and safety
Huckvale, K., Venkatesh, S., & Christensen, H. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0166-1 · Cited by 433 · see citing works
Self-citationStandalone smartphone apps for mental health—a systematic review and meta-analysis
Weisel, K. K., Fuhrmann, L. M., Berking, M., Baumeister, H., Cuijpers, P., & Ebert, D. D. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0188-8 · Cited by 447 · see citing works
Real-world menstrual cycle characteristics of more than 600,000 menstrual cycles
Bull, J. R., Rowland, S. P., Scherwitzl, E. B., Scherwitzl, R., Danielsson, K. G., & Harper, J. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0152-7 · Cited by 404 · see citing works
Stade, E. C., Stirman, S. W., Ungar, L. H., Boland, C. L., Schwartz, H. A., Yaden, D. B., Sedoc, J., DeRubeis, R. J., Willer, R., & Eichstaedt, J. C. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00056-z · Cited by 391 · see citing works
“However, these and other existing chatbots frequently struggle to understand and respond to unanticipated user responses 10,13 , which likely contributes to their low engagement and high dropout rates 14,15 .”
BackgroundSurvey of conversational agents in health
Montenegro, J. L. Z., da Costa, C. A., & da Rosa Righi, R. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.03.054 · Cited by 322 · see citing works
User engagement in mental health apps: A review of measurement, reporting, and validity
Ng, M. M., Firth, J., Minen, M., & Torous, J. (2019) · Psychiatric Services · doi:10.1176/appi.ps.201800519 · Cited by 293 · see citing works
Self-citationUsing science to sell apps: Evaluation of mental health app store quality claims
Larsen, M. E., Huckvale, K., Nicholas, J., Torous, J., Birrell, L., Li, E., & Reda, B. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0093-1 · Cited by 334 · see citing works
Self-citationStein, D. J., Shoptaw, S. J., Vigo, D. V., Lund, C., Cuijpers, P., Bantjes, J., Sartorius, N., & Maj, M. (2022) · World Psychiatry · doi:10.1002/wps.20998 · Cited by 331 · see citing works
Torous, J., Linardon, J., Goldberg, S. B., Sun, S., Bell, I., Nicholas, J., Hassan, L., Hua, Y., Milton, A., & Firth, J. (2025) · World Psychiatry · doi:10.1002/wps.21299 · Cited by 323 · see citing works
Self-citationGan, D. Z. Q., McGillivray, L., Han, J., Christensen, H., & Torok, M. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.764079 · Cited by 305 · see citing works
Self-citationUse of Social Robots in Mental Health and Well-Being Research: Systematic Review
Scoglio, A. A. J., Reilly, E. D., Gorman, J. A., & Drebing, C. E. (2019) · Journal of Medical Internet Research · doi:10.2196/13322 · Cited by 290 · see citing works
Insights from user reviews to improve mental health apps
Alqahtani, F., & Orji, R. (2020) · Health Informatics Journal · doi:10.1177/1460458219896492 · Cited by 286 · see citing works
Prochaska, J. J., Vogel, E. A., Chieng, A., Kendra, M., Baiocchi, M., Pajarito, S., & Robinson, A. (2021) · Journal of Medical Internet Research · doi:10.2196/24850 · Cited by 257 · see citing works
Wei, Y., Zheng, P., Deng, H., Wang, X., Li, X., & Fu, H. (2020) · Journal of Medical Internet Research · doi:10.2196/21687 · Cited by 238 · see citing works
Torous, J., Wisniewski, H., Liu, G., & Keshavan, M. (2018) · JMIR Mental Health · doi:10.2196/11715 · Cited by 201 · see citing works
Self-citationTorous, J., Wisniewski, H., Bird, B., Carpenter, E., David, G., Elejalde, E., Fulford, D., Guimond, S., Hays, R., Henson, P., et al. (2019) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-019-00095-w · Cited by 202 · see citing works
Self-citationDigital transformation of mental health services
Bond, R. R., Mulvenna, M. D., Potts, C., O’Neill, S., Ennis, E., & Torous, J. (2023) · npj Mental Health Research · doi:10.1038/s44184-023-00033-y · Cited by 199 · see citing works
Self-citationOyebode, O., Alqahtani, F., & Orji, R. (2020) · IEEE Access · doi:10.1109/access.2020.3002176 · Cited by 197 · see citing works
Moberg, C., Niles, A., & Beermann, D. (2019) · Journal of Medical Internet Research · doi:10.2196/12556 · Cited by 157 · see citing works
Digital privacy in mental healthcare: current issues and recommendations for technology use
Lustgarten, S. D., Garrison, Y. L., Sinnard, M. T., & Flynn, A. W. P. (2020) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2020.03.012 · Cited by 176 · see citing works
Litvin, S., Saunders, R., Maier, M. A., & Lüttke, S. (2020) · PLOS ONE · doi:10.1371/journal.pone.0237220 · Cited by 177 · see citing works
“Moreover, smartphone-delivered interventions for mental health problems suffer from low engagement, with mean attrition rates ranging from 9.6% for interventions targeting general stress to 50.0% for apps focusing on insomnia relief [23].”
BackgroundInfluentialWu, A., Scult, M. A., Barnes, E. D., Betancourt, J. A., Falk, A. E., & Gunning, F. M. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00386-8 · Cited by 164 · see citing works
Barriers to older adults’ uptake of mobile-based mental health interventions
Pywell, J., Vijaykumar, S., Dodd, A., & Coventry, L. (2020) · Digital Health · doi:10.1177/2055207620905422 · Cited by 156 · see citing works
Potential and Pitfalls of Mobile Mental Health Apps in Traditional Treatment: An Umbrella Review
Koh, J., Tng, G. Y. Q., & Hartanto, A. (2022) · Journal of Personalized Medicine · doi:10.3390/jpm12091376 · Cited by 155 · see citing works
Digital navigators to implement smartphone and digital tools in care
Wisniewski, H., & Torous, J. B. (2020) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.13149 · Cited by 144 · see citing works
Self-citationCarter, H., Araya, R., Anjur, K. I., Deng, D., & Naslund, J. A. (2020) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2020.12.016 · Cited by 155 · see citing works
Smartphone apps for the treatment of mental health conditions: status and considerations
Huckvale, K., Nicholas, J., Torous, J. B., & Larsen, M. E. (2020) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2020.04.008 · Cited by 137 · see citing works
Self-citationDahlhausen, F., Zinner, M., Bieske, L., Ehlers, J. P., Boehme, P., & Fehring, L. (2021) · JMIR mhealth and uhealth · doi:10.2196/33012 · Cited by 139 · see citing works
Gabrielli, S., Rizzi, S., Bassi, G., Carbone, S., Maimone, R., Marchesoni, M., & Forti, S. (2021) · JMIR mHealth and uHealth · doi:10.2196/27965 · Cited by 125 · see citing works
“Empirical studies on evidence-based digital interventions for mental health, including internet-based cognitive behavioral therapy (CBT), have shown that these interventions are effective, feasible, and acceptable to users [12-14], although some limitations have been found, mainly regarding low engagement by users and…”
BackgroundChatbots to Support Young Adults’ Mental Health: An Exploratory Study of Acceptability
Koulouri, T., Macredie, R. D., & Olakitan, D. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3485874 · Cited by 127 · see citing works
Human-Computer Interaction in Digital Mental Health
Balcombe, L., & Leo, D. D. (2022) · Informatics · doi:10.3390/informatics9010014 · Cited by 146 · see citing works
Cognitive-Behavioral Therapy in the Digital Age: Presidential Address
Wilhelm, S., Weingarden, H. M., Ladis, I., Braddick, V., Shin, J. Y., & Jacobson, N. C. (2019) · Behavior Therapy · doi:10.1016/j.beth.2019.08.001 · Cited by 122 · see citing works
Global research priorities for youth mental health
Mei, C., Fitzsimons, J., Allen, N. B., Álvarez‐Jiménez, M., Amminger, G. P., Browne, V., Cannon, M. D., Davis, M., Dooley, B. A., Hickie, I. B., et al. (2020) · Early Intervention in Psychiatry · doi:10.1111/eip.12878 · Cited by 117 · see citing works
Mohr, D. C., Schueller, S. M., Tomasino, K. N., Kaiser, S. M., Alam, N., Karr, C., Vergara, J. L., Gray, E. L., Kwasny, M. J., & Lattie, E. G. (2019) · Journal of Medical Internet Research · doi:10.2196/13609 · Cited by 116 · see citing works
Stawarz, K., Preist, C., & Coyle, D. (2019) · JMIR Mental Health · doi:10.2196/12546 · Cited by 107 · see citing works
Chan, A. H. Y., & Honey, M. L. L. (2021) · Journal of Psychiatric and Mental Health Nursing · doi:10.1111/jpm.12744 · Cited by 113 · see citing works
Oliveira, C., Pereira, A., Vagos, P., Nóbrega, C. A., Gonçalves, J., & Afonso, B. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.647606 · Cited by 116 · see citing works
Old-fashioned technology in the era of “bling”: is there a future for text messaging in health care?
Willcox, J. C., Dobson, R., & Whittaker, R. (2019) · Journal of Medical Internet Research · doi:10.2196/16630 · Cited by 95 · see citing works
A pilot digital intervention targeting loneliness in young people with psychosis
Lim, M. H., Gleeson, J., Rodebaugh, T. L., Eres, R., Long, K. M., Casey, K., Abbott, J.-A., Thomas, N., & Penn, D. L. (2019) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-019-01681-2 · Cited by 108 · see citing works
Human-Centered Design Approaches in Digital Mental Health Interventions: Exploratory Mapping Review
Vial, S., Boudhraâ, S., & Dumont, M. (2022) · JMIR Mental Health · doi:10.2196/35591 · Cited by 114 · see citing works
College student engagement with mental health apps: analysis of barriers to sustained use
Melcher, J., Camacho, E., Lagan, S., & Torous, J. (2022) · Journal of American College Health · doi:10.1080/07448481.2020.1825225 · Cited by 106 · see citing works
Self-citationUsability Evaluations of Mobile Mental Health Technologies: Systematic Review
İnal, Y., Wake, J. D., Guribye, F., & Nordgreen, T. (2019) · Journal of Medical Internet Research · doi:10.2196/15337 · Cited by 104 · see citing works
User‐centered design for technology‐enabled services for eating disorders
Graham, A. K., Wildes, J. E., Reddy, M. C., Munson, S. A., Taylor, C. B., & Mohr, D. C. (2019) · International Journal of Eating Disorders · doi:10.1002/eat.23130 · Cited by 103 · see citing works
Prochaska, J. J., Vogel, E. A., Chieng, A., Baiocchi, M., Maglalang, D. D., Pajarito, S., Weingardt, K. R., Darcy, A., & Robinson, A. (2021) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2021.108986 · Cited by 113 · see citing works
Clinically Meaningful Use of Mental Health Apps and its Effects on Depression: Mixed Methods Study
Zhang, R., Nicholas, J., Knapp, A. A., Graham, A. K., Gray, E., Kwasny, M. J., Reddy, M., & Mohr, D. C. (2019) · Journal of Medical Internet Research · doi:10.2196/15644 · Cited by 91 · see citing works
Self-citationDoes the Digital Therapeutic Alliance Exist? Integrative Review
Malouin-Lachance, A., Capolupo, J., Laplante, C., & Hudon, A. (2025) · JMIR Mental Health · doi:10.2196/69294 · Cited by 103 · see citing works
A systematic review of digital and face-to-face cognitive behavioral therapy for depression
Kambeitz-Ilankovic, L., Rzayeva, U., Völkel, L., Wenzel, J., Weiske, J., Jessen, F., Reininghaus, U., Uhlhaas, P. J., Alvarez-Jimenez, M., & Kambeitz, J. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00677-8 · Cited by 103 · see citing works
Preliminary Evaluation of the Engagement and Effectiveness of a Mental Health Chatbot
Daley, K., Hungerbuehler, I., Cavanagh, K., Claro, H. G., Swinton, P. A., & Kapps, M. (2020) · Frontiers in Digital Health · doi:10.3389/fdgth.2020.576361 · Cited by 101 · see citing works
Naslund, J. A., Gonsalves, P. P., Gruebner, O., Pendse, S. R., Smith, S. L., Sharma, A., & Raviola, G. J. (2019) · Current Treatment Options in Psychiatry · doi:10.1007/s40501-019-00186-8 · Cited by 108 · see citing works
Digital Mental Health Tools for Caregivers of Older Adults—A Scoping Review
Petrovic, M., & Gaggioli, A. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00128 · Cited by 100 · see citing works
Wasil, A. R., Taylor, M. E., Franzen, R. E., Steinberg, J. S., & DeRubeis, R. J. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.569785 · Cited by 89 · see citing works
Recommender systems for mental health apps: advantages and ethical challenges
Valentine, L., D’Alfonso, S., & Lederman, R. (2022) · AI & Society · doi:10.1007/s00146-021-01322-w · Cited by 97 · see citing works
A Pilot Digital Intervention Targeting Loneliness in Youth Mental Health
Lim, M. H., Rodebaugh, T. L., Eres, R., Long, K. M., Penn, D. L., & Gleeson, J. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00604 · Cited by 91 · see citing works
Weerdmeester, J., van Rooij, M. M. J. W., Engels, R. C. M. E., & Granic, I. (2020) · Journal of Medical Internet Research · doi:10.2196/14958 · Cited by 88 · see citing works
Gan, D. Z. Q., McGillivray, L., Larsen, M. E., Christensen, H. M., & Torok, M. H. (2022) · Digital Health · doi:10.1177/20552076221098268 · Cited by 81 · see citing works
Self-citationConnolly, S. L., Hogan, T. P., Shimada, S. L., & Miller, C. J. (2020) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-020-00165-4 · Cited by 78 · see citing works
Digital Mental Health: The Answer to the Global Mental Health Crisis?
Rudd, B. N., & Beidas, R. S. (2020) · JMIR Mental Health · doi:10.2196/18472 · Cited by 81 · see citing works
Linardon, J., Shatte, A. B. R., Rosato, J., & Fuller‐Tyszkiewicz, M. (2020) · Psychological Medicine · doi:10.1017/s0033291720003426 · Cited by 77 · see citing works
Evaluation of the Use of Digital Mental Health Platforms and Interventions: Scoping Review
Balcombe, L., & Leo, D. D. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20010362 · Cited by 83 · see citing works
Adu, M. D., Malabu, U. H., Malau‐Aduli, A. E. O., & Malau‐Aduli, B. S. (2018) · PLoS ONE · doi:10.1371/journal.pone.0208942 · Cited by 76 · see citing works
Böhm, A.-K., Jensen, M. L., Sørensen, M. R., & Stargardt, T. (2020) · JMIR mhealth and uhealth · doi:10.2196/22212 · Cited by 75 · see citing works
Zhang, R., Ringland, K. E., Paan, M., Mohr, D. C., & Reddy, M. C. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445771 · Cited by 75 · see citing works
Hensher, M., Cooper, P., Dona, S. W. A., Angeles, M. R., Nguyen, D., Heynsbergh, N., Chatterton, M. L., & Peeters, A. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab041 · Cited by 80 · see citing works
Chatbot-Based Assessment of Employees’ Mental Health: Design Process and Pilot Implementation
Hungerbuehler, I., Daley, K., Cavanagh, K., Garcia Claro, H., & Kapps, M. (2021) · JMIR Formative Research · doi:10.2196/21678 · Cited by 73 · see citing works
Mayer, G., Gronewold, N., Alvarez, S., Bruns, B., Hilbel, T., & Schultz, J. (2019) · JMIR Mental Health · doi:10.2196/14018 · Cited by 72 · see citing works
Direct-to-consumer digital health
Cohen, A. B., Mathews, S. C., Dorsey, E. R., Bates, D. W., & Safavi, K. C. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30057-1 · Cited by 70 · see citing works
Developments in Privacy and Data Ownership in Mobile Health Technologies, 2016-2019
Galvin, H. K., & DeMuro, P. R. (2020) · Yearbook of Medical Informatics · doi:10.1055/s-0040-1701987 · Cited by 70 · see citing works
Linardon, J. (2023) · Behaviour Research and Therapy · doi:10.1016/j.brat.2023.104421 · Cited by 69 · see citing works
Psychiatry in the Digital Age: A Blessing or a Curse?
Roth, C. B., Papassotiropoulos, A., Brühl, A. B., Lang, U. E., & Huber, C. G. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18168302 · Cited by 71 · see citing works
Jabir, A. I., Lin, X., Martinengo, L., Sharp, G., Theng, Y. L., & Car, L. T. (2023) · Journal of Medical Internet Research · doi:10.2196/48168 · Cited by 70 · see citing works
Danieli, M., Ciulli, T., Mousavi, S. M., & Riccardi, G. (2021) · JMIR Formative Research · doi:10.2196/30053 · Cited by 64 · see citing works
“However, there is little consensus on the usability and effectiveness of such systems [13].”
Bell, I. H., Thompson, A., Valentine, L., Adams, S., Alvarez-Jimenez, M., & Nicholas, J. (2022) · JMIR Mental Health · doi:10.2196/30716 · Cited by 60 · see citing works
“This is true both in clinical trials [15] and, particularly, in naturalistic studies in which apps are used in the wild [16], where good initial uptake is commonly followed by a dramatic drop in use over time [15].”
BackgroundSelf-citationThornton, L. K., Gardner, L. A., Osman, B., Green, O., Champion, K. E., Bryant, Z., Teesson, M., KAY‐LAMBKIN, F. J., Chapman, C., & The Health4Life Team (2021) · JMIR Formative Research · doi:10.2196/25513 · Cited by 66 · see citing works
Kornfield, R., Zhang, R., Nicholas, J., Schueller, S. M., Cambo, S. A., Mohr, D. C., & Reddy, M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376309 · Cited by 64 · see citing works
Self-citationTighe, J., Shand, F., McKay, K., Mcalister, T.-J., Mackinnon, A., & Christensen, H. (2020) · JMIR Mental Health · doi:10.2196/14296 · Cited by 63 · see citing works
Self-citationDeterminant Factors of Public Acceptance of Stress Management Apps: Survey Study
Apolinário-Hagen, J., Hennemann, S., Fritsche, L., Drüge, M., & Breil, B. (2019) · JMIR Mental Health · doi:10.2196/15373 · Cited by 63 · see citing works
Wilhelm, S., Weingarden, H. M., Greenberg, J. L., McCoy, T. H., Ladis, I., Summers, B. J., Matic, A., & Harrison, O. (2019) · Behavior Therapy · doi:10.1016/j.beth.2019.03.007 · Cited by 62 · see citing works
Heim, E., Ramia, J. A., Hana, R. A., Burchert, S., Carswell, K., Cornelisz, I., Cuijpers, P., El Chammay, R., Noun, P., van Klaveren, C., et al. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100380 · Cited by 61 · see citing works
Understanding Mental Health App Use Among Community College Students: Web-Based Survey Study
Borghouts, J. W., Eikey, E. V., Mark, G. J., Leon, C. D., Schueller, S. M., Schneider, M. L., Stadnick, N. A., Zheng, K., Mukamel, D. B., & Sorkin, D. H. (2021) · Journal of Medical Internet Research · doi:10.2196/27745 · Cited by 62 · see citing works
Pendse, S. R., Sharma, A., Vashistha, A., Choudhury, M. D., & Kumar, N. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445410 · Cited by 59 · see citing works
Kodish, T., Lau, A. S., Gong-Guy, E., Congdon, E. L., Arnaudova, I., Schmidt, M., Shoemaker, L., & Craske, M. G. (2021) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-021-01163-1 · Cited by 58 · see citing works
Bricker, J. B., Mull, K. E., Santiago-Torres, M., Miao, Z., Perski, O., & Di, C. (2022) · Journal of Medical Internet Research · doi:10.2196/39208 · Cited by 44 · see citing works
Single-session interventions embedded within Tumblr: acceptability, feasibility, and utility study
Dobias, M. L., Morris, R. R., & Schleider, J. L. (2022) · JMIR Formative Research · doi:10.2196/39004 · Cited by 56 · see citing works
Andalibi, N., & Flood, M. K. (2021) · JMIR Mental Health · doi:10.2196/21819 · Cited by 55 · see citing works
Hentati, A., Forsell, E., Ljótsson, B., Kaldo, V., Lindefors, N., & Kraepelien, M. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100448 · Cited by 56 · see citing works
Czyz, E. K., King, C. A., Prouty, D., Micol, V. J., Walton, M., & Nahum‐Shani, I. (2021) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13383 · Cited by 54 · see citing works
Oti, O., & Pitt, I. J. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100468 · Cited by 57 · see citing works
Ahmad, N. A., Ludin, A. F. M., Shahar, S., Noah, S. A. M., & Tohit, N. M. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-054561 · Cited by 56 · see citing works
The future of psychological treatments: The Marburg Declaration
Rief, W., Asmundson, G. J. G., Bryant, R. A., Clark, D. M., Ehlers, A., Holmes, E. A., McNally, R. J., Neufeld, C. B., Wilhelm, S., Jaroszewski, A. C., et al. (2024) · Clinical Psychology Review · doi:10.1016/j.cpr.2024.102417 · Cited by 54 · see citing works
Sawrikar, V., & Mote, K. (2022) · Health Policy and Technology · doi:10.1016/j.hlpt.2022.100686 · Cited by 53 · see citing works
Lemon, C. A., Huckvale, K., Carswell, K., & Torous, J. B. (2020) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462319003507 · Cited by 52 · see citing works
Self-citationXiong, A., Wang, T., Li, N., & Jha, S. (2020) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp40000.2020.00088 · Cited by 52 · see citing works
Exploring medical students' views on digital mental health interventions: A qualitative study
Dederichs, M., Weber, J., Pischke, C. R., Angerer, P., & Apolinário-Hagen, J. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100398 · Cited by 50 · see citing works
- Authors
- Torous, J., Nicholas, J., Larsen, M. E., Firth, J., & Christensen, H.
- Affiliations
- Beth Israel Deaconess Medical Center (US)
- Black Dog Institute (AU)
- Western Sydney University (AU)
- Corresponding author
- Torous, John
- Venue
- Evidence Based Mental Health
- Publisher
- BMJ
- Published
- 2018-06-05
- Volume
- 21
- Issue
- 3
- Pages
- 116-119
- ISSN
- 1362-0347 (ISSN-L)
- 1468-960X
- PMC
- PMC10270395
- OpenAlex
- W2806313543
- Cited by
- 790
- References
- 41
- Citations by year
- 2026: 99 · 2025: 135 · 2024: 103 · 2023: 96 · 2022: 106 · 2021: 119 · 2020: 68 · 2019: 57 · 2018: 7
- FWCI
- 8.51
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Evidence-Based Mental Health (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Research Explorer (The University of Manchester) (submittedVersion)
- Funders
- National Alliance for Research on Schizophrenia and Depression
- National Institute of Mental Health (K23MH116130)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Health Sciences › Health Professions › General Health Professions › Mental Health and Patient Involvement
- Keywords
- mental health apps, mobile application, user engagement, engagement metrics, mobile application design, privacy, trust, emergency relief, digital mental health, app uptake
- MeSH terms
- Smartphone, Mental Disorders, Humans, Mental Health Services, Patient Acceptance of Health Care, Patient Participation, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
