Characterizing Smartphone Engagement for Schizophrenia: Results of a Naturalist Mobile Health Study
Abstract
INTRODUCTION: Despite growing interest in smartphone apps for schizophrenia, little is known about how these apps are utilized in the real world. Understanding how app users are engaging with these tools outside of the confines of traditional clinical studies offers an important information on who is most likely to use apps and what type of data they are willing to share. METHODS: 2017 using the R programming language. We included all registered app users in our analysis with reported ages less than 100. RESULTS: We analyzed a total of 43,451 mood, medication and symptom entries from 622 registered users, and excluded a single patient with a reported age of 114. Seventy one percent of the 622 users tried the mood-tracking feature at least once, 49% the symptom tracking feature, and 36% the medication-tracking feature. The mean number of uses of the mood feature was two, the symptom feature 10, and the medication feature 14. However, a small subset of users were very engaged with the app and the top 10 users for each feature accounted for 35% or greater of all entries for that feature. We find that user engagement follows a power law distribution for each feature, and this fit was largely invariant when stratifying for age or gender. DISCUSSION: Engagement with this app for schizophrenia was overall low, but similar to prior naturalistic studies for mental health app use in other diseases. The low rate of engagement in naturalistic settings, compared to higher rates of use in clinical studies, suggests the importance of clinical involvement as one factor in driving engagement for mental health apps. Power law relationships suggest strongly skewed user engagement, with a small subset of users accounting for the majority of substantial engagements. There is a need for further research on app engagement in schizophrenia.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Torous et al found that future use of a schizophrenia self-management app is directly related to prior app use, suggesting those who engage more often have a higher probability of future engagement.
“They found that future use with the app is directly related to prior app use, suggesting that those who engage with the app more often will have a higher probability of app engagement in the future.”
From A Library of Analytic Indicators to Evaluate Effective Engagement with Consumer mHealth Apps for Chronic Conditions: Scoping Review (2019) · cites this paper as [44]
Cited by
Sorted by citations
Torous, J., Nicholas, J., Larsen, M. E., Firth, J., & Christensen, H. (2018) · Evidence Based Mental Health · doi:10.1136/eb-2018-102891 · Cited by 790 · see citing works
Self-citationBidargaddi, N., Almirall, D., Murphy, S., Nahum-Shani, I., Kovalcik, M., Pituch, T., Maaieh, H., & Strecher, V. (2018) · JMIR mHealth and uHealth · doi:10.2196/10123 · Cited by 189 · see citing works
“Even among those who use the apps, the amount of use depends on an individual’s health and behavioral characteristics [10-12].”
BackgroundDigital navigators to implement smartphone and digital tools in care
Wisniewski, H., & Torous, J. B. (2020) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.13149 · Cited by 144 · see citing works
Self-citationPham, Q., Graham, G., Carrion, C., Morita, P. P., Seto, E., Stinson, J. N., & Cafazzo, J. A. (2019) · JMIR mHealth and uHealth · doi:10.2196/11941 · Cited by 131 · see citing works
“Torous et al also characterized engagement for a schizophrenia self-management app through fitting frequency of interaction data to a piecewise power law distribution [44].”
College student engagement with mental health apps: analysis of barriers to sustained use
Melcher, J., Camacho, E., Lagan, S., & Torous, J. (2022) · Journal of American College Health · doi:10.1080/07448481.2020.1825225 · Cited by 106 · see citing works
Self-citationTorous, J., Firth, J., Huckvale, K., Larsen, M. E., Cosco, T. D., Carney, R., Chan, S., Pratap, A., Yellowlees, P., Wykes, T., et al. (2018) · Journal of Nervous & Mental Disease · doi:10.1097/nmd.0000000000000864 · Cited by 101 · see citing works
Self-citationHOFFMAN, L., WISNIEWSKI, H., HAYS, R., HENSON, P., VAIDYAM, A., HENDEL, V., KESHAVAN, M., & TOROUS, J. (2020) · Journal of Psychiatric Practice · doi:10.1097/pra.0000000000000450 · Cited by 97 · see citing works
Self-citationDigital Technologies in the Treatment of Anxiety: Recent Innovations and Future Directions
Firth, J., Torous, J., Carney, R., Newby, J., Cosco, T. D., Christensen, H., & Sarris, J. (2018) · Current Psychiatry Reports · doi:10.1007/s11920-018-0910-2 · Cited by 70 · see citing works
Self-citationArnold, C., Villagonzalo, K.-A., Meyer, D., Farhall, J., Foley, F., Kyrios, M., & Thomas, N. (2019) · Internet Interventions · doi:10.1016/j.invent.2019.100266 · Cited by 63 · see citing works
Bell, I. H., Rossell, S. L., Farhall, J., Hayward, M., Lim, M. H., Fielding-Smith, S. F., & Thomas, N. (2019) · Schizophrenia Research · doi:10.1016/j.schres.2019.10.026 · Cited by 70 · see citing works
Use of Smartphones, Computers and Social Media Among People with SMI: Opportunity for Intervention
Brunette, M. F., Achtyes, E., Pratt, S., Stilwell, K., Opperman, M., Guarino, S., & Kay-Lambkin, F. (2019) · Community Mental Health Journal · doi:10.1007/s10597-019-00431-7 · Cited by 67 · see citing works
Bell, I. H., Thompson, A., Valentine, L., Adams, S., Alvarez-Jimenez, M., & Nicholas, J. (2022) · JMIR Mental Health · doi:10.2196/30716 · Cited by 60 · see citing works
“This is true both in clinical trials [15] and, particularly, in naturalistic studies in which apps are used in the wild [16], where good initial uptake is commonly followed by a dramatic drop in use over time [15].”
BackgroundHatch, A., Hoffman, J. E., ROSS, R. W., & Docherty, J. P. (2018) · JMIR Mental Health · doi:10.2196/mental.9777 · Cited by 42 · see citing works
“Although digital health tools hold significant promise for the improvement of various elements of behavioral health intervention, and surveys have indicated that people with SMI are enthusiastic about using technology [21,22], adoption has been slow, testing of outcomes from acceptability to effectiveness has been lim…”
BackgroundArnold, C., Williams, A., & Thomas, N. (2020) · JMIR Mental Health · doi:10.2196/16730 · Cited by 42 · see citing works
“However, a large proportion will have only initial or infrequent use of an intervention, thus limiting potential associated benefits [17].”
BackgroundDigital Trajectories to Care in First-Episode Psychosis
Birnbaum, M., Rizvi, A., Faber, K., Addington, J., Correll, C., Gerber, C., Lahti, A., Loewy, R., Mathalon, D., Nelson, L., et al. (2018) · Psychiatric Services · doi:10.1176/appi.ps.201800180 · Cited by 34 · see citing works
“Unfortunately, utilization and engagement with existing digital mental health tools are poor (12), and new and innovative solutions need to be explored.”
Smelror, R. E., Bless, J. J., Hugdahl, K., & Agartz, I. (2019) · JMIR Formative Research · doi:10.2196/13882 · Cited by 25 · see citing works
“There are at least 11 mobile phone apps that have been developed for schizophrenia [20,27-33].”
BackgroundAccessibility and availability of smartphone apps for schizophrenia
Kwon, S., Firth, J., Joshi, D., & Torous, J. B. (2022) · Schizophrenia · doi:10.1038/s41537-022-00313-0 · Cited by 32 · see citing works
Self-citationCarpenter-Song, E. A., Acquilano, S. C., Noel, V. A., Al‐Abdulmunem, M., Torous, J. B., & Drake, R. E. (2021) · Community Mental Health Journal · doi:10.1007/s10597-021-00798-6 · Cited by 32 · see citing works
Self-citationDigital Technology to Enhance Clinical Care of Early Psychosis
Bell, I. H., & Álvarez‐Jiménez, M. (2019) · Current Treatment Options in Psychiatry · doi:10.1007/s40501-019-00182-y · Cited by 23 · see citing works
Minen, M. T., Jaran, J., Boyers, T., & Corner, S. E. (2020) · Headache The Journal of Head and Face Pain · doi:10.1111/head.13851 · Cited by 22 · see citing works
Wang, J., Rogge, A. A., Armour, M., Smith, C. A., D’Adamo, C. R., Pischke, C. R., Yen, H., Wu, M., Moré, A. O. O., Witt, C. M., et al. (2019) · JMIR mhealth and uhealth · doi:10.2196/14661 · Cited by 15 · see citing works
“User adherence and survey response rate are usually considered to be the measurements for evaluating engagement in app studies [40,44,45].”
Ortiz, A., Park, Y., Gonzalez‐Torres, C., Alda, M., Blumberger, D. M., Burnett, R., Husain, M. I., Sanches, M., & Mulsant, B. H. (2023) · International Journal of Bipolar Disorders · doi:10.1186/s40345-023-00297-5 · Cited by 15 · see citing works
Jin, S., Kim, B., & Han, K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713732 · Cited by 15 · see citing works
“…on certain devices or platforms, unmet initial expectations after download, and discomfort during the onboarding process often lead to early user dropout [3, 151, 210] Users with depression may find it difficult to use apps due to low energy and lack of motivation [8, 28, 51, 144, 162, 198].”
Lo, B., Shi, J., Hollenberg, E., Abi-Jaoudé, A., Johnson, A., & Wiljer, D. (2020) · JMIR Mental Health · doi:10.2196/15942 · Cited by 13 · see citing works
“Studies such as Torous et al [111] and Connolly et al [112] have identified that individuals with different mental health conditions and culture can impact their engagement with technology.”
Health Information Technology Resources to Support Measurement-Based Care
Torous, J. B., Powell, A. C., & Rodriguez-Villa, E. (2020) · Child and Adolescent Psychiatric Clinics of North America · doi:10.1016/j.chc.2020.06.011 · Cited by 11 · see citing works
Self-citationHan, H. J., Mendu, S., Jaworski, B. K., Owen, J. E., & Abdullah, S. (2023) · JMIR Formative Research · doi:10.2196/45894 · Cited by 12 · see citing works
“CAs are defined as dialogue systems that provide support for specific tasks [27,28].”
BackgroundVerbal memory measurement towards digital perspectives in first-episode psychosis: A review
Kilçiksiz, C. M., Keefe, R. S. E., Benoit, J. R. A., Öngür, D., & Torous, J. B. (2020) · Schizophrenia Research Cognition · doi:10.1016/j.scog.2020.100177 · Cited by 12 · see citing works
Self-citationHan, H. J., Mendu, S., Jaworski, B. K., Owen, J. E., & Abdullah, S. (2021) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3460418.3479332 · Cited by 16 · see citing works
“The mobile app for schizophrenia patients revealed that the degree of involvement decreases dramatically over time [43].”
BackgroundAustin, S. F., Frøsig, A. J., Buus, N., Lincoln, T. M., von Malachowski, A., Schlier, B., Frost, M. M., & Simonsen, E. H. (2021) · Qualitative Health Research · doi:10.1177/1049732320986556 · Cited by 13 · see citing works
“The use of mobile-based interventions (mHealth) can have a number of advantages such as facilitating access to evidence-based treatment; enhancing the potency of psychotherapy; reducing stigmatization; enabling users to work at their own pace; and promoting autonomy and the flexible integration of mental health interv…”
BackgroundAssessing Digital Phenotyping for App Recommendations and Sustained Engagement: Cohort Study
Dwyer, B., Flathers, M., Burns, J., Mikkelson, J., Perlmutter, E., Chen, K., Ram, N., & Torous, J. B. (2024) · JMIR Formative Research · doi:10.2196/62725 · Cited by 9 · see citing works
“Numerous other studies confirm exponential decay in app engagement, regardless of the health condition, age, gender, or race of users [6,7].”
Self-citationPrice, G. D., Heinz, M. V., Nemesure, M., McFadden, J., & Jacobson, N. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.807116 · Cited by 9 · see citing works
“These findings are important in light of evidence suggesting overall lower digital app engagement among populations with schizophrenia, likely skewed by small subgroups of heavily engaged participants (58).”
BackgroundPassive Digital Phenotyping: objective quantification of human behaviour through smartphones
Jongs, N. (2021) · doi:10.33612/diss.171368248 · Cited by 6 · see citing works
Strauss, J., Zhang, J., Jarrett, M. L., Patterson, B. A., & Ameringen, M. V. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-822201-0.00006-x · Cited by 8 · see citing works
Designing an Evidence-based Mental Health Intervention Alongside University Students
Williams, L., Pinto, M. D., Abrahim, H. L., Bhattacharya, A., Harrison, T., Burton, C. W., McIntyre, J. R. S., Negi, L. T., & Hayes, G. R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35572-1_13 · Cited by 5 · see citing works
von Malachowski, A., Schlier, B., Austin, S. F., Frost, M. M., Frøsig, A. J., Heinzle, M., Holzapfel, H., Lipps, A., Simonsen, E. H., & Lincoln, T. M. (2022) · Schizophrenia Research · doi:10.1016/j.schres.2021.12.011 · Cited by 5 · see citing works
Arigo, D., & Torous, J. B. (2022) · JMIR Formative Research · doi:10.2196/36541 · Cited by 5 · see citing works
“Modest compliance with mobile assessments is common among individuals with schizophrenia and other severe and persistent mental illnesses [30-33].”
BackgroundSelf-citationSmart Steps for Psychiatric Education: Approaching Smartphone Apps for Learning and Care
Torous, J. B., Bauer, A. M., Chan, S. R., BOLAND, R. J., & Ramo, D. E. (2018) · Academic Psychiatry · doi:10.1007/s40596-018-0901-y · Cited by 5 · see citing works
“Evidence suggests that most who download a mental health app may not stick with that app for more than two or three uses, and many patients may struggle to navigate how to actually use apps in daily life [28].”
Self-citationFrench, C., Sawyer, C. S., Fabian, H., Hassan, L., Sainsbury, J., Scrivano, L., Stubbs, B., Bucci, S., Torous, J. B., & Firth, J. (2026) · Schizophrenia Research · doi:10.1016/j.schres.2026.03.018 · Cited by 2 · see citing works
Self-citationSong, S., Ashton, M., Yoo, R. H., Lkhagvajav, Z., Wright, R. A., Mathews, D., & Taylor, C. O. (2024) · Journal of Medical Internet Research · doi:10.2196/51955 · Cited by 2 · see citing works
“Library journal subscriptions and interlibrary loan were used to obtain full text, with the full text of 1 publication [39] not available from either source.”
Des repères pour la conception des apps ?
Khazaal, Y., Potvin, S., Pennou, A., Djomo, W., Borgeat, F., & Lecomte, T. (2021) · Santé mentale au Québec · doi:10.7202/1081512ar · Cited by 2 · see citing works
Halverson, T. F., Browne, J., Thomas, S. M., Palenski, P. E., & Vilardaga, R. (2022) · Schizophrenia Research Cognition · doi:10.1016/j.scog.2022.100236 · Cited by 2 · see citing works
Xia, W., Hau, C., Firth, J., Kimhy, D., & Torous, J. B. (2025) · FOCUS The Journal of Lifelong Learning in Psychiatry · doi:10.1176/appi.focus.20250014 · Cited by 1 · see citing works
Self-citation
- Authors
- Torous, J., Staples, P., Slaters, L., Adams, J., Sandoval, L., Onnela, J., & Keshavan, M.
- Affiliations
- Beth Israel Deaconess Medical Center (US)
- Harvard University (US)
- Schizophrenia and Related Disorders Alliance of America, Houston, TX
- Self Care Catalyst, San Francisco, CA
- Venue
- Clinical Schizophrenia & Related Psychoses
- Publisher
- Longdom Group
- Published
- 2017-08-04
- Issue
- aop
- ISSN
- 1935-1232 (ISSN-L)
- OpenAlex
- W2743267105
- Cited by
- 44
- References
- 0
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 2 · 2023: 3 · 2022: 5 · 2021: 7 · 2020: 8 · 2019: 8 · 2018: 8
- FWCI
- 7.53
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Clinical Schizophrenia & Related Psychoses (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Keywords
- schizophrenia, mobile application, mobile health, user engagement, mood tracking, symptom monitoring, medication tracking, power-law distribution, mental health apps
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
