User Experience, Engagement, and Popularity in Mental Health Apps: Secondary Analysis of App Analytics and Expert App Reviews
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Background: User experience and engagement are critical elements of mental health apps’ abilities to support users. However, work examining the relationships among user experience, engagement, and popularity has been limited. Understanding how user experience relates to engagement with and popularity of mental health apps can demonstrate the relationship between subjective and objective measures of app use. In turn, this may inform efforts to develop more effective and appealing mental health apps and ensure that they reach wide audiences. Objective: We aimed to examine the relationship among subjective measures of user experience, objective measures of popularity, and engagement in mental health apps. Methods: We conducted a preregistered secondary data analysis in a sample of 56 mental health apps. To measure user experience, we used expert ratings on the Mobile App Rating Scale (MARS) and consumer ratings from the Apple App Store and Google Play. To measure engagement, we acquired estimates of monthly active users (MAU) and user retention. To measure app popularity, we used download count, total app revenue, and MAU again. Results: MARS total score was moderately positively correlated with app-level revenue (Kendall rank [T]=0.30, P=.002), MAU (T=0.39, P<.001), and downloads (T=0.41, P<.001). However, the MARS total score and each of its subscales (Engagement, Functionality, Aesthetics, and Information) showed extremely small correlations with user retention 1, 7, and 30 days after downloading. Furthermore, the total MARS score only correlated with app store rating at T=0.12, which, at P=.20, did not meet our threshold for significance. Conclusions: More popular mental health apps receive better ratings of user experience than less popular ones. However, user experience does not predict sustained engagement with mental health apps. Thus, mental health app developers and evaluators need to better understand user experience and engagement, as well as to define sustained engagement, what leads to it, and how to create products that achieve it. This understanding might be supported by better collaboration between industry and academic teams to advance a science of engagement.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mobile app rating scale: a new tool for assessing the quality of health mobile apps
Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D., & Mani, M. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3422 · Cited by 2,233
Objective user engagement with mental health apps: systematic search and panel-based usage analysis
Baumel, A., Muench, F., Edan, S., & Kane, J. M. (2019) · Journal of Medical Internet Research · doi:10.2196/14567 · Cited by 965 · cited 3×
“Our study echoes previous research suggesting that retention is particularly challenging to understand [9].”
BackgroundResultKaryotaki, E., Efthimiou, O., Miguel, C., Maas genannt Bermpohl, F., Furukawa, T. A., Cuijpers, P., Individual Patient Data Meta-Analyses for Depression (IPDMA-DE) Collaboration, Riper, H., Patel, V., Mira, A., et al. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.4364 · Cited by 866
Linardon, J., Cuijpers, P., Carlbring, P., Messer, M., & Fuller‐Tyszkiewicz, M. (2019) · World Psychiatry · doi:10.1002/wps.20673 · Cited by 860
Review and evaluation of mindfulness-based iPhone apps
Mani, M., Kavanagh, D. J., Hides, L., & Stoyanov, S. R. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4328 · Cited by 495
User‐centered design for psychosocial intervention development and implementation.
Lyon, A. R., & Koerner, K. (2016) · Clinical Psychology: Science and Practice · doi:10.1111/cpsp.12154 · Cited by 478 · cited 1×
“While this work is in its early stages, some promising strategies include incorporating human-centered design principles [13], branding digital mental health interventions in ways that appeal to specific subgroups of users [14], sending reminders and “digital triggers” [15], and incorporating human support [16].”
BackgroundTorous, J., & Roberts, L. W. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.0262 · Cited by 362
Schleider, J., & Weisz, J. (2018) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12811 · Cited by 320 · cited 1×
“An example comes from the growing literature on digital single-session interventions, which are designed to produce benefits after just one sitting [41-43].”
BackgroundUser engagement in mental health apps: A review of measurement, reporting, and validity
Ng, M. M., Firth, J., Minen, M., & Torous, J. (2019) · Psychiatric Services · doi:10.1176/appi.ps.201800519 · Cited by 293
Insights from user reviews to improve mental health apps
Alqahtani, F., & Orji, R. (2020) · Health Informatics Journal · doi:10.1177/1460458219896492 · Cited by 286 · cited 2×
“Additionally, a study examining over 13,000 reviews of 106 mental health apps noted that user interface and user-friendliness were two of the most common aspects commented on by users and that poor usability was often noted as a reason for abandoning apps [23].”
BackgroundMohr, D. C., Schueller, S. M., Riley, W. T., Brown, C. H., Cuijpers, P., Duan, N., Kwasny, M. J., Stiles-Shields, C., & Cheung, K. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4391 · Cited by 216 · cited 1×
“For example, industry teams tend to create more popular and engaging apps than academic teams do; however, solutions to user engagement problems plaguing these apps might be best pursued by rigorous research combining quantitative, qualitative, and experimental approaches [45].”
BackgroundSelf-citationStawarz, K., Preist, C., Tallon, D., Wiles, N., & Coyle, D. (2018) · Journal of Medical Internet Research · doi:10.2196/10120 · Cited by 198
Schueller, S. M., Neary, M., O'Loughlin, K., & Adkins, E. C. (2018) · Journal of Medical Internet Research · doi:10.2196/10141 · Cited by 167
Self-citationSalazar, A., de Sola, H., Failde, I., & Moral-Munoz, J. A. (2018) · JMIR mHealth and uHealth · doi:10.2196/10718 · Cited by 154 · cited 1×
“It has been used in various evaluations of health apps, including mindfulness apps [20] and pain-management apps [21].”
MethodHermes, E. D., Lyon, A. R., Schueller, S. M., & Glass, J. E. (2019) · Journal of Medical Internet Research · doi:10.2196/11752 · Cited by 149 · cited 1×
“This is logically similar to efforts in implementation science frameworks to map key implementation outcomes including adoption, appropriateness, and sustainment, and consistent with work that has mapped those outcomes onto different variables relevant to technology-enabled services [36].”
ResultSelf-citationKelders, S. M., Kip, H., & Greeff, J. (2020) · Journal of Medical Internet Research · doi:10.2196/17757 · Cited by 88
Examining the Reach of Smartphone Apps for Depression and Anxiety
Wasil, A. R., Gillespie, S., Shingleton, R., Wilks, C. R., & Weisz, J. R. (2020) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2019.19090905 · Cited by 87 · cited 2×
“Most mental health apps have nearly no active users, while a few apps have millions of active users [5].”
BackgroundSelf-citationWasil, A. R., Gillespie, S., Patel, R., Petre, A., Venturo-Conerly, K. E., Shingleton, R. M., Weisz, J. R., & DeRubeis, R. J. (2020) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000604 · Cited by 86
Self-citationMental Health Apps in Psychiatric Treatment: A Patient Perspective on Real World Technology Usage
Chiauzzi, E., & Newell, A. (2019) · JMIR Mental Health · doi:10.2196/12292 · Cited by 46
Assessment of Real-World Use of Behavioral Health Mobile Applications by a Novel Stickiness Metric
Carlo, A. D., Hosseini Ghomi, R., Renn, B. N., Strong, M. A., & Areán, P. A. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.11978 · Cited by 31 · cited 3×
“To date, nearly all of the most engaging digital mental health interventions have been developed by industry teams, rather than academic teams [12,32].”
BackgroundWhat's in a name? Branding of online mental health programming for university students.
Rith-Najarian, L., Sun, W., Chen, A., Chorpita, B., Chavira, D., Mougalian, S., & Gong-Guy, E. (2019) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000383 · Cited by 17
Cited by
Potential and Pitfalls of Mobile Mental Health Apps in Traditional Treatment: An Umbrella Review
Koh, J., Tng, G. Y. Q., & Hartanto, A. (2022) · Journal of Personalized Medicine · doi:10.3390/jpm12091376 · Cited by 155
Smartphone apps for menstrual pain and symptom management: a scoping review
Trépanier, L. C. M., Lamoureux, É., Bjornson, S. E., Mackie, C., Alberts, N. M., & Gagnon, M. M. (2023) · Internet Interventions · doi:10.1016/j.invent.2023.100605 · Cited by 50
Eylem-van Bergeijk, O., Robinson, T., Manktelow, M., Olympios, M., Poulter, S., Mane, P., Panagioti, M., Condell, J., & Leavey, G. (2025) · JMIR mHealth and uHealth · doi:10.2196/70691 · Cited by 2
Metadata
- Authors
- Kaveladze, B. T., Wasil, A. R., Bunyi, J. B., Ramirez, V., & Schueller, S. M.
- Venue
- JMIR Human Factors
- Publisher
- JMIR Publications Inc.
- Published
- 2022-01-31
- Volume
- 9
- Issue
- 1
- Pages
- e30766
- ISSN
- 2292-9495 (ISSN-L)
- OpenAlex
- W3215375111
- Cited by
- 104
- References
- 21
- FWCI
- 11.32
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.