Estimating the real‐world usage of mobile apps for mental health: development and application of two novel metrics
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Mobile apps for health and wellness (mHealth apps) have the potential to expand access to information and support, especially for people who are unable to access face-to-face care. The role of these apps is becoming especially salient during the ongoing COVID-19 crisis. According to recent estimates, there are over 325,000 mHealth apps, with 78,000 added in 2017 alone1. Estimating the effectiveness of these apps has become a topic of great interest2, 3. Few studies have examined the extent to which mHealth apps reach real-world users. Some recent work suggests that mHealth app marketplaces may be highly skewed, with a small number of apps attracting a large share of users4. To rigorously test this assumption, an empirical characterization of asymmetry in mHealth markets is needed. Existing approaches, while useful, often lack an intuitive or easily interpretable meaning5. This presents a challenge that is especially important in the context of mental health research, which requires communication between experts from a variety of disciplines (e.g., psychiatry, clinical psychology, digital health, economics, public policy). We examined the dissemination of mHealth apps for a variety of mental health conditions, searching the Apple App Store and Google Play Store in March 2020. We applied the following search terms: “addiction”, “anxiety”, “depression”, “eating disorders”, “fitness”, “mood tracking”, “schizophrenia”, and “sleep”. Consistent with previous work, apps within the top 50 search hits on either app store were screened4. We included apps designed to offer treatment, support or information. We collected monthly active user (MAU) data from Mobile Action, a mobile app market research firm, for a one-month period from March 14, 2020 to April 13, 2020. Total MAUs per category ranged from 264,763 (addiction) to 47,133,801 (fitness), with a median of 6,254,650. We then developed two novel metrics to characterize the mHealth app marketspaces: the market share index-n (MSI-n) and the number needed to reach-p (NNR-p). The MSI-n refers to the percentage of MAUs in a category (e.g., depression apps) that is accounted for by the top n apps. For example, “MSI-3” refers to the percentage of MAUs that is accounted for by the three most popular apps. Higher MSI values indicate that the top apps are responsible for a greater proportion of active users. The NNR-p refers to the minimum number of apps that are needed to account for p percentage of active users. For example, “NNR-90” refers to the number of apps that are required to account for 90% of MAUs in a category. Lower NNR values indicate that the top apps are responsible for a greater proportion of active users. For each of the above-mentioned categories, we calculated the MSI-3, MSI-10 and NNR-90. In six of the eight categories, the top three apps were responsible for more than 50% of MAUs. The MSI-3 values were 41.5% for fitness (indicating that the top three apps accounted for 41.5% of MAUs in the fitness category), 45.6% for addiction, 66.2% for depression, 66.4% for sleep, 75.7% for anxiety, 79.2% for mood tracking, 88.9% for eating disorders, and 98.1% for schizophrenia, with a median MSI-3 value of 71.1%. The median MSI-10 value was 91.4% (ranging from 67.6% for fitness to 99.97% for schizophrenia). The NNR-90 values were 2 for schizophrenia (indicating that the top two schizophrenia apps accounted for 90% of MAUs), 4 for eating disorders, 6 for mood tracking, 7 for anxiety, 11 for depression, 12 for addiction, 16 for sleep, and 25 for fitness. The median NNR-90 value was 9. Thus, app marketplaces for mental illnesses (e.g., schizophrenia, eating disorders) were more asymmetrical than those for overall health and wellness (e.g., fitness, sleep). These findings have important implications for the study and evaluation of mHealth apps. To better characterize the content that real-world users encounter through these apps, we recommend that usage data be incorporated to adjust the findings of mHealth app reviews6. Additionally, there are over 45 app evaluation frameworks, and there has been enormous interest in developing tools that help consumers sift through crowded app marketplaces7, 8. However, the reliability and validity of such tools has been criticized, as many of them yield different and sometimes conflicting conclusions8. Due to overwhelming volume of mHealth apps and app evaluation methods, it is not surprising that such issues arise: investigators commonly evaluate hundreds or thousands of apps, a labor-intensive process that can yield cursory or unreliable evaluations. Given the skewness of mHealth app marketplaces, consumers may benefit more from highly detailed and reliable evaluations of a much smaller number of apps – those that they are most likely to encounter and use9. The exact number of popular apps may vary by mHealth category. To account for this, investigators could apply the MSI-n and NNR-p metrics. For example, using the NNR-p metric,
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Towards a consensus around standards for smartphone apps and digital mental health
Torous, J., Andersson, G., Bertagnoli, A., Christensen, H., Cuijpers, P., Firth, J., Haim, A., Hsin, H., Hollis, C., Lewis, S., et al. (2019) · World Psychiatry · doi:10.1002/wps.20592 · Cited by 325
By the numbers: ratings and utilization of behavioral health mobile applications
Carlo, A. D., Hosseini Ghomi, R., Renn, B. N., & Areán, P. A. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0129-6 · Cited by 164
Wasil, A. R., Venturo-Conerly, K. E., Shingleton, R. M., & Weisz, J. R. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103498 · Cited by 160
Self-citationExamining the Reach of Smartphone Apps for Depression and Anxiety
Wasil, A. R., Gillespie, S., Shingleton, R., Wilks, C. R., & Weisz, J. R. (2020) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2019.19090905 · Cited by 87
Self-citationWasil, A. R., Gillespie, S., Patel, R., Petre, A., Venturo-Conerly, K. E., Shingleton, R. M., Weisz, J. R., & DeRubeis, R. J. (2020) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000604 · Cited by 86
Self-citation
Cited by
Lagan, S., D'Mello, R., Vaidyam, A., Bilden, R., & Torous, J. (2021) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.13306 · Cited by 65
“personal health information? 108 (39) 24 (29) 9 (11) 75 (44)”
Volpe, U., Orsolini, L., Salvi, V., Albert, U., Carmassi, C., Carrà, G., Cirulli, F., Dell’Osso, B., Luciano, M., Menculini, G., et al. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19031539 · Cited by 57
Sobowale, K., & Humphrey, D. K. (2025) · JMIR Formative Research · doi:10.2196/65605 · Cited by 15
Metadata
- Authors
- Wasil, A. R., Gillespie, S., Schell, T., Lorenzo‐Luaces, L., & DeRubeis, R. J.
- Venue
- World Psychiatry
- Publisher
- Wiley
- Published
- 2021-01-12
- Volume
- 20
- Issue
- 1
- Pages
- 137-138
- ISSN
- 1723-8617 (ISSN-L)
- 2051-5545
- OpenAlex
- W3119669232
- Cited by
- 38
- References
- 5
- FWCI
- 6.18
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Estimating the real‐world usage of mobile apps for mental health: development and application of two novel metrics. https://app.cochat.ai/papers/10.1002/wps.20827
- The paper
- Wasil, A. R., Gillespie, S., Schell, T., Lorenzo‐Luaces, L., & DeRubeis, R. J. (2021). Estimating the real‐world usage of mobile apps for mental health: development and application of two novel metrics. World Psychiatry. https://doi.org/10.1002/wps.20827
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.