Reassessing evidence-based content in popular smartphone apps for depression and anxiety: Developing and applying user-adjusted analyses.
Open accessThe DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Abstract
Objective: To assess the dissemination of evidence-based content within smartphone apps for depression and anxiety by developing and applying user-adjusted analysis—a method for weighting app content based on each app’s number of active users. Method: We searched the Apple App Store and Google Play Store and identified 27 apps within the top search hits, which real-world users are most likely to encounter. We developed a codebook of evidence-based treatment elements by reviewing past research on empirically supported treatments. We coded the apps to develop an initial tally of the frequency of treatment elements within the mental health (MH) apps. We then developed and applied user-adjusted analysis to refine the tallies based on each app’s number of monthly active users. Results: The 2 most popular apps were responsible for 90% of monthly active users, and user-adjusted analysis markedly altered conclusions of prior reports based on tallies alone. For example, mindfulness was present in 37% of apps but reached 96% of monthly active users, cognitive restructuring was present in 22% but reached only 2%, and exposure was present in 7% but reached only 0.0004%. Conclusions: The potential impact of MH apps on mental health may be best evaluated via assessment that combines tallies of evidence-based content with data on the content users are actually accessing. Given wide variation in the popularity of MH apps, findings weighted by usage data may differ markedly from findings based on raw tallies alone. (PsycInfo Database Record (c) 2020 APA, all rights reserved)What is the public health significance of this article?: Traditionally, analyses of mobile health and mental health applications have not accounted for the fact that apps differ markedly in their dissemination. In this study, we developed and applied a new method—user adjusted analysis—to better understand the content that users are receiving from mobile apps for depression and anxiety. From a public health perspective, this study (a) enhances our understanding of mobile apps for depression and anxiety and (b) presents a new way of estimating the impact of mobile apps. (PsycInfo Database Record (c) 2020 APA, all rights reserved) © 2020 American Psychological Association
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Meditation practice and meditation-based psychotherapies have become increasingly popular.”
From Prevalence of meditation-related adverse effects in a population-based sample in the United States (2022) · cites this paper as Wasil et al., 2020
In the citing paper
“Meditation apps are widely used (Wasil et al., 2020) and the predominantly unguided format may contribute to the frequency of MRAE (given these experiences are more likely to occur when practicing alone; Schlosser et al., 2019).”
From Prevalence of meditation-related adverse effects in a population-based sample in the United States (2022) · cites this paper as Wasil et al., 2020
In the citing paper
“Across users of all 27 apps, mindfulness content reaches 96% of monthly active users while cognitive restructuring and exposure (core elements of CBT) reach only 2% and 0.0004%, respectively.”
From A common factors perspective on mindfulness-based interventions (2022) · cites this paper as [2]
In the citing paper
“The prominence of mindfulness meditation within mental health apps2 suggests that MBIs are already at the forefront of unguided mHealth interventions (those delivered without live instructor guidance).”
From A common factors perspective on mindfulness-based interventions (2022) · cites this paper as [2]
In the citing paper
“Among the 27 most popular mental health apps, the two most popular (Headspace and Calm), which account for 90% of monthly active users, have meditation as their primary content2.”
From A common factors perspective on mindfulness-based interventions (2022) · cites this paper as [2]
Citations
Cites
Gulliver, A., Griffiths, K. M., & Christensen, H. (2010) · BMC Psychiatry · doi:10.1186/1471-244x-10-113 · Cited by 3,292
Defining empirically supported therapies.
Chambless, D. L., & Hollon, S. D. (1998) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.66.1.7 · Cited by 3,067
Mobile app rating scale: a new tool for assessing the quality of health mobile apps
Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D., & Mani, M. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3422 · Cited by 2,233
Empirically supported psychological interventions: Controversies and evidence
Chambless, D. L., & Ollendick, T. H. (2001) · Annual Review of Psychology · doi:10.1146/annurev.psych.52.1.685 · Cited by 2,097
Andrews, G., Cuijpers, P., Craske, M. G., McEvoy, P., & Titov, N. (2010) · PLoS ONE · doi:10.1371/journal.pone.0013196 · Cited by 1,311
Barriers to mental health treatment: results from the WHO World Mental Health surveys
Andrade, L. H., Alonso, J., Mneimneh, Z., Wells, J. E., Al-Hamzawi, A., Borges, G., Bromet, E., Bruffaerts, R., de Girolamo, G., de Graaf, R., et al. (2014) · Psychological Medicine · doi:10.1017/s0033291713001943 · Cited by 1,289
Rebooting psychotherapy research and practice to reduce the burden of mental illness
Kazdin, A. E., & Blase, S. L. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393527 · Cited by 1,254
Smartphones for smarter delivery of mental health programs: A systematic review
Donker, T., Petrie, K., Proudfoot, J., Clarke, J., Birch, M.-R., & Christensen, H. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2791 · Cited by 1,136
Firth, J., Torous, J., Nicholas, J., Carney, R., Pratap, A., Rosenbaum, S., & Sarris, J. (2017) · World Psychiatry · doi:10.1002/wps.20472 · Cited by 1,097
Cuijpers, P., Berking, M., Andersson, G., Quigley, L., Kleiboer, A., & Dobson, K. S. (2013) · The Canadian Journal of Psychiatry · doi:10.1177/070674371305800702 · Cited by 1,066
Psychotherapy for depression in adults: A meta-analysis of comparative outcome studies.
Cuijpers, P., van Straten, A., Andersson, G., & van Oppen, P. (2008) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0013075 · Cited by 1,050
Objective user engagement with mental health apps: systematic search and panel-based usage analysis
Baumel, A., Muench, F., Edan, S., & Kane, J. M. (2019) · Journal of Medical Internet Research · doi:10.2196/14567 · Cited by 965
Mental Health Smartphone Apps: Review and Evidence-Based Recommendations for Future Developments
Bakker, D., Kazantzis, N., Rickwood, D., & Rickard, N. (2016) · JMIR Mental Health · doi:10.2196/mental.4984 · Cited by 886
Linardon, J., Cuijpers, P., Carlbring, P., Messer, M., & Fuller‐Tyszkiewicz, M. (2019) · World Psychiatry · doi:10.1002/wps.20673 · Cited by 860
Chorpita, B. F., Daleiden, E. L., & Weisz, J. R. (2005) · Mental Health Services Research · doi:10.1007/s11020-005-1962-6 · Cited by 717
Self-citationChorpita, B. F., & Daleiden, E. L. (2009) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0014565 · Cited by 687
Firth, J., Torous, J., Nicholas, J., Carney, R., Rosenbaum, S., & Sarris, J. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.04.046 · Cited by 675
The Winner-Take-All Society: Why the Few at the Top Get So Much More Than the Rest of Us
Frank, R. H., & Cook, P. J. (2010) · Medical Entomology and Zoology · Cited by 615
Serious Games and Gamification for Mental Health: Current Status and Promising Directions
Fleming, T. M., Bavin, L., Stasiak, K., Hermansson-Webb, E., Merry, S. N., Cheek, C., Lucassen, M., Lau, H. M., Pollmuller, B., & Hetrick, S. (2017) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2016.00215 · Cited by 582
Evidence Base Update: 50 Years of Research on Treatment for Child and Adolescent Anxiety
Higa-McMillan, C. K., Francis, S. E., Rith-Najarian, L., & Chorpita, B. F. (2016) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2015.1046177 · Cited by 557
Kazdin, A. E. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.06.004 · Cited by 554
Ebert, D. D., Zarski, A.-C., Christensen, H., Stikkelbroek, Y., Cuijpers, P., Berking, M., & Riper, H. (2015) · PLOS ONE · doi:10.1371/journal.pone.0119895 · Cited by 533
Review and evaluation of mindfulness-based iPhone apps
Mani, M., Kavanagh, D. J., Hides, L., & Stoyanov, S. R. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4328 · Cited by 495
The impact of digital technology on psychological treatments and their dissemination
Fairburn, C. G., & Patel, V. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.08.012 · Cited by 448
Standalone smartphone apps for mental health—a systematic review and meta-analysis
Weisel, K. K., Fuhrmann, L. M., Berking, M., Baumeister, H., Cuijpers, P., & Ebert, D. D. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0188-8 · Cited by 447
Prolonged exposure therapy: past, present, and future
Foa, E. B. (2011) · Depression and Anxiety · doi:10.1002/da.20907 · Cited by 358
Josephine, K., Josefine, L., Philipp, D., David, E., & Harald, B. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.07.021 · Cited by 341
A Systematic Review of Cognitive Behavioral Therapy and Behavioral Activation Apps for Depression
Huguet, A., Rao, S., McGrath, P. J., Wozney, L., Wheaton, M., Conrod, J., & Rozario, S. (2016) · PLOS ONE · doi:10.1371/journal.pone.0154248 · Cited by 328
User engagement in mental health apps: A review of measurement, reporting, and validity
Ng, M. M., Firth, J., Minen, M., & Torous, J. (2019) · Psychiatric Services · doi:10.1176/appi.ps.201800519 · Cited by 293
Finding a Depression App: A Review and Content Analysis of the Depression App Marketplace
Shen, N., Levitan, M.-J., Johnson, A., Bender, J. L., Hamilton-Page, M., Jadad, A. (. R., & Wiljer, D. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3713 · Cited by 282
Smartphone Apps for Schizophrenia: A Systematic Review
Firth, J., & Torous, J. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4930 · Cited by 268
State of the Field of Mental Health Apps
Neary, M., & Schueller, S. M. (2018) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2018.01.002 · Cited by 247
Weisz, J. R., Ng, M. Y., & Bearman, S. K. (2014) · Clinical Psychological Science · doi:10.1177/2167702613501307 · Cited by 246
Self-citationVan Ameringen, M., Turna, J., Khalesi, Z., Pullia, K., & Patterson, B. (2017) · Depression and Anxiety · doi:10.1002/da.22657 · Cited by 234
Maximizing the Impact of e-Therapy and Serious Gaming: Time for a Paradigm Shift
Fleming, T. M., de Beurs, D., Khazaal, Y., Gaggioli, A., Riva, G., Botella, C., Baños, R. M., Aschieri, F., Bavin, L. M., Kleiboer, A., et al. (2016) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2016.00065 · Cited by 212
A Solution-Focused Research Approach to Achieve an Implementable Revolution in Digital Mental Health
Mohr, D. C., Riper, H., & Schueller, S. M. (2018) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.3838 · Cited by 204
Deriving a practical framework for the evaluation of health apps
Henson, P., David, G., Albright, K., & Torous, J. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30013-5 · Cited by 182
Lindsey, M. A., Brandt, N. E., Becker, K. D., Lee, B. R., Barth, R. P., Daleiden, E. L., & Chorpita, B. F. (2014) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-013-0163-x · Cited by 180
Implementation of Behavior Change Techniques in Mobile Applications for Physical Activity
Yang, C.-H., Maher, J. P., & Conroy, D. E. (2015) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2014.10.010 · Cited by 171
A Systematic, Multi-domain Review of Mobile Smartphone Apps for Evidence-Based Stress Management
Coulon, S. M., Monroe, C. M., & West, D. S. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.01.026 · Cited by 161
Wasil, A. R., Venturo-Conerly, K. E., Shingleton, R. M., & Weisz, J. R. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103498 · Cited by 160
Self-citationKaplan, R. M., & Stone, A. A. (2013) · Annual Review of Psychology · doi:10.1146/annurev-psych-113011-143736 · Cited by 144
The Guide to Interpersonal Psychotherapy
Weissman, M. M., Markowitz, J. C., & Klerman, G. L. (2017) · Oxford Clinical Psychology · doi:10.1093/med-psych/9780190662592.001.0001 · Cited by 141
Bry, L. J., Chou, T., Miguel, E., & Comer, J. S. (2018) · Behavior Therapy · doi:10.1016/j.beth.2017.07.008 · Cited by 106
Dogruel, L., Joeckel, S., & Bowman, N. D. (2015) · Mobile Media & Communication · doi:10.1177/2050157914557509 · Cited by 72
Craske, M. G. (2013) · American Psychological Association eBooks · doi:10.1037/14295-009 · Cited by 62
Evolution of cognitive-behavioral therapy for eating disorders
Agras, W. S., Fitzsimmons-Craft, E. E., & Wilfley, D. E. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.09.004 · Cited by 59
Normal Versus Pathological Mood: Implications for Diagnosis.
Ruscio, A. M. (2019) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050718-095644 · Cited by 56
Psychotherapy for Depression in Adults: A Meta-Analysis of Comparative Outcome Studies
Cuijpers, P., van Straten, A., Andersson, G., & van Oppen, P. (2010) · Focus · doi:10.1176/foc.8.1.foc75 · Cited by 47
Grant, D. M., & Wingate, L. R. (2011) · Elsevier eBooks · doi:10.1016/b978-0-12-385146-8.00011-0 · Cited by 40
Cognitive-behavioral interventions—Theory, research and procedures
Eysenck, H. J. (1981) · Behaviour Research and Therapy · doi:10.1016/0005-7967(81)90019-x · Cited by 28
Three questions to consider before developing a mental health app
Wasil, A. R., Weisz, J. R., & DeRubeis, R. J. (2020) · World Psychiatry · doi:10.1002/wps.20757 · Cited by 24
Self-citationWasil, A. R., Venturo‐Conerly, K. E., Shingleton, R. M., et al. (2019) · doi:10.31234/osf.io/zp362 · Cited by 11
Self-citation
Cited by
Torous, J., Linardon, J., Goldberg, S. B., Sun, S., Bell, I., Nicholas, J., Hassan, L., Hua, Y., Milton, A., & Firth, J. (2025) · World Psychiatry · doi:10.1002/wps.21299 · Cited by 323
Goldberg, S. B., Lam, S. U., Simonsson, O., Torous, J., & Sun, S. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000002 · Cited by 237
The Impact of the COVID-19 Pandemic on Mental Disorders. A Critical Review
Clemente-Suárez, V. J., Martínez-González, M. B., Benitez-Agudelo, J. C., Navarro-Jiménez, E., Beltran-Velasco, A. I., Ruisoto, P., Diaz Arroyo, E., Laborde-Cárdenas, C. C., & Tornero-Aguilera, J. F. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph181910041 · Cited by 161
Providing Human Support for the Use of Digital Mental Health Interventions: Systematic Meta-review
Werntz, A., Amado, S., Jasman, M., Ervin, A., & Rhodes, J. E. (2023) · Journal of Medical Internet Research · doi:10.2196/42864 · Cited by 158
Prevalence of meditation-related adverse effects in a population-based sample in the United States
Goldberg, S. B., Lam, S. U., Britton, W. B., & Davidson, R. J. (2022) · Psychotherapy Research · doi:10.1080/10503307.2021.1933646 · Cited by 115
Metadata
- Authors
- Wasil, A. R., Gillespie, S., Patel, R., Petre, A., Venturo-Conerly, K. E., Shingleton, R. M., Weisz, J. R., & DeRubeis, R. J.
- ORCID
- Venue
- Journal of Consulting and Clinical Psychology
- Published
- 2020-11
- Volume
- 88
- Issue
- 11
- Pages
- 983-993
- ISSN
- 0022-006X (ISSN-L)
- 1939-2117
- OpenAlex
- W3083378332
- Cited by
- 86
- References
- 49
- Citations by year
- 2026: 8 · 2025: 14 · 2024: 12 · 2023: 17 · 2022: 15 · 2021: 18 · 2020: 1
- FWCI
- 11.26
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- cc-by
- Locations
- Journal of Consulting and Clinical Psychology (publishedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- PubMed (publishedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Keywords
- mental health apps, mobile application, depression, anxiety, app popularity, mindfulness, cognitive restructuring, exposure therapy, empirically supported treatments
- MeSH terms
- Smartphone, Correlation of Data, Anxiety Disorders, Depressive Disorder, Humans, Mental Health, Sleep Deprivation, Telemedicine, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.