The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Using science to sell apps: Evaluation of mental health app store quality claims
Open accessAbstract
Despite the emergence of curated app libraries for mental health apps, personal searches by consumers remain a common method for discovering apps. App store descriptions therefore represent a key channel to inform consumer choice. This study examined the claims invoked through these app store descriptions, the extent to which scientific language is used to support such claims, and the corresponding evidence in the literature. Google Play and iTunes were searched for apps related to depression, self-harm, substance use, anxiety, and schizophrenia. The descriptions of the top-ranking, consumer-focused apps were coded to identify claims of acceptability and effectiveness, and forms of supporting statement. For apps which invoked ostensibly scientific principles, a literature search was conducted to assess their credibility. Seventy-three apps were coded, and the majority (64%) claimed effectiveness at diagnosing a mental health condition, or improving symptoms, mood or self-management. Scientific language was most frequently used to support these effectiveness claims (44%), although this included techniques not validated by literature searches (8/24 = 33%). Two apps described low-quality, primary evidence to support the use of the app. Only one app included a citation to published literature. A minority of apps (14%) described design or development involving lived experience, and none referenced certification or accreditation processes such as app libraries. Scientific language was the most frequently invoked form of support for use of mental health apps; however, high-quality evidence is not commonly described. Improved knowledge translation strategies may improve the adoption of other strategies, such as certification or lived experience co-design.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Other reviews have evaluated mental health apps more broadly, concluding that the quality of evidence supporting their efficacy is insufficient (Larsen et al. 2019; Wisniewski et al. 2019)”
From Evaluating the quality, safety, and functionality of commonly used smartphone apps for bipolar disorder mood and sleep self-management (2022) · cites this paper as Larsen et al. 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Whilst reviews of the broader BD app landscape report very low incidence of scientific evidence for available apps (Larsen et al. 2019), 55% of the apps most commonly used by individuals with BD have published evidence (albeit, with the limitations discussed above).”
From Evaluating the quality, safety, and functionality of commonly used smartphone apps for bipolar disorder mood and sleep self-management (2022) · cites this paper as Larsen et al. 2019
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The empirical status of cognitive-behavioral therapy: A review of meta-analyses
Butler, A. C., Chapman, J. E., Forman, E. M., & Beck, A. T. (2006) · Clinical Psychology Review · doi:10.1016/j.cpr.2005.07.003 · Cited by 3,215
Estimating the true global burden of mental illness
Vigo, D., Thornicroft, G., & Atun, R. (2016) · The Lancet Psychiatry · doi:10.1016/s2215-0366(15)00505-2 · Cited by 2,494
Plummer, F., Manea, L., Trepel, D., & McMillan, D. (2016) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2015.11.005 · Cited by 1,882
Gilbody, S., Richards, D., Brealey, S., & Hewitt, C. (2007) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0333-y · Cited by 1,290
Firth, J., Torous, J., Nicholas, J., Carney, R., Pratap, A., Rosenbaum, S., & Sarris, J. (2017) · World Psychiatry · doi:10.1002/wps.20472 · Cited by 1,097
Self-citationTorous, J., Nicholas, J., Larsen, M. E., Firth, J., & Christensen, H. (2018) · Evidence Based Mental Health · doi:10.1136/eb-2018-102891 · Cited by 790
Self-citationFirth, J., Torous, J., Nicholas, J., Carney, R., Rosenbaum, S., & Sarris, J. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.04.046 · Cited by 675
Self-citationMobile Apps for Bipolar Disorder: A Systematic Review of Features and Content Quality
Nicholas, J., Larsen, M. E., Proudfoot, J., & Christensen, H. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4581 · Cited by 438
Self-citationMohr, D. C., Lyon, A. R., Lattie, E. G., Reddy, M., & Schueller, S. M. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7725 · Cited by 432
A Systematic Review of Cognitive Behavioral Therapy and Behavioral Activation Apps for Depression
Huguet, A., Rao, S., McGrath, P. J., Wozney, L., Wheaton, M., Conrod, J., & Rozario, S. (2016) · PLOS ONE · doi:10.1371/journal.pone.0154248 · Cited by 328
Research works printed but not linked: 1
- OCEBM Levels of Evidence Working Group. The Oxford Levels of Evidence 2 https://www.cebm.net/index.aspx?o=5653 (2016).
Cited by
Sorted by citations
Items 301–333 of 333
Dietel, F. A., Pletzer-Zelgert, E., Portenhauser, A. A., Baumeister, H., Reininghaus, U., Pryss, R. C., Stach, M., Vocks, S., & Terhorst, Y. (2025) · doi:10.2196/preprints.82765
Löchner, J. (2025) · Kindheit und Entwicklung · doi:10.1026/0942-5403/a000487
Treatment of Obsessive-Compulsive Disorder via Mental Health Technology
Trent, E. S., Jagannathan, N., & Storch, E. A. (2025) · Psychiatric Clinics of North America · doi:10.1016/j.psc.2025.08.024
Foley, G., Cassiano, M. S., & Ricciardelli, R. (2025) · Digital Health · doi:10.1177/20552076251384142
(2024) · Emerald Group Publishing Limited eBooks · doi:10.1108/979-8-88730-594-320251004
Integrative approaches to specialized populations
Reddy, K. J. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-43908-7.00011-2
Log in, lie down: ethics and the digital turn in psychotherapy
Branford, J., & Conrad, J. A. (2025) · Philosophical Psychology · doi:10.1080/09515089.2025.2599894
Götzl, C., Hiller, S., Rauschenberg, C., Schick, A., Fechtelpeter, J., Fischer Abaigar, U., Koppe, G., Durstewitz, D., Reininghaus, U., & Krumm, S. (2022) · Figshare · doi:10.6084/m9.figshare.21582095
Götzl, C., Hiller, S., Rauschenberg, C., Schick, A., Fechtelpeter, J., Fischer Abaigar, U., Koppe, G., Durstewitz, D., Reininghaus, U., & Krumm, S. (2022) · Figshare · doi:10.6084/m9.figshare.21582092
Götzl, C., Hiller, S., Rauschenberg, C., Schick, A., Fechtelpeter, J., Fischer Abaigar, U., Koppe, G., Durstewitz, D., Reininghaus, U., & Krumm, S. (2022) · Figshare · doi:10.6084/m9.figshare.21582101
Götzl, C., Hiller, S., Rauschenberg, C., Schick, A., Fechtelpeter, J., Fischer Abaigar, U., Koppe, G., Durstewitz, D., Reininghaus, U., & Krumm, S. (2022) · Figshare · doi:10.6084/m9.figshare.21582089
Mayer, G., Lemmer, D., Michelsen, I., Schrader, P., Friederich, H.-C., & Bauer, S. (2024) · Figshare · doi:10.6084/m9.figshare.26684901
Götzl, C., Hiller, S., Rauschenberg, C., Schick, A., Fechtelpeter, J., Fischer Abaigar, U., Koppe, G., Durstewitz, D., Reininghaus, U., & Krumm, S. (2022) · Figshare · doi:10.6084/m9.figshare.21582098
Mayer, G., Lemmer, D., Michelsen, I., Schrader, P., Friederich, H.-C., & Bauer, S. (2024) · Figshare · doi:10.6084/m9.figshare.25282813
Götzl, C., Hiller, S., Rauschenberg, C., Schick, A., Fechtelpeter, J., Fischer Abaigar, U., Koppe, G., Durstewitz, D., Reininghaus, U., & Krumm, S. (2022) · Figshare · doi:10.6084/m9.figshare.21582104
Mayer, G., Lemmer, D., Michelsen, I., Schrader, P., Friederich, H.-C., & Bauer, S. (2024) · Figshare · doi:10.6084/m9.figshare.25282913
ATEŞ, E. G. Ç., Daşcı, E., Önal, A. A., & Güç, E. (2026) · Erzincan Üniversitesi Eğitim Fakültesi Dergisi · doi:10.17556/erziefd.1804448
Attoe, C., Szekely, R., Vega, M. O., Browne, E., Thomas, R., Kwan, C. W. T., Altchouler, P., De, A., Hariharan, N., Clay, G., et al. (2026) · Digital Health · doi:10.1177/20552076261428307
“59,60 A similar limitation applied to PsyberGuide, a mental health app rating platform developed by OneMind, 61 – 63 which is no longer operational.”
A Structured Decision-Making Framework for Clinicians Evaluating Mental Health Apps
Yu, R., & Haddock, A. D. (2026) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-026-00610-w
DeJong, S. M. (2026) · Springer eBooks · doi:10.1007/978-3-032-14062-3_14
Huberty, J., Baez, L. M., McAlister, K. L., & Kerppola, M. (2026) · JMIR Formative Research · doi:10.2196/85642
“…DMH technology emerged as a promising solution to the mental health treatment gap, most commercial DMH products have not been rigorously evaluated [3,4], and there is a concern that many DMH products may be ineffective or have iatrogenic effects, such as excessive self-monitoring, privacy…”
The Changing Concept in the History of Schizophrenia
Cavalli, E., Nicoletti, G. R. P., & Nicoletti, F. (2026) · Brain Sciences · doi:10.3390/brainsci16050447
“In addition, behavioral signals derived from smartphones or wearable devices may be influenced by contextual and environmental factors unrelated to illness progression [119].”
Technologies and Mental Health
Hon, C. K., Sunindijo, R. Y., Sun, C., & Frimpong, S. (2026) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-83662-690-920261011
Zárate, E., Velasco-Téllez, A. C., Sanchez-Reyes, L. A., Coll-Muñoz, A. M., Ramírez-Ramírez, M. Á., Tello, A. L., Toledo-Lozano, C. G., Moreno-Coutiño, A. B., Callejas-Gómez, M. D. P., Durón-Figueroa, R., et al. (2026) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2026.1824079 · Cited by 1
“Nevertheless, despite a proliferation of mental health apps, the vast majority lack rigorous clinical validation, regulatory oversight, or grounding in established therapeutic models (14, 15).”
Ng, A. T., & Koyfman, S. (2024) · American Psychiatric Association Publishing eBooks · doi:10.1176/appi.books.9780873182539.lg04
Dietel, F. A., Pletzer-Zelgert, E., Portenhauser, A. A., Baumeister, H., Reininghaus, U., Pryss, R. C., Stach, M., Vocks, S., & Terhorst, Y. (2025) · JMIR mhealth and uhealth · doi:10.2196/82765
Lipinska, G., Boland, J., Seef, S. E., & Rink, J. (2026) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2026.1651928
“%) associated with application use (33).”
Bamber, J. D. L., Ahlberg, S., & Cordier, R. (2026) · Global Implementation Research and Applications · doi:10.1007/s43477-026-00249-y
Whīria moe: a pilot trial of the Sleep Ninja CBT-I app for adolescents in Aotearoa, New Zealand
Ludin, N. M., Challis, H., Pawley, M. M. D. M., Fortune, S., Warman, G. R., Galland, B. C., Werner‐Seidler, A., Hetrick, S. E., & Cargo, T. (2026) · Australasian Psychiatry · doi:10.1177/10398562261491672
Sağlık Hizmetlerinde Akıllı Kalite ve Akreditasyon: Karma Yöntemli Bir Çalışma
Kocaman, E., & Terlemez, B. (2026) · Mehmet Akif Ersoy Üniversitesi Uygulamalı Bilimler Dergisi · doi:10.31200/makuubd.2029510
Schwaninger, I., Ighanian, P., Borga, L. G., Giraitis, M., Hoogendoorn, P., & Klucken, J. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001730
“Users such as patients can retrieve unstructured, limited and biased information on the quality of apps via app descriptions (metadata provided by manufacturers) and user feedback (provided by user ratings and reviews) [20,21].”
Chauhan, P., Gupta, S., Gupta, H., Singh, P., & Deb, K. S. (2026) · doi:10.2196/preprints.113988
Metadata
- Authors
- Larsen, M. E., Huckvale, K., Nicholas, J., Torous, J., Birrell, L., Li, E., & Reda, B.
- Affiliations
- Black Dog Institute (AU)
- Northwestern University (US)
- Beth Israel Deaconess Medical Center (US)
- UNSW Sydney (AU)
- Corresponding author
- Larsen, Mark Erik
- Venue
- npj Digital Medicine
- Published
- 2019-03-22
- Volume
- 2
- Issue
- 1
- Pages
- 18-18
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC6550255
- OpenAlex
- W2925166053
- Cited by
- 334
- References
- 23
- Citations by year
- 2026: 16 · 2025: 53 · 2024: 42 · 2023: 48 · 2022: 47 · 2021: 60 · 2020: 54 · 2019: 14
- FWCI
- 49.43
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- npj Digital Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Health Sciences › Health Professions › Health Information Management › Health, Technology, Consumer Behavior
- Keywords
- mental health apps, scientific language, quality of evidence, consumer choice, knowledge translation, depression, anxiety, schizophrenia, substance use, self-harm, Google Play Store, iTunes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
