The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Using science to sell apps: Evaluation of mental health app store quality claims
Open accessAbstract
Despite the emergence of curated app libraries for mental health apps, personal searches by consumers remain a common method for discovering apps. App store descriptions therefore represent a key channel to inform consumer choice. This study examined the claims invoked through these app store descriptions, the extent to which scientific language is used to support such claims, and the corresponding evidence in the literature. Google Play and iTunes were searched for apps related to depression, self-harm, substance use, anxiety, and schizophrenia. The descriptions of the top-ranking, consumer-focused apps were coded to identify claims of acceptability and effectiveness, and forms of supporting statement. For apps which invoked ostensibly scientific principles, a literature search was conducted to assess their credibility. Seventy-three apps were coded, and the majority (64%) claimed effectiveness at diagnosing a mental health condition, or improving symptoms, mood or self-management. Scientific language was most frequently used to support these effectiveness claims (44%), although this included techniques not validated by literature searches (8/24 = 33%). Two apps described low-quality, primary evidence to support the use of the app. Only one app included a citation to published literature. A minority of apps (14%) described design or development involving lived experience, and none referenced certification or accreditation processes such as app libraries. Scientific language was the most frequently invoked form of support for use of mental health apps; however, high-quality evidence is not commonly described. Improved knowledge translation strategies may improve the adoption of other strategies, such as certification or lived experience co-design.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Other reviews have evaluated mental health apps more broadly, concluding that the quality of evidence supporting their efficacy is insufficient (Larsen et al. 2019; Wisniewski et al. 2019)”
From Evaluating the quality, safety, and functionality of commonly used smartphone apps for bipolar disorder mood and sleep self-management (2022) · cites this paper as Larsen et al. 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Whilst reviews of the broader BD app landscape report very low incidence of scientific evidence for available apps (Larsen et al. 2019), 55% of the apps most commonly used by individuals with BD have published evidence (albeit, with the limitations discussed above).”
From Evaluating the quality, safety, and functionality of commonly used smartphone apps for bipolar disorder mood and sleep self-management (2022) · cites this paper as Larsen et al. 2019
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The empirical status of cognitive-behavioral therapy: A review of meta-analyses
Butler, A. C., Chapman, J. E., Forman, E. M., & Beck, A. T. (2006) · Clinical Psychology Review · doi:10.1016/j.cpr.2005.07.003 · Cited by 3,215
Estimating the true global burden of mental illness
Vigo, D., Thornicroft, G., & Atun, R. (2016) · The Lancet Psychiatry · doi:10.1016/s2215-0366(15)00505-2 · Cited by 2,494
Plummer, F., Manea, L., Trepel, D., & McMillan, D. (2016) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2015.11.005 · Cited by 1,882
Gilbody, S., Richards, D., Brealey, S., & Hewitt, C. (2007) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0333-y · Cited by 1,290
Firth, J., Torous, J., Nicholas, J., Carney, R., Pratap, A., Rosenbaum, S., & Sarris, J. (2017) · World Psychiatry · doi:10.1002/wps.20472 · Cited by 1,097
Self-citationTorous, J., Nicholas, J., Larsen, M. E., Firth, J., & Christensen, H. (2018) · Evidence Based Mental Health · doi:10.1136/eb-2018-102891 · Cited by 790
Self-citationFirth, J., Torous, J., Nicholas, J., Carney, R., Rosenbaum, S., & Sarris, J. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.04.046 · Cited by 675
Self-citationMobile Apps for Bipolar Disorder: A Systematic Review of Features and Content Quality
Nicholas, J., Larsen, M. E., Proudfoot, J., & Christensen, H. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4581 · Cited by 438
Self-citationMohr, D. C., Lyon, A. R., Lattie, E. G., Reddy, M., & Schueller, S. M. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7725 · Cited by 432
A Systematic Review of Cognitive Behavioral Therapy and Behavioral Activation Apps for Depression
Huguet, A., Rao, S., McGrath, P. J., Wozney, L., Wheaton, M., Conrod, J., & Rozario, S. (2016) · PLOS ONE · doi:10.1371/journal.pone.0154248 · Cited by 328
Research works printed but not linked: 1
- OCEBM Levels of Evidence Working Group. The Oxford Levels of Evidence 2 https://www.cebm.net/index.aspx?o=5653 (2016).
Cited by
Sorted by citations
Torous, J., Bucci, S., Bell, I. H., Kessing, L. V., Faurholt‐Jepsen, M., Whelan, P., Carvalho, A. F., Keshavan, M., Linardon, J., & Firth, J. (2021) · World Psychiatry · doi:10.1002/wps.20883 · Cited by 1,199
Self-citationTorous, J., Jän Myrick, K., Rauseo-Ricupero, N., & Firth, J. (2020) · JMIR Mental Health · doi:10.2196/18848 · Cited by 946
“Picking from those different apps is challenging as many offer little protection of user data, make exaggerated claims, may be ineffective, and often are quickly abandoned because of usability issues [18,23,24].”
BackgroundSelf-citationDigital interventions for the treatment of depression: a meta-analytic review
Moshe, I., Terhorst, Y., Philippi, P., Domhardt, M., Cuijpers, P., Cristea, I., Pulkki-Råback, L., Baumeister, H., & Sander, L. B. (2021) · Psychological Bulletin · doi:10.1037/bul0000334 · Cited by 390
Stein, D. J., Shoptaw, S. J., Vigo, D. V., Lund, C., Cuijpers, P., Bantjes, J., Sartorius, N., & Maj, M. (2022) · World Psychiatry · doi:10.1002/wps.20998 · Cited by 331
Beyond validation: getting health apps into clinical practice
Gordon, W. J., Landman, A., Zhang, H., & Bates, D. W. (2020) · npj Digital Medicine · doi:10.1038/s41746-019-0212-z · Cited by 322
Virtual reality as a clinical tool in mental health research and practice
Bell, I. H., Nicholas, J., Alvarez-Jimenez, M., Thompson, A., & Valmaggia, L. (2020) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2020.22.2/lvalmaggia · Cited by 302
Self-citationResponsible AI—two frameworks for ethical design practice
Peters, D., Vold, K., Robinson, D., & Calvo, R. A. (2020) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2020.2974991 · Cited by 293
“A number of authors within human–computer interaction have articulated some of the ethically loaded socio-technical challenges facing digital mental health developers [28]–[31], [43], [44].”
BackgroundThe Need for a Mental Health Technology Revolution in the COVID-19 Pandemic
Figueroa, C. A., & Aguilera, A. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00523 · Cited by 227
Linardon, J., Torous, J. B., Firth, J., Cuijpers, P. J., Messer, M., & Fuller‐Tyszkiewicz, M. (2024) · World Psychiatry · doi:10.1002/wps.21183 · Cited by 260
“…with depression and anxiety apps, such as privacy violations; possible easy access to ineffective, inaccurate or potentially harmful content 15,16 ; low rates of engagement 17,18 , and exclusion of the potential therapeutic ingredient represented by the personal relationship between a…”
BackgroundSelf-citationCarlo, F. D., Sociali, A., Picutti, E., Pettorruso, M., Vellante, F., Verrastro, V., Martinotti, G., & Giannantonio, M. D. (2020) · International Journal of Clinical Practice · doi:10.1111/ijcp.13716 · Cited by 225
“Through a search on dedicated platforms conducted in 2019, 1435 mobile apps were identified, 449 of which addressing anxiety problems, 450 addressing Depression, 282 addressing Schizophrenia, 124 addressing self-harm and 140 addressing Substance Use Disorders [37].”
Rauschenberg, C., Schick, A., Hirjak, D., Seidler, A., Paetzold, I., Apfelbacher, C., Riedel-Heller, S. G., & Reininghaus, U. (2021) · Journal of Medical Internet Research · doi:10.2196/23365 · Cited by 205
“There are a number of reviews that have concluded that mHealth apps have problematic data-sharing and privacy practices [8,27,28] and that there may not only be a lack of quality of offered content but even harmful intervention components.”
BackgroundInfluentialFiol-deRoque, M. A., Serrano-Ripoll, M. J., Jiménez, R., Zamanillo‐Campos, R., Yáñez, A. M., Bennasar‐Veny, M., Leiva, A., Gervilla, E., Buades, E. G., Garcı́a-Toro, M., et al. (2021) · JMIR mhealth and uhealth · doi:10.2196/27039 · Cited by 203
Bauer, M., Glenn, T., Geddes, J., Gitlin, M., Grof, P., Kessing, L. V., Monteith, S., Faurholt-Jepsen, M., Severus, E., & Whybrow, P. C. (2020) · International Journal of Bipolar Disorders · doi:10.1186/s40345-019-0164-x · Cited by 163
Scaling evidence-based treatments through digital mental health.
Schueller, S. M., & Torous, J. B. (2020) · American Psychologist · doi:10.1037/amp0000654 · Cited by 191
Self-citationRauschenberg, C., Schick, A., Goetzl, C., Roehr, S., Riedel-Heller, S. G., Koppe, G., Durstewitz, D., Krumm, S., & Reininghaus, U. (2021) · European Psychiatry · doi:10.1192/j.eurpsy.2021.17 · Cited by 158
Digital interventions in mental health: An overview and future perspectives
Löchner, J., Carlbring, P., Schuller, B. W., Torous, J. B., & Sander, L. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100824 · Cited by 182
“It becomes problematic when many private companies are less than transparent and methodologically thorough in their processing of personal data, e.g., for commercial and unintended purposes (Larsen et al., 2019).”
Self-citationActionable health app evaluation: translating expert frameworks into objective metrics
Lagan, S., Aquino, P. R., Emerson, M. R., Fortuna, K. L., Walker, R., & Torous, J. B. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00312-4 · Cited by 146
Self-citationCarter, H., Araya, R., Anjur, K. I., Deng, D., & Naslund, J. A. (2020) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2020.12.016 · Cited by 164
“For instance, over 10,000 mental health apps are available for download via commercial app stores (Torous et al., 2019); though, most of these commercially available apps have no reliable or high-quality scientific evidence as support (Larsen et al., 2019; Lau et al., 2020).”
BackgroundAdvances in the use of virtual reality to treat mental health conditions
Bell, I. H., Pot-Kolder, R., Rizzo, A., Rus-Calafell, M., Cardi, V., Cella, M., Ward, T., Riches, S., Reinoso, M., Thompson, A., et al. (2024) · Nature Reviews Psychology · doi:10.1038/s44159-024-00334-9 · Cited by 140
“With the proliferation of digital health products available directly to consumers, many of which make strong claims without sufficient scientific evidence 148 , there is a need to address growing concerns about ethics and regulation in the digital mental health field.”
Smartphone apps for the treatment of mental health conditions: status and considerations
Huckvale, K., Nicholas, J., Torous, J. B., & Larsen, M. E. (2020) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2020.04.008 · Cited by 148
Self-citationHealth app policy: international comparison of nine countries’ approaches
Essén, A., Stern, A. D., Haase, C. B., Car, J., Greaves, F., Paparova, D., Vandeput, S., Wehrens, R., & Bates, D. W. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00573-1 · Cited by 131
Van Daele, T., Karekla, M., Kassianos, A. P., Compare, A., Haddouk, L., Salgado, J., Ebert, D. D., Trebbi, G., Bernaerts, S., Van Assche, E., et al. (2020) · Journal of Psychotherapy Integration · doi:10.1037/int0000218 · Cited by 122
“Despite the increasing evidence-base for e-mental health, commercially available applications often lack scientific background and are hardly ever validated in research (Anthes, 2016; Larsen et al., 2019).”
BackgroundCognitive-Behavioral Therapy in the Digital Age: Presidential Address
Wilhelm, S., Weingarden, H. M., Ladis, I., Braddick, V., Shin, J. Y., & Jacobson, N. C. (2019) · Behavior Therapy · doi:10.1016/j.beth.2019.08.001 · Cited by 128
“However, when these descriptions referenced specific treatment approaches, only about half described methods supported in the literature—furthermore, only two descriptions cited research about the app itself to back up effectiveness claims (Larsen et al., 2019).”
BackgroundInfluentialZidaru, T., Morrow, E. M., & Stockley, R. (2021) · Health Expectations · doi:10.1111/hex.13299 · Cited by 143
Better, Virtually: the Past, Present, and Future of Virtual Reality Cognitive Behavior Therapy
Lindner, P. (2021) · International Journal of Cognitive Therapy · doi:10.1007/s41811-020-00090-7 · Cited by 120
“Research on consumer smartphone applications for mental health (Larsen et al. 2019; Shen et al. 2015) suggests that few future consumer VR applications that will be released on ordinary digital marketplaces can be expected to be evidence-based and effective.”
BackgroundHuman-Centered Design Approaches in Digital Mental Health Interventions: Exploratory Mapping Review
Vial, S., Boudhraâ, S., & Dumont, M. (2022) · JMIR Mental Health · doi:10.2196/35591 · Cited by 133
“Several issues raised in recent years have not been resolved: there is some distrust in the field that is not served by the lack of empirical validation of its benefit [6-9]; it raises privacy and data security concerns [9,10]; it presents commercial issues (eg, financial interest, user access, advertising) [11,12]; a…”
BackgroundCollege student engagement with mental health apps: analysis of barriers to sustained use
Melcher, J., Camacho, E., Lagan, S., & Torous, J. (2022) · Journal of American College Health · doi:10.1080/07448481.2020.1825225 · Cited by 106
“They are mindful that today anyone can put an app on the app store and make it look legitimate.(35) Survey results demonstrate that students are cognizant of false information with 60% of respondents reporting that accuracy of recommendations from mental health apps was among their top concerns for using these apps.”
BackgroundSelf-citationFriis-Healy, E. A., Nagy, G. A., & Kollins, S. H. (2021) · JMIR Mental Health · doi:10.2196/25456 · Cited by 115
“Reliance on the “wisdom of the crowd” to guide decision making is not advised since, among other reasons, user ratings do not correlate well with clinical utility or quality [27].”
BackgroundAI-Based and Digital Mental Health Apps: Balancing Need and Risk
Hamdoun, S., Monteleone, R., Bookman, T., & Michael, K. (2023) · IEEE Technology and Society Magazine · doi:10.1109/mts.2023.3241309 · Cited by 104
“…optimistic about the use of digital mental health apps, with commercial entities, in particular, touting a range of benefits for self-help services [32], such as the convenience of on-demand access to resources [33], the ability to address a range of mental health concerns holistically using a…”
BackgroundUnderstanding People’s Use of and Perspectives on Mood-Tracking Apps: Interview Study
Schueller, S. M., Neary, M., Lai, J., & Epstein, D. A. (2021) · JMIR Mental Health · doi:10.2196/29368 · Cited by 102
Serrano-Ripoll, M. J., Zamanillo-Campos, R., Fiol-DeRoque, M. A., Castro, A., & Ricci-Cabello, I. (2022) · JMIR mHealth and uHealth · doi:10.2196/29621 · Cited by 96
“The digital market is full of apps designed to improve the mental health of people with depression; however, most of them remain untested in clinical trials and suffer from numerous limitations, such as being designed without content based on the recommendations of experts [11,12].”
BackgroundSeiferth, C., Vogel, L., Aas, B., Brandhorst, I., Carlbring, P., Conzelmann, A., Esfandiari, N., Finkbeiner, M., Hollmann, K., Lautenbacher, H., et al. (2023) · Nature Mental Health · doi:10.1038/s44220-023-00085-1 · Cited by 105
Self-citationRoberts, A., Davenport, T. A., Wong, T., Moon, H.-W., Hickie, I. B., & LaMonica, H. M. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100379 · Cited by 99
“Consistent with previous research (Alyami et al., 2017; Larsen et al., 2019; Sucala et al., 2017; Van Ameringen et al., 2017), the majority of apps and e-tools rated in this study did not meet this standard (mean = 3.43, standard deviation = 0.87).”
ResultInfluentialBennett, C. B., Ruggero, C. J., Sever, A. C., & Yanouri, L. (2020) · Journal of Psychotherapy Integration · doi:10.1037/int0000217 · Cited by 87
Götzl, C., Hiller, S., Rauschenberg, C., Schick, A., Fechtelpeter, J., Fischer-Abaigar, U., Koppe, G., Durstewitz, D., Reininghaus, U., & Krumm, S. (2022) · Child and Adolescent Psychiatry and Mental Health · doi:10.1186/s13034-022-00522-6 · Cited by 95
“Existing apps available in major app stores rarely use evidence–based frameworks [16–19], frequently do not involve young people in the development [20], and therefore often remain insufficiently tailored to the needs of young users [4, 20, 21].”
BackgroundSmartphone-Enhanced Symptom Management In Psychosis: Open, Randomized Controlled Trial
Lewis, S. W., Ainsworth, J. D., Sanders, C., Stockton-Powdrell, C., Machin, M., Whelan, P., Hopkins, R., He, Z., Applegate, E., Drake, R., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/17019 · Cited by 83
Weber, S., Lorenz, C., & Hemmings, N. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02745 · Cited by 74
“We assessed resilience with the 13-item Resilience Scale (RS13; Leppert et al., 2008), a short form of the Resilience Scale (RS-25; Wagnild and Young, 1993).”
MethodConnolly, S. L., Hogan, T. P., Shimada, S. L., & Miller, C. J. (2020) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-020-00165-4 · Cited by 78
Ethics of Digital Mental Health During COVID-19: Crisis and Opportunities
Martinez‐Martin, N., Dasgupta, I., Carter, A., Chandler, J. A., Kellmeyer, P., Kreitmair, K., Weiss, A. P., & Cabrera, L. Y. (2020) · JMIR Mental Health · doi:10.2196/23776 · Cited by 89
“Furthermore, the majority of consumer apps are not evidence-based and some even contain harmful content [40].”
BackgroundRegulatory considerations to keep pace with innovation in digital health products
Torous, J. B., Stern, A. D., & Bourgeois, F. T. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00668-9 · Cited by 83
Self-citationDigital Psychiatry: Risks and Opportunities for Public Health and Wellbeing
Burr, C., Morley, J., Taddeo, M., & Floridi, L. (2020) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2020.2977059 · Cited by 85
“, “motivational therapist” or “wellness coach”) or hide behind disclaimers to avoid scrutiny [72].”
BackgroundMerchant, R., Torous, J., Rodriguez-Villa, E., & Naslund, J. A. (2020) · Current Opinion in Psychiatry · doi:10.1097/yco.0000000000000626 · Cited by 70
“A 2015 review that assessed 82 commercially available smartphone apps to manage bipolar disorder concluded that in general, the content was not that only 14% of a sample of 73 smartphone apps involved individuals with lived experience in the design or development process [19 & ].”
BackgroundSelf-citationKarkosz, S., Szymański, R., Sanna, K., & Michałowski, J. M. (2023) · JMIR Formative Research · doi:10.2196/47960 · Cited by 121
“Nevertheless, applications that are available on the market often lack appropriate scientific evidence of feasibility or efficacy [4].”
BackgroundRodriguez-Villa, E., & Torous, J. B. (2019) · BMC Medicine · doi:10.1186/s12916-019-1447-x · Cited by 71
“This lack of transparency, complicated by unmeasured and unfounded claims of many apps [25], makes evaluation cumbersome and time consuming.”
BackgroundSelf-citationThe Debate on the Ethics of AI in Health Care: a Reconstruction and Critical Review
Morley, J., Machado, C., Burr, C., Cowls, J., Taddeo, M., & Floridi, L. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3486518 · Cited by 66
Psychiatry in the Digital Age: A Blessing or a Curse?
Roth, C. B., Papassotiropoulos, A., Brühl, A. B., Lang, U. E., & Huber, C. G. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18168302 · Cited by 76
Is There an App for That?: Ethical Issues in the Digital Mental Health Response to COVID-19
Skorburg, J. A., & Yam, J. (2021) · AJOB Neuroscience · doi:10.1080/21507740.2021.1918284 · Cited by 65
Determinant Factors of Public Acceptance of Stress Management Apps: Survey Study
Apolinário-Hagen, J., Hennemann, S., Fritsche, L., Drüge, M., & Breil, B. (2019) · JMIR Mental Health · doi:10.2196/15373 · Cited by 63
“However, the evidence base for the quality and efficacy of most mental health apps is limited [112], even among those mental health apps that claim to be effective [113].”
Assessing the Clinical Robustness of Digital Health Startups: Cross-sectional Observational Analysis
Day, S., Shah, V., Kaganoff, S., Powelson, S., & Mathews, S. C. (2022) · Journal of Medical Internet Research · doi:10.2196/37677 · Cited by 57
Hong, R. H., Murphy, J., Michalak, E. E., Chakrabarty, T., Wang, Z., Parikh, S. V., Culpepper, L., Yatham, L. N., Lam, R. W. M., & Chen, J. (2021) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s283731 · Cited by 58
Nwe, K., Larsen, M. E., Nelissen, N., & Wong, D. (2020) · Journal of Medical Internet Research · doi:10.2196/17457 · Cited by 56
“To date, the evidence for their effectiveness is varied [4-6] and, in general, published evidence is extremely sparse [7].”
BackgroundSelf-citationExploring medical students' views on digital mental health interventions: A qualitative study
Dederichs, M., Weber, J., Pischke, C. R., Angerer, P., & Apolinário-Hagen, J. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100398 · Cited by 50
Tark, R., Metelitsa, M., Akkermann, K., Saks, K., Mikkel, S., & Haljas, K. (2019) · JMIR Serious Games · doi:10.2196/13776 · Cited by 57
“Similarly, another recent review brought out the mental health app efficacy evaluation as a concern, showing that 38% of app store descriptions included phrases related to claims of effectiveness, whereas only less than 3% provided scientific evidence for such declarations [24].”
ResultShidhaye, R. R. (2023) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2023.101661 · Cited by 57
Digital Health COVID-19 Impact Assessment: Lessons Learned and Compelling Needs
Microsoft, Lee, P., Abernethy, A., Verily, Shaywitz, D., HealthTech, A., Gundlapalli, A., Prevention, U. S. C. F. D. C. A., Weinstein, J., Healthcare, M., et al. (2022) · NAM Perspectives · doi:10.31478/202201c · Cited by 49
“eff ectiveness through scientifi c phrasing, but only two provided direct evidence from a trial [51].”
BackgroundIndividual differences in receptivity to scientific bullshit
Evans, A. M., Sleegers, W. W. A., & Mlakar, Ž. (2020) · Judgment and Decision Making · doi:10.1017/s1930297500007191 · Cited by 48
The Future of Peer Support in Digital Psychiatry: Promise, Progress, and Opportunities
Fortuna, K. L., Venegas, M. D. R. A., Umucu, E., Mois, G., Walker, R., & Brooks, J. (2019) · Current Treatment Options in Psychiatry · doi:10.1007/s40501-019-00179-7 · Cited by 51
Spadaro, B., Martin-Key, N. A., & Bahn, S. (2021) · Journal of Medical Internet Research · doi:10.2196/27507 · Cited by 45
“…commonly downloaded mental health apps mainly fall under the umbrella of well-being apps offering relaxation, meditation, or mindfulness skills rather than validated treatments, and that the scientific language used to support claims often lacks corresponding evidence in the literature [28,29].”
Research Trends on Mobile Mental Health Application for General Population: A Scoping Review
Hwang, W. J., Ha, J. S., & Kim, M. J. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18052459 · Cited by 51
Catuara-Solarz, S., Skorulski, B., Estella-Aguerri, I., Avella-Garcia, C. B., Shepherd, S., Stott, E., Hemmings, N. R., de Villa, A. R., Schulze, L., & Dix, S. (2021) · JMIR mhealth and uhealth · doi:10.2196/30976 · Cited by 44
“Although some apps have been shown to have positive effects on mental well-being [10,12,13], concerns remain regarding the level of credibility and the robustness of evidence underlying the majority of the thousands of available apps on the market [10,14].”
Baumel, A., Torous, J. B., Edan, S., & Kane, J. M. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.05.011 · Cited by 40
Self-citationTools for Wellbeing-Supportive Design: Features, Characteristics, and Prototypes
Peters, D., Ahmadpour, N., & Calvo, R. A. (2020) · Multimodal Technologies and Interaction · doi:10.3390/mti4030040 · Cited by 41
“Taking digital mental health as a cautionary example, failures of rigor have led to an exploding industry of mental health tools that lack evidence, inclusivity, and effectiveness (and at worst, cause harm) [44].”
BackgroundFeldman, N., Back, D., BOLAND, R. J., & Torous, J. B. (2021) · Archives of Women s Mental Health · doi:10.1007/s00737-021-01138-z · Cited by 43
“In terms of the marketplace and app stores, increasing research suggests that apps available to consumers are often untested (Larsen et al. 2019) and may present privacy risks that may expose sensitive mental health data (Huckvale et al.”
BackgroundSelf-citationFagernäs, S., Hamilton, W., Espinoza, N., Miloff, A., Carlbring, P., & Lindner, P. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100370 · Cited by 38
“…past research on consumer smartphone applications, which have been found to vary considerably in both use of evidence-based therapeutic components (Shen et al., 2015), direct empirical support (Linardon et al., 2019) and use of scientific language to support these claims (Larsen et al., 2019).”
Backgroundd'Halluin, A., Costa, M., Morgiève, M., & Sebbane, D. (2023) · Journal of Medical Internet Research · doi:10.2196/43102 · Cited by 34
“Although many of these apps were evidence informed and were inspired by scientifically validated techniques and methods, very few were evidence based (ie, the subject of rigorous scientific studies demonstrating their effectiveness and reliability [10,11]).”
Gasteiger, N., Dowding, D., Norman, G., McGarrigle, L., Eost‐Telling, C., Jones, D. J., Vercell, A., Ali, S. M., & O’Connor, S. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073283 · Cited by 37
Satre, D. D., Meacham, M. C., Asarnow, L. D., Fisher, W. S., Fortuna, L. R., & Iturralde, E. (2021) · American Journal of Health Promotion · doi:10.1177/08901171211055314 · Cited by 37
“Providers and health systemsmust contendwith a shifting landscape of available apps.(68) The generally unregulated status of mental health apps(69) and the unclear value of app store user reviews have sparked new approaches for endorsing apps and cataloguing recommendations.”
BackgroundHrynyschyn, R., & Dockweiler, C. (2021) · JMIR mHealth and uHealth · doi:10.2196/24703 · Cited by 35
“Currently, thousands of mental health apps are available in the market, and these apps are designed for a brief frequent use throughout the day [24].”
BackgroundGenerating value with mental health apps
Powell, A. C., Torous, J. B., Firth, J., & Kaufman, K. R. (2020) · BJPsych Open · doi:10.1192/bjo.2019.98 · Cited by 34
Self-citationCOVID-19 has inspired global healthcare innovation
Palanica, A., & Fossat, Y. (2020) · Canadian Journal of Public Health · doi:10.17269/s41997-020-00406-2 · Cited by 33
Digital technology for addressing cognitive impairment in recent-onset psychosis: A perspective
Bell, I., Pot-Kolder, R., Wood, S. J., Nelson, B., Acevedo, N., Stainton, A., Nicol, K., Kean, J. D., Bryce, S., Bartholomeusz, C. F., et al. (2022) · Schizophrenia Research Cognition · doi:10.1016/j.scog.2022.100247 · Cited by 34
“doscience and scientific jargon to make unsubstantiated claims of effectiveness (Larsen et al., 2019).”
Self-citationCaiani, E. G., Kemps, H., Hoogendoorn, P., Asteggiano, R., Böhm, A., Borregaard, B., Boriani, G., Rocca, H. B., Casado-Arroyo, R., Castelletti, S., et al. (2024) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztae042 · Cited by 32
“17,18 Access to mHealth solutions, including wearables such as smart-watches, is further complicated by the fact that they can assume the role of a lifestyle gadget and/or that of an MD, depending on the model and the country in which they are sold.”
BackgroundLeong, Q. Y., Sridhar, S., Blasiak, A., Tadeo, X., Yeo, G., Remus, A., & Ho, D. (2021) · Journal of Medical Internet Research · doi:10.2196/27388 · Cited by 39
“Instead of objectively informing about the evidence, the function of the app description is to appeal to consumers and drive the number of app downloads, independent of the platform’s objective quality [187].”
BackgroundBowie-DaBreo, D., Sas, C., Iles‐Smith, H., & Sünram‐Lea, S. I. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517498 · Cited by 36
“There is therefore need for more evidence on the use of apps for depression in the wild to better understand user experiences and implications for design to improve the quality and acceptance of these apps.”
Schick, A., Paetzold, I., Rauschenberg, C., Hirjak, D., Banaschewski, T., Meyer-Lindenberg, A., Boehnke, J. R., Boecking, B., & Reininghaus, U. (2021) · JMIR Research Protocols · doi:10.2196/27462 · Cited by 30
“However, most mHealth apps that are currently available in major app stores are not evidence-based, often use problematic data sharing and privacy practices, and sometimes contain harmful content [34-36].”
Critical review of self‐diagnosis of mental health conditions using artificial intelligence
Wimbarti, S., Kairupan, B. H. R., & Tallei, T. E. (2024) · International Journal of Mental Health Nursing · doi:10.1111/inm.13303 · Cited by 36
Selaskowski, B., Reiland, M., Schulze, M., Aslan, B., Kannen, K., Wiebe, A., Wallbaum, T., Boll, S., Lux, S., Philipsen, A., et al. (2023) · BJPsych Open · doi:10.1192/bjo.2023.573 · Cited by 33
Haaf, R., Vock, P., Wächtershäuser, N., Correll, C. U., Köhler, S., & Klein, J. P. (2024) · Der Nervenarzt · doi:10.1007/s00115-023-01587-0 · Cited by 30
Koch, E. D., Moukhtarian, T. R., Skirrow, C., Bozhilova, N. S., Asherson, P. J., & Ebner‐Priemer, U. (2021) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2021.06.002 · Cited by 34
Is the world ready for ChatGPT therapists?
Graber-Stiehl, I. (2023) · Nature · doi:10.1038/d41586-023-01473-4 · Cited by 32
Building Better Schools with Evidence-based Policy
Allen, K., Reupert, A., & Oades, L. G. (2021) · Routledge eBooks · doi:10.4324/9781003025955 · Cited by 35
Naslund, J. A., & Aschbrenner, K. A. (2021) · Journal of Affective Disorders Reports · doi:10.1016/j.jadr.2021.100227 · Cited by 29
“It is worrisome that so few of these commercially available apps are supported with reliable or high-quality scientific evidence, and in the most egregious examples even report false or misleading claims of clinical effectiveness (Larsen et al., 2019; Lau et al., 2020).”
Bautista, J., & Schueller, S. M. (2022) · JMIR Mental Health · doi:10.2196/43942 · Cited by 28
“Although there are mental health apps that are effective, many mental health apps are not evidence based [38].”
Tong, A. C. Y., Wong, K. T. Y., Chung, W. W. T., & Mak, W. W. S. (2025) · Journal of Medical Internet Research · doi:10.2196/70436 · Cited by 43
“Many of these tools are not empirically tested or are misinformed [59-61].”
Recommendations for Successful Development and Implementation of Digital Health Technology Tools
Loo, R. T. J., Nasta, F., Macchi, M., Baudot, A., Burstein, F., Bove, R. M., Greve, M., Fröhlich, H., Khalid, S., Küderle, A., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/56747 · Cited by 28
“Larsen et al [70] found that among 73 top-ranked mental health apps, only 2 provided primary evidence, 1 cited published literature, and 14% described development with lived experience.”
Self-citationBarnett, S., Huckvale, K., Christensen, H. M., Venkatesh, S., Mouzakis, K., & Vasa, R. (2019) · Journal of Medical Internet Research · doi:10.2196/16399 · Cited by 29
“…efforts to build sensing and intervention apps, many appear to have limited potential for clinical translation because they are not backed by evidence of efficacy [28], or do not clearly satisfy clinical quality [29] and safety expectations (such as those concerning diagnostic accuracy [30]).”
BackgroundSelf-citationBowie-DaBreo, D., Sünram‐Lea, S. I., Sas, C., & Iles‐Smith, H. (2020) · JMIR Formative Research · doi:10.2196/14988 · Cited by 26
“…authors commented that although some apps seemed to be evidence informed, reflecting some theoretical principles and strategies, the apps did not demonstrate high fidelity to evidence-based treatments such as CBT or BT [25,32] and generally lacked evidence supporting use and efficacy [21,23,25,27].”
BackgroundResultInfluentialWang, M., Chen, H., Yang, F., Xu, X., & Li, J. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.06.057 · Cited by 25
The use of eHealth interventions among persons experiencing homelessness: A systematic review
Polillo, A., Gran-Ruaz, S., Sylvestre, J. C., & Kerman, N. (2021) · Digital Health · doi:10.1177/2055207620987066 · Cited by 25
Simon, L., Reimann, J., Steubl, L., Stach, M., Spiegelhalder, K., Sander, L., Baumeister, H., Meßner, E.-M., & Terhorst, Y. (2022) · Journal of Sleep Research · doi:10.1111/jsr.13642 · Cited by 24
Publication Trends in Psychotherapy: Bibliometric Analysis of the Past 5 Decades
Soares, E. E., Thrall, J., Stephens, T. N., Biglieri, R. R., Consoli, A. J., & Bunge, E. L. (2020) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.20190045 · Cited by 27
“Additionally, a growing number of publications have addressed the utilization of behavioral intervention technologies (52), such as smartphone applications (53, 54), artificial intelligence (55), virtual reality (56), chatbots (57, 58), and even video games utilizing CBT for depression (59) and anxiety (60).”
BackgroundMercurio, M., Larsen, M. E., Wisniewski, H., Henson, P., Lagan, S., & Torous, J. B. (2020) · Evidence-Based Mental Health · doi:10.1136/ebmental-2019-300137 · Cited by 23
Self-citationCan mental health apps be effective for depression, anxiety, and stress during a pandemic?
Longyear, R. L., & Kushlev, K. (2021) · Practice Innovations · doi:10.1037/pri0000142 · Cited by 21
Digital Health Literacy in Bipolar Disorder: International Web-Based Survey
Morton, E. E., Ho, K., Barnes, S. J., & Michalak, E. E. (2021) · JMIR Mental Health · doi:10.2196/29764 · Cited by 24
“…10,000 publicly available offerings [51], app store search algorithms lack transparency and may be influenced by paid advertising [52], turnover in apps is high [53], and app descriptions often make scientific claims regarding effectiveness despite lack of appropriate high-quality evidence [54,55].”
Kheirinejad, S., Visuri, A., Suryanarayana, S. A., & Hosio, S. J. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e16586 · Cited by 24
Lin, X., Martinengo, L., Jabir, A. I., Ho, A. H. Y., Car, J., Atun, R. A., & Car, L. T. (2023) · Journal of Medical Internet Research · doi:10.2196/45984 · Cited by 24
“This could imply that some app developers might be using scientific terminology to attract users [63].”
Digital Technology to Enhance Clinical Care of Early Psychosis
Bell, I., & Álvarez‐Jiménez, M. (2019) · Current Treatment Options in Psychiatry · doi:10.1007/s40501-019-00182-y · Cited by 27
Digital health & low-value care
O’Reilly-Jacob, M., Mohr, P. E., Ellen, M., Petersen, C. C., Sarkisian, C. A., Attipoe, S., & Rich, E. C. (2021) · Healthcare · doi:10.1016/j.hjdsi.2021.100533 · Cited by 23
Kang, R. M., & Reynolds, T. L. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642178 · Cited by 27
“In a study conducted by Larsen and colleagues [38], researchers investigated app store description pages for mental health apps, fnding that 64% of apps included in their study claimed efcacy in reducing symptoms of mental illness or diagnosing a mental illness, and 44% of apps used “scientifc…”
BackgroundKadirvelu, B., Bel, T. B., Freccero, A., Simplicio, M. C. D., Nicholls, D., & Faisal, A. A. (2025) · Journal of Medical Internet Research · doi:10.2196/72501 · Cited by 32
“The proliferation of smartphones has enabled a new class of digital mental health interventions that leverage their unique data collection capabilities [12-14].”
Metadata
- Authors
- Larsen, M. E., Huckvale, K., Nicholas, J., Torous, J., Birrell, L., Li, E., & Reda, B.
- Affiliations
- Black Dog Institute (AU)
- Northwestern University (US)
- Beth Israel Deaconess Medical Center (US)
- UNSW Sydney (AU)
- Corresponding author
- Larsen, Mark Erik
- Venue
- npj Digital Medicine
- Published
- 2019-03-22
- Volume
- 2
- Issue
- 1
- Pages
- 18-18
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC6550255
- OpenAlex
- W2925166053
- Cited by
- 334
- References
- 23
- Citations by year
- 2026: 16 · 2025: 53 · 2024: 42 · 2023: 48 · 2022: 47 · 2021: 60 · 2020: 54 · 2019: 14
- FWCI
- 49.43
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- npj Digital Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Health Sciences › Health Professions › Health Information Management › Health, Technology, Consumer Behavior
- Keywords
- mental health apps, scientific language, quality of evidence, consumer choice, knowledge translation, depression, anxiety, schizophrenia, substance use, self-harm, Google Play Store, iTunes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
