Apps with maps-anxiety and depression mobile apps with evidence-based frameworks: Systematic search of major app stores
Open accessPublished version of a preprint
The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index; 20 of the 36 sources its claims cite were checked for support.
Abstract
Background: Mobile mental health apps have become ubiquitous tools to assist people in managing symptoms of anxiety and depression. However, due to the lack of research and expert input that has accompanied the development of most apps, concerns have been raised by clinicians, researchers, and government authorities about their efficacy. Objective: This review aimed to estimate the proportion of mental health apps offering comprehensive therapeutic treatments for anxiety and/or depression available in the app stores that have been developed using evidence-based frameworks. It also aimed to estimate the proportions of specific frameworks being used in an effort to understand which frameworks are having the most influence on app developers in this area. Methods: A systematic review of the Apple App Store and Google Play store was performed to identify apps offering comprehensive therapeutic interventions that targeted anxiety and/or depression. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist was adapted to guide this approach. Results: Of the 293 apps shortlisted as offering a therapeutic treatment for anxiety and/or depression, 162 (55.3%) mentioned an evidence-based framework in their app store descriptions. Of the 293 apps, 88 (30.0%) claimed to use cognitive behavioral therapy techniques, 46 (15.7%) claimed to use mindfulness, 27 (9.2%) claimed to use positive psychology, 10 (3.4%) claimed to use dialectical behavior therapy, 5 (1.7%) claimed to use acceptance and commitment therapy, and 20 (6.8%) claimed to use other techniques. Of the 162 apps that claimed to use a theoretical framework, only 10 (6.2%) had published evidence for their efficacy. Conclusions: The current proportion of apps developed using evidence-based frameworks is unacceptably low, and those without tested frameworks may be ineffective, or worse, pose a risk of harm to users. Future research should establish what other factors work in conjunction with evidence-based frameworks to produce efficacious mental health apps. © Jamie M Marshall, Debra A Dunstan, Warren Bartik. Originally published in JMIR Mental Health (http://mental.jmir.org),24.06.2020. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.
Overview
CoChat Summary
## TL;DR
This systematic search of the Apple App Store and Google Play found 293 apps that appeared to offer therapeutic treatment for anxiety and/or depression, but only 162 (55.3%) mentioned an evidence-based framework in their descriptions. Of those 162, only 10 (6.2%) had published evidence for efficacy.
## Key findings
- The search identified 293 apps whose descriptions inferred a therapeutic treatment for anxiety and/or depression; 162/293 (55.3%) said they used an evidence-based theoretical framework.
- Among all 293 apps, the most commonly claimed frameworks were CBT 88/293 (30.0%), mindfulness 46/293 (15.7%), positive psychology 27/293 (9.2%), DBT 10/293 (3.4%), ACT 5/293 (1.7%), and other techniques 20/293 (6.8%).
- Restricting to the 162 apps that claimed a theoretical framework, CBT made up 54.3% (88/162), followed by mindfulness 28.4% (46/162) and positive psychology 16.7% (27/162).
- Only 10/162 (6.2%) apps with a claimed framework had published evidence for effectiveness; across those studies there were 1017 participants in intervention conditions and 447 in control groups, with a mean age of 32.7 years (SD 9.3) where reported.
## Methods
- The authors conducted a systematic review/search of the Apple App Store and Google Play Store, using an adapted PRISMA/AMSTAR-based protocol called PASSR.
- Searches were run by four researchers in December 2018 and again in July 2019 using 19 keywords across all categories; apps were counted once even if available in both stores.
- Inclusion was limited to English-language apps whose descriptions indicated a comprehensive therapeutic treatment for anxiety and/or depression rather than only a single element of therapy; analysis was based on app store descriptions.
## Limitations
- The paper states that app store searching has restricted search capabilities, results can be influenced by developer-completed questionnaires and store algorithms, and app ordering can be ambiguous compared with literature searches.
- A stated limitation is that the study was based on app store descriptions only; none of the shortlisted apps were trialed to verify that descriptions matched actual content.
- Evident from the text: the review may miss apps whose descriptions did not clearly signal treatment for anxiety/depression, and it does not assess the quality of the claimed frameworks or whether apps were actually available/downloadable to all users despite being listed.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
12 Valid · 1 Not valid · 9 Not checked · 46 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Over the last two decades, e-mental health programs with a CBT framework and designed for use on computers have been found to be effective for both adults and children in reducing anxiety and depression [19,20]”
§ Apps With Maps—Anxiety and Depression Mobile Apps With Evidence-Based Frameworks: Systematic Search of Major App Stores > Introduction > Apps and Evidence-Based Frameworks
What the sources say
- Supportedcites [19] source
The Introduction states that computer- and Internet-based CBT is effective for depressive and anxiety disorders in adults. The Discussion reports significant effects on anxiety and depressive symptoms in youth, supporting the claim for both age groups. (LLM, conf=0.92)
- Not checked — no full text of the sourcecites [20] source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The nine research articles [47-55] that examined the 10 apps were made up of seven randomized controlled trials (RCTs)—the Flett et al article contained one RCT with 2 of the listed apps: Headspace and Smiling Mind [52]—and two feasibility or pilot studies: Kinderman et al [48] and Carey et al [55].”
§ Apps With Maps—Anxiety and Depression Mobile Apps With Evidence-Based Frameworks: Systematic Search of Major App Stores > Results
What the sources say
- Supportedcites [47] source
This paper supports its role as one of the cited randomized controlled trials examining two apps, not the aggregate count across all nine articles. The Discussion > Limitations calls it an RCT directly comparing an agoraphobia-specific app with a generic anxiety-related app, and Methods > Sample Size names the two apps as Agoraphobia Free and Stress Free. (LLM, conf=0.91)
- Supportedcites [48] source
The citation to Kinderman et al. [48] identifies it as one of the feasibility or pilot studies, rather than as support for the count of nine articles. Its Abstract (Aims) describes the first trial of Catch It, and its Discussion says a randomized controlled trial should be the next step. (LLM, conf=0.91)
- Not checked — no full text of the sourcecites [49] source
- Not checked — no full text of the sourcecites [50] source
- Partly supportedcites [51] sourceExperimental — accuracy not yet measured
The Method > Treatments passage describes a randomized comparison involving two apps, and the Discussion calls this a pilot trial. The paper does not establish the citing sentence’s totals of nine articles and ten apps, its seven-RCT/two-pilot classification, or the named studies’ places in that classification (pages unavailable in our copy). (LLM, conf=0.77)
- Not checked — no full text of the sourcecites [52] source
- Supportedcites [53] source
This paper supports its inclusion as one of the cited randomized trials. The Procedure section (page unavailable) says participants were randomized to DeStressify or a wait-list control, and the Design section calls it a pilot, exploratory study. (LLM, conf=0.91)
- Not checked — no full text of the sourcecites [54] source
- Supportedcites [55] source
Carey et al. identify their research as a pilot study in the Discussion (page unavailable). That supports the part of the citing sentence for which Carey et al. [55] is cited: its classification as a pilot study. The source is not being cited to establish the counts across all nine articles. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“over 90% of available apps for depression can be found in either of these stores [46].”
§ Apps With Maps—Anxiety and Depression Mobile Apps With Evidence-Based Frameworks: Systematic Search of Major App Stores > Methods
What the sources say
- Supportedcites [46] source
The Results > General Characteristics section reports that Google accounted for 53.5% and Apple for 37.0% of the 243 included depression apps. Together, that is 90.5%, supporting the claim for the apps identified in this study’s five-store search. (LLM, conf=0.92)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“and improving happiness and well-being [21-23].”
§ Apps With Maps—Anxiety and Depression Mobile Apps With Evidence-Based Frameworks: Systematic Search of Major App Stores > Introduction > Apps and Evidence-Based Frameworks
What the sources say
- Partly supportedcites [21] sourceExperimental — accuracy not yet measured
The Results (Intervention Characteristics) include programs aimed at general psychological well-being, and the Discussion (Conclusions) cautiously supports the effectiveness of Internet-delivered CBT for university students’ mental-health outcomes. The paper does not report an improvement in happiness specifically, and its demonstrated benefits chiefly concern depression, anxiety, and stress rather than a measured overall well-being outcome. (LLM, conf=0.85)
- Partly supportedcites [22] sourceExperimental — accuracy not yet measured
The Discussion’s Principal Findings (page unavailable) report improved well-being scores among young people using the web-based Bite Back program. But the Methods identify Bite Back as a positive psychology program, not one with a CBT framework, and the paper does not report a separate measure of happiness. (LLM, conf=0.87)
- Partly supportedcites [23] sourceExperimental — accuracy not yet measured
The Methods (Intervention) describe MoodGYM as an Internet-based program teaching cognitive-behavioral skills, and the Results report significantly higher mental well-being scores than the control group at 6 and 12 weeks. The Methods (Outcome Measures) identify mental well-being as the primary outcome, but the paper does not report a separate measure or finding for happiness. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Some studies have compared the effectiveness of e-mental health programs with face-to-face therapy and have found comparable [24,25]”
§ Apps With Maps—Anxiety and Depression Mobile Apps With Evidence-Based Frameworks: Systematic Search of Major App Stores > Introduction > Apps and Evidence-Based Frameworks
What the sources say
- Supportedcites [24] source
The Results report five studies comparing computerized CBT with face-to-face CBT for depression or panic disorder. The difference between treatments was not statistically significant, supporting the claim that the programs were found comparable in those studies. (LLM, conf=0.92)
- Not checked — no full text of the sourcecites [25] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
33 more claims
Create a free account to see all 68 claims and every source verdict.
- It may state two separate claims as one. 17
- Probably not a substantial claim. 19
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Mental health apps that saw a surge in downloads during the COVID-19 pandemic (eg, Headspace, Calm, Talkspace, Teladoc's BetterHelp, and Noom) have since experienced declining downloads.”
From Digital health technologies and major depressive disorder (2023) · cites this paper as [152]
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · cited 3×
“ACT: acceptance and commitment therapy AMSTAR: A MeaSurement Tool to Assess systematic Reviews CBT: cognitive behavioral therapy DBT: dialectical behavior therapy IPT: interpersonal therapy mHealth: mobile health PASSR: Protocol for App Store Systematic Reviews PRISMA: Preferred Reporting Items for Systematic Reviews…”
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
Wang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,657
Positive psychology: An introduction.
Seligman, M. E. P., & Csikszentmihalyi, M. (2000) · American Psychologist · doi:10.1037/0003-066x.55.1.5 · Cited by 11,351
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al. (2017) · BMJ · doi:10.1136/bmj.j4008 · Cited by 10,636 · cited 1×
“This systematic review used the AMSTAR (A MeaSurement Tool to Assess systematic Reviews) 2 [43] and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) [44] protocols for guidance in conducting the app store search.”
MethodPositive Psychology Progress: Empirical Validation of Interventions.
Seligman, M. E. P., Steen, T. A., Park, N., & Peterson, C. (2005) · American Psychologist · doi:10.1037/0003-066x.60.5.410 · Cited by 5,947 · cited 1×
“For example, Freedman and Enright [33] showed that forgiveness significantly decreased depression, Froh et al [34] implemented a gratitude intervention that resulted in significantly greater optimism and life satisfaction as well as decreased negative affect, Lyubomirsky et al [35] reported an increase in happiness an…”
MethodCognitive Therapy of Depression
HOLLON, S. D., & BECK, A. T. (1979) · Cognitive-Behavioral Interventions · doi:10.1016/b978-0-12-404480-7.50012-5 · Cited by 4,685
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
The effect of mindfulness based therapy on anxiety and depression: A meta analytic review
Hofmann, S. G., Sawyer, A. T., Witt, A. A., & Oh, D. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0018555 · Cited by 3,846 · cited 1×
“Various mindfulness interventions have proved efficacious in the past [38], and these meditation and breathing activities are easily incorporated into an app.”
Background
Research works printed but not linked: 1
- SmithAPew Research Center201504012019-01-20Washington, DCPew Internet & American Life ProjectChapter one: A portrait of smartphone ownershiphttp://www.pewinternet.org/2015/04/01/chapter-one-a-portrait-of-smartphone-ownership/
Cited by
Torous, J., Bucci, S., Bell, I. H., Kessing, L. V., Faurholt‐Jepsen, M., Whelan, P., Carvalho, A. F., Keshavan, M., Linardon, J., & Firth, J. (2021) · World Psychiatry · doi:10.1002/wps.20883 · Cited by 1,199
Rauschenberg, C., Schick, A., Goetzl, C., Roehr, S., Riedel-Heller, S. G., Koppe, G., Durstewitz, D., Krumm, S., & Reininghaus, U. (2021) · European Psychiatry · doi:10.1192/j.eurpsy.2021.17 · Cited by 158
A review of mobile chatbot apps for anxiety and depression and their self-care features
Ahmed, A., Ali, N., Aziz, S., Abd-alrazaq, A. A., Hassan, A., Khalifa, M., Elhusein, B., Ahmed, M., Ahmed, M. A. S., & Househ, M. (2021) · Computer Methods and Programs in Biomedicine Update · doi:10.1016/j.cmpbup.2021.100012 · Cited by 112
Esophageal Hypervigilance and Symptom-Specific Anxiety in Patients with Eosinophilic Esophagitis
Taft, T., Carlson, D. A., Simons, M. L., Zavala, S., Hirano, I., Gonsalves, N. P., & Pandolfino, J. E. (2021) · Gastroenterology · doi:10.1053/j.gastro.2021.06.023 · Cited by 101
Götzl, C., Hiller, S., Rauschenberg, C., Schick, A., Fechtelpeter, J., Fischer-Abaigar, U., Koppe, G., Durstewitz, D., Reininghaus, U., & Krumm, S. (2022) · Child and Adolescent Psychiatry and Mental Health · doi:10.1186/s13034-022-00522-6 · Cited by 105
“Existing apps available in major app stores rarely use evidence–based frameworks [16–19], frequently do not involve young people in the development [20], and therefore often remain insufficiently tailored to the needs of young users [4, 20, 21].”
BackgroundLagan, S., D'Mello, R., Vaidyam, A., Bilden, R., & Torous, J. (2021) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.13306 · Cited by 65
Chen, B., Yang, T., Xiao, L., Xu, C., & Zhu, C. (2022) · Journal of Medical Internet Research · doi:10.2196/39128 · Cited by 89
“Mindfulness meditation through smartphone apps may improve muscle relaxation and diaphragm breathing, focus attention on different XSL • FO RenderX parts of the body, raise noncritical awareness of events occurring continuously, and enhance self-reflection [59].”
Martinengo, L., Stona, A., Griva, K., Dazzan, P., Pariante, C. M., von Wangenheim, F., & Car, J. (2021) · Journal of Medical Internet Research · doi:10.2196/27619 · Cited by 68
“Publications in peer-reviewed journals were available for 17 apps, of which only 6 (35%) apps were evaluated using a randomized clinical trial, confirming the lack of evidence supporting the use of publicly available apps [91,92].”
MethodCameron, G., Mulvenna, M. D., Ennis, E., O’Neill, S. M., Bond, R., Cameron, D., & Bunting, A. (2024) · JMIR Mental Health · doi:10.2196/67785 · Cited by 81
Eis, S., Solà-Morales, O., Duarte‐Díaz, A., Vidal‐Alaball, J., Perestelo‐Pérez, L., Robles, N., & Carrión, C. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19042186 · Cited by 45
“Some studies suggest that this could be due to difficulty in identifying the right applications [29] or the lack of evidence supporting its potential effectiveness [30].”
BackgroundDigital health technologies and major depressive disorder
McIntyre, R. S., Greenleaf, W., Bulaj, G., Taylor, S. T., Mitsi, G., Saliu, D., Czysz, A., Silvesti, G., Garcia, M., & Jain, R. (2023) · CNS Spectrums · doi:10.1017/s1092852923002225 · Cited by 38
“92 Digital therapeutics have been approved by the FDA for the treatment of depression.”
MethodIs the world ready for ChatGPT therapists?
Graber-Stiehl, I. (2023) · Nature · doi:10.1038/d41586-023-01473-4 · Cited by 32
Huberty, J., Green, J., Puzia, M., & Stecher, C. D. (2021) · JMIR mhealth and uhealth · doi:10.2196/27106 · Cited by 30
“Mindfulness meditation apps are a popular type of mental health app, with over 290 available for download [1].”
BackgroundMeditation and mental health apps: relaxation, yoga, and meditation apps
Gál, É. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-99271-8.00013-9 · Cited by 23
Reyes, A. T. (2022) · Issues in Mental Health Nursing · doi:10.1080/01612840.2021.1953652 · Cited by 21
“In recent years, the concepts of mindfulness and acceptance have been increasingly integrated into mental health mobile apps to manage posttraumatic stress disorder (PTSD) symptoms (Marshall et al., 2020; Neary & Schueller, 2018).”
BackgroundRickard, N. S., Kurt, P., & Meade, T. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.1003181 · Cited by 30
Marshall, J. M., Dunstan, D. A., & Bartik, W. J. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.775775 · Cited by 26
Self-citationStress and Supportive Care Needs of Millennial Caregivers: A Qualitative Analysis
Hebdon, M., Jones, M., Neller, S. A., Kent‐Marvick, J., Thomas, M., Stewart, E., Aaron, S., Wilson, C. M., Peterson, N. E., & Ellington, L. (2021) · Western Journal of Nursing Research · doi:10.1177/01939459211056689 · Cited by 22
“Self-directed mental health treatment through mHealth apps is also growing, although the evidence is less robust (Marshall et al., 2020).”
BackgroundWu, X., Lin, X., Li, P., Tang, T., & Huang, C. (2021) · JMIR mhealth and uhealth · doi:10.2196/34054 · Cited by 20
“This is different from foreign mental health apps [39-41].”
BackgroundMontgomery, R. M., Boucher, E. M., Honomichl, R. D., Powell, T. A., Guyton, S. L., Bernecker, S. L., Stoeckl, S. E., & Parks, A. C. (2021) · JMIR Cardio · doi:10.2196/32351 · Cited by 19
“However, most of these interventions lack evidence regarding their safety and effectiveness, and those that do have empirical support are generally not widely accessible [88,89].”
BackgroundInsights From Longitudinal Evaluation of Moodie Mental Health App
Alslaity, A., Chan, G., Orji, R., & Wilson, R. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3519851 · Cited by 14
Carli, V., Petros, N. G., Hadlaczky, G., Vitcheva, T., Berchialla, P., Bianchi, S., Carletto, S., Christinaki, E., Citi, L., Dinis, S., et al. (2022) · EClinicalMedicine · doi:10.1016/j.eclinm.2022.101423 · Cited by 15
Yeager, C. M., & Benight, C. C. (2022) · JMIR Mental Health · doi:10.2196/35048 · Cited by 20
“Several systematic reviews have concluded that although DMHIs are growing in popularity, evidence of their efficacy is still limited [10,11].”
BackgroundLi, W., Meng, X., Zhang, K., Yang, Z., Feng, Z., Tong, K., & Tian, J. (2022) · Arthroscopy The Journal of Arthroscopic and Related Surgery · doi:10.1016/j.arthro.2022.09.008 · Cited by 12
Moulaei, K., Bahaadinbeigy, K., Mashoof, E., & Dinari, F. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02308-y · Cited by 19
“Mobile applications have become an all-encompassing tool for helping people to manage and control anxiety and depression symptoms [18], provide quick and easy access to health information, and improve interaction with therapists [19].”
BackgroundTreating Psychological Trauma in the Midst of COVID-19: The Role of Smartphone Apps
Marshall, J. M., Dunstan, D. A., & Bartik, W. J. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00402 · Cited by 13
Self-citationRajeshkumar, L., Mishkind, M. C., Coleman, J. J., Pahwa, M., LaPreze, D., Solenske, S., & Wright, J. H. (2024) · Journal of Psychiatric Practice · doi:10.1097/pra.0000000000000815 · Cited by 15
“124 Although the evidence is less robust, several MHapps or independent MHapp features have been evaluated against an evidence-based framework or with an RCT.”
Mobile applications in adolescent psychotherapy during the COVID-19 pandemic: a systematic review
Wüllner, S., Hermenau, K., Krutkova, M., Petras, I.-K., Hecker, T., & Siniatchkin, M. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1345808 · Cited by 12
“As other researchers have pointed out, most of the available mental health apps do not provide evidence on their effectiveness (12, 13, 32, 37, 41).”
BackgroundMethodMcCallum, M., Ho, A. S., Mitchell, E. S., May, C. N., Behr, H., Ritschel, L. A., Mochrie, K. D., & Michaelides, A. P. (2022) · JMIR Formative Research · doi:10.2196/36794 · Cited by 13
“Moreover, users are self-selected, meaning that the problems they are experiencing may or may not map onto the content including the mobile app (ie, problems with appropriateness) [25].”
BackgroundInfluentialFeeling Moodie: Insights from a Usability Evaluation to Improve the Design of mHealth Apps
Chan, G., Alslaity, A., Wilson, R., & Orji, R. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2241613 · Cited by 11
“However, a recent review of 293 mobile apps interventions that target anxiety or depression, found that most of the apps reviewed were ineffective at reducing anxiety or depression and potentially harm the user because they provide the user with misleading information (Marshall et al., 2020).”
Auva’a-Alatimu, T., Alefaio‐Tugia, S., & Ioane, J. (2024) · International Journal of Mental Health Systems · doi:10.1186/s13033-024-00633-x · Cited by 14
Ghafoor, R. Z., Nawaz, S., Zahra, T., & Hakeem, T. A. (2022) · Pakistan Journal of Medical and Health Sciences · doi:10.53350/pjmhs22169618 · Cited by 14
Ooi, P. B., & Wilkinson, G. G. (2024) · British Journal of Guidance and Counselling · doi:10.1080/03069885.2024.2373180 · Cited by 14
“…regard to the delivery of cognitive behavioural therapy (CBT) (Finazzi et al., 2023; Huguet et al., 2016) especially in relation to the treatment of anxiety disorder and depression (e.g. Marshall et al., 2020), and on the effectiveness of online therapy during Covid 19 (e.g. Barker & Barker, 2022).”
McCarthy, K., & Horwitz, A. G. (2025) · Journal of American College Health · doi:10.1080/07448481.2025.2458085 · Cited by 8
“25 Numerous mobile health applications (apps) posit incorporating and delivering components of evidence-based interventions such as cognitive behavioral therapy techniques (eg, cognitive restructuring, behavioral activation, problem solving) 26 yet many randomized control trial studies have failed…”
Mehrotra, S., Tripathi, R., Sengupta, P., Karishiddimath, A., Francis, A., Sharma, P., Sudhir, P. M., Srikanth, T. K., Rao, G. N., & Sagar, R. (2025) · JMIR mhealth and uhealth · doi:10.2196/79238 · Cited by 9
“Moreover, many well-researched apps remain unavailable to the public, while the majority of readily accessible apps lack direct evidence of effectiveness [20,21].”
Evaluating mobile mental health apps using the mobile application rating scale (MARS) in the U.S.
Ki, E., Jang, J., & Kang, D.-Y. (2024) · Advances in Mental Health · doi:10.1080/18387357.2024.2388680 · Cited by 8
“CBT is based on the idea that emotions can be effectively managed by changing thinking patterns (Marshall et al., 2020).”
Dietvorst, E., de Vries, L. P., van Eijl, S., Mesman, E., Legerstee, J. S., Keijsers, L., Hillegers, M. H. J., & Vreeker, A. (2024) · Digital Health · doi:10.1177/20552076241294105 · Cited by 13
Zamyatin, K. A., Игоревич, Н. Д., & Solovieva, M. (2021) · Russian Journal of Cardiology · doi:10.15829/1560-4071-2021-4681 · Cited by 8
Francis, L., Spaulding, E. M., Bloom, I., Patel, A., & Perrin, N. (2023) · Public Health Nutrition · doi:10.1017/s1368980023000526 · Cited by 4
“…search, appraisal and reporting of nutrition-related apps for children according to the Protocol for App Store Systematic Reviews (PASSR), (25) which was adapted from the A MeaSurement Tool to Assess Systematic Reviews (AMSTAR) (26) and the Preferred Reporting Items for Systematic Reviews…”
Being Present for the Future: Exploring Mindfulness and Prospective Memory
Nuno, C., & Shelton, J. T. (2023) · Journal of Cognitive Enhancement · doi:10.1007/s41465-023-00274-x · Cited by 6
“In comparison, app-based mindfulness training has sparsely been studied within the academic literature—in spite of its exponential mainstream popularization (Marshall et al., 2020).”
Technological Advances in Treating Anxiety Disorders
Goetter, E. M., & Yuen, E. K. (2024) · Psychiatric Clinics of North America · doi:10.1016/j.psc.2024.04.020 · Cited by 6
Effects of Using Mobile Apps for Mental Health Care in Korea: A Systematic Review
Lee, J., & Choi, H. (2022) · Journal of Korean Academy of Psychiatric and Mental Health Nursing · doi:10.12934/jkpmhn.2022.31.1.88 · Cited by 6
“그러나 대다수의 애플리케이 션은 그 효과가 검증되지 않은 채 사용되고 있다[13-15].”
Real-World Usage of a Digital Employee Wellbeing Platform: k-Means Clustering Analysis
Cameron, G., Mulvenna, M. D., Bond, R., Ennis, E., O’Neill, S. M., Cameron, D., & Bunting, A. (2024) · IEEE International Conference on e-Health Networking, Applications and Services (Healthcom) · doi:10.1109/healthcom60970.2024.10880783 · Cited by 4
Lessons Learned in Selection and Review of Depression Apps for Primary Care Settings
Emerson, M. R., Watanabe‐Galloway, S., Dinkel, D. M., Kotcherlakota, S., & Fok, L. (2020) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-020-00156-5 · Cited by 4
Science‐Based Mobile Apps for Reducing Anxiety: A Systematic Review and Meta‐Analysis
Doğan, T., Koçtürk, N., Akın, E., Kurnaz, M. F., Öztürk, C. D., Şen, A., & Yalçın, M. (2024) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.3058 · Cited by 9
Laure, T., Meija, C. V., Remmerswaal, D., Dulloo, D., der Hallen, R. V., Mayer, B., Littel, M., Dierckx, B., Legerstee, J. S., Engels, R. C. M. E., et al. (2025) · JMIR Formative Research · doi:10.2196/63325 · Cited by 2
“For instance, while some apps incorporate evidence-based therapeutic techniques such as cognitive behavioral therapy (CBT) or mindfulness practices, only a very small fraction have been empirically tested and validated [33].”
Varker, T., Metcalf, O., Mok, Y. F., Lamb, K. E., Fredrickson, J., Arjmand, H., Howlett, P., Gibson, K., Pham, L., Everett, K., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02092-1 · Cited by 4
“However,lessthan10%ofmentalhealthappsthatarecurrentlyavailable havepublishedevidencetosupporttheiref fi cacy 17 .”
Benefits, Implications and Ethical Concerns of Machine Learning Tools Serving Mental Health Purposes
Gamboa, P., Quaresma, C., Varandas, R. S. P., & Gamboa, H. F. S. (2021) · IFIP advances in information and communication technology · doi:10.1007/978-3-030-78288-7_27 · Cited by 2
Psikolojik Yardım Hizmetlerindeki Mobil Uygulamalarla İlgili Bir İnceleme
Kabadayı, F., & Güven, M. (2023) · Türk Eğitim Bilimleri Dergisi · doi:10.37217/tebd.1291285 · Cited by 2
Psychiatry, caught in the web of the internet
Amarakoon, L., & Fernando, R. (2024) · Sri Lanka Journal of Psychiatry · doi:10.4038/sljpsyc.v14i2.8532 · Cited by 1
“Marshall et al., 2020, in their systematic review of 293 commercially available apps for depression and/or anxiety, found that only 55.3% of the apps indicate an evidence-based framework.”
BackgroundInfluentialMunteanu, T., Kannis‐Dymand, L., Millear, P. M. R., & Jones, M. (2024) · Cyberpsychology Journal of Psychosocial Research on Cyberspace · doi:10.5817/cp2024-4-6 · Cited by 4
“…an app specific to GAD that provides quality, evidence-based, therapeutic strategies and supports previous research that has found that few mental health apps have a strong evidence base, including apps for GAD (Kertz et al., 2017; Marshall et al., 2020; Sucala et al., 2017; Wasil et al., 2019).”
ResultBoege, S., Milne‐Ives, M., Ananthakrishnan, A., Cong, C., Sharma, A., Anderson, D. O., & Meinert, E. (2024) · JMIR Research Protocols · doi:10.2196/56400 · Cited by 3
AI Mental Health Support and the Case for Distributed Networked Regulation
Paterson, J. (2026) · Cambridge University Press eBooks · doi:10.1017/9781009860796 · Cited by 1
A-Tjak, J. G. L., & Hayes, L. (2022) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197550076.013.14 · Cited by 1
Mandelbaum, M. M., Rumelt, A. M. A., Wein, S. N., & Vidair, H. B. (2024) · Journal of Child and Family Studies · doi:10.1007/s10826-024-02789-z · Cited by 1
“Moreover, apps can be used in conjunction with therapy to aid with homework compliance, skills generalization, and contacting therapists (Lui et al., 2017; Marshall et al., 2020a).”
InfluentialShaw, J. (2021) · Open Access Victoria University of Wellington | Te Herenga Waka (Figshare) · doi:10.26686/wgtn.14471310.v1
Chen, H., Chen, B., & Yang, T. (2021) · doi:10.2196/preprints.33882
Exploring the role of health apps in supporting young people with depression: a mixed methods study
Martinengo, L. (2021) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/157124
Mancini, A., & Fabbo, A. (2023) · Springer eBooks · doi:10.1007/978-3-031-27968-3_33
A Transparency Framework for App Store Descriptions
Alhejaili, A., & Blustein, J. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-35989-7_56 · Cited by 1
Freyer, O., Bergey, F., Cotte, F., Gilbert, S., & Mehl, A. (2023) · doi:10.2196/preprints.52486
Chan, G., Alslaity, A., Wilson, R., Rajeshsingh, P., & Orji, R. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-47718-8_28 · Cited by 1
Martínez-Vispo, C., García-Huércano, C., Cerón, S. C., Morejón, A. R., & Moreno-Peral, P. (2024) · Digital Health · doi:10.1177/20552076241292418 · Cited by 1
“14,42 A main strength of the prevANS intervention is that a group of experts with clinical experience designed it after reviewing the current scienti fi c literature and choosing the components based on the evidence for the prevention of anxiety disorders.”
BackgroundTelemedicine in Mental Health: Systematic Review of Online Psychological Therapy
Ramos-Galarza, C., García-Cruz, P., Guerrero-Tates, B., & Bolaños-Pasquel, M. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-74825-7_1
Gasteiger, N., Norman, G., Grainger, R., van der Veer, S. N., McGarrigle, L., Jones, D. J., Eost‐Telling, C., Vercell, A., Ford, C., Ali, S. M., et al. (2024) · JAMIA Open · doi:10.1093/jamiaopen/ooae159 · Cited by 1
“Marshall et al. 10 developed the Protocol for App Store Systematic Reviews guidance by simply amending some of the PRISMA 3 and AMSTAR 2 items, the latter of which is a critical appraisal tool for systematic reviews that include randomized or nonrandomized studies.”
Mental Health Apps Available in App Stores for Indian Users: Protocol for a Systematic Review
Mehrotra, S., Tripathi, R., Sengupta, P., Karishiddimath, A., Francis, A., Sharma, P., Sudhir, P. M., Srikanth, T. K., Rao, G. N., & Sagar, R. (2025) · JMIR Research Protocols · doi:10.2196/71071 · Cited by 4
“However, only 6.2% had published evidence of their efficacy [13].”
InfluentialAttoe, C., Szekely, R., Vega, M. O., Browne, E., Thomas, R., Kwan, T., Altchouler, P., De, A., Hariharan, N., Clay, G., et al. (2025) · doi:10.2196/preprints.73963
Attoe, C., Szekely, R., Vega, M. O., Browne, E., Thomas, R., Kwan, C. W. T., Altchouler, P., De, A., Hariharan, N., Clay, G., et al. (2025) · doi:10.2196/preprints.81498
Artificial Intelligence and Human Psychology
Karahan, F. Ş. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-1479-2.ch003
Kuberka, P., Chilton, M., Pike, K. C., Shafran, R., & Yardley, L. (2025) · doi:10.2196/preprints.84384
Attoe, C., Szekely, R., Vega, M. O., Browne, E., Thomas, R., Kwan, C. W. T., Altchouler, P., De, A., Hariharan, N., Clay, G., et al. (2026) · Digital Health · doi:10.1177/20552076261428307
“15 Even more concerning, another systematic review 16 found that 6 out of 82 apps for bipolar disorder, a condition with speci fi c and distinct guidelines, contained inaccurate information.”
Identifying what works in mental health apps through meta-regression analyses of 169 trials
Kraiss, J. T., Fiß, F., Chakhssi, F., Aktas, F. B., Koelen, J. A., & Simões, J. P. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02466-z · Cited by 1
“While these might not strictly fall under the umbrella of CBT, we still expected them to be relatively prevalent in mental health apps 28 .”
Jerome, L., Adams, K., Bird, V., & Ougrin, D. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001276
“A 2020 review by Marshall et al. [6] found 293 smartphone applications available through the Apple App Store or Google Play Store that offered a therapeutic treatment for anxiety and/or depression.”
InfluentialLicht-Handler, A. S., Lin, C., Stelzer, E., & Killikelly, C. (2026) · JMIR Research Protocols · doi:10.2196/87021
“However, a systematic search of apps on the app store revealed that only 6.2% of apps for depression and anxiety, which claimed that they use a theoretical evidence-based framework, actually published evidence for the apps’ efficacy [43], thus emphasizing the importance of well-tested interventions.”
InfluentialGörtz‐Dorten, A., Schindler, L., Klee, J., Wähnke, L., & Döpfner, M. (2026) · Trials · doi:10.1186/s13063-026-09806-0
Kuberka, P., Chilton, M., Pike, K. C., Shafran, R., & Yardley, L. (2026) · JMIR Pediatrics and Parenting · doi:10.2196/84384
“While many digital mental health interventions are publicly accessible, a substantial number have not undergone empirical evaluation [45,74,75], raising concerns about their effectiveness and safety.”
Metadata
- Authors
- Marshall, J. M., Dunstan, D. A., & Bartik, W.
- Affiliations
- University of New England (AU)
- Corresponding author
- Marshall, Jamie M
- Venue
- JMIR Mental Health
- Publisher
- JMIR Publications Inc.
- Published
- 2020-06-24
- Volume
- 7
- Issue
- 6
- Pages
- e16525
- ISSN
- 2368-7959 (ISSN-L)
- PMC
- PMC7381031
- OpenAlex
- W3037015118
- Cited by
- 83
- References
- 65
- Citations by year
- 2026: 7 · 2025: 9 · 2024: 18 · 2023: 11 · 2022: 18 · 2021: 18 · 2020: 2
- FWCI
- 11.07
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Mental Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RUNE (Research UNE) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Keywords
- mental health apps, anxiety, depression, evidence-based framework, cognitive behavioral therapy, mindfulness, positive psychology, dialectical behavior therapy, acceptance and commitment therapy, app efficacy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.