CoChat
Sign Up
Cited by 68Digital Mental Health InterventionsMobile Health and mHealth ApplicationsArtificial Intelligence in Healthcare and Education

Artificial intelligence and machine learning in mobile apps for mental health: A scoping review

Open access
Milne-Ives, Madison, Selby, Emma, Inkster, Becky, Lam, Ching, Meinert, Edward·2022·PLOS Digital Health
DOI 10.1371/journal.pdig.0000079PMC PMC9931284
Open this paper in CoChat — free accountOpen PDF

Abstract

Mental health conditions can have significant negative impacts on wellbeing and healthcare systems. Despite their high prevalence worldwide, there is still insufficient recognition and accessible treatments. Many mobile apps are available to the general population that aim to support mental health needs; however, there is limited evidence of their effectiveness. Mobile apps for mental health are beginning to incorporate artificial intelligence and there is a need for an overview of the state of the literature on these apps. The purpose of this scoping review is to provide an overview of the current research landscape and knowledge gaps regarding the use of artificial intelligence in mobile health apps for mental health. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) and Population, Intervention, Comparator, Outcome, and Study types (PICOS) frameworks were used to structure the review and the search. PubMed was systematically searched for randomised controlled trials and cohort studies published in English since 2014 that evaluate artificial intelligence- or machine learning-enabled mobile apps for mental health support. Two reviewers collaboratively screened references (MMI and EM), selected studies for inclusion based on the eligibility criteria and extracted the data (MMI and CL), which were synthesised in a descriptive analysis. 1,022 studies were identified in the initial search and 4 were included in the final review. The mobile apps investigated incorporated different artificial intelligence and machine learning techniques for a variety of purposes (risk prediction, classification, and personalisation) and aimed to address a wide range of mental health needs (depression, stress, and suicide risk). The studies’ characteristics also varied in terms of methods, sample size, and study duration. Overall, the studies demonstrated the feasibility of using artificial intelligence to support mental health apps, but the early stages of the research and weaknesses in the study designs highlight the need for more research into artificial intelligence- and machine learning-enabled mental health apps and stronger evidence of their effectiveness. This research is essential and urgent, considering the easy availability of these apps to a large population.

AI snapshot

Written from the paper’s full text.

TL;DR

This scoping review maps research on AI/ML-enabled mobile apps for mental health and finds mostly feasibility studies, with limited evidence of effectiveness. It included 17 studies (the abstract states 4) and calls for more rigorous RCTs.

Key findings

  • Across the 17 included studies, AI/ML was used mainly for prediction of mood, stress, and risk (8/17) and natural-language conversation for support (6/17); Random Forest was the most common classifier (5/17). Apps addressed depression (6/17), mood disorders (3/17), stress (3/17), general well-being (3/17), suicide risk (1/17), and broad diagnostic support (1/17).
  • Sample sizes ranged from 6 to 354; 12/17 studies had fewer than 100 participants, and all studies focused on adults. Only 3 RCTs were identified, all small-scale pilot RCTs.
  • 7/17 studies evaluated mental health as an outcome. Two found significant app-vs-control differences: Woebot reduced depression but not anxiety, and Circadian Rhythm for Mood reduced depressive, manic or hypomanic, and mood episodes. Three others found benefits linked to engagement/adherence, and one found no difference between “intelligent” and non-intelligent notifications.
  • The review concludes that feasibility is demonstrated, but evidence of effectiveness remains preliminary and limited.

Methods

  • Scoping review using PRISMA-ScR and PICOS; PubMed was searched on 1 March 2021 for English-language studies published since 2014 evaluating AI/ML-enabled mobile apps for mental health support.
  • Inclusion allowed any age and interventional, observational, or validation designs evaluating an AI/ML mental health app; exclusions included substance abuse interventions, protocols, reviews, meta-analyses, perspectives, and design/development descriptions.
  • Two reviewers screened references and extracted data; synthesis was descriptive. In accordance with scoping review standards, methodological quality and risk of bias of included studies were not assessed.

Limitations

  • Stated: only one database was searched, so relevant studies may have been overlooked; the review was not conducted by two independent reviewers, lacking independent validation of screening and analysis.
  • Stated: methodological quality and risk of bias of the included studies were not examined.
  • Evident: the abstract reports 4 included studies while the results and discussion report 17; included studies had small samples (6–354), short intervention periods (5/12 under one month; only 3/12 longer than one year), and heterogeneous designs, limiting the strength of conclusions.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1371/journal.pdig.0000079

Cites

40 of 40 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 149 cited sources checked · 0 supported · 0 flagged

  • Findingnone given

    All of the studies focused on adults.

    “All of the studies focused on adults.”
    Quote in the paperNot a substantial claim
  • Findingown data

    The second most common use of artificial intelligence in the mental health apps was natural language conversations to deliver mental health support (6/17 studies).

    “and natural language conversations to deliver mental health support (6/17 studies) [29,30,38,40,42,43].”
    Quote in the paperNot a substantial claim0 of 6 sources checked
    • cites [29] KK Fitzpatrick, 2017, Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial, JMIR Ment Health source

      Not checked in the free run.

    • cites [30] KH Ly, 2017, A fully automated conversational agent for promoting mental well-being: A pilot RCT using mixed methods, Internet Interv source

      Not checked in the free run.

    • cites [38] V Ta, 2020, User Experiences of Social Support From Companion Chatbots in Everyday Contexts: Thematic Analysis., J Med Internet Res source

      Not checked in the free run.

    • cites [40] G Dosovitsky, 2020, Artificial Intelligence Chatbot for Depression: Descriptive Study of Usage., JMIR Form Res source

      Not checked in the free run.

    • cites [42] EP Green, 2020, Expanding Access to Perinatal Depression Treatment in Kenya Through Automated Psychological Support: Development and Usability Study., JMIR Form Res source

      Not checked in the free run.

    • cites [43] B Inkster, 2018, An Empathy-Driven, Conversational Artificial Intelligence Agent (Wysa) for Digital Mental Well-Being: Real-World Data Evaluation Mixed-Methods Study, JMIR Mhealth Uhealth source

      Not checked in the free run.

  • Findingown data

    None of the systematic reviews that were identified examined evidence for the use of AI in mobile apps for mental health.

    “None of the systematic reviews that were identified examined evidence for the use of AI in mobile apps for mental health.”
    Quote in the paperSubstantial claim
  • Findingcitation

    In some studies, AI/ML was used to personalise aspects of the app such as notification delivery.

    “or to personalise aspects of the app such as notification delivery [28]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [28] LG Morrison, 2017, The Effect of Timing and Frequency of Push Notifications on Usage of a Smartphone-Based Stress Management Intervention: An Exploratory Trial, PLoS One source

      Not checked in the free run.

  • Findingown data

    The most common use of artificial intelligence in the mental health apps was prediction (of mood, stress, and risk) (8/17 studies).

    “The most common uses of artificial intelligence in the mental health apps were prediction (of mood, stress, and risk) (8/17 studies) [31–37,41]”
    Quote in the paperNot a substantial claim0 of 8 sources checked
    • cites [31] A Sarda, 2019, Using Passive Smartphone Sensing for Improved Risk Stratification of Patients With Depression and Diabetes: Cross-Sectional Observational Study., JMIR mHealth and uHealth source

      Not checked in the free run.

    • cites [32] A Sano, 2018, Identifying Objective Physiological Markers and Modifiable Behaviors for Self-Reported Stress and Mental Health Status Using Wearable Sensors and Mobile Phones: Observational Study., J Med Internet Res source

      Not checked in the free run.

    • cites [33] A Rozet, 2019, Using Machine Learning to Derive Just-In-Time and Personalized Predictors of Stress: Observational Study Bridging the Gap Between Nomothetic and Ideographic Approaches., J Med Internet Res source

      Not checked in the free run.

    • cites [34] C-H Cho, 2019, Mood Prediction of Patients With Mood Disorders by Machine Learning Using Passive Digital Phenotypes Based on the Circadian Rhythm: Prospective Observational Cohort Study., J Med Internet Res source

      Not checked in the free run.

    • cites [35] C-H Cho, 2020, Effectiveness of a Smartphone App With a Wearable Activity Tracker in Preventing the Recurrence of Mood Disorders: Prospective Case-Control Study., JMIR Ment Health source

      Not checked in the free run.

    • cites [36] A Mikus, 2018, Predicting short term mood developments among depressed patients using adherence and ecological momentary assessment data, Internet Interv source

      Not checked in the free run.

    • cites [37] J Busk, 2020, Forecasting Mood in Bipolar Disorder From Smartphone Self-assessments: Hierarchical Bayesian Approach., JMIR mHealth and uHealth source

      Not checked in the free run.

    • cites [41] A Haines-Delmont, 2020, Testing Suicide Risk Prediction Algorithms Using Phone Measurements With Patients in Acute Mental Health Settings: Feasibility Study, JMIR mHealth and uHealth source

      Not checked in the free run.

  • Classified as probably not a substantial claim. 23
  • May combine two claims. 18

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    After duplicates (n = 206) were removed using EndNote X9, 1,022 articles were screened using EndNote's keyword search.

    “After duplicates (n = 206) were removed using EndNote X9, 1,022 articles were screened using EndNote’s keyword search.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    A second screening with revised keyword searches identified 17 articles for full text review; 2 were already included, and 13 were determined to be eligible for inclusion, resulting in a total of 17 articles included in the final review.

    “A second screening was conducted with revised keyword searches and identified 17 articles for full text review (see S3 Appendix); 2 were already included, and 13 were determined to be eligible for inclusion, resulting in a total of 17 articles included in the final review (see Fig 1).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The search of PubMed retrieved 1,228 articles.

    “The search of PubMed retrieved 1,228 articles (see S2 Appendix).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    The sample sizes included in the studies ranged from 6 to 354, but the number analysed was often lower than the recruited sample.

    “The sample sizes included in the studies ranged from 6 [39] to 354 [40], but the number analysed was often lower than the recruited sample.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [39] SM Jungmann, 2019, Accuracy of a Chatbot (Ada) in the Diagnosis of Mental Disorders: Comparative Case Study With Lay and Expert Users, JMIR Formative Research source

      Not checked in the free run.

    • cites [40] G Dosovitsky, 2020, Artificial Intelligence Chatbot for Depression: Descriptive Study of Usage., JMIR Form Res source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    14 articles remained after 6 rounds of screening and 4 articles were determined to be eligible for final inclusion upon title/abstract and full-text review.

    “14 articles remained after 6 rounds of screening (see S3 Appendix) and 4 articles were determined to be eligible for final inclusion upon title/abstract and full-text review.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    One study aimed to identify suicide risk in inpatients.

    “one aimed to identify suicide risk in inpatients [41]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [41] A Haines-Delmont, 2020, Testing Suicide Risk Prediction Algorithms Using Phone Measurements With Patients in Acute Mental Health Settings: Feasibility Study, JMIR mHealth and uHealth source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Many of the studies successfully modelled data collected by the app or associated inputs to identify behaviours or responses associated with mental health conditions, provide natural language support, develop risk profiles, and deploy context-sensitive notifications or interventions.

    “Many of the studies successfully modelled data collected by the app or associated inputs to identify behaviours or responses associated with mental health conditions [44], provide natural language support [29,30,38,40,42,43], develop risk profiles [31–37,41], and deploy context-sensitive notifications [28] or interventions [44].”
    Quote in the paperSubstantial claim0 of 10 sources checked
    • cites [44] F Wahle, 2016, Mobile Sensing and Support for People With Depression: A Pilot Trial in the Wild., JMIR mHealth and uHealth

      Not checked in the free run.

    • cites [29] KK Fitzpatrick, 2017, Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial, JMIR Ment Health source

      Not checked in the free run.

    • cites [30] KH Ly, 2017, A fully automated conversational agent for promoting mental well-being: A pilot RCT using mixed methods, Internet Interv source

      Not checked in the free run.

    • cites [38] V Ta, 2020, User Experiences of Social Support From Companion Chatbots in Everyday Contexts: Thematic Analysis., J Med Internet Res source

      Not checked in the free run.

    • cites [40] G Dosovitsky, 2020, Artificial Intelligence Chatbot for Depression: Descriptive Study of Usage., JMIR Form Res source

      Not checked in the free run.

    • cites [42] EP Green, 2020, Expanding Access to Perinatal Depression Treatment in Kenya Through Automated Psychological Support: Development and Usability Study., JMIR Form Res source

      Not checked in the free run.

    • cites [43] B Inkster, 2018, An Empathy-Driven, Conversational Artificial Intelligence Agent (Wysa) for Digital Mental Well-Being: Real-World Data Evaluation Mixed-Methods Study, JMIR Mhealth Uhealth source

      Not checked in the free run.

    • cites [31] A Sarda, 2019, Using Passive Smartphone Sensing for Improved Risk Stratification of Patients With Depression and Diabetes: Cross-Sectional Observational Study., JMIR mHealth and uHealth source

      Not checked in the free run.

    • cites [32] A Sano, 2018, Identifying Objective Physiological Markers and Modifiable Behaviors for Self-Reported Stress and Mental Health Status Using Wearable Sensors and Mobile Phones: Observational Study., J Med Internet Res source

      Not checked in the free run.

    • cites [33] A Rozet, 2019, Using Machine Learning to Derive Just-In-Time and Personalized Predictors of Stress: Observational Study Bridging the Gap Between Nomothetic and Ideographic Approaches., J Med Internet Res source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Random Forest classifier was used for feature selection, risk-stratification of diabetic patients based on symptoms of depression, inferring context for depression level prediction, and predicting mood in the included studies.

    “It is used for feature selection [41], risk-stratification of diabetic patients based on symptoms of depression [31], inferring context for depression level prediction [44], and predicting mood [34,35].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [41] A Haines-Delmont, 2020, Testing Suicide Risk Prediction Algorithms Using Phone Measurements With Patients in Acute Mental Health Settings: Feasibility Study, JMIR mHealth and uHealth source

      Not checked in the free run.

    • cites [31] A Sarda, 2019, Using Passive Smartphone Sensing for Improved Risk Stratification of Patients With Depression and Diabetes: Cross-Sectional Observational Study., JMIR mHealth and uHealth source

      Not checked in the free run.

    • cites [44] F Wahle, 2016, Mobile Sensing and Support for People With Depression: A Pilot Trial in the Wild., JMIR mHealth and uHealth

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    The studies all recognized the need for further research, highlighting the early stage of the state of the literature.

    “The studies all recognized the need for further research, highlighting the early stage of the state of the literature.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    This review shows the diversity and feasibility of the use of artificial intelligence to support mental health care in a variety of different ways.

    “This review shows the diversity and feasibility of the use of artificial intelligence to support mental health care in a variety of different ways.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingreasoning

    Several strengths of using AI in mental health apps were identified from these studies.

    “Several strengths of using AI in mental health apps were identified from these studies.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    Two studies only reported user experiences; both found generally positive perceptions of the app in question.

    “Two studies only reported user experiences; both found generally positive perceptions of the app in question [38,42].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [38] V Ta, 2020, User Experiences of Social Support From Companion Chatbots in Everyday Contexts: Thematic Analysis., J Med Internet Res source

      Not checked in the free run.

    • cites [42] EP Green, 2020, Expanding Access to Perinatal Depression Treatment in Kenya Through Automated Psychological Support: Development and Usability Study., JMIR Form Res source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    The studies included in the review demonstrated the potential feasibility of incorporating AI/ML into mental health apps as well as the variety of applications of AI/ML in mental health care to provide support to individuals and an overburdened healthcare system.

    “The studies included in the review demonstrated the potential feasibility of incorporating AI/ML into mental health apps as well as the variety of applications of AI/ML in mental health care to provide support to individuals and an overburdened healthcare system.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    The sample sizes of the studies were relatively low, ranging from 6 to 354, with the latter being a retrospective descriptive study of real-world users, not recruited participants.

    “The sample sizes of the studies were relatively low, ranging from 6 [39] to 354 [40], with the latter being a retrospective descriptive study of real-world users, not recruited participants.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [39] SM Jungmann, 2019, Accuracy of a Chatbot (Ada) in the Diagnosis of Mental Disorders: Comparative Case Study With Lay and Expert Users, JMIR Formative Research source

      Not checked in the free run.

    • cites [40] G Dosovitsky, 2020, Artificial Intelligence Chatbot for Depression: Descriptive Study of Usage., JMIR Form Res source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Although the studies demonstrated feasibility and potential for the use of AI in mental health apps, they had a number of significant limitations.

    “Although the studies demonstrated feasibility and potential for the use of AI in mental health apps, they had a number of significant limitations.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    A large body of research was initially identified, but the number of randomised control trials evaluating AI/ML technologies incorporated in mental health apps was limited.

    “A large body of research was initially identified, but the number of randomised control trials evaluating AI/ML technologies incorporated in mental health apps was limited.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Mental health conditions, such as anxiety and depression, can have significant negative impacts on a range of mental and physical wellbeing, social, and employment outcomes.

    “Mental health conditions, such as anxiety and depression, can have significant negative impacts on a range of mental and physical wellbeing, social, and employment outcomes [1,2].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [1] The Improving Access to Psychological Therapies (IAPT) Pathway for People with Long-term Physical Health Conditions and Medically Unexplained Symptoms. NHS England; 2018.

      Not checked in the free run.

    • cites [2] C Naylor, 2012, Long-term conditions and mental health: The cost of co-morbidities, The King’s Fund

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In the UK, approximately a quarter of the population is seeking mental health treatment.

    “in the UK, approximately a quarter of the population is seeking mental health treatment [5].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [5] Adult Psychiatric Morbidity Survey: Survey of Mental Health and Wellbeing, England, 2014. NHS Digital 2016. https://digital.nhs.uk/data-and-information/publications/statistical/adult-psychiatric-morbidity-survey/adult-psychiatric-morbidity-survey-survey-of-mental-health-and-wellbeing-england-2014 (accessed March 24, 2021).

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Globally, the cost of mental health issues exceeds US$1 trillion each year.

    “Globally, this cost exceeds US$1 trillion each year [7].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] Mental health in the workplace. World Health Organisation n.d. https://www.who.int/teams/mental-health-and-substance-use/mental-health-in-the-workplace (accessed March 24, 2021).

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    AI/ML techniques have been widely applied in healthcare to generate insights from massive amounts of data.

    “AI/ML techniques have been widely applied in healthcare to generate insights from massive amounts of data [20–22]”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [20] RD Balicer, 2020, Advancing Healthcare Through Data-Driven Medicine and Artificial Intelligence., Healthcare and Artificial Intelligence source

      Not checked in the free run.

    • cites [21] F Jiang, 2017, Artificial intelligence in healthcare: past

      Not checked in the free run.

    • cites [22] M Chen, 2017, Disease Prediction by Machine Learning Over Big Data From Healthcare Communities, IEEE Access source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    People with severe, long-term mental illness have an average of 15 years shorter life expectancies than the general population.

    “People with severe, long-term mental illness have an average of 15 years shorter life expectancies than the general population [3].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [3] 2016, The Five Year Forward View for Mental Health, The Mental Health Taskforce

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Worldwide, there is a high prevalence of mental health issues and conditions.

    “Worldwide, there is a high prevalence of mental health issues and conditions [4];”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [4] Mental disorders. World Health Organisation n.d. https://www.who.int/news-room/fact-sheets/detail/mental-disorders (accessed March 24, 2021).

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    It is estimated that 75% of people who need mental health support do not receive it.

    “it is estimated that 75% of people who need mental health support do not receive it”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [3] 2016, The Five Year Forward View for Mental Health, The Mental Health Taskforce

      Not checked in the free run.

    • cites [6] J Alonso, 2018, Treatment gap for anxiety disorders is global: Results of the World Mental Health Surveys in 21 countries, Depress Anxiety source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The costs to the UK economy of unmet mental health support are approximately £100 billion annually.

    “resulting in costs to the UK economy of approximately £100 billion annually [3,6].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [3] 2016, The Five Year Forward View for Mental Health, The Mental Health Taskforce

      Not checked in the free run.

    • cites [6] J Alonso, 2018, Treatment gap for anxiety disorders is global: Results of the World Mental Health Surveys in 21 countries, Depress Anxiety source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The search strategy was developed using the PICOS framework.

    “The search strategy was developed using the Population, Intervention, Comparator, Outcome, and Study types (PICOS) framework (see Table 1).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The review was updated after a second screening of the retrieved studies to ensure that all of the relevant papers were included.

    “The review was updated after a second screening of the retrieved studies to ensure that all of the relevant papers were included.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    No separate protocol has been published for the review.

    “No separate protocol has been published.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The review used the PRISMA-ScR framework to structure the review.

    “The Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR; S1 Appendix) [25] was used to structure the review.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [25] AC Tricco, 2018, PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation., Ann Intern Med source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The remaining titles and abstracts were screened by two reviewers (MMI and EM) and a full-text review was conducted by two authors to determine final eligibility (MMI and CL).

    “The remaining titles and abstracts were screened by two reviewers (MMI and EM) and a full-text review was conducted by two authors to determine final eligibility (MMI and CL).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Data analysis was conducted collaboratively by two researchers (MMI and CL) to apply their different areas of expertise to the extraction and interpretation of the data.

    “Data analysis was conducted collaboratively by two researchers (MMI and CL) to apply their different areas of expertise to the extraction and interpretation of the data.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The research on AI/ML-enabled mental health apps is in early stages and the study designs have weaknesses.

    “but the early stages of the research and weaknesses in the study designs highlight the need for more research into artificial intelligence- and machine learning-enabled mental health apps and stronger evidence of their effectiveness.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Although generally positive, the evidence gathered by the studies was preliminary.

    “Although generally positive, the evidence gathered by the studies was preliminary.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    This demonstrates the potential benefit of this approach, but identifies the importance of maintaining user engagement with the intervention for any potential mental health benefits.

    “This demonstrates the potential benefit of this approach, but identifies the importance of maintaining user engagement with the intervention for any potential mental health benefits.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationcitation

    This demonstrates the importance of rigorously evaluating the effectiveness of the use of AI/ML for specific purposes and intended outcomes.

    “This demonstrates the importance of rigorously evaluating the effectiveness of the use of AI/ML for specific purposes and intended outcomes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Two reviewers collaborated on the screening and data analysis.

    “Although two reviewers collaborated on the screening and data analysis”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The PRISMA-ScR framework was used to guide the structure of the review and ensure the necessary components were addressed.

    “the PRISMA-ScR framework was used to guide the structure of the review and ensure the necessary components were addressed”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The review demonstrates a clear need for more high-quality randomised controlled trials of these apps, to evaluate whether they are actually achieving their intended purposes and providing a benefit to users.

    “However, the review demonstrates a clear need for more high-quality randomised controlled trials of these apps, to evaluate whether they are actually achieving their intended purposes and providing a benefit to users.”
    Quote in the paperSubstantial claim
37 of 37

  • The paper relies on this without support.

    The 17 included studies are representative of the current literature on AI/ML-enabled mental health apps.

    “Overall, the studies demonstrated the feasibility of using artificial intelligence to support mental health apps”

  • The paper relies on this without support.

    PROSPERO is a comprehensive registry such that finding no records means no registered systematic reviews exist.

    “A search of PROSPERO for registered reviews using the keywords “mental health apps” AND “AI OR artificial intelligence OR machine learning OR chatbot” also found no records.”

  • The paper relies on this without support.

    Several highly-used AI-enabled mental health apps are available on the Apple App Store and Google Play (e.g. Woebot, Reflectly, Wysa, Youper).

    “despite the availability of several highly-used AI-enabled mental health apps on the Apple App Store and Google Play (e.g. Woebot, Reflectly, Wysa, Youper).”

  • The paper relies on this without support.

    Mobile devices can collect active data from participants and capture passive sensing data, providing a rich body of information for analysis and to tailor the provision of support.

    “Mobile devices can collect active data from participants and capture passive sensing data, providing a rich body of information for analysis and to tailor the provision of support.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Abstract reports 4 included studies; results report 17

    Minor

    The abstract says 4 studies were included in the final review, but the results say 17 were included.

  • Data extraction is attributed to two reviewers and to one

    Minor

    The abstract attributes data extraction to two reviewers, but the data-extraction methods say one reviewer extracted the key data.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. A. de Jesus, 2018, Chatbots for Mental Health and Therapy–Comparing 5 Current Apps and Use Cases., Emerj
  2. F Jiang, 2017, Artificial intelligence in healthcare: past, 2, 230

Cited by

Sorted by citations

Related

Authors
Milne-Ives, M., Selby, E., Inkster, B., Lam, C., & Meinert, E.
Affiliations
  • University of Cambridge (GB)
  • University of Oxford (GB)
  • Harvard University (US)
Corresponding author
Meinert, Edward
ORCID
Venue
PLOS Digital Health
Publisher
Public Library of Science (PLoS)
Published
2022-08-15
Volume
1
Issue
8
Pages
e0000079
ISSN
2767-3170 (ISSN-L)
DOI
10.1371/journal.pdig.0000079
PMC
PMC9931284
OpenAlex
W4291798901
Semantic Scholar
ec7784de2a85710c36481e93d1cf10dba78022d4
Cited by
68
References
40
Citations by year
2026: 14 · 2025: 31 · 2024: 15 · 2023: 8
FWCI
11.92
Type
journal article
Language
en
Open access
Licence
Locations
  • PLOS Digital Health (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PEARL (University of Plymouth) (submittedVersion)
  • Apollo (University of Cambridge) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • Apollo
  • Apollo (cc-by)
Funders
National Institute for Health Research (AI_AWARD02176)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
artificial intelligence, machine learning, mobile health apps, mental health apps, depression, stress, suicide risk prediction, personalization, risk classification, digital mental health interventions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review