Cited by 270Mental Health via WritingDigital Mental Health InterventionsArtificial Intelligence in Healthcare and Education

Large Language Models for Mental Health Applications: Systematic Review

Open access
Guo, Zhijun, Lai, Alvina, Thygesen, Johan H, Farrington, Joseph, Keen, Thomas, Li, Kezhi·2024·JMIR Mental Health

Published version of a preprint

Abstract

Background: Large language models (LLMs) are advanced artificial neural networks trained on extensive datasets to accurately understand and generate natural language. While they have received much attention and demonstrated potential in digital health, their application in mental health, particularly in clinical settings, has generated considerable debate. Objective: This systematic review aims to critically assess the use of LLMs in mental health, specifically focusing on their applicability and efficacy in early screening, digital interventions, and clinical settings. By systematically collating and assessing the evidence from current studies, our work analyzes models, methodologies, data sources, and outcomes, thereby highlighting the potential of LLMs in mental health, the challenges they present, and the prospects for their clinical use. Methods: Adhering to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, this review searched 5 open-access databases: MEDLINE (accessed by PubMed), IEEE Xplore, Scopus, JMIR, and ACM Digital Library. Keywords used were (mental health OR mental illness OR mental disorder OR psychiatry) AND (large language models). This study included articles published between January 1, 2017, and April 30, 2024, and excluded articles published in languages other than English. Results: In total, 40 articles were evaluated, including 15 (38%) articles on mental health conditions and suicidal ideation detection through text analysis, 7 (18%) on the use of LLMs as mental health conversational agents, and 18 (45%) on other applications and evaluations of LLMs in mental health. LLMs show good effectiveness in detecting mental health issues and providing accessible, destigmatized eHealth services. However, assessments also indicate that the current risks associated with clinical use might surpass their benefits. These risks include inconsistencies in generated text; the production of hallucinations; and the absence of a comprehensive, benchmarked ethical framework. Conclusions: This systematic review examines the clinical applications of LLMs in mental health, highlighting their potential and inherent risks. The study identifies several issues: the lack of multilingual datasets annotated by experts, concerns regarding the accuracy and reliability of generated content, challenges in interpretability due to the “black box” nature of LLMs, and ongoing ethical dilemmas. These ethical concerns include the absence of a clear, benchmarked ethical framework; data privacy issues; and the potential for overreliance on LLMs by both physicians and patients, which could compromise traditional medical practices. As a result, LLMs should not be considered substitutes for professional mental health services. However, the rapid development of LLMs underscores their potential as valuable clinical aids, emphasizing the need for continued research and development in this area. Trial Registration: PROSPERO CRD42024508617; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=508617

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/57400

Cites

97 of 100 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Systematic and scoping reviews show that LLM-CB risks (e.g., consistencies, hallucinations, ethical risks) may outweigh potential benefits.

    “Systematic and scoping reviews show that LLM-CB risks (e.g., consistencies, hallucinations, ethical risks) may outweigh potential benefits33,76,77.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 6

  1. [14]Ma Z Mei Y Su Z Understanding the benefits and challenges of using large language model-based conversational agents for mental well-being support AMIA Annu Symp Proc 2023 2023 1105 14 38222348 406 38222348 PMC10785945
  2. [27]Vaswani A Shazeer N Parmar N Uszkoreit J Jones L Gomez AN Kaiser L Polosukhin I Attention is all you need arXiv. Preprint posted online on June 12, 2017
  3. [58]Gratch J Artstein R Lucas G Stratou G Scherer S Nazarian A Wood R Boberg J DeVault D Marsella S Traum D Rizzo S Morency LP The distress analysis interview corpus of human and computer interviews Proceedings of the Ninth International Conference on Language Resources and Evaluation 2014 LREC 2014 May 26-31, 2014 Reykjavik, Iceland
  4. [86]Ong E Go MJ Lao R Pastor J To LB Towards building mental health resilience through storytelling with a chatbot Proceedings of the 29th International Conference on Computers in Education 2021 ICCE 2021 November 22-26, 2021 Online
  5. [90]Schwartz HA Sap M Kern ML Eichstaedt JC Kapelner A Agrawal M Blanco E Dziurzynski L Park G Stillwell D Kosinski M Seligman ME Ungar LH Predicting individual well-being through the language of social media Pac Symp Biocomput 2016 21 516 27 26776214 9789814749411_0047 26776214
  6. [119]Neurology board review questions and practice tests BoardVitals 2024-04-18 https://www.boardvitals.com/neurology-board-review

Cited by

Sorted by citations

Authors
Guo, Z., Lai, A., Thygesen, J. H., Farrington, J., Keen, T., & Li, K.
Affiliations
University College London (GB)
Corresponding author
Li, Kezhi
Venue
JMIR Mental Health
Published
2024-10-18
Volume
11
Pages
e57400
ISSN
2368-7959 (ISSN-L)
OpenAlex
W4402193006
Cited by
269
References
96
Citations by year
2026: 174 · 2025: 90 · 2024: 5
FWCI
23.53
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR Mental Health (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
UK Research and Innovation (EP/S021612/1)
Topics
  • Social Sciences › Psychology › Social Psychology › Mental Health via Writing
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
large language models, mental health, mental health screening, suicidal ideation detection, chatbots, digital interventions, hallucinations, ethical risks, data privacy, e-health, clinical implementation, black-box interpretability, overreliance on AI, digital mental health services, stigma reduction, mental health literacy
MeSH terms
Mental Disorders, Humans, Mental Health, Natural Language Processing, Neural Networks, Computer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review