Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 85 of the 85 references it cites were found in a registry or an index.

Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation

Open access

Published version of a preprint

Abstract

Large language models (LLMs) such as Open AI’s GPT-4 (which power ChatGPT) and Google’s Gemini, built on artificial intelligence, hold immense potential to support, augment, or even eventually automate psychotherapy. Enthusiasm about such applications is mounting in the field as well as industry. These developments promise to address insufficient mental healthcare system capacity and scale individual access to personalized treatments. However, clinical psychology is an uncommonly high stakes application domain for AI systems, as responsible and evidence-based therapy requires nuanced expertise. This paper provides a roadmap for the ambitious yet responsible application of clinical LLMs in psychotherapy. First, a technical overview of clinical LLMs is presented. Second, the stages of integration of LLMs into psychotherapy are discussed while highlighting parallels to the development of autonomous vehicle technology. Third, potential applications of LLMs in clinical care, training, and research are discussed, highlighting areas of risk given the complex nature of psychotherapy. Fourth, recommendations for the responsible development and evaluation of clinical LLMs are provided, which include centering clinical science, involving robust interdisciplinary collaboration, and attending to issues like assessment, risk detection, transparency, and bias. Lastly, a vision is outlined for how LLMs might enable a new generation of studies of evidence-based interventions at scale, and how these studies may challenge assumptions about psychotherapy.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s44184-024-00056-z

Cites

85 of 85 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Methods for tailoring clinical large language models. Figure was designed using image components from Flaticon.com.
Fig. 1 Methods for tailoring clinical large language models. Figure was designed using image components from Flaticon.com.CC BY · PubMed Central
Fig. 2 Example clinical skills of large language models. Note. Figure was designed using image component from Flaticon.com.
Fig. 2 Example clinical skills of large language models. Note. Figure was designed using image component from Flaticon.com.CC BY · PubMed Central
Fig. 3 Stages of integrating large language models into psychotherapy. Figure was designed using image components from Flaticon.com.
Fig. 3 Stages of integrating large language models into psychotherapy. Figure was designed using image components from Flaticon.com.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 78 references

Research works printed but not linked: 3

  1. Markov, A. A. Essai d’une recherche statistique sur le texte du roman “Eugene Onegin” illustrant la liaison des epreuve en chain (‘Example of a statistical investigation of the text of “Eugene Onegin” illustrating the dependence between samples in chain’). Izvistia Imperatorskoi Akad. Nauk Bull. L’Academie Imp. Sci. St-Petersbourg 7, 153–162 (1913).
  2. Jurafsky, D. & Martin, J. H. N-gram language models. in Speech and language processing: An introduction to natural language processing, computational linguistics, and speech recognition (Pearson Prentice Hall, 2009).
  3. Norden, J. G. & Shah, N. R. What AI in health care can learn from the long road to autonomous vehicles. NEJM Catal. Innov. Care Deliv. https://doi.org/10.1056/CAT.21.0458 (2022). · doi:10.1056/cat.21.0458

Cited by

See all 394

Metadata

Authors
Stade, E. C., Stirman, S. W., Ungar, L. H., Boland, C. L., Schwartz, H. A., Yaden, D. B., Sedoc, J., DeRubeis, R. J., Willer, R., & Eichstaedt, J. C.
Affiliations
  • VA Palo Alto Health Care System (US)
  • University of Pennsylvania (US)
  • Stony Brook University (US)
  • Johns Hopkins University (US)
  • New York University (US)
  • Stanford University (US)
Corresponding author
  • Stade, Elizabeth C.
  • Eichstaedt, Johannes C.
Venue
npj Mental Health Research
Published
2024-04-02
Volume
3
Issue
1
Pages
12-12
ISSN
2731-4251 (ISSN-L)
OpenAlex
W4393397034
Cited by
391
References
78
Citations by year
2026: 182 · 2025: 152 · 2024: 54
FWCI
36.35
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Mental Health Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • PsyArXiv (OSF Preprints) (submittedVersion)
  • PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
Funders
  • National Institute of Mental Health (RF1-MH128785, R01-MH125702)
  • Stanford Institute for Human-Centered Artificial Intelligence, Stanford University
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
Keywords
large language models, psychotherapy, behavioral health services, responsible AI development, AI evaluation, risk detection, algorithmic bias, transparency, clinical psychology training, evidence-based interventions, access to mental health care

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation. https://app.cochat.ai/papers/10.1038/s44184-024-00056-z
The paper
Stade, E. C., Stirman, S. W., Ungar, L. H., Boland, C. L., Schwartz, H. A., Yaden, D. B., Sedoc, J., DeRubeis, R. J., Willer, R., & Eichstaedt, J. C. (2024). Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation. npj Mental Health Research. https://doi.org/10.1038/s44184-024-00056-z

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.