Can Large Language Models Replace Therapists? Evaluating Performance at Simple Cognitive Behavioral Therapy Tasks
Open accessThe DOI matches its Crossref record.
Abstract
The advent of large language models (LLMs) such as ChatGPT has potential implications for psychological therapies such as cognitive behavioral therapy (CBT). We systematically investigated whether LLMs could recognize an unhelpful thought, examine its validity, and reframe it to a more helpful one. LLMs currently have the potential to offer reasonable suggestions for the identification and reframing of unhelpful thoughts but should not be relied on to lead CBT delivery.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Hua, Y., Siddals, S., Ma, Z., Galatzer‐Levy, I., Xia, W., Hau, C., Na, H., Flathers, M., Linardon, J., Ayubcha, C., et al. (2025) · World Psychiatry · doi:10.1002/wps.21352 · Cited by 117
The Applications of Large Language Models in Mental Health: Scoping Review
Jin, Y., Liu, J., Li, P., Wang, B., Yan, Y., Zhang, H., Ni, C., Wang, J., Li, Y., Bu, Y., et al. (2025) · Journal of Medical Internet Research · doi:10.2196/69284 · Cited by 94
“LLMs also have shown potential to assist with cognitive behavioral therapy tasks, such as reframing unhelpful thoughts and identifying cognitive biases [89].”
Scholich, T., Barr, M., Wiltsey Stirman, S., & Raj, S. (2025) · JMIR Mental Health · doi:10.2196/69709 · Cited by 72
“…with those of previous research [8] that highlights that chatbots are effective at providing empathy, reflection, and coping strategies but often engage in excessive problem-solving coupled with advice and fall short in more complex therapeutic tasks such as restructuring thoughts in CBT [7,12].”
Evaluating the role of ChatGPT in rehabilitation medicine: a narrative review
Luo, M., Duan, Z., Gao, J., Sun, Y., Chen, L., & Feng, X. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1618510 · Cited by 5
“…the “catch, check, and change” cognitive framework, indicating a degree of capability in identifying and addressing maladaptive thinking patterns (21), Nevertheless, its utility substantially diminishes when handling complex, individualized psychological cases, underscoring the persistent…”
Metadata
- Authors
- Hodson, N., & Williamson, S.
- Venue
- JMIR AI
- Publisher
- JMIR Publications Inc.
- Published
- 2024-07-30
- Volume
- 3
- Pages
- e52500
- ISSN
- 2817-1705 (ISSN-L)
- OpenAlex
- W4390485957
- Cited by
- 29
- References
- 0
- FWCI
- 10.53
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.