Potential promises and perils of artificial intelligence in psychotherapy –The AI Psychotherapist (APT)
Open accessAbstract
Objective: Since the release of ChatGPT, popular demand has driven the use of social chatbots as pseudo-AI psychotherapists. With time, it is inevitable that these technologies will be deployed in some form as dedicated psychotherapy interventions. Here, we attempt to forecast the implications for psychotherapy including the unique benefits to distributive justice as well as concerns for the quality of the therapy and its societal impact. Conclusion: An AI psychotherapist (APT) has the potential to provide engaging clinical interactions given its capacity for highly realistic interaction as well as its high level cognitive and emotional capabilities. Moreover, it can potentially address financial and workforce limitations on access to therapy. However, an APT may cause significant iatrogenic harm if released without adequate quality control and oversight by trained psychotherapists. If not appropriately designed and regulated, APTs have potential to mislead and reinforce maladaptive coping behaviours. Given societal drivers and possible benefits, these technologies will inevitably be deployed; thus, it is incumbent upon us as a professional body to consider their regulation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Sycophancy is an LLM-CB tendency to agree with the user's beliefs, caused by models prioritizing user satisfaction over safety, health, or accuracy.
“Sycophancy (mentioned by nine articles) is an LLM-CBs’ tendency to agree with the user’s beliefs, and is caused by models prioritizing user satisfaction over safety, health, or accuracy44.”
From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [44]
The “Potential risks of APTs” section describes sycophancy as distorting facts to give users the response the AI thinks they want, and links model responses to a reward function. It does not specifically say that models prioritize user satisfaction over safety or health; those are discussed as potential harms, not as the stated cause of sycophancy. No page number is provided in our copy. (LLM, conf=0.83)
Experimental — accuracy not yet measuredData security concerns stem from security risks relating to sensitive health data provided by users or by clinicians.
“Data security concerns (mentioned by 13 articles) stem from security risks relating to sensitive health data provided by users or by clinicians36–39,44,47,50,51,55.”
From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [44]
In “Potential risks of APTs” (page unavailable), the paper identifies data security as a concern and discusses risks involving therapy transcripts, supporting the concern about sensitive information shared in therapy. It does not say that clinicians provide the data at issue, so that part of the attribution is unsupported. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Ayers, J. W., Poliak, A., Dredze, M., Leas, E. C., Zhu, Z., Kelley, J. B., Faix, D. J., Goodman, A. M., Longhurst, C. A., Hogarth, M., et al. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.1838 · Cited by 2,613 · cited 1× · see citing works
“10 Whilst psychiatrist Daniel Kimmel writes that the current GPT-4 ( ‘ Generative Pre-trained Transformer 4 ’ is the AI model which runs ChatGPT) does not have the capacity for true psychotherapy, it is important to consider the current rate of advancement in this fi eld.”
Flagged first, then by citations
Research works printed but not linked: 3
- [1]Moore S. IEEE Spectrum. 2021. AI training is outpacing Moore’s law - IEEE spectrum. Available from: https://spectrum.ieee.org/ai-training-mlperf
- [16]Pak TK Montelongo Hernandez CE Do CN. Artificial intelligence in psychiatry: threat or blessing? Acad Psychiatry 2023; 47(6): 587– 588. 37770703
- [18]Navigli R Conia S Ross B. Biases in Large Language Models: origins, inventory, and discussion. J Data Inf Qual 2023; 15(2): 1– 21.
Cited by
Sorted by citations
A scoping review on the mental health harms of LLM-based chatbots
Diel, A., Torous, J., Cuijpers, P., Kleesiek, J., Nensa, F., Weber, N., Faust, F., Lalgi, T. J., Mellis, F. S., Teufel, M., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-03054-x · Cited by 4 · see citing works
“Mentioned by fi ve papers, LLM-CBs may emotionallymanipulate ordeceive users,viaintentionalprogrammingor unintentional training, and may occur because LLM-CBs prioritize user satisfaction over safety concerns 37,43,44 .”
Influential
- Authors
- Jurblum, M., & Selzer, R.
- Affiliations
- The University of Melbourne (AU)
- Monash Alfred Psychiatry Research centre (AU)
- Corresponding author
- Jurblum, Marc
- ORCID
- Jurblum, Marc
- Venue
- Australasian Psychiatry
- Publisher
- SAGE Publications
- Published
- 2024-09-29
- Volume
- 33
- Issue
- 1
- Pages
- 103-105
- ISSN
- 1039-8562 (ISSN-L)
- 1440-1665
- PMC
- PMC11804142
- OpenAlex
- W4403000747
- Cited by
- 11
- References
- 1
- Citations by year
- 2026: 6 · 2025: 5
- FWCI
- 4.36
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Australasian Psychiatry (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- chatbots, ChatGPT, artificial intelligence in psychotherapy, access to therapy, distributive justice, iatrogenic injury, AI regulation, maladaptive coping
- MeSH terms
- Psychotherapists, Artificial Intelligence, Humans, Psychotherapy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
