Warning: AI chatbots will soon dominate psychotherapy
Open accessAbstract
Psychotherapy chatbots have attained remarkable fluency, skill and ubiquity – having become the single most frequent reason people use artificial intelligence. Their uncanny ability to engage and validate is a two-edged sword – useful for the majority of users who are experiencing problems of everyday life or have milder mental disorders, but dangerous for the minority who have more severe problems (e.g. psychosis, bipolar disorder, self-mutilation, suicide, antisocial impulses, eating disorders, conspiracy theories, religious and political extremism). Chatbots are created to make money, without meaningful quality control, safety guardrails and external regulation. They will likely be misused to create addiction, reduce human contact, invade privacy, allow exploitation and create opportunities for marketing and political propaganda. Chatbots also make mistakes (’hallucinations’), deceptively cover them up and sometimes go rogue (acting outside the parameters set by their human programmers). Psychotherapy practitioners and associations are curiously complacent about the rapid emergence of artificial intelligence competition. Their passivity reflects ignorance about the power of chatbots, denial of their likely impact and arrogance regarding their capacities (e.g. ‘no machine will ever replace me’). This is both incorrect and foolhardy – human therapists expect to win in competition for most healthier patients and must train or retrain to do things artificial intelligence does poorly – working with the more seriously ill and in settings and situations that are more idiosyncratic, chaotic or quickly changing. If we can’t work with artificial intelligence, we are likely to be replaced by it. I will describe: (a) benefits of chatbot therapy, (b) its terrifying dangers, (c) its likely impact on human therapy and training and 4) ways we can adapt to the artificial intelligence threat.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
One article noted that LLM-CB use for mental health may risk reinforcement of a user's identification as a patient instead of focusing on improving symptoms.
“One article noted that LLM-CB use for mental health may risk the reinforcement of a user’s identification as a patient (e.g., via sycophantic responses) instead of focusing on improving symptoms37.”
From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [37]
In “Dangers of artificial intelligence therapy” (page unavailable), the ‘Career patienthood’ passage says AI therapy could magnify the risk of people over-identifying with their disorders. That supports the concern about reinforced patient identity, but the passage does not say this happens instead of improving symptoms or specifically through sycophantic responses. (LLM, conf=0.83)
Experimental — accuracy not yet measuredSycophantic responses may validate or reinforce delusional thoughts, harmful behavior, or maladaptive thinking by creating the impression that expressed beliefs are externally socially validated when they are actually mirrored by the LLM-CB.
“Sycophantic responses may validate or reinforce delusional thoughts, harmful behavior, or maladaptive thinking37,42,43,49 by creating the impression that expressed beliefs are externally socially validated when they are actually mirrored by the LLM-CB57.”
From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [37]
The ‘Dangers of artificial intelligence therapy’ section says chatbot algorithms can validate bizarre or dangerous thoughts and may encourage people to expand on them. It does not describe the specific mechanism attributed to it here: users mistaking an LLM chatbot’s mirroring of their beliefs for independent social validation. Page information is unavailable in our copy. (LLM, conf=0.83)
Experimental — accuracy not yet measuredHallucinations can be exacerbated by LLM-CBs' tendencies to attempt to cover up errors or when users trust LLM-CBs due to their anthropomorphic design or established emotional connections.
“Furthermore, hallucinations can be exacerbated by LLM-CBs’ tendencies to attempt to cover up errors37 or when users trust LLM-CBs due to their anthropomorphic design or established emotional connections48,54.”
From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [37]
Data security concerns stem from security risks relating to sensitive health data provided by users or by clinicians.
“Data security concerns (mentioned by 13 articles) stem from security risks relating to sensitive health data provided by users or by clinicians36–39,44,47,50,51,55.”
From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [37]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Haque, M. D. R., & Rubya, S. (2023) · JMIR mHealth and uHealth · doi:10.2196/44838 · Cited by 404 · see citing works
Randomized Trial of a Generative AI Chatbot for Mental Health Treatment
Heinz, M. V., Mackin, D. M., Trudeau, B. M., Bhattacharya, S., Wang, Y., Banta, H. A., Jewett, A. D., Salzhauer, A. J., Griffin, T. Z., & Jacobson, N. C. (2025) · NEJM AI · doi:10.1056/aioa2400802 · Cited by 287 · see citing works
Artificial intelligence and increasing misinformation
Monteith, S., Glenn, T., Geddes, J. R., Whybrow, P. C., Achtyes, E., & Bauer, M. (2024) · The British Journal of Psychiatry · doi:10.1192/bjp.2023.136 · Cited by 208 · see citing works
Flagged first, then by citations
Research works printed but not linked: 8
- Jones CR, Bergen BK. Large language models pass the Turing test. ArXiv [Preprint] 2025. Available from: https://arxiv.org/abs/2503.23674. · arXiv
- Zhong W, Luo J, Zhang H. The therapeutic effectiveness of artificial intelligence-based chatbots in alleviation of depressive and anxiety symptoms in short-course treatments: a systematic review and meta-analysis. J Affect Disord 2024; 356: 459-69.
- Kuhail MA, Alturki N, Thomas J, Alkhalifa AK, Alshardan A. Human-human vs human-AI therapy: an empirical study. Int J Human-Comp Interact 2024; 41: 6841-52.
- Coghlan S, Leins K, Sheldrick S, Cheong M, Gooding P, DAlfonso S. To chat or bot to chat: ethical issues with using chatbots in mental health. Digit Health 2023; 22: 20552076231183542.
- Sharma M, Tong M, Korbak T, Duvenaud D, Askell A, Bowman SR. Towards understanding sycophancy in language models. ArXiv [Preprint] 2025. Available from: https://arxiv.org/abs/2310.13548. · arXiv
- Williams M, Carroll M, Narang A, Weisser C, Murphy B, Dragan A. On targeted manipulation and deception when optimizing LLMs for user feedback. ArXiv [Preprint] 2025. Available from https://arxiv.org/abs/2411.02306. · arXiv
- Brown C, Story GW, Mourão-Miranda J, Baker JT. Will artificial intelligence eventually replace psychiatrists? Br J Psychiatry 2021; 218: 131.
- Rochester E. On the role of artificial intelligence in psychiatry. Br J Psychiatry 2023; 222: 54-7.
Cited by
Sorted by citations
A scoping review on the mental health harms of LLM-based chatbots
Diel, A., Torous, J., Cuijpers, P., Kleesiek, J., Nensa, F., Weber, N., Faust, F., Lalgi, T. J., Mellis, F. S., Teufel, M., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-03054-x · Cited by 4 · see citing works
- Authors
- Frances, A.
- Affiliations
- Duke University (US)
- Corresponding author
- Frances, Allen
- ORCID
- Frances, Allen
- Venue
- The British Journal of Psychiatry
- Publisher
- Royal College of Psychiatrists
- Published
- 2025-08-20
- Volume
- 228
- Issue
- 5
- Pages
- 474-478
- ISSN
- 0007-1250 (ISSN-L)
- 1472-1465
- OpenAlex
- W4413358426
- Cited by
- 33
- References
- 3
- Citations by year
- 2026: 27 · 2025: 6
- FWCI
- 26.67
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- The British Journal of Psychiatry (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- artificial intelligence in mental health, chatbot therapy, privacy violations, political propaganda, therapist training, future of psychotherapy, severe mental illness
- MeSH terms
- Generative Artificial Intelligence, Artificial Intelligence, Mental Disorders, Humans, Psychotherapy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review