Illusions of intelligence, connection and reality: Perils of large-language AI models for people with severe mental illness
Abstract
For people with mental illnesses that impair reality testing, such as psychosis, severe depression and bipolar disorder, Artificial Intelligence (AI) Large-Language Models (LLMs) may represent threats to mental health. LLMs are unable to detect delusional beliefs, may encourage and validate delusions and cognitive distortions, miss opportunities to reinforce reality-based thinking, and exacerbate risks of self-harm and harm to others. Psychiatrists need to understand these risks of LLMs for people with severe mental illnesses, and educate patients and carers on avoiding these potential harms. Risk assessments need to be informed by an awareness of the inputs that patients receive from LLMs.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Sycophantic responses may validate or reinforce delusional thoughts, harmful behavior, or maladaptive thinking by creating the impression that expressed beliefs are externally socially validated when they are actually mirrored by the LLM-CB.
“Sycophantic responses may validate or reinforce delusional thoughts, harmful behavior, or maladaptive thinking37,42,43,49 by creating the impression that expressed beliefs are externally socially validated when they are actually mirrored by the LLM-CB57.”
From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [42]
Looi et al. noted several real-life cases of severe mental illness that LLM-CBs were not suitable to deal with or even reinforced.
“Looi et al. note several real-life cases of severe mental illness that LLM-CBs were not suitable to deal with or even reinforced42”
From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [42]
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 6 · Cited by 4
- Authors
- Looi, J. C., Allison, S., Bastiampillai, T., Reutens, S., & Looi, R. C.
- Affiliations
- Australian National University (AU)
- Flinders University (AU)
- UNSW Sydney (AU)
- Independent Scholar
- Corresponding author
- Looi, Jeffrey CL
- Venue
- Australasian Psychiatry
- Publisher
- SAGE Publications
- Published
- 2025-09-15
- Volume
- 34
- Issue
- 1
- Pages
- 5-7
- ISSN
- 1039-8562 (ISSN-L)
- 1440-1665
- OpenAlex
- W4414166896
- Cited by
- 4
- References
- 6
- Citations by year
- 2026: 4
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Australasian Psychiatry (publishedVersion)
- PubMed (publishedVersion)
- ANU Open Research (Australian National University) (acceptedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Keywords
- large language models, artificial intelligence, severe mental illness, psychosis, delusions, cognitive distortions, reality testing, self-harm, harm to others, risk assessment, patient and carer education
- MeSH terms
- Artificial Intelligence, Mental Disorders, Delusions, Humans, Language, Psychotic Disorders
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
