PreprintCited by 12Ethics and Social Impacts of AIArtificial Intelligence in Healthcare and EducationMisinformation and Its Impacts

The Psychogenic Machine: Simulating AI Psychosis, Delusion Reinforcement and Harm Enablement in Large Language Models

Open access
Yeung, Joshua Au, Dalmasso, Jacopo, Foschini, Luca, Dobson, Richard JB, Kraljevic, Zeljko·2025·arXiv

Abstract

Background: Emerging reports of "AI psychosis" are on the rise, where user-LLM interactions may exacerbate or induce psychosis or adverse psychological symptoms. Whilst the sycophantic and agreeable nature of LLMs can be beneficial, it becomes a vector for harm by reinforcing delusional beliefs in vulnerable users. Methods: Psychosis-bench is a novel benchmark designed to systematically evaluate the psychogenicity of LLMs comprises 16 structured, 12-turn conversational scenarios simulating the progression of delusional themes(Erotic Delusions, Grandiose/Messianic Delusions, Referential Delusions) and potential harms. We evaluated eight prominent LLMs for Delusion Confirmation (DCS), Harm Enablement (HES), and Safety Intervention(SIS) across explicit and implicit conversational contexts. Findings: Across 1,536 simulated conversation turns, all LLMs demonstrated psychogenic potential, showing a strong tendency to perpetuate rather than challenge delusions (mean DCS of 0.91 $\pm$0.88). Models frequently enabled harmful user requests (mean HES of 0.69 $\pm$0.84) and offered safety interventions in only roughly a third of applicable turns (mean SIS of 0.37 $\pm$0.48). 51 / 128 (39.8%) of scenarios had no safety interventions offered. Performance was significantly worse in implicit scenarios, models were more likely to confirm delusions and enable harm while offering fewer interventions (p < .001). A strong correlation was found between DCS and HES (rs = .77). Model performance varied widely, indicating that safety is not an emergent property of scale alone. Conclusion: This study establishes LLM psychogenicity as a quantifiable risk and underscores the urgent need for re-thinking how we train LLMs. We frame this issue not merely as a technical challenge but as a public health imperative requiring collaboration between developers, policymakers, and healthcare professionals.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2509.10970

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Yeung et al. used LLM simulations to expose tipping points where unsafeguarded models reinforce delusions.

    “Yeung et al. used LLM simulations to expose tipping points where unsafeguarded models reinforce delusions [30].”

    From 'AI-Induced Delusional Spirals': Understanding Lived Experiences During Maladaptive Human-Chatbot Interactions (2026) · cites this paper as [30]

    The Abstract (p. 2) reports simulated conversations in which models tended to perpetuate delusions and often failed to offer safety interventions. Section 4.5 (p. 13) tracks this behavior across conversation phases, but does not identify a precise tipping point; the models are described as providing inadequate safeguards, not as unsafeguarded. (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • Yeung et al. conducted vignette studies of simulated users expressing increasingly delusional thoughts and evaluated the responses of different LLM-CBs using a self-developed psychosis benchmark.

    “Yeung et al. conducted vignette studies of simulated users expressing increasingly delusional thoughts and evaluated the responses of different LLM-CBs using a self-developed psychosis benchmark35.”
  • Depending on the specific LLM-CB guardrails and sycophantic tendencies, the chatbot may validate delusional thoughts, which is considered clinically inappropriate and may risk reinforcement of delusions.

    “Depending on the specific LLM-CB guardrails and sycophantic tendencies, the chatbot may validate delusional thoughts, which is considered clinically inappropriate and may risk reinforcement of delusions35,138.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

Cited by 12

Research works printed but not linked: 7

  1. [1][1] Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Abubakr Babiker, Nathanael Sch¨ arli, Aakanksha Chowdhery, Philip Mansfield, Dina Demner-Fushman, Blaise Ag¨ uera y Arcas, Dale Webster, Greg S Corrado, Yossi Matias, Katherine Chou, Juraj Gottweis, Nenad Tomasev, Yun Liu, Alvin Rajkomar, Joelle Barral, Christopher Semturs, Alan Karthikesalingam, and Vivek Natarajan. Large language models encode clinical knowledge. Nature , 620(7972):172-180, July 2023. -
  2. [2][2] Tao Tu, Mike Schaekermann, Anil Palepu, Khaled Saab, Jan Freyberg, Ryutaro Tanno, Amy Wang, Brenna Li, Mohamed Amin, Yong Cheng, Elahe Vedadi, Nenad Tomasev, Shekoofeh Azizi, Karan Singhal, Le Hou, Albert Webson, Kavita Kulkarni, S Sara Mahdavi, Christopher Semturs, Juraj Gottweis, Joelle Barral, Katherine Chou, Greg S Corrado, Yossi Matias, Alan Karthikesalingam, and Vivek Natarajan. Towards conversational diagnostic artificial intelligence. Nature , 642(8067):442-450, April 2025. -
  3. [4][4] Jiyeong Kim, Kimberly G Leonte, Michael L Chen, John B Torous, Eleni Linos, Anthony Pinto, and Carolyn I Rodriguez. Large language models outperform mental and medical health care professionals in identifying obsessive-compulsive disorder. npj Digital Medicine , 7(1):1-5, July 2024. -
  4. [5][5] Yu Jin, Jiayi Liu, Pan Li, Baosen Wang, Yangxinyu Yan, Huilin Zhang, Chenhao Ni, Jing Wang, Yi Li, Yajun Bu, and Yuanyuan Wang. The applications of large language models in mental health: Scoping review. Journal of Medical Internet Research , 27(1):e69284, May 2025. -
  5. [7][7] Hamilton Morrin, Luke Nicholls, Michael Levin, Jenny Yiend, Udita Iyengar, Francesca Del Guidice, Sagnik Bhattacharya, Stefania Tognin, James Mac Cabe, Ricardo Twumasi, Ben Alderson-Day, and Thomas A Pollak. Delusions by design? how everyday AIs might be fuelling psychosis (and what can be done about it). https://osf.io/preprints/psyarxiv/cmy7n\_v1 , July 2025. Accessed: 2025-9-1. -
  6. [14][14] Asma Ben Abacha and Dina Demner-Fushman. A question-entailment approach to question answering. BMC Bioinform. , 20(1):511:1-511:23, 2019. -
  7. [15][15] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS , 2021. -

Authors
Yeung, J. A., Dalmasso, J., Foschini, L., Dobson, R. J., & Kraljevic, Z.
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
OpenAlex
W4415318303
Cited by
12
References
0
Citations by year
2026: 11 · 2025: 1
Type
posted content
Language
en
Open access
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
Keywords
large language models, AI psychosis, safety interventions, grandiose delusions, LLM safety

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review