CoChat
Sign Up
PreprintCited by 25Ethics and Social Impacts of AIArtificial Intelligence in Healthcare and EducationDigital Mental Health Interventions

Delusions by design? How everyday AIs might be fuelling psychosis (and what can be done about it)

Open access
Morrin, Hamilton, Nicholls, Luke, Levin, Michael, Yiend, Jenny, Iyengar, Udita, DelGuidice, Francesca, Bhattacharyya, Sagnik, MacCabe, James +4 more·2025
DOI 10.31234/osf.io/cmy7n_v5
Open this paper in CoChat — free accountOpen PDF

Abstract

Large language models (LLMs) are poised to become a ubiquitous feature of our lives, mediating communication, decision-making and information curation across nearly every domain. Within psychiatry and psychology the focus to date has remained largely on bespoke therapeutic applications, sometimes narrowly focused and often diagnostically siloed, rather than on the broader and more pressing reality that individuals with mental illness will increasingly engage in agential interactions with AI systems as a routine part of daily existence. While their capacity to model therapeutic dialogue, provide 24/7 companionship and assist with cognitive support has sparked understandable enthusiasm, recent reports suggest that these same systems may contribute to the onset or exacerbation of psychotic symptoms: so-called ‘AI psychosis’ or ‘ChatGPT psychosis’. Emerging, and rapidly accumulating, evidence indicates that agential AI may mirror, validate or amplify delusional or grandiose content, particularly in users already vulnerable to psychosis, due in part to the models’ design to maximise engagement and affirmation, although notably it is not clear whether these interactions have resulted or can result in the emergence of de novo psychosis in the absence of pre-existing vulnerability. Even if some individuals may benefit from AI interactions, for example where the AI functions as a benign and predictable conversational anchor, there is a growing concern that these agents may also reinforce epistemic instability, blur reality boundaries and disrupt self-regulation. In this paper, we outline both the potential harms and therapeutic possibilities of agential AI for people with psychotic disorders. In this perspective piece, we propose a framework of AI-integrated care involving personalised instruction protocols, reflective check-ins, digital advance statements and escalation safeguards to support epistemic security in vulnerable users. These tools reframe the AI agent as an epistemic ally (as opposed to ‘only’ a therapist or a friend) which functions as a partner in relapse prevention and cognitive containment. Given the rapid adoption of LLMs across all domains of digital life, these protocols must be urgently trialled and co-designed with service users and clinicians.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.31234/osf.io/cmy7n_v5

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Morrin et al. (2025) list several recent incidents in which chatbot users felt they were interacting with a conscious god-like AI, and this severely disrupted their life functioning, sometimes leading to suicide.

    “Morrin et al. (2025) list several recent incidents in which chatbot users felt they were interacting with a conscious”

    From Are Attributions of Consciousness to AI Chatbots Epistemically Innocent? (2026) · cites this paper as Morrin et al., 2025

    The case summaries in “Future directions” describe users treating AI as sentient or a higher power, alongside serious disruptions to relationships, work, and mental health. One case reports an attempted hanging, but the passages found do not report a completed suicide; another user was shot and killed by police. The claim’s reference to suicide therefore overstates the reported outcomes. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Romantic non-bizarre delusions can emerge when a chatbot's ability to mimic human conversation is misinterpreted by the user as a conscious AI's affection.

    “romantic non-bizarre delusions can emerge when a chatbot's ability to mimic human conversation is misinterpreted (e.g., due to motivated reasoning) by the user as a conscious AI's affection (Morrin et al., 2025).”

    From Are Attributions of Consciousness to AI Chatbots Epistemically Innocent? (2026) · cites this paper as Morrin et al., 2025

    The Introduction describes romantic or attachment-based delusions in which a user interprets an AI’s human-like conversation as genuine love from a sentient AI. It does not characterize these delusions as “non-bizarre” or identify motivated reasoning as the mechanism. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • These incidents suggest that polythematic bizarre delusions about chatbot consciousness may exist.

    “these incidents (and others, see Morrin et al., 2025) suggest that polythematic bizarre delusions about chatbot consciousness may exist.”

    From Are Attributions of Consciousness to AI Chatbots Epistemically Innocent? (2026) · cites this paper as Morrin et al., 2025

    In “AI is programmed to provide the confirmation that psychotic thinking may require” (page unavailable), the paper says that perceiving an AI as conscious may become part of a delusional belief system, and it describes users seeing AI as supernatural or omniscient. Its case descriptions include elaborate beliefs involving chatbots, but our copy does not establish that the reported delusions were specifically polythematic and bizarre. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • Even if delusion-based consciousness attributions to chatbots are sometimes epistemically innocent, they can also reinforce epistemic instability by blurring reality boundaries and disrupting self-regulation.

    “However, even if delusion-based consciousness attributions to chatbots are sometimes epistemically innocent, they can also reinforce epistemic instability, understood here as a reduced ability to distinguish between accurate and inaccurate representations of reality and to respond appropriately to counterevidence, by blurring reality boundaries and disrupting selfregulation (Morrin et al., 2025).”

    From Are Attributions of Consciousness to AI Chatbots Epistemically Innocent? (2026) · cites this paper as Morrin et al., 2025

  • Prolonged interaction with LLM-CBs may facilitate delusional thoughts and cause an epistemic drift towards delusional beliefs.

    “Prolonged interaction with LLM-CBs may facilitate delusional thoughts and cause an epistemic drift towards delusional beliefs139.”

    From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [139]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 41

  1. Siddals S, Torous J, Coxon A. "It happened to be the perfect thing": experiences of generative AI chatbots for mental health. Npj Ment Health Res. 2024;3:48.
  2. Sharma M, Tong M, Korbak T, Duvenaud D, Askell A, Bowman SR, et al. Towards Understanding Sycophancy in Language Models [Internet]. arXiv; 2025 [cited 2025 Jul 8]. Available from: http://arxiv.org/abs/2310.13548 · arXiv
  3. Moore JR, Grabb D, Agnew W, Klyman K, Chancellor S, Ong DC, et al. Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers [Internet]. 2025. Available from: http://dx.doi.org/10.48550/arXiv.2504.18412 · DOI
  4. Fuchs T. Understanding Sophia? On human interaction with artificial agents. Phenomenol Cogn Sci. 2024;23:21-42.
  5. Ben-Zion Z, Witte K, Jagadish AK, Duek O, Harpaz-Rotem I, Khorsandian MC, et al. Assessing and alleviating state anxiety in large language models. Npj Digit Med [Internet]. 2025 Mar 3 [cited 2025 Jul 8];8(1). Available from: https://www.nature.com/articles/s41746-025-01512-6
  6. Nass C, Moon Y. Machines and Mindlessness: Social Responses to Computers. J Soc Issues. 2000 Jan;56(1):81-103.
  7. Rouleau N, Levin M. Discussions of machine versus living intelligence need more clarity. Nat Mach Intell. 2024;6(12):1424-6.
  8. Tausk V. Über die Entstehung des "Beeinflussungsapparates" in der Schizophrenie. Int Z Für Psychoanal. 1919;5(1):1-19.
  9. Higgins O, Short BL, Chalup SK, Wilson RL. Interpretations of innovation: The role of technology in explanation seeking related to psychosis. Perspect Psychiatr Care. 2023;2023:4464934.
  10. Sconce J. Haunted Media: Electronic Presence from Telegraphy to Television. Durham: Duke University Press; 2000.
  11. Farhall J, Greenwood KM, Jackson HJ. Coping with hallucinated voices in schizophrenia: a review of self-initiated strategies and therapeutic interventions. Clin Psychol Rev. 2007;27(4):476-93.
  12. Feder R. Auditory hallucinations treated by radio headphones. Am J Psychiatry. 1982;139(9):1188-90.
  13. Margo A, Hemsley DR, Slade PD. The effects of varying auditory input on schizophrenic hallucinations. Br J Psychiatry. 1981;139:122-7.
  14. Denno P, Wallis S, Caldwell K, Ives J, Wood SJ, Broome MR, et al. Listening to voices: understanding and self-management of auditory verbal hallucinations in young adults. Psychosis. 2022;14(3):281-92.
  15. Cruz-Gonzalez P, He AWJ, Lam EP, Ng IMC, Li MW, Hou R, et al. Artificial intelligence in mental health care: a systematic review of diagnosis, monitoring, and intervention applications. Psychol Med. 2025 Feb 6;55:e18.
  16. Corlett PR, Horga G, Fletcher PC, Alderson-Day B, Schmack K, Powers AR. Hallucinations and Strong Priors. Trends Cogn Sci. 2019 Feb;23(2):114-27.
  17. Rosset E. It's no accident: Our bias for intentional explanations. Cognition. 2008;108(3):771-80.
  18. Clemmensen L, van Os J, Skovgaard AM, Væver M, Blijd-Hoogewys EMA, Bartels-Velthuis AA, et al. Hyper-theory-of-mind in children with psychotic experiences. PLoS One. 2014;9(11):e113082.
  19. Abu-Akel A, Bailey AL. The possibility of different forms of theory of mind impairment in psychiatric and developmental disorders. Psychol Med. 2000;30(3):735-8.
  20. Csibra G, Gergely G. 'Obsessed with goals': functions and mechanisms of teleological interpretation of actions in humans. Acta Psychol Amst. 2007;124(1):60-78.
  21. Ford JM, Mathalon DH. Efference Copy, Corollary Discharge, Predictive Coding, and Psychosis. Biol Psychiatry Cogn Neurosci Neuroimaging. 2019 Sep 1;4(9):764-7.
  22. Garety PA, Kuipers E, Fowler D, Freeman D, Bebbington PE. A cognitive model of the positive symptoms of psychosis. Psychol Med. 2001 Feb;31(2):189-95.
  23. Blakemore SJ, Sarfati Y, Bazin N, Decety J. The detection of intentional contingencies in simple animations in patients with delusions of persecution. Psychol Med. 2003;33(8):1433-41.
  24. Walter H, Ciaramidaro A, Adenzato M, Vasic N, Ardito RB, Erk S, et al. Dysfunction of the social brain in schizophrenia is modulated by intention type: an fMRI study. Soc Cogn Affect Neurosci. 2009 Jun;4(2):166-76.
  25. McCarthy-Jones S, Trauer T, Mackinnon A, Sims E, Thomas N, Copolov DL. A new phenomenological survey of auditory hallucinations: evidence for subtypes and implications for theory and practice. Schizophr Bull. 2014 Jan;40(1):231-5.
  26. Østergaard SD. Will Generative Artificial Intelligence Chatbots Generate Delusions in Individuals Prone to Psychosis? Schizophr Bull. 2023;49(6):1418-9.
  27. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention Is All You Need [Internet]. arXiv; 2023 [cited 2025 Jul 9]. Available from: http://arxiv.org/abs/1706.03762 · arXiv
  28. Whybrow PC, Akiskal HS, McKinney WT. Mood Disorders: Toward a New Psychobiology. First. Springer New York, NY; 2011. 244 p. (Critical Issues in Psychiatry).
  29. Russinovich M, Salem A, Eldan R. Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack [Internet]. arXiv; 2025 [cited 2025 Jul 8]. Available from: http://arxiv.org/abs/2404.01833 · arXiv
  30. Alderson-Day B, Woods A, Moseley P, Common S, Deamer F, Dodgson G, et al. Voice-Hearing and Personification: Characterizing Social Qualities of Auditory Verbal Hallucinations in Early Psychosis. Schizophr Bull. 2021 Jan 1;47(1):228-36.
  31. Kosmyna N, Hauptmann E, Yuan YT, Situ J, Liao XH, Beresnitzky AV, et al. Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task [Internet]. arXiv; 2025 [cited 2025 Jul 8]. Available from: http://arxiv.org/abs/2506.08872 · arXiv
  32. Wu S, Liu Y, Ruan M. Human-generative AI collaboration enhances task performance but undermines human's intrinsic motivation. Sci Rep. 2025;15:15105.
  33. Weidinger L, Mellor J, Rauh M, Griffin C, Uesato J, Huang PS, et al. Ethical and social risks of harm from Language Models [Internet]. arXiv; 2021 [cited 2025 Jul 8]. Available from: http://arxiv.org/abs/2112.04359 · arXiv
  34. Kidd C, Birhane A. How AI can distort human beliefs. Science. 2023 Jun 23;380(6651):1222-3.
  35. Qiu J, He Y, Juan X, Wang Y, Liu Y. EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safety. 2025 Apr 13; Available from: https://arxiv.org/abs/2504.09689 · arXiv
  36. Charlson FJ, Ferrari AJ, Santomauro DF, Diminic S, Stockings E, Scott JG, et al. Global Epidemiology and Burden of Schizophrenia: Findings From the Global Burden of Disease Study 2016. Schizophr Bull. 2018 Oct 17;44(6):1195-203.
  37. Grabb D, Lamparth M, Vasan N. Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementation [Internet]. arXiv; 2024 [cited 2025 Jul 8]. Available from: http://arxiv.org/abs/2406.11852 · arXiv
  38. Feyaerts J, Henriksen MG, Vanheule S, Myin-Germeys I, Sass LA. Delusions beyond beliefs: a critical overview of diagnostic, aetiological, and therapeutic schizophrenia research from a clinical-phenomenological perspective. Lancet Psychiatry. 2021 Mar 1;8(3):237-49.
  39. Pappa E, Baah F, Lynch J, Shiel L, Blackman G, Raihani N, et al. Delusional Themes are More Varied Than Previously Assumed: A Comprehensive Systematic Review and Meta-Analysis. Schizophr Bull. 2025 May 8;51(3):637-45.
  40. Waaler PN, Hussain M, Molchanov I, Bongo LA, Elvevåg B. Prompt engineering a schizophrenia chatbot: utilizing a multi-agent approach for enhanced compliance with prompt instructions [Internet]. 2024. Available from: https://arxiv.org/abs/2410.12848 · arXiv
  41. Ben-Zion Z. Why we need mandatory safeguards for emotionally responsive AI. Nature. 2025 Jul;643(8070):9.

Cited by

Sorted by citations

Related

Authors
Morrin, H., Nicholls, L., Levin, M., Yiend, J., Iyengar, U., DelGuidice, F., Bhattacharyya, S., MacCabe, J., Tognin, S., Twumasi, R., Alderson-Day, B., & Pollak, T.
Affiliations
  • King's College London (GB)
  • The Graduate Center, CUNY (US)
  • Tufts University (US)
ORCID
Publisher
Center for Open Science
Published
2025-07-11
DOI
10.31234/osf.io/cmy7n_v5
OpenAlex
W4412347940
Cited by
24
References
0
Citations by year
2026: 17 · 2025: 7
Type
posted content
Language
en
Open access
Licence
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
Keywords
AI psychosis, large language models, psychosis, delusions, grandiosity, epistemic instability, relapse prevention, epistemic security, AI companionship

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review