Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.

Characterizing Delusional Spirals through Human-LLM Chat Logs

Open access
Moore, Jared, Mehta, Ashish, Agnew, William, Anthis, Jacy Reese, Louie, Ryan, Mai, Yifan, Yin, Peggy, Cheng, Myra, Paech, Samuel J., Klyman, Kevin, Chancellor, Stevie, Lin, Eric, Haber, Nick, Ong, Desmond C. · 2026 · Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency

Abstract

As large language models (LLMs) have proliferated, disturbing anecdotal reports of negative psychological effects, such as delusions, self-harm, and “AI psychosis,” have emerged in global media and legal discourse. However, it remains unclear how users and chatbots interact over the course of lengthy delusional “spirals,” limiting our ability to understand and mitigate the harm. In our work, we analyze logs of conversations with LLM chatbots from 19 users who report having experienced psychological harms from chatbot use. Many of our participants come from a support group for such chatbot users. We also include chat logs from participants covered by media outlets in widely-distributed stories about chatbot-reinforced delusions. In contrast to prior work that speculates on potential AI harms to mental health, to our knowledge we present the first in-depth study of such high-profile and veridically harmful cases. We develop an inventory of 28 codes and apply it to the 391, 562 messages in the logs. Codes include whether a user demonstrates delusional thinking (15.5% of user messages), a user expresses suicidal thoughts (69 validated user messages), or a chatbot misrepresents itself as sentient (21.2% of chatbot messages). We analyze the co-occurrence of message codes. We find, for example, that messages that declare romantic interest and messages where the chatbot describes itself as sentient occur much more often in longer conversations, suggesting that these topics could promote or result from user over-engagement and that safeguards in these areas may degrade in multi-turn settings. We conclude with concrete recommendations for how policymakers, LLM chatbot developers, and users can use our inventory and conversation analysis tool to understand and mitigate harm from LLM chatbots.

Overview

Integrity signals

Registry

Crossref · 10.1145/3805689.3806443

Cites

58 of 58 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

The text we found for this paper looks like another document, so its claims and self-check weren’t run. The registry and citation checks still apply.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The text we found for this paper looks like another document, so the self-check wasn’t run.

Citations

Cites

Show all 45 references

Research works printed but not linked: 13

  1. Substance Abuse, Mental Health Services Administration, et al. 2016. Impact of the DSM-IV to DSM-5 Changes on the National Survey on Drug Use and Health. (2016).
  2. American Psychiatric Association. 2022. Diagnostic and statistical manual of mental disorders DSM-5-TR (fifth edition, text revision ed.). American Psychiatric Association Publishing.
  3. Social Media Victims Law Center. 2025. Social Media Victims Law Center and Tech Justice Law Project lawsuits accuse ChatGPT of emotional manipulation supercharging AI delusions and acting as a “suicide coach”. https://socialmediavictims.org/press-releases/smvlc-tech-justice-law-project-lawsuits-accuse-chatgpt-of-emotional-manipulation-supercharging-ai-delusions-and-acting-as-a-suicide-coach/
  4. Julian De Freitas, Zeliha Oguz-Uguralp, and Ahmet Kaan-Uguralp. 2025. Emotional manipulation by AI companions. arXiv preprint arXiv:2508.19258 (2025). · arXiv
  5. W Bradley Knox, Katie Bradford, Samanta Varela Castro, Desmond C Ong, Sean Williams, Jacob Romanow, Carly Nations, Peter Stone, and Samuel Baker. 2025. Harmful traits of AI companions. (2025).
  6. Marsha Linehan. 1993. Cognitive-behavioral treatment of borderline personality disorder. Guilford press.
  7. Zilin Ma, Yiyang Mei, and Zhaoyuan Su. 2024. Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Support. 2023 (2024), 1105–1114. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10785945/
  8. John J McGrath, Sukanta Saha, Ali Al-Hamzawi, Jordi Alonso, Evelyn J Bromet, Ronny Bruffaerts, José Miguel Caldas-de Almeida, Wai Tat Chiu, Peter de Jonge, John Fayyad, et al. 2015. Psychotic experiences in the general population: a cross-national analysis based on 31 261 respondents from 18 countries. JAMA psychiatry 72, 7 (2015), 697–705.
  9. Ashish Mehta, Jared Moore, Jacy Reese Anthis, William Agnew, Eric Lin, Peggy Yin, Desmond C. Ong, Nick Haber, and Carol Dweck. 2026. The Dynamics of Delusion: Modeling Bidirectional False Belief Amplification in Human-Chatbot Dialogue. https://spirals.stanford.edu/p/dynamics Preprint.
  10. John E Overall and Donald R Gorham. 1988. The Brief Psychiatric Rating Scale (BPRS): recent developments in ascertainment and scaling. Psychopharmacol Bull 24, 1 (1988), 97–99.
  11. Joseph M. Pierre, Ben Gaeta, Govind Raghavan, and Karthik V. Sarma. 2025. “You're Not Crazy”: A Case of New-onset AI-associated Psychosis. 22, 10 (2025), 11. https://innovationscns.com/youre-not-crazy-a-case-of-new-onset-ai-associated-psychosis/
  12. Michael B. Robb and Supreet Mann. 2025. Talk, trust, and trade-offs: How and why teens use AI companions. https://www.commonsensemedia.org/sites/default/files/research/report/talk-trust-and-trade-offs_2025_web.pdf
  13. Viktor Tausk. 1919. Über die Entstehung des “Beeinflussungsapparates” in der Schizophrenie. 5 (1919), 1–33. https://pep-web.org/browse/document/izpa.005.0001a

Cited by

See all 15

Metadata

Authors
Moore, J., Mehta, A., Agnew, W., Anthis, J. R., Louie, R., Mai, Y., Yin, P., Cheng, M., Paech, S. J., Klyman, K., Chancellor, S., Lin, E., Haber, N., & Ong, D. C.
Affiliations
  • Stanford University (US)
  • Carnegie Mellon University (US)
  • University of Chicago (US)
  • Independent Researcher, N/A, Australia
  • Harvard University (US)
  • University of Minnesota (US)
  • The University of Texas at Austin (US)
Venue
Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency
Publisher
ACM
Published
2026-06-25
Pages
7631-7674
OpenAlex
W7166582319
Cited by
15
References
45
Citations by year
2026: 15
FWCI
142.46
Type
conference paper
Language
en
Open access
Locations
ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
large language model chatbots, AI psychosis, suicidal ideation, chat logs, conversation analysis, multi-turn dialogue

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A second model labelled each claim on 2026-10-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Characterizing Delusional Spirals through Human-LLM Chat Logs. https://app.cochat.ai/papers/10.1145/3805689.3806443
The paper
Moore, J., Mehta, A., Agnew, W., Anthis, J. R., Louie, R., Mai, Y., Yin, P., Cheng, M., Paech, S. J., Klyman, K., Chancellor, S., Lin, E., Haber, N., & Ong, D. C. (2026). Characterizing Delusional Spirals through Human-LLM Chat Logs. Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3805689.3806443

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.