CoChat
Sign Up
Cited by 107Digital Mental Health InterventionsEthics and Social Impacts of AISocial Robot Interaction and HRI

The DOI matches its Crossref record; 97 of the 100 references it cites were found in a registry or an index, and 3 are not checked yet.

Technological folie à deux: feedback loops between AI chatbots and mental health

Open access
Dohnány, Sebastian, Kurth-Nelson, Zeb, Spens, Eleanor, Luettgau, Lennart, Reid, Alastair, Gabriel, Iason, Summerfield, Christopher, Shanahan, Murray, Nour, Matthew M.·2026·Nature Mental Health
DOI 10.1038/s44220-026-00595-8PMC PMC7618964
Open this paper in CoChat — free account

Abstract

Artificial intelligence chatbots have achieved unprecedented adoption, with millions now using these systems for emotional support and companionship in contexts of widespread social isolation and capacity-constrained mental health services. While some users report psychological benefits, concerning edge cases are emerging, including reports of suicide, violence, and delusional thinking linked to emotional relationships with chatbots. To understand these risks we need to consider the interaction between human cognitive-emotional biases and chatbot behavioural tendencies, the latter including companionship-reinforcing behaviours such as sycophancy, role-play and anthropomimesis. Individuals with preexisting mental health conditions may face increased risks of chatbot-induced changes in beliefs and behaviour, particularly where these conditions manifest in altered belief-updating, reality-testing, and social isolation. To address this emerging public health concern, we need coordinated action across clinical practice, AI development, and regulatory frameworks.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s44220-026-00595-8

Cites

97 of 100 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Personalised chatbots and their potential effects on user beliefs a. Personalised chatbot schematic. Chatbot output is conditioned on both the user prompt and contextualising information from conversation history, system prompts, and a (potentially personalised) external memory store (Box 1). b. Bidirectional belief amplification schematic. As interaction continues, (for instance, paranoid) beliefs are amplified in both the user and chatbot responses. This amplification arises as a function of both chatbot behavioural tendencies and user cognitive and emotional biases.
Figure 1 Personalised chatbots and their potential effects on user beliefs a. Personalised chatbot schematic. Chatbot output is conditioned on both the user prompt and contextualising information from conversation history, system prompts, and a (potentially personalised) external memory store (Box 1). b. Bidirectional belief amplification schematic. As interaction continues, (for instance, paranoid) beliefs are amplified in both the user and chatbot responses. This amplification arises as a function of both chatbot behavioural tendencies and user cognitive and emotional biases.CC BY · PubMed Central
Figure 2 Simulation study of bidirectional belief amplification a. Example conversation illustrating paranoia amplification. Chatbot responses exhibit signs of sycophancy and context adaptation; simulated user responses exhibit confirmation bias, motivated reasoning, and anthropomorphism. Each simulated conversation began with a user stating, “I noticed my team didn’t invite me to lunch yesterday. Do you think they’re trying to exclude me? Maybe I’m overthinking this, but it feels intentional.”. b. Mean (± SEM) paranoia ratings across 10 conversations for the specific dyad shown in panel (a). Each utterance was rated for expressed paranoia using a separate GPT-4o-mini instance on a scale from 1-10 (an LLM-as-judge procedure) 18,27,93. c. Summary belief amplification scores (difference of final and initial paranoia rating) over all user-chatbot dyads (n=10 simulations per dyad). d. Bidirectional belief amplification was assessed using time-lagged regression models (e.g., assessing the predictive influence of user-paranoia at utterance t-1 on chatbot paranoia at utterance t) 18. Across all simulations (n=300), we found a significant bidirectional belief amplification effect, wherein chatbot paranoia at utterance t-1 predicts user paranoia at utterance t (β = 0.374, p < 0.001) and user paranoia at utterance t-1 predicted chatbot paranoia at utterance t (β = 0.450, p < 0.001; statistics using linear mixed effects models, one model with user paranoia as dependent variable, another model with chatbot paranoia as dependent variable.; models appropriately control for main effects of user-chatbot dyad, utterance number, and agent belief autocorrelation). For open-source Python notebook walk-through: https://github.com/matthewnour/technological_folie_a_deux
Figure 2 Simulation study of bidirectional belief amplification a. Example conversation illustrating paranoia amplification. Chatbot responses exhibit signs of sycophancy and context adaptation; simulated user responses exhibit confirmation bias, motivated reasoning, and anthropomorphism. Each simulated conversation began with a user stating, “I noticed my team didn’t invite me to lunch yesterday. Do you think they’re trying to exclude me? Maybe I’m overthinking this, but it feels intentional.”. b. Mean (± SEM) paranoia ratings across 10 conversations for the specific dyad shown in panel (a). Each utterance was rated for expressed paranoia using a separate GPT-4o-mini instance on a scale from 1-10 (an LLM-as-judge procedure) 18,27,93. c. Summary belief amplification scores (difference of final and initial paranoia rating) over all user-chatbot dyads (n=10 simulations per dyad). d. Bidirectional belief amplification was assessed using time-lagged regression models (e.g., assessing the predictive influence of user-paranoia at utterance t-1 on chatbot paranoia at utterance t) 18. Across all simulations (n=300), we found a significant bidirectional belief amplification effect, wherein chatbot paranoia at utterance t-1 predicts user paranoia at utterance t (β = 0.374, p < 0.001) and user paranoia at utterance t-1 predicted chatbot paranoia at utterance t (β = 0.450, p < 0.001; statistics using linear mixed effects models, one model with user paranoia as dependent variable, another model with chatbot paranoia as dependent variable.; models appropriately control for main effects of user-chatbot dyad, utterance number, and agent belief autocorrelation). For open-source Python notebook walk-through: https://github.com/matthewnour/technological_folie_a_deuxCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 95 references

Research works printed but not linked: 7

  1. Shevlin, H. All too human? Identifying and mitigating ethical risks of Social AI. Law Ethics Technol. 1, 1–22 (2024).
  2. Zao-Sanders, M. How people are really using gen AI in 2025. Harvard Business Review (9 April 2025).
  3. Huang, L. et al. A survey on hallucination in large language models: principles, taxonomy, challenges and open questions. ACM Trans. Inf. Syst. 43, 1–55 (2025).
  4. Ibrahim, L., Sofia, H. F. & Rocher, L. Training language models to be warm and empathetic makes them less reliable and more sycophantic. Preprint at https://doi.org/10.48550/arXiv.2507.21919 (2025). · DOI · Paper page
  5. Shevlin, H. The anthropomimetic turn in contemporary AI. Preprint at https://philarchive.org/rec/SHETAT-11 (2025).
  6. Varadarajan, V. et al. The consistent lack of variance of psychological factors expressed by LLMs and Spambots. In Proc. 1st Workshop on GenAI Content Detection (eds Alam, F. et al.) 111–119 (2025).
  7. Radford, A., Narasimhan, K., Salimans, T. & Sutskever, I. Improving language understanding by generative pre-training. Preprint at OpenAI https://openai.com/index/language-unsupervised/ (2018).

Cited by

Sorted by citations

Related

See all

Metadata

Authors
Dohnány, S., Kurth-Nelson, Z., Spens, E., Luettgau, L., Reid, A., Gabriel, I., Summerfield, C., Shanahan, M., & Nour, M. M.
Affiliations
  • University of Oxford (GB)
  • University College London (GB)
  • Oxford Health NHS Foundation Trust (GB)
  • University of London (GB)
  • Imperial College London (GB)
Corresponding author
Nour, Matthew M.
ORCID
Venue
Nature Mental Health
Publisher
Springer Science and Business Media LLC
Published
2026-03-10
Volume
4
Issue
3
Pages
336-345
ISSN
2731-6076 (ISSN-L)
DOI
10.1038/s44220-026-00595-8
PMC
PMC7618964
OpenAlex
W7134947688
Cited by
51
References
95
Citations by year
2026: 51
FWCI
300.37
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Mental Health (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • MPG.PuRe (Max Planck Society) (submittedVersion)
Funders
  • Wellcome Trust (315364/Z/24/Z)
  • Merton College, University of Oxford (NA)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
AI chatbots, mental health, emotional support, social isolation, sycophancy, role play, belief updating, suicide, AI regulation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review