Cited by 107Digital Mental Health InterventionsEthics and Social Impacts of AISocial Robot Interaction and HRI

The DOI matches its Crossref record; 97 of the 100 references it cites were found in a registry or an index, and 3 are not checked yet.

Technological folie à deux: feedback loops between AI chatbots and mental health

Open access

Dohnány, Sebastian, Kurth-Nelson, Zeb, Spens, Eleanor, Luettgau, Lennart, Reid, Alastair, Gabriel, Iason, Summerfield, Christopher, Shanahan, Murray, Nour, Matthew M.·2026·Nature Mental Health

DOI 10.1038/s44220-026-00595-8PMC PMC7618964

Open this paper in CoChat — free account

Abstract

Artificial intelligence chatbots have achieved unprecedented adoption, with millions now using these systems for emotional support and companionship in contexts of widespread social isolation and capacity-constrained mental health services. While some users report psychological benefits, concerning edge cases are emerging, including reports of suicide, violence, and delusional thinking linked to emotional relationships with chatbots. To understand these risks we need to consider the interaction between human cognitive-emotional biases and chatbot behavioural tendencies, the latter including companionship-reinforcing behaviours such as sycophancy, role-play and anthropomimesis. Individuals with preexisting mental health conditions may face increased risks of chatbot-induced changes in beliefs and behaviour, particularly where these conditions manifest in altered belief-updating, reality-testing, and social isolation. To address this emerging public health concern, we need coordinated action across clinical practice, AI development, and regulatory frameworks.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1038/s44220-026-00595-8

Cites

97 of 100 references verified0 citation errors · 0 not checkable

Statistics

Figure 1 Personalised chatbots and their potential effects on user beliefs a. Personalised chatbot schematic. Chatbot output is conditioned on both the user prompt and contextualising information from conversation history, system prompts, and a (potentially personalised) external memory store (Box 1). b. Bidirectional belief amplification schematic. As interaction continues, (for instance, paranoid) beliefs are amplified in both the user and chatbot responses. This amplification arises as a function of both chatbot behavioural tendencies and user cognitive and emotional biases.CC BY · PubMed Central

Figure 2 Simulation study of bidirectional belief amplification a. Example conversation illustrating paranoia amplification. Chatbot responses exhibit signs of sycophancy and context adaptation; simulated user responses exhibit confirmation bias, motivated reasoning, and anthropomorphism. Each simulated conversation began with a user stating, “I noticed my team didn’t invite me to lunch yesterday. Do you think they’re trying to exclude me? Maybe I’m overthinking this, but it feels intentional.”. b. Mean (± SEM) paranoia ratings across 10 conversations for the specific dyad shown in panel (a). Each utterance was rated for expressed paranoia using a separate GPT-4o-mini instance on a scale from 1-10 (an LLM-as-judge procedure) 18,27,93. c. Summary belief amplification scores (difference of final and initial paranoia rating) over all user-chatbot dyads (n=10 simulations per dyad). d. Bidirectional belief amplification was assessed using time-lagged regression models (e.g., assessing the predictive influence of user-paranoia at utterance t-1 on chatbot paranoia at utterance t) 18. Across all simulations (n=300), we found a significant bidirectional belief amplification effect, wherein chatbot paranoia at utterance t-1 predicts user paranoia at utterance t (β = 0.374, p < 0.001) and user paranoia at utterance t-1 predicted chatbot paranoia at utterance t (β = 0.450, p < 0.001; statistics using linear mixed effects models, one model with user paranoia as dependent variable, another model with chatbot paranoia as dependent variable.; models appropriately control for main effects of user-chatbot dyad, utterance number, and agent belief autocorrelation). For open-source Python notebook walk-through: https://github.com/matthewnour/technological_folie_a_deuxCC BY · PubMed Central

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Attention Is All You Need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Birds of a Feather: Homophily in Social Networks
McPherson, M., Smith-Lovin, L., & Cook, J. M. (2001) · Annual Review of Sociology · doi:10.1146/annurev.soc.27.1.415 · Cited by 18,567
The case for motivated reasoning.
Kunda, Z. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.108.3.480 · Cited by 9,022
Confirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888
Lost in the Middle: How Language Models Use Long Contexts
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00638 · Cited by 1,612
Concrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314
Untitled work
Sharma, M., Tong, M., Korbak, T., Duperrouzel, D., Was, H., Korbak, J., Cheng, M., Askell, A., Bai, Y., Brain, C., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.13548 · Cited by 1,313 · cited 3×
“Beyond these seemingly benign modifications, however, RLHF can also render models sycophantic 52–55, These undesirable behavioural tendencies emerge because the human judgements that RLHF uses as a training signal are themselves shaped by human cognitive biases.”
Attachment-related psychodynamics
Shaver, P. R., & Mikulincer, M. (2002) · Attachment & Human Development · doi:10.1080/14616730210154171 · Cited by 1,294
Feeling validated versus being correct: A meta-analysis of selective exposure to information
Hart, W., Albarracín, D., Eagly, A. H., Brechan, I., Lindberg, M. J., & Merrill, L. (2009) · Psychological Bulletin · doi:10.1037/a0015701 · Cited by 1,292
Instruction Tuning for Large Language Models: A Survey
Zhang, S., Dong, L., Li, X., Zhang, S., Sun, X., Wang, S., Li, J., Hu, R., Zhang, T., Wang, G., et al. (2025) · ACM Computing Surveys · doi:10.1145/3777411 · Cited by 1,271
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058
Computational psychiatry as a bridge from neuroscience to clinical applications
Huys, Q. J. M., Maia, T. V., & Frank, M. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4238 · Cited by 1,002
The Predictive Coding Account of Psychosis
Sterzer, P., Adams, R. A., Fletcher, P., Frith, C., Lawrie, S. M., Muckli, L., Petrovic, P., Uhlhaas, P., Voss, M., & Corlett, P. R. (2018) · Biological Psychiatry · doi:10.1016/j.biopsych.2018.05.015 · Cited by 941 · cited 1×
“For example, people with psychosis are liable to form overly confident beliefs based on minimal evidence (“jumping to conclusions”) 97,98 , potentially indicating a tendency to overweight new information in favour of prior beliefs 99 .”
Detecting hallucinations in large language models using semantic entropy
Farquhar, S., Kossen, J., Kuhn, L., & Gal, Y. (2024) · Nature · doi:10.1038/s41586-024-07421-0 · Cited by 934 · cited 2×
“Generation of false information, which is nevertheless presented with high apparent confidence and linguistic coherence 33,37,41,42 (sometimes termed “confabulations”).”
Retrieval-Augmented Generation for Large Language Models: A Survey
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2312.10997 · Cited by 876
AI models collapse when trained on recursively generated data
Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R., & Gal, Y. (2024) · Nature · doi:10.1038/s41586-024-07566-y · Cited by 793 · cited 1×
“Post-training procedures designed to shape chatbot responses, such as RLHF, carry an inherent risk of proxy failure and suffer from inadequate data coverage (reduced sample diversity 106,107 ).”
Creating Social Connection Through Inferential Reproduction: Loneliness and Perceived Agency in Gadgets, Gods, and Greyhounds
Epley, N., Akalis, S., Waytz, A., & Cacioppo, J. T. (2008) · Psychological Science · doi:10.1111/j.1467-9280.2008.02056.x · Cited by 632
Systematic review and meta-analysis of AI-based conversational agents for promoting mental health and well-being
Li, H., Zhang, R., Lee, Y.-C., Kraut, R. E., & Mohr, D. C. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00979-5 · Cited by 598
Role play with large language models
Shanahan, M., McDonell, K., & Reynolds, L. (2023) · Nature · doi:10.1038/s41586-023-06647-8 · Cited by 491 · cited 7×
“LLM output is generated primarily through autoregressive sampling from the encoded probability distribution, such that the sequence of emitted tokens has a high joint probability given the sum total of contextualising information 49,64 .”
InfluentialSelf-citation
Social isolation in mental health: a conceptual and methodological review
Wang, J., Lloyd-Evans, B., Giacco, D., Forsyth, R., Nebo, C., Mann, F., & Johnson, S. (2017) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-017-1446-1 · Cited by 462
Social connectedness as a determinant of mental health: A scoping review
WICKRAMARATNE, P. J., Yangchen, T., Lepow, L., Patra, B. G., Glicksburg, B., Talati, A., Adekkanattu, P. M., Ryu, E., Biernacka, J. M., Charney, A. W., et al. (2022) · PLoS ONE · doi:10.1371/journal.pone.0275004 · Cited by 461
Psychosis, delusions and the “jumping to conclusions” reasoning bias: a systematic review and meta-analysis
Dudley, R., Taylor, P., Wickham, S., & Hutton, P. (2016) · Schizophrenia Bulletin · doi:10.1093/schbul/sbv150 · Cited by 353
Dissociating language and thought in large language models
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.011 · Cited by 345
Untitled work
Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., & Kaplan, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.08073 · Cited by 326 · cited 10×
“All modern chatbots are built on AI large language models (LLMs).”
Influential
How human–AI feedback loops alter human perceptual, emotional and social judgements
Glickman, M., & Sharot, T. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02077-2 · Cited by 289
Zoom In: An Introduction to Circuits
Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M., & Carter, S. (2020) · Distill · doi:10.23915/distill.00024.001 · Cited by 288
Computational psychiatry: towards a mathematically informed understanding of mental illness
Adams, R. A., Huys, Q. J. M., & Roiser, J. P. (2015) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp-2015-310737 · Cited by 285
Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting
Turpin, M., Michael, J., Perez, E., & Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2305.04388 · Cited by 262
Loneliness and suicide mitigation for students using GPT3-enabled chatbots
Maples, B., Cerit, M., Vishwanath, A., & Pea, R. (2024) · npj Mental Health Research · doi:10.1038/s44184-023-00047-6 · Cited by 250 · cited 3×
“…number still may use chatbots as companions or mental health therapists, in an attempt to mitigate loneliness or as a consequence of barriers to human-administered psychotherapy, respectively 1,7,8,13–15 , We have highlighted one theoretical risk profile that may arise in these latter use cases.”
“It happened to be the perfect thing”: experiences of generative AI chatbots for mental health
Siddals, S., Torous, J., & Coxon, A. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00097-4 · Cited by 241
How People Use ChatGPT
Chatterji, A., Cunningham, T., Deming, D., Hitzig, Z., Ong, C., Shan, C. Y., & Wadman, K. (2025) · doi:10.3386/w34255 · Cited by 233
Association of the Jumping to Conclusions and Evidence Integration Biases With Delusions in Psychosis: A Detailed Meta-analysis
McLean, B. F., Mattiske, J. K., & Balzan, R. P. (2016) · Schizophrenia Bulletin · doi:10.1093/schbul/sbw056 · Cited by 231
Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers.
Moore, J., Grabb, D., Agnew, W., Klyman, K., Chancellor, S., Ong, D. C., & Haber, N. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732039 · Cited by 204
Formal thought disorders: from phenomenology to neurobiology
Kircher, T. T. J., Bröhl, H., Meier, F., & Engelen, J. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(18)30059-2 · Cited by 177
What Is ChatGPT Doing … and Why Does It Work?
Wolfram, S. (2023) · doi:10.31855/bc47ee6b-75c · Cited by 172
Why human–AI relationships need socioaffective alignment
Kirk, H. R., Gabriel, I., Summerfield, C., Vidgen, B., & Hale, S. A. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-04532-5 · Cited by 146
Self-citation
Language models transmit behavioural traits through hidden signals in data
Cloud, A., Le, M. H., Chua, J., Betley, J., Sztyber, A., Mindermann, S., Hilton, J., Marks, S., & Evans, O. (2026) · Nature · doi:10.1038/s41586-026-10319-8 · Cited by 146
The levers of political persuasion with conversational artificial intelligence
Hackenburg, K., Tappin, B. M., Hewitt, L., Saunders, E., Black, S., Lin, H., Fist, C., Margetts, H. Z., Rand, D. G., & Summerfield, C. (2025) · Science · doi:10.1126/science.aea3884 · Cited by 128
Self-citation
The benefits, risks and bounds of personalizing the alignment of large language models to individuals
Kirk, H. R., Vidgen, B., Röttger, P., & Hale, S. A. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00820-y · Cited by 122
Training language models to be warm can reduce accuracy and increase sycophancy
Ibrahim, L., Hafner, F. S., & Rocher, L. (2026) · Nature · doi:10.1038/s41586-026-10410-0 · Cited by 118
The benefits and dangers of anthropomorphic conversational agents
Peter, S., Riemer, K., & West, J. D. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2415898122 · Cited by 110 · cited 10×
“With regard to human factors, the potential relates to a capacity for anthropomorphism : the attribution of human-like qualities such as agency, intentionality, emotional states, and consciousness to non-human systems 3,5,27,39,40 .”
Influential
How AI and Human Behaviors Shape Psychosocial Effects of Extended Chatbot Use: A Longitudinal Randomized Controlled Study
Fang, C. M., Liu, A. R., Danry, V., Lee, E., Chan, S. W. T., Pataranutaporn, P., Maes, P., Phang, J., Lampe, M., Ahmad, L., et al. (2026) · doi:10.21203/rs.3.rs-8148142/v1 · Cited by 109
How AI and Human Behaviors Shape Psychosocial Effects of Extended Chatbot Use: A Longitudinal Randomized Controlled Study
Fang, C. M., Liu, A. R., Danry, V., Lee, E., Chan, S. W. T., Pataranutaporn, P., Maes, P., Phang, J., Lampe, M., Ahmad, L., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.17473 · Cited by 109
Are lonely youngsters turning to chatbots for companionship? The relationship between chatbot usage and social connectedness in Danish high-school students
Herbener, A. B., & Damholdt, M. F. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103409 · Cited by 107
Embers of autoregression show how large language models are shaped by the problem they are trained to solve
McCoy, R. T., Yao, S., Friedman, D., Hardy, M. D., & Griffiths, T. L. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2322420121 · Cited by 96 · cited 2×
“…responses are often expressed with a high level of linguistic fluency and confidence regardless of the accuracy of the conveyed information 36,41 , reflecting a training objective that optimises for the generation of plausible text completions, rather than accurate and unbiased information 42 .”
The health risks of generative AI-based wellness apps
De Freitas, J., & Cohen, I. G. (2024) · Nature Medicine · doi:10.1038/s41591-024-02943-6 · Cited by 95
Position: Towards Bidirectional Human-AI Alignment
Shen, H., Knearem, T., Ghosh, R., Alkiek, K., Krishna, K., Yachuan, L., Petridis, S., Peng, Y.-H., Qiwei, L., Si, C., et al. (2025) · neural information processing systems · doi:10.52202/085713-4186 · Cited by 89
PATIENT-𝜓: Using Large Language Models to Simulate Patients for Training Mental Health Professionals
Wang, R., Milani, S., Chiu, J. C., Zhi, J., Eack, S. M., Labrum, T., Murphy, S. M., Jones, N., Hardy, K. V., Hong, S., et al. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.711 · Cited by 88
Motivated Thinking
Molden, D. C., & Higgins, E. T. (2012) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199734689.013.0020 · Cited by 86
The impact of advanced AI systems on democracy
Summerfield, C., Argyle, L. P., Bakker, M. A., Collins, T., Durmus, E., Eloundou, T., Gabriel, I., Ganguli, D., Hackenburg, K., Hadfield, G. K., et al. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02309-z · Cited by 65
Self-citation
Multi-turn Evaluation of Anthropomorphic Behaviours in Large Language ModelsPreprint
Ibrahim, L., Akbulut, C., Elasmar, R., Rastogi, C., Kahng, M., Morris, M. R., McKee, K. R., Rieser, V., Shanahan, M., & Weidinger, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.07077 · Cited by 63
Self-citation
SycEval: Evaluating LLM Sycophancy
Fanous, A., Goldberg, J., Agarwal, A., Lin, J., Zhou, A., Xu, S., Bikia, V., Daneshjou, R., & Koyejo, S. (2025) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1609/aies.v8i1.36598 · Cited by 62
SycEval: Evaluating LLM Sycophancy
Fanous, A., Goldberg, J., Agarwal, A. A., Lin, J., Zhou, A., Daneshjou, R., & Koyejo, S. (2025) · arXiv · doi:10.48550/arxiv.2502.08177 · Cited by 62
A systematic review of echo chamber research: comparative analysis of conceptualizations, operationalizations, and varying outcomes
Hartmann, D., Wang, S. M., Pohlmann, L., & Berendt, B. (2025) · Journal of Computational Social Science · doi:10.1007/s42001-025-00381-z · Cited by 59
The Ethics of Advanced AI Assistants
Gabriel, I., Manzini, A., Keeling, G., Hendricks, L. A., Rieser, V., Iqbal, H., Tomašev, N., Ktena, I., Kenton, Z., Rodriguez, M., et al. (2024) · arXiv · doi:10.48550/arxiv.2404.16244 · Cited by 57
Self-citation
Discovering Language Model Behaviors with Model-Written Evaluations
Perez, E., Ringer, S., Lukošiūtė, K., Nguyen, K., Chen, E., Heiner, S., Pettit, C., Olsson, C., Kundu, S., Kadavath, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.09251 · Cited by 53 · cited 2×
“Beyond these seemingly benign modifications, however, RLHF can also render models sycophantic 52–55, These undesirable behavioural tendencies emerge because the human judgements that RLHF uses as a training signal are themselves shaped by human cognitive biases.”
Belief updating in psychosis, depression and anxiety disorders: A systematic review across computational modelling approaches
Gibbs-Dean, T., Katthagen, T., Tsenkova, I., Ali, R., Liang, X., Spencer, T. J., & Diederen, K. M. J. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105087 · Cited by 52
LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals
Park, J. S., Zou, C. Q., Kamphorst, J., Egan, N., Shaw, A., Hill, B. M., Cai, C., Morris, M. R., Liang, P., Willer, R., et al. (2024) · arXiv · doi:10.48550/arxiv.2411.10109 · Cited by 49
Shifting Work Patterns with Generative AI
Dillon, E. W., Jaffe, S., Immorlica, N., & Stanton, C. (2025) · National Bureau of Economic Research · doi:10.3386/w33795 · Cited by 47
PUB: A Pragmatics Understanding Benchmark for Assessing LLMs’ Pragmatics Capabilities
Sravanthi, S., Doshi, M., Tankala, P., Murthy, R., Dabre, R., & Bhattacharyya, P. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.719 · Cited by 46
The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
Shojaee, P., Mirzadeh, I., Alizadeh, K., Horton, M., Bengio, S., & Farajtabar, M. (2025) · arXiv · doi:10.48550/arxiv.2506.06941 · Cited by 45
Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts
Samvelyan, M., Raparthy, S. C., Lupu, A., Hambro, E., Markosyan, A. H., Bhatt, M., Mao, Y., Jiang, M., Parker-Holder, J., Foerster, J., et al. (2024) · neural information processing systems · doi:10.52202/079017-2229 · Cited by 42
LLM-empowered Chatbots for Psychiatrist and Patient Simulation: Application and EvaluationPreprint
Chen, S., Wu, M., Zhu, K. Q., Lan, K., Zhang, Z., & Cui, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.13614 · Cited by 41
Confabulation: The Surprising Value of Large Language Model Hallucinations
Sui, P., Duede, E., Wu, S., & So, R. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.770 · Cited by 41
Can AI Relate: Testing Large Language Model Response for Mental Health Support
Gabriel, S., Puri, I., Xu, X., Malgaroli, M., & Ghassemi, M. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.120 · Cited by 40
Dead rats, dopamine, performance metrics, and peacock tails: Proxy failure is an inherent risk in goal-oriented systems
John, Y. J., Caldwell, L., McCoy, D. E., & Braganza, O. (2023) · Behavioral and Brain Sciences · doi:10.1017/s0140525x23002753 · Cited by 39
The broader spectrum of in-context learningPreprint
Lampinen, A. K., Chan, S. C. Y., Singh, A. K., & Shanahan, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.03782 · Cited by 39
Self-citation
Meta-learned models of cognition
Binz, M., Dasgupta, I., Jagadish, A. K., Botvinick, M., Wang, J. X., & Schulz, E. (2024) · Behavioral and Brain Sciences · doi:10.1017/s0140525x23003266 · Cited by 38
The influence of mental state attributions on trust in large language models
Colombatto, C., Birch, J., & Fleming, S. M. (2025) · Communications Psychology · doi:10.1038/s44271-025-00262-1 · Cited by 34 · cited 3×
“For example, as chatbots become ubiquitous, their influence on beliefs in the broader population may increasingly operate through their perceived impartiality and access to knowledge, rather than companionship-reinforcement 88 .”
Invisible Saboteurs: Sycophantic LLMs Mislead Novices in Problem-Solving Tasks
Bo, J. Y., Kazemitabaar, M., Deng, M., Inzlicht, M., & Anderson, A. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791365 · Cited by 33
Measuring Faithfulness in Chain-of-Thought Reasoning
Lanham, T., Chen, A., Radhakrishnan, A., Steiner, B., Denison, C., Hernandez, D., Li, D., Durmus, E., Hubinger, E., Kernion, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.13702 · Cited by 33
Real-world use of large language models for mental health in 2024
Stade, E. C., Tait, Z., Campione, S. T., Stirman, S. W., & Eichstaedt, J. C. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02842-9 · Cited by 32
Investigating Affective Use and Emotional Well-being on ChatGPT
Phang, J., Lampe, M., Ahmad, L., Agarwal, S., Fang, C. M., Liu, A. R., Danry, V., Lee, E., Chan, S. W. T., Pataranutaporn, P., et al. (2025) · arXiv · doi:10.48550/arxiv.2504.03888 · Cited by 29
Sycophantic AI increases attitude extremity and overconfidencePreprint
Rathje, S., Ye, M., Globig, L. K., Pillai, R. M., de Mello, V. O., & Bavel, J. J. V. (2025) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/vmyek_v1 · Cited by 29
Many-shot Jailbreaking
Anil, C., Durmus, E., Panickssery, N., Sharma, M., Benton, J., Kundu, S., Batson, J., Tong, M., Mu, J., Ford, D., et al. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-4121 · Cited by 25
On Targeted Manipulation and Deception when Optimizing LLMs for User FeedbackPreprint
Williams, M., Carroll, M., Narang, A., Weisser, C. N., Murphy, B. J., & Dragan, A. D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.02306 · Cited by 24
Can AI really help solve the loneliness epidemic?
Montag, C., Spapé, M., & Becker, B. (2025) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2025.08.002 · Cited by 22
A17 HD brain-train: neuroplasticity as a target to improve function in huntington’s disease
Papoutsi, M., Weiskopf, N., Langbehn, D. R., Reilmann, R., Rees, G. E., & Tabrizi, S. J. (2016) · Journal of Neurology Neurosurgery & Psychiatry · doi:10.1136/jnnp-2016-314597.17 · Cited by 21
INTIMA: A Benchmark for Human-AI Companionship BehaviorPreprint
Kaffee, L.-A., Pistilli, G., & Jernite, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.09998 · Cited by 20
Anthropomorphic tendencies in autism: A conceptual replication and extension of White and Remington (2019) and preliminary development of a novel anthropomorphism measure
Clutterbuck, R. A., Shah, P., Leung, H. S., Callan, M. J., Gjersoe, N. L., & Livingston, L. A. (2021) · Autism · doi:10.1177/13623613211039387 · Cited by 16
Accounting for Sycophancy in Language Model Uncertainty Estimation
Sicilia, A., İnan, M., & Alikhani, M. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.438 · Cited by 16
The Use of AI Chatbots for Autistic People: A Double-Edged Sword of Digital Support and Companionship
Papadopoulos, C. (2025) · Neurodiversity · doi:10.1177/27546330251370657 · Cited by 16
EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health SafetyPreprint
Qiu, J., He, Y., Juan, X., Wang, Y., Liu, Y., Yao, Z., Wu, Y., Jiang, X., Yang, L., & Wang, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.09689 · Cited by 14
Transdiagnostic types of formal thought disorder and their association with gray matter brain structure: a model-based cluster analytic approach
Stein, F., Gudjons, A. M., Brosch, K., Keunecke, L. M., Pfarr, J., Teutenberg, L., Thomas‐Odenthal, F., Usemann, P. L., Wersching, H., Wroblewski, A., et al. (2025) · Molecular Psychiatry · doi:10.1038/s41380-025-03009-w · Cited by 12
The Psychogenic Machine: Simulating AI Psychosis, Delusion Reinforcement and Harm Enablement in Large Language Models
Yeung, J. A., Dalmasso, J., Foschini, L., Dobson, R. J., & Kraljevic, Z. (2025) · arXiv · doi:10.48550/arxiv.2509.10970 · Cited by 12
The problem of atypicality in LLM-powered psychiatry
Garcia, B., Chua, E. Y. S., & Brah, H. S. (2025) · Journal of Medical Ethics · doi:10.1136/jme-2025-110972 · Cited by 10
Increasing happiness through conversations with artificial intelligence
Heffner, J., Qin, C., Chadwick, M., Knutsen, C., Summerfield, C., Kurth-Nelson, Z., & Rutledge, R. B. (2025) · arXiv · doi:10.48550/arxiv.2504.02091 · Cited by 10
Self-citation
Large Language Models Are Echo Chambers
Nehring, J., Gabryszak, A., Jürgens, P., Burchardt, A., Schaffer, S., Spielkamp, M., & Stark, B. (2024) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4zo8zahv3d3q · Cited by 10
Real-World Use of Large Language Models for Mental Health in 2024
Stade, E. C., Tait, Z., Campione, S. T., et al. (2026) · doi:10.31219/osf.io/ygx5q_v1 · Cited by 10
Persona Vectors: Monitoring and Controlling Character Traits in Language Models
Chen, R., Arditi, A., Sleight, H., Evans, O., & Lindsey, J. (2025) · arXiv · doi:10.48550/arxiv.2507.21509 · Cited by 7
Structured AI Dialogues Can Increase Happiness and Meaning in LifePreprint
Schöne, J. P., Salecha, A., Lyubomirsky, S., Eichstaedt, J. C., & Willer, R. (2025) · doi:10.31234/osf.io/2bf7t_v1 · Cited by 6
Conversational AI increases political knowledge as effectively as self-directed internet search
Luettgau, L., Kirk, H. R., Hackenburg, K., Bergs, J., Davidson, H., Ogden, H., Siddarth, D., Huang, S., & Summerfield, C. (2026) · PNAS Nexus · doi:10.1093/pnasnexus/pgag305 · Cited by 3
Self-citation
Human-inspired Episodic Memory for Infinite Context LLMs
Fountas, Z., Benfeghoul, M. A., Oomerjee, A., Christopoulou, F., Lampouras, G., Bou-Ammar, H., & Wang, J. (2024) · arXiv · doi:10.48550/arxiv.2407.09450 · Cited by 3
Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis
Kumar, A., Clune, J., Lehman, J., & Stanley, K. O. (2025) · arXiv · doi:10.48550/arxiv.2505.11581 · Cited by 2
How Overconfidence in Initial Choices and Underconfidence Under Criticism Modulate Change of Mind in Large Language ModelsPreprint
Kumaran, D., Fleming, S. M., Markeeva, L., Heyward, J., Banino, A., Mathur, M., Pascanu, R., Osindero, S., Martino, B. D., Veličković, P., et al. (2025) · arXiv (Cornell University) · Cited by 1

Research works printed but not linked: 7

Shevlin, H. All too human? Identifying and mitigating ethical risks of Social AI. Law Ethics Technol. 1, 1–22 (2024).
Zao-Sanders, M. How people are really using gen AI in 2025. Harvard Business Review (9 April 2025).
Huang, L. et al. A survey on hallucination in large language models: principles, taxonomy, challenges and open questions. ACM Trans. Inf. Syst. 43, 1–55 (2025).
Ibrahim, L., Sofia, H. F. & Rocher, L. Training language models to be warm and empathetic makes them less reliable and more sycophantic. Preprint at https://doi.org/10.48550/arXiv.2507.21919 (2025). · DOI · Paper page
Shevlin, H. The anthropomimetic turn in contemporary AI. Preprint at https://philarchive.org/rec/SHETAT-11 (2025).
Varadarajan, V. et al. The consistent lack of variance of psychological factors expressed by LLMs and Spambots. In Proc. 1st Workshop on GenAI Content Detection (eds Alam, F. et al.) 111–119 (2025).
Radford, A., Narasimhan, K., Salimans, T. & Sutskever, I. Improving language understanding by generative pre-training. Preprint at OpenAI https://openai.com/index/language-unsupervised/ (2018).

Sorted by citations

Delusional Experiences Emerging From AI Chatbot Interactions or “AI Psychosis”
Hudon, A., & Stip, E. (2025) · JMIR Mental Health · doi:10.2196/85799 · Cited by 88
Mindbench.ai: an actionable platform to evaluate the profile and performance of large language models in a mental healthcare context
Dwyer, B., Flathers, M., Sano, A., Dempsey, A. G., Cipriani, A., Gazi, A. H., Hill, B., Gorban, C., Rodríguez, C. I., Stromeyer, C., et al. (2025) · NPP—Digital Psychiatry and Neuroscience · doi:10.1038/s44277-025-00049-6 · Cited by 31
Can AI chatbots trigger psychosis? What the science says
Fieldhouse, R. (2025) · Nature · doi:10.1038/d41586-025-03020-9 · Cited by 21
Hallucinating with AI: Distributed Delusions and “AI Psychosis”
Osler, L. (2026) · Philosophy & Technology · doi:10.1007/s13347-026-01034-3 · Cited by 21
Characterizing Delusional Spirals through Human-LLM Chat Logs
Moore, J., Mehta, A., Agnew, W., Anthis, J. R., Louie, R., Mai, Y., Yin, P., Cheng, M., Paech, S. J., Klyman, K., et al. (2026) · Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3805689.3806443 · Cited by 15

See all 107

Authors: Dohnány, S., Kurth-Nelson, Z., Spens, E., Luettgau, L., Reid, A., Gabriel, I., Summerfield, C., Shanahan, M., & Nour, M. M.
Affiliations: University of Oxford (GB)
University College London (GB)
Oxford Health NHS Foundation Trust (GB)
University of London (GB)
Imperial College London (GB)
Corresponding author: Nour, Matthew M.
ORCID: Dohnány, Sebastian
Kurth-Nelson, Zeb
Spens, Eleanor
Luettgau, Lennart
Reid, Alastair
Gabriel, Iason
Summerfield, Christopher
Shanahan, Murray
Nour, Matthew M.
Venue: Nature Mental Health
Publisher: Springer Science and Business Media LLC
Published: 2026-03-10
Volume: 4
Issue: 3
Pages: 336-345
ISSN: 2731-6076 (ISSN-L)
DOI: 10.1038/s44220-026-00595-8
PMC: PMC7618964
OpenAlex: W7134947688
Cited by: 51
References: 95
Citations by year: 2026: 51
FWCI: 300.37
Type: journal article
Language: en
Open access: green
pmc.ncbi.nlm.nih.gov
Licence: cc-by
www.springernature.com/gp/researchers/text-and-data-mining (tdm)
www.springernature.com/gp/researchers/text-and-data-mining (vor)
Locations: Nature Mental Health (publishedVersion)
PubMed (publishedVersion)
PubMed Central (publishedVersion, cc-by)
MPG.PuRe (Max Planck Society) (submittedVersion)
Funders: Wellcome Trust (315364/Z/24/Z)
Merton College, University of Oxford (NA)
Topics: Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords: AI chatbots, mental health, emotional support, social isolation, sycophancy, role play, belief updating, suicide, AI regulation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Technological folie à deux: feedback loops between AI chatbots and mental health

Abstract

Overview

Integrity signals

Figures

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks