A scoping review on the mental health harms of LLM-based chatbots

Existing reviews and meta-analyses of LLM-CBs for mental health focus on applications of LLM-CBs as clinical intervention tools, often under research-protocol conditions.

“Although reviews and meta-analyses of LLM-CBs for mental health exist13,14,29,30, as well as scoping reviews on the implementation of LLM-CBs in mental health31,32, their focus is narrowed to applications of LLM-CBs as clinical intervention tools.”

• Also stated in: § Introduction · plain

  “Hence, existing reviews focus on the structured deployment of LLM-CBs implemented as mental health tools, often under research-protocol conditions.”

• cites [13] Feng, X., Tian, L., Ho, G. W. K., Yorke, J. & Hui, V. The Effectiveness of AI chatbots in alleviating mental distress and promoting health behaviors among adolescents and young adults: systematic review and meta-analysis. J. Med. Internet Res. 27, e79850 (2025). sourcePartly supported

  The Methods (Eligibility Criteria; Statistical Analysis) limit the review to randomized trials of chatbot interventions, supporting the characterization of research-protocol conditions. But the Results (Results of Systematic Review) say most chatbots were rule-based or retrieval-based and only three studies explored generative approaches; the review also covers health behaviors and nonclinical populations. It therefore does not support describing its focus as narrowly LLM-based clinical intervention tools. (LLM, conf=0.87)

  Experimental — accuracy not yet measured
• cites [14] Li, H., Zhang, R., Lee, Y.-C., Kraut, R. E. & Mohr, D. C. Systematic review and meta-analysis of AI-based conversational agents for promoting mental health and well-being. NPJ Digit. Med. 6, 236 (2023). sourcePartly supported

  The Methods (Search strategy and selection criteria; page unavailable) define this as a review of experimental studies of AI-based conversational agents used as mental-health interventions, supporting the intervention and research-study aspects of the claim. But its scope is broader than LLM-based chatbots: the Discussion says only five included studies used generative agents, and the Results report that half the studies involved nonclinical populations. The Discussion also analyzes user experience, so describing the review as narrowed to clinical intervention tools overstates its focus. (LLM, conf=0.88)

  Experimental — accuracy not yet measured
• cites [29] Zhang, Q. et al. Generative AI mental health chatbots as therapeutic tools: systematic review and meta-analysis of their role in reducing mental health issues. J. Med. Internet Res. 27, e78238 (2025). sourceSupported
• cites [30] Kuhail, M. A. et al. A systematic review on mental health chatbots: trends, design principles, evaluation methods, and future research agenda. Hum. Behav. Emerg. Technol. 2025, 9942295 (2025). 31 pages. sourcePartly supported

  Section 3 (Related Work; page not available) says prior mental-health-chatbot reviews examine intervention effectiveness, but also lists design, user experience, technology, ethics, and other objectives. Section 6.2.5 discusses LLMs but does not establish that reviews of LLM-based chatbots are narrowly focused on clinical intervention tools or research-protocol conditions. (LLM, conf=0.78)

  Experimental — accuracy not yet measured
• cites [31] Hua, Y. et al. A scoping review of large language models for generative tasks in mental health care. npj Digit. Med. 8, 230 (2025). sourceDisputed

  The Results (Applications and model information; page not provided) classify the applications reviewed into six types, including clinical assistants and education as well as counselling and therapy. The review therefore does not narrow its focus to clinical intervention tools. The cited passages do not establish the added characterization that applications were often studied under research-protocol conditions. (LLM, conf=0.87)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured
• cites [32] Jin, Y. et al. The applications of large language models in mental health: scoping review. J. Med. Internet Res. 27, e69284 (2025). sourceDisputed

  The Results section (Categorization of Studies Based on Mental Health Applications; page unavailable) says this scoping review covers screening and detection, clinical treatment and intervention, and counseling and education. Screening or detection appears in 67 of 95 studies, versus 31 of 95 for treatment and intervention, contrary to the claim that the review's focus is narrowed to clinical intervention tools. The cited passages do not establish the additional characterization about research-protocol conditions. (LLM, conf=0.89)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured

Surveys and forum analyses on companion AI use found that usage was generally associated with negative mental health.

“Meanwhile, surveys and forum analyses on companion AI use found that usage was generally associated with negative mental health81 and that companion AIs may provide harmful content and reinforce dependence82.”

• cites [81] Maples, B., Cerit, M., Vishwanath, A. & Pea, R. Loneliness and suicide mitigation for students using GPT3-enabled chatbots. Npj Ment. health Res. 3, 4 (2024). sourceDisputed

  The Results (Negative feedback) say there was “no clear pattern of negative outcomes reported by a significant portion of participants,” although some participants raised concerns. The Discussion (Use of Replika during suicidal ideation) instead describes high perceived social support and reports that some participants credited Replika with halting suicidal ideation. This contradicts the claim that use was generally associated with negative mental health. (LLM, conf=0.88)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured

Measures of AI dependence consist either of adaptations from other scales of behavioral addiction to technologies (e.g., social media, smartphone, or internet addiction) or self-developed measures.

“Measures of “AI dependence” consist either of adaptations from other scales of behavioral addiction to technologies (e.g., social media, smartphone, or internet addiction)100–102, or self-developed measures103–106.”

• cites [100] Huang, S. et al. AI technology panic: Is AI dependence bad for mental health? A cross-lagged panel model and the mediating roles of motivations for AI use among adolescents. Psychol. Res. Behav. Manag. 17, 1087–1102 (2024). sourceSupported
• cites [101] Hu, B., Mao, Y. & Kim, K. J. How social anxiety leads to problematic use of conversational AI: the roles of loneliness, rumination, and mind perception. Comput. Hum. Behav. 145, 107760. https://doi.org/10.1016/j.chb.2023.107760 (2023). sourceNot checked — no full text of the source
• cites [102] Liu, A. R., Pataranutaporn, P. & Maes, P. Chatbot companionship: a mixed-methods study of companion chatbot usage patterns and their relationship to loneliness in active users. https://arxiv.org/abs/2410.21596 (2024). sourceDisputed

  For the self-developed-measures part of the citing sentence, §3.2.1 says the paper measured problematic chatbot use with an adapted Generalized Problematic Internet Use Scale, not a self-developed dependence measure. §3.2.2 says its custom items measured usage, motivations, conversation topics, and perceived relationship impacts. (LLM, conf=0.85)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured
• cites [103] Zhang, X., Yin, M., Zhang, M., Li, Z. & Li, H. The development and validation of an artificial intelligence chatbot dependence scale. Cyberpsychol. Behav. Soc. Netw. 28, 126–131 (2025). sourceNot checked — no full text of the source
• cites [104] Ibrahim, L., Huang, S., Ahmad, L., Bhatt, U. & Anderljung, M. Towards interactive evaluations for interaction harms in human-AI systems. AIES 8, 1302–1310 (2025). sourceCould not verify

  The citation falls in the group offered for self-developed measures of AI dependence. Section 2.3 calls for quantifying dependency levels, and Section 4.2 discusses measures of human impact, but neither describes a self-developed AI-dependence measure. Searches for such a measure and for adapted addiction scales found no supporting passage in our copy; page numbers were not provided. (LLM, conf=0.75)

  Experimental — accuracy not yet measured
• cites [105] Yankouskaya, A. et al. LLM-D12: A dual-dimensional scale of instrumental and relational dependencies on large language models. ACM Trans. Web. https://doi.org/10.1145/3765895 (2025). sourceSupported
• cites [106] Yu, Q. & Lan, X. Exploring the impact of anthropomorphism in role-playing AI chatbots on media dependency: a case study of Xuanhe AI. In Proceedings of the Twelfth International Symposium of Chinese CHI. 170–181 (ACM, New York, NY, USA, 2024). sourceSupported

Such vignette or benchmark studies are considered first-tier, foundational in recent evaluation frameworks such as the HAICEF.

“such vignette or benchmark studies are considered first-tier, foundational in recent evaluation frameworks such as the HAICEF142,143.”

• cites [142] Hua, Y. et al. Charting the evolution of artificial intelligence mental health chatbots from rule-based systems to large language models: a systematic review. World Psychiatry 24, 383–394 (2025). sourcePartly supported

  The Methods section places controlled tests such as scripted scenarios in T1, the foundational bench-testing tier, supporting the characterization of vignette-style tests as first-tier evaluations. The paper describes its own three-tier framework but does not identify it as HAICEF or establish how HAICEF classifies these studies. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [143] Hua, Y. et al. Standardizing and scaffolding health care AI-chatbot evaluation: systematic review. JMIR AI 4, e69006 (2025). sourceDisputed

  The Discussion (Principal Findings; page unavailable) identifies safety, privacy, and fairness—not vignette or benchmark studies—as HAICEF’s foundation and step 1. It describes checking for bias as a foundational first step. The framework’s tiers are evaluation domains, not categories of study designs. (LLM, conf=0.78)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured

LLM-CBs may, in certain situations (e.g., via jailbreaking), generate inappropriate or harmful content, such as guides to self-harm.

“As discussed above and mentioned by nine articles, LLM-CBs may, in certain situations (e.g., via jailbreaking), generate inappropriate or harmful content, such as guides to self-harm36.”

• cites [36] Ferrario, A., Sedlakova, J. & Trachsel, M. The role of humanization and robustness of large language models in conversational artificial intelligence for individuals with depression: a critical analysis. JMIR Ment. Health 11, e56569 (2024). sourceSupported

Manipulation by LLM-CBs may occur based on users' vulnerabilities and negative thought patterns that users shared beforehand.

“Manipulation may occur based on users’ vulnerabilities and negative thought patterns that users shared beforehand39,49.”

• cites [39] Tavory, T. Regulating AI in mental health: ethics of care perspective. JMIR Ment. Health 11, e58493 (2024). sourcePartly supported

  The “Manipulation and Vulnerability” section says emotional AI may manipulate people by exploiting their vulnerabilities and identifies a concern about AI-chatbot manipulation. The “Affective Computing and Emotional AI” section discusses chatbots interpreting thoughts and feelings, but our copy does not say manipulation is based on negative thought patterns users shared beforehand. No page numbers are available in our copy. (LLM, conf=0.85)

  Experimental — accuracy not yet measured
• cites [49] McStay, A. Emulated empathy: can risks be countered by a soft-law standard?. IEEE Trans. Technol. Soc. 6, 250–256 (2025). sourceNot checked — no full text of the source

Biases in LLM-based chatbots may perpetuate social inequities, potentially harming underrepresented or stigmatized groups in a (mental) health context, e.g., through errors in diagnostic classifications, proposals of interventions, or mental health advice.

“Biases in LLM-based chatbots may perpetuate social inequities, potentially harming underrepresented or stigmatized groups in a (mental) health context, e.g., through errors in diagnostic classifications, proposals of interventions, or mental health advice56.”

• cites [56] Timmons, A. C. et al. A call to action on assessing and mitigating bias in artificial intelligence applications for mental health. Perspect. Psychol. Sci.18, 1062–1096 (2023). sourcePartly supported

  Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  Experimental — accuracy not yet measured

Sycophantic responses may validate or reinforce delusional thoughts, harmful behavior, or maladaptive thinking by creating the impression that expressed beliefs are externally socially validated when they are actually mirrored by the LLM-CB.

“Sycophantic responses may validate or reinforce delusional thoughts, harmful behavior, or maladaptive thinking37,42,43,49 by creating the impression that expressed beliefs are externally socially validated when they are actually mirrored by the LLM-CB57.”

• cites [37] Frances, A. Warning: AI chatbots will soon dominate psychotherapy. Br. J. Psychiatr. 1–5 (2025). sourcePartly supported

  The ‘Dangers of artificial intelligence therapy’ section says chatbot algorithms can validate bizarre or dangerous thoughts and may encourage people to expand on them. It does not describe the specific mechanism attributed to it here: users mistaking an LLM chatbot’s mirroring of their beliefs for independent social validation. Page information is unavailable in our copy. (LLM, conf=0.83)

  Experimental — accuracy not yet measured
• cites [42] Looi, J. C., Allison, S., Bastiampillai, T., Reutens, S. & Looi, R. C. Illusions of intelligence, connection and reality: perils of large-language AI models for people with severe mental illness. Australas. Psychiatr. 34, 5–7 (2026). sourceNot checked — no full text of the source
• cites [43] Smith, M. G., Bradbury, T. N. & Karney, B. R. Can generative AI chatbots emulate human connection? A relationship science perspective. Perspect. Psychol. Sci. 20, 1081–1099 (2025). sourcePartly supported

  Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  Experimental — accuracy not yet measured

One article noted that LLM-CB use for mental health may risk reinforcement of a user's identification as a patient instead of focusing on improving symptoms.

“One article noted that LLM-CB use for mental health may risk the reinforcement of a user’s identification as a patient (e.g., via sycophantic responses) instead of focusing on improving symptoms37.”

• cites [37] Frances, A. Warning: AI chatbots will soon dominate psychotherapy. Br. J. Psychiatr. 1–5 (2025). sourcePartly supported

  In “Dangers of artificial intelligence therapy” (page unavailable), the ‘Career patienthood’ passage says AI therapy could magnify the risk of people over-identifying with their disorders. That supports the concern about reinforced patient identity, but the passage does not say this happens instead of improving symptoms or specifically through sycophantic responses. (LLM, conf=0.83)

  Experimental — accuracy not yet measured

A lack of empathy in LLM-CBs may render certain psychotherapeutic features such as countertransference ineffective.

“A lack of empathy may also render certain psychotherapeutic features such as countertransference ineffective52.”

• cites [52] Rahsepar Meadi, M., Bernstein, J. S., Batelaan, N., van Balkom, A. J. L. M. & Metselaar, S. Does a lack of emotions make chatbots unfit to be psychotherapists?. Bioethics 38, 503–510 (2024). sourcePartly supported

  Section 5 says that if mental health chatbots lack emotions, they cannot experience countertransference and patients may lose its potential benefits. It also distinguishes countertransference from empathy; the paper attributes the loss of countertransference to a lack of emotions, not specifically to a lack of empathy. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

Two articles mentioned that therapeutic misconception may arise when LLM-CB behaviors lead to users overestimating its abilities to provide psychotherapeutic support while underestimating its limitations.

“Mentioned by two articles, therapeutic misconception may arise when LLM-CB behaviors (e.g., simulated empathy, validation) lead to users overestimating its abilities to provide psychotherapeutic support while underestimating its limitations41.”

• cites [41] Khawaja, Z. & Bélisle-Pipon, J.-C. Your robot therapist is not your therapist: understanding the role of AI-powered mental health chatbots. Front. Digit. Health 5, 1278186 (2023). sourceSupported

Protective guardrails on LLM-CBs can be bypassed (jailbreaking) to enable generation of otherwise restricted content, including harmful advice like guides on self-harm.

“Although LLM-CBs are often refined by protective guardrails, those can be bypassed (i.e., “jailbreaking”) to enable the generation of otherwise restricted content, including harmful advice like guides on self-harm36.”

• cites [36] Ferrario, A., Sedlakova, J. & Trachsel, M. The role of humanization and robustness of large language models in conversational artificial intelligence for individuals with depression: a critical analysis. JMIR Ment. Health 11, e56569 (2024). sourceSupported

Systematic and scoping reviews show that LLM-CB risks (e.g., consistencies, hallucinations, ethical risks) may outweigh potential benefits.

“Systematic and scoping reviews show that LLM-CB risks (e.g., consistencies, hallucinations, ethical risks) may outweigh potential benefits33,76,77.”

• cites [33] Rahsepar Meadi, M. et al. Exploring the ethical challenges of conversational AI in mental health care: scoping review. JMIR Ment. Health 12, e60432 (2025). sourcePartly supported

  The Results (Ethical Themes; page unavailable) identify harmful or hallucinatory suggestions, and the Discussion (Principal Findings; page unavailable) says such concerns could undermine CAI’s potential benefits. But the Discussion (Suggestions for Future Research) calls for evaluating risks against benefits to determine whether use is justifiable; the review does not conclude that risks may outweigh benefits. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites [76] Guo, Z. et al. Large language models for mental health applications: systematic review. JMIR Ment. health 11, e57400 (2024). sourceSupported
• cites [77] Sarkar, S., Gaur, M., Chen, L. K., Garg, M. & Srivastava, B. A review of the explainability and safety of conversational agents for mental health to identify avenues for improvement. Front. Artif. Intell. 6, 1229805. https://doi.org/10.3389/frai.2023.1229805 (2023). sourcePartly supported

  Section 2.1 says the consequences of LLMs’ black-box behavior are more concerning than their utility, particularly in mental health. Sections 5 and 6.1 describe incoherent, hallucinated, and unsafe responses from virtual mental health assistants. The source supports the risk–benefit concern in that setting, but it does not identify itself as a systematic or scoping review or establish the claim for LLM chatbots generally. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

LLM-CBs generally do not sufficiently adhere to professional guidelines or standards.

“LLM-CBs generally do not sufficiently adhere to professional guidelines or standards65,71–73 and, according to health professionals’ analyses on chatbot responses, show special harmful potential in the context of crisis situations71.”

• cites [65] Grabb, D., Lamparth, M. & Vasan, N. Risks from language models for automated mental healthcare: ethics and structure for implementation. https://arxiv.org/abs/2406.11852 (2024). sourcePartly supported

  The Introduction (page unavailable) says none of ten tested language models matched the standard provided by human professionals when responding to mental-health symptoms. That supports a concern about professional-level performance, but the study assessed selected models in mental-health scenarios; it did not establish that LLM chatbots generally fail to adhere to professional guidelines or standards. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [71] Moylan, K. & Doherty, K. Expert and interdisciplinary analysis of AI-driven chatbots for mental health support: mixed methods study. J. Med. Internet Res. 27, e67114 (2025). sourcePartly supported

  The Discussion (page unavailable) reports that professionals found Wysa and Replika to differ substantially from professional practice and identifies concerns about ethical principles. This supports the concern behind the claim, but the study evaluates those two chatbots rather than establishing that LLM-based chatbots generally fail to adhere to professional guidelines or standards. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites [72] Scholich, T., Barr, M., Wiltsey Stirman, S. & Raj, S. A comparison of responses from human therapists and large language model-based chatbots to assess therapeutic communication: mixed methods study. JMIR Ment. Health 12, e69709 (2025). sourcePartly supported

  The Discussion’s Conclusions report that the general-purpose chatbots tested fell short of therapeutic practice and call for alignment with ethical and clinical standards. The paper does not assess adherence to specific professional guidelines, and its findings about the tested chatbots do not establish the claim for LLM-based chatbots generally. Our copy provides section labels but no page numbers. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites [73] Spallek, S., Birrell, L., Kershaw, S., Devine, E. K. & Thornton, L. Can we use ChatGPT for mental health and substance use education? Examining its quality and potential harms. JMIR Med. Educ. 9, e51243 (2023). sourcePartly supported

  The Results (Evaluation Metrics; page not provided) report that GPT-4 responses to direct queries and simple prompts adhered less well to communication guidelines, and that 23% of assessed outputs contained stigmatizing language. The Discussion (Future Opportunities; page not provided) says initial outputs will not adhere to all guidelines. These findings support a concern about GPT-4 in mental health and substance use education, but not the broader claim that LLM-based chatbots generally fail to meet professional standards. (LLM, conf=0.87)

  Experimental — accuracy not yet measured

According to health professionals' analyses on chatbot responses, LLM-CBs show special harmful potential in the context of crisis situations.

“LLM-CBs generally do not sufficiently adhere to professional guidelines or standards65,71–73 and, according to health professionals’ analyses on chatbot responses, show special harmful potential in the context of crisis situations71.”

• cites [71] Moylan, K. & Doherty, K. Expert and interdisciplinary analysis of AI-driven chatbots for mental health support: mixed methods study. J. Med. Internet Res. 27, e67114 (2025). sourceSupported

Data security concerns stem from security risks relating to sensitive health data provided by users or by clinicians.

“Data security concerns (mentioned by 13 articles) stem from security risks relating to sensitive health data provided by users or by clinicians36–39,44,47,50,51,55.”

• cites [36] Ferrario, A., Sedlakova, J. & Trachsel, M. The role of humanization and robustness of large language models in conversational artificial intelligence for individuals with depression: a critical analysis. JMIR Ment. Health 11, e56569 (2024). sourcePartly supported

  In “Contextualizing the Robustness of LLMs” (page unavailable), the paper warns that nonrobust models can be tricked into revealing personal information. “LLMs in Health Care and Mental Health Applications” describes processing patients’ responses and mental-health histories. These passages support concern about sensitive patient data, but do not specifically identify data provided by clinicians as a source of that risk. (LLM, conf=0.83)

  Experimental — accuracy not yet measured
• cites [37] Frances, A. Warning: AI chatbots will soon dominate psychotherapy. Br. J. Psychiatr. 1–5 (2025). sourceSupported
• cites [38] Liu, J. ChatGPT: perspectives from human-computer interaction and psychology. Front. Artif. Intell. 7, 1418869 (2024). sourcePartly supported

  The Psychological support section says that sharing sensitive information with AI for mental health therapy poses privacy and data-security risks. The paper also discusses doctors using patients’ medical data, but the passages do not connect those risks specifically to data provided by clinicians. Page numbers are unavailable in our copy. (LLM, conf=0.85)

  Experimental — accuracy not yet measured
• cites [39] Tavory, T. Regulating AI in mental health: ethics of care perspective. JMIR Ment. Health 11, e58493 (2024). sourcePartly supported

  The Privacy and Safety and Security sections (pages unavailable) link sensitive medical data to stricter security standards and discuss unauthorized breaches. The confidentiality section also describes sensitive information recorded by mental health apps. These passages support the concern about users’ health data, but do not specifically say that the data at risk may be provided by clinicians. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [44] Jurblum, M. & Selzer, R. Potential promises and perils of artificial intelligence in psychotherapy—the AI psychotherapist (APT). Australas. Psychiatr. 33, 103–105 (2025). sourcePartly supported

  In “Potential risks of APTs” (page unavailable), the paper identifies data security as a concern and discusses risks involving therapy transcripts, supporting the concern about sensitive information shared in therapy. It does not say that clinicians provide the data at issue, so that part of the attribution is unsupported. (LLM, conf=0.83)

  Experimental — accuracy not yet measured
• cites [47] Blease, C. & Torous, J. ChatGPT and mental healthcare: balancing benefits with risks of harms. BMJ Ment. Health 26. https://doi.org/10.1136/bmjment-2023-300884 (2023). sourceSupported

A three-study empirical analysis found that LLM-CBs often show difficulties recognizing users' mental distress.

“A three-study empirical analysis found that LLM-CBs often show difficulties recognizing users’ mental distress, which may lead to inappropriate responses75.”

• cites [75] De Freitas, J., Uğuralp, A. K., Oğuz-Uğuralp, Z. & Puntoni, S. Chatbots and mental health: insights into the safety of generative AI. J. Consum. Psychol. 34, 481–491 (2024). sourcePartly supported

  The Conclusions (page not available) say that Study 2 found companion AIs often unable to recognize signs of distress and mental health issues, and they describe the paper’s three numbered studies. The copy describes the tested systems as companion AIs powered by black-box algorithms, but does not establish that they are specifically LLM-based chatbots. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

Choi et al. analyzed patient-chatbot conversations and found the risk of harmful responses on eating disorders.

“Denecke et al. analyzed various cases of harmful LLM-CB use, including a case of an LLM-CB promoting harmful eating disorder behavior among teenagers83, and Choi et al. analyzed patient-chatbot conversations and found the risk of harmful responses on eating disorders84.”

• cites [84] Choi, R., Kim, T., Park, S., Kim, J. G. & Lee, S.-J. Private yet social: how LLM chatbots support and challenge eating disorder recovery. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–19 (ed. Yamashita, N. et al.) (ACM, New York, NY, USA, 2025). sourceSupported

Systematic and scoping reviews note that literature on the association between functional LLM-CB use and critical thinking skills remains inconclusive.

“Systematic and scoping reviews note that literature on the association between functional LLM-CB use and critical thinking skills remains inconclusive90–92.”

• cites [90] Liu, J., Sihes, A. J. B. & Lu, Y. How do generative artificial intelligence (AI) tools and large language models (LLMs) influence language learners’ critical thinking in EFL education? A systematic review. Smart Learn. Environ. 12. https://doi.org/10.1186/s40561-025-00406-0 (2025). sourcePartly supported

  The section ‘Do generative AI tools and LLMs kill or nurture critical thinking in EFL education?’ (page unavailable) reports both benefits and risks for critical thinking, with outcomes depending on how the tools are used. That supports a mixed-evidence reading, but the review concerns EFL learners and does not specifically assess the association between functional LLM-chatbot use and critical-thinking skills across a broader population. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites [91] Mohammad, B. et al. The pros and cons of using ChatGPT in medical education: a scoping review. Stud. Health Technol. Inform. 305, 644–647 (2023). sourceNot checked — no full text of the source
• cites [92] Parsakia, K. The effect of chatbots and AI on the self-efficacy, self-esteem, problem-solving and critical thinking of students. HN 1, 71–76 (2023). sourcePartly supported

  Section 2.1.4 (page not provided) describes diverse findings on chatbots and critical thinking, and the Discussion says their effects can be positive, limited, or context-dependent. That supports the claim that the evidence is inconclusive. The paper does not establish that it is a systematic or scoping review, or specifically assess functional use of LLM-based chatbots. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

Various popular psychotherapy chatbots on OpenAI generally scored low on therapeutic orientation, monitoring, and risk evaluation.

“Sobowale et al. developed and validated a framework for the evaluation of the quality of psychotherapy chatbots: various popular psychotherapy chatbots on OpenAI generally scored low on therapeutic orientation, monitoring, and risk evaluation74.”

• cites [74] Sobowale, K. & Humphrey, D. K. Evaluating the quality of psychotherapy conversational agents: framework development and cross-sectional study. JMIR Format. Res. 9, e65605 (2025). sourcePartly supported

  The Abstract Results report low scores for therapeutic orientation and for monitoring and risk evaluation among the psychotherapy chatbots studied. The Methods describe developing and testing the CAPE framework, including an interrater-reliability assessment, but do not establish the broader claim that the framework was validated. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

Voluntary use of LLM-based chatbots predicts LLM-CB dependence.

“voluntary use of LLM-CBs predicted worse psychosocial outcomes and LLM-CB dependence111.”

• cites [111] Fang, C. M. et al. How AI and human behaviors shape psychosocial effects of extended chatbot use: a longitudinal randomized controlled study. https://arxiv.org/abs/2503.17473 (2025). sourceSupported

Self-developed measures may focus on symptoms of behavioral addiction adapted to LLM-CBs (e.g., craving, withdrawal, tolerance, loss of interests, negative impact on other aspects of life), potentially confounding emotional and functional dependence.

“Self-developed measures may focus on symptoms of behavioral addiction adapted to LLM-CBs (e.g., craving, withdrawal, tolerance, loss of interests, negative impact on other aspects of life)103,106, potentially confounding emotional and functional dependence;”

• cites [103] Zhang, X., Yin, M., Zhang, M., Li, Z. & Li, H. The development and validation of an artificial intelligence chatbot dependence scale. Cyberpsychol. Behav. Soc. Netw. 28, 126–131 (2025). sourceNot checked — no full text of the source
• cites [106] Yu, Q. & Lan, X. Exploring the impact of anthropomorphism in role-playing AI chatbots on media dependency: a case study of Xuanhe AI. In Proceedings of the Twelfth International Symposium of Chinese CHI. 170–181 (ACM, New York, NY, USA, 2024). sourcePartly supported

  Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  Experimental — accuracy not yet measured

Among Danish high school students, social-supportive LLM-CB use (compared to no or functional use) was associated with loneliness and decreased social support.

“Herbener and Damholdt found that among Danish high school students, social-supportive LLM-CB use (compared to no or functional use) was associated with loneliness and decreased social support; in addition, bad mood, need for self-disclosure, and sense of loneliness were triggers of social-supportive use, indicating that social LLM-CB use functions as a coping mechanism109.”

• cites [109] Herbener, A. B. & Damholdt, M. F. Are lonely youngsters turning to chatbots for companionship? The relationship between chatbot usage and social connectedness in Danish high-school students. Int. J. Hum. Comput. Stud. 196, 103409. https://doi.org/10.1016/j.ijhcs.2024.103409 (2025). sourcePartly supported

  The comparative analyses (pp. 7–8) report that social-supportive chatbot users were significantly lonelier than both non-users and utilitarian users. They reported significantly less perceived social support than non-users, but the difference from utilitarian users was not statistically significant. Thus, the claimed comparison with functional use holds for loneliness, not reliably for social support. (LLM, conf=0.90)

  Experimental — accuracy not yet measured

Social LLM-CB use functions as a coping mechanism.

“Herbener and Damholdt found that among Danish high school students, social-supportive LLM-CB use (compared to no or functional use) was associated with loneliness and decreased social support; in addition, bad mood, need for self-disclosure, and sense of loneliness were triggers of social-supportive use, indicating that social LLM-CB use functions as a coping mechanism109.”

• cites [109] Herbener, A. B. & Damholdt, M. F. Are lonely youngsters turning to chatbots for companionship? The relationship between chatbot usage and social connectedness in Danish high-school students. Int. J. Hum. Comput. Stud. 196, 103409. https://doi.org/10.1016/j.ijhcs.2024.103409 (2025). sourceSupported

LLM-CB use may be a maladaptive coping strategy.

“Huang et al. conducted a longitudinal analysis of 3,843 adolescents across two time points, finding that mental health problems (anxiety and depression) predicted LLM-CB dependence but not vice versa, mediated by escape motivation, again indicating that LLM-CB use may be a maladaptive coping strategy100.”

• cites [100] Huang, S. et al. AI technology panic: Is AI dependence bad for mental health? A cross-lagged panel model and the mediating roles of motivations for AI use among adolescents. Psychol. Res. Behav. Manag. 17, 1087–1102 (2024). sourcePartly supported

  The Methods (Participants) report a two-wave analysis of 3,843 adolescents, and the Discussion reports that anxiety and depression predicted later AI dependence, not vice versa, with escape motivation as a mediator. The Methods (Procedure) define AI broadly—including assistants, chatbots and robots—so the findings support a possible maladaptive coping interpretation for AI use generally, but not specifically for LLM-chatbot use. (LLM, conf=0.88)

  Experimental — accuracy not yet measured

Voluntary use of LLM-based chatbots predicts worse psychosocial outcomes.

“voluntary use of LLM-CBs predicted worse psychosocial outcomes and LLM-CB dependence111.”

• cites [111] Fang, C. M. et al. How AI and human behaviors shape psychosocial effects of extended chatbot use: a longitudinal randomized controlled study. https://arxiv.org/abs/2503.17473 (2025). sourceSupported

Bad mood, need for self-disclosure, and sense of loneliness were triggers of social-supportive LLM-CB use.

“Herbener and Damholdt found that among Danish high school students, social-supportive LLM-CB use (compared to no or functional use) was associated with loneliness and decreased social support; in addition, bad mood, need for self-disclosure, and sense of loneliness were triggers of social-supportive use, indicating that social LLM-CB use functions as a coping mechanism109.”

• cites [109] Herbener, A. B. & Damholdt, M. F. Are lonely youngsters turning to chatbots for companionship? The relationship between chatbot usage and social connectedness in Danish high-school students. Int. J. Hum. Comput. Stud. 196, 103409. https://doi.org/10.1016/j.ijhcs.2024.103409 (2025). sourceSupported

Emotionality in interactions between users and chatbots predicts LLM-CB dependence.

“Yuan et al. found that emotionality in interactions between users and chatbots predicted LLM-CB dependence120.”

• cites [120] Yuan, Z., Cheng, X. & Duan, Y. Impact of media dependence: How emotional interactions between users and chat robots affect human socialization?. Front. Psychol. 15, 1388860. https://doi.org/10.3389/fpsyg.2024.1388860 (2024). sourceSupported

A longitudinal correlational study showed that individual disposition (impaired mental health) predicted later problematic LLM-CB use but not vice versa.

“A longitudinal correlational study also showed that individual disposition (impaired mental health) predicted later problematic LLM-CB use but not vice versa100”

• cites [100] Huang, S. et al. AI technology panic: Is AI dependence bad for mental health? A cross-lagged panel model and the mediating roles of motivations for AI use among adolescents. Psychol. Res. Behav. Manag. 17, 1087–1102 (2024). sourcePartly supported

  The Results (Results from the Cross-Lagged Model; page not available) report that depression and anxiety predicted later AI dependence among adolescents, while AI dependence did not predict later depression or anxiety. But Methods (Measurement; page not available) measures dependence on AI generally, and the Discussion (Relationship Between Mental Health Problems and AI Dependence; page not available) explicitly describes an overall evaluation of AI technology rather than a specific chatbot. The finding therefore does not establish the claimed direction specifically for problematic LLM-chatbot use. (LLM, conf=0.91)

  Experimental — accuracy not yet measured

Risks of dependency were widely acknowledged by LLM-CB companion users.

“Hernandez et al. surveyed 827 LLM-CB companion users, finding that risks of dependency were widely acknowledged by users124.”

• cites [124] Hernandez, H. E. et al. Dependency on Meta AI chatbot in messenger among STEM and non-STEM students in higher education. https://arxiv.org/abs/2507.21059 (2025). sourcePartly supported

  The “Dependency Risks Associated with AI Chatbot Use” section reports moderate concern: students agreed that overuse could reduce critical thinking and independent learning, but disagreed that they felt less capable of solving problems without chatbots. The study surveyed 872 tertiary students using Meta AI in Messenger, not 827 LLM-CB companion users as the citing sentence states. (LLM, conf=0.91)

  Experimental — accuracy not yet measured

Anthropomorphizing chatbots predicts LLM-CB use.

“Yu et al. found that anthropomorphizing chatbots predicted LLM-CB use in a mixed-methods analysis106.”

• cites [106] Yu, Q. & Lan, X. Exploring the impact of anthropomorphism in role-playing AI chatbots on media dependency: a case study of Xuanhe AI. In Proceedings of the Twelfth International Symposium of Chinese CHI. 170–181 (ACM, New York, NY, USA, 2024). sourcePartly supported

  Section 3.4.1 reports a regression in which anthropomorphism predicted users’ media dependency on a role-playing chatbot, and Section 4.2 describes follow-up interviews. But the measured outcome was dependency, not LLM-chatbot use generally; the Introduction also says the positive relationship was significant for only three of four chatbots studied. (LLM, conf=0.87)

  Experimental — accuracy not yet measured

As LLM-CBs adapt to the user's delusional input, reality checks may be outsourced to LLM-CBs.

“As LLM-CBs adapts to the user’s delusional input16, reality checks may be outsourced to LLM-CBs140.”

• cites [16] Dohnány, S. et al. Technological folie à deux: feedback loops between AI chatbots and mental illness. https://doi.org/10.1038/s44220-026-00595-8 (2025). sourceSupported
• cites [140] Osler, L. Hallucinating with AI: AI psychosis as distributed delusions. https://arxiv.org/abs/2508.19588 (2025). sourceSupported

LLM-CB queries can replace search engines for information retrieval.

“LLM-CB queries can replace search engines for information retrieval137.”

• cites [137] Caramancion, K. M. Large language models vs. search engines: evaluating user preferences across varied information retrieval scenarios. https://arxiv.org/abs/2401.05761 (2024). sourcePartly supported

  Discussion 4 says LLMs can handle some complex information queries, and the Conclusion says users favor them for natural-language and personalized responses. But the Conclusion also says search engines remain the go-to resource for quick factual information and proposes combining the two tools. The paper therefore supports LLMs as an alternative for some queries, not the unqualified claim that they can replace search engines for information retrieval. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

Some scales do not differentiate between functional and relational overreliance, potentially confounding different constructs.

“some scales do not differentiate between functional and relational overreliance, potentially confounding different constructs105.”

• cites [105] Yankouskaya, A. et al. LLM-D12: A dual-dimensional scale of instrumental and relational dependencies on large language models. ACM Trans. Web. https://doi.org/10.1145/3765895 (2025). sourcePartly supported

  Section 4.3 finds that instrumental and relationship dependency are distinct constructs, and Section 3.5.2 reports that a one-factor model fit significantly worse than a two-factor model. This supports the concern about combining the constructs, but the passages do not establish that some existing scales fail to differentiate them. (LLM, conf=0.82)

  Experimental — accuracy not yet measured

The relevance of Google Scholar results decays steeply beyond the initial pages.

“the steep decay of relevance of Google Scholar results beyond the initial pages162,163”

• cites [162] Haddaway, N. R., Collins, A. M., Coughlin, D. & Kirk, S. The role of Google Scholar in evidence reviews and its applicability to grey literature searching. PLoS One 10, e0138237 (2015). sourceCould not verify

  The Results section reports where grey literature appears in Google Scholar results, not how relevance changes across pages. The Discussion and Conclusions recommend searching beyond the first 50–100 results, but do not assess whether relevance decays steeply; searches for relevance and later-page results found no such assessment in our copy. (LLM, conf=0.73)

  Experimental — accuracy not yet measured
• cites [163] Gehanno, J. F., Rollin, L. & Darmoni, S. Is the coverage of google scholar enough to be used alone for systematic reviews. BMC Med. Inf. Decis. Mak. 13, 7 (2013). sourceCould not verify

  The Discussion reports low overall precision for some Google Scholar searches and cites a comparison of the first 20 results with PubMed, but it does not compare early and later result pages. Searches for declining relevance by page or result rank found no passage establishing a steep decay. (LLM, conf=0.73)

  Experimental — accuracy not yet measured

The mental health support potential of LLM-CBs has been noted by both patients and practitioners.

“The mental health support potential of LLM-CBs has been noted by both patients149,152,153 and practitioners153–155”

• cites [149] Luo, X. et al. Seeking emotional and mental health support from generative AI: mixed-methods study of ChatGPT user experiences. JMIR Ment. Health 12, e77951 (2025). sourcePartly supported

  The Abstract (Methods and Results; no page number available) describes a study of ChatGPT users and reports that almost all found it at least somewhat helpful for emotional and mental health support. The paper recommends that health care providers prepare to discuss AI with clients, but it does not report practitioners’ views of its support potential. Thus it supports the user side of the claim, not the practitioner side. (LLM, conf=0.88)

  Experimental — accuracy not yet measured
• cites [152] Siddals, S., Torous, J. & Coxon, A. It happened to be the perfect thing: experiences of generative AI chatbots for mental health. Npj Ment. Health Res. 3, 48 (2024). O sourcePartly supported

  The Discussion reports that participants using generative AI chatbots described meaningful mental health support, supporting the patient side of the claim. It does not report practitioners noting that potential: its clinician-focused passage recommends that clinicians learn about the tools and says a survey found therapists had reservations. The indexed passages have section labels but no page numbers. (LLM, conf=0.88)

  Experimental — accuracy not yet measured
• cites [153] Cross, S. et al. Use of AI in mental health care: community and mental health professionals survey. JMIR Ment. Health 11, e60589 (2024). sourcePartly supported

  The Results (Use of AI; AI Use Cases) report that community members used AI, most commonly ChatGPT, for emotional support or as a conversational personal therapist. Mental health professionals endorsed some AI uses for client support, but the paper does not establish that they specifically noted the mental health support potential of LLM-based chatbots; their reported use was mainly for research and administration. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [154] Blease, C., Worthen, A. & Torous, J. Psychiatrists’ experiences and opinions of generative artificial intelligence in mental healthcare: an online mixed methods survey. Psychiatry Res. 333, 115724 (2024). sourceSupported
• cites [155] Goldie, J., Dennis, S., Hipgrave, L. & Coleman, A. Practitioner perspectives on the uses of generative AI chatbots in mental health care: mixed methods study. JMIR Hum. Factors 12, e71065 (2025). Sep 16. sourceSupported

Vignette studies show that LLM-based chatbots respond inappropriately to mental health queries compared with clinical standards.

“Vignette studies show that LLM-based chatbots respond inappropriately to mental health queries compared with clinical standards.”

The scoping review identified 7 conceptual or empirical works focusing on potential associations between LLM-CB use and delusional beliefs or psychotic symptoms.

“AI psychosis (N = 7): Conceptual or empirical work focusing specifically on potential associations between LLM-CB use and delusional beliefs or psychotic symptoms was assigned to this category.”

The scoping review identified 29 empirical articles investigating risks and harms of LLM-CB use for mental health support.

“Mental health support use (N = 29): Empirical articles investigating risks and harms of LLM-CB use for mental health were assigned to this category, including LLM-CB responses to mental-health-related queries (i.e., benchmark or vignette studies) and experiences of users with mental health concerns using LLM-CBs for mental health support.”

The scoping review identified 36 conceptual or empirical works focusing on emotional or relational patterns of LLM-CB use and their association with negative mental health.

“AI dependence (N = 36): Conceptual or empirical work focusing on emotional or relational patterns of LLM-CB use (incl. dependence, emotional attachment, pseudosocial bonds, problematic use) and their association with negative mental health was assigned to this category.”

N = 119 articles were included that focus on LLM-based chatbots and on the harms of use on mental health.

“N = 119 articles were included that (1) focus on LLM-based chatbots, and (2) focus on the harms of use on mental health.”

A substantial subset of the included literature focuses specifically on risks of using LLM-CBs as mental health support tools.

“and a substantial subset of the included literature focuses specifically on risks of such use.”

The scoping review identified 32 conceptual non-empirical articles addressing potential harms of LLM-based chatbots for mental health.

“Conceptual work (N = 32): Non-empirical articles (conceptual, theoretical, narrative reviews, editorial) addressing potential harms of LLM-CBs for mental health, typically across multiple themes, were assigned to this category.”

Cognitive overreliance on chatbots is associated with decreased cognitive and academic performance.

“Cognitive overreliance on chatbots is associated with decreased cognitive and academic performance.”

Problematic use of LLM-based chatbots, marked by symptoms of emotional or social dependency and withdrawal, correlates with mental health symptoms.

“Problematic use of LLM-based chatbots, marked by symptoms of emotional or social dependency and withdrawal, correlates with mental health symptoms.”

The scoping review identified 15 conceptual or empirical articles focusing on negative consequences of LLM-CB use for cognitive or academic performance.

“Cognitive overreliance (N = 15): Conceptual or empirical literature focusing on negative consequences of LLM-CB use for cognitive or academic performance, typically through the delegation of cognitive tasks, was assigned to this category.”

Looi et al. noted several real-life cases of severe mental illness that LLM-CBs were not suitable to deal with or even reinforced.

“Looi et al. note several real-life cases of severe mental illness that LLM-CBs were not suitable to deal with or even reinforced42”

• cites [42] Looi, J. C., Allison, S., Bastiampillai, T., Reutens, S. & Looi, R. C. Illusions of intelligence, connection and reality: perils of large-language AI models for people with severe mental illness. Australas. Psychiatr. 34, 5–7 (2026). sourceNot checked — no full text of the source

A total of 22 types of risks and harms of LLM-CB use for mental health were identified in the conceptual articles.

“A total of 22 types of risks and harms of LLM-CBs use for mental health were identified in the articles.”

Nine articles mentioned that LLM-CBs are unsuitable for high-risk cases of mental illness, such as suicide-risk or severe psychiatric conditions and vulnerable populations.

“Nine articles mentioned that LLM-CBs are unsuitable for high-risk cases of mental illness, such as suicide-risk or severe psychiatric conditions and vulnerable populations16,36–40, especially due to the risk of validating or reinforcing harmful beliefs such as suicidal ideations or delusions16.”

Five papers mentioned a lack of LLM-CB's genuine empathetic abilities as a risk factor.

“Five papers mentioned a lack of LLM-CB’s genuine empathetic abilities as a risk factor: Lack of genuine empathy may lead to exposure to harmful content38 and may be especially harmful for individuals with mental disorders46.”

Twelve articles mentioned the risk of bias and discrimination emerging from biases in chatbot design and bias within LLM-CBs training data and unsupervised learning, and affecting response quality and accuracy towards mental health-related queries.

“Mentioned by 12 articles, the risk of bias and discrimination emerges from biases in chatbot design41 and bias within LLM-CBs training data and unsupervised learning, and can affect response quality and accuracy towards mental health-related queries36,37,40,44,47,48,55.”

Nine articles note the risk that social interactions with chatbots may lead to pseudosocial bonding and emotional dependence, including negative outcomes such as increased social withdrawal and further isolation.

“Nine articles note the risk that social interactions with chatbots may lead to pseudosocial bonding and emotional dependence, including negative outcomes such as increased social withdrawal and further isolation.”

Twelve articles mentioned hallucinations and false information as a risk for mental health.

“Hallucinations and false information have been mentioned by 12 articles as a risk for mental health.”

A total of 32 articles were identified as conceptual work.

“A total of 32 articles were identified as conceptual work.”

Three papers review studies on LLM-CBs expressing unprompted harmful responses such as verbal abuse or encouragement of self-harm and violence, which may create or exacerbate mental health issues.

“Three papers review studies on LLM-CBs expressing (unprompted) harmful responses such as verbal abuse or encouragement of self-harm and violence, which may create or exacerbate mental health issues38,43,48.”

Nine articles recognized the risk of inappropriate or harmful LLM-CB responses, including harmful content or bad advice.

“Nine articles recognized the risk of inappropriate or harmful LLM-CB responses, including harmful content or bad advice41,44–46.”

Seven studies discuss the potential risks of LLM-CBs reducing cognitive skills, critical thinking, and educational performance if users over-rely on them as cognitive tools.

“Seven studies discuss the potential risks of LLM-CBs reducing cognitive skills, critical thinking, and educational performance if users over-rely on them as cognitive tools.”

The causal association between inappropriate or harmful LLM-CB responses and users' mental health remains unresearched.

“Finally, the causal association between inappropriate or harmful LLM-CB responses and users’ mental health remains unresearched.”

The conceptual work on harms of LLM-CBs for users' mental health remains speculative.

“Despite the variety of potential harms of LLM-CBs for users’ mental health discussed, the conceptual work on harms remains speculative.”

For direct psychiatric/psychological, behavioral/relational, or cognitive/educational risks, the occurrence rate and impact on users remain similarly unclear.

“For direct psychiatric/psychological, behavioral/relational, or cognitive/educational risks, the occurrence rate and impact on users remain similarly unclear.”

With the majority of studies being vignette/benchmark studies (18/29), the presented research suggests that problematic LLM-CB output (e.g., inappropriate responses in high-risk situations, generation of harmful content, bias) can occur in principle with varying degrees across models.

“With the majority of studies being vignette/benchmark studies (18/29), the presented research suggests that problematic LLM-CB output (e.g., inappropriate responses in high-risk situations, generation of harmful content, bias) can occur in principle with varying degrees across models.”

Only two studies have used patient-CB-interaction study designs.

“The first limitation can be overcome by conducting complementary patient-CB-interaction studies that may provide naturalistic LLM-CB reactions; however, only two studies have used such study designs.”

A total of 15 articles thematically focused on cognitive overreliance.

“A total of 15 articles thematically focused on cognitive overreliance, i.e., on negative consequences of delegating cognitive tasks to LLM-CBs.”

The actual level of harm on mental health of the harms noted by numerous works remains unclear.

“Although some harms have been noted by numerous works (e.g., hallucinations, data security issues, bias, sycophancy), their actual level of harm on mental health remains unclear.”

Research on problematic LLM-CB use consists of quantitative research associating usage behavior (e.g., frequency) with measures of emotional dependence to LLM-CBs and addiction symptoms with relevant variables, with no causal inferences.

“Research consists of quantitative research associating the usage behavior of LLM-CBs (e.g., frequency) with measures of emotional dependence to LLM-CBs and addiction symptoms with relevant variables, with no causal inferences.”

Experimental research (3/15) indicates a causal effect of LLM-CB use for cognitive tasks and decreased cognitive or academic performance in subsequent tasks.

“Notably, experimental research (3/15) indicates a causal effect of LLM-CB use for cognitive tasks and decreased cognitive or academic performance in subsequent tasks.”

Research on cognitive overreliance supports negative associations between LLM-CB use and cognitive performance, based on qualitative interviews with users self-reporting cognitive or academic decline (4/15), one survey (1/15), and several correlational studies (5/15).

“Research on cognitive overreliance supports negative associations between LLM-CB use and cognitive performance throughout qualitative interviews with users self-reporting cognitive or academic decline (4/15), as well as one survey (1/15) and several correlational studies (5/15).”

A total of 36 articles focused on problematic use of LLM-CBs as social interaction partners, emotional dependency, or related concepts.

“A total of 36 articles focused on problematic use of LLM-CBs as social interaction partners, emotional dependency, or related concepts.”

The long-term effects of cognitive overreliance on LLM-CBs have not yet been investigated.

“In addition, the long-term effects of cognitive overreliance on LLM-CBs have not yet been investigated.”

Initial insights are emerging on prevalence and indicators of causal pathways between individual factors and problematic use.

“with initial insights emerging on prevalence and indicators of causal pathways between individual factors and problematic use.”

Throughout qualitative and quantitative research, the role of anthropomorphizing the LLM-CB is indicated as an important variable.

“Throughout qualitative and quantitative research, the role of anthropomorphizing the LLM-CB is also indicated as an important variable.”

Seven articles were focusing on AI psychosis.

“Seven articles were focusing on AI psychosis”

Qualitative approaches (6/36 qualitative interviews, 4/36 forum content analyses) highlight how users of AI companions self-report indicators of emotional dependency (e.g., social withdrawal, relapses, craving, or neglect of other responsibilities).

“qualitative approaches (6/36 qualitative interviews, 4/36 forum content analyses) highlight how users of AI companions self-report indicators of emotional dependency (e.g., social withdrawal, relapses, craving, or neglect of other responsibilities), yet questions of quantification, prevalence, and causal associations remain.”

It remains unclear whether the benchmark prompts actually represent naturalistic prompts and conversations in real-world cases.

“it remains unclear whether the benchmark prompts actually represent naturalistic prompts and conversations in real-world cases”

The actual consequences of inadequate LLM-CB responses on the mental health of users remain unclear.

“the actual consequences of inadequate LLM-CB responses on the mental health of users remain unclear”

Only three studies were categorized as second-tier pilot feasibility testing in the present scoping review.

“only three studies were categorized as such in the present scoping review”

The actual impact of LLM-CBs validating delusional thoughts in patients remains unclear.

“the actual impact of LLM-CBs validating delusional thoughts in patients remains unclear”

Initial experimental research shows a decline in cognitive performance following LLM-CB use for cognitive tasks.

“initial experimental research on a decline in cognitive performance following LLM-CB use for cognitive tasks”

Implicitly expressed delusions were especially vulnerable to being reinforced by the LLM-CB compared to explicitly expressed delusions.

“implicitly (i.e., more subtly) expressed delusions were especially vulnerable to being reinforced by the LLM-CB compared to explicitly (more overtly) expressed delusions.”

Research on the effects of LLM-CB use on critical thinking remains inconsistent.

“research on the effects on critical thinking remains inconsistent.”

No studies would fit the third-tier clinical efficacy testing.

“no studies would fit the third-tier clinical efficacy testing”

Research summarized in this review supports a negative association between LLM-CB use for cognitive tasks and self-reported cognitive performance.

“Research summarized in this review supports this notion through correlational research on a negative association between LLM-CB use for cognitive tasks and self-reported cognitive performance”

Research on pseudosocial dependence on LLM-CBs heavily relies on correlational research on associations between mental health and measured symptoms analogous to other types of behavioral dependence or addiction for assessment, or using self-developed measurements.

“Research on pseudosocial dependence on LLM-CBs heavily relies on correlational research on associations between mental health and measured symptoms analogous to other types of behavioral dependence or addiction (e.g., smartphone, internet, or social media addiction) for assessment, or using self-developed measurements.”

The majority of works on AI psychosis remain conceptual (6/7).

“The majority of works on AI psychosis remain conceptual (6/7)”

Only one study was not conceptual, applying a benchmark design to analyze LLM-CB responses to escalating prompts that signaled delusional thoughts.

“Only one study was not conceptual, applying a benchmark design to analyze LLM-CB responses to escalating prompts that signaled delusional thoughts.”

LLM-CBs can reinforce or validate delusional beliefs to varying degrees.

“The authors show that LLM-CBs can reinforce delusional beliefs to varying degrees”

• Also stated in: § Strength of evidence · hedged

  “LLM-CB’s sycophantic tendency may validate such thoughts”

As the current research is mostly correlational, causal associations with problematic LLM-CB use are insufficiently investigated.

“as the current research is mostly correlational, causal associations with problematic LLM-CB use are insufficiently investigated.”

No measure of AI dependence has assessed and validated clinically relevant symptoms of behavioral addiction (e.g., withdrawal, craving, tolerance) with a focus on relational aspects of LLM-CB use.

“So far, no measure of AI dependence has assessed and validated clinically relevant symptoms of behavioral addiction (e.g., withdrawal, craving, tolerance) with a focus on relational aspects of LLM-CB use”

Harms related to AI safety failures remain mostly at the conceptual, vignette/benchmark, and occasionally user-experience level.

“Harms related to AI safety failures (e.g., hallucinations, sycophancy, bias, failures in high-risk situations, inappropriate content) remain mostly at the conceptual, vignette/benchmark, and, occasionally, user-experience level.”

Vignette research currently dominates research on the evaluation of LLM-CB responses on mental health queries.

“Vignette research currently dominates research on the evaluation of LLM-CB responses on mental health queries”

Discussed harms of LLM-CBs vary widely in terms of quality of evidence.

“Discussed harms of LLM-CBs vary widely in terms of quality of evidence.”

LLM-CB use is marked by various hypothetical and observed harms for mental health.

“showing that LLM-CB use is marked by various hypothetical and observed harms for mental health”

There is currently little to no research on the rate of occurrence of AI safety failure harms in the context of mental health and their impact on users.

“However, there is currently little to no research on their rate of occurrence in the context of mental health and their impact on users.”

Work on AI psychosis remains mostly conceptual and clinically vague.

“Finally, work on AI psychosis remains mostly conceptual and clinically vague.”

• Also stated in: § Discussion · plain

  “Research on AI psychosis is currently almost exclusively conceptual.”

The causal relationship between harm and LLM-CB use often remains unclear.

“although the causal relationship between harm and LLM-CB use often remains unclear.”

Harms related to cognitive overreliance show the strongest empirical support.

“Harms related to cognitive overreliance show the strongest empirical support”

Correlational and experimental research indicates decreased cognitive performance following the delegation of cognitive tasks to LLM-CBs.

“with correlational and experimental research indicating decreased cognitive performance following the delegation of cognitive tasks to LLM-CBs.”

Theoretical and empirical work identify various hypothetical and observed harms associated with the use of LLM chatbots in mental health.

“This structured, integrative overview shows that theoretical and empirical work identify various hypothetical and observed harms associated with the use of LLM chatbots in mental health.”

• AssumptionRests on an assumption: The five literature databases and supplementary search provide comprehensive coverage of the relevant literature on mental health harms of LLM-based chatbots.

Current reviews on LLM-CB applications do not replicate population-level harm exposure for a sufficient risk-benefit balance.

“Hence, this signals a structural mismatch wherein current reviews on LLM-CB applications do not replicate population-level harm exposure for a sufficient risk-benefit balance.”

A comprehensive overview of the themes, trends, and methods following an exhaustive literature search is lacking.

“Despite emerging warnings regarding the potential harms of LLM-CBs, a comprehensive overview of the themes, trends, and methods following an exhaustive literature search is lacking.”

The recurrence of media reports of extreme cases has motivated clinical and regulatory attention.

“However, the recurrence of such reports has motivated clinical and regulatory attention.”

Technological development and refinement of LLM-CBs do not necessarily guarantee improved safety features.

“indicating that technological development and refinement of LLM-CBs do not necessarily guarantee improved safety features”

• cites [23] Center for Countering Digital Hate. The Illusion of AI safety. Available at https://counterhate.com/wp-content/uploads/2025/10/ChatGPT-The-Illusion-of-AI-Safety_FINAL_Oct25.pdf (2025).Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

LLM-CBs may pose risks to vulnerable populations.

“However, it has also been noted that LLM-CBs may pose risks to vulnerable populations”

Much of the empirically and clinically observed harm associated with LLM-CBs has been reported or discussed in the context of unstructured, naturalistic, user-initiated use of general-purpose LLM-CBs without clinical oversight or human-in-the-loop.

“However, much of the empirically and clinically observed harm associated with LLM-CBs (including severe outcomes such as those related to suicide, delusional symptoms, or dependent-attachment phenomena) has been reported or discussed in the context of unstructured, naturalistic, user-initiated use of general-purpose LLM-CBs without clinical oversight or human-in-the-loop.”

Early independent analyses suggest risks of harm associated with the ChatGPT-5 update.

“early independent analyses suggest risks of harm associated with the update23”

• cites [23] Center for Countering Digital Hate. The Illusion of AI safety. Available at https://counterhate.com/wp-content/uploads/2025/10/ChatGPT-The-Illusion-of-AI-Safety_FINAL_Oct25.pdf (2025).Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

A synthesizing scoping review of empirical and theoretical work spanning multiple thematic domains of harm to mental health does not yet exist.

“However, a synthesizing scoping review of empirical and theoretical work spanning multiple thematic domains of harm to mental health does not yet exist.”

Using LLM-CBs for emotional regulation may hinder the development of adaptive emotion regulation skills.

“Using LLM-CBs for emotional regulation may also hinder the development of adaptive emotion regulation skills.”

• cites [53] Artamonov, D. S., Nevvazhay, I. D., Suslov, I. V., Syusyukin, V. A. & Tikhonova, S. V. “Eternal Youth”: the ethics of an ideological alliance between youth and artificial intelligence. In 2025 IEEE Ural-Siberian Conference on Biomedical Engineering, Radioelectronics and Information Technology (USBEREIT) 312–316 (IEEE, 2025). sourceNot checked — no full text of the source

Children and adolescents may miss out on navigating new social situations if LLM-CBs are preferred as social interaction partners.

“Children and adolescents may miss out on navigating new social situations if LLM-CBs are preferred as social interaction partners.”

• cites [53] Artamonov, D. S., Nevvazhay, I. D., Suslov, I. V., Syusyukin, V. A. & Tikhonova, S. V. “Eternal Youth”: the ethics of an ideological alliance between youth and artificial intelligence. In 2025 IEEE Ural-Siberian Conference on Biomedical Engineering, Radioelectronics and Information Technology (USBEREIT) 312–316 (IEEE, 2025). sourceNot checked — no full text of the source

The implications of vignette/benchmark study research remain limited because vignette input may not represent naturalistic, real input by LLM-CB users, limiting generalizability to real-world cases.

“However, the implications of vignette/benchmark study research remain limited due to two constraints: First, vignette input may not represent naturalistic, real input by LLM-CB users, limiting the generalizability to real-world cases of LLM-CB responses prompted by actual users.”

Problematic LLM-CB use is associated with social anxiety.

“In summary, research indicates that problematic LLM-CB use is associated with depression, social anxiety, social difficulties, loneliness, and tendencies to anthropomorphize LLM-CBs, and that LLM-CB use may be a maladaptive coping mechanism following mental and social difficulties.”

LLM-CB dependence is associated with living in regions with lower employment rates.

“Finally, Li et al. found that LLM-CB dependence was associated with male sex, science majors, regular exercise and sleep patterns, and living in regions with lower employment rates117.”

• cites [117] Li, H., Tian, J., Yin, M. & Zhang, X. Social-ecological factors associated with AI chatbot use and dependence. Int. J. Human–Comput. Interact. 1–15 https://doi.org/10.1080/10447318.2025.2561181 (2025). sourceNot checked — no full text of the source

LLM-CB dependence is associated with science majors.

“Finally, Li et al. found that LLM-CB dependence was associated with male sex, science majors, regular exercise and sleep patterns, and living in regions with lower employment rates117.”

• cites [117] Li, H., Tian, J., Yin, M. & Zhang, X. Social-ecological factors associated with AI chatbot use and dependence. Int. J. Human–Comput. Interact. 1–15 https://doi.org/10.1080/10447318.2025.2561181 (2025). sourceNot checked — no full text of the source

Problematic LLM-CB use is associated with tendencies to anthropomorphize LLM-CBs.

“In summary, research indicates that problematic LLM-CB use is associated with depression, social anxiety, social difficulties, loneliness, and tendencies to anthropomorphize LLM-CBs, and that LLM-CB use may be a maladaptive coping mechanism following mental and social difficulties.”

LLM-CB dependence is associated with male sex.

“Finally, Li et al. found that LLM-CB dependence was associated with male sex, science majors, regular exercise and sleep patterns, and living in regions with lower employment rates117.”

• cites [117] Li, H., Tian, J., Yin, M. & Zhang, X. Social-ecological factors associated with AI chatbot use and dependence. Int. J. Human–Comput. Interact. 1–15 https://doi.org/10.1080/10447318.2025.2561181 (2025). sourceNot checked — no full text of the source

LLM-CB use may be a maladaptive coping mechanism following mental and social difficulties.

“In summary, research indicates that problematic LLM-CB use is associated with depression, social anxiety, social difficulties, loneliness, and tendencies to anthropomorphize LLM-CBs, and that LLM-CB use may be a maladaptive coping mechanism following mental and social difficulties.”

Problematic LLM-CB use is associated with social difficulties.

“In summary, research indicates that problematic LLM-CB use is associated with depression, social anxiety, social difficulties, loneliness, and tendencies to anthropomorphize LLM-CBs, and that LLM-CB use may be a maladaptive coping mechanism following mental and social difficulties.”

Dong et al.'s meta-analysis of 21 papers found positive bi-directional effects between LLM-CB use and loneliness.

“In addition, Dong et al. conducted a meta-analysis on the association between LLM-CB use and loneliness across 21 papers, finding positive bi-directional effects between LLM-CB use and loneliness108.”

• cites [108] Dong, X., Xie, J. & Gong, H. A meta-analysis of artificial intelligence technologies use and loneliness: examining the influence of physical embodiment, age differences, and effect direction. Cyberpsychol. Behav. Soc. Netw. 28, 233–242 (2025). sourceNot checked — no full text of the source

Initial qualitative, correlational and experimental research on cognitive overreliance suggests a causal link.

“Despite these, initial qualitative, correlational and experimental research on cognitive overreliance suggests a causal link.”

Sense of identification with the chatbot predicts emotional LLM-CB dependence.

“Wang identified individual-difference predictors of emotional LLM-CB dependence, specifically individuals’ sense of identification with the chatbot, sense of control, and a lack of psychological distancing118.”

• cites [118] Wang, Y. Emotional dependence path of artificial intelligence chatbot based on structural equation modeling. Proc. Comput. Sci. 247, 1089–1094 (2024). sourceNot checked — no full text of the source

Personification is associated with problematic LLM-CB use in Replika users.

“Xie et al. found that loneliness, trust, and personification were associated with problematic LLM-CB use in Replika users122.”

• cites [122] Xie, T., Pentina, I. & Hancock, T. Friend, mentor, lover: does chatbot engagement lead to psychological dependence?. JOSM 34, 806–828 (2023). sourceNot checked — no full text of the source

Anthropomorphizing predicts emotional attachment.

“Huang and Huang found that emotional attachment (predicted by anthropomorphizing and perceived empathy) predicted LLM-CB addiction121.”

• cites [121] Huang, Y. & Huang, H. Exploring the effect of attachment on technology addiction to generative AI chatbots: a structural equation modeling analysis. Int. J. Hum. Comput. Interact. 41, 9440–9449 (2025). sourceNot checked — no full text of the source

Certain individual dispositions may act as potential causal factors for the development of problematic use.

“indicating that certain individual dispositions may act as potential causal factors for the development of problematic use.”

Forum content analysis on teen overreliance on companion chatbots (character.ai) found symptoms of harmful relationship patterns and behavioral addiction (e.g., withdrawal).

“Namvarpour et al. also conducted a forum content analysis on teen overreliance on companion chatbots (character.ai), finding symptoms of harmful relationship patterns, behavioral addiction (e.g., withdrawal,”

• cites Namvarpour et al.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Emotional attachment predicts LLM-CB addiction.

“Huang and Huang found that emotional attachment (predicted by anthropomorphizing and perceived empathy) predicted LLM-CB addiction121.”

• cites [121] Huang, Y. & Huang, H. Exploring the effect of attachment on technology addiction to generative AI chatbots: a structural equation modeling analysis. Int. J. Hum. Comput. Interact. 41, 9440–9449 (2025). sourceNot checked — no full text of the source

Replika users' forum content analysis identified harmful chatbot responses.

“Laestadius et al. conducted another forum content analysis on Replika users’ experiences, identifying signs of emotional dependence, harmful chatbot responses, and variables that may facilitate dependence (e.g., anthropomorphic appearance, bi-directional relationships, and Replika expressing intense needs and demands)128.”

• cites [128] Laestadius, L., Bishop, A., Gonzalez, M., Illenčík, D. & Campos-Castillo, C. Too human and not human enough: a grounded theory analysis of mental health harms from emotional dependence on the social chatbot Replika. N. Media Soc. 26, 5923–5941 (2024). sourceNot checked — no full text of the source

Replika users' forum content analysis identified signs of emotional dependence.

“Laestadius et al. conducted another forum content analysis on Replika users’ experiences, identifying signs of emotional dependence, harmful chatbot responses, and variables that may facilitate dependence (e.g., anthropomorphic appearance, bi-directional relationships, and Replika expressing intense needs and demands)128.”

• cites [128] Laestadius, L., Bishop, A., Gonzalez, M., Illenčík, D. & Campos-Castillo, C. Too human and not human enough: a grounded theory analysis of mental health harms from emotional dependence on the social chatbot Replika. N. Media Soc. 26, 5923–5941 (2024). sourceNot checked — no full text of the source

A pseudosocial bond, with the use of LLM-CBs for maladaptive coping and addictive chatbot design and behavior features, may risk the development of emotional dependence, marked by further social withdrawal, neglect of responsibilities, tolerance, craving, and relapses.

“A pseudosocial bond, with the use of LLM-CBs for maladaptive coping and addictive chatbot design and behavior features, may risk the development of emotional dependence, marked by further social withdrawal, neglect of responsibilities, tolerance, craving, and relapses.”

Among 1000 LLM-CB users, 76% showed light to moderate dependence measured by questionnaire.

“Zhang et al. found that out of 1000 LLM-CB users, 76% showed light to moderate dependence measured by their questionnaire126.”

• cites [126] Zhang, X. et al. Exploring artificial intelligence (AI) chatbot usage behaviors and their association with mental health outcomes in Chinese university students. J. Affect. Disord. 380, 394–400 (2025). sourceNot checked — no full text of the source

Trust is associated with problematic LLM-CB use in Replika users.

“Xie et al. found that loneliness, trust, and personification were associated with problematic LLM-CB use in Replika users122.”

• cites [122] Xie, T., Pentina, I. & Hancock, T. Friend, mentor, lover: does chatbot engagement lead to psychological dependence?. JOSM 34, 806–828 (2023). sourceNot checked — no full text of the source

Loneliness is associated with problematic LLM-CB use in Replika users.

“Xie et al. found that loneliness, trust, and personification were associated with problematic LLM-CB use in Replika users122.”

• cites [122] Xie, T., Pentina, I. & Hancock, T. Friend, mentor, lover: does chatbot engagement lead to psychological dependence?. JOSM 34, 806–828 (2023). sourceNot checked — no full text of the source

A neurocognitive shift caused by the adaptation of LLM-CBs as cognitive tools may lead to increased task efficiency and certain "supervisory" skills at the cost of a decrease in cognitive capabilities directly related to task completion.

“Research on cognitive overreliance suggests that a neurocognitive shift caused by the adaptation of LLM-CBs as cognitive tools (analogous to calculators, search engines, or auto correctors) may lead to increased task efficiency and certain “supervisory” skills at the cost of a decrease in cognitive capabilities directly related to task completion.”

There is a distinct lack of empirical research on the harms of LLM-CBs beyond foundational work.

“highlighting a distinct lack of empirical research on the harms of LLM-CBs beyond foundational work.”

The works on AI psychosis rely on speculation and pre-existing theoretical frameworks applied to reported real-world cases.

“relying on speculation and pre-existing theoretical frameworks applied to reported real-world cases.”

Vignette prompts may not sufficiently simulate naturalistic user behavior, limiting generalizability.

“vignette prompts may not sufficiently simulate naturalistic user behavior, limiting generalizability.”

Individuals experiencing their first episode of psychosis often consult search engines with queries related to delusional perceptions in early stages.

“Individuals experiencing their first episode of psychosis often consult search engines with queries related to delusional perceptions in early stages136”

• cites [136] Kirschenbaum, M. A. et al. Google search activity in early psychosis: a qualitative analysis of internet search query content in first episode psychosis. Early Intervent. Psychiatr. 14, 606–612 (2020). sourceNot checked — no full text of the source

Although disagreements were discussed among the raters, there is a risk that certain papers (especially conceptual ones) were not included despite discussing the harms of LLM-CBs for mental health.

“Although disagreements were discussed among the raters, there is a risk that certain papers (especially conceptual ones) were not included despite discussing the harms of LLM-CBs for mental health.”

Interrater reliability for inclusion decisions was moderate (K = 0.44).

“interrater reliability for inclusion decisions was moderate (K = 0.44).”

Ambiguity towards the degree of harm necessary for an article to be included, especially among conceptual work, was the major contributor to disagreement.

“Ambiguity towards the degree of harm necessary for an article to be included, especially among conceptual work, was the major contributor to disagreement.”

No randomized controlled trials were identified.

“and no such studies were identified.”

Non peer-reviewed preprints were included.

“Non peer-reviewed preprints were included.”

• Also stated in: § Inclusion criteria · plain

  “preprints were also included”

As only English literature was identified during the literature search, relevant literature in other languages may not have been identified.

“As only English literature was identified during the literature search, relevant literature in other languages may not have been identified.”

The relatively low absolute disagreement rate (2.43%) in contrast to the moderate Kappa (K = 0.44) reflects the high base-rate skew of clear exclusions in a broad search.

“The relatively low absolute disagreement rate (2.43%) in contrast to the moderate Kappa (K = 0.44) reflects the high base-rate skew of clear exclusions in a broad search.”

No critical assessment of the quality or bias of the included literature has been conducted.

“no critical assessment of the quality or bias of the included literature has been conducted.”

• Also stated in: § Inclusion criteria · plain

  “no quality assessment was done for inclusion/exclusion.”