CoChat
Sign Up
Cited by 46Digital Mental Health InterventionsArtificial Intelligence in Healthcare and EducationSocial Robot Interaction and HRI

Expert and Interdisciplinary Analysis of AI-Driven Chatbots for Mental Health Support: Mixed Methods Study

Open access
Moylan, Kayley, Doherty, Kevin·2025·Journal of Medical Internet Research
DOI 10.2196/67114PMC PMC12064976

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Recent years have seen an immense surge in the creation and use of chatbots as social and mental health companions. Aiming to provide empathic responses in support of the delivery of personalized support, these tools are often presented as offering immense potential. However, it is also essential that we understand the risks of their deployment, including their potential adverse impacts on the mental health of users, including those most at risk. Objective: The study aims to assess the ethical and pragmatic clinical implications of using chatbots that claim to aid mental health. While several studies within human-computer interaction and related fields have examined users’ perceptions of such systems, few studies have engaged mental health professionals in critical analysis of their conduct as mental health support tools. This paper comprises, in turn, an effort to assess the ethical and pragmatic clinical implications of using chatbots that claim to aid mental health. Methods: This study included 8 interdisciplinary mental health professional participants (from psychology and psychotherapy to social care and crisis volunteer workers) in a mixed methods and hands-on analysis of 2 popular mental health–related chatbots’ data handling, interface design, and responses. This analysis was carried out through profession-specific tasks with each chatbot, eliciting participants’ perceptions through both the Trust in Automation scale and semistructured interviews. Through thematic analysis and a 2-tailed, paired t test, these chatbots’ implications for mental health support were thus evaluated. Results: Qualitative analysis revealed emphatic initial impressions among mental health professionals of chatbot responses likely to produce harm, exhibiting a generic mode of care, and risking user dependence and manipulation given the central role of trust in the therapeutic relationship. Trust scores from the Trust in Automation scale, while exhibiting no statistically significant differences between the chatbots (t6=–0.76; P=.48), indicated medium to low trust scores for each chatbot. The findings of this work highlight that the design and development of artificial intelligence (AI)–driven mental health–related solutions must be undertaken with utmost caution. The mental health professionals in this study collectively resist these chatbots and make clear that AI-driven chatbots used for mental health by at-risk users invite several potential and specific harms. Conclusions: Through this work, we contributed insights into the mental health professional perspective on the design of chatbots used for mental health and underscore the necessity of ongoing critical assessment and iterative refinement to maximize the benefits and minimize the risks associated with integrating AI into mental health support.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/67114

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • LLM-CBs generally do not sufficiently adhere to professional guidelines or standards.

    “LLM-CBs generally do not sufficiently adhere to professional guidelines or standards65,71–73 and, according to health professionals’ analyses on chatbot responses, show special harmful potential in the context of crisis situations71.”

    From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [71]

    The Discussion (page unavailable) reports that professionals found Wysa and Replika to differ substantially from professional practice and identifies concerns about ethical principles. This supports the concern behind the claim, but the study evaluates those two chatbots rather than establishing that LLM-based chatbots generally fail to adhere to professional guidelines or standards. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • According to health professionals' analyses on chatbot responses, LLM-CBs show special harmful potential in the context of crisis situations.

    “LLM-CBs generally do not sufficiently adhere to professional guidelines or standards65,71–73 and, according to health professionals’ analyses on chatbot responses, show special harmful potential in the context of crisis situations71.”

    From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [71]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 107 · Cited by 46

Research works printed but not linked: 61

  1. [5]McGorry PD Mei C Chanen A Hodges C Alvarez-Jimenez M Killackey E Designing and scaling up integrated youth mental health care World Psychiatry 2022 02 21 1 61 76 10.1002/wps.20938 35015367 35015367 PMC8751571 · DOI · Paper page
  2. [6]O'Connor K Wrigley M Jennings R Hill M Niazi A Mental health impacts of COVID-19 in Ireland and the need for a secondary care mental health service response Ir J Psychol Med 2021 06 38 2 99 107 10.1017/ipm.2020.64 32456713 S0790966720000646 32456713 PMC7471570 · DOI
  3. [7]Ziebland S Hyde E Powell J Power, paradox and pessimism: on the unintended consequences of digital health technologies in primary care Soc Sci Med 2021 11 289 114419 10.1016/j.socscimed.2021.114419 34619631 S0277-9536(21)00751-6 34619631 · DOI
  4. [8]Sanderson C Kouzoupi N Hall CL Technology matters: the human touch in a digital age - a blended approach in mental healthcare delivery with children and young people Child Adolesc Ment Health 2020 05 25 2 120 2 10.1111/camh.12385 32307846 32307846 · DOI
  5. [9]Banks J The human touch: practical and ethical implications of putting AI and robotics to work for patients IEEE Pulse 2018 5 11 9 3 15 8 10.1109/mpul.2018.2814238 29757747 · DOI
  6. [10]Tewari A Chhabria A Singh Khalsa A Chaudhary S Kanal H A survey of mental health chatbots using NLP Proceedings of the International Conference on Innovative Computing & Communication 2021 2021 ICICC 2021 February 20-21, 2021 Delhi, India 10.2139/ssrn.3833914 · DOI
  7. [12]Cheshmehzangi A Zou T Su Z The digital divide impacts on mental health during the COVID-19 pandemic Brain Behav Immun 2022 03 101 211 3 10.1016/j.bbi.2022.01.009 35041937 S0889-1591(22)00012-5 35041937 PMC8760094 · DOI
  8. [13]Maglieri V Zanoli A Giunchi D Palagi E Social isolation affects the mimicry response in the use of smartphones: an ethological experiment during the COVID-19 pandemic Hum Nat 2023 03 34 1 88 102 10.1007/s12110-023-09443-5 36806091 10.1007/s12110-023-09443-5 36806091 PMC9942080 · DOI
  9. [17]Denecke K Vaaheesan S Arulnathan A A mental health chatbot for regulating emotions (SERMO) - concept and usability test IEEE Trans Emerg Topics Comput 2021 9 3 1170 82 10.1109/tetc.2020.2974478 · DOI
  10. [18]Coghlan S Leins K Sheldrick S Cheong M Gooding P D'Alfonso S To chat or bot to chat: ethical issues with using chatbots in mental health Digit Health 2023 9 20552076231183542 10.1177/20552076231183542 37377565 10.1177_20552076231183542 37377565 PMC10291862 · DOI
  11. [21]Hamdoun S Monteleone R Bookman T Michael K Michael K AI-based and digital mental health apps: balancing need and risk IEEE Technol Soc Mag 2023 3 42 1 25 36 10.1109/mts.2023.3241309 · DOI
  12. [26]Cohen JA Mannarino AP Murray LK Igelman R Psychosocial interventions for maltreated and violence‐exposed children J Soc Issues 2006 10 30 62 4 737 66 10.1111/j.1540-4560.2006.00485.x · DOI
  13. [29]Alduhishy M The overprescription of antidepressants and its impact on the elderly in Australia Trends Psychiatry Psychother 2018 08 23 40 3 241 3 10.1590/2237-6089-2016-0077 30156645 S2237-60892018005004101 30156645 · DOI
  14. [30]Annema JH Verstraete M Abeele VV Desmet S Geerts D Videogames in therapy: a therapist's perspective Proceedings of the 3rd International Conference on Fun and Games 2010 Fun and Games '10 September 15-17, 2010 Leuven, Belgium 10.1145/1823818.1823828 · DOI
  15. [31]Tochilnikova E Patnaik A Alsebayel G “Guilty of talking too much”: how psychotherapists gamify therapy Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems 2022 CHI '22 April 29-May 5, 2022 New Orleans, LA 10.1145/3491102.3517437 · DOI
  16. [32]Flobak E Wake JD Vindenes J Kahlon S Nordgreen T Guribye F Participatory design of VR scenarios for exposure therapy Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems 2019 CHI '19 May 4-9, 2019 Glasgow, UK 10.1145/3290605.3300799 · DOI
  17. [33]Schuemie M Design of virtual reality exposure therapy systems: task analysis Proceedings of the CHI '00 Extended Abstracts on Human Factors in Computing Systems 2000 CHI EA '00 April 1-6, 2000 The Hague, The Netherlands 10.1145/633292.633499 · DOI
  18. [34]Hamid A Arshad R Shahid S What are you thinking?: using CBT and storytelling to improve mental health among college students Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems 2022 CHI '22 April 29-May 5, 2022 New Orleans, LA 10.1145/3491102.3517603 · DOI
  19. [35]Pendse SR Lalani FM De Choudhury M Sharma A Kumar N "Like shock absorbers": understanding the human infrastructures of technology-mediated mental health support Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems 2020 CHI '20 April 25-30, 2020 Honolulu, HI 10.1145/3313831.3376465 PMC9128312 35615053 · DOI · Paper page
  20. [36]Roquet CD Sas C Evaluating mindfulness meditation apps Proceedings of the Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems 2018 CHI EA '18 April 21-26, 2018 Montreal, QC 10.1145/3170427.3188616 · DOI
  21. [37]Legaspi CM Jr Pacana TR Loja K Sing C Ong E User perception of Wysa as a mental well-being support tool during the COVID-19 pandemic Proceedings of the Asian HCI Symposium 2022 2022 Asian HCI '22 April 29-May 5, 2022 New Orleans, LA 10.1145/3516492.3559064 · DOI
  22. [39]Cooney O Doherty K Barry M Coyle D Doherty G Cowan BR Using speech agents for mood logging within blended mental healthcare: mental healthcare practitioners' perspectives Proceedings of the 6th ACM Conference on Conversational User Interfaces 2024 CUI '24 July 8-10, 2024 Luxembourg, Luxembourg 10.1145/3640794.3665540 · DOI
  23. [41]Skjuve M Følstad A Brandtzæg PB A longitudinal study of self-disclosure in human–chatbot relationships Interact Comput 2023 35 1 24 39 10.1093/iwc/iwad022 · DOI
  24. [43]Jarvie C Renaud K Online age verification: government legislation, supplier responsibilization, and public perceptions Children (Basel) 2024 08 30 11 9 1068 10.3390/children11091068 39334601 children11091068 39334601 PMC11429505 · DOI
  25. [45]Cowls J Morley J Floridi L App store governance: implications, limitations, and regulatory responses Telecommun Policy 2023 02 47 1 102460 10.1016/j.telpol.2022.102460 · DOI
  26. [46]Balaskas A Schueller SM Doherty K Cox AL Doherty G Designing personalized mental health interventions for anxiety: CBT therapists’ perspective Int J Hum Comput Stud 2024 10 190 103319 10.1016/j.ijhcs.2024.103319 · DOI
  27. [47]Maharjan R Doherty K Rohani DA Bækgaard P Bardram JE Experiences of a speech-enabled conversational agent for the self-report of well-being among people living with affective disorders: an in-the-wild study ACM Trans Interact Intell Syst 2022 07 20 12 2 1 29 10.1145/3484508 · DOI
  28. [48]Doherty K Doherty G Engagement in HCI: conception, theory and measurement ACM Comput Surv 2018 11 19 51 5 1 39 10.1145/3234149 · DOI
  29. [50]Sax M Optimization of what? For-profit health apps as manipulative digital environments Ethics Inf Technol 2021 01 03 23 3 345 61 10.1007/s10676-020-09576-6 · DOI
  30. [51]Spors V Wagner HG Flintham M Brundell P Murphy D Selling glossy, easy futures: a feminist exploration of commercial mental-health-focused self-care apps’ descriptions in the Google Play store Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems 2021 CHI '21 May 8-13, 2021 Yokohama, Japan 10.1145/3411764.3445500 · DOI
  31. [52]Burr C Morley J Burr C Milano S Empowerment or engagement? Digital health technologies for mental healthcare The 2019 Yearbook of the Digital Ethics Lab 2020 Cham, Switzerland Springer
  32. [53]Brown JE Halpern J AI chatbots cannot replace human interactions in the pursuit of more inclusive mental healthcare SSM Mental Health 2021 12 1 100017 10.1016/j.ssmmh.2021.100017 · DOI · Paper page
  33. [54]McCutchen C Hyland P Shevlin M Cloitre M The occurrence and co-occurrence of ACEs and their relationship to mental health in the United States and Ireland Child Abuse Negl 2022 07 129 105681 10.1016/j.chiabu.2022.105681 35643057 S0145-2134(22)00201-0 35643057 · DOI
  34. [55]Iwaya LH Babar MA Rashid A Wijayarathna C On the privacy of mental health apps: an empirical investigation and its implications for app development Empir Softw Eng 2023 11 08 28 1 2 10.1007/s10664-022-10236-0 36407814 10236 36407814 PMC9643945 · DOI · Paper page
  35. [56]Warrier U Warrier A Khandelwal K Ethical considerations in the use of artificial intelligence in mental health Egypt J Neurol Psychiatry Neurosurg 2023 10 27 59 139 10.1186/s41983-023-00735-2 · DOI
  36. [57]Brouwers EP Social stigma is an underestimated contributing factor to unemployment in people with mental illness or mental health issues: position paper and future directions BMC Psychol 2020 04 21 8 1 36 10.1186/s40359-020-00399-0 32317023 10.1186/s40359-020-00399-0 32317023 PMC7171845 · DOI
  37. [59]Young G Kenny MC Focusing the APA ethics code to include development: applications to abuse J Child Adolesc Trauma 2023 03 13 16 1 109 22 10.1007/s40653-022-00484-z 36776633 484 36776633 PMC9908791 · DOI
  38. [60]Barkerd P Mental Health Ethics: The Human Context 2010 Milton Park, UK Routledge
  39. [62]Guest G Bunce A Johnson L How many interviews are enough?: an experiment with data saturation and variability Field Methods 2006 02 01 18 1 59 82 10.1177/1525822x05279903 · DOI · Paper page
  40. [63]Shan Y Philosophical foundations of mixed methods research Philos Compass 2021 12 16 17 e12804 10.1111/phc3.12804 · DOI
  41. [64]Doyle L Brady AM Byrne G An overview of mixed methods research J Res Nurs 2009 03 01 14 2 175 85 10.1177/1744987108093962 · DOI
  42. [65]Field A Miles J Field Z Discovering Statistics Using R 2012 Thousand Oaks, CA SAGE Publications
  43. [66]Matthews M Gay G Doherty G Taking part: role-play in the design of therapeutic systems Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 2014 CHI '14 April 26-May 1, 2014 Toronto, ON 10.1145/2556288.2557103 · DOI
  44. [67]Kohn SC de Visser EJ Wiese E Lee YC Shaw TH Measurement of trust in automation: a narrative review and reference guide Front Psychol 2021 10 19 12 604977 10.3389/fpsyg.2021.604977 34737716 34737716 PMC8562383 · DOI
  45. [68]Parmar S Thomas RP Effects of probabilistic risk situation awareness tool (RSAT) on aeronautical weather-hazard decision making Front Psychol 2020 12 16 11 566780 10.3389/fpsyg.2020.566780 33391082 33391082 PMC7772149 · DOI
  46. [69]Oliffe JL Kelly MT Gonzalez Montaner G Yu Ko WF Zoom interviews: benefits and concessions Int J Qual Methods 2021 10 20 20 10.1177/16094069211053522 · DOI
  47. [70]Green J Thorogood N Qualitative Methods for Health Research 2009 Thousand Oaks, CA SAGE Publications
  48. [73]Osborne JW Is data cleaning and the testing of assumptions relevant in the 21st century? Front Psychol 2013 4 370 10.3389/fpsyg.2013.00370 23805118 23805118 PMC3691503 · DOI
  49. [74]Eagle T Blau C Bales S Desai N Li V Whittaker S “I don’t know what you mean by `I am anxious'”: a new method for evaluating conversational agent responses to standardized mental health inputs for anxiety and depression ACM Trans Interact Intell Syst 2022 07 20 12 2 1 23 10.1145/3488057 · DOI
  50. [75]De Nieva JO Joaquin JA Tan CB Te RK Ong E Investigating students’ use of a mental health chatbot to alleviate academic stress Proceedings of the 6th International ACM In-Cooperation HCI and UX Conference 2020 CHIuXiD '20 October 21-23, 2020 Jakarta & Bandung, Indonesia 10.1145/3431656.3431657 · DOI
  51. [76]Lin BB Thompson S Mitchell R Astell-Burt T De Leeuw E Jalaludin B Feng X Policymaker and practitioner perceptions of parks for health and wellbeing: scoping a holistic approach Sustainability 2023 03 16 15 6 5251 10.3390/su15065251 · DOI
  52. [77]Murphy E Responding to the needs of students with mental health difficulties in higher education: an Irish perspective Eur J Spec Needs Educ 2016 12 23 32 1 110 24 10.1080/08856257.2016.1254966 · DOI
  53. [78]Thieme A Wallace J Meyer TD Olivier P Designing for mental wellbeing: towards a more holistic approach in the treatment and prevention of mental illness Proceedings of the 2015 British HCI Conference 2015 British HCI '15 July 13-17, 2015 Lincoln, UK 10.1145/2783446.2783586 · DOI
  54. [79]Maylea C Victoria, Australia, is getting a new Mental Health and Wellbeing Bill Bioethical Inq 2022 10 03 19 4 527 32 10.1007/s11673-022-10212-9 PMC9908623 36190671 · DOI
  55. [80]Bipeta R Legal and ethical aspects of mental health care Indian J Psychol Med 2019 03 01 41 2 108 12 10.4103/ijpsym.ijpsym_59_19 30983656 PMC6436399 · DOI
  56. [82]Schomerus G Schindler S Rechenberg T Gfesser T Grabe HJ Liebergesell M Sander C Ulke C Speerforck S Stigma as a barrier to addressing childhood trauma in conversation with trauma survivors: a study in the general population PLoS One 2021 10 18 16 10 e0258782 10.1371/journal.pone.0258782 34662349 PONE-D-21-09346 34662349 PMC8523057 · DOI
  57. [83]Foley GN Gentile JP Nonverbal communication in psychotherapy Psychiatry (Edgmont) 2010 06 7 6 38 44 20622944 PMC2898840 20622944
  58. [84]Doherty K Kallina E Moylan K Silva MP Karimian S Shumsher S Brennan R Imagining better AI-enabled healthcare futures: the case for care by design arXiv Preprint posted online on November 4, 2024 10.48550/arXiv.2411.02067 · DOI
  59. [87]Gutzwiller RS Chiou EK Craig SD Lewis CM Lematta GJ Hsiung CP Positive bias in the ‘Trust in Automated Systems Survey’? An examination of the Jian et al. (2000) scale Proc Hum Factors Ergon Soc Annu Meet 2019 11 20 63 1 217 21 10.1177/1071181319631201 · DOI · Paper page
  60. [88]Stokoe E The (in)authenticity of simulated talk: comparing role-played and actual interaction and the implications for communication training Res Lang Soc Interact 2013 04 14 46 2 165 85 10.1080/08351813.2013.780341 · DOI
  61. [89]Blease C Rodman A Generative artificial intelligence in mental healthcare: an ethical evaluation Curr Treat Options Psych 2024 12 09 12 5 10.1007/s40501-024-00340-x · DOI

Authors
Moylan, K., & Doherty, K.
Affiliations
University College Dublin (IE)
Corresponding author
Moylan, Kayley
ORCID
Venue
Journal of Medical Internet Research
Publisher
JMIR Publications Inc.
Published
2025-04-25
Volume
27
Pages
e67114
ISSN
1438-8871 (ISSN-L)
DOI
10.2196/67114
PMC
PMC12064976
OpenAlex
W4409783700
Cited by
46
References
107
Citations by year
2026: 42 · 2025: 4
FWCI
36.81
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
Keywords
AI chatbots, mental health support, mental health professionals' perspectives, trust in automation, chatbot design, data handling, interface design
MeSH terms
Generative Artificial Intelligence, Adult, Artificial Intelligence, Female, Humans, Male, Mental Health, Mental Health Services

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review