Generative artificial intelligence (GenAI) and decision-making: Legal & ethical hurdles for implementation in mental health
Open accessAbstract
This article argues that significant risks are being taken with using GenAI in mental health that should be assessed urgently. It recommends that guidelines for using generative artificial intelligence (GenAI) in mental health care must be established promptly. Currently, clinicians using chatbots without appropriate approval risk undermining legal protections for patients. This could harm the patient and undermine the standards of the profession, undermining trust in an area where human involvement in decision-making is critical. To explore these concerns, this paper is divided into three parts. First, it examines the needs of patients in mental health. Second, it explores the potential benefits of GenAI in mental health and highlights the risks of its use as it pertains to patient needs. Third, it notes the ethical and legal concerns around data use and medical liability that require careful attention. The impact of the European Union's (EU) Artificial Intelligence Act (AI-Act) is also considered. It will be seen that these laws are insufficient in the context of mental health. As such, the paper recommends that guidelines should be developed to help resolve the existing legal gaps until codified rules are established.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Hallucinations can be exacerbated by LLM-CBs' tendencies to attempt to cover up errors or when users trust LLM-CBs due to their anthropomorphic design or established emotional connections.
“Furthermore, hallucinations can be exacerbated by LLM-CBs’ tendencies to attempt to cover up errors37 or when users trust LLM-CBs due to their anthropomorphic design or established emotional connections48,54.”
From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [48]
As general-purpose LLMs are not medical applications, entered data is not protected with the standards typical for medical data, and data protection often fails HIPAA standards.
“As general-purpose LLMs are not medical applications, entered data is not protected with the standards typical for medical data, and data protection often fails HIPAA standards41,48.”
From A scoping review on the mental health harms of LLM-based chatbots (2026) · cites this paper as [48]
Cites
Artificial hallucinations in ChatGPT: implications in scientific writing
Alkaissi, H., & McFarlane, S. I. (2023) · Cureus · doi:10.7759/cureus.35179 · Cited by 1,540 · see citing works
Dave, T., Athaluri, S. A., & Singh, S. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1169595 · Cited by 1,204 · see citing works
Ethical and legal challenges of artificial intelligence-driven healthcare
Gerke, S., Minssen, T., & Cohen, G. (2020) · Artificial Intelligence in Healthcare · doi:10.1016/b978-0-12-818438-7.00012-5 · Cited by 1,170 · see citing works
Radez, J., Reardon, T., Creswell, C., Lawrence, P. J., Evdoka-Burton, G., & Waite, P. (2021) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-019-01469-4 · Cited by 1,119 · see citing works
Sharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00593-2 · Cited by 500 · see citing works
Accuracy of a Generative Artificial Intelligence Model in a Complex Diagnostic Challenge
Kanjee, Z., Crowe, B., & Rodman, A. (2023) · JAMA · doi:10.1001/jama.2023.8288 · Cited by 449 · see citing works
The legal and ethical concerns that arise from using complex predictive analytics in health care
Cohen, I. G., Amarasingham, R., Shah, A., Xie, B., & Lo, B. (2014) · Health Affairs · doi:10.1377/hlthaff.2014.0048 · Cited by 300 · see citing works
Patient involvement in mental health care: one size does not fit all
Tambuyzer, E., Pieters, G., & Audenhove, C. V. (2011) · Health Expectations · doi:10.1111/j.1369-7625.2011.00743.x · Cited by 240 · see citing works
Will Generative Artificial Intelligence Deliver on Its Promise in Health Care?
Wachter, R. M., & Brynjolfsson, E. (2023) · JAMA · doi:10.1001/jama.2023.25054 · Cited by 190 · see citing works
The Challenges for Regulating Medical Use of ChatGPT and Other Large Language Models
Minssen, T., Vayena, E., & Cohen, I. G. (2023) · JAMA · doi:10.1001/jama.2023.9651 · Cited by 171 · see citing works
Assessing the Effectiveness of ChatGPT in Delivering Mental Health Support: A Qualitative Study
Alanezi, F. (2024) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s447368 · Cited by 139 · see citing works
ChatGPT is not ready yet for use in providing mental health assessment and interventions
Dergaa, I., Fekih-Romdhane, F., Hallit, S., Loch, A. A., Glenn, J. M., Fessi, M. S., Ben Aissa, M., Souissi, N., Guelmami, N., Swed, S., et al. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1277756 · Cited by 125 · see citing works
ChatGPT and mental healthcare: balancing benefits with risks of harms
Blease, C., & Torous, J. (2023) · BMJ Mental Health · doi:10.1136/bmjment-2023-300884 · Cited by 122 · see citing works
Continuity of care as experienced by mental health service users - a qualitative study
Biringer, E., Hartveit, M., Sundfør, B., Ruud, T., & Borg, M. (2017) · BMC Health Services Research · doi:10.1186/s12913-017-2719-9 · Cited by 118 · see citing works
Blease, C., Worthen, A., & Torous, J. (2024) · Psychiatry Research · doi:10.1016/j.psychres.2024.115724 · Cited by 113 · see citing works
What Should ChatGPT Mean for Bioethics?
Cohen, I. G. (2023) · The American Journal of Bioethics · doi:10.1080/15265161.2023.2233357 · Cited by 95 · see citing works
Beyond Discrimination: Generative AI Applications and Ethical Challenges in Forensic Psychiatry
Tortora, L. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1346059 · Cited by 72 · see citing works
Froomkin, A. M., Kerr, I., & Pineau, J. (2025) · Edward Elgar Publishing eBooks · doi:10.4337/9781800887305.00012 · Cited by 62 · see citing works
Factor structure in the Camberwell Assessment of Need
Wennström, E., Sörbom, D., & Wiesel, F. (2004) · The British Journal of Psychiatry · doi:10.1192/bjp.185.6.505 · Cited by 60 · see citing works
Elyoseph, Z., & Levkovich, I. (2024) · JMIR Mental Health · doi:10.2196/53043 · Cited by 58 · see citing works
Behavioral health and generative AI: a perspective on future of therapies and patient care
Sezgın, E., & McKay, I. T. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00067-w · Cited by 55 · see citing works
Mental Health and Substance Abuse
Saraceno, B., Funk, M., & Poznyak, V. B. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-803678-5.00283-6 · Cited by 55 · see citing works
Patient-Centered Values and Experiences with Emergency Department and Mental Health Crisis Care
Thomas, K. C., Owino, H., Ansari, S., Adams, L. B., Cyr, J. M., Gaynes, B. N., & Glickman, S. W. (2018) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-018-0849-y · Cited by 48 · see citing works
Will artificial intelligence eventually replace psychiatrists?
Brown, C., Story, G. W., Mourão-Miranda, J., & Baker, J. T. (2019) · The British Journal of Psychiatry · doi:10.1192/bjp.2019.245 · Cited by 43 · see citing works
An assessment of the patients’ needs in Mental Health Education
Chien, W. T., Kam, C., & Lee, I. (2001) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2001.01759.x · Cited by 41 · see citing works
Artificial Intelligence in Healthcare
(2021) · Advanced technologies and societal change · doi:10.1007/978-981-16-6265-2 · Cited by 36 · see citing works
Manderius, C., Clintståhl, K., Sjöström, K., & Örmon, K. (2023) · BMC Nursing · doi:10.1186/s12912-023-01186-z · Cited by 35 · see citing works
Harmful biases in artificial intelligence
King, M. (2022) · The Lancet Psychiatry · doi:10.1016/s2215-0366(22)00312-1 · Cited by 33 · see citing works
Solaiman, B., & Malik, A. (2024) · Medical Law Review · doi:10.1093/medlaw/fwae033 · Cited by 28 · see citing works
Self-citationMental Health Applications of Generative AI and Large Language Modeling in the United States
Banerjee, S. K., Dunn, P. J., Conard, S. E., & Ali, A. (2024) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph21070910 · Cited by 27 · see citing works
Research Handbook on Health, AI and the Law
Biasin, E., Kamenjašević, E., & Ludvigsen, K. R. (2024) · Edward Elgar Publishing eBooks · doi:10.4337/9781802205657 · Cited by 26 · see citing works
Liability for use of artificial intelligence in medicine
Price, W. N., Gerke, S., & Cohen, I. G. (2024) · Edward Elgar Publishing eBooks · doi:10.4337/9781802205657.00016 · Cited by 26 · see citing works
Solaiman, B., Malik, A., & Ghuloum, S. (2023) · American Journal of Law & Medicine · doi:10.1017/amj.2023.30 · Cited by 26 · see citing works
Self-citationChatGPT: Opportunities, risks and priorities for psychiatry
Wei, Y., Guo, L., Lian, C., & Chen, J. (2023) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2023.103808 · Cited by 23 · see citing works
Meaningful Relationships in Community and Clinical Samples: Their Importance for Mental Health
Block, V. J., Haller, E., Villanueva, J., Meyer, A. H., Benoy, C., Walter, M., Lang, U. E., & Gloster, A. T. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.832520 · Cited by 19 · see citing works
Bellier-Teichmann, T., Golay, P., Bonsack, C., & Pomini, V. (2016) · Frontiers in Public Health · doi:10.3389/fpubh.2016.00022 · Cited by 12 · see citing works
Jørgensen, K., Hansen, M. S., Andersen, T. G., Hansen, M., & Karlsson, B. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20031965 · Cited by 11 · see citing works
van Dusseldorp, L., Groot, M., van Vught, A., Goossens, P. J. J., Hulshof, H., & Peters, J. W. B. (2023) · Journal of the American Association of Nurse Practitioners · doi:10.1097/jxx.0000000000000867 · Cited by 11 · see citing works
Professionals’ Perspective on Needs of Persons Who Frequently Use Psychiatric Emergency Services
Schmidt, M., Garmy, P., Stjernswärd, S., & Janlöv, A. (2020) · Issues in Mental Health Nursing · doi:10.1080/01612840.2019.1663565 · Cited by 10 · see citing works
Artificial intelligence and the law of informed consent
Cohen, I. G., & Slottje, A. (2024) · Edward Elgar Publishing eBooks · doi:10.4337/9781802205657.ch10 · Cited by 9 · see citing works
Solaiman, B. (2024) · doi:10.31235/osf.io/s459p · Cited by 7 · see citing works
Self-citationSolaiman, B. (2025) · Law Innovation and Technology · doi:10.1080/17579961.2025.2469986 · Cited by 7 · see citing works
Self-citationGoverning AI in the European Union: emerging infrastructures and regulatory ecosystems in health
Minssen, T., Solaiman, B., Köttering, L., Wested, J. B., & Malik, A. (2024) · Edward Elgar Publishing eBooks · doi:10.4337/9781802205657.00027 · Cited by 7 · see citing works
Self-citationAI in hospital administration and management: ethical and legal implications
Malik, A., & Solaiman, B. (2024) · Edward Elgar Publishing eBooks · doi:10.4337/9781802205657.00008 · Cited by 6 · see citing works
Self-citationPrivacy and Medical Confidentiality in Healthcare
Vansweevelt, T., & Glover‐Thomas, N. (2023) · Edward Elgar Publishing eBooks · doi:10.4337/9781035309436 · Cited by 5 · see citing works
Towards community care: Qatar's rapidly evolving mental health landscape
Solaiman, B., & Ghuloum, S. (2021) · BJPsych International · doi:10.1192/bji.2021.21 · Cited by 5 · see citing works
Self-citationThe legal considerations of AI-blockchain for securing health data
Solaiman, B., & Dimitropoulos, G. (2024) · Edward Elgar Publishing eBooks · doi:10.4337/9781802205657.00015 · Cited by 4 · see citing works
Self-citationData protection, privacy, and confidentiality in Qatar's health system
Solaiman, B. (2023) · Edward Elgar Publishing eBooks · doi:10.4337/9781035309436.00012 · Cited by 3 · see citing works
Self-citationSolaiman, B. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4886518 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
1 of the 50 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Medicine, healthcare and the AI act: Gaps, challenges and future implications
Vardas, E. P., Marketou, M., & Vardas, P. E. (2025) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztaf041 · Cited by 60 · see citing works
Hughes, L., Davies, F., Li, K., Senali, M. G., Malik, T., & Dwivedi, Y. K. (2025) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2025.102982 · Cited by 51 · see citing works
Understanding Attitudes and Trust of Generative AI Chatbots for Social Anxiety Support
Wang, Y., Wang, Y., Crace, K., & Zhang, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714286 · Cited by 25 · see citing works
Hui, L., & Wu, X. (2025) · AI and Ethics · doi:10.1007/s43681-025-00827-0 · Cited by 16 · see citing works
Mauro, L. D., Capasso, E., Tettamanti, C., Casella, C., Francaviglia, M., Volonnino, G., Rinaldi, R., Esposito, M., & Chisari, M. G. (2025) · Journal of Forensic and Legal Medicine · doi:10.1016/j.jflm.2025.102941 · Cited by 16 · see citing works
Sustainability of AI-Assisted Mental Health Intervention: A Review of the Literature from 2020–2025
Carrasco, D. K. E., Alcántara, M. D. R. P., Várgas, C. G. A. P., Espino, B. M. S. C., Valdera, L. J. D., Cabrera, C. V., Carrasco, M. E., Valdera, A. K. D., & Córdova, L. M. A. (2025) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph22091382 · Cited by 8 · see citing works
A scoping review on the mental health harms of LLM-based chatbots
Diel, A., Torous, J., Cuijpers, P., Kleesiek, J., Nensa, F., Weber, N., Faust, F., Lalgi, T. J., Mellis, F. S., Teufel, M., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-03054-x · Cited by 4 · see citing works
“Furthermore, hallucinations can be exacerbated by LLM-CBs ’ tendencies to attempt to cover up errors 37 or when users trust LLM-CBs due to their anthropomorphic design or established emotional connections 48,54 .”
Arabic Mental Health Apps: A Narrative Review of Therapeutic and Engagement Features
Harrasi, A. A., Al-Mahrouqi, T., Al‐Alawi, M., Lawati, A. A., Al-Fahdi, A., Alzaabi, A. A., & Sinawi, H. A. (2025) · Saudi Journal of Medicine and Medical Sciences · doi:10.4103/sjmms.sjmms_62_25 · Cited by 4 · see citing works
Generative AI in healthcare: foundations, applications, challenges and future directions
Zheng, J., Li, B., Li, H., & Lu, Y. (2026) · Journal of Management Analytics · doi:10.1080/23270012.2026.2647940 · Cited by 5 · see citing works
Digital mental health needs a purpose-driven approach
Househ, M. S., Shah, H. A., Tariq, Z. U. A., Hassan, D. A., Khalifa, M., Schneider, J., Hamdi, M., Abd-Alrazaq, A. A., Ahmad, A., Solaiman, B., et al. (2026) · Nature Human Behaviour · doi:10.1038/s41562-025-02380-6 · Cited by 3 · see citing works
Self-citationGenerative Artificial Intelligence as a Psychological Intervention: Between Illusion and Risk
Izquierdo‐Condoy, J. S., Paz, C., Busch, F., & Ortiz‐Prado, E. (2026) · Journal of Medical Systems · doi:10.1007/s10916-026-02381-6 · Cited by 3 · see citing works
Generative AI in healthcare: redefining clinical practice through digital transformation
Li, J. Z., Li, X., Wu, Q., Ting, Q., & Lao, Y. (2026) · Health Economics Review · doi:10.1186/s13561-026-00812-1 · Cited by 2 · see citing works
Hosen, M. S., Fakir, M. S. I., Hawlader, S. C., Rahman, F., Karim, T., & Uddin, M. H. (2025) · European Journal of Medical and Health Research · doi:10.59324/ejmhr.2025.3(2).11 · Cited by 1 · see citing works
Acim, B., Izouaouen, Z., Kharmoum, N., & Ziti, S. (2025) · IEEE Access · doi:10.1109/access.2025.3647279 · Cited by 1 · see citing works
Goyal, R., Deshmukh, S. G., & Bolia, N. B. (2026) · Socio-Economic Planning Sciences · doi:10.1016/j.seps.2026.102461 · Cited by 2 · see citing works
On the Opportunities and Risks of Generative AI in Mental Health
Premanandan, S., Gross, J., & Ouhbi, S. (2025) · Ambient intelligence and smart environments · doi:10.3233/aise250007
Machines, Minds, and the Future of Work: Generative Ai in Strategic ForesightPreprint
Hajal, G. E., & Ivanov, S. H. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5353166
Ethical and Empathetic Requirements for Trustworthy ChatBots
Jit, S. S., & Dvornik, N. (2025) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew66121.2025.00077
AI Detection and Risk Estimation in Family Violence: Challenges and Governance
Maddox, A., Singh, S., & Jones, B. (2025) · IEEE International Symposium on Technology and Society (ISTAS) · doi:10.1109/istas65609.2025.11269633
Wu, Y., & Xu, L. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-00864-6
Butt, S., & Ceballos, H. G. (2026) · Law, governance and technology series · doi:10.1007/978-3-032-13063-1_6
Scaling-up Mental Health Services Artificial Intelligence: Regulatory Pathways and Case Study
Hooley, C., Schwarze, A. C., & Boyd, Z. M. (2026) · Journal of Technology in Human Services · doi:10.1080/15228835.2026.2670322
Machines, minds, and the future of work: GenAI and future work imaginaries
Hajal, G. E., & Ivanov, S. H. (2026) · Futures · doi:10.1016/j.futures.2026.103934
Ethische Aspekte zur KI-Anwendung in der Psychotherapie
Schürmann, J. M., Sedláková, J., & Trachsel, M. (2026) · Psychotherapie-Wissenschaft · doi:10.30820/1664-9583-2026-2-17
Kwiatkowska, A., Kołodziej, F. M., Majchrzak, M. K., Konopka, K., Kozłowska, M. J., Kabat, E. Z., Liasota, M., Komin, Z. J., Lis, M., & Juskowiak, J. (2026) · International Journal of Innovative Technologies in Social Science · doi:10.31435/ijitss.4(52).2026.6752
- Authors
- Solaiman, B.
- Affiliations
- Weill Cornell Medical College in Qatar (QA)
- Corresponding author
- Solaiman, Barry
- ORCID
- Solaiman, Barry
- Venue
- International Journal of Law and Psychiatry
- Publisher
- Elsevier BV
- Published
- 2024-11
- Volume
- 97
- Pages
- 102028
- ISSN
- 0160-2527 (ISSN-L)
- OpenAlex
- W4403526037
- Cited by
- 24
- References
- 50
- Citations by year
- 2026: 11 · 2025: 13
- FWCI
- 2.27
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International Journal of Law and Psychiatry (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Figshare (submittedVersion, cc-by)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Keywords
- generative AI, mental health services, chatbots, clinical decision-making, legal liability, medical liability, data protection, AI ethics, EU AI Act, professional guidelines, patient trust
- MeSH terms
- Artificial Intelligence, Decision Making, European Union, Humans, Mental Health, Mental Health Services, Liability, Legal
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
