CoChat
Sign Up
Cited by 21Suicide and Self-Harm StudiesArtificial Intelligence in Healthcare and EducationMental Health via Writing

The impact of history of depression and access to weapons on suicide risk assessment: a comparison of ChatGPT-3.5 and ChatGPT-4

Open access
Shinan-Altman, Shiri, Elyoseph, Zohar, Levkovich, Inbar·2024·PeerJ
DOI 10.7717/peerj.17468PMC PMC11143969
Open this paper in CoChat — free account

Abstract

The aim of this study was to evaluate the effectiveness of ChatGPT-3.5 and ChatGPT-4 in incorporating critical risk factors, namely history of depression and access to weapons, into suicide risk assessments. Both models assessed suicide risk using scenarios that featured individuals with and without a history of depression and access to weapons. The models estimated the likelihood of suicidal thoughts, suicide attempts, serious suicide attempts, and suicide-related mortality on a Likert scale. A multivariate three-way ANOVA analysis with Bonferroni post hoc tests was conducted to examine the impact of the forementioned independent factors (history of depression and access to weapons) on these outcome variables. Both models identified history of depression as a significant suicide risk factor. ChatGPT-4 demonstrated a more nuanced understanding of the relationship between depression, access to weapons, and suicide risk. In contrast, ChatGPT-3.5 displayed limited insight into this complex relationship. ChatGPT-4 consistently assigned higher severity ratings to suicide-related variables than did ChatGPT-3.5. The study highlights the potential of these two models, particularly ChatGPT-4, to enhance suicide risk assessment by considering complex risk factors.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.7717/peerj.17468

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • GPT-3.5 has limited analytical capabilities, whereas GPT-4 demonstrates greater accuracy in assessing human psychological aspects, such as suicide risk assessments.

    “Indeed, various studies have found that the free version (GPT-3.5)has limited analytical capabilities, whereas the paid version (GPT-4) demonstrates greater accuracy in assessing human psychological aspects, such as suicide risk assessments (Shinan-Altman et al., 2024).”

    From Teachers vs. Chat GPT: identifying and naming acts of online sexual abuse (2025) · cites this paper as Shinan-Altman et al., 2024

    The Discussion reports that ChatGPT-4 showed a more nuanced assessment of suicide-risk factors than ChatGPT-3.5, supporting the comparison of their capabilities. But it describes more accurate assessments as a potential outcome, not an established finding: the study did not directly test accuracy against mental health professionals and calls for real-world validation. It also does not establish the broader claim about assessing human psychological aspects generally. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 45

  1. Baek IC Jo S Kim EJ Lee GR Lee DH Jeon HJ A review of suicide risk assessment tools and their measured psychometric properties in Korea Frontiers in Psychiatry 2021 12 679779 10.3389/fpsyt.2021.679779 34239463 PMC8258346 · DOI
  2. Betz ME Thomas AC Simonetti JA Firearms, suicide, and approaches for prevention The Journal of the American Medical Association 2022 328 12 1179 1180 10.1001/jama.2022.16663 36166015 · DOI
  3. Bolton JM Walld R Chateau D Finlayson G Sareen J Risk of suicide and suicide attempts associated with physical disorders: a population-based, balancing score-matched analysis Psychological Medicine 2015 45 3 495 504 10.1017/S0033291714001639 25032807 · DOI
  4. Bryan CJ Bryan AO Anestis MD Rates of preparatory suicidal behaviors across subgroups of protective firearm owners Archives of Suicide Research 2022 26 2 948 960 10.1080/13811118.2020.1848672 33211635 · DOI
  5. Burnette C Ramchand R Ayer L Gatekeeper training for suicide prevention: a theoretical model and review of the empirical literature Rand Health Quarterly 2015 5 1 16 PMC5158249 28083369
  6. Cascella M Montomoli J Bellini V Bignami E Evaluating the feasibility of ChatGPT in healthcare: an analysis of multiple clinical and research scenarios Journal of Medical Systems 2023 47 1 33 10.1007/s10916-023-01925-4 36869927 PMC9985086 · DOI · Paper page
  7. Chiu CC Liu HC Li WH Tsai SY Chen CC Kuo CJ Incidence, risk and protective factors for suicide mortality among patients with major depressive disorder Asian Journal of Psychiatry 2023 80 103399 10.1016/j.ajp.2022.103399 36542894 · DOI
  8. Demesmaeker A Chazard E Hoang A Vaiva G Amad A Suicide mortality after a nonfatal suicide attempt: a systematic review and meta-analysis Australian & New Zealand Journal of Psychiatry 2022 56 6 603 616 10.1177/00048674211043455 34465221 · DOI
  9. Elyoseph Z Gur T Haber Y Simon T Angert T Navon Y Tal A Asman O An ethical perspective on the democratization of mental health with generative artificial intelligence Preprint JMIR Mental Health 2024c PMC11500620 39417792
  10. Elyoseph Z Levkovich I Beyond human expertise: the promise and limitations of ChatGPT in suicide risk assessment Frontiers in Psychiatry 2023 14 574 10.3389/fpsyt.2023.1213141 PMC10427505 37593450 · DOI · Paper page
  11. Elyoseph Z Levkovich I Comparing the perspectives of generative AI, mental health experts, and the general public on schizophrenia recovery: case vignette study JMIR Mental Health 2024 11 2 e53043 10.2196/53043 38533615 PMC11004608 · DOI
  12. Elyoseph Z Levkovich I Rabin E Shemo G Szpiler T Shoval DH Belz YL Applying language models for suicide prevention: evaluating news article adherence to WHO reporting guidelines Research Square 2024b 10.21203/rs.3.rs-4180591/v1 PMC12181428 40542269 · DOI
  13. Elyoseph Z Levkovich I Shinan-Altman S Assessing prognosis in depression: comparing perspectives of AI models, mental health professionals and the general public Family Medicine and Community Health 2024d 12 Suppl 1 e002583 10.1136/fmch-2023-002583 38199604 PMC10806564 · DOI
  14. Elyoseph Z Refoua E Asraf K Lvovsky M Shimoni Y Hadar-Shoval D Capacity of generative AI to interpret human emotions from visual and textual data: pilot evaluation study JMIR Mental Health 2024a 11 2 e54369. 10.2196/54369 38319707 PMC10879976 · DOI
  15. Ghanadian H Nejadgholi I Osman HA ChatGPT for suicide risk assessment on social media: quantitative evaluation of model performance, potentials and limitations ArXiv 2023 10.48550/arXiv.2306.09390 · DOI
  16. Gvion Y Levi-Belz Y Serious suicide attempts: systematic review of psychological risk factors Frontiers in Psychiatry 2018 9 56 10.3389/fpsyt.2018.00056 29563886 PMC5845877 · DOI
  17. Haasz M Myers MG Rowhani-Rahbar A Zimmerman MA Seewald L Sokol RL Cunningham RM Carter PM Firearms availability among high-school age youth with recent depression or suicidality Pediatrics 2023 151 6 e2022059532. 10.1542/peds.2022-059532 37212021 PMC10233739 · DOI
  18. Haber Y Levkovich I Hadar-Shoval D Elyoseph Z The artificial third: a broad view of the effects of introducing generative artificial intelligence on psychotherapy JMIR Mental Health 2024 10.2196/preprints.54781 PMC11137430 38787297 · DOI
  19. Hadar-Shoval D Asraf K Mizrachi Y Haber Y Elyoseph Z Assessing the alignment of large language models with human values for mental health integration: cross-sectional study using Schwartz’s theory of basic values JMIR Mental Health 2024 11 e55988. 10.2196/55988 38593424 PMC11040439 · DOI
  20. Hadar-Shoval D Elyoseph Z Lvovsky M The plasticity of ChatGPT’s mentalizing abilities: personalization for personality structures Frontiers in Psychiatry 2023 14 1234397 10.3389/fpsyt.2023.1234397 37720897 PMC10503434 · DOI
  21. Hawton K Comabella CC Haw C Saunders K Risk factors for suicide in individuals with depression: a systematic review Journal of Affective Disorders 2013 147 1–3 17 28 10.1016/j.jad.2013.01.004 23411024 · DOI · Paper page
  22. He N Yan Y Wu Z Cheng Y Liu F Li X Zhai S Chat GPT-4 significantly surpasses GPT-3.5 in drug information queries Journal of Telemedicine and Telecare 2023 1 3 10.1177/1357633X231181922 37350055 · DOI
  23. Helm PJ Medrano MR Allen JJ Greenberg J Existential isolation, loneliness, depression, and suicide ideation in young adults Journal of Social and Clinical Psychology 2020 39 8 641 674 10.1521/jscp.2020.39.8.641 · DOI
  24. Ilic I Zivanovic Macuzic I Kocic S Ilic M Worldwide suicide mortality trends by firearm (1990-2019): a joinpoint regression analysis PLOS ONE 2022 17 5 e0267817. 10.1371/journal.pone.0267817 35613099 PMC9132310 · DOI
  25. Junior ARC de Guadalupe Correa JFD Lemos T Teixeira EP de Souza MDL Risk factors for suicide: systematic review Saudi Journal for Health Sciences 2020 9 3 183 193 10.4103/sjhs.sjhs_83_20 · DOI
  26. Levi-Belz Y Gamliel E The effect of perceived burdensomeness and thwarted belongingness on therapists’ assessment of patients’ suicide risk Psychotherapy Research 2016 26 4 436 445 10.1080/10503307.2015.1013161 25751580 · DOI
  27. Levkovich I Elyoseph Z Suicide risk assessments through the eyes of ChatGPT-3.5 vs ChatGPT-4: vignette study JMIR Mental Health 2023 10 7 e51232 10.2196/51232 37728984 PMC10551796 · DOI
  28. Lewandowski M Łukowicz P Świetlik D Barańska-Rybak W An original study of ChatGPT-3.5 and ChatGPT-4 dermatological knowledge level based on the dermatology specialty certificate examinations Clinical and Experimental Dermatology 2023 255 505 10.1093/ced/llad255 37540015 · DOI
  29. Lewiecki EM Miller SA Suicide, guns, and public policy American Journal of Public Health 2013 103 1 27 31 10.2105/AJPH.2012.300964 23153127 PMC3518361 · DOI
  30. Li Y Guo Z Tian W Wang X Dou W Chen Y Huang S Ni S Wang H Wang C An investigation of the relationships between suicidal ideation, psychache, and meaning in life using network analysis BMC Psychiatry 2023 23 257 10.1186/s12888-023-04700-4 37069569 PMC10111716 · DOI
  31. McCourt AD Firearm access and suicide: lethal means counseling and safe storage education in a comprehensive prevention strategy American Journal of Public Health 2021 111 2 185 187 10.2105/AJPH.2020.306059 33439717 PMC7811084 · DOI
  32. Nori H King N McKinney SM Carignan D Horvitz E Capabilities of GPT-4 on medical challenge problems ArXiv 2023 10.48550/arXiv.2303.13375 · DOI · Paper page
  33. Orsolini L Latini R Pompili M Serafini G Volpe U Vellante F Fornaro M Valchera A Tomasetti C Fraticelli S Alessandrini M La Rovere R Trotta S Martinotti G Di Giannantonio M De Berardis D Understanding the complex of suicide in depression: from research to clinics Psychiatry Investigation 2020 17 3 207 221 10.30773/pi.2019.0171 32209966 PMC7113180 · DOI
  34. O’Connor EA Perdue LA Coppola EL Henninger ML Thomas RG Gaynes BN Depression and suicide risk screening: updated evidence report and systematic review for the US preventive services task force The Journal of the American Medical Association 2023 329 23 2068 2085 10.1001/jama.2023.7787 37338873 PMC13533462 · DOI
  35. Pallin R Barnhorst A Clinical strategies for reducing firearm suicide Injury Epidemiology 2021 8 1 1 10 10.1186/s40621-021-00352-8 34607607 PMC8489372 · DOI
  36. Parray AA Inam ZM Ramonfaur D Haider SS Mistry SK Pandya AK ChatGPT and global public health: applications, challenges, ethical considerations and mitigation strategies Global Transitions 2023 5 1 5 54 10.1016/j.glt.2023.05.001 · DOI · Paper page
  37. Perret G Abudureheman A Perret-Catipovic M Flomenbaum M La Harpe R Suicides in the young people of Geneva, Switzerland, from 1993 to 2002 Journal of Forensic Science 2006 51 1169 1173 10.1111/j.1556-4029.2006.00230.x 17018104 · DOI
  38. Shahsavar Y Choudhury A User intentions to use ChatGPT for self-diagnosis and health-related purposes: cross-sectional survey study JMIR Human Factors 2023 10 1 e47564. 10.2196/47564 37195756 PMC10233444 · DOI · Paper page
  39. Smith EG Kim HM Ganoczy D Stano C Pfeiffer PN Valenstein M Suicide risk assessment received prior to suicide death by veterans health administration patients with a history of depression The Journal of Clinical Psychiatry 2013 74 3 226 232 10.4088/JCP.12m07853 23561227 PMC4055158 · DOI
  40. Swahn MH Ali B Bossarte RM Van Dulmen M Crosby A Jones AC Schinka KC Self-harm and suicide attempts among high-risk, urban youth in the US: shared and unique risk and protective factors International Journal of Environmental Research and Public Health 2012 9 1 178 191 10.3390/ijerph9010178 22470286 PMC3315085 · DOI
  41. Swanson JW McGinty EE Fazel S Mays VM Mental illness and reduction of gun violence and suicide: bringing epidemiologic research to policy Annals of Epidemiology 2015 25 5 366 376 10.1016/j.annepidem.2014.03.004 24861430 PMC4211925 · DOI
  42. Tal A Elyoseph Z Haber Y Angert T Gur T Simon T Asman O The artificial third: utilizing ChatGPT in mental health The American Journal of Bioethics 2023 23 10 74 77 10.1080/15265161.2023.2250297 37812102 · DOI
  43. Teebagy S Colwell L Wood E Yaghy A Faustina M Improved performance of ChatGPT-4 on the OKAP exam: a comparative study with ChatGPT-3.5 MedRxiv 2023 10.1101/2023.04.03.23287957 PMC10495224 37701862 · DOI
  44. Witt JK Brockmole JR Action alters object identification: wielding a gun increases the bias to see guns Journal of Experimental Psychology: Human Perception and Performance 2012 38 5 1159 1167 10.1037/a0027881 22506781 · DOI
  45. Zalsman G Suicide: epidemiology, etiology, treatment and prevention Harefuah 2019 158 7 468 472 31339248

Cited by

Sorted by citations

Authors
Shinan-Altman, S., Elyoseph, Z., & Levkovich, I.
Affiliations
  • School of Social Work, Bar Ilan University, Ramat Gan, Israel
  • The Max Stern Yezreel Valley College (IL)
  • Oranim Academic College of Education (IL)
Corresponding author
Elyoseph, Zohar
ORCID
Venue
PeerJ
Publisher
PeerJ
Published
2024-05-29
Volume
12
Pages
e17468
ISSN
2167-8359 (ISSN-L)
DOI
10.7717/peerj.17468
PMC
PMC11143969
OpenAlex
W4399130765
Cited by
21
References
7
Citations by year
2026: 8 · 2025: 12 · 2024: 1
FWCI
8.59
Type
journal article
Language
en
Open access
Licence
Locations
  • PeerJ (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Topics
  • Social Sciences › Psychology › Clinical Psychology › Suicide and Self-Harm Studies
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Psychology › Social Psychology › Mental Health via Writing
Keywords
suicide risk assessment, history of depression, ChatGPT-3.5, ChatGPT-4, large language models, suicidal ideation, suicide attempts, risk factors
MeSH terms
Suicide Prevention, Adult, Depression, Female, Humans, Male, Middle Aged, Risk Factors, Suicide, Suicide, Attempted, Risk Assessment, Weapons, Young Adult, Suicidal Ideation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review