Reasoning language models for more transparent prediction of suicide risk
Open accessAbstract
Background: We previously demonstrated that a large language model could estimate suicide risk using hospital discharge notes. Objective: With the emergence of reasoning models that can be run on consumer-grade hardware, we investigated whether these models can approximate the performance of much larger and costlier models. Methods: From 458 053 adults hospitalised at one of two academic medical centres between 4 January 2005 and 2 January 2014, we identified 1995 who died by suicide or accident, and matched them with 5 control individuals. We used Llama-DeepSeek-R1 8B to generate predictions of risk. Beyond discrimination and calibration, we examined the aspects of model reasoning—that is, the topics in the chain of thought—associated with correct or incorrect predictions. Findings: The cohort included 1995 individuals who died by suicide or accidental death and 9975 individuals matched 5:1, totalling 11 954 discharges and 58 933 person-years of follow-up. In Fine and Grey regression, hazard as estimated by the Llama3-distilled model was significantly associated with observed risk (unadjusted HR 4.65 (3.58–6.04)). The corresponding c-statistic was 0.64 (0.63–0.65), modestly poorer than the GPT4o model (0.67 (0.66–0.68)). In chain-of-thought reasoning, topics including Substance Abuse, Surgical Procedure, and Age-related Comorbidities were associated with correct predictions, while Fall-related Injury was associated with incorrect prediction. Conclusions: Application of a reasoning model using local, consumer-grade hardware only modestly diminished performance in stratifying suicide risk. Clinical implications: Smaller models can yield more secure, scalable and transparent risk prediction.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Chain-of-Thought prompting and natural language explanation generation produce outputs that align with clinical reasoning conventions, mimicking the differential diagnosis process familiar to practitioners.
“Chain-of-Thought prompting [62,64,78] and natural language explanation generation [72,74,75] produce outputs that align with clinical reasoning conventions, mimicking the differential diagnosis process familiar to practitioners.”
From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [74]
The Methods section (Analysis of chain of thought; page unavailable) describes the model’s chain of thought as natural-language text, and the Discussion (Clinical implications; page unavailable) says it may help clinicians understand risk factors behind a prediction. The paper does not establish that these outputs mimic differential diagnosis or align with clinical reasoning conventions. Its Discussion also says prompting was not investigated or optimised in this work. (LLM, conf=0.86)
Experimental — accuracy not yet measuredThe verbose reasoning traces produced by agentic decomposition systems and reasoning models are valuable for retrospective audit but may overwhelm a clinician seeking a rapid decision aid during a time-pressured consultation.
“The verbose reasoning traces produced by agentic decomposition systems and reasoning models [74, 78] are valuable for retrospective audit but may overwhelm a clinician seeking a rapid decision aid during a time-pressured consultation.”
From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [74]
The Discussion and Clinical implications sections say that examining chains of thought aids analysis of predictions and helps clinicians understand risk factors. The paper does not say that the traces are verbose, that they overwhelm clinicians, or that they impede rapid decisions in time-pressured consultations. No page numbers are available in our copy. (LLM, conf=0.86)
Experimental — accuracy not yet measuredA third, more recent group uses a clinical instrument to organize the reasoning process itself rather than the explanation surface.
“A third, more recent group uses a clinical instrument to organize the reasoning process itself rather than the explanation surface [74,77-79].”
From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [74]
The Background distinguishes the model’s self-directed chain of thought from post hoc explanations, and Methods > LLM scoring describes a prompt that supplies suicide risk and protective factors. But that section calls the factors a synopsis adapted from Veterans Administration resources, not a clinical instrument, and does not show an instrument organizing the reasoning process. (LLM, conf=0.83)
Experimental — accuracy not yet measuredThe few studies that move beyond purely local explanations point to promising but underexplored directions.
“The few studies that move beyond purely local explanations [11, 74,80,81,104,116,125,129] point to promising but underexplored directions.”
From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [74]
Open-source alternatives support privacy-preserving local deployment with transparent CoT traces through distilled reasoning models.
“and, through distilled reasoning models, support privacy-preserving local deployment with transparent CoT traces [74].”
From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [74]
Open-source models support privacy-preserving local deployment.
“and privacy-preserving local deployment [74]”
From Explainability of decoder-only clinical large language models: A scoping review (2026) · cites this paper as [74]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Charlson, M. E., Pompei, P., Ales, K. L., & MacKenzie, C. R. (1987) · Journal of Chronic Diseases · doi:10.1016/0021-9681(87)90171-8 · Cited by 49,742 · see citing works
A Proportional Hazards Model for the Subdistribution of a Competing Risk
Fine, J. P., & Gray, R. J. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10474144 · Cited by 14,290 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,649 · see citing works
Roth, G. A., Abate, D., Abate, K. H., Abay, S. M., Abbafati, C., Abbasi, N., Abbastabar, H., Abd-Allah, F., Abdela, J., Abdelalim, A., et al. (2018) · The Lancet · doi:10.1016/s0140-6736(18)32203-7 · Cited by 8,600 · see citing works
(2009) · Cited by 1,900 · see citing works
DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al. (2025) · Nature · doi:10.1038/s41586-025-09422-z · Cited by 1,701 · see citing works
The Leading Causes of Death in the US for 2020
Ahmad, F. B., & Anderson, R. N. (2021) · JAMA · doi:10.1001/jama.2021.5469 · Cited by 954 · see citing works
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.12948 · Cited by 538 · see citing works
Suicidal risk following hospital discharge: a review
Forte, A., Buscajoni, A., Fiorillo, A., Pompili, M., & Baldessarini, R. J. (2019) · Harvard Review of Psychiatry · doi:10.1097/hrp.0000000000000222 · Cited by 150 · see citing works
McCoy, T. H., Castro, V. M., Roberson, A. M., Snapper, L. A., & Perlis, R. H. (2016) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2016.2172 · Cited by 130 · see citing works
Self-citationA Comprehensive Evaluation of Quantization Strategies for Large Language Models
Jin, R., Du, J., Huang, W., Liu, W., Luan, J., Wang, B., & Xiong, D. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.726 · Cited by 71 · see citing works
Comparative Analysis of Suicide, Accidental, and Undetermined Cause of Death Classification
Gray, D., Coon, H. H., McGlade, E., Callor, W. B., Byrd, J., Viskochil, J., Bakian, A. V., Yurgelun-Todd, D. A., Grey, T. C., & McMahon, W. M. (2014) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12079 · Cited by 69 · see citing works
Reconciling the contrasting narratives on the environmental impact of large language models
Ren, S., Tomlinson, B., Black, R. W., & Torrance, A. W. (2024) · Scientific Reports · doi:10.1038/s41598-024-76682-6 · Cited by 68 · see citing works
Vasiliadis, H., Ngui, A. N., & LESAGE, A. D. (2014) · Psychiatric Services · doi:10.1176/appi.ps.201400133 · Cited by 64 · see citing works
Temporal trends and characteristics of reportable health data breaches, 2010-2017
McCoy, T. H., & Perlis, R. H. (2018) · JAMA · doi:10.1001/jama.2018.9222 · Cited by 62 · see citing works
Self-citationMcCoy, T. H., Han, L., Pellegrini, A. M., Tanzi, R. E., Berretta, S., & Perlis, R. H. (2019) · Alzheimer s & Dementia · doi:10.1016/j.jalz.2019.09.084 · Cited by 51 · see citing works
Self-citationMcCoy, T. H., Pellegrini, A. M., & Perlis, R. H. (2019) · Depression and Anxiety · doi:10.1002/da.22882 · Cited by 31 · see citing works
Self-citationBoag, W., Kovaleva, O., McCoy, T. H., Rumshisky, A., Peter Szolovits, B., & Perlis, R. H. (2021) · Translational Psychiatry · doi:10.1038/s41398-020-01104-w · Cited by 30 · see citing works
Self-citationEnhancing Delirium Case Definitions in Electronic Health Records Using Clinical Free Text
McCoy, T. H., Chaukos, D. C., Snapper, L. A., Hart, K. L., Stern, T. A., & Perlis, R. H. (2016) · Psychosomatics · doi:10.1016/j.psym.2016.10.007 · Cited by 26 · see citing works
Self-citationAhmed, A., Hou, M., Xi, R., Zeng, X., & Shah, S. A. (2024) · Companion Proceedings of the ACM Web Conference 2024 · doi:10.1145/3589335.3651904 · Cited by 22 · see citing works
Zero Suicide Model Implementation and Suicide Attempt Rates in Outpatient Mental Health Care
Ahmedani, B. K., Penfold, R. B., Frank, C. B., Richards, J. A., Stewart, C. C., Boggs, J. M., Coleman, K. J., Sterling, S., Yarborough, B. J. H., Clarke, G. N., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.3721 · Cited by 21 · see citing works
Environmental impact of large language models in medicine
Kleinig, O., Sinhal, S., Khurram, R., Gao, C., Spajic, L., Zannettino, A. C. W., Schnitzler, M., Guo, C. W., Zaman, S. J., Smallbone, H., et al. (2024) · Internal Medicine Journal · doi:10.1111/imj.16549 · Cited by 20 · see citing works
McCoy, T. H., & Hughes, M. C. (2018) · JAMA Network Open · doi:10.1001/jamanetworkopen.2018.6029 · Cited by 13 · see citing works
Self-citationHard Truths About Suicide Prevention
Perlis, R. H., & Fihn, S. D. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.22713 · Cited by 12 · see citing works
Self-citationApplying large language models to stratify suicide risk using narrative clinical notes
McCoy, T. H., & Perlis, R. H. (2025) · Journal of Mood and Anxiety Disorders · doi:10.1016/j.xjmad.2025.100109 · Cited by 12 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Generative Large Language Models in Mental Health Care Settings: Systematic Review and Meta-Analysis
Leung, J., Johnson, B., Mcrae, K., Fong, S., Gonzalez, P. C., McClure‐Thomas, C., Sun, T., Kern, N., Wong, Y. M., Liu, R., et al. (2026) · JMIR AI · doi:10.2196/87730 · Cited by 3 · see citing works
Reichenpfader, D., Gabarrón, E., Lüthi, S., Bürki, J. R., Kroißenbrunner, A., & Denecke, K. (2026) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-026-00647-x · Cited by 1 · see citing works
Applications of DeepSeek in Medicine: Bibliometric Analysis and Scoping Review
Zhang, H., Wang, D., Xu, Y., Han, S., & Wang, G. (2026) · Journal of Medical Internet Research · doi:10.2196/93354 · Cited by 1 · see citing works
Li, K. C., Zirikly, A., Collica, S. C., Goes, F. S., Zhao, C., Nguyen, T. Q., Gagliardi, J. P., Goldstein, B. A., Hong, H., Stuart, E. A., et al. (2026) · JMIR Formative Research · doi:10.2196/86906 · Cited by 1 · see citing works
Ay, M., Öztan, D. T., Asri, Ş., Karaveli, A., Öztürk, N., Alparslan, A. Ş., & Ay, S. A. (2026) · Medicina · doi:10.3390/medicina62091680 · Cited by 1 · see citing works
Explainability of decoder-only clinical large language models: A scoping review
Mishra, N., Abu-Hanna, A., & Calixto, I. (2026) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2026.103501
Positive sentiments in early academic literature on DeepSeek: a cross-disciplinary mini review
He, Y., Giangan, A., Vu, N., & Watters, C. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1725853
Wrzosek, K., Udrycka, K., Kuchta, K., Poleszak, J., Leśniak, M., & Kowalewski, P. (2026) · International Journal of Innovative Technologies in Social Science · doi:10.31435/ijitss.3(51).2026.6484
- Authors
- McCoy, T. H., & Perlis, R. H.
- Affiliations
- Harvard University (US)
- Corresponding author
- Perlis, Roy H
- Venue
- BMJ Mental Health
- Publisher
- BMJ
- Published
- 2025-05
- Volume
- 28
- Issue
- 1
- Pages
- e301654
- ISSN
- 2755-9734 (ISSN-L)
- PMC
- PMC12067846
- OpenAlex
- W4410268067
- Cited by
- 8
- References
- 25
- Citations by year
- 2026: 8
- FWCI
- 9.45
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- BMJ Mental Health (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institute of Mental Health (1U01MH136059-01, 5R01MH120991-05)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Suicide and Self-Harm Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Keywords
- suicide risk prediction, reasoning language models, chain-of-thought, model distillation, consumer hardware, risk stratification, model transparency, accidental death, substance abuse, fall-related injuries
- MeSH terms
- Adult, Aged, Female, Humans, Male, Middle Aged, Suicide, Risk Assessment, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
