Prediction Models for Suicide Attempts and Deaths: A Systematic Review and Simulation
Abstract
Importance: Suicide prediction models have the potential to improve the identification of patients at heightened suicide risk by using predictive algorithms on large-scale data sources. Suicide prediction models are being developed for use across enterprise-level health care systems including the US Department of Defense, US Department of Veterans Affairs, and Kaiser Permanente. Objectives: To evaluate the diagnostic accuracy of suicide prediction models in predicting suicide and suicide attempts and to simulate the effects of implementing suicide prediction models using population-level estimates of suicide rates. Evidence Review: A systematic literature search was conducted in MEDLINE, PsycINFO, Embase, and the Cochrane Library to identify research evaluating the predictive accuracy of suicide prediction models in identifying patients at high risk for a suicide attempt or death by suicide. Each database was searched from inception to August 21, 2018. The search strategy included search terms for suicidal behavior, risk prediction, and predictive modeling. Reference lists of included studies were also screened. Two reviewers independently screened and evaluated eligible studies. Findings: From a total of 7306 abstracts reviewed, 17 cohort studies met the inclusion criteria, representing 64 unique prediction models across 5 countries with more than 14 million participants. The research quality of the included studies was generally high. Global classification accuracy was good (≥0.80 in most models), while the predictive validity associated with a positive result for suicide mortality was extremely low (≤0.01 in most models). Simulations of the results suggest very low positive predictive values across a variety of population assessment characteristics. Conclusions and Relevance: To date, suicide prediction models produce accurate overall classification models, but their accuracy of predicting a future event is near 0. Several critical concerns remain unaddressed, precluding their readiness for clinical applications across health systems.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Even complex risk assessment models using artificial intelligence do not allow for precise prediction of suicidal behavior.
“Even complex risk assessment models do not allow for precise prediction of suicidal behavior (26, 27)”
From Suicide risk assessment: clinical implications of the unpredictability of suicidal behavior (2026) · cites this paper as [27]
Artificial intelligence models do not allow for prediction of the time at which suicidal behavior will occur.
“and, in particular, they do not allow for prediction of the time at which suicidal behavior will occur.”
From Suicide risk assessment: clinical implications of the unpredictability of suicidal behavior (2026) · cites this paper as [27]
Events with a low base rate, such as suicide attempts and suicides, can never be precisely predicted, no matter which assessments and methods are used.
“In principle, it is important to acknowledge that events with a low base rate, such as suicide attempts and suicides, can never be precisely predicted-no matter which assessments and methods are used (Belsher et al., 2019;Bryan, 2021).”
From Suicide Crisis Syndrome: A systematic review (2024) · cites this paper as Belsher et al., 2019
Most suicide risk prediction models had a positive predictive value less than 1% for predicting suicide mortality.
“most models had a PPV <1% for predicting suicide mortality”
From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [10]
Multiple research groups have developed suicide risk prediction models using electronic health record data.
“multiple research groups have developed suicide risk prediction models using electronic health record data.10,11,12,13”
From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [10]
A recent systematic review identified 17 suicide risk prediction models.
“A recent systematic review10 identified 17 such predictive models that have been described in the literature.”
From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [10]
A 2019 review of 64 longitudinal suicide prediction models found most models reported extremely low PPVs (ie, <.01).
“A 2019 review of 64 longitudinal suicide prediction models36 found most models reported extremely low (ie, <.01) PPVs, such that 99 out of every 100 individuals predicted by these models to attempt suicide will not.”
From A pilot study using frequent inpatient assessments of suicidal thinking to predict short-term postdischarge suicidal behavior (2021) · cites this paper as [36]
The dynamic feature model's predictive accuracy (AUC 0.89) is at the high end of the range of reported AUC values for longitudinal prediction models of suicidal behaviors.
“The dynamic feature model predicted posthospital suicide attempts with good accuracy (AUC, 0.89), which is at the high end of the range of reported AUC values for longitudinal prediction models of suicidal behaviors.36”
From A pilot study using frequent inpatient assessments of suicidal thinking to predict short-term postdischarge suicidal behavior (2021) · cites this paper as [36]
The average cross-validated PPV for the dynamic feature model (0.39) represents a substantial improvement beyond previous work and the baseline and mean models.
“In comparison, the average cross-validated PPV for our dynamic feature model was 0.39. This represents a substantial improvement beyond both previous work and our baseline and mean models”
From A pilot study using frequent inpatient assessments of suicidal thinking to predict short-term postdischarge suicidal behavior (2021) · cites this paper as [36]
Machine learning methods have yielded high AUCs, but the PPV of most models has been low.
“Although machine learning methods have yielded high AUCs, the PPV of most models has been low.”
From A pilot study using frequent inpatient assessments of suicidal thinking to predict short-term postdischarge suicidal behavior (2021) · cites this paper as [36]
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · see citing works
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859 · see citing works
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199 · see citing works
Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research
Franklin, J. C., Ribeiro, J. D., Fox, K. R., Bentley, K. H., Kleiman, E. M., Huang, X., Musacchio, K. M., Jaroszewski, A. C., Chang, B. P., & Nock, M. K. (2017) · Psychological Bulletin · doi:10.1037/bul0000084 · Cited by 3,784 · see citing works
Wang, Y., Kung, L., & Byrd, T. A. (2018) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2015.12.019 · Cited by 1,571 · see citing works
Prediction of Suicide in Psychiatric Patients
Pokorny, A. D. (1983) · Archives of General Psychiatry · doi:10.1001/archpsyc.1983.01790030019002 · Cited by 749 · see citing works
Predicting Risk of Suicide Attempts Over Time Through Machine Learning
Walsh, C. G., Ribeiro, J. D., & Franklin, J. C. (2017) · Clinical Psychological Science · doi:10.1177/2167702617691560 · Cited by 571 · see citing works
Uses and abuses of screening tests
Grimes, D. A., & Schulz, K. F. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07948-5 · Cited by 545 · see citing works
Kessler, R. C., Warner, C. H., Ivany, C., Petukhova, M. V., Rose, S., Bromet, E. J., Brown, M., Cai, T., Colpe, L. J., Cox, K. L., et al. (2015) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2014.1754 · Cited by 447 · see citing works
Simon, G. E., Johnson, E., Lawrence, J. M., Rossom, R. C., Ahmedani, B., Lynch, F. L., Beck, A., Waitzfelder, B., Ziebell, R., Penfold, R. B., et al. (2018) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2018.17101167 · Cited by 400 · see citing works
Carter, G., Milner, A., McGill, K., Pirkis, J., Kapur, N., & Spittal, M. J. (2017) · British Journal of Psychiatry · doi:10.1192/bjp.bp.116.182717 · Cited by 361 · see citing works
Predicting suicidal behavior from longitudinal electronic health records.
Barak-Corren, Y., Castro, V. M., Javitt, S., Hoffnagle, A. G., Dai, Y., Perlis, R. H., Nock, M. K., Smoller, J. W., & Reis, B. Y. (2017) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2016.16010077 · Cited by 339 · see citing works
Diagnosing Tests: Using and Misusing Diagnostic and Screening Tests
Streiner, D. L. (2003) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8103_03 · Cited by 299 · see citing works
Schoenbaum, M., Kessler, R. C., Gilman, S. E., Colpe, L. J., Heeringa, S. G., Stein, M. B., Ursano, R. J., & Cox, K. L. (2014) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2013.4417 · Cited by 281 · see citing works
McCarthy, J. F., Bossarte, R. M., Katz, I. R., Thompson, C., Kemp, J., Hannemann, C. M., Nielson, C., & Schoenbaum, M. (2015) · American Journal of Public Health · doi:10.2105/ajph.2015.302737 · Cited by 251 · see citing works
Runeson, B., Odeberg, J., Pettersson, A., Edbom, T., Jildevik Adamsson, I., & Waern, M. (2017) · PLOS ONE · doi:10.1371/journal.pone.0180292 · Cited by 250 · see citing works
Kessler, R. C., Hwang, I., Hoffmire, C. A., McCarthy, J. F., Petukhova, M. V., Rosellini, A. J., Sampson, N. A., Schneider, A. L., Bradley, P. A., Katz, I. R., et al. (2017) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1575 · Cited by 209 · see citing works
Nock, M. K., Millner, A. J., Joiner, T. E., Gutierrez, P. M., Han, G., Hwang, I., King, A., Naifeh, J. A., Sampson, N. A., Zaslavsky, A. M., et al. (2018) · Journal of Abnormal Psychology · doi:10.1037/abn0000317 · Cited by 193 · see citing works
Kessler, R. C., Stein, M. B., Petukhova, M. V., Bliese, P., Bossarte, R. M., Bromet, E. J., Fullerton, C. S., Gilman, S. E., Ivany, C., Lewandowski-Romps, L., et al. (2017) · Molecular Psychiatry · doi:10.1038/mp.2016.110 · Cited by 163 · see citing works
Suicide prevention: an emerging priority for health care
Hogan, M. F., & Grumet, J. G. (2016) · Health Affairs · doi:10.1377/hlthaff.2015.1672 · Cited by 159 · see citing works
LeFevre, M. L., & on behalf of the U.S. Preventive Services Task Force* (2014) · Annals of Internal Medicine · doi:10.7326/m14-0589 · Cited by 154 · see citing works
Tran, T., Luo, W., Phung, D., Harvey, R., Berk, M., Kennedy, R. L., & Venkatesh, S. (2014) · BMC Psychiatry · doi:10.1186/1471-244x-14-76 · Cited by 150 · see citing works
Detecting and treating suicide ideation in all settings.
(2016) · PubMed · Cited by 143 · see citing works
Walkup, J. T., Townsend, L. D., Crystal, S., & Olfson, M. (2012) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.2335 · Cited by 133 · see citing works
The futility of risk prediction in psychiatry
Mulder, R., Newton‐Howes, G., & Coid, J. W. (2016) · The British Journal of Psychiatry · doi:10.1192/bjp.bp.116.184960 · Cited by 117 · see citing works
Choi, S. B., Lee, W., Yoon, J., Won, J., & Kim, D. W. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2018.01.019 · Cited by 85 · see citing works
Department of Defense Suicide Event Report Calendar Year 2013 Annual Report
Smolenski, D. J., Reger, M. A., Bush, N. E., Skopp, N. A., Zhang, Y., & Campise, R. L. (2014) · doi:10.21236/ada617001 · Cited by 74 · see citing works
Self-citationUsing Neural Networks with Routine Health Records to Identify Suicide Risk: Feasibility Study
DelPozo‐Baños, M., John, A. M., Petkov, N., Berridge, D. M., Southern, K., Lloyd, K. R., Jones, C. H. D., Spencer, S., & Travieso, C. M. (2018) · JMIR Mental Health · doi:10.2196/10144 · Cited by 58 · see citing works
Suicide Risk Assessment and Prevention: A Systematic Review Focusing on Veterans
Nelson, H. D., Denneson, L. M., Low, A., Bauer, B. W., O'Neil, M. E., Kansagara, D. L., & Teo, A. R. (2017) · Psychiatric Services · doi:10.1176/appi.ps.201600384 · Cited by 56 · see citing works
Predicting High-Risk Behaviors in Veterans With Posttraumatic Stress Disorder
Hartl, T. L., Rosen, C. S., Drescher, K. D., Lee, T. T., & Gusman, F. D. (2005) · The Journal of Nervous and Mental Disease · doi:10.1097/01.nmd.0000168238.13252.b3 · Cited by 55 · see citing works
An Assessment of the Utility of Suicide Prediction*
MacKinnon, D. R., & Farberow, N. L. (1976) · Suicide and Life-Threatening Behavior · doi:10.1111/j.1943-278x.1976.tb00673.x · Cited by 47 · see citing works
Amini, P., Ahmadinia, H., Poorolajal, J., & Amiri, M. M. (2016) · PubMed Central · Cited by 46 · see citing works
Mustafa, R. A., Wiercioch, W., Cheung, A., Prediger, B., Brożek, J., Bossuyt, P. M., Garg, A. X., Lelgemann, M., Bühler, D., & Schünemann, H. J. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.09.003 · Cited by 45 · see citing works
Karmakar, C. K., Luo, W., Tran, T., Berk, M., & Venkatesh, S. (2016) · JMIR Mental Health · doi:10.2196/mental.5475 · Cited by 40 · see citing works
Ilgen, M. A., Downing, K., Zivin, K., Hoggatt, K. J., Kim, H. M., Ganoczy, D. A., Austin, K., McCarthy, J. F., Patel, J. M., & Valenstein, M. M. (2009) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.08m04795 · Cited by 39 · see citing works
Evaluation of clinical prognostic models for suicide attempts after a major depressive episode
Galfalvy, H., Oquendo, M. A., & Mann, J. J. (2008) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2008.01162.x · Cited by 37 · see citing works
Suicide Rates and Methods in Active Duty Military Personnel, 2005 to 2011
Anglemyer, A. T., Miller, M. L., Buttrey, S. E., & Whitaker, L. R. (2016) · Annals of Internal Medicine · doi:10.7326/m15-2785 · Cited by 34 · see citing works
Tucker, R. P., Tackett, M. J., Glickman, D. R., & Reger, M. A. (2018) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12431 · Cited by 28 · see citing works
Ban, J.-W., Emparanza, J. I., Urreta, I., & Burls, A. (2016) · PLOS ONE · doi:10.1371/journal.pone.0145779 · Cited by 18 · see citing works
Reported gastrointestinal infections in the U.S. Air Force, 2000-2012.
Cover, K. E., Ruiz, S. A., & Chapman, A. S. (2014) · PubMed · Cited by 9 · see citing works
Models of suicide risk: nice persons.
Motto, J. A., & Ag, B. (1990) · PubMed · Cited by 8 · see citing works
Flagged first, then by citations
3 of the 46 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Torous, J., Bucci, S., Bell, I. H., Kessing, L. V., Faurholt‐Jepsen, M., Whelan, P., Carvalho, A. F., Keshavan, M., Linardon, J., & Firth, J. (2021) · World Psychiatry · doi:10.1002/wps.20883 · Cited by 1,199 · see citing works
Time for united action on depression: a Lancet–World Psychiatric Association Commission
Herrman, H., Patel, V., Kieling, C., Berk, M., Buchweitz, C., Cuijpers, P., Furukawa, T. A., Kessler, R. C., Kohrt, B. A., Maj, M., et al. (2022) · The Lancet · doi:10.1016/s0140-6736(21)02141-3 · Cited by 1,104 · see citing works
Turecki, G., Brent, D. A., Gunnell, D., O’Connor, R. C., Oquendo, M. A., Pirkis, J., & Stanley, B. H. (2019) · Nature Reviews Disease Primers · doi:10.1038/s41572-019-0121-0 · Cited by 1,074 · see citing works
Supervised Machine Learning: A Brief Primer
Jiang, T., Gradus, J. L., & Rosellini, A. J. (2020) · Behavior Therapy · doi:10.1016/j.beth.2020.05.002 · Cited by 880 · see citing works
Machine learning and natural language processing in mental health: systematic review
Le Glaz, A., Haralambous, Y., Kim-Dufor, D.-H., Lenca, P., Billot, R., Ryan, T. C., Marsh, J., DeVylder, J., Walter, M., Berrouiguet, S., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/15708 · Cited by 576 · see citing works
Hossain, E., Rana, R., Higgins, N., Soar, J., Barua, P. D., Pisani, A. R., & Turner, K. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.106649 · Cited by 407 · see citing works
Candidate biomarkers in psychiatric disorders: state of the field
Abi‐Dargham, A., Moeller, S. J., Ali, F., DeLorenzo, C., Domschke, K., Horga, G., Jutla, A., Kotov, R., Paulus, M. P., Rubio, J. M., et al. (2023) · World Psychiatry · doi:10.1002/wps.21078 · Cited by 390 · see citing works
Artificial intelligence and the future of psychiatry: Insights from a global physician survey
Doraiswamy, P. M., Blease, C., & Bodner, K. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.101753 · Cited by 253 · see citing works
Nebeker, C., Torous, J., & Bartlett Ellis, R. J. (2019) · BMC Medicine · doi:10.1186/s12916-019-1377-7 · Cited by 241 · see citing works
Kessler, R. C., Bossarte, R. M., Luedtke, A., Zaslavsky, A. M., & Zubizarreta, J. R. (2020) · Molecular Psychiatry · doi:10.1038/s41380-019-0531-0 · Cited by 234 · see citing works
Su, C., Aseltine, R. H., Doshi, R., Chen, K., Rogers, S. C., & Wang, F. (2020) · Translational Psychiatry · doi:10.1038/s41398-020-01100-0 · Cited by 163 · see citing works
Nock, M. K., Millner, A. J., Ross, E. L., Kennedy, C. J., Al-Suwaidi, M., Barak-Corren, Y., Castro, V. M., Castro-Ramirez, F., Lauricella, T., Murman, N., et al. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.44373 · Cited by 157 · see citing works
The Changing Characteristics of African-American Adolescent Suicides, 2001–2017
Price, J. H., & Khubchandani, J. (2019) · Journal of Community Health · doi:10.1007/s10900-019-00678-x · Cited by 141 · see citing works
Identification of Suicide Attempt Risk Factors in a National US Survey Using Machine Learning
de la Garza, Á. G., Blanco, C., Olfson, M., & Wall, M. M. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.4165 · Cited by 139 · see citing works
Genome-Wide Association Study of Suicide Death and Polygenic Prediction of Clinical Antecedents
Docherty, A. R., Shabalin, A. A., DiBlasi, E., Monson, E., Mullins, N., Adkins, D. E., Bacanu, S.-A., Bakian, A. V., Crowell, S., Chen, D., et al. (2020) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2020.19101025 · Cited by 124 · see citing works
Gee, B. L., Han, J., Benassi, H. P., & Batterham, P. J. (2020) · Digital Health · doi:10.1177/2055207620963958 · Cited by 122 · see citing works
Walsh, C. G., Chaudhry, B., Dua, P., Goodman, K. W., Kaplan, B., Kavuluru, R., Solomonides, A., & Subbian, V. (2020) · JAMIA Open · doi:10.1093/jamiaopen/ooz054 · Cited by 118 · see citing works
Leaune, E., Samuel, M., Oh, H., Poulet, E., & Brunelin, J. (2020) · Preventive Medicine · doi:10.1016/j.ypmed.2020.106264 · Cited by 116 · see citing works
Physician Suicide: A Call to Action.
Kalmoe, M. C., Chapman, M. B., Gold, J. A., & Giedinghagen, A. M. (2019) · PubMed Central · Cited by 110 · see citing works
Singh, B., Swartz, H. A., Barboza, A. B. C., Schaffer, A., Kato, T., Dols, A., Sperry, S. H., Vassilev, A. B., Burdick, K. E., & Frye, M. A. (2025) · The Lancet · doi:10.1016/s0140-6736(25)01140-7 · Cited by 106 · see citing works
Tsui, F. R., Shi, L., Ruiz, V., Ryan, N. D., Biernesser, C. L., Iyengar, S., Walsh, C. G., & Brent, D. A. (2021) · JAMIA Open · doi:10.1093/jamiaopen/ooab011 · Cited by 105 · see citing works
Bjureberg, J., Dahlin, M., Carlborg, A., Edberg, H., Haglund, A., & Runeson, B. (2021) · Psychological Medicine · doi:10.1017/s0033291721000751 · Cited by 104 · see citing works
Barak‐Corren, Y., Castro, V. M., Nock, M. K., Mandl, K. D., Madsen, E. M., Seiger, A. N., Adams, W. G., Applegate, R. J., Bernstam, E. V., Klann, J. G., et al. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.1262 · Cited by 102 · see citing works
King, C. A., Grupp‐Phelan, J., Brent, D., Dean, J. M., Webb, M., Bridge, J. A., Spirito, A., Chernick, L. S., Mahabee‐Gittens, E. M., Mistry, R. D., et al. (2019) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13087 · Cited by 101 · see citing works
Wang, S. B., Coppersmith, D. D. L., Kleiman, E. M., Bentley, K. H., Millner, A. J., Fortgang, R., Mair, P., Dempsey, W., Huffman, J. C., & Nock, M. K. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.0591 · Cited by 98 · see citing works
Passos, I. C., Ballester, P. L., Barros, R. C., Librenza‐Garcia, D., Mwangi, B., Birmaher, B. B., Brietzke, E. M., Hájek, T., López‐Jaramillo, C., Mansur, R. B., et al. (2019) · Bipolar Disorders · doi:10.1111/bdi.12828 · Cited by 98 · see citing works
Translating promise into practice: a review of machine learning in suicide research and prevention
Kirtley, O. J., van Mens, K., Hoogendoorn, M., Kapur, N., & de Beurs, D. (2022) · The Lancet Psychiatry · doi:10.1016/s2215-0366(21)00254-6 · Cited by 100 · see citing works
Kusuma, K., Larsen, M. E., Quiroz, J. C., Gillies, M. B., Burnett, A., Qian, J., & Torok, M. H. (2022) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2022.09.050 · Cited by 90 · see citing works
Levis, M., Leonard Westgate, C., Gui, J., Watts, B. V., & Shiner, B. (2021) · Psychological Medicine · doi:10.1017/s0033291720000173 · Cited by 87 · see citing works
How accurate are suicide risk prediction models? asking the right questions for clinical practice.
Whiting, D., & Fazel, S. (2019) · Evidence Based Mental Health · doi:10.1136/ebmental-2019-300102 · Cited by 83 · see citing works
Genetic contributions to suicidal thoughts and behaviors
DiBlasi, E., Kang, J., & Docherty, A. R. (2021) · Psychological Medicine · doi:10.1017/s0033291721001720 · Cited by 83 · see citing works
Chen, Q., Zhang‐James, Y., Barnett, E. J., Lichtenstein, P., Jokinen, J., D'Onofrio, B. M., Faraone, S. V., Larsson, H., & Fazel, S. (2020) · PLoS Medicine · doi:10.1371/journal.pmed.1003416 · Cited by 82 · see citing works
Schafer, K. M., Duffy, M. E., Kennedy, G. A., Stentz, L. A., Leon, J., Herrerias, G., Fulcher, S., & Joiner, T. E. (2021) · Military Psychology · doi:10.1080/08995605.2021.1976544 · Cited by 80 · see citing works
National Trends of Mental Health Care Among US Adults Who Attempted Suicide in the Past 12 Months
Bommersbach, T. J., Rosenheck, R. A., & Rhee, T. G. (2022) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2021.3958 · Cited by 80 · see citing works
Prospective Validation of an Electronic Health Record–Based, Real-Time Suicide Risk Model
Walsh, C. G., Johnson, K. B., Ripperger, M., Sperry, S., Harris, J., Clark, N., Fielstein, E., Novak, L., Robinson, K., & Stead, W. W. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.1428 · Cited by 78 · see citing works
A direct comparison of theory-driven and machine learning prediction of suicide: A meta-analysis
Schafer, K. M., Kennedy, G. A., Gallyer, A. J., & Resnik, P. (2021) · PLoS ONE · doi:10.1371/journal.pone.0249833 · Cited by 78 · see citing works
Conceptualising fairness: three pillars for medical algorithms and health equity
Sikstrom, L., Maslej, M. M., Hui, K., Findlay, Z., Buchman, D. Z., & Hill, S. L. (2022) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2021-100459 · Cited by 76 · see citing works
Cusick, M. M., Adekkanattu, P. M., Campion, T. R., Sholle, E., Myers, A., Banerjee, S., Alexopoulos, G. S., Wang, Y., & Pathak, J. (2021) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2021.01.052 · Cited by 74 · see citing works
Machine learning-based prediction for self-harm and suicide attempts in adolescents
Su, R., John, J. R., & Lin, P. (2023) · Psychiatry Research · doi:10.1016/j.psychres.2023.115446 · Cited by 72 · see citing works
Psychiatry in the Digital Age: A Blessing or a Curse?
Roth, C. B., Papassotiropoulos, A., Brühl, A. B., Lang, U. E., & Huber, C. G. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18168302 · Cited by 71 · see citing works
Ehtemam, H., Esfahlani, S. S., Sanaei, A., Ghaemi, M. M., Hajesmaeel‐Gohari, S., Rahimisadegh, R., Bahaadinbeigy, K., Ghasemian, F., & Shirvani, H. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02524-0 · Cited by 69 · see citing works
Kleiman, E. M., Glenn, C. R., & Liu, R. T. (2019) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2019.1666400 · Cited by 68 · see citing works
Miché, M., Studerus, E., Meyer, A., Gloster, A., Beesdo-Baum, K., Wittchen, H., & Lieb, R. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.11.093 · Cited by 62 · see citing works
Acoustic and language analysis of speech for suicidal ideation among US veterans
Belouali, A., Gupta, S., Sourirajan, V., Yu, J., Allen, N. J., Alaoui, A., Dutton, M. A., & Reinhard, M. J. (2021) · BioData Mining · doi:10.1186/s13040-021-00245-y · Cited by 67 · see citing works
Ross, E. L., Zuromski, K. L., Reis, B. Y., Nock, M. K., Kessler, R. C., & Smoller, J. W. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2021.0089 · Cited by 65 · see citing works
Annual Research Review: Translational machine learning for child and adolescent psychiatry
Dwyer, D., & Koutsouleris, N. (2022) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13545 · Cited by 65 · see citing works
Can machine-learning methods really help predict suicide?
McHugh, C. M., & Large, M. M. B. (2020) · Current Opinion in Psychiatry · doi:10.1097/yco.0000000000000609 · Cited by 64 · see citing works
The use of advanced technology and statistical methods to predict and prevent suicide
Kleiman, E. M., Glenn, C. R., & Liu, R. T. (2023) · Nature Reviews Psychology · doi:10.1038/s44159-023-00175-y · Cited by 63 · see citing works
Chiauzzi, E. J., Clayton, A. R., & Huh, J. (2020) · JMIR Mental Health · doi:10.2196/24021 · Cited by 62 · see citing works
Meta-analysis of the strength of exploratory suicide prediction models; from clinicians to computers
Corke, M., Mullin, K., Angel-Scott, H. N., Xia, S., & Large, M. M. B. (2021) · BJPsych Open · doi:10.1192/bjo.2020.162 · Cited by 57 · see citing works
Expectations for Artificial Intelligence (AI) in Psychiatry
Monteith, S., Glenn, T., Geddes, J. R., Whybrow, P. C., Achtyes, E. D., & Bauer, M. (2022) · Current Psychiatry Reports · doi:10.1007/s11920-022-01378-5 · Cited by 56 · see citing works
Evidence of Inflated Prediction Performance: A Commentary on Machine Learning and Suicide Research
Jacobucci, R., Littlefield, A. K., Millner, A. J., Kleiman, E. M., & Steinley, D. (2021) · Clinical Psychological Science · doi:10.1177/2167702620954216 · Cited by 56 · see citing works
Martínez‐Alés, G., Hernández‐Calle, D., Khauli, N., & Keyes, K. M. (2020) · Current topics in behavioral neurosciences · doi:10.1007/7854_2020_158 · Cited by 54 · see citing works
Suicide in the pediatric population: screening, risk assessment and treatment
Cwik, M. F., O’Keefe, V. M., & Haroz, E. E. (2020) · International Review of Psychiatry · doi:10.1080/09540261.2019.1693351 · Cited by 54 · see citing works
Integrating a functional view on suicide risk into idiographic statistical models
Kaurin, A., Dombrovski, A. Y., Hallquist, M. N., & Wright, A. G. C. (2021) · Behaviour Research and Therapy · doi:10.1016/j.brat.2021.104012 · Cited by 53 · see citing works
van Eeden, W. A., Luo, C. W., van Hemert, A. M., Carlier, I. V. E., Penninx, B. W. J. H., Wardenaar, K. J., Hoos, H. H., & Giltay, E. J. (2021) · Psychiatry Research · doi:10.1016/j.psychres.2021.113823 · Cited by 52 · see citing works
Interpersonal theory of suicide: prospective examination
Forkmann, T., Glaesmer, H., Paashaus, L., Rath, D., Schönfelder, A., Stengler, K., Juckel, G., Assion, H., & Teismann, T. (2020) · BJPsych Open · doi:10.1192/bjo.2020.93 · Cited by 51 · see citing works
Wilimitis, D., Turer, R. W., Ripperger, M. A., McCoy, A. B., Sperry, S. H., Fielstein, E. M., Kurz, T. L., & Walsh, C. G. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.12095 · Cited by 51 · see citing works
Baldaçara, L., Rocha, G. A., Leite, V. D. S., Porto, D. M., Grüdtner, R. R., Díaz, A. P., Meleiro, A. M. A. D. S., Corrêa, H., Tung, T. C., Quevedo, J., et al. (2020) · Brazilian Journal of Psychiatry · doi:10.1590/1516-4446-2020-0994 · Cited by 49 · see citing works
Eskin, M., Baydar, N., Harlak, H., Hamdan, M., Mechri, A., Isayeva, U., Abdel‐Khalek, A. M., Rezaeian, M., Asad, N., El-Nayal, M., et al. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.07.050 · Cited by 47 · see citing works
Haroz, E. E., Kitchen, C., Nestadt, P. S., Wilcox, H. C., DeVylder, J., & Kharrazi, H. H. K. (2021) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12800 · Cited by 47 · see citing works
Vidal-Ribas, P., Janiri, D., Doucet, G. E., Pornpattananangkul, N., Nielson, D. M., Frangou, S., & Stringaris, A. (2021) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2020.20020120 · Cited by 45 · see citing works
Reconciling Statistical and Clinicians’ Predictions of Suicide Risk
Simon, G. E., Matarazzo, B. B., Walsh, C. G., Smoller, J. W., Boudreaux, E. D., Yarborough, B. J. H., Shortreed, S. M., Coley, R. Y., Ahmedani, B. K., Doshi, R., et al. (2021) · Psychiatric Services · doi:10.1176/appi.ps.202000214 · Cited by 44 · see citing works
Agne, N. A., Tisott, C. G., Ballester, P. L., Passos, I. C., & Ferrão, Y. A. (2020) · Psychological Medicine · doi:10.1017/s0033291720002329 · Cited by 44 · see citing works
van Vuuren, C. L., van Mens, K., Beurs, D. P. D., Lokkerbol, J., van der Wal, M. F., Cuijpers, P. J., & Chinapaw, M. J. M. (2021) · Journal of Affective Disorders · doi:10.1016/j.jad.2021.09.018 · Cited by 44 · see citing works
Analysis and evaluation of explainable artificial intelligence on suicide risk assessment
Hao Tang, A., Rekavandi, A. M., Rooprai, D., Dwivedi, G., Sanfilippo, F. M., Boussaïd, F., & Bennamoun, M. (2024) · Scientific Reports · doi:10.1038/s41598-024-53426-0 · Cited by 42 · see citing works
Konttinen, H. M., Sjöholm, K., Jacobson, P., Svensson, P., Carlsson, L. M. S., & Peltonen, M. (2019) · Annals of Surgery · doi:10.1097/sla.0000000000003742 · Cited by 42 · see citing works
Iorfino, F., Ho, N., Carpenter, J. S., Cross, S. P. M., Davenport, T. A., Hermens, D. F., Yee, H. F., Nichles, A., Zmicerevska, N., Guastella, A. J., et al. (2020) · PLoS ONE · doi:10.1371/journal.pone.0243467 · Cited by 42 · see citing works
Clinical epidemiological research on suicide-related behaviors—where we are and where we need to go
Kessler, R. C. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.1238 · Cited by 41 · see citing works
Firearm Suicides in the Elderly: A Narrative Review and Call for Action
Price, J. H., & Khubchandani, J. (2021) · Journal of Community Health · doi:10.1007/s10900-021-00964-7 · Cited by 41 · see citing works
Morgiève, M., Genty, C., Azé, J., Dubois, J., Leboyer, M., Vaïva, G., Berrouiguet, S., & Courtet, P. (2020) · JMIR mhealth and uhealth · doi:10.2196/15741 · Cited by 40 · see citing works
Clary, K. L., Pena, S., & Smith, D. C. (2021) · Current Psychology · doi:10.1007/s12144-021-01768-7 · Cited by 39 · see citing works
Spittal, M. J., Guo, X. A., Kang, L., Kirtley, O. J., Clapperton, A., Hawton, K. E., Kapur, N., Pirkis, J., & Carter, G. L. (2025) · PLoS Medicine · doi:10.1371/journal.pmed.1004581 · Cited by 39 · see citing works
Machine learning for suicidology: A practical review of exploratory and hypothesis-driven approaches
Cox, C. R., Moscardini, E. H., Cohen, A. S., & Tucker, R. P. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101940 · Cited by 39 · see citing works
Torous, J. B., & Walker, R. L. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.1231 · Cited by 38 · see citing works
A comparative study of machine learning techniques for suicide attempts predictive model
Nordin, N., Zainol, Z. B., Noor, M. H. M., & Fong, C. L. (2021) · Health Informatics Journal · doi:10.1177/1460458221989395 · Cited by 38 · see citing works
Martínez‐Alés, G., & Keyes, K. M. (2019) · Current Psychiatry Reports · doi:10.1007/s11920-019-1080-6 · Cited by 37 · see citing works
Gardner, W. P., PAJER, K. A., Cloutier, P., Currie, L. M., Colman, I., Zemek, R. L., Hatcher, S., Lima, I. S. F., & Cappelli, M. (2019) · Canadian Medical Association Journal · doi:10.1503/cmaj.190188 · Cited by 35 · see citing works
Torous, J. B., Choudhury, T., Barnett, I., Keshavan, M. S., & Kane, J. M. (2020) · World Psychiatry · doi:10.1002/wps.20805 · Cited by 35 · see citing works
Steine, I. M., Nielsen, B., Porter, P. A., Krystal, J. H., Winje, D., Grønli, J., Milde, A. M., Bjorvatn, B., Nordhus, I. H., & Pallesen, S. (2020) · European Journal of Psychotraumatology · doi:10.1080/20008198.2020.1815282 · Cited by 35 · see citing works
Development of a Suicide Prediction Model for the Elderly Using Health Screening Data
Cho, S.-E., Geem, Z., & Na, K.-S. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph181910150 · Cited by 32 · see citing works
A review and meta-analysis of gene expression profiles in suicide
Piras, I. S., Huentelman, M. J., Pinna, F., Paribello, P., Solmi, M., Murru, A., Carpiniello, B., Manchia, M., & Zai, C. C. H. (2021) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2021.12.003 · Cited by 35 · see citing works
Joo, Y. Y., Moon, S., Wang, H., Kim, H., Lee, E., Kim, J. H., Posner, J. E., Ahn, W., Choi, I., Kim, J.-W., et al. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.48585 · Cited by 35 · see citing works
The Veterans Health Administration REACH VET Program: Suicide Predictive Modeling in Practice
Matarazzo, B. B., Eagan, A. E., Landes, S. J., Mina, L. K., Clark, K., Gerard, G. R., McCarthy, J. F., Trafton, J. A., Bahraini, N. H., Brenner, L. A., et al. (2022) · Psychiatric Services · doi:10.1176/appi.ps.202100629 · Cited by 34 · see citing works
Averill, L. A., Averill, C. L., Gueorguieva, R., Fouda, S., Sherif, M., Ahn, K.-H., Ranganathan, M., D'Souza, D. C., Southwick, S. M., Sanacora, G., et al. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.01.104 · Cited by 33 · see citing works
Gifuni, A. J., Chakravarty, M. M., Lepage, M., Ho, T. C., Geoffroy, M.-C., Lacourse, E., Gotlib, I. H., Turecki, G., Renaud, J., & Jollant, F. (2021) · Journal of Psychiatry and Neuroscience · doi:10.1503/jpn.200198 · Cited by 33 · see citing works
Bentley, K. H., Zuromski, K. L., Fortgang, R. G., Madsen, E., Kessler, D. T., Lee, H., Nock, M. K., Reis, B. Y., Castro, V. M., & Smoller, J. W. (2022) · JMIR Formative Research · doi:10.2196/30946 · Cited by 33 · see citing works
What is bipolar disorder? A disease model of dysregulated energy expenditure
Mansur, R. B., Lee, Y. L., McIntyre, R. S., & Brietzke, E. M. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.04.006 · Cited by 33 · see citing works
Parghi, N., Chennapragada, L., Barzilay, S., Newkirk, S., Ahmedani, B. K., Lok, B. C. L., & GALYNKER, I. I. (2020) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1863 · Cited by 32 · see citing works
Machado, C. D. S., Ballester, P. L., Cao, B., Mwangi, B., Caldieraro, M. A., Kapczinski, F. P., & Passos, I. C. (2021) · Psychological Medicine · doi:10.1017/s0033291720004997 · Cited by 32 · see citing works
van Velzen, L. S., Toenders, Y. J., Àvila-Parcet, A., Dinga, R., Rabinowitz, J. A., Campos, A. I., Jahanshad, N., Rentería, M. E., & Schmaal, L. (2022) · The British Journal of Psychiatry · doi:10.1192/bjp.2022.7 · Cited by 32 · see citing works
A Review of Suicide Risk Assessment Tools and Their Measured Psychometric Properties in Korea
Baek, I., Jo, S., Kim, E. J., Lee, G. R., Lee, D. H., & Jeon, H. J. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.679779 · Cited by 31 · see citing works
Suicide Crisis Syndrome: A systematic review
Melzer, L., Forkmann, T., & Teismann, T. (2024) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.13065 · Cited by 30 · see citing works
Ćosić, K., Popović, S., & Wiederhold, B. K. (2024) · Cyberpsychology Behavior and Social Networking · doi:10.1089/cyber.2023.0737 · Cited by 30 · see citing works
Liu, L., Li, Z., Hu, Y., Li, C., He, S., Zhang, S., Gao, J., Zhu, H., & Huang, G. (2025) · Journal of Medical Internet Research · doi:10.2196/73052 · Cited by 30 · see citing works
Levis, M., Levy, J., Dufort, V., Gobbel, G. T., Watts, B. V., & Shiner, B. R. (2022) · Psychiatry Research · doi:10.1016/j.psychres.2022.114703 · Cited by 30 · see citing works
Wu, C., Lee, M., Liao, S., Chan, C., Liu, L.-Y. D., & Chen, C.-Y. (2020) · Journal of the Formosan Medical Association · doi:10.1016/j.jfma.2020.04.033 · Cited by 29 · see citing works
Gioia, A. N., Forrest, L. N., & Smith, A. R. (2022) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12915 · Cited by 29 · see citing works
Machine learning to advance the prediction, prevention and treatment of eating disorders
Wang, S. B. (2021) · European Eating Disorders Review · doi:10.1002/erv.2850 · Cited by 29 · see citing works
Bryan, C. J., Rozek, D. C., & Khazem, L. R. (2019) · Crisis · doi:10.1027/0227-5910/a000639 · Cited by 28 · see citing works
- Authors
- Belsher, B. E., Smolenski, D. J., Pruitt, L. D., Bush, N. E., Beech, E. H., Workman, D. E., Morgan, R. L., Evatt, D. P., Tucker, J., & Skopp, N. A.
- Affiliations
- Uniformed Services University of the Health Sciences (US)
- Defense Health Agency (US)
- McMaster University (CA)
- ORCID
- Venue
- JAMA Psychiatry
- Publisher
- American Medical Association (AMA)
- Published
- 2019-06-01
- Volume
- 76
- Issue
- 6
- Pages
- 642
- ISSN
- 2168-622X (ISSN-L)
- OpenAlex
- W2921616123
- Cited by
- 540
- References
- 46
- Citations by year
- 2026: 81 · 2025: 73 · 2024: 63 · 2023: 63 · 2022: 82 · 2021: 83 · 2020: 64 · 2019: 31
- FWCI
- 10.24
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- JAMA Psychiatry (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Suicide and Self-Harm Studies
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- suicide attempts, suicide mortality, diagnostic accuracy, predictive validity, positive predictive value, risk prediction, clinical implementation, health care system
- MeSH terms
- Algorithms, Humans, Models, Theoretical, Suicide, Suicide, Attempted, Suicidal Ideation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
