The DOI matches its Crossref record, with a small difference in title or year; 43 of the 46 references it cites were found in a registry or an index.
Prediction Models for Suicide Attempts and Deaths: A Systematic Review and Simulation
Abstract
Importance: Suicide prediction models have the potential to improve the identification of patients at heightened suicide risk by using predictive algorithms on large-scale data sources. Suicide prediction models are being developed for use across enterprise-level health care systems including the US Department of Defense, US Department of Veterans Affairs, and Kaiser Permanente. Objectives: To evaluate the diagnostic accuracy of suicide prediction models in predicting suicide and suicide attempts and to simulate the effects of implementing suicide prediction models using population-level estimates of suicide rates. Evidence Review: A systematic literature search was conducted in MEDLINE, PsycINFO, Embase, and the Cochrane Library to identify research evaluating the predictive accuracy of suicide prediction models in identifying patients at high risk for a suicide attempt or death by suicide. Each database was searched from inception to August 21, 2018. The search strategy included search terms for suicidal behavior, risk prediction, and predictive modeling. Reference lists of included studies were also screened. Two reviewers independently screened and evaluated eligible studies. Findings: From a total of 7306 abstracts reviewed, 17 cohort studies met the inclusion criteria, representing 64 unique prediction models across 5 countries with more than 14 million participants. The research quality of the included studies was generally high. Global classification accuracy was good (≥0.80 in most models), while the predictive validity associated with a positive result for suicide mortality was extremely low (≤0.01 in most models). Simulations of the results suggest very low positive predictive values across a variety of population assessment characteristics. Conclusions and Relevance: To date, suicide prediction models produce accurate overall classification models, but their accuracy of predicting a future event is near 0. Several critical concerns remain unaddressed, precluding their readiness for clinical applications across health systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Even complex risk assessment models do not allow for precise prediction of suicidal behavior (26, 27)”
From Suicide risk assessment: clinical implications of the unpredictability of suicidal behavior (2026) · cites this paper as [27]
In the citing paper
“and, in particular, they do not allow for prediction of the time at which suicidal behavior will occur.”
From Suicide risk assessment: clinical implications of the unpredictability of suicidal behavior (2026) · cites this paper as [27]
In the citing paper
“In principle, it is important to acknowledge that events with a low base rate, such as suicide attempts and suicides, can never be precisely predicted-no matter which assessments and methods are used (Belsher et al., 2019;Bryan, 2021).”
From Suicide Crisis Syndrome: A systematic review (2024) · cites this paper as Belsher et al., 2019
In the citing paper
“most models had a PPV <1% for predicting suicide mortality”
From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [10]
In the citing paper
“multiple research groups have developed suicide risk prediction models using electronic health record data.10,11,12,13”
From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [10]
In the citing paper
“A recent systematic review10 identified 17 such predictive models that have been described in the literature.”
From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [10]
In the citing paper
“A 2019 review of 64 longitudinal suicide prediction models36 found most models reported extremely low (ie, <.01) PPVs, such that 99 out of every 100 individuals predicted by these models to attempt suicide will not.”
From A pilot study using frequent inpatient assessments of suicidal thinking to predict short-term postdischarge suicidal behavior (2021) · cites this paper as [36]
In the citing paper
“The dynamic feature model predicted posthospital suicide attempts with good accuracy (AUC, 0.89), which is at the high end of the range of reported AUC values for longitudinal prediction models of suicidal behaviors.36”
From A pilot study using frequent inpatient assessments of suicidal thinking to predict short-term postdischarge suicidal behavior (2021) · cites this paper as [36]
In the citing paper
“In comparison, the average cross-validated PPV for our dynamic feature model was 0.39. This represents a substantial improvement beyond both previous work and our baseline and mean models”
From A pilot study using frequent inpatient assessments of suicidal thinking to predict short-term postdischarge suicidal behavior (2021) · cites this paper as [36]
In the citing paper
“Although machine learning methods have yielded high AUCs, the PPV of most models has been low.”
From A pilot study using frequent inpatient assessments of suicidal thinking to predict short-term postdischarge suicidal behavior (2021) · cites this paper as [36]
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research
Franklin, J. C., Ribeiro, J. D., Fox, K. R., Bentley, K. H., Kleiman, E. M., Huang, X., Musacchio, K. M., Jaroszewski, A. C., Chang, B. P., & Nock, M. K. (2017) · Psychological Bulletin · doi:10.1037/bul0000084 · Cited by 3,784
Wang, Y., Kung, L., & Byrd, T. A. (2018) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2015.12.019 · Cited by 1,571
Prediction of Suicide in Psychiatric Patients
Pokorny, A. D. (1983) · Archives of General Psychiatry · doi:10.1001/archpsyc.1983.01790030019002 · Cited by 749
Predicting Risk of Suicide Attempts Over Time Through Machine Learning
Walsh, C. G., Ribeiro, J. D., & Franklin, J. C. (2017) · Clinical Psychological Science · doi:10.1177/2167702617691560 · Cited by 571
Uses and abuses of screening tests
Grimes, D. A., & Schulz, K. F. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07948-5 · Cited by 545
Kessler, R. C., Warner, C. H., Ivany, C., Petukhova, M. V., Rose, S., Bromet, E. J., Brown, M., Cai, T., Colpe, L. J., Cox, K. L., et al. (2015) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2014.1754 · Cited by 447
Simon, G. E., Johnson, E., Lawrence, J. M., Rossom, R. C., Ahmedani, B., Lynch, F. L., Beck, A., Waitzfelder, B., Ziebell, R., Penfold, R. B., et al. (2018) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2018.17101167 · Cited by 400
Flagged first, then by citations
3 of the 46 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- 2018, Supporting our veterans during their transition from uniformed service to civilian life., Federal Register, 83, 1513
- Pruitt, 2017, Department of Defense Suicide Event Report: Calendar Year 2016 Annual Report
- 2018, VA National Suicide Data Report: 2005-2015
- 2018, Joint Action Plan for Supporting Veterans During Their Transition From Uniformed Service to Civilian Life
- VA REACH VET initiative helps save veterans lives: program signals when more help is needed for at-risk veterans [press release]. Washington DC: US Department of Veterans Affairs; April 3, 2017.
Cited by
Sorted by citations
Items 501–542 of 542
de Filippis, R., & Foysal, A. A. (2026) · OALib · doi:10.4236/oalib.1115138
Kawabe, K., MATSUMOTO, Y., Ihara, Y., & Horiuchi, F. (2026) · Psychiatry and Clinical Neurosciences Reports · doi:10.1002/pcn5.70353
Shao, W., Ammerman, B. A., & Jacobucci, R. (2026) · medRxiv · doi:10.64898/2026.06.04.26354928
Kimbrel, N. A., Dennis, P. A., Patel, T. A., Halverson, T. F., Dillon, K. H., Stephenson, M. E., Bourassa, K. J., Patel, P. R., Qin, X. J., Barr, P. B., et al. (2026) · Archives of Suicide Research · doi:10.1080/13811118.2026.2683477
Artificial intelligence in Australian prison psychiatry: Promise, pitfalls, and ethical imperatives
Aga, F. (2026) · Australasian Psychiatry · doi:10.1177/10398562261464441
Metts, A. V., Creech, S. K., Leri, J., Dodd, C. G., Newport, D. J., Shahidullah, J. D., Rousseau, J. F., Foshee, K. L., Richmond, R., Garrett, A. S., et al. (2026) · Journal of Affective Disorders · doi:10.1016/j.jad.2026.122170
Noyon, A., Riedel, A., & Heidenreich, T. (2026) · Springer eBooks · doi:10.1007/978-3-662-72796-6_6
Li, B., Li, M., Li, M., Wang, X., Wang, X., Li, J., Chen, Z., Sun, L., Zhang, X., & Zhang, X. (2026) · Scientific Reports · doi:10.1038/s41598-026-59370-5
Orbán, L. L., Assaf, M., Williams, L., Mayer, Q., & Sharma, B. (2026) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro70003.2026.11591659
Rodríguez, M. D. O., Apellániz, J. S., Coronel, C. P., García, M. A. V., Álvarez-García, R., Córdoba, R., Pfang, B., & Baca-García, E. (2026) · BJPsych International · doi:10.1192/bji.2026.10111
Huang, S., Goodin, A. J., Fairless, J. A., Guo, J. S., Bian, J., Xu, J., & Jiao, T. (2026) · Depression and Anxiety · doi:10.1155/da/5305506
Large language models in adolescent suicide prevention: from language signals to accountable action
Li, M., Huang, W., Wu, J., Kuang, N., Luo, Y., Huang, J., & Huang, N. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1872450
Ciobanu, A. M., Tebeanu, A. V., Tau, V., Chendea, A. A., Franti, E. D., Niculae, C., Vasile, C. I., Macarie, G. F., Neagu, L., Dascălu, M., et al. (2026) · Psychiatry International · doi:10.3390/psychiatryint7040174
Machine Learning Prediction of First Suicide Attempts in Early Adolescence Among Child Ideators
Nguyen, J., Dwyer, D., Tagliaferri, S. D., Hartmann, S., Clark, S. R., Scott, I., Wigman, J. T. W., Lin, A. P., Thompson, A. D., Gao, C. X., et al. (2026) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2026.07.1245
Tison, M., Zagury, J., Leboyer, M., & Tamouza, R. (2025) · Elsevier eBooks · doi:10.1016/b978-2-294-78794-2.00011-7
Li, X., Xiang, S.-T., & Dong, J. (2026) · BMC Public Health · doi:10.1186/s12889-026-28952-3
Shafqat, A., & Park, E. (2026) · Acta Psychologica · doi:10.1016/j.actpsy.2026.107579
Assessing the Utility of Social Determinants of Health Data in Suicide Prediction Models
Castner, M. D., Kitchen, C., Xiong, C., Bittle, M. J., Nestadt, P. S., Wilcox, H. C., & Kharrazi, H. H. K. (2026) · Archives of Suicide Research · doi:10.1080/13811118.2026.2722144
Suicidology’s Faulty Paradigm: Suicide Overrides Superordinate Impetus for Survival (“SOSIS”)
Soper, C. A. (2026) · Evolutionary Psychological Science · doi:10.1007/s40806-026-00469-3
Cao, Q., Xu, J., Li, W., Zhang, Y., Huang, X., Zhou, K., Zhang, J., & Jiang, L. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15176719
Fu, C., Su, D., Zhang, J., Pei, S., & Ma, S. (2026) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2026.105143
Damiani, S., Marika, O., Provenzani, U., Bortolin, E., Rovida, M., Stefanelli, R., Addington, J., Alameda, L., Arango, C., Breitborde, N. J. K., et al. (2026) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.70138
Saha, S., Singh, M. K., & Choudhary, P. K. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-9109-0.ch005
Zhang, M., Xia, Y., Mo, H., Zhu, Y., Fang, L., Xiang, Q., Shen, J., Wang, L., & Lou, J. (2026) · Epidemiologic Reviews · doi:10.1093/epirev/mxag013
A machine learning framework for the multi-class prediction of suicide attempt risk
Nordin, N., Noor, M. H. M., Zainol, Z. B., Chan, L. F., & Buji, R. I. (2026) · Acta Psychologica · doi:10.1016/j.actpsy.2026.107815
Human-Centered AI in Crisis Counseling: A Three-Tier Governance FrameworkPreprint
Thangaraju, A. (2026) · Research Square · doi:10.21203/rs.3.rs-11148168/v1
Predicting suicide attempts in people with autism and co-morbid mood or psychotic disordersPreprint
Baker, M. J., Lam, W. Y., Virtosu, M., & de Lacy, N. (2026) · medRxiv · doi:10.64898/2026.09.25.26364048
Pérez-Barrero, A., Pérez‐Esteban, A., Díez-Gómez, A., Pérez‐Albéniz, A., Al‐Halabí, S., Lucas‐Molina, B., Inchausti-Gómez, F., & Fonseca‐Pedrero, E. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1918211
Ward, E. J., Bentley, K. H., Dworkin, E. R., Hawrilenko, M., Ark, E. V., DeLorenzo, J., Brown, M. D., & Chekroud, A. M. (2026) · Research Square · doi:10.21203/rs.3.rs-10882429/v1
Belouali, A., Kitchen, C., Haroz, E. E., Lehmann, H. P., Nestadt, P. S., Wilcox, H. C., & Kharrazi, H. H. K. (2026) · BioData Mining · doi:10.1186/s13040-026-00608-3
Liang, S., Jiang, Q., Zhang, J., Deng, Y., Xu, Y., Zhao, G., Liu, X., Xing, Y., Yu, H., Hu, C., et al. (2026) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2026.1950186
From discovery to proof: a Translation Evidence Map for precision psychiatry
Baune, B. T. (2026) · The Lancet Psychiatry · doi:10.1016/s2215-0366(26)00267-1
Dayal, R. R., Goklish, N. A., Adams, R. J., Rebman, P., Suttle, R., Garcia, M., Mega, J., de la Sierra, V. Q., & Haroz, E. E. (2026) · medRxiv · doi:10.64898/2026.10.05.26364799
Yu, H., Wang, H., Gong, K., Wang, L., Lu, Y., Wang, Y., Zhang, X., Huang, C., Liang, J., Gao, L., et al. (2026) · Journal of Affective Disorders · doi:10.1016/j.jad.2026.122600
Rossom, R. C., Johnson, E. A., Papini, S., Shortreed, S. M., Coleman, K. J., Ziebell, R. A., Ahmedani, B. K., Lynch, F. L., Beck, A. L., Daida, Y. G., et al. (2026) · Psychiatric Services · doi:10.1176/appi.ps.20250697
Omigbodun, F. T. (2026) · Journal of Medicine Surgery and Public Health · doi:10.1016/j.glmedi.2026.100258
Tatlı, M. (2026) · Özgür Yayınları eBooks · doi:10.58830/ozgur.pub1519.c6126
Accompaniment Planning: Re-sequencing Suicide-Related Care in ClinicalSocial Work PracticePreprint
Hightower, H. H. (2026) · Research Square · doi:10.21203/rs.3.rs-11180341/v1
Anxiety and suicide risk: mechanism, timing, and clinical utility
Allan, N. P., Ko, H., & Cho, G. Y. (2026) · Current Opinion in Psychiatry · doi:10.1097/yco.0000000000001132
Kozłowska, M. J., Majchrzak, M. K., Kołodziej, F. M., Kwiatkowska, A., Juskowiak, J., Lis, M., Komin, Z. J., Liasota, M., Kabat, E. Z., & Konopka, K. (2026) · International Journal of Innovative Technologies in Social Science · doi:10.31435/ijitss.4(52).2026.6396
Salma, S. U., Sterling, E., & Sakib, N. (2026) · Studies in big data · doi:10.1007/978-3-032-23522-0_10
Metadata
- Authors
- Belsher, B. E., Smolenski, D. J., Pruitt, L. D., Bush, N. E., Beech, E. H., Workman, D. E., Morgan, R. L., Evatt, D. P., Tucker, J., & Skopp, N. A.
- Affiliations
- Uniformed Services University of the Health Sciences (US)
- Defense Health Agency (US)
- McMaster University (CA)
- ORCID
- Venue
- JAMA Psychiatry
- Publisher
- American Medical Association (AMA)
- Published
- 2019-06-01
- Volume
- 76
- Issue
- 6
- Pages
- 642
- ISSN
- 2168-622X (ISSN-L)
- OpenAlex
- W2921616123
- Cited by
- 540
- References
- 46
- Citations by year
- 2026: 81 · 2025: 73 · 2024: 63 · 2023: 63 · 2022: 82 · 2021: 83 · 2020: 64 · 2019: 31
- FWCI
- 10.24
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- JAMA Psychiatry (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Suicide and Self-Harm Studies
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- suicide attempts, suicide mortality, diagnostic accuracy, predictive validity, positive predictive value, risk prediction, clinical implementation, health care system
- MeSH terms
- Algorithms, Humans, Models, Theoretical, Suicide, Suicide, Attempted, Suicidal Ideation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
