The DOI matches its Crossref record, with a small difference in title or year; 43 of the 46 references it cites were found in a registry or an index.
Prediction Models for Suicide Attempts and Deaths: A Systematic Review and Simulation
Abstract
Importance: Suicide prediction models have the potential to improve the identification of patients at heightened suicide risk by using predictive algorithms on large-scale data sources. Suicide prediction models are being developed for use across enterprise-level health care systems including the US Department of Defense, US Department of Veterans Affairs, and Kaiser Permanente. Objectives: To evaluate the diagnostic accuracy of suicide prediction models in predicting suicide and suicide attempts and to simulate the effects of implementing suicide prediction models using population-level estimates of suicide rates. Evidence Review: A systematic literature search was conducted in MEDLINE, PsycINFO, Embase, and the Cochrane Library to identify research evaluating the predictive accuracy of suicide prediction models in identifying patients at high risk for a suicide attempt or death by suicide. Each database was searched from inception to August 21, 2018. The search strategy included search terms for suicidal behavior, risk prediction, and predictive modeling. Reference lists of included studies were also screened. Two reviewers independently screened and evaluated eligible studies. Findings: From a total of 7306 abstracts reviewed, 17 cohort studies met the inclusion criteria, representing 64 unique prediction models across 5 countries with more than 14 million participants. The research quality of the included studies was generally high. Global classification accuracy was good (≥0.80 in most models), while the predictive validity associated with a positive result for suicide mortality was extremely low (≤0.01 in most models). Simulations of the results suggest very low positive predictive values across a variety of population assessment characteristics. Conclusions and Relevance: To date, suicide prediction models produce accurate overall classification models, but their accuracy of predicting a future event is near 0. Several critical concerns remain unaddressed, precluding their readiness for clinical applications across health systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Even complex risk assessment models do not allow for precise prediction of suicidal behavior (26, 27)”
From Suicide risk assessment: clinical implications of the unpredictability of suicidal behavior (2026) · cites this paper as [27]
In the citing paper
“and, in particular, they do not allow for prediction of the time at which suicidal behavior will occur.”
From Suicide risk assessment: clinical implications of the unpredictability of suicidal behavior (2026) · cites this paper as [27]
In the citing paper
“In principle, it is important to acknowledge that events with a low base rate, such as suicide attempts and suicides, can never be precisely predicted-no matter which assessments and methods are used (Belsher et al., 2019;Bryan, 2021).”
From Suicide Crisis Syndrome: A systematic review (2024) · cites this paper as Belsher et al., 2019
In the citing paper
“most models had a PPV <1% for predicting suicide mortality”
From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [10]
In the citing paper
“multiple research groups have developed suicide risk prediction models using electronic health record data.10,11,12,13”
From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [10]
In the citing paper
“A recent systematic review10 identified 17 such predictive models that have been described in the literature.”
From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [10]
In the citing paper
“A 2019 review of 64 longitudinal suicide prediction models36 found most models reported extremely low (ie, <.01) PPVs, such that 99 out of every 100 individuals predicted by these models to attempt suicide will not.”
From A pilot study using frequent inpatient assessments of suicidal thinking to predict short-term postdischarge suicidal behavior (2021) · cites this paper as [36]
In the citing paper
“The dynamic feature model predicted posthospital suicide attempts with good accuracy (AUC, 0.89), which is at the high end of the range of reported AUC values for longitudinal prediction models of suicidal behaviors.36”
From A pilot study using frequent inpatient assessments of suicidal thinking to predict short-term postdischarge suicidal behavior (2021) · cites this paper as [36]
In the citing paper
“In comparison, the average cross-validated PPV for our dynamic feature model was 0.39. This represents a substantial improvement beyond both previous work and our baseline and mean models”
From A pilot study using frequent inpatient assessments of suicidal thinking to predict short-term postdischarge suicidal behavior (2021) · cites this paper as [36]
In the citing paper
“Although machine learning methods have yielded high AUCs, the PPV of most models has been low.”
From A pilot study using frequent inpatient assessments of suicidal thinking to predict short-term postdischarge suicidal behavior (2021) · cites this paper as [36]
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research
Franklin, J. C., Ribeiro, J. D., Fox, K. R., Bentley, K. H., Kleiman, E. M., Huang, X., Musacchio, K. M., Jaroszewski, A. C., Chang, B. P., & Nock, M. K. (2017) · Psychological Bulletin · doi:10.1037/bul0000084 · Cited by 3,784
Wang, Y., Kung, L., & Byrd, T. A. (2018) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2015.12.019 · Cited by 1,571
Prediction of Suicide in Psychiatric Patients
Pokorny, A. D. (1983) · Archives of General Psychiatry · doi:10.1001/archpsyc.1983.01790030019002 · Cited by 749
Predicting Risk of Suicide Attempts Over Time Through Machine Learning
Walsh, C. G., Ribeiro, J. D., & Franklin, J. C. (2017) · Clinical Psychological Science · doi:10.1177/2167702617691560 · Cited by 571
Uses and abuses of screening tests
Grimes, D. A., & Schulz, K. F. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07948-5 · Cited by 545
Kessler, R. C., Warner, C. H., Ivany, C., Petukhova, M. V., Rose, S., Bromet, E. J., Brown, M., Cai, T., Colpe, L. J., Cox, K. L., et al. (2015) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2014.1754 · Cited by 447
Simon, G. E., Johnson, E., Lawrence, J. M., Rossom, R. C., Ahmedani, B., Lynch, F. L., Beck, A., Waitzfelder, B., Ziebell, R., Penfold, R. B., et al. (2018) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2018.17101167 · Cited by 400
Research works printed but not linked: 5
- 2018, Supporting our veterans during their transition from uniformed service to civilian life., Federal Register, 83, 1513
- Pruitt, 2017, Department of Defense Suicide Event Report: Calendar Year 2016 Annual Report
- 2018, VA National Suicide Data Report: 2005-2015
- 2018, Joint Action Plan for Supporting Veterans During Their Transition From Uniformed Service to Civilian Life
- VA REACH VET initiative helps save veterans lives: program signals when more help is needed for at-risk veterans [press release]. Washington DC: US Department of Veterans Affairs; April 3, 2017.
Cited by
Sorted by citations
Torous, J., Bucci, S., Bell, I. H., Kessing, L. V., Faurholt‐Jepsen, M., Whelan, P., Carvalho, A. F., Keshavan, M., Linardon, J., & Firth, J. (2021) · World Psychiatry · doi:10.1002/wps.20883 · Cited by 1,199
Time for united action on depression: a Lancet–World Psychiatric Association Commission
Herrman, H., Patel, V., Kieling, C., Berk, M., Buchweitz, C., Cuijpers, P., Furukawa, T. A., Kessler, R. C., Kohrt, B. A., Maj, M., et al. (2022) · The Lancet · doi:10.1016/s0140-6736(21)02141-3 · Cited by 1,104
Turecki, G., Brent, D. A., Gunnell, D., O’Connor, R. C., Oquendo, M. A., Pirkis, J., & Stanley, B. H. (2019) · Nature Reviews Disease Primers · doi:10.1038/s41572-019-0121-0 · Cited by 1,074
Supervised Machine Learning: A Brief Primer
Jiang, T., Gradus, J. L., & Rosellini, A. J. (2020) · Behavior Therapy · doi:10.1016/j.beth.2020.05.002 · Cited by 953
Machine learning and natural language processing in mental health: systematic review
Le Glaz, A., Haralambous, Y., Kim-Dufor, D.-H., Lenca, P., Billot, R., Ryan, T. C., Marsh, J., DeVylder, J., Walter, M., Berrouiguet, S., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/15708 · Cited by 576
Metadata
- Authors
- Belsher, B. E., Smolenski, D. J., Pruitt, L. D., Bush, N. E., Beech, E. H., Workman, D. E., Morgan, R. L., Evatt, D. P., Tucker, J., & Skopp, N. A.
- Affiliations
- Uniformed Services University of the Health Sciences (US)
- Defense Health Agency (US)
- McMaster University (CA)
- ORCID
- Venue
- JAMA Psychiatry
- Publisher
- American Medical Association (AMA)
- Published
- 2019-06-01
- Volume
- 76
- Issue
- 6
- Pages
- 642
- ISSN
- 2168-622X (ISSN-L)
- OpenAlex
- W2921616123
- Cited by
- 540
- References
- 46
- Citations by year
- 2026: 81 · 2025: 73 · 2024: 63 · 2023: 63 · 2022: 82 · 2021: 83 · 2020: 64 · 2019: 31
- FWCI
- 10.24
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- JAMA Psychiatry (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Suicide and Self-Harm Studies
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- suicide attempts, suicide mortality, diagnostic accuracy, predictive validity, positive predictive value, risk prediction, clinical implementation, health care system
- MeSH terms
- Algorithms, Humans, Models, Theoretical, Suicide, Suicide, Attempted, Suicidal Ideation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
