How accurate are suicide risk prediction models? asking the right questions for clinical practice.
Open accessAbstract
Prediction models assist in stratifying and quantifying an individual’s risk of developing a particular adverse outcome, and are widely used in cardiovascular and cancer medicine. Whether these approaches are accurate in predicting self-harm and suicide has been questioned. We searched for systematic reviews in the suicide risk assessment field, and identified three recent reviews that have examined current tools and models derived using machine learning approaches. In this clinical review, we present a critical appraisal of these reviews, and highlight three major limitations that are shared between them. First, structured tools are not compared with unstructured assessments routine in clinical practice. Second, they do not sufficiently consider a range of performance measures, including negative predictive value and calibration. Third, the potential role of these models as clinical adjuncts is not taken into consideration. We conclude by presenting the view that the current role of prediction models for self-harm and suicide is currently not known, and discuss some methodological issues and implications of some machine learning and other analytic techniques for clinical utility.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
US guidelines recommend offering bisphosphonates to patients with low bone density whose annual risk of hip fracture or major osteoporotic fracture exceeds 0.3% or 2%, respectively.
“offering bisphosphonates to patients with low bone density whose annual risk of hip fracture or major osteoporotic fracture exceeds 0.3% or 2%, respectively40,41”
From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [40]
These PPV estimates are in line with risk thresholds used for primary prevention of other similarly impactful conditions.
“Notably, these PPV estimates are in line with risk thresholds used for primary prevention of other similarly impactful conditions.”
From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [40]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199 · see citing works
Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research
Franklin, J. C., Ribeiro, J. D., Fox, K. R., Bentley, K. H., Kleiman, E. M., Huang, X., Musacchio, K. M., Jaroszewski, A. C., Chang, B. P., & Nock, M. K. (2017) · Psychological Bulletin · doi:10.1037/bul0000084 · Cited by 3,784 · see citing works
PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group† (2019) · Annals of Internal Medicine · doi:10.7326/m18-1376 · Cited by 2,984 · see citing works
Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research
Steyerberg, E. W., Moons, K. G. M., van der Windt, D. A., Hayden, J. A., Perel, P., Schroter, S., Riley, R. D., Hemingway, H., Altman, D. G., & for the PROGRESS Group (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001381 · Cited by 1,508 · see citing works
Prediction Models for Suicide Attempts and Deaths: A Systematic Review and Simulation
Belsher, B. E., Smolenski, D. J., Pruitt, L. D., Bush, N. E., Beech, E. H., Workman, D. E., Morgan, R. L., Evatt, D. P., Tucker, J., & Skopp, N. A. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.0174 · Cited by 540 · see citing works
Carter, G., Milner, A., McGill, K., Pirkis, J., Kapur, N., & Spittal, M. J. (2017) · British Journal of Psychiatry · doi:10.1192/bjp.bp.116.182717 · Cited by 361 · see citing works
Suicide risk assessment and intervention in people with mental illness
Bolton, J. M., Gunnell, D., & Turecki, G. (2015) · BMJ · doi:10.1136/bmj.h4978 · Cited by 294 · see citing works
Runeson, B., Odeberg, J., Pettersson, A., Edbom, T., Jildevik Adamsson, I., & Waern, M. (2017) · PLOS ONE · doi:10.1371/journal.pone.0180292 · Cited by 250 · see citing works
Can we usefully stratify patients according to suicide risk?
Large, M. M., Ryan, C. J., Carter, G., & Kapur, N. (2017) · BMJ · doi:10.1136/bmj.j4627 · Cited by 112 · see citing works
The role of prediction in suicide prevention
Large, M. M. B. (2018) · Dialogues in Clinical Neuroscience · doi:10.31887/dcns.2018.20.3/mlarge · Cited by 103 · see citing works
Fazel, S., Wolf, A., Larsson, H., Mallett, S., & Fanshawe, T. R. (2019) · Translational Psychiatry · doi:10.1038/s41398-019-0428-3 · Cited by 91 · see citing works
Self-citationRobinson, J. J., Meehan, L., & Appleby, L. (2002) · Research Explorer (The University of Manchester) · Cited by 49 · see citing works
Selecting a risk assessment tool to use in practice:a 10-point guide
Fazel, S., & Wolf, A. (2017) · Evidence-Based Mental Health · doi:10.1136/eb-2017-102861 · Cited by 41 · see citing works
Self-citationEffect of Patient and Therapist Factors on Suicide Risk Assessment
Berman, N. C., Stark, A., Cooperman, A., Wilhelm, S., & Cohen, I. G. (2015) · Death Studies · doi:10.1080/07481187.2014.958630 · Cited by 33 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [1]Health Education England The Topol Review. Preparing the healthcare workforce to deliver the digital future Health Education England 2019
- [2]National Institute for Health and Care Excellence Self-harm in over 8s: long-term management [CG 133] NICE 2011 31891461
- [14]National Institute for Health and Care Excellence Depression in adults: recognition and management [CG90]] NICE 2009 31990491
Cited by
Sorted by citations
Hawton, K., Lascelles, K., Pitman, A., Gilbert, S., & Silverman, M. (2022) · The Lancet Psychiatry · doi:10.1016/s2215-0366(22)00232-2 · Cited by 196 · see citing works
Ross, E. L., Zuromski, K. L., Reis, B. Y., Nock, M. K., Kessler, R. C., & Smoller, J. W. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2021.0089 · Cited by 65 · see citing works
Gatekeeper training for friends and family of individuals at risk of suicide: a systematic review
Morton, M., Wang, S., Tse, K., Chung, C., Bergmans, Y., Ceniti, A., Flam, S., Johannes, R., Schade, K., Terah, F., et al. (2021) · Journal of Community Psychology · doi:10.1002/jcop.22624 · Cited by 43 · see citing works
- Authors
- Whiting, D., & Fazel, S.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Fazel, Seena
- Venue
- Evidence Based Mental Health
- Publisher
- BMJ
- Published
- 2019-06-27
- Volume
- 22
- Issue
- 3
- Pages
- 125-128
- ISSN
- 1362-0347 (ISSN-L)
- 1468-960X
- PMC
- PMC7012643
- OpenAlex
- W2955434830
- Cited by
- 83
- References
- 14
- Citations by year
- 2026: 13 · 2025: 11 · 2024: 9 · 2023: 14 · 2022: 18 · 2021: 9 · 2020: 8 · 2019: 1
- FWCI
- 10.50
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Evidence-Based Mental Health (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Funders
- National Institute for Health Research
- Wellcome Trust
- Topics
- Social Sciences › Psychology › Clinical Psychology › Suicide and Self-Harm Studies
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- suicide risk prediction, self-harm, prediction model accuracy, machine learning, clinical utility, negative predictive value, model calibration
- MeSH terms
- Systematic Reviews as Topic, Humans, Suicide, Risk Assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
