The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Risk stratification using data from electronic medical records better predicts suicide risks than clinician assessments
Open accessAbstract
BACKGROUND: To date, our ability to accurately identify patients at high risk from suicidal behaviour, and thus to target interventions, has been fairly limited. This study examined a large pool of factors that are potentially associated with suicide risk from the comprehensive electronic medical record (EMR) and to derive a predictive model for 1-6 month risk. METHODS: 7,399 patients undergoing suicide risk assessment were followed up for 180 days. The dataset was divided into a derivation and validation cohorts of 4,911 and 2,488 respectively. Clinicians used an 18-point checklist of known risk factors to divide patients into low, medium, or high risk. Their predictive ability was compared with a risk stratification model derived from the EMR data. The model was based on the continuation-ratio ordinal regression method coupled with lasso (which stands for least absolute shrinkage and selection operator). RESULTS: In the year prior to suicide assessment, 66.8% of patients attended the emergency department (ED) and 41.8% had at least one hospital admission. Administrative and demographic data, along with information on prior self-harm episodes, as well as mental and physical health diagnoses were predictive of high-risk suicidal behaviour. Clinicians using the 18-point checklist were relatively poor in predicting patients at high-risk in 3 months (AUC 0.58, 95% CIs: 0.50 - 0.66). The model derived EMR was superior (AUC 0.79, 95% CIs: 0.72 - 0.84). At specificity of 0.72 (95% CIs: 0.70-0.73) the EMR model had sensitivity of 0.70 (95% CIs: 0.56-0.83). CONCLUSION: Predictive models applied to data from the EMR could improve risk stratification of patients presenting with potential suicidal behaviour. The predictive factors include known risks for suicide, but also other information relating to general health and health service utilisation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Numerous recent studies predicted suicide death or attempted suicide in high-risk patient populations from electronic health record data using machine learning (ML) methods.”
From Using Administrative Data to Predict Suicide After Psychiatric Hospitalization in the Veterans Health Administration System (2020) · cites this paper as [69]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Comorbidity measures for use with administrative data.
Elixhauser, A., Steiner, C., Harris, D. R., & Coffey, R. M. (1998) · Medical Care · doi:10.1097/00005650-199801000-00004 · Cited by 9,921
Regression Shrinkage and Selection via The Lasso: A Retrospective
Tibshirani, R. J. (2011) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2011.00771.x · Cited by 3,601
Nock, M. K., Green, J. G., Hwang, I., McLaughlin, K. A., Sampson, N. A., Zaslavsky, A. M., & Kessler, R. C. (2013) · JAMA Psychiatry · doi:10.1001/2013.jamapsychiatry.55 · Cited by 1,666
Contact With Mental Health and Primary Care Providers Before Suicide: A Review of the Evidence
Luoma, J. B., Martin, C. E., & Pearson, J. L. (2002) · American Journal of Psychiatry · doi:10.1176/appi.ajp.159.6.909 · Cited by 1,576
Risk factors for suicide in individuals with depression: a systematic review
Hawton, K., Casañas i Comabella, C., Haw, C., & Saunders, K. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.01.004 · Cited by 1,469
Borges, G., Nock, M. K., Haro Abad, J. M., Hwang, I., Sampson, N. A., Alonso, J., Andrade, L. H., Angermeyer, M. C., Beautrais, A., Bromet, E., et al. (2010) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.08m04967blu · Cited by 796
Regression models for ordinal responses: a review of methods and applications
Ananth, C. V. (1997) · International Journal of Epidemiology · doi:10.1093/ije/26.6.1323 · Cited by 594
Trends in US emergency department visits for attempted suicide and self-inflicted injury, 1993-2008.
Ting, S. A., Sullivan, A. F., Boudreaux, E. D., Miller, I., & Camargo, C. A. (2012) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2012.03.020 · Cited by 252
Suicidal behavior in bipolar disorder: Epidemiology, characteristics and major risk factors
Gonda, X., Pompili, M., Serafini, G., Montebovi, F., Campi, S., Döme, P., Duleba, T., Girardi, P., & Rihmer, Z. (2012) · Journal of Affective Disorders · doi:10.1016/j.jad.2012.04.041 · Cited by 219
Perry, I. J., Corcoran, P. E., Fitzgerald, T., Keeley, H., Reulbach, U., & Arensman, E. (2012) · PLoS ONE · doi:10.1371/journal.pone.0031663 · Cited by 179
Moons, K. G. M., Donders, A. R. T., Steyerberg, E. W., & Harrell, F. E. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.01.020 · Cited by 175
Challis, S., Nielssen, O. B., Harris, A. W. F., & Large, M. M. B. (2013) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12074 · Cited by 156
Suicide among adults aged 35-64 years--United States, 1999-2010.
Sullivan, E., Annest, J. L., Luo, F., Simon, T. R., & Dahlberg, L. L. (2013) · PubMed Central · Cited by 155
Emergency department contact prior to suicide in mental health patients.
Da Cruz, D., Pearson, A., Saini, P., Miles, C., While, D., Swinson, N., Williams, A., Shaw, J., Appleby, L., & Kapur, N. (2011) · Emergency Medicine Journal · doi:10.1136/emj.2009.081869 · Cited by 149
Suicide Intent Scale in the prediction of suicide
Stefánsson, J. G., Nordström, P., & Jokinen, J. (2010) · Journal of Affective Disorders · doi:10.1016/j.jad.2010.11.016 · Cited by 115
Suicide among adults aged 35-64 years--United States, 1999-2010.
(2013) · PubMed · Cited by 94
Predicting Suicide Attempts With the SAD PERSONS Scale
Bolton, J. M., Spiwak, R. P., & Sareen, J. (2012) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.11m07362 · Cited by 91
Hospitalization for physical illness and risk of subsequent suicide: a population study
Qin, P., Webb, R., Kapur, N., & Sørensen, H. T. (2013) · Journal of Internal Medicine · doi:10.1111/j.1365-2796.2012.02572.x · Cited by 89
Prediction and prevention of suicide in patients with unipolar depression and anxiety
Gonda, X., Fountoulakis, K. Ν., Kaprinis, G., & Rihmer, Z. (2007) · Annals of General Psychiatry · doi:10.1186/1744-859x-6-23 · Cited by 79
Allen, M. H., Abar, B. W., McCormick, M., Barnes, D. H., Haukoos, J. S., Garmel, G. M., & Boudreaux, E. D. (2013) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12018 · Cited by 68
Archer, K. J., & Williams, A. (2012) · Statistics in Medicine · doi:10.1002/sim.4484 · Cited by 63
The Rorschach Suicide Constellation: Assessing Various Degrees of Lethality
Fowler, J. C., Piers, C., Hilsenroth, M. J., Holdwick, D. J., & Padawer, J. R. (2001) · Journal of Personality Assessment · doi:10.1207/s15327752jpa7602_13 · Cited by 60
Development of the Risk Assessment Suicidality Scale (RASS): A population-based study
Fountoulakis, K. Ν., Pantoula, E., Siamouli, M., Moutou, K., Gonda, X., Rihmer, Z., Iacovides, A., & Akiskal, H. S. (2012) · Journal of Affective Disorders · doi:10.1016/j.jad.2011.12.045 · Cited by 59
A Profile of Medically Serious Suicide Attempts
Elliott, A. J., Pages, K. P., Russo, J. E., Wilson, L. G., & Roy‐Byrne, P. (1996) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v57n1202 · Cited by 52
Epidemiological trends of suicide and attempted suicide by poisoning in the US: 2000–2008
Spiller, H. A., Appana, S. N., & Brock, G. N. (2010) · Legal Medicine · doi:10.1016/j.legalmed.2010.04.005 · Cited by 51
Clinical and epidemiological aspects of suicide in patients with schizophrenia.
Gómez‐Durán, E. L., Martín-Fumadó, C., & Hurtado-Ruíz, G. (2013) · PubMed · Cited by 41
Evaluating the predictive validity of suicidal intent and medical lethality in youth.
Sapyta, J. J., Goldston, D. B., Erkanli, A., Daniel, S. S., Heilbron, N., Mayfield, A., & Treadway, S. L. (2012) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0026870 · Cited by 41
Choi, J. W., Park, S., Yi, K. K., & Hong, J. P. (2012) · Suicide and Life-Threatening Behavior · doi:10.1111/j.1943-278x.2012.00085.x · Cited by 40
Wærn, M., Sjöström, N., Marlow, T., & Hetta, J. E. (2010) · European Psychiatry · doi:10.1016/j.eurpsy.2010.03.014 · Cited by 40
Living alone and deliberate self-harm: a case–control study of characteristics and risk factors
Haw, C., & Hawton, K. E. (2010) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-010-0278-z · Cited by 39
Evaluation of clinical prognostic models for suicide attempts after a major depressive episode
Galfalvy, H., Oquendo, M. A., & Mann, J. J. (2008) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2008.01162.x · Cited by 37
Huffman, L. C., Wang, N. E., Saynina, O., Wren, F. J., Wise, P. H., & Horwitz, S. M. (2012) · Psychiatric Services · doi:10.1176/appi.ps.201000482 · Cited by 31
Epidemiological trends of suicide and attempted suicide by poisoning in the US: 2000–2008
Spiller, H. A., Appana, S. N., & Brock, G. N. (2010) · Journal of Forensic Medicine · Cited by 26
Suicide risk assessment: where are we now?
Ryan, C. J., & Large, M. M. B. (2013) · The Medical Journal of Australia · doi:10.5694/mja13.10437 · Cited by 25
Suicide risk assessment: where are we now?
Ryan, C. J., Large, M., & Callaghan, S. (2013) · The Medical Journal of Australia · doi:10.5694/mja13.10840 · Cited by 23
de Achaval, S., Feudtner, C., Palla, S. L., & Suarez-Almazor, M. E. (2013) · BMC Health Services Research · doi:10.1186/1472-6963-13-72 · Cited by 20
Risk factors for suicide in Japan: A model of predicting suicide in 2008 by risk factors of 2007
Ishii, N., Terao, T., Araki, Y., Kohno, K., Mizokami, Y., Arasaki, M., & Iwata, N. (2012) · Journal of Affective Disorders · doi:10.1016/j.jad.2012.11.038 · Cited by 18
Outpatient Health Care Utilization of Suicide Decedents in Their Last Year of Life
Liu, H., Chen, L.-H., & Huang, S. (2012) · Suicide and Life-Threatening Behavior · doi:10.1111/j.1943-278x.2012.00103.x · Cited by 13
Bilén, K., Ponzer, S., Ottosson, C., Castrén, M. K., Owe‐Larsson, B., Ekdahl, K., & Pettersson, H. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.01.037 · Cited by 10
Sakinofsky, I. (2005) · Evidence-Based Mental Health · doi:10.1136/ebmh.8.4.97 · Cited by 9
Cited by
Sorted by citations
Luo, W., Phung, D., Tran, T., Gupta, S., Rana, S., Karmakar, C., Shilton, A., Yearwood, J., Dimitrova, N., Ho, T. B., et al. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5870 · Cited by 988
Goldstein, B. A., Navar, A. M., Pencina, M. J., & Ioannidis, J. P. A. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw042 · Cited by 936
Machine Learning Approaches for Clinical Psychology and Psychiatry
Dwyer, D. B., Falkai, P., & Koutsouleris, N. (2018) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032816-045037 · Cited by 966
Prediction Models for Suicide Attempts and Deaths: A Systematic Review and Simulation
Belsher, B. E., Smolenski, D. J., Pruitt, L. D., Bush, N. E., Beech, E. H., Workman, D. E., Morgan, R. L., Evatt, D. P., Tucker, J., & Skopp, N. A. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.0174 · Cited by 540
Lee, E., Torous, J. B., Choudhury, M. D., Depp, C. A., Graham, S. A., Kim, H., Paulus, M. P., Krystal, J. H., & Jeste, D. V. (2021) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2021.02.001 · Cited by 523
Metadata
- Authors
- Tran, T., Luo, W., Phung, D., Harvey, R., Berk, M., Kennedy, R. L., & Venkatesh, S.
- Affiliations
- Deakin University (AU)
- Barwon Health (AU)
- Corresponding author
- Venkatesh, Svetha
- Venue
- BMC Psychiatry
- Published
- 2014-03-14
- Volume
- 14
- Issue
- 1
- Pages
- 76-76
- ISSN
- 1471-244X (ISSN-L)
- PMC
- PMC3984680
- OpenAlex
- W2123409929
- Cited by
- 150
- References
- 41
- Citations by year
- 2026: 5 · 2025: 9 · 2024: 12 · 2023: 9 · 2022: 8 · 2021: 32 · 2020: 17 · 2019: 12 · 2018: 14 · 2017: 14 · 2016: 10 · 2015: 7 · 2014: 1
- FWCI
- 12.62
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Psychiatry (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- eSpace (Curtin University) (submittedVersion)
- eSpace (Curtin University) (submittedVersion, cc-by)
- Figshare (submittedVersion, cc-by)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Suicide and Self-Harm Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Keywords
- suicide risk, risk stratification, electronic health records, clinical risk assessment, prediction model, emergency department visits, hospitalization, health care utilization, LASSO
- MeSH terms
- Suicide Prevention, Adolescent, Adult, Aged, Australia, Emergency Service, Hospital, Female, Humans, Male, Medical History Taking, Middle Aged, Prognosis, Retrospective Studies, Risk Factors, Suicide, Models, Statistical, Risk Assessment, Young Adult, Electronic Health Records, Suicidal Ideation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
