CoChat
Sign Up
Cited by 400Suicide and Self-Harm StudiesMachine Learning in HealthcareMental Health via Writing

Predicting Suicide Attempts and Suicide Deaths Following Outpatient Visits Using Electronic Health Records

Open access
Simon, Gregory E., Johnson, Eric, Lawrence, Jean M., Rossom, Rebecca C., Ahmedani, Brian, Lynch, Frances L., Beck, Arne, Waitzfelder, Beth +3 more·2018·American Journal of Psychiatry
DOI 10.1176/appi.ajp.2018.17101167PMC PMC6167136
Open this paper in CoChat — free accountOpen PDF

Abstract

OBJECTIVE: The authors sought to develop and validate models using electronic health records to predict suicide attempt and suicide death following an outpatient visit. METHOD: Across seven health systems, 2,960,929 patients age 13 or older (mean age, 46 years; 62% female) made 10,275,853 specialty mental health visits and 9,685,206 primary care visits with mental health diagnoses between Jan. 1, 2009, and June 30, 2015. Health system records and state death certificate data identified suicide attempts (N=24,133) and suicide deaths (N=1,240) over 90 days following each visit. Potential predictors included 313 demographic and clinical characteristics extracted from records for up to 5 years before each visit: prior suicide attempts, mental health and substance use diagnoses, medical diagnoses, psychiatric medications dispensed, inpatient or emergency department care, and routinely administered depression questionnaires. Logistic regression models predicting suicide attempt and death were developed using penalized LASSO (least absolute shrinkage and selection operator) variable selection in a random sample of 65% of the visits and validated in the remaining 35%. RESULTS: Mental health specialty visits with risk scores in the top 5% accounted for 43% of subsequent suicide attempts and 48% of suicide deaths. Of patients scoring in the top 5%, 5.4% attempted suicide and 0.26% died by suicide within 90 days. C-statistics (equivalent to area under the curve) for prediction of suicide attempt and suicide death were 0.851 (95% CI=0.848, 0.853) and 0.861 (95% CI=0.848, 0.875), respectively. Primary care visits with scores in the top 5% accounted for 48% of subsequent suicide attempts and 43% of suicide deaths. C-statistics for prediction of suicide attempt and suicide death were 0.853 (95% CI=0.849, 0.857) and 0.833 (95% CI=0.813, 0.853), respectively. CONCLUSIONS: Prediction models incorporating both health record data and responses to self-report questionnaires substantially outperform existing suicide risk prediction tools.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1176/appi.ajp.2018.17101167

Cites

46 of 46 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • One study of machine learning of health records examined the medical and psychiatric records of ~3 million patients with at least one mental health diagnosis across seven different health care systems.

    “One study of machine learning of health records for the prediction of suicide and suicide attempts examined the medical and psychiatric records of ~3 million patients with at least one mental health diagnosis across seven different health care systems 222 .”

    From Suicide and suicide risk (2019) · cites this paper as [222]

    The Abstract Methods and Results report 2,960,929 patients across seven health systems and describe using health records to predict suicide attempts and deaths. The Methods specify that eligible visits were either to a specialty mental health clinic or to primary care when a mental health diagnosis was recorded; they do not establish that every patient had a mental health diagnosis. Page numbers are not provided in our copy. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • In this study, an algorithm was developed that could identify 5% of the population that accounted for 43-48% of the suicides and suicide attempts, and was able to predict suicide and suicide attempts in the next 90 days with a specificity of ~70% and a sensitivity of ~80% in both specialty mental health and primary care settings.

    “In this study, an algorithm was developed that could identify 5% of the population that accounted for 43-48% of the suicides and suicide attempts, and was able to predict suicide and suicide attempts in the next 90 days with a specificity of ~70% and a sensitivity of ~80% in both specialty mental health and primary care settings 222 .”

    From Suicide and suicide risk (2019) · cites this paper as [222]

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • Multiple research groups have developed suicide risk prediction models using electronic health record data.

    “multiple research groups have developed suicide risk prediction models using electronic health record data.10,11,12,13”

    From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [12]

  • Corresponding PPVs were 2% to 6% for predicting suicide attempts.

    “and 2% to 6%11,12 for predicting suicide attempts”

    From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [12]

  • Corresponding PPVs were 0.1% to 0.3% for predicting suicide death.

    “The corresponding PPVs were 0.1% to 0.3% for predicting suicide death12,13”

    From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [12]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Simon, G. E., Johnson, E., Lawrence, J. M., Rossom, R. C., Ahmedani, B., Lynch, F. L., Beck, A., Waitzfelder, B., Ziebell, R., Penfold, R. B., & Shortreed, S. M.
Affiliations
Kaiser Permanente Washington Health Research Institute (US)
ORCID
Venue
American Journal of Psychiatry
Publisher
American Psychiatric Association Publishing
Published
2018-10-01
Volume
175
Issue
10
Pages
951-960
ISSN
  • 0002-953X (ISSN-L)
  • 1535-7228
DOI
10.1176/appi.ajp.2018.17101167
PMC
PMC6167136
OpenAlex
W2804266670
Cited by
400
References
46
Citations by year
2026: 42 · 2025: 57 · 2024: 37 · 2023: 56 · 2022: 53 · 2021: 49 · 2020: 55 · 2019: 44 · 2018: 7
FWCI
45.53
Type
journal article
Language
en
Open access
Locations
  • American Journal of Psychiatry (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
National Institute of Mental Health (U19MH092201)
Topics
  • Social Sciences › Psychology › Clinical Psychology › Suicide and Self-Harm Studies
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Social Sciences › Psychology › Social Psychology › Mental Health via Writing
Keywords
suicide attempts, suicide mortality, suicide risk prediction, outpatient mental health visits, primary care visits, electronic health records, LASSO feature selection, risk score, substance use, psychiatric medication, mental health diagnoses, prediction model validation, death certificates, emergency department care, inpatient care, self-report questionnaire
MeSH terms
Adolescent, Adult, Aged, Female, Humans, Male, Middle Aged, Models, Psychological, Outpatients, Surveys and Questionnaires, Risk Factors, Suicide, Attempted, Young Adult, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review