The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Predicting suicidal behavior from longitudinal electronic health records.
Abstract
OBJECTIVE: The purpose of this article was to determine whether longitudinal historical data, commonly available in electronic health record (EHR) systems, can be used to predict patients' future risk of suicidal behavior. METHOD: Bayesian models were developed using a retrospective cohort approach. EHR data from a large health care database spanning 15 years (1998-2012) of inpatient and outpatient visits were used to predict future documented suicidal behavior (i.e., suicide attempt or death). Patients with three or more visits (N=1,728,549) were included. ICD-9-based case definition for suicidal behavior was derived by expert clinician consensus review of 2,700 narrative EHR notes (from 520 patients), supplemented by state death certificates. Model performance was evaluated retrospectively using an independent testing set. RESULTS: Among the study population, 1.2% (N=20,246) met the case definition for suicidal behavior. The model achieved sensitive (33%-45% sensitivity), specific (90%-95% specificity), and early (3-4 years in advance on average) prediction of patients' future suicidal behavior. The strongest predictors identified by the model included both well-known (e.g., substance abuse and psychiatric disorders) and less conventional (e.g., certain injuries and chronic conditions) risk factors, indicating that a data-driven approach can yield more comprehensive risk profiles. CONCLUSIONS: Longitudinal EHR data, commonly available in clinical settings, can be useful for predicting future risk of suicidal behavior. This modeling approach could serve as an early warning system to help clinicians identify high-risk patients for further screening. By analyzing the full phenotypic breadth of the EHR, computerized risk screening approaches may enhance prediction beyond what is feasible for individual clinicians.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“multiple research groups have developed suicide risk prediction models using electronic health record data.10,11,12,13”
From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [11]
In the citing paper
“33%, from a model developed in Massachusetts academic medical centers11”
From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [11]
In the citing paper
“and 2% to 6%11,12 for predicting suicide attempts”
From Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US (2021) · cites this paper as [11]
Citations
Cites
Nock, M. K., Borges, G., Bromet, E. J., Cha, C. B., Kessler, R. C., & Lee, S. (2008) · Epidemiologic Reviews · doi:10.1093/epirev/mxn002 · Cited by 2,897
Self-citationCross-national prevalence and risk factors for suicidal ideation, plans and attempts
Nock, M. K., Borges, G., Bromet, E. J., Alonso, J., Angermeyer, M., Beautrais, A., Bruffaerts, R., Chiu, W. T., de Girolamo, G., Gluzman, S., et al. (2008) · British Journal of Psychiatry · doi:10.1192/bjp.bp.107.040113 · Cited by 2,654
Self-citationClinical versus mechanical prediction: a meta-analysis
Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000) · Psychological Assessment · doi:10.1037/1040-3590.12.1.19 · Cited by 1,640
Machine learning for medical diagnosis: history, state of the art and perspective
Kononenko, I. (2001) · Artificial Intelligence in Medicine · doi:10.1016/s0933-3657(01)00077-x · Cited by 1,306
Clinical correlates of inpatient suicide
Busch, K. A., Fawcett, J., & Jacobs, D. G. (2003) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v64n0105 · Cited by 750
Measuring the Suicidal Mind: Implicit Cognition Predicts Suicidal Behavior
Nock, M. K., Park, J. M., Finn, C. T., Deliberto, T. L., Dour, H. J., & Banaji, M. R. (2010) · Psychological Science · doi:10.1177/0956797610364762 · Cited by 624
Self-citationKessler, R. C., Warner, C. H., Ivany, C., Petukhova, M. V., Rose, S., Bromet, E. J., Brown, M., Cai, T., Colpe, L. J., Cox, K. L., et al. (2015) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2014.1754 · Cited by 447
Self-citationImproving the Short-Term Prediction of Suicidal Behavior
Glenn, C. R., & Nock, M. K. (2014) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2014.06.004 · Cited by 328
Self-citationTrends in US emergency department visits for attempted suicide and self-inflicted injury, 1993-2008.
Ting, S. A., Sullivan, A. F., Boudreaux, E. D., Miller, I., & Camargo, C. A. (2012) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2012.03.020 · Cited by 252
Clements, C. C., Castro, V. M., Blumenthal, S. R., Rosenfield, H. R., Murphy, S. N., Fava, M., Erb, J. L., Churchill, S. E., Kaimal, A. J., Doyle, A. E., et al. (2015) · Molecular Psychiatry · doi:10.1038/mp.2014.90 · Cited by 197
Self-citationPredicting the Risk of Suicide by Analyzing the Text of Clinical Notes
Poulin, C., Shiner, B. R., Thompson, P., Vepštas, L., Young‐Xu, Y., Goertzel, B., Watts, B. V., Flashman, L. A., & McAllister, T. W. (2014) · PLoS ONE · doi:10.1371/journal.pone.0085733 · Cited by 168
Tran, T., Luo, W., Phung, D., Harvey, R., Berk, M., Kennedy, R. L., & Venkatesh, S. (2014) · BMC Psychiatry · doi:10.1186/1471-244x-14-76 · Cited by 150
Walkup, J. T., Townsend, L. D., Crystal, S., & Olfson, M. (2012) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.2335 · Cited by 133
Measuring the impact of health policies using Internet search patterns: the case of abortion
Reis, B. Y., & Brownstein, J. S. (2010) · BMC Public Health · doi:10.1186/1471-2458-10-514 · Cited by 103
Self-citationLongitudinal histories as predictors of future diagnoses of domestic abuse: modelling study
Reis, B. Y., Kohane, I. S., & Mandl, K. D. (2009) · BMJ · doi:10.1136/bmj.b3677 · Cited by 71
Self-citationIdentification of hospitalizations for intentional self‐harm when E‐codes are incompletely recorded
Patrick, A. R., Miller, M., Barber, C. W., Wang, P. S., Canning, C. F., & Schneeweiß, S. (2010) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.2037 · Cited by 70
Methods for identifying suicide or suicidal ideation in EHRs.
Haerian, K., Salmasian, H., & Friedman, C. (2012) · PubMed · Cited by 68
An Epidemiological Network Model for Disease Outbreak Detection
Reis, B. Y., Kohane, I. S., & Mandl, K. D. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040210 · Cited by 67
Self-citationUsing Data Mining to Explore Complex Clinical Decisions
Baca‐García, E., Pérez-Rodríguez, M. M., Basurte-Villamor, I., Sáiz, P. A., Leiva-Murillo, J. M., de Prado‐Cumplido, M., Mozos, R. S., Artés-Rodrı́guez, A., & de León, J. (2006) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v67n0716 · Cited by 58
Syndromic surveillance: the effects of syndrome grouping on model accuracy and outbreak detection
Reis, B. Y., & Mandl, K. D. (2004) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2004.03.030 · Cited by 57
Self-citationCastro, V. M., Gallagher, P., Clements, C. C., Murphy, S. N., Gainer, V. S., D, M. F. M., Weilburg, J. B., Churchill, S. E., Kohane, I. S., Iosifescu, D. V., et al. (2012) · BMJ Open · doi:10.1136/bmjopen-2011-000544 · Cited by 53
Self-citationArea Disease Estimation Based on Sentinel Hospital Records
Wang, J., Reis, B. Y., Hu, M.-G., Christakos, G., Yang, W., Sun, Q., Li, Z., Li, X., Lai, S., Chen, H., et al. (2011) · PLoS ONE · doi:10.1371/journal.pone.0023428 · Cited by 44
Self-citationIdentifying suicidal behavior among adolescents using administrative claims data
Callahan, S. T., Fuchs, D. C., Shelton, R. C., Balmer, L. S., Dudley, J. A., Gideon, P. S., DeRanieri, M. M., Stratton, S. M., Williams, C. L., Ray, W. A., et al. (2013) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3421 · Cited by 40
Ilgen, M. A., Downing, K., Zivin, K., Hoggatt, K. J., Kim, H. M., Ganoczy, D. A., Austin, K., McCarthy, J. F., Patel, J. M., & Valenstein, M. M. (2009) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.08m04795 · Cited by 39
How complete are E‐codes in commercial plan claims databases?
Lu, C. Y., Stewart, C. C., Ahmed, A. T., Ahmedani, B. K., Coleman, K. J., Copeland, L. A., Hunkeler, E. M., Lakoma, M. D., Madden, J. M., Penfold, R. B., et al. (2014) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3551 · Cited by 36
Concordance and predictive value of two adverse drug event data sets
Cami, A., & Reis, B. Y. (2014) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-14-74 · Cited by 4
Self-citation
Flagged first, then by citations
Research works printed but not linked: 2
- Nalichowski R, 2006, AMIA Annu Symp Proc, 1044
- Haerian K, 2012, AMIA Annu Symp Proc, 2012, 1244
Cited by
Sorted by citations
Improving Suicide Prevention Through Evidence-Based Strategies: A Systematic Review
Mann, J. J., Michel, C. A., & Auerbach, R. P. (2021) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2020.20060864 · Cited by 655
Prediction Models for Suicide Attempts and Deaths: A Systematic Review and Simulation
Belsher, B. E., Smolenski, D. J., Pruitt, L. D., Bush, N. E., Beech, E. H., Workman, D. E., Morgan, R. L., Evatt, D. P., Tucker, J., & Skopp, N. A. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.0174 · Cited by 540
Simon, G. E., Johnson, E., Lawrence, J. M., Rossom, R. C., Ahmedani, B., Lynch, F. L., Beck, A., Waitzfelder, B., Ziebell, R., Penfold, R. B., et al. (2018) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2018.17101167 · Cited by 400
Severity and Variability of Depression Symptoms Predicting Suicide Attempt in High-Risk Individuals
Melhem, N., Porta, G., Oquendo, M. A., Zelazny, J. M., Keilp, J. G., Iyengar, S., Burke, A. K., Birmaher, B. B., Stanley, B., Mann, J. J., et al. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2018.4513 · Cited by 261
Characterisation of mental health conditions in social media using Informed Deep Learning
Gkotsis, G., Oellrich, A., Velupillai, S. U., Liakata, M., Hubbard, T., Dobson, R., & Dutta, R. (2017) · Scientific Reports · doi:10.1038/srep45141 · Cited by 255
Metadata
- Authors
- Barak-Corren, Y., Castro, V. M., Javitt, S., Hoffnagle, A. G., Dai, Y., Perlis, R. H., Nock, M. K., Smoller, J. W., & Reis, B. Y.
- Affiliations
- From the Predictive Medicine Group, Boston Children’s Hospital Informatics Program, Boston; the Technion, Israeli Institute of Technology, Haifa, Israel; the Partners Research Information Systems and Computing, Boston; the Department of Psychiatry, Massachusetts General Hospital, Boston; the Psychiatric and Neurodevelopmental Genetics Unit, Center for Human Genetic Research, Massachusetts General Hospital, Boston; the Department of Psychology, Harvard University, Boston; and Harvard Medical School, Boston.
- Boston Children's Hospital (US)
- Venue
- American Journal of Psychiatry
- Published
- 2017-02-01
- Volume
- 174
- Issue
- 2
- Pages
- 154-162
- ISSN
- 0002-953X (ISSN-L)
- 1535-7228
- OpenAlex
- W2509888018
- Cited by
- 339
- References
- 26
- Citations by year
- 2026: 23 · 2025: 43 · 2024: 32 · 2023: 41 · 2022: 38 · 2021: 48 · 2020: 37 · 2019: 41 · 2018: 26 · 2017: 10
- FWCI
- 34.32
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Journal of Psychiatry (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Pfizer
- Massachusetts General Hospital
- Tommy Fuss Fund
- Harvard University
- Department of Psychology, Harvard University
- National Institute of Mental Health (K24MH094614)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Suicide and Self-Harm Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Keywords
- suicidal behavior, suicide risk prediction, electronic health records, suicide attempts, suicide mortality, risk factors, substance abuse, mental illness, injuries, chronic disease, early warning systems, risk screening
- MeSH terms
- Adult, Aged, Mental Disorders, Female, Humans, Longitudinal Studies, Male, Massachusetts, Middle Aged, Registries, Suicide, Suicide, Attempted, Case-Control Studies, Risk Assessment, Substance-Related Disorders, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
