Developing a practical suicide risk prediction model for targeting high‐risk patients in the Veterans health Administration
Open accessAbstract
Objectives: The US Veterans Health Administration (VHA) has begun using predictive modeling to identify Veterans at high suicide risk to target care. Initial analyses are reported here. Methods: A penalized logistic regression model was compared with an earlier proof‐of‐concept logistic model. Exploratory analyses then considered commonly‐used machine learning algorithms. Analyses were based on electronic medical records for all 6,360 individuals classified in the National Death Index as having died by suicide in fiscal years 2009–2011 who used VHA services the year of their death or prior year and a 1% probability sample of time‐matched VHA service users alive at the index date ( n = 2,112,008). Results: A penalized logistic model with 61 predictors had sensitivity comparable to the proof‐of‐concept model (which had 381 predictors) at target thresholds. The machine learning algorithms had relatively similar sensitivities, the highest being for Bayesian additive regression trees, with 10.7% of suicides occurred among the 1.0% of Veterans with highest predicted risk and 28.1% among the 5.0% of with highest predicted risk. Conclusions: Based on these results, VHA is using penalized logistic regression in initial intervention implementation. The paper concludes with a discussion of other practical issues that might be explored to increase model performance.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
BERT models effectively capture the interaction between a key clinical concept and their surrounding context, and have shown superior performance in various NLP tasks including text classification.
“In particular, the BERT (Bidirectional Encoder Representations from Transformers) models effectively capture the interaction between a key clinical concept and their surrounding context, and have shown superior performance in various NLP tasks including text classification16.”
From Deep learning for identifying personal and family history of suicidal thoughts and behaviors from EHRs (2024) · cites this paper as [16]
The Analysis methods (§2.3) and Results (§3.2) discuss BART—Bayesian additive regression trees—for predicting suicide risk, not BERT. Searches for BERT, transformers, clinical-context interactions, and NLP text classification found no passage addressing the attributed claim. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482 · see citing works
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957 · see citing works
Regularization Paths for Generalized Linear Models via Coordinate Descent
Friedman, J., Hastie, T., & Tibshirani, R. (2010) · Journal of Statistical Software · doi:10.18637/jss.v033.i01 · Cited by 19,927 · see citing works
Classification and Regression by randomForest
Liaw, A. I., & Wiener, M. C. (2007) · Cited by 18,594 · see citing works
Statistical Comparisons of Classifiers over Multiple Data Sets
Demšar, J. (2006) · doi:10.5555/1248547.1248548 · Cited by 11,966 · see citing works
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359 · see citing works
Ridge regression: Biased estimation for nonorthogonal problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.1080/00401706.1970.10488634 · Cited by 9,648 · see citing works
Multivariate Adaptive Regression Splines
Friedman, J. H. (1991) · The Annals of Statistics · doi:10.1214/aos/1176347963 · Cited by 8,108 · see citing works
Top 10 algorithms in data mining
Wu, X., Kumar, V., Ross Quinlan, J., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G. J., Ng, A., Liu, B., Yu, P. S., et al. (2008) · Knowledge and Information Systems · doi:10.1007/s10115-007-0114-2 · Cited by 5,667 · see citing works
A Short Introduction to Boosting
Freund, Y., & Schapire, R. E. (1999) · Cited by 2,991 · see citing works
Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine LearningPreprint
Lemaître, G., Nogueira, F., & Aridas, C. K. (2016) · Journal of machine learning research · doi:10.48550/arxiv.1609.06570 · Cited by 2,733 · see citing works
Do we need hundreds of classifiers to solve real world classification problems
Fernández-Delgado, M., Cernadas, E., Barro, S., & AmorimDinani (2014) · Journal of Machine Learning Research · doi:10.5555/2627435.2697065 · Cited by 2,131 · see citing works
Ridge Regression: Biased Estimation for Nonorthogonal Problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.2307/1267351 · Cited by 1,978 · see citing works
caret: Classification and Regression Training
Kühn, M. (2007) · doi:10.32614/cran.package.caret · Cited by 1,964 · see citing works
BART: Bayesian additive regression trees
Chipman, H. A., George, E. I., & McCulloch, R. E. (2010) · The Annals of Applied Statistics · doi:10.1214/09-aoas285 · Cited by 1,875 · see citing works
Graph-based approach for airborne light detection and ranging segmentation
Vilariño, D. L., Cabaleiro, J. C., Sánchez, J. M., Rivera, F. F., & Pena, T. F. (2017) · Journal of Applied Remote Sensing · doi:10.1117/1.jrs.11.015020 · Cited by 1,760 · see citing works
Data mining and knowledge discovery handbook
(2011) · Choice Reviews Online · doi:10.5860/choice.48-5729 · Cited by 1,174 · see citing works
BART: Bayesian Additive Regression Trees
Chipman, H., George, E. I., & McCulloch, R. E. (2006) · ScholarlyCommons (University of Pennsylvania) · Cited by 889 · see citing works
Harnessing Smartphone-Based Digital Phenotyping to Enhance Behavioral and Mental Health
Onnela, J.-P., & Rauch, S. L. (2016) · Neuropsychopharmacology · doi:10.1038/npp.2016.7 · Cited by 726 · see citing works
Steinwart, I., & Christmann, A. (2008) · Information science and statistics · doi:10.1007/978-0-387-77242-4 · Cited by 654 · see citing works
Misc Functions of the Department of Statistics, ProbabilityTheory Group (Formerly: E1071), TU Wien
MEYER, D. A., Dimitriadou, E., Hornik, K., Weingessel, A., & Leisch, F. (2015) · Cited by 523 · see citing works
Estimating treatment effect heterogeneity in randomized program evaluation
Imai, K., & Ratkovic, M. (2013) · The Annals of Applied Statistics · doi:10.1214/12-aoas593 · Cited by 487 · see citing works
Kessler, R. C., Warner, C. H., Ivany, C., Petukhova, M. V., Rose, S., Bromet, E. J., Brown, M., Cai, T., Colpe, L. J., Cox, K. L., et al. (2015) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2014.1754 · Cited by 447 · cited 1× · see citing works
“…here but only note that they assessed variables in five broad domains that have been shown in previous research to predict future suicides (Kessler et al., 2015; Kessler, Stein et al., 2017): intensity‐recency of inpatient, outpatient, and emergency service use for various mental…”
BackgroundSelf-citationPolynomial splines and their tensor products in extended linear modeling: 1994 Wald memorial lecture
Stone, C. J., Hansen, M. H., Kooperberg, C. L., & Truong, Y. K.-N. (1997) · The Annals of Statistics · doi:10.1214/aos/1031594728 · Cited by 397 · see citing works
Addressing the class imbalance problem in medical datasets
Rahman, M. M., & Davis, D. N. (2013) · International Journal of Machine Learning and Computing · doi:10.7763/ijmlc.2013.v3.307 · Cited by 387 · see citing works
Willett, J. B., & Singer, J. D. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.61.6.952 · Cited by 357 · see citing works
Dynamic Prediction in Clinical Survival Analysis
van Houwelingen, H. C., & Putter, H. (2011) · CRC Press eBooks · doi:10.1201/b11311 · Cited by 352 · see citing works
Modelling suicide and unemployment: a longitudinal analysis covering 63 countries, 2000–11
Nordt, C., Warnke, I., Seifritz, E., & Kawohl, W. (2015) · The Lancet Psychiatry · doi:10.1016/s2215-0366(14)00118-7 · Cited by 322 · see citing works
Willett, J. B., & Singer, J. D. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.61.6.952 · Cited by 276 · see citing works
Shmilovici, A. (2009) · Springer eBooks · doi:10.1007/978-0-387-09823-4_12 · Cited by 252 · see citing works
McCarthy, J. F., Bossarte, R. M., Katz, I. R., Thompson, C., Kemp, J., Hannemann, C. M., Nielson, C., & Schoenbaum, M. (2015) · American Journal of Public Health · doi:10.2105/ajph.2015.302737 · Cited by 251 · see citing works
Self-citationReger, M. A., Smolenski, D. J., Skopp, N. A., Metzger-Abamukang, M. J., Kang, H. K., Bullman, T. A., Perdue, S., & Gahm, G. A. (2015) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2014.3195 · Cited by 222 · see citing works
Using patient self-reports to study heterogeneity of treatment effects in major depressive disorder
Kessler, R. C., van Loo, H. M., Wardenaar, K. J., Bossarte, R. M., Brenner, L. A., Ebert, D. D., de Jonge, P., Nierenberg, A. A., Rosellini, A. J., Sampson, N. A., et al. (2016) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796016000020 · Cited by 193 · see citing works
Self-citationKessler, R. C., Stein, M. B., Petukhova, M. V., Bliese, P., Bossarte, R. M., Bromet, E. J., Fullerton, C. S., Gilman, S. E., Ivany, C., Lewandowski-Romps, L., et al. (2017) · Molecular Psychiatry · doi:10.1038/mp.2016.110 · Cited by 163 · see citing works
Self-citationMachine learning approaches for the discovery of gene-gene interactions in disease data
Upstill‐Goddard, R., Eccles, D., Fliege, J., & Collins, A. R. (2012) · Briefings in Bioinformatics · doi:10.1093/bib/bbs024 · Cited by 91 · see citing works
Optimizing randomized trial designs to distinguish which subpopulations benefit from treatment
Rosenblum, M., & van der Laan, M. J. (2011) · Biometrika · doi:10.1093/biomet/asr055 · Cited by 62 · see citing works
Resampling methods improve the predictive power of modeling in class-imbalanced datasets
Lee, P. (2014) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph110909776 · Cited by 62 · see citing works
Alam, M. G. R., Cho, E. J., Huh, E., & Hong, C. S. (2014) · International Conference on Ubiquitous Information Management and Communication (IMCOM) · doi:10.1145/2557977.2558020 · Cited by 36 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Christodoulou, E., Ma, J., Collins, G. S., Steyerberg, E. W., Verbakel, J. Y., & Van Calster, B. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.02.004 · Cited by 2,119 · see citing works
Machine learning in mental health: a scoping review of methods and applications
Shatte, A. B. R., Hutchinson, D. M., & Teague, S. J. (2019) · Psychological Medicine · doi:10.1017/s0033291719000151 · Cited by 1,047 · see citing works
Artificial intelligence for mental health and mental illnesses: an overview
Graham, S., Depp, C., Lee, E. E., Nebeker, C., Tu, X., Kim, H.-C., & Jeste, D. V. (2019) · Current Psychiatry Reports · doi:10.1007/s11920-019-1094-0 · Cited by 970 · see citing works
Prediction Models for Suicide Attempts and Deaths: A Systematic Review and Simulation
Belsher, B. E., Smolenski, D. J., Pruitt, L. D., Bush, N. E., Beech, E. H., Workman, D. E., Morgan, R. L., Evatt, D. P., Tucker, J., & Skopp, N. A. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.0174 · Cited by 540 · see citing works
Simon, G. E., Johnson, E., Lawrence, J. M., Rossom, R. C., Ahmedani, B., Lynch, F. L., Beck, A., Waitzfelder, B., Ziebell, R., Penfold, R. B., et al. (2018) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2018.17101167 · Cited by 400 · see citing works
Understanding Links among Opioid Use, Overdose, and Suicide
Bohnert, A. S. B., & Ilgen, M. A. (2019) · New England Journal of Medicine · doi:10.1056/nejmra1802148 · Cited by 321 · see citing works
Jacobs, M., Pradier, M. F., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01224-x · Cited by 247 · see citing works
Kessler, R. C., Bossarte, R. M., Luedtke, A., Zaslavsky, A. M., & Zubizarreta, J. R. (2020) · Molecular Psychiatry · doi:10.1038/s41380-019-0531-0 · Cited by 234 · see citing works
Self-citationBernert, R. A., Hilberg, A. M., Melia, R., Kim, J. P., Shah, N. H., & Abnousi, F. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17165929 · Cited by 225 · see citing works
Torous, J., Larsen, M. E., Depp, C., Cosco, T. D., Barnett, I., Nock, M. K., & Firth, J. (2018) · Current Psychiatry Reports · doi:10.1007/s11920-018-0914-y · Cited by 207 · see citing works
Burke, T. A., Ammerman, B. A., & Jacobucci, R. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2018.11.073 · Cited by 202 · see citing works
Richards, J. E., Whiteside, U., Ludman, E. J., Pabiniak, C., Kirlin, B., Hidalgo, R., & Simon, G. (2019) · Psychiatric Services · doi:10.1176/appi.ps.201800342 · Cited by 167 · see citing works
Luedtke, A., Sadikova, E., & Kessler, R. C. (2019) · Clinical Psychological Science · doi:10.1177/2167702618815466 · Cited by 143 · see citing works
Self-citationO’Connor, R. C., & Portzky, G. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.02139 · Cited by 146 · see citing works
Identification of Suicide Attempt Risk Factors in a National US Survey Using Machine Learning
de la Garza, Á. G., Blanco, C., Olfson, M., & Wall, M. M. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.4165 · Cited by 139 · see citing works
The Secondary Use of Electronic Health Records for Data Mining: Data Characteristics and Challenges
Sarwar, T., Seifollahi, S., Chan, J., Zhang, X., Aksakallı, V., Hudson, I. L., Verspoor, K. M., & Cavedon, L. (2022) · ACM Computing Surveys · doi:10.1145/3490234 · Cited by 129 · see citing works
Leist, A. K., Klee, M., Kim, J. H., Rehkopf, D. H., Bordas, S., Terrera, G. M., & Wade, S. (2022) · Science Advances · doi:10.1126/sciadv.abk1942 · Cited by 127 · see citing works
Artificial intelligence and suicide prevention: A systematic review
Lejeune, A., Glaz, A. L., Perron, P.-A., Sebti, J., Baca‐García, E., Walter, M., Lemey, C., & Berrouiguet, S. (2022) · European Psychiatry · doi:10.1192/j.eurpsy.2022.8 · Cited by 123 · see citing works
Zheng, L., Wang, O., Hao, S., Ye, C., Liu, M., Xia, M., Sabo, A. N., Markovic, L., Stearns, F., Kanov, L., et al. (2020) · Translational Psychiatry · doi:10.1038/s41398-020-0684-2 · Cited by 112 · see citing works
Tsui, F. R., Shi, L., Ruiz, V., Ryan, N. D., Biernesser, C. L., Iyengar, S., Walsh, C. G., & Brent, D. A. (2021) · JAMIA Open · doi:10.1093/jamiaopen/ooab011 · Cited by 105 · see citing works
McCarthy, J. F., Cooper, S. A., Dent, K. R., Eagan, A. E., Matarazzo, B. B., Hannemann, C. M., Reger, M. A., Landes, S. J., Trafton, J. A., Schoenbaum, M., et al. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.29900 · Cited by 98 · see citing works
Self-citationTranslating promise into practice: a review of machine learning in suicide research and prevention
Kirtley, O. J., van Mens, K., Hoogendoorn, M., Kapur, N., & de Beurs, D. (2022) · The Lancet Psychiatry · doi:10.1016/s2215-0366(21)00254-6 · Cited by 100 · see citing works
Haines-Delmont, A., Chahal, G., Bruen, A. J., Wall, A., Khan, C. T., Sadashiv, R., & Fearnley, D. (2020) · JMIR mhealth and uhealth · doi:10.2196/15901 · Cited by 87 · see citing works
Levis, M., Leonard Westgate, C., Gui, J., Watts, B. V., & Shiner, B. (2021) · Psychological Medicine · doi:10.1017/s0033291720000173 · Cited by 87 · see citing works
Kusuma, K., Larsen, M. E., Quiroz, J. C., Gillies, M. B., Burnett, A., Qian, J., & Torok, M. H. (2022) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2022.09.050 · Cited by 90 · see citing works
Prospective Validation of an Electronic Health Record–Based, Real-Time Suicide Risk Model
Walsh, C. G., Johnson, K. B., Ripperger, M., Sperry, S., Harris, J., Clark, N., Fielstein, E., Novak, L., Robinson, K., & Stead, W. W. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.1428 · Cited by 78 · see citing works
Ebert, D. D., Harrer, M., Apolinário-Hagen, J., & Baumeister, H. (2019) · Advances in Experimental Medicine and Biology · doi:10.1007/978-981-32-9721-0_29 · Cited by 76 · see citing works
Boudreaux, E. D., Rundensteiner, E. A., Liu, F., Wang, B., Larkin, C., Agu, E., Ghosh, S., Semeter, J., Simon, G. E., & Davis-Martin, R. E. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.707916 · Cited by 76 · see citing works
Suicidal behaviour prediction models using machine learning techniques: A systematic review
Nordin, N., Zainol, Z. B., Noor, M. H. M., & Chan, L. F. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102395 · Cited by 74 · see citing works
Shortreed, S. M., Cook, A. J., Coley, R. Y., Bobb, J. F., & Nelson, J. C. (2019) · American Journal of Epidemiology · doi:10.1093/aje/kwy292 · Cited by 68 · see citing works
Ross, E. L., Zuromski, K. L., Reis, B. Y., Nock, M. K., Kessler, R. C., & Smoller, J. W. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2021.0089 · Cited by 65 · see citing works
Self-citationHostetter, T. A., Hoffmire, C. A., Forster, J. E., Adams, R. S., Stearns-Yoder, K. A., & Brenner, L. A. (2019) · Journal of Head Trauma Rehabilitation · doi:10.1097/htr.0000000000000489 · Cited by 64 · see citing works
Self-citationArtificial Intelligence: Review of Current and Future Applications in Medicine
Thomas, L. B. (2021) · Federal Practitioner · doi:10.12788/fp.0174 · Cited by 69 · see citing works
Hoffmire, C. A., Monteith, L. L., Forster, J. E., Bernhard, P. A., Blosnich, J. R., Vogt, D. S., Maguen, S., Smith, A. A., & Schneiderman, A. (2021) · Medical Care · doi:10.1097/mlr.0000000000001431 · Cited by 62 · see citing works
Self-citationCan machine-learning methods really help predict suicide?
McHugh, C. M., & Large, M. M. B. (2020) · Current Opinion in Psychiatry · doi:10.1097/yco.0000000000000609 · Cited by 64 · see citing works
Moral injury and peri- and post-military suicide attempts among post-9/11 veterans
Maguen, S., Griffin, B. J., Vogt, D., Hoffmire, C. A., Blosnich, J. R., Bernhard, P. A., Akhtar, F. Z., Cypel, Y. S., & Schneiderman, A. I. (2023) · Psychological Medicine · doi:10.1017/s0033291721005274 · Cited by 58 · see citing works
Self-citationRecent trends in the rural–urban suicide disparity among veterans using VA health care
Shiner, B. R., Peltzman, T., Cornelius, S. L., Gui, J., Forehand, J. A., & Watts, B. V. (2020) · Journal of Behavioral Medicine · doi:10.1007/s10865-020-00176-9 · Cited by 57 · see citing works
Meta-analysis of the strength of exploratory suicide prediction models; from clinicians to computers
Corke, M., Mullin, K., Angel-Scott, H. N., Xia, S., & Large, M. M. B. (2021) · BJPsych Open · doi:10.1192/bjo.2020.162 · Cited by 57 · see citing works
The association of lung ultrasound images with COVID‐19 infection in an emergency room cohort
Bar, S., Lecourtois, A., Diouf, M., Goldberg, E., Bourbon, C., Arnaud, É., Domisse, L., Dupont, H., & Gosset, P. (2020) · Anaesthesia · doi:10.1111/anae.15175 · Cited by 53 · see citing works
van Eeden, W. A., Luo, C. W., van Hemert, A. M., Carlier, I. V. E., Penninx, B. W. J. H., Wardenaar, K. J., Hoos, H. H., & Giltay, E. J. (2021) · Psychiatry Research · doi:10.1016/j.psychres.2021.113823 · Cited by 52 · see citing works
McKernan, L. C., Clayton, E. W., & Walsh, C. G. (2018) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2018.00650 · Cited by 52 · see citing works
Naifeh, J. A., Mash, H. B. H., Stein, M. B., Fullerton, C. S., Kessler, R. C., & Ursano, R. J. (2018) · Molecular Psychiatry · doi:10.1038/s41380-018-0197-z · Cited by 50 · see citing works
Self-citationIntegrating Predictive Modeling Into Mental Health Care: An Example in Suicide Prevention
Reger, G. M., McClure, M. L., Ruskin, D., Carter, S. P., & Reger, M. A. (2018) · Psychiatric Services · doi:10.1176/appi.ps.201800242 · Cited by 50 · see citing works
Mortality among U.S. military participants at eight aboveground nuclear weapons test series
Boice, J. D., Cohen, S. S., Mumma, M. T., Chen, H., Golden, A. P., Beck, H. L., & Till, J. E. (2020) · International Journal of Radiation Biology · doi:10.1080/09553002.2020.1787543 · Cited by 48 · see citing works
Haroz, E. E., Kitchen, C., Nestadt, P. S., Wilcox, H. C., DeVylder, J., & Kharrazi, H. H. K. (2021) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12800 · Cited by 47 · see citing works
Reconciling Statistical and Clinicians’ Predictions of Suicide Risk
Simon, G. E., Matarazzo, B. B., Walsh, C. G., Smoller, J. W., Boudreaux, E. D., Yarborough, B. J. H., Shortreed, S. M., Coley, R. Y., Ahmedani, B. K., Doshi, R., et al. (2021) · Psychiatric Services · doi:10.1176/appi.ps.202000214 · Cited by 44 · see citing works
Shortreed, S. M., Walker, R. L., Johnson, E., Wellman, R., Cruz, M., Ziebell, R. A., Coley, R. Y., Yaseen, Z. S., Dharmarajan, S. H., Penfold, R. B., et al. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00772-4 · Cited by 46 · see citing works
Addressing Suicide in the Veteran Population: Engaging a Public Health Approach
Carroll, D., Kearney, L. K., & Miller, M. A. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.569069 · Cited by 45 · see citing works
The Role of Big Data Analytics in Predicting Suicide
Kessler, R. C., Bernecker, S. L., Bossarte, R. M., Luedtke, A. R., McCarthy, J. F., Nock, M. K., Pigeon, W. R., Petukhova, M. V., Sadikova, E., VanderWeele, T. J., et al. (2019) · Personalized Psychiatry · doi:10.1007/978-3-030-03553-2_5 · Cited by 44 · see citing works
Self-citationSuicide in Transgender Veterans: Prevalence, Prevention, and Implications of Current Policy
Tucker, R. P. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691618812680 · Cited by 44 · see citing works
Fink, D. S., Stohl, M., Mannes, Z. L., Shmulewitz, D., Wall, M. M., Gutkind, S., Olfson, M., Gradus, J. L., Keyhani, S., Maynard, C. C., et al. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08899-y · Cited by 45 · see citing works
Artificial intelligence for dementia prevention
Newby, D., Orgeta, V., Marshall, C. R., Lourida, I., Albertyn, C. P., Tamburin, S., Raymont, V., Veldsman, M., Koychev, I., Bauermeister, S., et al. (2023) · Alzheimer's & Dementia · doi:10.1002/alz.13463 · Cited by 42 · see citing works
Danielsen, A. A., Fenger, M. H. J., Østergaard, S. D., Nielbo, K. L., & Mors, O. (2019) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.13061 · Cited by 38 · see citing works
Bentley, K. H., Zuromski, K. L., Fortgang, R. G., Madsen, E., Kessler, D. T., Lee, H., Nock, M. K., Reis, B. Y., Castro, V. M., & Smoller, J. W. (2022) · JMIR Formative Research · doi:10.2196/30946 · Cited by 33 · see citing works
What health records data are required for accurate prediction of suicidal behavior?
Simon, G. E., Shortreed, S. M., Johnson, E., Rossom, R. C., Lynch, F. L., Ziebell, R. A., & Penfold, A. R. B. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz136 · Cited by 35 · see citing works
Edgcomb, J. B., Thiruvalluru, R. K., Pathak, J., & Brooks, J. O. (2021) · Medical Care · doi:10.1097/mlr.0000000000001467 · Cited by 35 · see citing works
Ryan, A. T., Addington, J., Bearden, C. E., Cadenhead, K. S., Cornblatt, B. A., Mathalon, D. H., McGlashan, T. H., Perkins, D. O., Seidman, L. J., Tsuang, M. T., et al. (2017) · Schizophrenia Research · doi:10.1016/j.schres.2017.12.001 · Cited by 34 · see citing works
Jiang, T., Rosellini, A. J., Horváth–Puhó, E., Shiner, B. R., Street, A. E., Lash, T. L., Sørensen, H. T., & Gradus, J. L. (2021) · The British Journal of Psychiatry · doi:10.1192/bjp.2021.19 · Cited by 32 · see citing works
Self-citationClinical implementation of suicide risk prediction models in healthcare: a qualitative study
Yarborough, B. J. H., Stumbo, S. P., Schneider, J. L., Richards, J. A., Hooker, S. A., & Rossom, R. C. (2022) · BMC Psychiatry · doi:10.1186/s12888-022-04400-5 · Cited by 30 · see citing works
Predictive modeling of housing instability and homelessness in the Veterans Health Administration
Byrne, T. H., Montgomery, A. E., & Fargo, J. D. (2018) · Health Services Research · doi:10.1111/1475-6773.13050 · Cited by 30 · see citing works
Big Data From Health Records in Mental Health Care
Simon, G. E. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2018.4510 · Cited by 30 · see citing works
Levis, M., Levy, J., Dufort, V., Gobbel, G. T., Watts, B. V., & Shiner, B. R. (2022) · Psychiatry Research · doi:10.1016/j.psychres.2022.114703 · Cited by 30 · see citing works
The Veterans Health Administration REACH VET Program: Suicide Predictive Modeling in Practice
Matarazzo, B. B., Eagan, A. E., Landes, S. J., Mina, L. K., Clark, K., Gerard, G. R., McCarthy, J. F., Trafton, J. A., Bahraini, N. H., Brenner, L. A., et al. (2022) · Psychiatric Services · doi:10.1176/appi.ps.202100629 · Cited by 34 · see citing works
Self-citationHaroz, E. E., Grubin, F., Goklish, N. A., Pioche, S., Cwik, M. F., Barlow, A., Waugh, E., Usher, J., Lenert, M. C., & Walsh, C. G. (2021) · JMIR Public Health and Surveillance · doi:10.2196/24377 · Cited by 27 · see citing works
Stanley, I. H., Chu, C., Gildea, S. M., Hwang, I. H., King, A. J., Kennedy, C. J., Luedtke, A. R., Marx, B. P., O’Brien, R. W., Petukhova, M. V., et al. (2022) · Molecular Psychiatry · doi:10.1038/s41380-021-01423-4 · Cited by 30 · see citing works
Self-citationMethodological advances in statistical prediction.
Garb, H. N., & Wood, J. M. (2019) · Psychological Assessment · doi:10.1037/pas0000673 · Cited by 28 · see citing works
Haroz, E. E., Walsh, C. G., Goklish, N. A., Cwik, M. F., O’Keefe, V. M., & Barlow, A. (2019) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12598 · Cited by 25 · see citing works
Richards, J. A., Simon, G. E., Boggs, J. M., Beidas, R. S., Yarborough, B. J. H., Coleman, K. J., Sterling, S., Beck, A. L., Flores, J. P., Bruschke, C., et al. (2021) · Implementation Research and Practice · doi:10.1177/26334895211011769 · Cited by 25 · see citing works
Ji, X., Zhao, J., Fan, L., Li, H., Lin, P., Zhang, P., Fang, S., Law, S., Yao, S., & Wang, X. (2021) · Journal of Clinical Psychology · doi:10.1002/jclp.23246 · Cited by 24 · see citing works
Patient Feedback on the Use of Predictive Analytics for Suicide Prevention
Reger, M. A., Ammerman, B. A., Carter, S. P., Gebhardt, H. M., Rojas, S. M., Lee, J. M., & Buchholz, J. R. (2020) · Psychiatric Services · doi:10.1176/appi.ps.202000092 · Cited by 21 · see citing works
Atkins, D., Makridis, C. A., Alterovitz, G., Ramoni, R. B., & Clancy, C. M. (2022) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-122220-110053 · Cited by 24 · see citing works
Levis, M., Levy, J., Dent, K. R., Dufort, V., Gobbel, G. T., Watts, B. V., & Shiner, B. R. (2023) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.22m14568 · Cited by 22 · see citing works
Preventing Suicide Among Women Veterans: Gender-Sensitive, Trauma-Informed Conceptualization
Monteith, L. L., Holliday, R. P., Dichter, M. E., & Hoffmire, C. A. (2022) · Current Treatment Options in Psychiatry · doi:10.1007/s40501-022-00266-2 · Cited by 22 · see citing works
Self-citationRelative accuracy of social and medical determinants of suicide in electronic health records
Alemi, F., Avramovic, S., Renshaw, K. D., Kanchi, R., & Schwartz, M. D. (2020) · Health Services Research · doi:10.1111/1475-6773.13540 · Cited by 21 · see citing works
Luk, J. W., Pruitt, L. D., Smolenski, D. J., Tucker, J., Workman, D. E., & Belsher, B. E. (2021) · Journal of Clinical Psychology · doi:10.1002/jclp.23202 · Cited by 21 · see citing works
Prediction models for self-harm and suicide: a systematic review and critical appraisal
Seyedsalehi, A., Bailey, J. P. M., Ogonah, M. G. T., Fanshawe, T. R., & Fazel, S. (2025) · BMC Medicine · doi:10.1186/s12916-025-04367-6 · Cited by 23 · see citing works
AI-Based Prediction and Prevention of Psychological and Behavioral Changes in Ex-COVID-19 Patients
Ćosić, K., Popović, S., Šarlija, M., Kesedžić, I., Gambiraža, M., Dropuljić, B., Mijić, I., Henigsberg, N., & Jovanović, T. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.782866 · Cited by 25 · see citing works
Dhaubhadel, S., Ganguly, K., Ribeiro, R. M., Cohn, J. D., Hyman, J. M., Hengartner, N., Kolade, B., Singley, A., Tanmoy Bhattacharya, A., Finley, P. D., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-51762-9 · Cited by 20 · see citing works
Self-citationAmmerman, B. A., & Reger, M. A. (2020) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12618 · Cited by 20 · see citing works
Sex‐Specific Risk Profiles for Suicide Among Persons with Substance Use Disorders in Denmark
Adams, R. S., Jiang, T., Rosellini, A. J., Horváth–Puhó, E., Street, A. E., Keyes, K. M., Cerdá, M., Lash, T. L., Sørensen, H. T., & Gradus, J. L. (2021) · Addiction · doi:10.1111/add.15455 · Cited by 18 · see citing works
Self-citationPremature Death, Suicide, and Nonlethal Intentional Self-Harm After Psychiatric Discharge
Mortier, P., Conde, S., Alayo, I., Amigo, F., Ballester, L., Amell, R. C., Guinart, D., Contaldo, S. F., Ferrer, M., Leis, Á., et al. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.17131 · Cited by 18 · see citing works
Self-citationMachine Learning Analysis of Handgun Transactions to Predict Firearm Suicide Risk
Laqueur, H. S., Smirniotis, C., McCort, C. D., & Wintemute, G. J. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.21041 · Cited by 16 · see citing works
Evaluation of Electronic Health Record-Based Suicide Risk Prediction Models on Contemporary Data
Walker, R. L., Shortreed, S. M., Ziebell, R. A., Johnson, E., Boggs, J. M., Lynch, F. L., Daida, Y. G., Ahmedani, B. K., Rossom, R. C., Coleman, K. J., et al. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1733908 · Cited by 16 · see citing works
Rawat, B. P. S., Reisman, J., Pogoda, T. K., Liu, W., Rongali, S., Aseltine Jr, R. H., Chen, K., Tsai, J., Berlowitz, D., Yu, H., et al. (2023) · JMIR Public Health and Surveillance · doi:10.2196/42803 · Cited by 15 · see citing works
Military and Veteran Suicide Prevention
Jobes, D. A., Haddock, L. A., & Olivares, M. (2019) · Springer eBooks · doi:10.1007/978-3-030-05384-0_5 · Cited by 14 · see citing works
Yarborough, B. J. H., Stumbo, S. P., Schneider, J. L., Richards, J. A., Hooker, S. A., & Rossom, R. C. (2022) · BMC Psychiatry · doi:10.1186/s12888-022-04129-1 · Cited by 14 · see citing works
Richards, J. A., Shortreed, S. M., Simon, G. E., Penfold, R. B., Glass, J. E., Ziebell, R. A., & Williams, E. C. (2019) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2019.12.002 · Cited by 15 · see citing works
Suicide among Scottish military veterans: follow-up and trends
Bergman, B. P., Mackay, D., & Pell, J. P. (2021) · Occupational and Environmental Medicine · doi:10.1136/oemed-2021-107713 · Cited by 14 · see citing works
Suicide and the agent–host–environment triad: leveraging surveillance sources to inform prevention
Keyes, K. M., Kandula, S., Olfson, M., Gould, M. S., Martínez‐Alés, G., Rutherford, C. G., & Shaman, J. L. (2021) · Psychological Medicine · doi:10.1017/s003329172000536x · Cited by 11 · see citing works
Artificial intelligence, machine learning and mental health
Delgadillo, J., & Atzil–Slonim, D. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-91497-0.00177-6 · Cited by 14 · see citing works
Adekkanattu, P., Furmanchuk, A., Wu, Y., Pathak, A., Patra, B. G., Bost, S., Morrow, D., Wang, G. H.-M., Yang, Y., Forrest, N. J., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01266-7 · Cited by 12 · see citing works
Suicide Risk Among Veterans Who Receive Evidence-Based Therapy for Posttraumatic Stress Disorder
Saulnier, K. G., Brabbs, S., Szymanski, B. R., Harpaz‐Rotem, I., McCarthy, J. F., & Sripada, R. K. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.52144 · Cited by 11 · see citing works
Self-citationPredictive risk modelling in mental health issues using machine learning on graphs
Lu, H., Uddin, S., Hajati, F., Khushi, M., & Moni, M. A. (2022) · Australasian Computer Science Week Multiconference (ACSW) · doi:10.1145/3511616.3513112 · Cited by 11 · see citing works
Meerwijk, E. L., Tamang, S. R., Finlay, A. K., Ilgen, M. A., Reeves, R., & Harris, A. H. S. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-065088 · Cited by 10 · see citing works
Luci, K., Simons, K. V., Hagemann, L., Jacobs, M. L., Bower, E. S., Eichorst, M. K., & Hilgeman, M. M. (2019) · Clinical Gerontologist · doi:10.1080/07317115.2019.1666444 · Cited by 11 · see citing works
Gromatsky, M. A., Edwards, E. R., Sullivan, S. R., Lissa, C. J. V., Lane, R., Spears, A. P., Mitchell, E. L., Armey, M. F., Cáceda, R., & Goodman, M. S. (2022) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2022.02.016 · Cited by 11 · see citing works
Suicide Death Prediction Using the Maryland Suicide Data Warehouse: A Sensitivity Analysis
Kitchen, C., Zirikly, A., Belouali, A., Kharrazi, H. H. K., Nestadt, P. S., & Wilcox, H. C. (2024) · Archives of Suicide Research · doi:10.1080/13811118.2024.2363227 · Cited by 13 · see citing works
Accurate Prediction and Detection of Suicidal Risk using Random Forest Algorithm
Saravanan, N., Moheshkumar, G., Shaid, V. M. M., Purushothman, S., & Sanjai, V. G. (2024) · International Conference on Pervasive Computing and Social Networking (ICPCSN) · doi:10.1109/icpcsn62568.2024.00053 · Cited by 12 · see citing works
The Application of AI to Ecological Momentary Assessment Data in Suicide Research: Systematic Review
Melia, R., Schafer, K. M., Rogers, M. L., Wilson, E., & Joiner, T. E. (2025) · Journal of Medical Internet Research · doi:10.2196/63192 · Cited by 11 · see citing works
Zelkowitz, R. L., Jiang, T., Horváth–Puhó, E., Street, A. E., Lash, T. L., Sørensen, H. T., Rosellini, A. J., & Gradus, J. L. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.03.034 · Cited by 11 · see citing works
Self-citation
- Authors
- Kessler, R. C., Hwang, I., Hoffmire, C. A., McCarthy, J. F., Petukhova, M. V., Rosellini, A. J., Sampson, N. A., Schneider, A. L., Bradley, P. A., Katz, I. R., Thompson, C., & Bossarte, R. M.
- Affiliations
- Harvard University (US)
- VA Rocky Mountain Network (US)
- VA Center for Clinical Management Research (US)
- Boston University (US)
- PricewaterhouseCoopers (United States) (US)
- United States Department of Veterans Affairs (US)
- West Virginia University (US)
- Corresponding author
- Kessler, Ronald C.
- ORCID
- Venue
- International Journal of Methods in Psychiatric Research
- Publisher
- Wiley
- Published
- 2017-07-04
- Volume
- 26
- Issue
- 3
- ISSN
- 1049-8931 (ISSN-L)
- 1557-0657
- PMC
- PMC5614864
- OpenAlex
- W2725461439
- Cited by
- 209
- References
- 37
- Citations by year
- 2026: 19 · 2025: 29 · 2024: 21 · 2023: 16 · 2022: 31 · 2021: 36 · 2020: 18 · 2019: 23 · 2018: 14 · 2017: 2
- FWCI
- 19.33
- Type
- journal article
- Language
- en
- Open access
- green
- europepmc.org
- Locations
- International Journal of Methods in Psychiatric Research (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- The Research Repository @ WVU (West Virginia University) (submittedVersion)
- Funders
- U.S. Department of Veterans Affairs (VA118‐14‐C‐0046)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Suicide and Self-Harm Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- suicide risk prediction, veterans, Veterans Health Administration, penalized logistic regression, Bayesian additive regression trees, machine learning, suicide mortality, electronic health records
- MeSH terms
- Adolescent, Adult, Aged, Aged, 80 and over, Female, Humans, Male, Middle Aged, Suicide, United States, United States Department of Veterans Affairs, Models, Statistical, Risk Assessment, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
