Using Administrative Data to Predict Suicide After Psychiatric Hospitalization in the Veterans Health Administration System

May combine two claims.

The prediction models had AUC .79-.82 for time horizons between 1 week and 6 months and AUC .74 for 12 months.

“The models had AUC=.79–.82 for time horizons between 1 week and 6 months and AUC=.74 for 12 months.”

May combine two claims.

Variable importance analysis shows that 51.1% of model performance is due to psychopathological risk factors, 26.2% to social determinants of health, 14.8% to prior history of suicidal behaviors, and 6.6% to physical disorders.

“Variable importance analysis shows that 51.1% of model performance is due to psychopathological risk factors accounted, 26.2% to social determinants of health, 14.8% to prior history of suicidal behaviors, and 6.6% to physical disorders.”

May combine two claims.

At the 5% threshold, these patients accounted for 24.1% of all suicides in the holdout sample in the 1 week after hospital discharge, 32.2% in 1 month, 26.9% in 3 months, 26.4% in 6 months, and 22.4% in 12 months.

“The sensitivities at this threshold show that these patients accounted for 24.1% of all suicides in the holdout sample that occurred in the 1 week after hospital discharge, 32.2% in 1 month, 26.9% in 3 months, 26.4% in 6 months, and 22.4% in 12 months”

May combine two claims.

In the 1-week model, non-Hispanic Whites had an OR of 4.2 and 'other' race/ethnicity had an OR of 5.3 for suicide compared to non-Hispanic Blacks.

“race/ethnicity, with significant ORs of 4.2 for non-Hispanic Whites and 5.3 for “other” race/ethnicity (not non-Hispanic White, non-Hispanic Black, and other Hispanic) compared to non-Hispanic Blacks”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The majority were non-Hispanic White (60.1%-61.3%) and others mostly non-Hispanic Black (24.3%-24.9%).

“The majority were non-Hispanic White (60.1%–61.3%) and others mostly non-Hispanic Black (24.3%–24.9%).”

May combine two claims.

The 12-month model had higher AUC than other models for each shorter time horizon (AUC = .79 versus .67-.77 to predict suicides within the first week; .82 versus .63-.77 within the first month; .78 versus .61-.73 within 3 months; .80 versus .61-.74 within 6 months; .74 versus .60-.71 within 12 months).

“AUC = .79 versus .67–.77 to predict suicides within the first week of hospital discharge, AUC = .82 versus .63–.77 to predict suicides within the first month of discharge, AUC = .78 versus .61–.73 to predict suicides within 3 months of discharge, AUC = .80 versus .61–.74 to predict suicides within 6 months of discharge, and AUC = .74 versus .60–.71 to predict suicides within 12 months of discharge”

Probably not a substantial claim: may not pass S4 (based).

Conditional suicide risk is much higher in the first weeks and months after hospital discharge.

“conditional suicide risk is much higher in the first weeks and months after hospital discharge.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The feature selection process resulted in the selection of 1,221 features for the 1-week model, 2,411 for the 1-month model, 4,074 for the 3-month model, 5,675 for the 6-month model, and 8,071 for the 12-month model.

“This process resulted in the selection of 1,221 features for the 1-week model, 2,411 for the 1-month model, 4,074 for the 3-month model, 5,675 for the 6-month model, and 8,071 for the 12-month model.”

May combine two claims.

Age had ORs of 0.4-0.5 for the two youngest quintiles, increasing to 0.6-0.8 over longer time horizons; and nonsignificant ORs of 0.6-0.9 for the two oldest quintiles, decreasing to significant OR=0.7 in the 12-month model.

“age (OR=0.4–0.5 for the two youngest quintiles, increasing to 0.6–0.8 over longer time horizons; and nonsignificant OR=0.6–0.9 for the two oldest quintiles, decreasing to significant OR=0.7 in the 12-month model)”

May combine two claims.

Compared to all patients making VHA visits for any reason over the same time periods weighted by number of visits, the stratification variables most strongly associated with psychiatric hospitalization were ages 20-55, never married, separated-divorced, post-Vietnam era, Persian Gulf War era, currently or recently homeless, and having a high risk flag.

“the stratification variables most strongly associated with psychiatric hospitalization were ages 20–55 (55.4%–57.7% versus 26.7%–27.6%), never married (23.0%–23.3% versus 12.4%–12.6%), separated-divorced (47.3%–61.7% versus 31.5%–31.8%), post-Vietnam era (22.9%–25.1% versus 12.0%–13.4%), Persian Gulf War era (29.9%–36.3% versus 18.0%–20.5%), currently or recently homeless (35.6%–41.1% versus 8.7%–”

May combine two claims.

Extreme gradient boosting was one of the top 5 classifiers in all models, support vector machines (with varying kernels) in 4 of the 5, generalized linear models in 4 of the 5, Bayesian additive regression trees in 1 of the 5, and elastic net in 1 of the 5.

“Extreme gradient boosting was one of the top 5 classifiers in all models, support vector machines (with varying kernels) in 4 of the 5, generalized linear models in 4 of the 5, Bayesian additive regression trees in 1 of the 5, and elastic net in 1 of the 5.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The plurality served most recently either in the Vietnam era (37.1%-39.4%), followed by the Persian Gulf War era (29.9%-36.3%) and the years between the Vietnam and Persian Gulf War eras (22.9%-25.1%).

“the plurality served most recently either in the Vietnam era (37.1%–39.4%), followed by the Persian Gulf War era (29.9%–36.3%) and the years between the Vietnam and Persian Gulf War eras (22.9%–25.1%).”

May combine two claims.

Most of these predictors remained significant in models for longer time horizons (OR=2.4-3.0 for non-Hispanic White; OR=3.4-4.5 for 'other' race/ethnicity; OR=0.26-0.5 for Baptists, OR=1.8-2.6 high risk flag).

“Most of these predictors remained significant in models for longer time horizons (OR=2.4–3.0 for non-Hispanic White; OR=3.4–4.5 for “other” race/ethnicity; OR=0.26–0.5 for Baptists, OR=1.8–2.6 high risk flag)”

May combine two claims.

In the 1-week model, Black and other Baptists had significantly reduced ORs of 0.00-0.17 and 'other' Christians had an OR of 0.12 compared to other Protestants.

“religion, with significantly reduced ORs of 0.00–0.17 for Black and other Baptists and 0.12 for “other” Christians (not Roman Catholic) compared to other Protestants”

May combine two claims.

A search of FDA drug labeling documents found 49 medications with suicide as an adverse reaction in the box warning, 112 in warnings and precautions, and 79 in adverse reactions.

“we searched Food and Drug Administration approved drug labeling documents in the Food and Drug Administration Label Database (67) using the search terms suicidality, suicidal behavior, suicidal ideation, suicide attempt, suicidal, and suicide and found 49 medications that indicated suicide as an adverse reaction in the box warning section of the drug label, 112 that included suicide in the”

• cites [67] 2019Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The great majority of patients in both the training sample and the holdout sample were male (92.6%-93.4%) and had a median age of 54.

“The great majority of patients in both the training sample and the holdout sample were male (92.6%–93.4%) and had a median age of 54.”

May combine two claims.

Most recent era of active duty service: Persian Gulf War era had OR=2.6 in the 1-month model and OR=2.0-2.3 in longer time horizons; Pre-Vietnam era had nonsignificant OR=1.0 that became significant OR=1.9 in the 12-month model; Vietnam era had nonsignificant OR=0.9 that became significant ORs=1.5-1.8 in the 6- and 12-month models compared to the post-Vietnam era.

“most recent era of active duty service (OR=2.6 for Persian Gulf War era in the 1-month model and OR=2.0–2.3 in models for longer time horizons; a nonsignificant OR=1.0 for the Pre-Vietnam era that became significant OR=1.9 in the 12-month model; and a nonsignificant OR=0.9 for the Vietnam era that became significant ORs=1.5–1.8 in the 6- and 12-month models compared to the post-Vietnam era)”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The plurality was either divorced (38.4%-38.8%) and the others mostly either married (24.3%-26.0%) or never married (23.0%-23.3%).

“The plurality was either divorced (38.4%–38.8%) and the others mostly either married (24.3%–26.0%) or never married (23.0%–23.3%).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The majority lived in Census-defined metropolitan statistical areas of more than 1 million residents (51.4%-53.5%) or 250,000-1 million residents (23.1%-23.8%) and only a small proportion (8.0%-9.8%) lived in areas with populations of less than 20,000.

“The majority lived in Census-defined metropolitan statistical areas of more than 1 million residents (51.4%–53.5%) or 250,000–1 million residents (23.1%–23.8%) and only a small proportion (8.0%–9.8%) lived in areas with populations of less than 20,000.”

May combine two claims.

Neural network was the best classifier for the 1-week model and random forest was best for the other models and second most important for the 1-week model.

“Neural network was the best classifier for the 1-week model and random forest was best for the other models and second most important for the 1-week model”

May combine two claims.

Male gender became significant in the 1-month model with OR=3.3, decreasing from 2.2 to 1.3 over longer time horizons.

“male gender (OR=3.3, decreasing from 2.2 to 1.3 over longer time horizons)”

May combine two claims.

Homelessness became significant in the 3-month model: currently homeless had OR=0.7 remaining significant OR=0.6-0.07 in the 6- and 12-month models; recently homeless had OR=0.7 remaining significant in the 6-month model OR=0.7 but not in the 12-month model OR=0.9.

“Homelessness became significant in the 3-month model (OR=0.7 for currently homeless remaining significant OR=0.6–0.07 in the 6- and 12-month models; OR=0.7 for recently homeless remaining significant in the 6-month model OR=0.7 but not in the 12-month model OR=0.9).”

May combine two claims.

A strikingly high 35.6%-41.1% had been homeless at some time in the 12 months before hospitalization, including 16.1%-19.0% homeless at time of admission.

“A strikingly high 35.6%–41.1% had been homeless at some time in the 12 months before hospitalization, including 16.1%–19.0% homeless at time of admission.”

May combine two claims.

Marital status: never married had OR=2.1 increasing to OR=2.5-2.9 in the 6- and 12-month models; currently married had nonsignificant OR=1.1-1.3 in the 1- and 3-month models increasing to significant OR=1.6-1.7 in the 6- and 12-month models compared to currently separated.

“marital status (OR=2.1 for never married increasing to OR=2.5–2.9 in the 6- and 12-month models; and nonsignificant OR=1.1–1.3 for currently married in the 1- and 3-month models increasing to significant OR=1.6–1.7 in the 6- and 12-month models compared to currently separated)”

May combine two claims.

The observed suicide rate at the level of hospitalization over the 12 months after hospital discharge was 315.5 per 100,000 person-years in the training sample and 282.5 in the holdout sample.

“The observed suicide rate at the level of hospitalization over the 12 months after hospital discharge was 315.5 per 100,000 person-years in the training sample (Figure 1A) and 282.5 in the holdout sample (Figure 1B).”

May combine two claims.

The other categories of predictors were much less important (physical disorders 6.6%; medications classified by the US Food and Drug Administration as potential risk factors for suicide 0.7%).

“The other categories of predictors were much less important (physical disorders 6.6%; medications classified by the US Food and Drug Administration as potential risk factors for suicide 0.7%).”

May combine two claims.

Sorting these predictors into categories shows that psychopathological risk factors were most important (accounting for 51.1% of overall model performance) followed by social determinants of health (26.2% of overall model performance, including 11.8% for socio-demographics, 2.9% for V codes, and 11.5% for small area geocode data) and history of suicidal behaviors (14.8%).

“Sorting these predictors into categories shows that psychopathological risk factors were most important (accounting for 51.1% of overall model performance) followed by social determinants of health (26.2% of overall model performance, including 11.8% for socio-demographics, 2.9% for V codes, and 11.5% for small area geocode data) and history of suicidal behaviors (14.8%)”

May combine two claims.

The top 10 predictors accounted for 43.7% of overall model performance based on the gain metric in the Extreme Gradient Boosting algorithm, the top 25 predictors for 58.1%, the top 50 predictors for 70.3%, and the top 100 predictors for 81.4% of model performance.

“The top 10 predictors accounted for 43.7% of overall model performance based on the gain metric in the Extreme Gradient Boosting algorithm (77), the top 25 predictors for 58.1%, the top 50 predictors for 70.3%, and the top 100 predictors for 81.4% of model performance.”

• cites [77] Chen, 2016, XGBoost: a scalable tree boosting system, arXiv: sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The gross association of homelessness with suicide is positive, but the OR for homelessness in the multivariate model suggests that homelessness is protective.

“The gross association of homelessness with suicide is positive, but the OR for homelessness in the multivariate model suggests that homelessness is protective.”

May combine two claims.

Bipolar disorder was a powerful predictor when considered alone in the initial screening models, but other predictors more efficiently captured the variance due to bipolar disorder in the SL analysis.

“bipolar disorder was a powerful predictor when considered alone in the initial screening models, but other predictors more efficiently captured the variance due to bipolar disorder in the SL analysis.”

May combine two claims.

Patients with bipolar disorder had a stronger history of suicidality and more risk factors involving social determinants of health than other patients.

“patients with bipolar disorder had a stronger history of suicidality and more risk factors involving social determinants of health than other patients.”

May combine two claims.

Psychopathological risk factors and prior suicidality were important predictors in our model.

“our finding that psychopathological risk factors and prior suicidality were important is not surprising.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Most VHA patients are males, which is a unique characteristic of our sample.

“The fact that most VHA patients are males is another unique characteristic of our sample.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Person-level data were weighted by the inverse of their probabilities of selection for analysis and population projection.

“The person-level data were weighted by the inverse of their probabilities of selection for purposes of analysis and population projection.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Recent UK research found that quality of care indicators, such as staff turnover and short average duration of stay, are significant predictors of postdischarge suicides.

“Recent research in the UK has found that quality of care indicators, such as extent of staff turnover and short average duration of stay, are significant predictors of postdischarge suicides (5, 14).”

• cites [5] Hunt, 2009, Suicide in recently discharged psychiatric patients: a case-control study, Psychol Med sourceNot checked yet

  Not checked in the free run.

• cites [14] Kapur, 2016, Mental health service changes, organisational factors, and patient suicide in England in 1997-2012: a before-and-after study, Lancet Psychiatry sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The case definition excludes deaths by other external cause, some of which might have been misclassified suicides.

“This case definition excludes deaths by other external cause (i.e., murder or other accidents), some of which might have been misclassified suicides.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

A postdischarge case management program would be more scalable if it focused on recently discharged patients at high suicide risk and was implemented remotely by centralized program staff.

“Such a program would be more scalable, though, if it focused on recently discharged patients at high suicide risk and was implemented remotely by centralized program staff to increase efficiency.”

Classified as probably not a substantial claim.

More than 50% of all suicides in the 12 months after hospital discharge in the Army STARRS study occurred among the 10% of soldiers defined as being at highest risk.

“More than 50% of all suicides in the 12 months after hospital discharge in that study occurred among the 10% of soldiers defined as being at highest risk.”

• cites [20] Kessler, 2015, Predicting suicides after psychiatric hospitalization in US Army soldiers: the Army Study To Assess Risk and Resilience in Servicemembers (Army STARRS), JAMA Psychiatry sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Small-area characteristics cluster into two dimensions associated with variation in suicide rates: neighborhood deprivation and neighborhood fragmentation.

“previous research has shown to cluster into two dimensions associated with variation in suicide rates: neighborhood deprivation (22 indicators; e.g., low median education, high unemployment and poverty rates, percent of households receiving public assistance) and neighborhood fragmentation (5 indicators: proportions of households with single-person occupant, vacant, occupant unmarried, occupant”

• cites [51] Evans, 2004, Social fragmentation, severe mental illness and suicide, Soc Psychiatry Psychiatr Epidemiol sourceNot checked yet

  Not checked in the free run.

• cites [52] Steelesmith, 2019, Contextual factors associated with county-level suicide rates in the United States, 1999 to 2016, JAMA Netw Open sourceNot checked yet

  Not checked in the free run.

• cites [53] Whitley, 1999, Ecological study of social fragmentation, poverty, and suicide, BMJ sourceNot checked yet

  Not checked in the free run.

• cites [54] Fontanella, 2018, Mapping suicide mortality in Ohio: a spatial epidemiological analysis of suicide clusters and area level correlates, Prev Med sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The 70% of case hospitalizations with earliest discharge dates and all control hospitalizations up to the same discharge date formed the training sample; the remaining 30% of cases and controls were held out to validate the model.

“The 70% of case hospitalizations with the earliest discharge dates were combined with all control hospitalizations up to the same discharge date to create a training sample in which the prediction model was developed. The remaining 30% of cases and controls were held out to validate the model.”

May combine two claims.

Super Learner has a guarantee to be at least as accurate and typically has considerably higher prediction accuracy than the best-performing algorithm in the ensemble.

“SL has a guarantee to be at least as accurate and typically has a considerably higher level of prediction accuracy than the best-performing algorithm in the ensemble.”

• cites [72] SuperLearner: Super Learner Prediction [computer program] PolleyE LeDellE van der LaanM 2016Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The study protocol was approved by the Research Ethics Committee of the Veterans Administration Center of Excellence for Suicide Prevention and Harvard Medical School with a waiver of informed consent based on deidentified data.

“The study protocol was approved by Research Ethics Committee of the Veterans Administration Center of Excellence for Suicide Prevention and Harvard Medical School with a waiver of informed consent based on the data being deidentified.”

May combine two claims.

A case management program delivered to the 20% or 60% of patients with highest predicted risk would capture 55.2%-66.1% (20% decision threshold) and 90.3%-100% (60% decision rule) of the patients who would otherwise die by suicide at one or more of the time horizons.

“A case management program delivered to the 20% or 60% of patients with highest predicted risk would capture 55.2%–66.1% (20% decision threshold) and 90.3%–100% (60% decision rule) of the patients who would otherwise die by suicide at one or more of the time horizons.”

Classified as probably not a substantial claim.

Individual-neighborhood differences sometimes predict suicides.

“Based on evidence that individual-neighborhood differences sometimes predict suicides (55, 56), we created interactions of patient race/ethnicity with the percent of neighborhood residents who were of the same race/ethnicity”

• cites [55] Wetherall, 2015, Explaining the income and suicidality relationship: income rank is more strongly associated with suicidal thoughts and attempts than income, Soc Psychiatry Psychiatr Epidemiol sourceNot checked yet

  Not checked in the free run.

• cites [56] Liu, 2017, To Compare is to despair? a population-wide study of neighborhood composition and suicide in Stockholm, Soc Probl sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Five-fold cross-validation was used for internal Super Learner cross-validation to build optimal models with each classifier and determine optimal weighting across classifiers.

“Five-fold cross-validation was used for internal SL cross-validation both to build optimal models with each classifier and to determine optimal weighting across classifiers in the ensemble.”

Probably not a substantial claim: may not pass S2 (checkable).

This comparatively high adjusted PPV speaks to the potential value of special interventions focused on the high rates of imminent risk among patients shortly after discharge.

“This comparatively high adjusted PPV speaks to the potential value of special interventions focused on the high rates of imminent risk among patients shortly after discharge.”

Classified as probably not a substantial claim.

The sample was stratified to match the population of all psychiatric hospital admissions over the study period on the cross-classification of diverse socio-demographic and geographic variables as well as on a measure of whether the patient had a suicide risk flag on their medical record in the month prior to their hospitalization.

“the sample was stratified to match the population of all psychiatric hospital admissions over the study period on the cross-classification of diverse socio-demographic and geographic variables as well as on a measure of whether the patient had a suicide risk flag on their medical record in the month prior to their hospitalization”

Probably not a substantial claim: may not pass S3 (load-bearing).

Some medications for physical disorders might predispose to suicide.

“Literature suggests that some medications for physical disorders might predispose to suicide (65, 66).”

• cites [65] Gorton, 2016, Non-psychotropic medication and risk of suicide or attempted suicide: a systematic review, BMJ Open sourceNot checked yet

  Not checked in the free run.

• cites [66] Gupta, 2016, Adverse psychiatric effects of non-psychotropic medications, BJPsych Adv sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Machine learning methods have considerable promise for predicting suicide, although prior studies were limited in a number of ways.

“Results showed that ML methods have considerable promise even though all these studies were limited in a number of ways discussed elsewhere (71).”

• cites [71] Kessler, 2019, The Role of Big Data Analytics in Predicting Suicide sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Adjusted PPV decreases as the time horizon increases when conditional suicide risk decreases in this same way.

“However, adjusted PPV decreases as the time horizon increases when conditional suicide risk decreases in this same way.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Missing values were handled by nearest neighbor imputations when values were nonmissing in earlier records, and remaining missing values and inconsistencies were reconciled using rational imputations.

“The small amount of missing values found in this data often were nonmissing in earlier records, allowing nearest neighbor imputations. Remaining missing values and inconsistencies were reconciled using rational imputations”

Classified as probably not a substantial claim. May combine two claims.

Strategies involving under-sampling controls, pseudo-sampling cases, and combinations have been developed to address class imbalance and shown to improve model performance.

“A number of strategies involving under-sampling controls, pseudo-sampling cases, and combinations have been developed to address this problem (83) and shown to improve model performance [e.g., (84, 85)].”

• cites [83] Chawla, 2010, Data mining for imbalanced datasets: An overviewNot checked yet

  Not checked in the free run.

• cites [84] Lee, 2014, Resampling methods improve the predictive power of modeling in class-imbalanced datasets, Int J Environ Res Public Health sourceNot checked yet

  Not checked in the free run.

• cites [85] Rahman, 2013, Addressing the class imbalance problem in medical datasets, Int J Mach Learn Comput sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Initial feature pruning can improve out-of-sample model performance substantially.

“This kind of initial feature pruning can improve out-of-sample model performance substantially (80).”

• cites [80] Chandrashekar, 2014, A survey on feature selection methods, Comput Electrical Eng sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The remaining 30% of case hospitalizations (n=331) and the associated controls were held out to validate the model.

“The remaining 30% of case hospitalizations (n=331) and the associated controls were held out to validate the model.”

Probably not a substantial claim: may not pass S3 (load-bearing).

All five replicates of a stratified 20% of case records were included in a single five-fold cross-validation fold to address overfitting when cases are duplicated.

“All five replicates of a stratified 20% of case records were included in a single five-fold cross-validation fold in order to address the problem that overfitting can occur when cases are duplicated.”

Classified as probably not a substantial claim.

Economic trends are associated with trends in suicide rates.

“Finally, based on evidence that economic trends are associated with trends in suicide rates (60), we obtained County-level data on the bankruptcy rate, the median debt-to-income ratio, and the unemployment rate averaged over the past 3–14 months.”

• cites [60] Reeves, 2014, Economic suicides in the Great Recession in Europe and North America, Br J Psychiatry sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Food insufficiency is more important than low income in predicting suicidality.

“Based on evidence that food insufficiency is more important than low income in predicting suicidality (57), data were obtained from the Department of Agriculture on the percent of the neighborhood that was a food desert (58).”

• cites [57] Alaimo, 2002, Family food insufficiency, but not low family income, is positively associated with dysthymia and suicide symptoms in adolescents, J Nutr sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Individuals who visit an emergency department with suicide intent, visit an emergency department after a suicide attempt, or are hospitalized for a psychiatric problem account for nearly one-third of all US suicides.

“such individuals account for nearly one-third of all US suicides (92).”

• cites [92] Schaffer, 2016, Population-based analysis of health care contacts among suicide decedents: identifying opportunities for more targeted suicide prevention strategies, World Psychiatry sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

About 2% of the US population in a given year either visit an emergency department with suicide intent, visit an emergency department after a suicide attempt, or are hospitalized for a psychiatric problem.

“Although only about 2% of the US population in a given year either visit an emergency department with suicide intent, visit an emergency department after a suicide attempt, or are hospitalized for a psychiatric problem”

• cites [92] Schaffer, 2016, Population-based analysis of health care contacts among suicide decedents: identifying opportunities for more targeted suicide prevention strategies, World Psychiatry sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The suicide rate after psychiatric hospital discharge is between 3,000/100,000 person-years in the first week after discharge and 650/100,000 person-years 4-12 months after discharge.

“that the suicide rate after psychiatric hospital discharge is between 3,000/100,000 person-years in the first week after discharge and 650/100,000 person-years 4–12 months after discharge (49, 91)”

• cites [49] Chung, 2017, Suicide rates after discharge from psychiatric facilities: a systematic review and meta-analysis, JAMA Psychiatry sourceNot checked yet

  Not checked in the free run.

• cites [91] Chung, 2019, Meta-analysis of suicide rates in the first week and the first month after psychiatric hospitalisation, BMJ Open sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Statistical models have long been known to be superior to unstructured clinical judgments in predicting a wide range of clinical outcomes, although varying over settings and decisions.

“Statistical models have long been known to be superior to unstructured clinical judgments in predicting a wide range of clinical outcomes (107), although varying over settings and decisions (108).”

• cites [107] Dawes, 1989, Clinical versus actuarial judgment, Science sourceNot checked yet

  Not checked in the free run.

• cites [108] Ægisdóttir, 2006, The meta-analysis of clinical judgment project: fifty-six years of accumulated research on clinical versus statistical prediction, Couns Psychol sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The suicide rate among emergency department patients presenting with suicide intent or after a suicide attempt is about 1,600/100,000 within 1 year of the emergency department visit.

“Meta-analyses suggest that the suicide rate among emergency department patients presenting with suicide intent or after a suicide attempt is about 1,600/100,000 within 1 year of the emergency department visit (89)”

• cites [89] Carroll, 2014, Hospital presenting self-harm and risk of fatal and non-fatal repetition: systematic review and meta-analysis, PloS One sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The 10% of ex-inpatients with highest risk account for approximately 4% of all VHA suicides, which is roughly twice as high a proportion as in the Reach Vet model.

“which means that the 10% of these ex-inpatients with highest risk account for approximately 4% of all VHA suicides (i.e., 35% of 12%), which is roughly twice as high a proportion as in the Reach Vet model.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

PPV will inevitably be very low for a rare outcome such as suicide for any decision threshold that includes more than a very small proportion of patients.

“PPV will inevitably be very low for a rare outcome such as suicide for any decision threshold that includes more than a very small proportion of patients.”

May combine two claims.

Intensive postdischarge case management programs, which are not used in VHA, have been shown elsewhere to be effective in reducing suicides after psychiatric hospital discharge.

“intensive postdischarge case management programs, which are not used in VHA, have been shown elsewhere to be effective in reducing suicides after psychiatric hospital discharge (9–16)”

• cites [9] Miller, 2016, The Coping Long Term with Active Suicide Program: description and pilot data, Suicide Life Threat Behav sourceNot checked yet

  Not checked in the free run.

• cites [10] Knox, 2012, An emergency department-based brief intervention for veterans at risk for suicide (SAFE VET), Am J Public Health sourceNot checked yet

  Not checked in the free run.

• cites [11] Matarazzo, 2017, Connecting veterans at risk for suicide to care through the HOME Program, Suicide Life Threat Behav sourceNot checked yet

  Not checked in the free run.

• cites [12] Stanley, 2018, Comparison of the safety planning intervention with follow-up vs usual care of suicidal patients treated in the emergency department, JAMA Psychiatry sourceNot checked yet

  Not checked in the free run.

• cites [13] Anderson, 2006, The national suicide prevention strategy for England: the reality of a national strategy for the nursing profession, J Psychiatr Ment Health Nurs sourceNot checked yet

  Not checked in the free run.

• cites [14] Kapur, 2016, Mental health service changes, organisational factors, and patient suicide in England in 1997-2012: a before-and-after study, Lancet Psychiatry sourceNot checked yet

  Not checked in the free run.

• cites [15] While, 2012, Implementation of mental health service recommendations in England and Wales and suicide rates, 1997-2006: a cross-sectional and before-and-after observational study, Lancet sourceNot checked yet

  Not checked in the free run.

• cites [16] De Leo, 2007, Intensive case management in suicide attempters following discharge from psychiatric care, Aust J Prim Health sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The suicide rate among inpatients is about 150/100,000 inpatient-years.

“that the suicide rate among inpatients is about 150/100,000 inpatient-years (90)”

• cites [90] Walsh, 2015, Meta-analysis of suicide rates among psychiatric in-patients, Acta Psychiatr Scand sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The patients at highest suicide risk are not necessarily the patients most likely to be helped by existing interventions.

“that the patients at highest suicide risk are not necessarily the patients most likely to be helped by existing interventions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The fact that social determinants of health indicators were important predictors does not necessarily mean that they would be useful intervention targets, as they might be risk markers rather than causal risk factors.

“the fact that social determinants of health indicators were important predictors does not necessarily mean that they would be useful intervention targets, as they might be risk markers rather than causal risk factors (96).”

• cites [96] Kraemer, 1997, Coming to terms with the terms of risk, Arch Gen Psychiatry sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

It is unclear whether hospitalization (which occurs after 50% of VHA outpatient SAs) reduces subsequent SRBs (either suicide or subsequent SAs).

“As detailed elsewhere (71), it is unclear whether hospitalization (which occurs after 50% of VHA outpatient SAs) reduces subsequent SRBs (either suicide or subsequent SAs).”

• cites [71] Kessler, 2019, The Role of Big Data Analytics in Predicting Suicide sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

These other outcomes might be reduced by intensive postdischarge interventions designed to reduce suicide death.

“These other outcomes might be reduced by intensive postdischarge interventions designed to reduce suicide death.”

May combine two claims.

The VHA Reach Vet model is based on a 30-day time horizon, which we showed clearly to be suboptimal in the current application to inpatients.

“the VHA Reach Vet model (88), which is used to target preventive interventions to the roughly 35,000 VHA patients each year considered to be at highest suicide risk, is based on a 30-day time horizon, which we showed clearly to be suboptimal in the current application to inpatients.”

• cites [88] McCarthy, 2015, Predictive modeling and concentration of the risk of suicide: implications for preventive interventions in the US Department of Veterans Affairs, Am J Public Health sourceNot checked yet

  Not checked in the free run.

May combine two claims.

It is infeasible to use controlled treatment trials to study the aggregate effects of hospitalization decisions after SAs given the large samples required and the rarity of SRBs other than among high-risk patients for whom randomization would be unethical.

“It is infeasible to use controlled treatment trials to study the aggregate effects of these decisions given the large samples required (130) and rarity of SRBs other than among high-risk patients for whom randomization would be unethical.”

• cites [130] Luedtke, 2019, Sample size requirements for multivariate models to predict between-patient differences in best treatments of major depressive disorder, Clin Psychol Sci sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

ML methods with a single classifier were used in most recent studies aimed at building suicide prediction models in high-risk patient populations using predictors that included patient self-report scales, clinical rating scales, and administrative data.

“As reviewed elsewhere, ML methods with a single classifier were used in most recent studies aimed at building suicide prediction models in these high-risk patient populations using predictors that included information collected from patient self-report scales, clinical rating scales, and administrative data (71).”

• cites [71] Kessler, 2019, The Role of Big Data Analytics in Predicting Suicide sourceNot checked yet

  Not checked in the free run.