Personalized prognostic prediction of treatment outcome for depressed patients in a naturalistic psychiatric hospital setting: A comparison of machine learning approaches.
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: Research on predictors of treatment outcome in depression has largely derived from randomized clinical trials involving strict standardization of treatments, stringent patient exclusion criteria, and careful selection and supervision of study clinicians. The extent to which findings from such studies generalize to naturalistic psychiatric settings is unclear. This study sought to predict depression outcomes for patients seeking treatment within an intensive psychiatric hospital setting and while comparing the performance of a range of machine learning approaches. METHOD: Depressed patients (N = 484; ages 18-72; 89% White) receiving treatment within a psychiatric partial hospital program delivering pharmacotherapy and cognitive behavioral therapy were split into a training sample and holdout sample. First, within the training sample, 51 pretreatment variables were submitted to 13 machine learning algorithms to predict, via cross-validation, posttreatment Patient Health Questionnaire-9 depression scores. Second, the best performing modeling approach (lowest mean squared error; MSE) from the training sample was selected to predict outcome in the holdout sample. RESULTS: The best performing model in the training sample was elastic net regularization (ENR; MSE = 20.49, R2 = .28), which had comparable performance in the holdout sample (MSE = 11.26; R2 = .38). There were 14 pretreatment variables that predicted outcome. To demonstrate the translation of an ENR model to personalized prediction of treatment outcome, a patient-specific prognosis calculator is presented. CONCLUSIONS: Informed by pretreatment patient characteristics, such predictive models could be used to communicate prognosis to clinicians and to guide treatment planning. Identified predictors of poor prognosis may suggest important targets for intervention. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Kuhn, M., & Johnson, K. (2013) · doi:10.1007/978-1-4614-6849-3 · Cited by 6,788 · cited 1×
“In an effort to minimize overfitting and identify a model that would best generalize to a holdout sample, we ran each of the 14 algorithms within 10-folds cross-validation, such that the training dataset (n = 387) was split into ten equal size samples (Kuhn & Johnson, 2013).”
MethodMissForest—Non-parametric missing value imputation for mixed-type data
Stekhoven, D. J., & Bühlmann, P. (2012) · Bioinformatics · doi:10.1093/bioinformatics/btr597 · Cited by 6,156
The PHQ-9: A New Depression Diagnostic and Severity Measure
Kroenke, K., & Spitzer, R. L. (2002) · Psychiatric Annals · doi:10.3928/0048-5713-20020901-06 · Cited by 5,898 · cited 8×
“The Patient Health Questionnaire - 9 (PHQ-9)(Kroenke & Spitzer, 2002) is a brief self-report measure assessing symptoms of depression over the past two weeks.”
BackgroundMethodInfluentialCognitive Therapy of Depression
HOLLON, S. D., & BECK, A. T. (1979) · Cognitive-Behavioral Interventions · doi:10.1016/b978-0-12-404480-7.50012-5 · Cited by 4,685
Big Data and Machine Learning in Health Care
Beam, A. L., & Kohane, I. S. (2018) · JAMA · doi:10.1001/jama.2017.18391 · Cited by 2,506
Psychometric properties of the credibility/expectancy questionnaire
Devilly, G. J., & Borkovec, T. D. (2000) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/s0005-7916(00)00012-4 · Cited by 2,295
Löwe, B., Kroenke, K., Herzog, W., & Gräfe, K. (2004) · Journal of Affective Disorders · doi:10.1016/s0165-0327(03)00198-8 · Cited by 1,558 · cited 1×
“The PHQ-9 has demonstrated good reliability and validity in previous validation studies as a screening tool for MDD (Kroenke & Spitzer, 2002; Löwe, Kroenke, Herzog, & Gräfe, 2004).”
MethodGrove, W. M., & Meehl, P. E. (1996) · Psychology, Public Policy, and Law · doi:10.1037/1076-8971.2.2.293 · Cited by 1,457
Sequenced treatment alternatives to relieve depression (STAR*D): Rationale and design
Rush, A. J., Fava, M., Wisniewski, S. R., Lavori, P. W., Trivedi, M. H., Sackeim, H. A., Thase, M. E., Nierenberg, A. A., Quitkin, F. M., Kashner, T. M., et al. (2004) · Controlled Clinical Trials · doi:10.1016/s0197-2456(03)00112-0 · Cited by 1,076 · cited 2×
“This is compared to RCTs of outpatient treatment where the r 2 is substantially lower, such as in the large STAR*D (Rush et al., 2004) and GENDEP trials (Uher et al., 2009; Uher, Tansey, Malki, & Perlis, 2012) (mean r 2 = .16 across both trials) and the recently completed EMBARC trial (r 2 = .09)…”
Method
Cited by
Supervised Machine Learning: A Brief Primer
Jiang, T., Gradus, J. L., & Rosellini, A. J. (2020) · Behavior Therapy · doi:10.1016/j.beth.2020.05.002 · Cited by 847
Using Machine Learning in Burnout Prediction: A Survey
Grządzielewska, M. (2021) · Child and Adolescent Social Work Journal · doi:10.1007/s10560-020-00733-w · Cited by 46
Cohen, Z. D., DeRubeis, R. J., Hayes, R., Watkins, E. R., Lewis, G., Byng, R., Byford, S., Crane, C., Kuyken, W., Dalgleish, T., et al. (2023) · Clinical Psychological Science · doi:10.1177/21677026221076832 · Cited by 25
Self-citationZilcha-Mano, S., & Webb, C. A. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.609585 · Cited by 25
Self-citationJiwani, Z., Goldberg, S. B., Stroud, J., Young, J., Curtin, J., Dunne, J. D., Simonsson, O., Webb, C. A., Carhart-Harris, R., & Schlosser, M. (2025) · PLOS ONE · doi:10.1371/journal.pone.0310160 · Cited by 6
Self-citation
Metadata
- Authors
- Webb, C. A., Cohen, Z. D., Beard, C., Forgeard, M., Peckham, A. D., & Björgvinsson, T.
- Venue
- Journal of Consulting and Clinical Psychology
- Published
- 2020-01
- Volume
- 88
- Issue
- 1
- Pages
- 25-38
- ISSN
- 0022-006X (ISSN-L)
- 1939-2117
- OpenAlex
- W2995547481
- Cited by
- 96
- References
- 27
- FWCI
- 6.01
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.