Prognostic models for predicting relapse or recurrence of major depressive disorder in adults
Open accessThe DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Relapse (the re-emergence of depressive symptoms after some level of improvement but preceding recovery) and recurrence (onset of a new depressive episode after recovery) are common in depression, lead to worse outcomes and quality of life for patients and exert a high economic cost on society. Outcomes can be predicted by using multivariable prognostic models, which use information about several predictors to produce an individualised risk estimate. The ability to accurately predict relapse or recurrence while patients are well (in remission) would allow the identification of high-risk individuals and may improve overall treatment outcomes for patients by enabling more efficient allocation of interventions to prevent relapse and recurrence. OBJECTIVES: To summarise the predictive performance of prognostic models developed to predict the risk of relapse, recurrence, sustained remission or recovery in adults with major depressive disorder who meet criteria for remission or recovery. SEARCH METHODS: We searched the Cochrane Library (current issue); Ovid MEDLINE (1946 onwards); Ovid Embase (1980 onwards); Ovid PsycINFO (1806 onwards); and Web of Science (1900 onwards) up to May 2020. We also searched sources of grey literature, screened the reference lists of included studies and performed a forward citation search. There were no restrictions applied to the searches by date, language or publication status . SELECTION CRITERIA: We included development and external validation (testing model performance in data separate from the development data) studies of any multivariable prognostic models (including two or more predictors) to predict relapse, recurrence, sustained remission, or recovery in adults (aged 18 years and over) with remitted depression, in any clinical setting. We included all study designs and accepted all definitions of relapse, recurrence and other related outcomes. We did not specify a comparator prognostic model. DATA COLLECTION AND ANALYSIS: Two review authors independently screened references; extracted data (using a template based on the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS)); and assessed risks of bias of included studies (using the Prediction model Risk Of Bias ASsessment Tool (PROBAST)). We referred any disagreements to a third independent review author. Where we found sufficient (10 or more) external validation studies of an individual model, we planned to perform a meta-analysis of its predictive performance, specifically with respect to its calibration (how well the predicted probabilities match the observed proportions of individuals that experience the outcome) and discrimination (the ability of the model to differentiate between those with and without the outcome). Recommendations could not be qualified using the GRADE system, as guidance is not yet available for prognostic model reviews. MAIN RESULTS: We identified 11 eligible prognostic model studies (10 unique prognostic models). Seven were model development studies; three were model development and external validation studies; and one was an external validation-only study. Multiple estimates of performance measures were not available for any of the models and, meta-analysis was therefore not possible. Ten out of the 11 included studies were assessed as being at high overall risk of bias. Common weaknesses included insufficient sample size, inappropriate handling of missing data and lack of information about discrimination and calibration. One paper (Klein 2018) was at low overall risk of bias and presented a prognostic model including the following predictors: number of previous depressive episodes, residual depressive symptoms and severity of the last depressive episode. The external predictive performance of this model was poor (C-statistic 0.59; calibration slope 0.56; confidence intervals not reported). None of the identified studies examined the clinical utility (net benefit) of the developed model. AUTHORS' CONCLUSIONS: Of the 10 prognostic models identified (across 11 studies), only four underwent external validation. Most of the studies (n = 10) were assessed as being at high overall risk of bias, and the one study that was at low risk of bias presented a model with poor predictive performance. There is a need for improved prognostic research in this clinical area, with future studies conforming to current best practice recommendations for prognostic model development/validation and reporting findings in line with the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group† (2019) · Annals of Internal Medicine · doi:10.7326/m18-1376 · Cited by 2,984
Self-citationCalculating the sample size required for developing a clinical prediction model
Riley, R. D., Ensor, J., Snell, K. I. E., Harrell, F. E., Martin, G. P., Reitsma, J. B., Moons, K. G. M., Collins, G., & van Smeden, M. (2020) · BMJ · doi:10.1136/bmj.m441 · Cited by 2,917
Frank, E. (1991) · Archives of General Psychiatry · doi:10.1001/archpsyc.1991.01810330075011 · Cited by 1,959
Transduction of psychosocial stress into the neurobiology of recurrent affective disorder
(1992) · American Journal of Psychiatry · doi:10.1176/ajp.149.8.999 · Cited by 1,848
Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S. (2019) · Annals of Internal Medicine · doi:10.7326/m18-1377 · Cited by 1,669
Self-citationPrognosis Research Strategy (PROGRESS) 3: Prognostic Model Research
Steyerberg, E. W., Moons, K. G. M., van der Windt, D. A., Hayden, J. A., Perel, P., Schroter, S., Riley, R. D., Hemingway, H., Altman, D. G., & for the PROGRESS Group (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001381 · Cited by 1,508
Self-citationRiley, R. D., Snell, K. I., Ensor, J., Burke, D. L., Harrell Jr, F. E., Moons, K. G., & Collins, G. S. (2019) · Statistics in Medicine · doi:10.1002/sim.7992 · Cited by 1,409
Self-citationExplainable AI: Interpreting, Explaining and Visualizing Deep Learning
(2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6 · Cited by 1,377
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018) · Management Science · doi:10.1287/mnsc.2016.2643 · Cited by 1,230
Risk prediction models: II. External validation, model updating, and impact assessment
Moons, K. G. M., Kengne, A. P., Grobbee, D. E., Royston, P., Vergouwe, Y., Altman, D. G., & Woodward, M. (2012) · Heart · doi:10.1136/heartjnl-2011-301247 · Cited by 1,195
Vickers, A. J., Van Calster, B., & Steyerberg, E. W. (2016) · BMJ · doi:10.1136/bmj.i6 · Cited by 1,163
A comparison of heterogeneity variance estimators in simulated random‐effects meta‐analyses
Langan, D., Higgins, J. P. T., Jackson, D., Bowden, J., Veroniki, A. A., Kontopantelis, E., Viechtbauer, W., & Simmonds, M. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1316 · Cited by 1,074
Risk for recurrence in depression
Burcusa, S. L., & Iacono, W. G. (2007) · Clinical Psychology Review · doi:10.1016/j.cpr.2007.02.005 · Cited by 1,023
Relapse prevention with antidepressant drug treatment in depressive disorders: a systematic review
Geddes, J. R., Carney, S. M., Davies, C., Furukawa, T. A., Kupfer, D. J., Frank, E., & Goodwin, G. M. (2003) · The Lancet · doi:10.1016/s0140-6736(03)12599-8 · Cited by 945
Recurrence after recovery from major depressive disorder during 15 years of observational follow-up
Mueller, T. I., Leon, A. C., Keller, M. B., Solomon, D. A., Endicott, J., Coryell, W., Warshaw, M., & Maser, J. D. (1999) · American Journal of Psychiatry · doi:10.1176/ajp.156.7.1000 · Cited by 897
Iorio, A., Spencer, F. A., Falavigna, M., Alba, C., Lang, E., Burnand, B., McGinn, T., Hayden, J., Williams, K., Shea, B., et al. (2015) · BMJ · doi:10.1136/bmj.h870 · Cited by 880
Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints
van der Ploeg, T., Austin, P. C., & Steyerberg, E. W. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-137 · Cited by 809
Collins, G. S., de Groot, J. A., Dutton, S., Omar, O., Shanyinde, M., Tajar, A., Voysey, M., Wharton, R., Yu, L.-M., Moons, K. G., et al. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-40 · Cited by 729
Report by the ACNP Task Force on Response and Remission in Major Depressive Disorder
Rush, A. J., Kraemer, H. C., Sackeim, H. A., Fava, M., Trivedi, M. H., Frank, E., Ninan, P. T., Thase, M. E., Gelenberg, A. J., Kupfer, D. J., et al. (2006) · Neuropsychopharmacology · doi:10.1038/sj.npp.1301131 · Cited by 707
Reporting of artificial intelligence prediction models
Collins, G. S., & Moons, K. G. M. (2019) · The Lancet · doi:10.1016/s0140-6736(19)30037-6 · Cited by 701
Cross-trial prediction of treatment outcome in depression: a machine learning approach
Chekroud, A. M., Zotti, R. J., Shehzad, Z., Gueorguieva, R., Johnson, M. K., Trivedi, M. H., Cannon, T. D., Krystal, J. H., & Corlett, P. R. (2016) · The Lancet Psychiatry · doi:10.1016/s2215-0366(15)00471-x · Cited by 676
A guide to systematic review and meta-analysis of prediction model performance
Debray, T. P. A., Damen, J. A. A. G., Snell, K. I. E., Ensor, J., Hooft, L., Reitsma, J. B., Riley, R. D., & Moons, K. G. M. (2017) · BMJ · doi:10.1136/bmj.i6460 · Cited by 657
Self-citationJudd, L. L., Akiskal, H. S., Maser, J. D., Zeller, P. J., Endicott, J., Coryell, W., Paulus, M. P., Kunovac, J. L., Leon, A. C., Mueller, T. I., et al. (1998) · Journal of Affective Disorders · doi:10.1016/s0165-0327(98)00138-4 · Cited by 590
Onukwugha, E., Yong, C., Hussain, A., Seal, B., & Mullins, C. D. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-1 · Cited by 589
Huguet, A., Hayden, J. A., Stinson, J., McGrath, P. J., Chambers, C. T., Tougas, M. E., & Wozney, L. (2013) · Systematic Reviews · doi:10.1186/2046-4053-2-71 · Cited by 531
Foroutan, F., Guyatt, G., Zuk, V., Vandvik, P. O., Alba, A. C., Mustafa, R., Vernooij, R., Arevalo-Rodriguez, I., Munn, Z., Roshanov, P., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.12.023 · Cited by 465
Self-citationBuckman, J. E. J., Underwood, A., Clarke, K., Saunders, R., Hollon, S. D., Fearon, P., & Pilling, S. (2018) · Clinical Psychology Review · doi:10.1016/j.cpr.2018.07.005 · Cited by 398
Six-Year Outcome for Cognitive Behavioral Treatment of Residual Symptoms in Major Depression
Fava, G. A., Rafanelli, C., Grandi, S., Canestrari, R., & Morphy, M. A. (1998) · American Journal of Psychiatry · doi:10.1176/ajp.155.10.1443 · Cited by 299
Bockting, C. L., Hollon, S. D., Jarrett, R. B., Kuyken, W., & Dobson, K. (2015) · Clinical Psychology Review · doi:10.1016/j.cpr.2015.02.003 · Cited by 295
A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes
Debray, T. P., Damen, J. A., Riley, R. D., Snell, K., Reitsma, J. B., Hooft, L., Collins, G. S., & Moons, K. G. (2019) · Statistical Methods in Medical Research · doi:10.1177/0962280218785504 · Cited by 278
Self-citation(1992) · American Journal of Psychiatry · doi:10.1176/ajp.149.8.1046 · Cited by 252
King, M., Walker, C., Levy, G., Bottomley, C., Royston, P., Weich, S., Bellón-Saameño, J. Á., Moreno, B., Švab, I., Rotar, D., et al. (2008) · Archives of General Psychiatry · doi:10.1001/archpsyc.65.12.1368 · Cited by 221
Hardeveld, F., Spijker, J., De Graaf, R., Nolen, W. A., & Beekman, A. T. F. (2013) · Psychological Medicine · doi:10.1017/s0033291712002395 · Cited by 176
Appelhof, B. C., Huyser, J., Verweij, M., Brouwer, J. P., van Dyck, R., Fliers, E., Hoogendijk, W. J. G., Tijssen, J. G. P., Wiersinga, W. M., & Schene, A. H. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.09.008 · Cited by 154
Relapse of depression in primary care: Rate and clinical predictors
Lin, E. H. B. (1998) · Archives of Family Medicine · doi:10.1001/archfami.7.5.443 · Cited by 152
Lynam, A. L., Dennis, J. M., Owen, K. R., Oram, R. A., Jones, A. G., Shields, B. M., & Ferrat, L. A. (2020) · Diagnostic and Prognostic Research · doi:10.1186/s41512-020-00075-2 · Cited by 142
Ali, S., Rhodes, L., Moreea, O., McMillan, D., Gilbody, S., Leach, C., Lucock, M., Lutz, W., & Delgadillo, J. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2017.04.006 · Cited by 139
Self-citationClarke, K., Mayo-Wilson, E., Kenny, J., & Pilling, S. (2015) · Clinical Psychology Review · doi:10.1016/j.cpr.2015.04.002 · Cited by 113
Berwian, I. M., Wenzel, J. G., Collins, A. G. E., Seifritz, E., Stephan, K. E., Walter, H., & Huys, Q. J. M. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.4971 · Cited by 113
Wojnarowski, C., Firth, N., Finegan, M., & Delgadillo, J. (2019) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465819000080 · Cited by 108
Longitudinal hypothalamic-pituitary-adrenal axis trait and state effects in recurrent depression
Lok, A., Mocking, R. J. T., Ruhé, H. G., Visser, I., Koeter, M. W. J., Assies, J., Bockting, C. L. H., Olff, M., & Schene, A. H. (2012) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2011.10.005 · Cited by 99
Hardeveld, F., Spijker, J., De Graaf, R., Hendriks, S. M., Licht, C. M. M., Nolen, W. A., Penninx, B. W. J. H., & Beekman, A. T. F. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2012.11.008 · Cited by 95
Remission onset and relapse in depression
O’Leary, D., Costello, F., Gormley, N., & Webb, M. (2000) · Journal of Affective Disorders · doi:10.1016/s0165-0327(99)00086-5 · Cited by 87
IsHak, W. W., Greenberg, J. M., & Cohen, R. M. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.05.048 · Cited by 78
Personality and Clinical Predictors of Recurrence of Depression
Berlanga, C., Heinze, G., Torres, M., Apiquián, R., & Caballero, A. (1999) · Psychiatric Services · doi:10.1176/ps.50.3.376 · Cited by 70
Ruhe, H. G., Mocking, R. J. T., Figueroa, C. A., Seeverens, P. W. J., Ikani, N., Tyborowska, A., Browning, M., Vrijsen, J. N., Harmer, C. J., & Schene, A. H. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00145 · Cited by 66
Predictors of recurrence in remitted late-life depression
Deng, Y., McQuoid, D. R., Potter, G. G., Steffens, D. C., Albert, K., Riddle, M., Beyer, J. L., & Taylor, W. D. (2018) · Depression and Anxiety · doi:10.1002/da.22772 · Cited by 65
Conradi, H. J., de Jonge, P., & Ormel, J. (2008) · Journal of Affective Disorders · doi:10.1016/j.jad.2007.04.017 · Cited by 62
An evaluation of an integrated model of relapse in depression
Backs-Dermott, B. J., Dobson, K. S., & Jones, S. L. (2010) · Journal of Affective Disorders · doi:10.1016/j.jad.2009.11.015 · Cited by 49
Sex similarities and differences in risk factors for recurrence of major depression
van Loo, H. M., Aggen, S. H., Gardner, C. O., & Kendler, K. S. (2018) · Psychological Medicine · doi:10.1017/s0033291717003178 · Cited by 45
Multiple risk factors predict recurrence of major depressive disorder in women
van Loo, H. M., Aggen, S. H., Gardner, C. O., & Kendler, K. S. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2015.03.045 · Cited by 45
Wang, J. L., Patten, S., Sareen, J., Bolton, J., Schmitz, N., & MacQueen, G. (2014) · Depression and Anxiety · doi:10.1002/da.22215 · Cited by 43
Klein, N. S., Holtman, G. A., Bockting, C. L. H., Heymans, M. W., & Burger, H. (2018) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2018.06.006 · Cited by 29
Cited by
Predicting relapse or recurrence of depression: systematic review of prognostic models
Moriarty, A. S., Meader, N., Snell, K. I. E., Riley, R. D., Paton, L. W., Dawson, S., Hendon, J., Chew-Graham, C. A., Gilbody, S., Churchill, R., et al. (2022) · The British Journal of Psychiatry · doi:10.1192/bjp.2021.218 · Cited by 71
Self-citationCohen, Z. D., DeRubeis, R. J., Hayes, R., Watkins, E. R., Lewis, G., Byng, R., Byford, S., Crane, C., Kuyken, W., Dalgleish, T., et al. (2023) · Clinical Psychological Science · doi:10.1177/21677026221076832 · Cited by 25
Saunders, R., Cohen, Z. D., Ambler, G., DeRubeis, R. J., Wiles, N., Kessler, D., Gilbody, S., Hollon, S. D., Kendrick, T., Watkins, E., et al. (2021) · Journal of Personalized Medicine · doi:10.3390/jpm11121295 · Cited by 21
“However, these have been hampered by methodological problems, including small sample sizes, inappropriate handling of missing data, and a lack of validation, and they have largely failed to make accurate predictions [44].”
BackgroundSelf-citationNunes, A., Pavlova, B., Cunningham, J. E. A., Nuñez, J.-J., Quilty, L. C., Foster, J. A., Harkness, K. L., Ho, K., Lam, R. W., Li, Q. S., et al. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.06.023 · Cited by 5
Metadata
- Authors
- Moriarty, A. S., Meader, N., Snell, K. I., Riley, R. D., Paton, L. W., Chew-Graham, C. A., Gilbody, S., Churchill, R., Phillips, R. S., Ali, S., & McMillan, D.
- Venue
- Cochrane Database of Systematic Reviews
- Publisher
- Wiley
- Published
- 2021-05-05
- Volume
- 2021
- Issue
- 5
- Pages
- CD013491-CD013491
- ISSN
- 1361-6137 (ISSN-L)
- 1465-1858
- OpenAlex
- W3159074378
- Cited by
- 36
- References
- 54
- FWCI
- 3.20
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.