Development and validation of a clinical prediction tool to estimate the individual risk of depressive relapse or recurrence in individuals with recurrent depression
Open accessAbstract
Objectives Many studies examined predictors of depressive relapse/recurrence but no simple tool based on well-established risk factors is available that estimates the risk within an individual. We developed and validated such a prediction tool in remitted recurrently depressed individuals.Methods The tool was developed using data (n = 235) from a pragmatic randomised controlled trial in remitted recurrently depressed participants and externally validated using data (n = 209) from a similar randomised controlled trial of remitted recurrently depressed participants using maintenance antidepressants. Cox regression was used with time to relapse/recurrence within 2 years as outcome and well-established risk factors as predictors. Performance measures and absolute risk scores were calculated, a practically applicable risk score was created, and the tool was externally validated.Results The 2-year cumulative proportion relapse/recurrence was 46.2% in the validation dataset. The tool included number of previous depressive episodes, residual depressive symptoms, severity of the last depressive episode, and treatment. The C-statistic and calibration slope were 0.56 and 0.81 respectively. The tool stratified participants into relapse/recurrence risk classes of 37%, 55%, and 72%. The C-statistic and calibration slope in the external validation were 0.59 and 0.56 respectively, and Kaplan Meier curves showed that the tool could differentiate between risk classes.Conclusions This is the first study that developed a simple prediction tool based on well-established risk factors of depressive relapse/recurrence, estimating the individual risk. Since the overall performance of the model was poor, more studies are needed to enhance the performance before recommending implementation into clinical practice.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior research has investigated a limited range of individual predictors, many of which are related to demographics and clinical history.
“prior research has investigated a limited range of individual predictors, many of which are related to demographics and clinical history.7,9,18”
Between one-fifth and one-half of patients in remission from a depressive episode will suffer a recurrence within 2 years.
“Studies indicate that between one-fifth to one-half of patients in remission from a depressive episode will suffer a recurrence within 2 years.4–10”
After treatment, a higher risk of relapse has been associated with the number and severity of previous episodes, the persistence of residual symptoms, and a history of childhood maltreatment.
“after treatment, a higher risk of relapse has been associated with the number and severity of previous episodes, the persistence of residual symptoms and a history of childhood maltreatment.9,15,19–23”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hamilton, M. (1960) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.23.1.56 · Cited by 32,817 · see citing works
Multiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792 · see citing works
General Cardiovascular Risk Profile for Use in Primary Care: The Framingham Heart Study
D’Agostino, R. B., Vasan, R. S., Pencina, M. J., Wolf, P. A., Cobain, M., Massaro, J. M., & Kannel, W. B. (2008) · Circulation · doi:10.1161/circulationaha.107.699579 · Cited by 7,491 · see citing works
(2010) · Cited by 7,034 · see citing works
Whiteford, H. A., Degenhardt, L., Rehm, J., Baxter, A. J., Ferrari, A. J., Erskine, H. E., Charlson, F. J., Norman, R. E., Flaxman, A. D., Johns, N., et al. (2013) · The Lancet · doi:10.1016/s0140-6736(13)61611-6 · Cited by 6,006 · see citing works
The Structured Clinical Interview for DSM-III-R (SCID): I: History, Rationale, and Description
Spitzer, R. L. (1992) · Archives of General Psychiatry · doi:10.1001/archpsyc.1992.01820080032005 · Cited by 4,336 · see citing works
Methodology of Social Sciences
Weber, M. (2017) · Routledge eBooks · doi:10.4324/9781315124445 · Cited by 2,547 · see citing works
The Inventory of Depressive Symptomatology (IDS): Psychometric properties
Rush, A. J., Gullion, C. M., Basco, M. R., Jarrett, R. B., & Trivedi, M. H. (1996) · Psychological Medicine · doi:10.1017/s0033291700035558 · Cited by 2,177 · see citing works
Peduzzi, P., Concato, J., Feinstein, A. R., & Holford, T. R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00048-8 · Cited by 2,154 · see citing works
Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research
Steyerberg, E. W., Moons, K. G. M., van der Windt, D. A., Hayden, J. A., Perel, P., Schroter, S., Riley, R. D., Hemingway, H., Altman, D. G., & for the PROGRESS Group (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001381 · Cited by 1,508 · see citing works
Nanni, V., Uher, R., & Danese, A. (2012) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.11020335 · Cited by 1,480 · see citing works
PRACTICE GUIDELINE FOR THE Treatment of Patients With Major Depressive Disorder
Gelenberg, A. J., Freeman, P. B., Markowitz, J. C., Rosenbaum, J. F., Thase, M. E., Trivedi, M. H., & Rhoads, R. S. (2010) · Cited by 1,372 · see citing works
Fried, E. I., & Nesse, R. M. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2014.10.010 · Cited by 1,043 · see citing works
Risk for recurrence in depression
Burcusa, S. L., & Iacono, W. G. (2007) · Clinical Psychology Review · doi:10.1016/j.cpr.2007.02.005 · Cited by 1,023 · see citing works
Prognosis and prognostic research: application and impact of prognostic models in clinical practice
Moons, K. G. M., Altman, D. G., Vergouwe, Y., & Royston, P. (2009) · BMJ · doi:10.1136/bmj.b606 · Cited by 934 · see citing works
External validation of a Cox prognostic model: principles and methods
Royston, P. J., & Altman, D. G. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-33 · Cited by 911 · see citing works
A calibration hierarchy for risk models was defined: from utopia to empirical data
Van Calster, B., Nieboer, D., Vergouwe, Y., De Cock, B., Pencina, M. J., & Steyerberg, E. W. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.12.005 · Cited by 910 · see citing works
Marshall, A., Altman, D. G., HOLDER, R. L., & Royston, P. J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-57 · Cited by 883 · see citing works
Multiple recurrences of major depressive disorder
Solomon, D. A., Keller, M. B., Leon, A. C., Mueller, T. I., Lavori, P. W., Shea, M. T., Coryell, W., Warshaw, M., Turvey, C., Maser, J. D., et al. (2000) · American Journal of Psychiatry · doi:10.1176/appi.ajp.157.2.229 · Cited by 835 · see citing works
Childhood maltreatment and characteristics of adult depression: Meta-analysis
Nelson, J., Klumparendt, A., Doebler, P., & Ehring, T. (2017) · British Journal of Psychiatry · doi:10.1192/bjp.bp.115.180752 · Cited by 822 · see citing works
Carr, C. P., Martins, C. M. S., Stingel, A. M., Lemgruber, V. B., & Juruena, M. F. (2013) · Journal of Nervous & Mental Disease · doi:10.1097/nmd.0000000000000049 · Cited by 802 · see citing works
Cognitive debiasing 1: origins of bias and theory of debiasing
Croskerry, P., Singhal, G., & Mamede, S. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-001712 · Cited by 605 · see citing works
Prevalence and predictors of recurrence of major depressive disorder in the adult population
Hardeveld, F., Spijker, J., De Graaf, R., Nolen, W. A., & Beekman, A. T. F. (2010) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2009.01519.x · Cited by 486 · see citing works
Depression in adults: recognition and management
(2017) · Clinical pharmacist · doi:10.1211/cp.2017.20202439 · Cited by 394 · see citing works
Impact of childhood life events and trauma on the course of depressive and anxiety disorders
Hovens, J. G. F. M., Giltay, E. J., Wiersma, J. E., Spinhoven, P., Penninx, B. W. J. H., & Zitman, F. G. (2012) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2011.01828.x · Cited by 328 · see citing works
Bockting, C. L., Hollon, S. D., Jarrett, R. B., Kuyken, W., & Dobson, K. (2015) · Clinical Psychology Review · doi:10.1016/j.cpr.2015.02.003 · Cited by 295 · see citing works
Self-citationten Haaf, K., Jeon, J., Tammemägi, M. C., Han, S. S., Kong, C. Y., Plevritis, S. K., Feuer, E. J., de Koning, H. J., Steyerberg, E. W., & Meza, R. (2017) · PLoS Medicine · doi:10.1371/journal.pmed.1002277 · Cited by 282 · see citing works
Depression in adults with a chronic physical health problem: Treatment and management
Haddad, M. (2009) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2009.08.007 · Cited by 271 · see citing works
Geersing, G., Zuithoff, N. P. A., Kearon, C., Anderson, D. R., ten Cate‐Hoek, A. J., Elf, J. L., Bates, S. M., Hoes, A. W., Kraaijenhagen, R. A., Oudega, R., et al. (2014) · BMJ · doi:10.1136/bmj.g1340 · Cited by 227 · see citing works
Moving forward: how depression heterogeneity hinders progress in treatment and research
Fried, E. (2017) · Expert Review of Neurotherapeutics · doi:10.1080/14737175.2017.1307737 · Cited by 220 · see citing works
Mental disorders and risk of suicide attempt: a national prospective study
Hoertel, N., Franco, S., Wall, M. M., Oquendo, M. A., Kerridge, B. T., Limosin, F., & Blanco, C. (2015) · Molecular Psychiatry · doi:10.1038/mp.2015.19 · Cited by 205 · see citing works
de Graaf, R., Bijl, R. V., Ravelli, A., Smit, F., & Vollebergh, W. (2002) · Acta Psychiatrica Scandinavica · doi:10.1034/j.1600-0447.2002.01397.x · Cited by 192 · see citing works
Hardeveld, F., Spijker, J., De Graaf, R., Nolen, W. A., & Beekman, A. T. F. (2013) · Psychological Medicine · doi:10.1017/s0033291712002395 · Cited by 176 · see citing works
Stangier, U., Hilling, C., Heidenreich, T., Risch, A. K., Barocka, A., Schlösser, R., Kronfeld, K., Ruckes, C., Berger, H., Röschke, J., et al. (2013) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2013.12060734 · Cited by 174 · see citing works
Tran, T., Luo, W., Phung, D., Harvey, R., Berk, M., Kennedy, R. L., & Venkatesh, S. (2014) · BMC Psychiatry · doi:10.1186/1471-244x-14-76 · Cited by 150 · see citing works
Depression in adults: recognition and management
(2017) · Pharmaceutical journal/The pharmaceutical journal · doi:10.1211/pj.2017.20202439 · Cited by 137 · see citing works
Hovens, J. G. F. M., Giltay, E. J., Spinhoven, P., van Hemert, A. M., & Penninx, B. W. J. H. (2015) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.14m09135 · Cited by 135 · see citing works
Negative life events and depressive symptoms in the elderly: A life span perspective
Kraaij, V., & de Wilde, E. J. (2001) · Aging & Mental Health · doi:10.1080/13607860020020681 · Cited by 131 · see citing works
Bockting, C. L. H., Klein, N. S., Elgersma, H. J., van Rijsbergen, G. D., Slofstra, C., Ormel, J., Buskens, E., Dekker, J., de Jong, P. J., Nolen, W. A., et al. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(18)30100-7 · Cited by 130 · see citing works
Self-citationPain, not chronic disease, is associated with the recurrence of depressive and anxiety disorders
Gerrits, M. M., van Oppen, P., Leone, S. S., van Marwijk, H. W., van der Horst, H. E., & Penninx, B. W. (2014) · BMC Psychiatry · doi:10.1186/1471-244x-14-187 · Cited by 106 · see citing works
Groenwold, R., Moons, K., Pajouheshnia, R., Altman, D., Collins, G., Debray, T. P. A., Reitsma, J., Riley, R., & Peelen, L. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.03.017 · Cited by 70 · see citing works
Fusar‐Poli, P., & Schultze‐Lutter, F. (2016) · Evidence-Based Mental Health · doi:10.1136/eb-2015-102295 · Cited by 66 · see citing works
Paterniti, S., Sterner, I., Caldwell, C., & Bisserbe, J. C. (2017) · BMC Psychiatry · doi:10.1186/s12888-017-1270-x · Cited by 63 · see citing works
Hoertel, N., Blanco, C., Oquendo, M. A., Wall, M. M., Olfson, M., Falissard, B., Franco, S., Peyre, H., Lemogne, C., & Limosin, F. (2017) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2017.07.022 · Cited by 63 · see citing works
Multiple risk factors predict recurrence of major depressive disorder in women
van Loo, H. M., Aggen, S. H., Gardner, C. O., & Kendler, K. S. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2015.03.045 · Cited by 45 · see citing works
Wang, J. L., Patten, S., Sareen, J., Bolton, J., Schmitz, N., & MacQueen, G. (2014) · Depression and Anxiety · doi:10.1002/da.22215 · Cited by 43 · see citing works
Clinical differences between first and recurrent episodes in depressive patients
Roca, M., Armengol, S., García-García, M., Rodríguez-Bayón, A., Ballesta, I., Serrano-Ripoll, M. J., Comas, À., & Gili, M. (2010) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2010.04.011 · Cited by 37 · see citing works
Bockting, C., Elgersma, H. J., van Rijsbergen, G. D., de Jonge, P., Ormel, J., Buskens, E., Stant, A. D., de Jong, P. J., Peeters, F. P. M. L., Huibers, M. J. H., et al. (2011) · BMC Psychiatry · doi:10.1186/1471-244x-11-8 · Cited by 29 · see citing works
Self-citationJudd, L. L., Schettler, P. J., & Rush, A. J. (2016) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2016.15111462 · Cited by 29 · see citing works
Biesheuvel-Leliefeld, K. E. M., Dijkstra‐Kersten, S. M. A., van Schaik, A., van Marwijk, H., Smit, F., van der Horst, H. E., & Bockting, C. (2017) · Psychotherapy and Psychosomatics · doi:10.1159/000472260 · Cited by 29 · see citing works
Self-citationFactors predicting the long-term illness course in a cohort of depressed inpatients
Kuehner, C., & Huffziger, S. (2012) · European Archives of Psychiatry and Clinical Neuroscience · doi:10.1007/s00406-012-0379-7 · Cited by 25 · see citing works
King, M., Bottomley, C., Bellón, J. Á., Torres‐González, F., Švab, I., Rotar, D., Xavier, M., & Nazareth, I. (2013) · Psychological Medicine · doi:10.1017/s0033291712002693 · Cited by 22 · see citing works
Klein, N. S., Kok, G. D., Burger, H., van Valen, E., Riper, H. M., Cuijpers, P. J., Dekker, J., Smit, F., van der Heiden, C., & Bockting, C. (2018) · Psychotherapy and Psychosomatics · doi:10.1159/000485039 · Cited by 20 · see citing works
Self-citationRelapse prevention for depression.
(2010) · American Psychological Association eBooks · doi:10.1037/12082-000 · Cited by 18 · see citing works
Kok, G. D., Bockting, C. L. H., Burger, H., Hannig, W., Pijnenborg, G. H. M., Cuijpers, P., & Hollon, S. D. (2013) · PLoS ONE · doi:10.1371/journal.pone.0057510 · Cited by 18 · see citing works
Self-citationKennedy, J. C., Dunlop, B. W., Craighead, L. W., Nemeroff, C. B., Mayberg, H. S., & Craighead, W. E. (2018) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000279 · Cited by 18 · see citing works
The persistence of depression score
Spijker, J., de Graaf, R., Ormel, J., Nolen, W. A., Grobbee, D. E., & Burger, H. (2006) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2006.00821.x · Cited by 11 · see citing works
Self-citationP127 Comprehensive Model For Implementation Of Guidelines For Disease Prevention
Elgstrand, M. B., Carlsson, I., Månsdotter, A., et al. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2013-002293.173 · Cited by 10 · see citing works
Recurrence in major depression: Assessing risk indicators in the context of risk estimates.
Monroe, S. M. (2010) · American Psychological Association eBooks · doi:10.1037/12082-001 · Cited by 5 · see citing works
Bockting, C., Kok, G. D., van der Kamp, L., Smit, F., van Valen, E., Schoevers, R., van Marwijk, H. V., Cuijpers, P., Riper, H., Dekker, J., et al. (2011) · BMC Psychiatry · doi:10.1186/1471-244x-11-12 · Cited by 3 · see citing works
Self-citationHarrell, F., Jr. (1996) · doi:10.1002/(sici)1097-0258(19960229)15:4
Flagged first, then by citations
Cited by
Sorted by citations
Predicting relapse or recurrence of depression: systematic review of prognostic models
Moriarty, A. S., Meader, N., Snell, K. I. E., Riley, R. D., Paton, L. W., Dawson, S., Hendon, J., Chew-Graham, C. A., Gilbody, S., Churchill, R., et al. (2022) · The British Journal of Psychiatry · doi:10.1192/bjp.2021.218 · Cited by 71 · see citing works
Kumagai, N., Tajika, A., Hasegawa, A., Kawanishi, N., Horikoshi, M., Shimodera, S., Kurata, K., Chino, B., & Furukawa, T. A. (2019) · BMC Psychiatry · doi:10.1186/s12888-019-2382-2 · Cited by 20 · see citing works
Saunders, R., Cohen, Z. D., Ambler, G., DeRubeis, R. J., Wiles, N., Kessler, D., Gilbody, S., Hollon, S. D., Kendrick, T., Watkins, E., et al. (2021) · Journal of Personalized Medicine · doi:10.3390/jpm11121295 · Cited by 21 · see citing works
Lam, R. W., Rnic, K., Nunez, J.-J., Ho, K., LeMoult, J., Nunes, A., Chakrabarty, T., Foster, J. A., Frey, B. N., Harkness, K. L., et al. (2025) · The Canadian Journal of Psychiatry · doi:10.1177/07067437251337603 · Cited by 6 · see citing works
- Authors
- Klein, N. S., Holtman, G. A., Bockting, C. L. H., Heymans, M. W., & Burger, H.
- Affiliations
- University of Groningen (NL)
- University Medical Center Groningen (NL)
- Amsterdam UMC Location Vrije Universiteit Amsterdam (NL)
- Corresponding author
- Bockting, Claudi L.H.
- Venue
- Journal of Psychiatric Research
- Publisher
- Elsevier BV
- Published
- 2018-09
- Volume
- 104
- Pages
- 1-7
- ISSN
- 0022-3956 (ISSN-L)
- OpenAlex
- W2806258855
- Cited by
- 29
- References
- 61
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 2 · 2023: 7 · 2022: 5 · 2021: 5 · 2020: 3 · 2019: 3
- FWCI
- 1.78
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Psychiatric Research (publishedVersion)
- PubMed (publishedVersion)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Pure Amsterdam UMC (submittedVersion)
- Topics
- Health Sciences › Medicine › Pharmacology › Treatment of Major Depression
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Health Sciences › Medicine › Psychiatry and Mental health › Bipolar Disorder and Treatment
- Keywords
- recurrent depression, depression relapse, depression recurrence, risk prediction tools, individual risk assessment, residual depressive symptoms, episode severity, remitted depression, risk score
- MeSH terms
- Adolescent, Adult, Depression, Female, Follow-Up Studies, Humans, Male, Middle Aged, Predictive Value of Tests, Psychiatric Status Rating Scales, Recurrence, Risk Factors, Proportional Hazards Models, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
