The effects of psychotherapy for adult depression are overestimated: a meta-analysis of study quality and effect size
The DOI matches its Crossref record; 49 of the 50 references it cites were found in a registry or an index.
Abstract
Background: No meta-analytical study has examined whether the quality of the studies examining psychotherapy for adult depression is associated with the effect sizes found. This study assesses this association. Method: We used a database of 115 randomized controlled trials in which 178 psychotherapies for adult depression were compared to a control condition. Eight quality criteria were assessed by two independent coders: participants met diagnostic criteria for a depressive disorder, a treatment manual was used, the therapists were trained, treatment integrity was checked, intention-to-treat analyses were used, N ⩾50, randomization was conducted by an independent party, and assessors of outcome were blinded. Results: Only 11 studies (16 comparisons) met the eight quality criteria. The standardized mean effect size found for the high-quality studies ( d =0.22) was significantly smaller than in the other studies ( d =0.74, p <0.001), even after restricting the sample to the subset of other studies that used the kind of care-as-usual or non-specific controls that tended to be used in the high-quality studies. Heterogeneity was zero in the group of high-quality studies. The numbers needed to be treated in the high-quality studies was 8, while it was 2 in the lower-quality studies. Conclusions: We found strong evidence that the effects of psychotherapy for adult depression have been overestimated in meta-analytical studies. Although the effects of psychotherapy are significant, they are much smaller than was assumed until now, even after controlling for the type of control condition used.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The quality of the included studies was considered using in part the criteria derived by Cuijpers et al. (2010) from work by others (Chambless & Hollon, 1998; Higgins & Green, 2006).”
From An update on the efficacy of psychological treatments for obsessive-compulsive disorder in adults (2013) · cites this paper as Cuijpers et al. (2010)
In the citing paper
“We rated each study on eight criteria developed by Cuijpers and colleagues (2010) and found that the mean quality rating for all RCTs included in this review was only 4.6 (SD 2.2) out of a maximum of 8.0.”
From An update on the efficacy of psychological treatments for obsessive-compulsive disorder in adults (2013) · cites this paper as Cuijpers et al., 2010
In the citing paper
“A similar difference has been noted for psychotherapy effectiveness in depression, whereby overall effectiveness was overestimated in earlier meta-analyses because of inclusion of non-experimental designs.57”
From Effectiveness of psychological interventions in prison to reduce recidivism: a systematic review and meta-analysis of randomised controlled trials (2021) · cites this paper as [57]
In the citing paper
“It is well known from controlled trials that poor-quality trials considerably overestimate the effects of therapies (Cuijpers et al. 2010(Cuijpers et al. , 2019b;;Furukawa et al. 2014).”
From The Role of Common Factors in Psychotherapy Outcomes (2019) · cites this paper as Cuijpers et al. 2010
In the citing paper
“In recent meta-analyses, it has been shown that the treatment effects of psychotherapy have been overestimated in lower quality studies [35,36].”
From Positive psychology interventions: a meta-analysis of randomized controlled studies (2013) · cites this paper as [35]
8 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
An inventory for measuring depression
BECK, A. T. (1961) · Archives of General Psychiatry · doi:10.1001/archpsyc.1961.01710120031004 · Cited by 38,032
Hamilton, M. (1960) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.23.1.56 · Cited by 32,817
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Statistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
Evidence-Based Medicine: How to Practice and Teach EBM
Sackett, D. L. (2000) · Cited by 5,950
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,784
Defining empirically supported therapies.
Chambless, D. L., & Hollon, S. D. (1998) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.66.1.7 · Cited by 3,067 · cited 1×
“These criteria were based on an authoritative review of empirically supported psychotherapies (Chambless & Hollon, 1998), and on the criteria proposed by the Cochrane Collaboration to assess the methodological validity of a study (Higgins & Green, 2006).”
MethodSelf-citationKirsch, I., Deacon, B. J., Huedo-Medina, T. B., Scoboria, A., Moore, T. J., & Johnson, B. T. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050045 · Cited by 2,442
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Elkin, I. (1989) · Archives of General Psychiatry · doi:10.1001/archpsyc.1989.01810110013002 · Cited by 2,300 · cited 2×
“All but one of the high-quality psychotherapy studies was published within the previous decade and that one exception was planned and conducted by the National Institute of Mental Health (NIMH) as an example of how to perform high-quality treatment research (Elkin et al. 1989).”
BackgroundMeta-Analysis: Recent Developments in Quantitative Methods for Literature Reviews
Rosenthal, R., & DiMatteo, M. R. (2001) · Annual Review of Psychology · doi:10.1146/annurev.psych.52.1.59 · Cited by 1,780
Evidence-Based Medicine: How to Practice and Teach EBM
Satya-Murti, S. (2000) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.284.18.2382 · Cited by 1,643
Dimidjian, S., Hollon, S. D., Dobson, K. S., Schmaling, K. B., Kohlenberg, R. J., Addis, M. E., Gallop, R., McGlinchey, J. B., Markley, D. K., Gollan, J. K., et al. (2006) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.74.4.658 · Cited by 1,607 · cited 1×
“…no better than psychotherapy in several of the high-quality studies we examined (Elkin et al. 1989 ; Jarrett et al. 1999 ; DeRubeis et al. 2005 ; Dimidjian et al. 2006), and that earlier metaanalyses found that the effects of medication also are smaller than was previously assumed (Turner et…”
BackgroundSelf-citationBehavioral activation treatments of depression: a meta-analysis.
Cuijpers, P., van Straten, A., & Warmerdam, L. (2007) · Clinical Psychology Review · doi:10.1016/j.cpr.2006.11.001 · Cited by 1,144
Self-citationDesign Sensitivity: Statistical Power for Experimental Research.
Eliason, S. R., & Lipsey, M. W. (1991) · Contemporary Sociology · doi:10.2307/2072167 · Cited by 1,040
Cognitive Therapy vs Medications in the Treatment of Moderate to Severe Depression
DeRubeis, R. J., Hollon, S. D., Amsterdam, J. D., Shelton, R. C., Young, P. R., Salomon, R. M., O’Reardon, J. P., Lovett, M. L., Gladis, M. M., Brown, L. L., et al. (2005) · Archives of General Psychiatry · doi:10.1001/archpsyc.62.4.409 · Cited by 989
Self-citationEffects of psychotherapy for depression in children and adolescents: A meta-analysis
Weisz, J. R., McCarty, C. A., & Valeri, S. M. (2006) · Psychological Bulletin · doi:10.1037/0033-2909.132.1.132 · Cited by 942 · cited 1×
“Recent meta-analyses in the field of psychotherapy for depression in children and adolescents also found indications that the effects have been overestimated considerably in earlier meta-analytical research (Weisz et al. 2006 ; Klein et al. 2007).”
BackgroundSize of Treatment Effects and Their Importance to Clinical Research and Practice
Kraemer, H. C., & Kupfer, D. J. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.09.014 · Cited by 850
A meta-analysis of the effects of cognitive therapy in depressed patients
Gloaguen, V., Cottraux, J., Cucherat, M., & Blackburn, I.-M. (1998) · Journal of Affective Disorders · doi:10.1016/s0165-0327(97)00199-7 · Cited by 809
Design Sensitivity: Statistical Power for Experimental Research.
Lachenbruch, P. A., & Lipsey, M. W. (1990) · Journal of the American Statistical Association · doi:10.2307/2289636 · Cited by 782
Robinson, L. A., Berman, J. S., & Neimeyer, R. A. (1990) · Psychological Bulletin · doi:10.1037//0033-2909.108.1.30 · Cited by 671
Treating Depression in Predominantly Low-Income Young Minority Women: A Randomized Controlled Trial
Miranda, J., Chung, J. Y., Green, B. L., Krupnick, J., Siddique, J., Revicki, D. A., & Belin, T. (2003) · JAMA · doi:10.1001/jama.290.1.57 · Cited by 520
Williams, Jr., Barrett, J., Oxman, T., Frank, E., Katon, W., Sullivan, M., Cornell, J., & Sengupta, A. (2000) · JAMA · doi:10.1001/jama.284.12.1519 · Cited by 460
GELLATLY, J., BOWER, P., HENNESSY, S., RICHARDS, D., GILBODY, S., & LOVELL, K. (2007) · Psychological Medicine · doi:10.1017/s0033291707000062 · Cited by 419 · cited 3×
“2006), or specific delivery formats (McDermut et al. 2001 ; Gellatly et al. 2007), and only one of these studies even addressed the association between effect size and study quality (Gellatly et al.”
BackgroundA meta-analysis of randomized trials of behavioural treatment of depression
Ekers, D., Richards, D., & Gilbody, S. (2008) · Psychological Medicine · doi:10.1017/s0033291707001614 · Cited by 400 · cited 1×
“Meta-analyses in this field have consistently found that psychotherapies have moderate to large effect sizes (Gloaguen et al. 1998 ; Churchill et al. 2001 ; Wampold et al. 2002 ; Cuijpers et al. 2007a ; Ekers et al. 2008).”
BackgroundCooper, P., Murray, L., Wilson, A., & Romaniuk, H. (2003) · PubMed · Cited by 390
Churchill, R., Hunot, V., Corney, R., Knapp, M., McGuire, H., Tylee, A., & Wessely, S. (2002) · Health Technology Assessment · doi:10.3310/hta5350 · Cited by 368 · cited 1×
“Meta-analyses in this field have consistently found that psychotherapies have moderate to large effect sizes (Gloaguen et al. 1998 ; Churchill et al. 2001 ; Wampold et al. 2002 ; Cuijpers et al. 2007a ; Ekers et al. 2008).”
BackgroundCooper, P. J., Murray, L., Wilson, A., & Romaniuk, H. (2003) · British Journal of Psychiatry · doi:10.1192/bjp.182.5.412 · Cited by 356
de Mello, M. F., de Jesus Mari, J., Bacaltchuk, J., Verdeli, H., & Neugebauer, R. (2005) · European Archives of Psychiatry and Clinical Neuroscience · doi:10.1007/s00406-004-0542-x · Cited by 346 · cited 1×
“Most meta-analyses in this field have focused on specific types of psychotherapy (De Mello et al. 2005 ; Cuijpers et al. 2007b ; * Address for correspondence : P. Cuijpers, Ph.D., Professor of Clinical Psychology, Department of Clinical Psychology, VU University Amsterdam, Van der Boechorststraat…”
BackgroundA meta-(re) analysis of the effects of cognitive therapy versus ‘other therapies’ for depression
Wampold, B. E., Minami, T., Baskin, T. W., & Callen Tierney, S. (2002) · Journal of Affective Disorders · doi:10.1016/s0165-0327(00)00287-1 · Cited by 305
Problem solving therapies for depression: A meta-analysis
Cuijpers, P., van Straten, A., & Warmerdam, L. (2007) · European Psychiatry · doi:10.1016/j.eurpsy.2006.11.001 · Cited by 261 · cited 2×
“Most meta-analyses in this field have focused on specific types of psychotherapy (De Mello et al. 2005 ; Cuijpers et al. 2007b ; * Address for correspondence : P. Cuijpers, Ph.D., Professor of Clinical Psychology, Department of Clinical Psychology, VU University Amsterdam, Van der Boechorststraat…”
BackgroundSelf-citationKLEIN, J. B., JACOBS, R. H., & REINECKE, M. A. (2007) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/chi.0b013e3180592aaa · Cited by 254 · cited 1×
“Recent meta-analyses in the field of psychotherapy for depression in children and adolescents also found indications that the effects have been overestimated considerably in earlier meta-analytical research (Weisz et al. 2006 ; Klein et al. 2007).”
BackgroundPsychological treatment of depression: A meta-analytic database of randomized studies
Cuijpers, P., van Straten, A., Warmerdam, L., & Andersson, G. (2008) · BMC Psychiatry · doi:10.1186/1471-244x-8-36 · Cited by 241
Self-citationDowrick, C., Dunn, G., Ayuso-Mateos, J. L., Dalgard, O. S., Page, H., Lehtinen, V., Casey, P., Wilkinson, C., Vazquez-Barquero, J. L., & Wilkinson, G. (2000) · BMJ · doi:10.1136/bmj.321.7274.1450 · Cited by 228
Evidence-based medicine: how to practice and teach EBM (2nd edn).
Samuel, O. (2000) · Family Practice · doi:10.1093/fampra/17.4.356 · Cited by 215
McDermut, W., Miller, I. W., & Brown, R. A. (2001) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.8.1.98 · Cited by 206 · cited 2×
“2006), or specific delivery formats (McDermut et al. 2001 ; Gellatly et al. 2007), and only one of these studies even addressed the association between effect size and study quality (Gellatly et al.”
BackgroundPinquart, M., Duberstein, P. R., & Lyness, J. M. (2006) · American Journal of Psychiatry · doi:10.1176/ajp.2006.163.9.1493 · Cited by 204
Jarrett, R. B., Schaffer, M., McIntire, D., Witt-Browder, A., Kraft, D., & Risser, R. C. (1999) · Archives of General Psychiatry · doi:10.1001/archpsyc.56.5.431 · Cited by 174
Treating Depression During Pregnancy and the Postpartum: A Preliminary Meta-Analysis
Bledsoe, S. E., & Grote, N. K. (2006) · Research on Social Work Practice · doi:10.1177/1049731505282202 · Cited by 160
McDermut, W. (2001) · Clinical Psychology Science and Practice · doi:10.1093/clipsy/8.1.98 · Cited by 140
Characteristics of effective psychological treatments of depression: a metaregression analysis
Cuijpers, P. J., van Straten, A., Warmerdam, L., & Smits, N. (2008) · Psychotherapy Research · doi:10.1080/10503300701442027 · Cited by 127
Self-citationSchool-Based Cognitive-Behavioral Therapy for Adolescent Depression: A Benchmarking Study
Shirk, S. R., Kaplinski, H. C., & Gudmundsen, G. R. (2008) · Journal of Emotional and Behavioral Disorders · doi:10.1177/1063426608326202 · Cited by 115
Interpersonal Psychotherapy for Elderly Patients in Primary Care
van Schaik, A., van Marwijk, H., Adèr, H. J., van Dyck, R., de Haan, M., Penninx, B. W. J. H., van der Kooy, K. G., van Hout, H. P. J., & Beekman, A. T. F. (2006) · American Journal of Geriatric Psychiatry · doi:10.1097/01.jgp.0000199341.25431.4b · Cited by 110
Effectiveness of Long-term Psychodynamic Psychotherapy: A Meta-analysis
Leichsenring, F. (2008) · JAMA · doi:10.1001/jama.300.13.1551 · Cited by 65 · cited 3×
“…: p.cuijpers@psy.vu.nl) Psychological Medicine (2010), 40, 211–223. f Cambridge University Press 2009 doi:10.1017/S0033291709006114 ORIGINAL ARTICLE Leichsenring & Rabung, 2008), specific target groups (Bledsoe & Grote, 2006 ; Pinquart et al. 2006), or specific delivery formats (McDermut et al.…”
BackgroundMethodInfluentialDunn, N. J., Rehm, L. P., Schillaci, J., Souchek, J., Mehta, P. D., Ashton, C. M., Yanasak, E. V., & Hamilton, J. D. (2007) · Journal of Traumatic Stress · doi:10.1002/jts.20214 · Cited by 61
[Psychological treatment of depression; a systematic review of meta-analyses].
Cuijpers, P. J., & Dekker, J. (2005) · PubMed · Cited by 56
Self-citationSmit, A. C., Kluiter, H., Conradi, H. J., van der Meer, K., Tiemens, B. G., Jenner, J. A., Os, T. W. D. P. V., & Ormel, J. (2005) · Psychological Medicine · doi:10.1017/s0033291705006318 · Cited by 41
Poor-Quality Medical Research: What Can Journals Do?
Altman, D. G. (2002) · JAMA · doi:10.1001/jama.287.21.2765 · Cited by 34
Research works printed but not linked: 2
- Higgins JPT , Green S (2006). Cochrane Handbook for Systematic Reviews of Interventions 4.2.6 (updated September 2006). In The Cochrane Library, Issue 4, 2006. John Wiley & Sons, Ltd: Chichester, UK.
- 10.1037/0033-2909.108.1.30 · doi:10.1037/0033-2909.108.1.30 · Paper page
Cited by
Positive psychology interventions: a meta-analysis of randomized controlled studies
Bolier, L., Haverman, M., Westerhof, G. J., Riper, H., Smit, F., & Bohlmeijer, E. (2013) · BMC Public Health · doi:10.1186/1471-2458-13-119 · Cited by 1,996
“Effectiveness research in psychotherapy shows that effect sizes are relatively small in high-quality studies compared with low-quality studies [35] and this might also be true for positive psychology interventions.”
BackgroundResultSelf-citationPostpartum depression: current status and future directions
O'Hara, M. W., & McCabe, J. E. (2013) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050212-185612 · Cited by 1,750
PRACTICE GUIDELINE FOR THE Treatment of Patients With Major Depressive Disorder
Gelenberg, A. J., Freeman, P. B., Markowitz, J. C., Rosenbaum, J. F., Thase, M. E., Trivedi, M. H., & Rhoads, R. S. (2010) · Cited by 1,510
Relative Deprivation: A Theoretical and Meta-Analytic Review
Smith, H. J., Pettigrew, T. F., Pippin, G. M., & Bialosiewicz, S. (2012) · Personality and Social Psychology Review · doi:10.1177/1088868311430825 · Cited by 1,255
Cuijpers, P., Berking, M., Andersson, G., Quigley, L., Kleiboer, A., & Dobson, K. S. (2013) · The Canadian Journal of Psychiatry · doi:10.1177/070674371305800702 · Cited by 1,066
Self-citation
Metadata
- Authors
- Cuijpers, P., van Straten, A., Bohlmeijer, E., Hollon, S. D., & Andersson, G.
- Affiliations
- Amsterdam Public Health (NL)
- Vanderbilt University (US)
- Linköping University (SE)
- Corresponding author
- Cuijpers, P.
- Venue
- Psychological Medicine
- Publisher
- Cambridge University Press (CUP)
- Published
- 2009-06-03
- Volume
- 40
- Issue
- 2
- Pages
- 211-223
- ISSN
- 0033-2917 (ISSN-L)
- 1469-8978
- OpenAlex
- W2396755030
- Cited by
- 533
- References
- 50
- Citations by year
- 2026: 18 · 2025: 26 · 2024: 43 · 2023: 27 · 2022: 36 · 2021: 31 · 2020: 38 · 2019: 38 · 2018: 31 · 2017: 29 · 2016: 31 · 2015: 30 · 2014: 41 · 2013: 35 · 2012: 24
- FWCI
- 3.51
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Medicine (publishedVersion)
- PubMed (publishedVersion)
- VU Research Portal (publishedVersion)
- University of Twente Research Information (publishedVersion)
- Digital Academic REpository of VU University Amsterdam (Vrije Universiteit Amsterdam) (submittedVersion)
- University of Twente Research Information (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Keywords
- psychotherapy, adult depression, study quality, effect size overestimation, randomized controlled trials, implementation fidelity, control conditions, number needed to treat, heterogeneity
- MeSH terms
- Adult, Major Depressive Disorder, Humans, Psychotherapy, Severity of Illness Index, Cohort Effect, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.