The effects of cognitive behavioral therapy are not systematically falling: A revision of Johnsen and Friborg (2015).
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
In a meta-analysis, Johnsen and Friborg (2015) reported a significant negative relationship between publication year and the effect sizes (ESs) of cognitive-behavioral therapy (CBT) for depressive disorders, suggesting its effectiveness was falling. We identified a series of methodological and conceptual caveats and consequently redid the meta-analysis. We used the same inclusion criteria, but only included randomized controlled trials and searched for additional eligible trials. We computed both within-group and between-group ESs for the CBT arm for the Beck Depression Inventory (BDI) and the Hamilton Rating Scale for Depression (HRSD). We assessed risk of bias, sample size, type of control group, and the study's country of origin and conducted subgroup, single, and multiple meta-regression analyses including publication year and other moderators. We identified 30 additional eligible trials. Within-group ESs presented huge heterogeneity estimates (I2 around 90%). Year of publication was significant in some single meta-regression analyses on the BDI, but not significant in others, in most analyses on the HRSD, and in any of the analyses on between-group ESs. Multiple regression models indicated that either year was not significantly related or that both year and country were significantly related to outcomes, with a temporal trend present solely in US studies. Year of publication does not appear to be a reliable and independent moderator of the effectiveness of CBT for depression. The linear "fall" reported by Johnsen and Friborg (2015) is most likely a spurious finding. (PsycINFO Database Record
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Elkin, I. (1989) · Archives of General Psychiatry · doi:10.1001/archpsyc.1989.01810110013002 · Cited by 2,300
Intention-to-treat concept: A review
Gupta, S. (2011) · Perspectives in Clinical Research · doi:10.4103/2229-3485.83221 · Cited by 1,921
Dimidjian, S., Hollon, S. D., Dobson, K. S., Schmaling, K. B., Kohlenberg, R. J., Addis, M. E., Gallop, R., McGlinchey, J. B., Markley, D. K., Gollan, J. K., et al. (2006) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.74.4.658 · Cited by 1,607
Self-citationA component analysis of cognitive-behavioral treatment for depression.
Jacobson, N. S., Dobson, K. S., Truax, P. A., Addis, M. E., Koerner, K., Gollan, J. K., Gortner, E., & Prince, S. E. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.64.2.295 · Cited by 1,413
Uncertainty in heterogeneity estimates in meta-analyses
Ioannidis, J. P. A., Patsopoulos, N. A., & Evangelou, E. (2007) · BMJ · doi:10.1136/bmj.39343.408449.80 · Cited by 1,254 · cited 2×
“We calculated 95% confidence intervals (CIs) around I2 (Ioannidis et al., 2007), using the noncentral 2-based approach with the heterogi module for STATA (Orsini, Bottai, Higgins, & Buchan, 2006).”
MethodResultCuijpers, P., Berking, M., Andersson, G., Quigley, L., Kleiboer, A., & Dobson, K. S. (2013) · The Canadian Journal of Psychiatry · doi:10.1177/070674371305800702 · Cited by 1,066 · cited 2×
“T hi s ar tic le is in te nd ed so le ly fo r th e pe rs on al us e of th e in di vi du al us er an d is no t to be di ss em in at ed br oa dl y. outliers, in line with previous meta-analyses of CBT compared to a control group (Barth et al., 2013; Cuijpers et al., 2013).”
ResultSelf-citationRush, A. J., Beck, A. T., Kovacs, M., & Hollon, S. (1977) · Cognitive Therapy and Research · doi:10.1007/bf01173502 · Cited by 913
Self-citationCuijpers, P., Donker, T., van Straten, A., Li, J., & Andersson, G. (2010) · Psychological Medicine · doi:10.1017/s0033291710000772 · Cited by 832
Self-citationForman, E. M., Herbert, J. D., Moitra, E., Yeomans, P. D., & Geller, P. A. (2007) · Behavior Modification · doi:10.1177/0145445507302202 · Cited by 687
Barth, J., Munder, T., Gerger, H., Nüesch, E., Trelle, S., Znoj, H., Jüni, P., & Cuijpers, P. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001454 · Cited by 569 · cited 5×
“T hi s ar tic le is in te nd ed so le ly fo r th e pe rs on al us e of th e in di vi du al us er an d is no t to be di ss em in at ed br oa dl y. outliers, in line with previous meta-analyses of CBT compared to a control group (Barth et al., 2013; Cuijpers et al., 2013).”
BackgroundResultInfluentialSelf-citationCuijpers, P., van Straten, A., Bohlmeijer, E., Hollon, S. D., & Andersson, G. (2010) · Psychological Medicine · doi:10.1017/s0033291709006114 · Cited by 533
Self-citationFurukawa, T. A., Noma, H., Caldwell, D. M., Honyashiki, M., Shinohara, K., Imai, H., Chen, P., Hunot, V., & Churchill, R. (2014) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12275 · Cited by 521 · cited 2×
“Moreover, other moderator variables could also be relevant, such as the type of control group used, given that the effects of CBT have been shown to vary with different control groups (Furukawa et al., 2014).”
BackgroundShapiro, D. A., Barkham, M., Rees, A., Hardy, G. E., Reynolds, S., & Startup, M. (1994) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.62.3.522 · Cited by 485
Cognitive Therapy and Pharmacotherapy for Depression: Singly and in Combination
Hollon, S. D. (1992) · Archives of General Psychiatry · doi:10.1001/archpsyc.1992.01820100018004 · Cited by 481 · cited 2×
“Much less consideration has been given to the actual methodology by which these results came about in Johnsen and Friborg (2015). Nevertheless, we identified a number of questionable or unclear aspects.”
BackgroundMethodSelf-citationPropst, L. R., Ostrom, R., Watkins, P., & Dean, T. (1992) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.60.1.94 · Cited by 423
Kennedy, S. H., Konarski, J. Z., Segal, Z. V., Lau, M. A., Bieling, P. J., McIntyre, R. S., & Mayberg, H. S. (2007) · American Journal of Psychiatry · doi:10.1176/ajp.2007.164.5.778 · Cited by 420
Wiles, N., Thomas, L., Abel, A., Ridgway, N., Turner, N., Campbell, J., Garland, A., Hollinghurst, S., Jerrom, B., Kessler, D., et al. (2013) · The Lancet · doi:10.1016/s0140-6736(12)61552-9 · Cited by 340
Mohr, D. C., Boudewyn, A. C., Goodkin, D. E., Bostrom, A., & Epstein, L. (2001) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.69.6.942 · Cited by 334
Johnsen, T. J., & Friborg, O. (2015) · Psychological Bulletin · doi:10.1037/bul0000015 · Cited by 331 · cited 51×
“Similar to Johnsen and Friborg (2015), we used the BDI and HRSD as outcome measures.”
BackgroundMethodResultInfluentialWright, J. H., Wright, A. S., Albano, A. M., Basco, M. R., Goldsmith, L. J., Raffield, T., & Otto, M. W. (2005) · American Journal of Psychiatry · doi:10.1176/appi.ajp.162.6.1158 · Cited by 319
Wagner, B., Horn, A. B., & Maercker, A. (2014) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.06.032 · Cited by 311
Cognitive-Behavioral Therapy for Depression in Parkinson's Disease: A Randomized, Controlled Trial
Dobkin, R. D., Menza, M., Allen, L. A., Gara, M. A., Mark, M. H., Tiu, J., Bienfait, K. L., & Friedman, J. (2011) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.10111669 · Cited by 310
Treatment of Depressive Symptoms in Human Immunodeficiency Virus–Positive Patients
Markowitz, J. C., Kocsis, J. H., Fishman, B., Spielman, L. A., Jacobsberg, L. B., Frances, A. J., Klerman, G. L., & Perry, S. W. (1998) · Archives of General Psychiatry · doi:10.1001/archpsyc.55.5.452 · Cited by 268
Nuesch, E., Trelle, S., Reichenbach, S., Rutjes, A. W. S., Burgi, E., Scherer, M., Altman, D. G., & Juni, P. (2009) · BMJ · doi:10.1136/bmj.b3244 · Cited by 248
Treatment of Depression After Coronary Artery Bypass Surgery: A Randomized Controlled Trial
Freedland, K. E., Skala, J. A., Carney, R. M., Rubin, E. H., Lustman, P. J., Dávila-Román, V. G., Steinmeyer, B. C., & Hogue, C. W. (2009) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2009.7 · Cited by 228
Gallagher-Thompson, D., & Steffen, A. M. (1994) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.62.3.543 · Cited by 221
Driessen, E., Cuijpers, P., Hollon, S. D., & Dekker, J. J. M. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0020570 · Cited by 208
Self-citationSerfaty, M. A., Haworth, D., Blanchard, M., Buszewicz, M., Murad, S., & King, M. (2009) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2009.165 · Cited by 190
Tovote, K. A., Fleer, J., Snippe, E., Peeters, A. C. T. M., Emmelkamp, P. M. G., Sanderman, R., Links, T. P., & Schroevers, M. J. (2014) · Diabetes Care · doi:10.2337/dc13-2918 · Cited by 163
Cuijpers, P., Weitz, E., Twisk, J., Kuehner, C., Cristea, I., David, D., DeRubeis, R. J., Dimidjian, S., Dunlop, B. W., Faramarzi, M., et al. (2014) · Depression and Anxiety · doi:10.1002/da.22328 · Cited by 158 · cited 1×
“Some of the moderators considered by Johnsen and Friborg (2015) (gender, baseline severity) have been shown to not be related to outcomes in meta-analyses of CBT or psychotherapy for depression (Cuijpers et al., 2014; Driessen, Cuijpers, Hollon, & Dekker, 2010).”
BackgroundSelf-citationO'Mahen, H., Himle, J. A., Fedock, G., Henshaw, E., & Flynn, H. (2013) · Depression and Anxiety · doi:10.1002/da.22050 · Cited by 136
A Clinical Trial of In-Home CBT for Depressed Mothers in Home Visitation
Ammerman, R. T., Putnam, F. W., Altaye, M., Stevens, J., Teeters, A. R., & Van Ginkel, J. B. (2013) · Behavior Therapy · doi:10.1016/j.beth.2013.01.002 · Cited by 123
Ashman, T., Cantor, J. B., Tsaousides, T., Spielman, L., & Gordon, W. (2014) · Journal of Head Trauma Rehabilitation · doi:10.1097/htr.0000000000000098 · Cited by 83
Armijo-Olivo, S., Fuentes, J., Ospina, M., Saltaji, H., & Hartling, L. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-116 · Cited by 60
Cited by
Digital interventions for the treatment of depression: a meta-analytic review
Moshe, I., Terhorst, Y., Philippi, P., Domhardt, M., Cuijpers, P., Cristea, I., Pulkki-Råback, L., Baumeister, H., & Sander, L. B. (2021) · Psychological Bulletin · doi:10.1037/bul0000334 · Cited by 390
Self-citationCuijpers, P. (2017) · Canadian Psychology / Psychologie canadienne · doi:10.1037/cap0000096 · Cited by 231
Self-citation50 years of rational-emotive and cognitive-behavioral therapy: a systematic review and meta-analysis
David, D., Cotet, C., Matu, S., Mogoase, C., & Stefan, S. (2018) · Journal of Clinical Psychology · doi:10.1002/jclp.22514 · Cited by 179
Self-citationBarry, T. J., Hallford, D. J., & Takano, K. (2021) · Psychological Bulletin · doi:10.1037/bul0000345 · Cited by 103
Bear, H. A., Edbrooke-Childs, J., Norton, S., Krause, K. R., & Wolpert, M. (2020) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2019.12.002 · Cited by 101
Metadata
- Authors
- Cristea, I. A., Stefan, S., Karyotaki, E., David, D., Hollon, S. D., & Cuijpers, P.
- Venue
- Psychological Bulletin
- Published
- 2017-03
- Volume
- 143
- Issue
- 3
- Pages
- 326-340
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2589763153
- Cited by
- 50
- References
- 34
- FWCI
- 5.06
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.