Does Publication Bias Inflate the Apparent Efficacy of Psychological Treatment for Major Depressive Disorder? A Systematic Review and Meta-Analysis of US National Institutes of Health-Funded Trials
Open accessThe DOI matches its Crossref record; 88 of the 89 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The efficacy of antidepressant medication has been shown empirically to be overestimated due to publication bias, but this has only been inferred statistically with regard to psychological treatment for depression. We assessed directly the extent of study publication bias in trials examining the efficacy of psychological treatment for depression. METHODS AND FINDINGS: We identified US National Institutes of Health grants awarded to fund randomized clinical trials comparing psychological treatment to control conditions or other treatments in patients diagnosed with major depressive disorder for the period 1972-2008, and we determined whether those grants led to publications. For studies that were not published, data were requested from investigators and included in the meta-analyses. Thirteen (23.6%) of the 55 funded grants that began trials did not result in publications, and two others never started. Among comparisons to control conditions, adding unpublished studies (Hedges' g = 0.20; CI95% -0.11~0.51; k = 6) to published studies (g = 0.52; 0.37~0.68; k = 20) reduced the psychotherapy effect size point estimate (g = 0.39; 0.08~0.70) by 25%. Moreover, these findings may overestimate the "true" effect of psychological treatment for depression as outcome reporting bias could not be examined quantitatively. CONCLUSION: The efficacy of psychological interventions for depression has been overestimated in the published literature, just as it has been for pharmacotherapy. Both are efficacious but not to the extent that the published literature would suggest. Funding agencies and journals should archive both original protocols and raw data from treatment trials to allow the detection and correction of outcome reporting bias. Clinicians, guidelines developers, and decision makers should be aware that the published literature overestimates the effects of the predominant treatments for depression.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“After adding the effect sizes of these unpublished trials to those of the published ones, the mean effect size for psychotherapy dropped by more than 25%.”
From How effective are cognitive behavior therapies for major depression and anxiety disorders? A meta-analytic update of the evidence (2016) · cites this paper as [15]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“a recent study found that almost one quarter of trials of psychotherapy for adult depression funded by the US National Institutes of Health were not published 15 .”
From How effective are cognitive behavior therapies for major depression and anxiety disorders? A meta-analytic update of the evidence (2016) · cites this paper as [15]
The cited source says what the claim attributes to it.
In the citing paper
“However, this was a relatively small drop compared to that reported in other studies on publication bias in psychotherapies for MDD 15,18,184 .”
From How effective are cognitive behavior therapies for major depression and anxiety disorders? A meta-analytic update of the evidence (2016) · cites this paper as [15]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Hollon, S. D., DeRubeis, R. J., Fawcett, J., Amsterdam, J. D., Shelton, R. C., Zajecka, J., Young, P. R., & Gallop, R. (2014) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2014.1054 · Cited by 179
Self-citationRetractedAn inventory for measuring depression
BECK, A. T. (1961) · Archives of General Psychiatry · doi:10.1001/archpsyc.1961.01710120031004 · Cited by 38,032
Hamilton, M. (1960) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.23.1.56 · Cited by 32,817
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
Zhang, Q., Wang, C., Zheng, M., Li, Y., Li, J., Zhang, L., Shang, X., & Yan, C. (2015) · Cerebrovascular Diseases · doi:10.1159/000369778 · Cited by 11,533
Projections of Global Mortality and Burden of Disease from 2002 to 2030
Mathers, C. D., & Loncar, D. (2006) · PLoS Medicine · doi:10.1371/journal.pmed.0030442 · Cited by 11,386
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Cited by
Time for united action on depression: a Lancet–World Psychiatric Association Commission
Herrman, H., Patel, V., Kieling, C., Berk, M., Buchweitz, C., Cuijpers, P., Furukawa, T. A., Kessler, R. C., Kohrt, B. A., Maj, M., et al. (2022) · The Lancet · doi:10.1016/s0140-6736(21)02141-3 · Cited by 1,104
Self-citationMalhi, G. S., Bassett, D., Boyce, P., Bryant, R., Fitzgerald, P. B., Fritz, K., Hopwood, M., Lyndon, B., Mulder, R., Murray, G., et al. (2015) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867415617657 · Cited by 842
“The scientific literature on psychological therapies is also influenced by publication bias (increased likelihood that positive outcome studies will be published) (Driessen et al., 2015) and researcher allegiance effects (Munder et al.”
BackgroundExercise as a treatment for depression: A meta-analysis
Kvam, S., Kleppe, C. L., Nordhus, I. H., & Hovland, A. (2016) · Journal of Affective Disorders · doi:10.1016/j.jad.2016.03.063 · Cited by 837
““file drawer problem,” where nonsignificant or negative findings are less likely to be published than positive findings (Rosenthal, 1979), may have led to an overestimation of the effect of psychological treatments and antidepressant medication for depression (Driessen et al., 2015) and this problem could also apply t…”
BackgroundKuyken, W., Warren, F. C., Taylor, R. S., Whalley, B., Crane, C., Bondolfi, G., Hayes, R., Huijbers, M., Ma, H., Schweizer, S., et al. (2016) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2016.0076 · Cited by 821
Malhi, G. S., Bell, E., Bassett, D., Boyce, P., Bryant, R., Hazell, P., Hopwood, M., Lyndon, B., Mulder, R., Porter, R., et al. (2021) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867420979353 · Cited by 614
“Short-term psychodynamic therapy appears to be effective in the treatment of depression, both as monotherapy (Driessen et al., 2015a, 2017) and as an adjunct to antidepressant medication (Driessen et al., 2020) and recent findings await robust replication.”
BackgroundCuijpers, P., Cristea, I. A., Karyotaki, E., Reijnders, M., & Huibers, M. J. H. (2016) · World Psychiatry · doi:10.1002/wps.20346 · Cited by 572
Self-citationEvidence Base Update of Psychosocial Treatments for Child and Adolescent Depression
Weersing, V. R., Jeffreys, M., Do, M.-C. T., Schwartz, K. T. G., & Bolano, C. (2017) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2016.1220310 · Cited by 505
“…meta-analytic evidence on treatment effects in youth and adulthood suggest that a ceiling may have been reached with current depression treatments (or that early effects may have been overestimates; Driessen, Hollon, Bockting, Cuijpers, & Turner, 2015; Johnsen & Friborg, 2015; Weisz et al., 2006).”
BackgroundLeichsenring, F., Steinert, C., Rabung, S., & Ioannidis, J. P. A. (2022) · World Psychiatry · doi:10.1002/wps.20941 · Cited by 485
User‐centered design for psychosocial intervention development and implementation.
Lyon, A. R., & Koerner, K. (2016) · Clinical Psychology: Science and Practice · doi:10.1111/cpsp.12154 · Cited by 478
Assessing risk of bias due to missing results in a synthesis
Page, M. J., Higgins, J. P., & Sterne, J. A. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch13 · Cited by 402
O. B. O. T. C.-S. G., Grant, S., Mayo-Wilson, E., Montgomery, P., Macdonald, G., Michie, S., Hopewell, S., & Moher, D. (2018) · Trials · doi:10.1186/s13063-018-2735-z · Cited by 348
“These inefficiencies contribute to suboptimal dissemination of effective interventions [29, 30], overestimation of intervention efficacy [31], and research waste [29].”
Backgroundon behalf of the CONSORT-SPI Group, Montgomery, P., Grant, S., Mayo-Wilson, E., Macdonald, G., Michie, S., Hopewell, S., & Moher, D. (2018) · Trials · doi:10.1186/s13063-018-2733-1 · Cited by 327
Sharma, T., Guski, L. S., Freund, N., & Gøtzsche, P. C. (2016) · BMJ · doi:10.1136/bmj.i65 · Cited by 319
The effects of fifteen evidence-supported therapies for adult depression: A meta-analytic review
Cuijpers, P., Karyotaki, E., de Wit, L., & Ebert, D. D. (2020) · Psychotherapy Research · doi:10.1080/10503307.2019.1649732 · Cited by 276
Self-citationPsychological Treatment of Depression in Primary Care: Recent Developments
Cuijpers, P., Quero, S., Dowrick, C., & Arroll, B. (2019) · Current Psychiatry Reports · doi:10.1007/s11920-019-1117-x · Cited by 296
Self-citationCuijpers, P. (2017) · Canadian Psychology / Psychologie canadienne · doi:10.1037/cap0000096 · Cited by 231
Self-citationCatalogue of bias: publication bias
DeVito, N. J., & Goldacre, B. (2019) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2018-111107 · Cited by 225
Schmucker, C. M., Blümle, A., Schell, L. K., Schwarzer, G., Oeller, P., Cabrera, L., von Elm, E., Briel, M., Meerpohl, J. J., & on behalf of the OPEN consortium (2017) · PLOS ONE · doi:10.1371/journal.pone.0176210 · Cited by 227
More treatment but no less depression: The treatment-prevalence paradox
Ormel, J., Hollon, S. D., Kessler, R. C., Cuijpers, P., & Monroe, S. M. (2022) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102111 · Cited by 213
Self-citationTargets and Outcomes of Psychotherapies for Mental Disorders: An Overview
Cuijpers, P. (2019) · World Psychiatry · doi:10.1002/wps.20661 · Cited by 206
Psychology's Replication Crisis and Clinical Psychological Science
Tackett, J. L., Brandes, C. M., King, K. M., & Markon, K. E. (2019) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050718-095710 · Cited by 198
“Various lines of evidence have confirmed long-standing concerns that the psychotherapy research literature itself has been affected by publication bias (e.g., Cuijpers et al. 2010,Driessen et al. 2015, Sohn 1996).”
Bennett, S. D., Cuijpers, P., Ebert, D. D., McKenzie Smith, M., Coughtrey, A. E., Heyman, I., Manzotti, G., & Shafran, R. (2019) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13010 · Cited by 162
“Significant small study effects for studies comparing against a control group, as is often found in studies of psychological interventions (Driessen, Hollon, Bockting, Cuijpers, & Turner, 2015), may have led to an overestimation of the effect of self-help against control groups.”
Self-citationWas Eysenck right after all? A reassessment of the effects of psychotherapy for adult depression
Cuijpers, P., Karyotaki, E., Reijnders, M., & Ebert, D. D. (2019) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796018000057 · Cited by 173
“In depression research, several other types have been tested, such as interpersonal psychotherapy (Cuijpers et al. 2016b), counselling (Cuijpers et al. 2012) and brief psychodynamic therapies (Driessen et al. 2015a).”
BackgroundSelf-citationA historical review of publication bias
Marks‐Anglin, A., & Chen, Y. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1452 · Cited by 165
Kline, A. C., Cooper, A. A., Rytwinksi, N. K., & Feeny, N. C. (2018) · Clinical Psychology Review · doi:10.1016/j.cpr.2017.10.009 · Cited by 162
Treating major depression with yoga: A prospective, randomized, controlled pilot trial
Prathikanti, S., Rivera, R., Cochran, A., Tungol, J. G., Fayazmanesh, N., & Weinmann, E. (2017) · PLOS ONE · doi:10.1371/journal.pone.0173869 · Cited by 157
Bandelow, B., Allgulander, C., Baldwin, D. S., Costa, D. L. D. C., Denys, D., Dilbaz, N., Domschke, K., Eriksson, E., Fineberg, N. A., Hättenschwiler, J., et al. (2023) · The World Journal of Biological Psychiatry · doi:10.1080/15622975.2022.2086295 · Cited by 158
“Publication bias is an issue both in pharmacotherapy and psychotherapy studies (Driessen et al. 2015; Turner et al. 2008).”
BackgroundSchoevers, R. A., van Borkulo, C. D., Lamers, F., Servaas, M. N., Bastiaansen, J. A., Beekman, A. T. F., van Hemert, A. M., Smit, J. H., Penninx, B. W. J. H., & Riese, H. (2021) · Psychological Medicine · doi:10.1017/s0033291720000689 · Cited by 149
“This lack of diagnostic specificity likely plays a role in the overall modest efficacy of current psychological and pharmacological treatments (Driessen, Hollon, Bockting, Cuijpers, & Turner, 2015; Turner, Matthews, Linardatos, Tell, & Rosenthal, 2008).”
BackgroundHistory repeating: guidelines to address common problems in psychedelic science
van Elk, M., & Fried, E. I. (2023) · Therapeutic Advances in Psychopharmacology · doi:10.1177/20451253231198466 · Cited by 137
“39 This means that significant findings in trials are much more likely to be published in scientific journals, whereas nonsignificant findings are often not published, inflating meta-analytic effect sizes and threatening statistical conclusion validity.”
BackgroundReplication Research, Publication Bias, and Applied Behavior Analysis
Tincani, M., & Travers, J. C. (2019) · Perspectives on Behavior Science · doi:10.1007/s40614-019-00191-5 · Cited by 144
How to prove that your therapy is effective, even when it is not: a guideline
Cuijpers, P., & Cristea, I. A. (2016) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796015000864 · Cited by 122
Self-citationWho benefits from psychotherapies for adult depression? A meta-analytic update of the evidence
Cuijpers, P., Karyotaki, E., Reijnders, M., & Huibers, M. J. H. (2018) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2017.1420098 · Cited by 115
Self-citationDoes psychotherapy work? An umbrella review of meta‐analyses of randomized controlled trials
Dragioti, E., Karathanos, V., Gerdle, B., & Evangelou, E. (2017) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12713 · Cited by 111
“This has been observed in other studies that have shown that the results of up to 25% of the psychotherapy RCTs have never been published (11, 14).”
BackgroundResultInfluentialCoronado-Montoya, S., Levis, A. W., Kwakkenbos, L., Steele, R. J., Turner, E. H., & Thombs, B. D. (2016) · PLOS ONE · doi:10.1371/journal.pone.0153220 · Cited by 107
“39 when data for non-published studies was integrated [63].”
BackgroundSelf-citationOrmel (2019) · doi:10.1097/yco.0000000000000505 · Cited by 102
de Vries, Y. A., Roest, A. M., de Jonge, P., Cuijpers, P., Munafò, M. R., & Bastiaansen, J. A. (2018) · Psychological Medicine · doi:10.1017/s0033291718001873 · Cited by 98
“Of 55 National Institutes of Health-funded psychotherapy trials, 13 (24%) remained unpublished (Driessen et al., 2015), and these had a markedly lower effect size than the published trials.”
BackgroundSelf-citationVogt, K. S., & Norman, P. (2019) · Psychology and Psychotherapy: Theory, Research and Practice · doi:10.1111/papt.12194 · Cited by 98
O’Connor, E., Perdue, L. A., Coppola, E. L., Henninger, M. L., Thomas, R. G., & Gaynes, B. N. (2023) · JAMA · doi:10.1001/jama.2023.7787 · Cited by 128
Bradley, H. A., Rucklidge, J. J., & Mulder, R. T. (2017) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12647 · Cited by 92
Cybulski, L., Mayo-Wilson, E., & Grant, S. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000115 · Cited by 90
Collaborative care for anxiety disorders in primary care: a systematic review and meta-analysis
Muntingh, A. D., van der Feltz-Cornelis, C. M., van Marwijk, H. W., Spinhoven, P., & van Balkom, A. J. (2016) · BMC Family Practice · doi:10.1186/s12875-016-0466-3 · Cited by 90
Antipsychotic polypharmacy and metabolic syndrome in schizophrenia: a review of systematic reviews
Ijaz, S., Bolea-Alamanac, B. M., Davies, S., Savović, J., Richards, A., Sullivan, S. A., & Moran, P. (2018) · BMC Psychiatry · doi:10.1186/s12888-018-1848-y · Cited by 115
Chu, C.-S., Stubbs, B., Chen, T.-Y., Tang, C.-H., Li, D.-J., Yang, W.-C., Wu, C.-K., Carvalho, A. F., Vieta, E., Miklowitz, D. J., et al. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.08.025 · Cited by 84
“efficacy of both pharmacotherapy and psychological treatment in clinical trials due to publication bias and the selective reporting of ‘positive’ findings (Driessen et al., 2015; Turner et al., 2008).”
BackgroundBiases in research: risk factors for non-replicability in psychotherapy and pharmacotherapy research
Leichsenring, F., Abbass, A., Hilsenroth, M. J., Leweke, F., Luyten, P., Keefe, J. R., Midgley, N., Rabung, S., Salzer, S., & Steinert, C. (2017) · Psychological Medicine · doi:10.1017/s003329171600324x · Cited by 78
“According to two recent meta-analyses, the effects of psychotherapy for depressionalso seemtobeoverestimateddue topublication bias (Cuijpers et al. 2010; Driessen et al. 2015).”
MethodAre Transbronchial Cryobiopsies Ready for Prime Time? A Systematic Review and Meta-Analysis
Sethi, J. S., Ali, M. S., Mohananey, D., Nanchal, R. S., Maldonado, F., & Musani, A. I. (2018) · Journal of Bronchology & Interventional Pulmonology · doi:10.1097/lbr.0000000000000519 · Cited by 83
O'Daffer, A., Colt, S. F., Wasil, A. R., & Lau, N. (2022) · JMIR Mental Health · doi:10.2196/40924 · Cited by 72
“It is plausible that similar concerns could be present in digital mental health interventions [20], especially in cases where companies are explicitly funding, sponsoring, or participating in clinical trials.”
BackgroundNull Effects and Publication Bias in Special Education Research
Cook, B. G., & Therrien, W. J. (2017) · Behavioral Disorders · doi:10.1177/0198742917709473 · Cited by 82
“Similarly, Driessen et al. (2015) indicated that 11 of the 13 unpublished studies in their review were never submitted for publication.”
BackgroundDietch, D., Kerr‐Gaffney, J., Hockey, M., Marx, W., Ruusunen, A. A., Young, A. H., Berk, M., & Mondelli, V. (2023) · BJPsych Open · doi:10.1192/bjo.2023.36 · Cited by 85
Duracinský, M., Lalanne, C., Rous, L., Dara, A. F., Baudoin, L., Pellet, C., Descamps, A., Péretz, F., & Chassany, O. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0371-z · Cited by 73
Eggart, M., Todd, J., & Valdés‐Stauber, J. (2021) · PLoS ONE · doi:10.1371/journal.pone.0253913 · Cited by 85
Cuijpers, P., Quero, S., Papola, D., Cristea, I. A., & Karyotaki, E. (2021) · Psychological Medicine · doi:10.1017/s0033291719003581 · Cited by 62
Self-citationDepression and psychodynamic psychotherapy
Ribeiro, Â. D. S. B., Ribeiro, J. P., & von Doellinger, O. (2017) · Brazilian Journal of Psychiatry · doi:10.1590/1516-4446-2016-2107 · Cited by 93
Selective outcome reporting and the effectiveness of psychotherapies for depression
Miguel, C., Karyotaki, E., Cuijpers, P., & Cristea, I. A. (2021) · World Psychiatry · doi:10.1002/wps.20900 · Cited by 54
Self-citationImproving the rigor of psychophysiology research
Baldwin, S. A. (2016) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2016.04.006 · Cited by 57
Ahmadpanah, M., Akbari, T., Akhondi, A., Haghighi, M., Jahangard, L., Bahmani, D. S., Bajoghli, H., Holsboer‐Trachsler, E., & Brand, S. (2017) · Psychiatry Research · doi:10.1016/j.psychres.2017.07.030 · Cited by 66
“This is especially important as Driessen et al. (2015) have shown that a publication bias might inflate the apparent efficacy of psychological treatments for MDDs (as is also the case for pharmacotherapy).”
Toward a paradigm shift in treatment and research of mental disorders
Leichsenring, F., Steinert, C., & Ioannidis, J. P. A. (2019) · Psychological Medicine · doi:10.1017/s0033291719002265 · Cited by 53
“For key mental disorders such as depressive disorders (Driessen et al., 2015; Cipriani et al., 2018; Cuijpers et al., 2019), anxiety disorders (Heeren et al.”
BackgroundInfluentialHengartner, M. P. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00256 · Cited by 60
“For instance, Driessen et al. (2015) found that 24% of all trials aimed at evaluating the efficacy of psychological treatments for major depression funded by the National Institutes of Health were not published, which led to a 25% reduction in the estimated efficacy of psychotherapy (g = 0.52 vs. g…”
BackgroundInfluentialDigital mental health interventions for the treatment of depression: A multiverse meta-analysis
Plessen, C. Y., Panagiotopoulou, O. M., Tong, L., Cuijpers, P., & Karyotaki, E. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.10.018 · Cited by 53
Self-citationLorenzo-Luaces, L., Zimmerman, M., & Cuijpers, P. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2018.02.066 · Cited by 42
Self-citationRogge, H., Artelt, N., Endlich, N., & Endlich, K. (2017) · Journal of Microscopy · doi:10.1111/jmi.12593 · Cited by 40
The effects of psychological treatment of perinatal depression: an overview
Cuijpers, P. J., & Karyotaki, E. (2021) · Archives of Women s Mental Health · doi:10.1007/s00737-021-01159-8 · Cited by 46
“Furthermore, publication bias is a major problem affecting the overall pooled effect sizes in metaanalyses of psychotherapies for depression (Driessen et al. 2015; Cuijpers et al. 2019a).”
BackgroundSelf-citationTreanor, L., Frank, R. A., Cherpak, L. A., Dehmoobad Sharifabadi, A., Salameh, J.-P., Hallgrimson, Z., Fabiano, N., McGrath, T. A., Kraaijpoel, N., Yao, J., et al. (2020) · European Radiology · doi:10.1007/s00330-019-06568-z · Cited by 38
Some properties of p-curves, with an application to gradual publication bias.
Ulrich, R., & Miller, J. O. (2018) · Psychological Methods · doi:10.1037/met0000125 · Cited by 38
(2017) · Europe PMC (PubMed Central) · Cited by 146
Amad, A., Jardri, R., Rousseau, C., Larochelle, Y., Ioannidis, J. P. A., & Naudet, F. (2019) · Psychotherapy and Psychosomatics · doi:10.1159/000502805 · Cited by 36
Evaluation of publication bias in response interruption and redirection: A meta‐analysis
Dowdy, A., Tincani, M., & Schneider, W. J. (2020) · Journal of Applied Behavior Analysis · doi:10.1002/jaba.724 · Cited by 37
Transparent, Open, and Reproducible Prevention Science
Grant, S., Wendt, K. E., Leadbeater, B. J. R., Supplee, L. H., Mayo‐Wilson, E., Gardner, F. E. M., & Bradshaw, C. P. (2022) · Prevention Science · doi:10.1007/s11121-022-01336-w · Cited by 46
“An evaluation of trials funded by the National Institute of Mental Health found that studies with small effects were less likely to be published than studies with large effects, inflating the apparent effectiveness of psychotherapies (Driessen et al., 2015).”
MethodHeyman, R. E., Slep, A. M. S., Lorber, M. F., Mitnick, D. M., Xu, S., Baucom, K. J. W., Halford, W. K., & Niolon, P. H. (2018) · Prevention Science · doi:10.1007/s11121-018-0961-y · Cited by 42
Recent Developments in the Treatment of Depression
Hollon, S. D., Cohen, Z. D., Singla, D. R., & Andrews, P. W. (2019) · Behavior Therapy · doi:10.1016/j.beth.2019.01.002 · Cited by 40
Self-citationKetamine effects on anxiety and fear-related behaviors: Current literature evidence and new findings
Silote, G. P., de Oliveira, S. F. S., Ribeiro, D. E., Machado, M. S., Andreatini, R., Joca, S. R. L., & Beijamini, V. (2020) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2020.109878 · Cited by 35
The Impact of Self-assessment on Academic Performance: A Meta-analysis Study
Karaman, P. (2021) · International Journal of Research in Education and Science · doi:10.46328/ijres.2344 · Cited by 60
“However, some evidence showed that published studies tend to show greater treatment effect than unpublished studies (Conn et al., 2003; Driessen et al., 2015; Hopewell et al, 2007).”
ResultTime to publication for results of clinical trials
Showell, M. G., Cole, S., Clarke, M. J., DeVito, N., Farquhar, C. M., & Jordan, V. (2024) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000011.pub3 · Cited by 57
Psychological factors for the onset of depression: a meta-analysis of prospective studies
Fu, Z., Brouwer, M. E., Kennis, M., Williams, A. D., Cuijpers, P. J., & Bockting, C. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-050129 · Cited by 45
Self-citationGrey matters; on the importance of publication bias in systematic reviews
Winters, M., & Weir, A. (2016) · British Journal of Sports Medicine · doi:10.1136/bjsports-2016-096679 · Cited by 36
Sylvester, R. J., Canfield, S. E., Lam, T. B. L., Marconi, L. S., MacLennan, S., Yuan, Y., MacLennan, G., Norrie, J. D., Omar, M. I., Bruins, H. M., et al. (2016) · European Urology · doi:10.1016/j.eururo.2016.11.023 · Cited by 37
“Seitz et al also found evidence of publication bias, which can lead to overestimation of treatment effects and compromise the validity of the MA findings [40].”
BackgroundNegative Results: Conceptual and Methodological Dimensions in Single-Case Intervention Research
Kratochwill, T. R., Levin, J. R., & Horner, R. H. (2017) · Remedial and Special Education · doi:10.1177/0741932517741721 · Cited by 35
History repeating: A roadmap to address common problems in psychedelic sciencePreprint
van Elk, M., & Fried, E. I. (2023) · doi:10.31234/osf.io/ak6gx · Cited by 30
Characteristics of mental health trials registered in ClinicalTrials.gov
Arnow, K. D., King, A. C., & Wagner, T. H. (2019) · Psychiatry Research · doi:10.1016/j.psychres.2019.112552 · Cited by 28
Representation and Extrapolation: Evidence from Clinical Trials
Alsan, M. M., Durvasula, M., Gupta, H., Schwartzstein, J., & Williams, H. (2023) · The Quarterly Journal of Economics · doi:10.1093/qje/qjad036 · Cited by 50
Baumel, A., Mathur, N., Pawar, A. K. S., & Muench, F. J. (2021) · Journal of Child and Family Studies · doi:10.1007/s10826-020-01863-6 · Cited by 33
Multidimensional perfectionism and poor sleep: A meta-analysis of bivariate associations
Stricker, J., Kröger, L., Johann, A. F., Küskens, A., Gieselmann, A., & Pietrowsky, R. (2022) · Sleep Health · doi:10.1016/j.sleh.2022.09.015 · Cited by 30
Cuijpers, P., & Gentili, C. (2017) · Research in Psychotherapy: Psychopathology, Process and Outcome · doi:10.4081/ripppo.2017.273 · Cited by 24
“…treatment (Malouff, Thorsteins- son, & Schutte, 2007), life review therapy in older adults (Lan, Xiao, & Chen, 2017), non-directive counseling (Cuijpers et al., 2012), psychodynamic therapies (Driessen et al., 2015; Leichsenring & Rabung, 2008) and third wave therapies (Churchill et al., 2013).”
BackgroundSelf-citationMeaningful change definitions: sample size planning for experimental intervention research
Gruijters, S. L. K., & Peters, G.-J. (2020) · Psychology and Health · doi:10.1080/08870446.2020.1841762 · Cited by 35
“This means that the effect sizes in published studies exhibit an upward bias of unknown magnitude (e.g., Driessen et al., 2015).”
ResultStatistical power in clinical trials of interventions for mood, anxiety, and psychotic disorders
de Vries, Y. A., Schoevers, R. A., Higgins, J. P. T., Munafo, M. R., & Bastiaansen, J. A. (2022) · Psychological Medicine · doi:10.1017/s0033291722001362 · Cited by 28
“…been inflated by reporting bias (de Vries et al., 2018; Roest et al., 2015; Turner, Knoepflmacher, & Shapley, 2012; Turner, Matthews, Linardatos, Tell, & Rosenthal, 2008), and the same is probably true for psychotherapy (de Vries et al., 2018; Driessen, Hollon, Bockting, Cuijpers, & Turner, 2015).”
There’s More to Groove than Bass in Electronic Dance Music: Why Some People Won’t Dance to Techno
Wesolowski, B. C., & Hofmann, A. (2016) · PLoS ONE · doi:10.1371/journal.pone.0163938 · Cited by 36
de Wit, H. M., Groen, M. T., Rovers, M. M., & Tack, C. J. (2016) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.12925 · Cited by 29
A Systematic Review of the Heterogeneity of Schema Therapy
van Dijk, S. D. M., Veenstra, M. S., van den Brink, R. H. S., van Alphen, S. P. J., & Voshaar, R. C. O. (2023) · Journal of Personality Disorders · doi:10.1521/pedi.2023.37.2.262 · Cited by 35
The antidepressant standoff: Why it continues and how to resolve it
Ormel, J., Spinhoven, P., de Vries, Y. A., Cramer, A. O. J., Siegle, G. J., Bockting, C. L. H., & Hollon, S. D. (2020) · Psychological Medicine · doi:10.1017/s0033291719003295 · Cited by 22
“Similar to ADM trials, accumulating biases have also resulted in overestimation of the short- and long-term efficacy of psychological treatments (Cuijpers et al., 2016; Driessen et al., 2015).”
Self-citationIncrease value and reduce waste in research on psychological therapies
Cristea, I. A., & Naudet, F. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103479 · Cited by 22
Homeopathy in the treatment of depression: a systematic review
Viksveen, P., Fibert, P., & Relton, C. (2018) · European Journal of Integrative Medicine · doi:10.1016/j.eujim.2018.07.004 · Cited by 22
Exploring the efficacy of psychotherapies for depression: a multiverse meta-analysis
Plessen, C. Y., Karyotaki, E., Miguel, C., Čihařová, M., & Cuijpers, P. J. (2023) · BMJ Mental Health · doi:10.1136/bmjment-2022-300626 · Cited by 28
“This is consistent with previous research, which has found significant publication bias for psychotherapies of major depressive disorder, 41 for digital psychological interventions for depression 42 and for most evidence-supported therapies for adult depression.”
ResultSelf-citationEvolutionary theory and the treatment of depression: It is all about the squids and the sea bass
Hollon, S. D., Andrews, P. W., Singla, D. R., Maslej, M. M., & Mulsant, B. H. (2021) · Behaviour Research and Therapy · doi:10.1016/j.brat.2021.103849 · Cited by 23
Self-citationSundell, K., & Olsson, T. (2017) · Oxford Bibliographies in Social Work · doi:10.1093/obo/9780195389678-0254 · Cited by 21
Identifying key-conservation areas for Posidonia oceanica seagrass beds
Houngnandan, F., Kéfi, S., & Déter, J. (2020) · Biological Conservation · doi:10.1016/j.biocon.2020.108546 · Cited by 22
Ulrich, R., Miller, J. O., & Erdfelder, E. (2018) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000319 · Cited by 21
Finazzi, E., & MacBeth, A. M. (2021) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2650 · Cited by 22
Introduction to Special Issue: Null Effects and Publication Bias in Learning Disabilities Research
Therrien, W. J., & Cook, B. G. (2018) · Learning Disabilities Research and Practice · doi:10.1111/ldrp.12163 · Cited by 22
Reporting bias in imaging: higher accuracy is linked to faster publication
Sharifabadi, A. D., Korevaar, D. A., McGrath, T. A., van Es, N., Frank, R. A., Cherpak, L. A., Dang, W., Salameh, J., Nguyen, F., Stanley, C., et al. (2018) · European Radiology · doi:10.1007/s00330-018-5354-x · Cited by 21
Faggion, C. M., Monje, A., & Wasiak, J. (2018) · Journal Of Clinical Periodontology · doi:10.1111/jcpe.12893 · Cited by 18
Halicka, M., Parkhouse, T., Webster, K. E., Spiga, F., Hines, L. A., Freeman, T. P., Sanghera, S., Dawson, S., Paterson, C. A., Savović, J., et al. (2025) · Addiction · doi:10.1111/add.70084 · Cited by 24
Metadata
- Authors
- Driessen, E., Hollon, S. D., Bockting, C. L. H., Cuijpers, P., & Turner, E. H.
- Affiliations
- Amsterdam Public Health (NL)
- Vanderbilt University (US)
- University of Groningen (NL)
- Oregon Health & Science University (US)
- Corresponding author
- Driessen, Ellen
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-09-30
- Volume
- 10
- Issue
- 9
- Pages
- e0137864
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4589340
- OpenAlex
- W2201728466
- Cited by
- 208
- References
- 89
- Citations by year
- 2026: 9 · 2025: 8 · 2024: 14 · 2023: 16 · 2022: 18 · 2021: 14 · 2020: 16 · 2019: 23 · 2018: 27 · 2017: 30 · 2016: 28 · 2015: 5
- FWCI
- 4.97
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- VU Research Portal (publishedVersion, other-oa)
- Ghent University Academic Bibliography (Ghent University) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Pure Amsterdam UMC (submittedVersion, other-oa)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Funders
- National Institutes of Health
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Health Sciences › Medicine › Psychiatry and Mental health › Mental Health Treatment and Access
- Keywords
- publication bias, major depressive disorder, psychological intervention, psychotherapy efficacy, unpublished trials, selective reporting, data archiving, effect size overestimation
- MeSH terms
- Antidepressive Agents, Clinical Trials as Topic, Depressive Disorder, Major, Major Depressive Disorder, Financing, Government, Humans, National Institutes of Health (U.S.), Psychotherapy, United States, Treatment Outcome, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.