The cumulative effect of reporting and citation biases on the apparent efficacy of treatments: the case of depression
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Evidence-based medicine is the cornerstone of clinical practice, but it is dependent on the quality of evidence upon which it is based. Unfortunately, up to half of all randomized controlled trials (RCTs) have never been published, and trials with statistically significant findings are more likely to be published than those without (Dwan et al., 2013). Importantly, negative trials face additional hurdles beyond study publication bias that can result in the disappearance of non-significant results (Boutron et al., 2010; Dwan et al., 2013; Duyx et al., 2017). Here, we analyze the cumulative impact of biases on apparent efficacy, and discuss possible remedies, using the evidence base for two effective treatments for depression: Antidepressants and psychotherapy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Dwan, K., Gamble, C., Williamson, P. R., Kirkham, J. J., & the Reporting Bias Group (2013) · PLoS ONE · doi:10.1371/journal.pone.0066844 · Cited by 1,127
Boutron, I. (2010) · JAMA · doi:10.1001/jama.2010.651 · Cited by 842 · cited 2×
“Reporting strategies that could distort the interpretation of results and mislead readers are defined as spin (Boutron et al., 2010).”
BackgroundTrial Publication after Registration in ClinicalTrials.Gov: A Cross-Sectional Analysis
Ross, J. S., Mulvey, G. K., Hines, E. M., Nissen, S. E., & Krumholz, H. M. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000144 · Cited by 401 · cited 1×
“gov were published within two years of completion (Ross et al., 2009), and non-reporting of protocol-specified outcomes or the silent addition of new outcomes is also common (Jones et al.”
BackgroundBoutron, I., Altman, D. G., Hopewell, S., Vera-Badillo, F., Tannock, I., & Ravaud, P. (2014) · Journal of Clinical Oncology · doi:10.1200/jco.2014.56.7503 · Cited by 368
Comparison of registered and published outcomes in randomized controlled trials: a systematic review
Jones, C. W., Keil, L. G., Holland, W. C., Caughey, M. C., & Platts-Mills, T. F. (2015) · BMC Medicine · doi:10.1186/s12916-015-0520-3 · Cited by 237
Driessen, E., Hollon, S. D., Bockting, C. L. H., Cuijpers, P., & Turner, E. H. (2015) · PLOS ONE · doi:10.1371/journal.pone.0137864 · Cited by 208 · cited 1×
“Of 55 National Institutes of Health-funded psychotherapy trials, 13 (24%) remained unpublished (Driessen et al., 2015), and these had a markedly lower effect size than the published trials.”
BackgroundSelf-citationScientific citations favor positive results: a systematic review and meta-analysis.
Duyx, B., Urlings, M. J. E., Swaen, G. M. H., Bouter, L. M., & Zeegers, M. P. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.06.002 · Cited by 197
Knüppel, H., Metz, C., Meerpohl, J. J., & Strech, D. (2013) · PLoS ONE · doi:10.1371/journal.pone.0075995 · Cited by 31 · cited 1×
“The International Committee of Medical Journal Editors (ICMJE) began requiring prospective registration of clinical trials as a precondition for publication in 2005, but many journals do not require registration (Knüppel et al., 2013) and others allow Fig.”
Backgroundde Vries, Y. A., Roest, A. M., Franzen, M., Munafò, M. R., & Bastiaansen, J. A. (2016) · Psychological Medicine · doi:10.1017/s0033291716000805 · Cited by 28
Self-citation
Cited by
Research Weaving: Visualizing the Future of Research Synthesis
Nakagawa, S., Samarasinghe, G., Haddaway, N. R., Westgate, M. J., O’Dea, R. E., Noble, D. W. A., & Lagisz, M. (2019) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2018.11.007 · Cited by 261
More treatment but no less depression: The treatment-prevalence paradox
Ormel, J., Hollon, S. D., Kessler, R. C., Cuijpers, P., & Monroe, S. M. (2022) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102111 · Cited by 213
Self-citationProtecting against researcher bias in secondary data analysis: challenges and potential solutions
Baldwin, J. R., Pingault, J.-B., Schoeler, T., Sallis, H. M., & Munafò, M. R. (2022) · European Journal of Epidemiology · doi:10.1007/s10654-021-00839-0 · Cited by 178
“While this is a positive outcome for research integrity, null results are less likely to be published [13, 41, 42] and cited [11], which could disadvantage researchers’ careers.”
ResultTriangulating Evidence through the Inclusion of Genetically Informed Designs
Munafò, M. R., Higgins, J. P. T., & Smith, G. D. (2021) · Cold Spring Harbor Perspectives in Medicine · doi:10.1101/cshperspect.a040659 · Cited by 125
Hengartner, M. P. (2020) · Therapeutic Advances in Psychopharmacology · doi:10.1177/2045125320921694 · Cited by 72
Metadata
- Authors
- de Vries, Y. A., Roest, A. M., de Jonge, P., Cuijpers, P., Munafò, M. R., & Bastiaansen, J. A.
- Venue
- Psychological Medicine
- Publisher
- Cambridge University Press (CUP)
- Published
- 2018-08-02
- Volume
- 48
- Issue
- 15
- Pages
- 2453-2455
- ISSN
- 0033-2917 (ISSN-L)
- 1469-8978
- OpenAlex
- W2887011384
- Cited by
- 98
- References
- 10
- FWCI
- 7.58
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The cumulative effect of reporting and citation biases on the apparent efficacy of treatments: the case of depression. https://app.cochat.ai/papers/10.1017/s0033291718001873
- The paper
- de Vries, Y. A., Roest, A. M., de Jonge, P., Cuijpers, P., Munafò, M. R., & Bastiaansen, J. A. (2018). The cumulative effect of reporting and citation biases on the apparent efficacy of treatments: the case of depression. Psychological Medicine. https://doi.org/10.1017/s0033291718001873
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.