Considering the methodological limitations in the evidence base of antidepressants for depression: a reanalysis of a network meta-analysis
Open accessThe DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Objectives: To investigate whether the conclusion of a recent systematic review and network meta-analysis (Cipriani et al) that antidepressants are more efficacious than placebo for adult depression was supported by the evidence. Design: Reanalysis of a systematic review, with meta-analyses. Data sources: 522 trials (116 477 participants) as reported in the systematic review by Cipriani et al and clinical study reports for 19 of these trials. Analysis: We used the Cochrane Handbook’s risk of bias tool and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to evaluate the risk of bias and the certainty of evidence, respectively. The impact of several study characteristics and publication status was estimated using pairwise subgroup meta-analyses. Results: Several methodological limitations in the evidence base of antidepressants were either unrecognised or underestimated in the systematic review by Cipriani et al. The effect size for antidepressants versus placebo on investigator-rated depression symptom scales was higher in trials with a ‘placebo run-in’ study design compared with trials without a placebo run-in design (p=0.05). The effect size of antidepressants was higher in published trials compared with unpublished trials (p<0.0001). The outcome data reported by Cipriani et al differed from the clinical study reports in 12 (63%) of 19 trials. The certainty of the evidence for the placebo-controlled comparisons should be very low according to GRADE due to a high risk of bias, indirectness of the evidence and publication bias. The mean difference between antidepressants and placebo on the 17-item Hamilton depression rating scale (range 0–52 points) was 1.97 points (95% CI 1.74 to 2.21). Conclusions: The evidence does not support definitive conclusions regarding the benefits of antidepressants for depression in adults. It is unclear whether antidepressants are more efficacious than placebo.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Studies without active placebo are considered to be at unclear or high risk of performance bias (Munkholm 2019).”
From Approaches for discontinuation versus continuation of long-term antidepressant use for depressive and anxiety disorders in adults (2021) · cites this paper as Munkholm 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The efficacy of antidepressant treatment for depression is modest compared with placebo [2, 3]”
From The relationship between dose and serotonin transporter occupancy of antidepressants—a systematic review (2022) · cites this paper as [2]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cipriani, A., Furukawa, T. A., Salanti, G., Chaimani, A., Atkinson, L. Z., Ogawa, Y., Leucht, S., Ruhe, H. G., Turner, E. H., Higgins, J. P. T., et al. (2018) · The Lancet · doi:10.1016/s0140-6736(17)32802-7 · Cited by 3,541 · cited 10×
“They classified the trials as low risk of bias if none of the domains assessed were rated as high risk of bias and three or less were rated as unclear risk; moderate if one domain was rated as high risk of bias or none were rated as high risk of bias but four or more were rated as unclear risk; and all other cases wer…”
InfluentialThe cost of dichotomising continuous variables
Altman, D. G., & Royston, P. (2006) · BMJ · doi:10.1136/bmj.332.7549.1080 · Cited by 2,688
Kirsch, I., Deacon, B. J., Huedo-Medina, T. B., Scoboria, A., Moore, T. J., & Johnson, B. T. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050045 · Cited by 2,442
Selective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435 · cited 2×
“antidepressant trials remain unpublished or are inadequately reported.(9) Cipriani et al(5) included 436 published and 86 unpublished studies, but as many as a thousand antidepressant studies may have been conducted.”
BackgroundIntHout, J., Ioannidis, J. P., & Borm, G. F. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-25 · Cited by 2,419 · cited 1×
“For the comparisons between antidepressants and placebo on rating scales, we used the Hartung-Knapp-SidikJonkman approach because it results in fewer type I errors than the DerSimonian and Laird approach.(16) We based our analyses on the number of participants from Cipriani et al’s ‘efficacy’ analyses.”
MethodAntidepressant Drug Effects and Depression Severity: A Patient-Level Meta-analysis
Fournier, J. C., DeRubeis, R. J., Hollon, S. D., Dimidjian, S., Amsterdam, J. D., Shelton, R. C., & Fawcett, J. (2010) · JAMA · doi:10.1001/jama.2009.1943 · Cited by 2,009
Yatham, L. N., Kennedy, S. H., Parikh, S. V., Schaffer, A., Bond, D. J., Frey, B. N., Sharma, V., Goldstein, B. I., Rej, S., Beaulieu, S., et al. (2018) · Bipolar Disorders · doi:10.1111/bdi.12609 · Cited by 1,852
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
GRADE guidelines: 8. Rating the quality of evidence—indirectness
Guyatt, G. H., Oxman, A. D., Kunz, R., Woodcock, J., Brozek, J., Helfand, M., Alonso-Coello, P., Falck-Ytter, Y., Jaeschke, R., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.04.014 · Cited by 1,843 · cited 1×
“Downgrading due to the indirectness of the network meta-analysis’ methodology should also be considered.(38) Taken together, the certainty of evidence should be ‘very low’.”
BackgroundIndustry sponsorship and research outcome
Lundh, A., Lexchin, J., Mintzes, B., Schroll, J. B., & Bero, L. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000033.pub3 · Cited by 1,697 · cited 1×
“Some argue that ‘vested interests’ should also be considered, since industry sponsored drug studies lead to more favourable effects than other studies by mechanisms that are not explained by the usual bias domains.(26) We explored whether industry sponsorship was associated with larger effect estimates, by performing…”
Fava, G. A., Gatti, A., Belaise, C., Guidi, J., & Offidani, E. (2015) · Psychotherapy and Psychosomatics · doi:10.1159/000370338 · Cited by 436 · cited 2×
“symptoms during the placebo run-in that are alleviated by the study drug.(44) It could be misinterpreted as an improvement of the depression (increased benefits in the drug group).”
BackgroundJakobsen, J. C., Katakam, K. K., Schou, A., Hellmuth, S. G., Stallknecht, S. E., Leth-Møller, K., Iversen, M., Banke, M. B., Petersen, I. J., Klingenberg, S. L., et al. (2017) · BMC Psychiatry · doi:10.1186/s12888-016-1173-2 · Cited by 366
Antidepressants versus placebo for depression in primary care
Arroll, B., Elley, C. R., Fishman, T., Goodyear-Smith, F. A., Kenealy, T., Blashki, G., Kerse, N., & MacGillivray, S. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007954 · Cited by 352
Active placebos versus antidepressants for depression
Moncrieff, J., Wessely, S., & Hardy, R. (2004) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003012.pub2 · Cited by 342 · cited 1×
“The degree of unblinding is extensive and leads to inflated effect estimates,(20) and smaller effects have been observed when the trials were better blinded by adding atropine to the placebo.(21) Given these issues, all placebo-controlled trials of antidepressants should arguably be categorised as at least unclear, or…”
BackgroundSharma, T., Guski, L. S., Freund, N., & Gøtzsche, P. C. (2016) · BMJ · doi:10.1136/bmj.i65 · Cited by 319
Strategies for dealing with missing data in clinical trials: from design to analysis.
(2013) · PubMed Central · Cited by 288
Antidepressant discontinuation reactions: Are preventable and simple to treat
Haddad, P., Lejoyeux, M., & Young, A. (1998) · BMJ · doi:10.1136/bmj.316.7138.1105 · Cited by 286
Page, M. J., Higgins, J. P. T., Clayton, G., Sterne, J. A. C., Hróbjartsson, A., & Savović, J. (2016) · PLOS ONE · doi:10.1371/journal.pone.0159267 · Cited by 284 · cited 1×
“Trials at high or unclear risk of bias within these two domains are likely to report inflated effect estimates, especially of subjective outcomes.(19) Cipriani et al did not describe how they assessed the risk of bias in relation to the randomisation sequence generation or the allocation concealment, and we were there…”
MethodGibbons, R. D., Hur, K., Brown, C. H., Davis, J. M., & Mann, J. J. (2012) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.2044 · Cited by 271
Effectiveness of antidepressants: an evidence myth constructed from a thousand randomized trials?
Ioannidis, J. P. (2008) · Philosophy, Ethics, and Humanities in Medicine · doi:10.1186/1747-5341-3-14 · Cited by 253 · cited 2×
“There are many methodological limitations in trials of antidepressant agents,(12) of which many have been acknowledged for decades.(13) Research aiming to inform clinical practice on the use of antidepressants for depression must recognise these limitations.”
BackgroundDrug safety assessment in clinical trials: methodological challenges and opportunities
Singh, S., & Loke, Y. K. (2012) · Trials · doi:10.1186/1745-6215-13-138 · Cited by 227 · cited 1×
“A further consequence of a short follow-up period is an underestimation of serious and non-serious adverse events.(28)”
BackgroundDouble-Blind, Randomized Comparison of Mirtazapine and Paroxetine in Elderly Depressed Patients
Schatzberg, A. F., Kremer, C., Rodrigues, H. E., & Murphy, G. M. (2002) · The American Journal of Geriatric Psychiatry · doi:10.1097/00019442-200209000-00007 · Cited by 215
Furukawa, T. A., Salanti, G., Atkinson, L. Z., Leucht, S., Ruhe, H. G., Turner, E. H., Chaimani, A., Ogawa, Y., Takeshima, N., Hayasaka, Y., et al. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010919 · Cited by 214 · cited 1×
“trial duration and long-term effects Cipriani et al(5) extracted outcome data as close to 8 weeks follow-up as possible within an interval of 4–12 weeks,(5) but they did not provide a rationale for this decision.(22) The common clinical practice is to prescribe antidepressants for much longer periods.”
BackgroundLeucht, S., Fennema, H., Engel, R., Kaspers–Janssen, M., Lepping, P., & Szegedi, A. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2012.12.001 · Cited by 211 · cited 1×
“scale of three points or less.(46) Others have interpreted the same data”
BackgroundThe Imperative to Share Clinical Study Reports: Recommendations from the Tamiflu Experience
Doshi, P., Jefferson, T., & Del Mar, C. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001201 · Cited by 165 · cited 1×
“Considering the multiple methodological limitations we have identified, it would be necessary to analyse data based on clinical study reports and individual patient data to make a reliable assessment of the benefits and harms of antidepressants because they are the most reliable source of trial data.(40) There are als…”
ResultDouble-Blind, Randomized Comparison of Mirtazapine and Paroxetine in Elderly Depressed Patients
Schatzberg, A. F. (2002) · American Journal of Geriatric Psychiatry · doi:10.1176/appi.ajgp.10.5.541 · Cited by 146
Moncrieff, J., & Kirsch, I. (2015) · Contemporary Clinical Trials · doi:10.1016/j.cct.2015.05.005 · Cited by 142 · cited 1×
“875, are necessary for a clinician to detect a minimal clinical improvement.(47) We found that the”
BackgroundOpening up data at the European Medicines Agency
Gotzsche, P. C., & Jorgensen, A. W. (2011) · BMJ · doi:10.1136/bmj.d2686 · Cited by 106 · cited 1×
“We cross-referenced the included trials with the clinical study reports that we previously obtained from the European Medicines Agency in 2010.(15) We compared the outcomes of total dropout rates and dropouts due to adverse events as reported in the clinical study reports with the data reported by Cipriani et al.”
A Reevaluation of the Exclusion Criteria Used in Antidepressant Efficacy Trials
Posternak, M. A., Zimmerman, M., Keitner, G. I., & Miller, I. W. (2002) · American Journal of Psychiatry · doi:10.1176/appi.ajp.159.2.191 · Cited by 97 · cited 1×
“These include psychiatric comorbidities, alcohol abuse, long duration of illness and ‘non-response’ to previous antidepressant treatment.(36) The majority of patients in a clinical setting would not be eligible to enter randomised trials due to such exclusion criteria,(37) and the evidence coming from these trials is…”
BackgroundPigott, H. E. (2015) · The Canadian Journal of Psychiatry · doi:10.1177/070674371506000104 · Cited by 93 · cited 1×
“Observational studies also indicate that the effectiveness of antidepressants in practice is very low: in the large, publicly funded, Sequenced Treatment Alternatives to Relieve Depression study, only 3% of the 4041 enrolled patients were considered ‘in remission’ after 1 year.(39) The recent finding based on clinical…”
BackgroundClinical trials and the response rate illusion
Kirsch, I., & Moncrieff, J. (2007) · Contemporary Clinical Trials · doi:10.1016/j.cct.2006.10.012 · Cited by 80 · cited 3×
“It is a problematic outcome because it lacks clinical relevance and it may create an illusion of clinical effectiveness.(33) Dichotomisation of outcomes measured on rating scales leads to loss of statistical power, and it increases the risk of false positive results(34) and spuriously inflated effect sizes.”
BackgroundInfluentialTurner, E. H., & Rosenthal, R. (2008) · BMJ · doi:10.1136/bmj.39510.531597.80 · Cited by 78
Hughes, S., Cohen, D., & Jaggi, R. (2014) · BMJ Open · doi:10.1136/bmjopen-2014-005535 · Cited by 72 · cited 1×
“harms, including death, of antidepressants are often not reported in published papers.(30)”
BackgroundNHS prescribed record number of antidepressants last year
Iacobucci, G. (2019) · BMJ · doi:10.1136/bmj.l1508 · Cited by 70
Huijbregts, K. M., Hoogendoorn, A., Slottje, P., van Balkom, A. J. L. M., & Batelaan, N. M. (2017) · Psychotherapy and Psychosomatics · doi:10.1159/000480456 · Cited by 68
Relevance of Exclusion Criteria in Antidepressant Clinical Trials: A Replication Study
Zetin, M., & Hoepner, C. T. (2007) · Journal of Clinical Psychopharmacology · doi:10.1097/jcp.0b013e318058263f · Cited by 61
Eiring, Ø., Landmark, B. F., Aas, E., Salkeld, G., Nylenna, M., & Nytrøen, K. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-007848 · Cited by 57
Woolley, S. B., Cardoni, A. A., & Goethe, J. W. (2009) · Pharmacotherapy: The Journal of Human Pharmacology and Drug Therapy · doi:10.1592/phco.29.12.1408 · Cited by 52
Hengartner, M. P. (2017) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2017.00275 · Cited by 52
Assuring That Double-Blind Is Blind
Perlis, R. H., Ostacher, M. J., D, M. F. M., Nierenberg, A. A., Sachs, G. S., & Rosenbaum, J. F. (2010) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2009.09060820 · Cited by 51
Effectiveness of antidepressants
McCormack, J., & Korownyk, C. (2018) · BMJ · doi:10.1136/bmj.k1073 · Cited by 43 · cited 1×
“The choice made by Cipriani et al(5) to report only the relative ORs and not the trials’ absolute response rates has been criticised.(35) However, even the absolute response rates are of limited clinical relevance.”
BackgroundPlacebo‐Controlled trial of paroxetine in depressive illness
Edwards, J. G., & Goldie, A. (1993) · Human Psychopharmacology: Clinical and Experimental · doi:10.1002/hup.470080306 · Cited by 14
Network meta-analysis of antidepressants
Boesen, K., Paludan‐Müller, A. S., & Munkholm, K. (2018) · The Lancet · doi:10.1016/s0140-6736(18)31783-5 · Cited by 10 · cited 1×
“We have already addressed some of the limitations in the risk of bias assessment in the Cipriani et al review.(14) However, given the potential implications of Cipriani et al’s review,(5) we here aimed to provide a more comprehensive assessment.”
MethodSelf-citation
Research works printed but not linked: 1
- Sharma T Effects of selective serotonin reuptake inhibitors (SSRIs) and serotonin - norepinephrine reuptake inhibitors (SNRIs) on suicidality, violence, and quality of life (PhD Thesis): University of Copenhagen, 2018.
Cited by
Leichsenring, F., Steinert, C., Rabung, S., & Ioannidis, J. P. A. (2022) · World Psychiatry · doi:10.1002/wps.20941 · Cited by 485
von Rotz, R., Schindowski, E. M., Jungwirth, J., Schuldt, A., Rieser, N. M., Zahoranszky, K., Seifritz, E., Nowak, A., Nowak, P., Jäncke, L., et al. (2023) · eClinicalMedicine · doi:10.1016/j.eclinm.2022.101809 · Cited by 354
Galvão-Coelho, N. L., Marx, W., Gonzalez, M., Sinclair, J., de Manincor, M., Perkins, D., & Sarris, J. (2021) · Psychopharmacology · doi:10.1007/s00213-020-05719-1 · Cited by 137
O’Connor, E., Perdue, L. A., Coppola, E. L., Henninger, M. L., Thomas, R. G., & Gaynes, B. N. (2023) · JAMA · doi:10.1001/jama.2023.7787 · Cited by 126
Hengartner, M. P., & Plöderl, M. (2022) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2020-111600 · Cited by 114
Metadata
- Authors
- Munkholm, K., Paludan-Müller, A. S., & Boesen, K.
- Affiliations
- Rigshospitalet (DK)
- Corresponding author
- Munkholm, Klaus
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2019-06
- Volume
- 9
- Issue
- 6
- Pages
- e024886
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC6597641
- OpenAlex
- W2954134279
- Semantic Scholar
- 1f02503fe1b4290d23f35fdcf0ce927d794271ed
- Cited by
- 151
- References
- 43
- Citations by year
- 2026: 13 · 2025: 15 · 2024: 17 · 2023: 31 · 2022: 20 · 2021: 29 · 2020: 15 · 2019: 11
- FWCI
- 3.80
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- Locations
- BMJ Open (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Research at the University of Copenhagen (University of Copenhagen) (publishedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- HighWire Press Open Archive (submittedVersion)
- Funders
- Nordic Cochrane Centre
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Pharmacology (medical) › Treatment of Major Depression
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Keywords
- antidepressants, depression, placebo response, risk of bias, publication bias, clinical study reports, quality of evidence, Hamilton Depression Rating Scale, antidepressant efficacy
- MeSH terms
- Network Meta-Analysis, Network Meta-Analysis as Topic, Systematic Reviews as Topic, Adult, Antidepressive Agents, Depression, Humans, Meta-Analysis as Topic, Bias, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.