Statistically Significant Antidepressant-Placebo Differences on Subjective Symptom-Rating Scales Do Not Prove That the Drugs Work: Effect Size and Method Bias Matter!
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
Following the publication of a recent meta-analysis by Cipriani et al. (1), various opinion leaders and news reports claimed that the effectiveness of antidepressants has been definitely proven (2).E.g., Dr. Pariante, spokesperson for the Royal College of Psychiatrists, stated that this study "finally puts to bed the controversy on antidepressants, clearly showing that these drugs do work in lifting mood and helping most people with depression" (https://www.theguardian.com/ science/2018/feb/21/the-drugs-do-work-antidepressants-are-effective-study-shows).We surely would embrace drug treatments that effectively help most people with depression, but based on work that has contested the validity of mostly industry-sponsored antidepressant trials (3-6) we remain skeptical about antidepressants' clinical benefits.The most recent meta-analysis indeed concludes that antidepressants are more effective than placebo but also acknowledges that risk of bias was substantial and that the mean effect size of d = 0.3 was modest (1).Unfortunately, no clarification is given what this effect size means and whether it can be expected to be clinically significant in real-world routine practice.In this opinion paper we therefore ponder over how the reported effect size of d = 0.3 relates to clinical significance and how method bias undermines its validity, in order that the public, clinicians, and patients can judge for themselves whether antidepressants clearly work in most people with depression.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The ASA Statement on p -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Cipriani, A., Furukawa, T. A., Salanti, G., Chaimani, A., Atkinson, L. Z., Ogawa, Y., Leucht, S., Ruhe, H. G., Turner, E. H., Higgins, J. P. T., et al. (2018) · The Lancet · doi:10.1016/s0140-6736(17)32802-7 · Cited by 3,541
A practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490 · cited 1×
“A statistically significant result neither proves that the null hypothesis is false nor that the alternative hypothesis is true (8, 9, 11).”
BackgroundSelective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Practical significance: a concept whose time has come
Kirk, R. E. (1996) · Educational and Psychological Measurement · doi:10.1177/0013164496056005002 · Cited by 1,778
The Hamilton Depression Rating Scale: Has the Gold Standard Become a Lead Weight?
Bagby, R. M., Ryder, A. G., Schuller, D. R., & Marshall, M. B. (2004) · American Journal of Psychiatry · doi:10.1176/appi.ajp.161.12.2163 · Cited by 1,148
Antidepressant use and risk of adverse outcomes in older people: population based cohort study
Coupland, C., Dhiman, P., Morriss, R., Arthur, A., Barton, G., & Hippisley-Cox, J. (2011) · BMJ · doi:10.1136/bmj.d4551 · Cited by 833
Carvalho, A. F., Sharma, M. S., Brunoni, A. R., Vieta, E., & Fava, G. A. (2016) · Psychotherapy and Psychosomatics · doi:10.1159/000447034 · Cited by 650
Stone, M., Laughren, T., Jones, M. L., Levenson, M., Holland, P. C., Hughes, A., Hammad, T. A., Temple, R., & Rochester, G. (2009) · BMJ · doi:10.1136/bmj.b2880 · Cited by 609
Cuijpers, P., Sijbrandij, M., Koole, S. L., Andersson, G., Beekman, A. T., & Reynolds, C. F. (2013) · World Psychiatry · doi:10.1002/wps.20038 · Cited by 540
Hróbjartsson, A., Thomsen, A. S. S., Emanuelsson, F., Tendal, B., Hilden, J., Boutron, I., Ravaud, P., & Brorson, S. (2013) · Canadian Medical Association Journal · doi:10.1503/cmaj.120744 · Cited by 527
Anticholinergic drugs and risk of dementia: case-control study
Richardson, K., Fox, C., Maidment, I., Steel, N., Loke, Y. K., Arthur, A., Myint, P. K., Grossi, C. M., Mattishent, K., Bennett, K., et al. (2018) · BMJ · doi:10.1136/bmj.k1315 · Cited by 520
Leucht, S., Hierl, S., Kissling, W., Dold, M., & Davis, J. M. (2012) · British Journal of Psychiatry · doi:10.1192/bjp.bp.111.096594 · Cited by 451
Fava, G. A., Gatti, A., Belaise, C., Guidi, J., & Offidani, E. (2015) · Psychotherapy and Psychosomatics · doi:10.1159/000370338 · Cited by 436
Jakobsen, J. C., Katakam, K. K., Schou, A., Hellmuth, S. G., Stallknecht, S. E., Leth-Møller, K., Iversen, M., Banke, M. B., Petersen, I. J., Klingenberg, S. L., et al. (2017) · BMC Psychiatry · doi:10.1186/s12888-016-1173-2 · Cited by 366
Active placebos versus antidepressants for depression
Moncrieff, J., Wessely, S., & Hardy, R. (2004) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003012.pub2 · Cited by 342
When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment
Szucs, D., & Ioannidis, J. P. A. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00390 · Cited by 298
A systematic review of comparative efficacy of treatments and controls for depression
Khan, A., Faucett, J., Lichtenberg, P., Kirsch, I., & Brown, W. A. (2012) · PLoS ONE · doi:10.1371/journal.pone.0041778 · Cited by 288
Antidepressant discontinuation reactions: Are preventable and simple to treat
Haddad, P., Lejoyeux, M., & Young, A. (1998) · BMJ · doi:10.1136/bmj.316.7138.1105 · Cited by 286
Furukawa, T. A., Cipriani, A., Atkinson, L. Z., Leucht, S., Ogawa, Y., Takeshima, N., Hayasaka, Y., Chaimani, A., & Salanti, G. (2016) · The Lancet Psychiatry · doi:10.1016/s2215-0366(16)30307-8 · Cited by 219 · cited 1×
“Finally, assuming a placebo response rate of 35–40% in moderate-to-severe depression (18), based on the Furukawa formula (19), the number needed to treat (NNT) is approximately 9 [see also (20), who calculated a NNT of 8–10 based on the results reported by (1)].”
BackgroundFava, G. A., Benasi, G., Lucente, M., Offidani, E., Cosci, F., & Guidi, J. (2018) · Psychotherapy and Psychosomatics · doi:10.1159/000491524 · Cited by 209
How to obtain nnt from cohen’s D: comparison of two methods
Furukawa, T. A., & Leucht, S. (2011) · PLoS ONE · doi:10.1371/journal.pone.0019070 · Cited by 179
Review: problems associated with long-term treatment with selective serotonin reuptake inhibitors
Moret, C., Isaac, M., & Briley, M. (2009) · Journal of Psychopharmacology · doi:10.1177/0269881108093582 · Cited by 135
Ebrahim, S., Bance, S., Athale, A., Malachowski, C., & Ioannidis, J. P. A. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.08.021 · Cited by 107
Braun, C., Bschor, T., Franklin, J., & Baethge, C. (2016) · Psychotherapy and Psychosomatics · doi:10.1159/000442293 · Cited by 103
Maslej, M. M., Bolker, B. M., Russell, M. J., Eaton, K., Durisko, Z., Hollon, S. D., Swanson, G. M., Thomson Jr., J. A., Mulsant, B. H., & Andrews, P. W. (2017) · Psychotherapy and Psychosomatics · doi:10.1159/000477940 · Cited by 98
Cited by
Leichsenring, F., Steinert, C., Rabung, S., & Ioannidis, J. P. A. (2022) · World Psychiatry · doi:10.1002/wps.20941 · Cited by 485
Doss, M. K., Považan, M., Rosenberg, M. D., Sepeda, N. D., Davis, A. K., Finan, P. H., Smith, G. S., Pekar, J. J., Barker, P. B., Griffiths, R. R., et al. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01706-y · Cited by 356
Almeida, R. N. D., Galvão, A. C. D. M., da Silva, F. S., Silva, E. A. D. S., Palhano-Fontes, F., Maia-de-Oliveira, J. P., de Araújo, L.-S. B., Lobão-Soares, B., & Galvão-Coelho, N. L. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.01234 · Cited by 177
Galvão-Coelho, N. L., Marx, W., Gonzalez, M., Sinclair, J., de Manincor, M., Perkins, D., & Sarris, J. (2021) · Psychopharmacology · doi:10.1007/s00213-020-05719-1 · Cited by 137
Changes in inflammatory biomarkers are related to the antidepressant effects of Ayahuasca
Galvão-Coelho, N. L., de Menezes Galvão, A. C., de Almeida, R. N., Palhano-Fontes, F., Campos Braga, I., Lobão Soares, B., Maia-de-Oliveira, J. P., Perkins, D., Sarris, J., & de Araujo, D. B. (2020) · Journal of Psychopharmacology · doi:10.1177/0269881120936486 · Cited by 128
Metadata
- Authors
- Hengartner, M. P., & Plöderl, M.
- Venue
- Frontiers in Psychiatry
- Publisher
- Frontiers Media SA
- Published
- 2018-10-17
- Volume
- 9
- Pages
- 517-517
- ISSN
- 1664-0640 (ISSN-L)
- OpenAlex
- W2894886297
- Cited by
- 93
- References
- 27
- FWCI
- 5.82
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.