Magnitude of effects in clinical trials published in high-impact general medical journals
The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Prestigious journals select for publication studies that are considered most important and informative. We aimed to examine whether high-impact general (HIG) medical journals systematically demonstrate more favourable results for experimental interventions compared with the rest of the literature. METHODS: We scrutinized systematic reviews of the Cochrane Database (Issue 4, 2009) and meta-analyses published in four general journals (2008-09). Eligible articles included ≥1 binary outcome meta-analysis(es) pertaining to effectiveness with ≥1 clinical trial(s) published in NEJM, JAMA or Lancet. Effect sizes in trials from NEJM, JAMA or Lancet were compared with those from other trials in the same meta-analyses by deriving summary relative odds ratios (sRORs). Additional analyses examined separately early- and late-published trials in HIG journals and journal-specific effects. RESULTS: A total of 79 meta-analyses including 1043 clinical trials were analysed. Trials in HIG journals had similar effects to trials in other journals, when there was large-scale evidence, but showed more favourable results for experimental interventions when they were small. When HIG trials had less than 40 events, the sROR was 1.64 [95% confidence interval (95% CI): 1.23-2.18). The difference was most prominent when small early trials published in HIG journals were compared with subsequent trials [sROR 2.68 (95% CI: 1.33-5.38)]. Late-published HIG trials showed no consistent inflation of effects. The patterns did not differ beyond chance between NEJM, JAMA or Lancet. CONCLUSIONS: Small trials published in the most prestigious journals show more favourable effects for experimental interventions, and this is most prominent for early-published trials in such journals. No effect inflation is seen for large trials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Wood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538
Quantitative Synthesis in Systematic Reviews
Lau, J., Ioannidis, J. P. A., & Schmid, C. H. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-9-199711010-00008 · Cited by 2,517
Self-citationSelective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435
Publication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282
Investigating and dealing with publication and other biases in meta-analysis
Sterne, J. A. C., Egger, M., & Smith, G. D. (2001) · BMJ · doi:10.1136/bmj.323.7304.101 · Cited by 2,225
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546
Self-citationWhy Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495
Self-citationContradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465
Self-citationStatistics in Medicine — Reporting of Subgroup Analyses in Clinical Trials
Wang, R., Lagakos, S. W., Ware, J. H., Hunter, D. J., & Drazen, J. M. (2007) · New England Journal of Medicine · doi:10.1056/nejmsr077003 · Cited by 1,445
Undue reliance on I 2 in assessing heterogeneity may mislead
Rücker, G., Schwarzer, G., Carpenter, J. R., & Schumacher, M. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-79 · Cited by 1,287
Cited by
Szucs, D., & Ioannidis, J. P. A. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2000797 · Cited by 761
Self-citationWhy Most Clinical Research Is Not Useful
Ioannidis, J. P. A. (2016) · PLOS Medicine · doi:10.1371/journal.pmed.1002049 · Cited by 655
Self-citationDeep impact: unintended consequences of journal rank
Brembs, B., Button, K., & Munafò, M. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00291 · Cited by 441
The earth is flat ( p > 0.05): significance thresholds and the crisis of unreplicable research
Amrhein, V., Korner-Nievergelt, F., & Roth, T. (2017) · PeerJ · doi:10.7717/peerj.3544 · Cited by 308
Empirical evaluation of which trial characteristics are associated with treatment effect estimates
Dechartres, A., Trinquart, L., Faber, T., & Ravaud, P. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.04.005 · Cited by 85
Metadata
- Authors
- Siontis, K. C., Evangelou, E., & Ioannidis, J. P.
- Venue
- International Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2011-09-08
- Volume
- 40
- Issue
- 5
- Pages
- 1280-1291
- ISSN
- 0300-5771 (ISSN-L)
- 1464-3685
- OpenAlex
- W2152904388
- Cited by
- 38
- References
- 16
- FWCI
- 4.05
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Magnitude of effects in clinical trials published in high-impact general medical journals. https://app.cochat.ai/papers/10.1093/ije/dyr095
- The paper
- Siontis, K. C., Evangelou, E., & Ioannidis, J. P. (2011). Magnitude of effects in clinical trials published in high-impact general medical journals. International Journal of Epidemiology. https://doi.org/10.1093/ije/dyr095
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.