The DOI matches its Crossref record; 89 of the 89 references it cites were found in a registry or an index.
Risk factors and interventions with statistically significant tiny effects
Abstract
BACKGROUND: Large studies may identify postulated risk factors and interventions with very small effect sizes. We aimed to assess empirically a large number of statistically significant relative risks (RRs) of tiny magnitude and their interpretation by investigators. METHODS: RRs in the range between 0.95 and 1.05 were identified in abstracts of articles of cohort studies; articles published in NEJM, JAMA or Lancet; and Cochrane reviews. For each eligible tiny effect and the respective study, we recorded information on study design, participants, risk factor/intervention, outcome, effect estimates, P-values and interpretation by study investigators. We also calculated the probability that each effect lies outside specific intervals around the null (RR interval 0.97-1.03, 0.95-1.05, 0.90-1.10). RESULTS: We evaluated 51 eligible tiny effects (median sample size 112 786 for risk factors and 36 021 for interventions). Most (37/51) appeared in articles published in 2006-10. The effects pertained to nutrition (n = 19), genetic and other biomarkers (n = 8), correlates of health care (n = 8) and diverse other topics (n = 16) of clinical or public health importance and mostly referred to major clinical outcomes. A total of 15 of the 51 effects were >80% likely to lie outside the RR interval 0.97-1.03, but only 8 were >40% likely to lie outside the RR interval 0.95-1.05 and none was >1.7% likely to lie outside the RR interval 0.90-1.10. The authors discussed at least one concern for 23 effects (small magnitude n = 19, residual confounding n = 11, selection bias n = 1). No concerns were expressed for 28 effects. CONCLUSIONS: Statistically significant tiny effects for risk factors and interventions of clinical or public health importance become more common in the literature. Cautious interpretation is warranted, since most of these effects could be eliminated with even minimal biases and their importance is uncertain.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A systematic methodologic review of 51 RCTs assessing complex interventions with statistically significant small effects showed that authors exercised no caution in their interpretation of results in about half of the reports (33).”
From Misrepresentation and distortion of research in biomedical literature (2018) · cites this paper as [33]
Citations
Cites
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationTrends in Prevalence and Outcome of Heart Failure with Preserved Ejection Fraction
Owan, T. E., Hodge, D. O., Herges, R. M., Jacobsen, S. J., Roger, V. L., & Redfield, M. M. (2006) · New England Journal of Medicine · doi:10.1056/nejmoa052256 · Cited by 4,191
Genome-wide association studies for complex traits: consensus, uncertainty and challenges
McCarthy, M. I., Abecasis, G. R., Cardon, L. R., Goldstein, D. B., Little, J., Ioannidis, J. P. A., & Hirschhorn, J. N. (2008) · Nature Reviews Genetics · doi:10.1038/nrg2344 · Cited by 2,891
Self-citationEducational Levels of Hospital Nurses and Surgical Patient Mortality
Aiken, L. H. (2003) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.290.12.1617 · Cited by 1,724
Long-term ozone exposure and mortality
Jerrett, M., Burnett, R. T., Pope, C. A., Ito, K., Thurston, G., Krewski, D., Shi, Y., Calle, E., & Thun, M. (2009) · New England Journal of Medicine · doi:10.1056/nejmoa0803894 · Cited by 1,602
Why Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495
Self-citationThe Effect of Fruit and Vegetable Intake on Risk for Coronary Heart Disease
Joshipura, K. J., Hu, F. B., Manson, J. E., Stampfer, M. J., Rimm, E. B., Speizer, F. E., Colditz, G., Ascherio, A., Rosner, B., Spiegelman, D., et al. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-12-200106190-00010 · Cited by 1,396
Discrepancies between Meta-Analyses and Subsequent Large Randomized, Controlled Trials
LeLorier, J., Grégoire, G., Benhaddad, A., Lapierre, J., & Derderian, F. (1997) · New England Journal of Medicine · doi:10.1056/nejm199708213370806 · Cited by 1,171
Fruit and Vegetable Intake in Relation to Risk of Ischemic Stroke
Joshipura, K. J. (1999) · JAMA · doi:10.1001/jama.282.13.1233 · Cited by 1,078
Research works printed but not linked: 2
- 2006, 186, 200
- Skrabanek, 1994, Epidemiology (Cambridge, Mass.), 5, 553
Cited by
Kivimäki, M., Batty, G. D., Pentti, J., Shipley, M. J., Sipilä, P. N., Nyberg, S. T., Suominen, S. B., Oksanen, T., Stenholm, S., Virtanen, M., et al. (2020) · The Lancet Public Health · doi:10.1016/s2468-2667(19)30248-8 · Cited by 734
Why clinical trial outcomes fail to translate into benefits for patients
Heneghan, C., Goldacre, B., & Mahtani, K. R. (2017) · Trials · doi:10.1186/s13063-017-1870-2 · Cited by 404
Misrepresentation and distortion of research in biomedical literature
Boutron, I., & Ravaud, P. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1710755115 · Cited by 273
When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment
Szucs, D., & Ioannidis, J. P. A. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00390 · Cited by 298
Self-citationImplausible results in human nutrition research
Ioannidis, J. P. A. (2013) · BMJ · doi:10.1136/bmj.f6698 · Cited by 270
Self-citation
Metadata
- Authors
- Siontis, G. C., & Ioannidis, J. P.
- Affiliations
- University of Ioannina (GR)
- Corresponding author
- Ioannidis, J. P.
- Venue
- International Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2011-07-06
- Volume
- 40
- Issue
- 5
- Pages
- 1292-1307
- ISSN
- 0300-5771 (ISSN-L)
- 1464-3685
- OpenAlex
- W2148152872
- Cited by
- 80
- References
- 89
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 4 · 2023: 4 · 2022: 4 · 2021: 4 · 2020: 3 · 2019: 2 · 2018: 12 · 2017: 8 · 2016: 6 · 2015: 6 · 2014: 5 · 2013: 10 · 2012: 5
- FWCI
- 7.15
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- relative risk, risk factors, intervention, residual confounding, selection bias, nutrition, genetic biomarkers, large cohort studies
- MeSH terms
- Data Interpretation, Statistical, Humans, Risk, Risk Factors, Meta-Analysis as Topic, Bias, Confidence Intervals, Treatment Outcome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Risk factors and interventions with statistically significant tiny effects. https://app.cochat.ai/papers/10.1093/ije/dyr099
- The paper
- Siontis, G. C., & Ioannidis, J. P. (2011). Risk factors and interventions with statistically significant tiny effects. International Journal of Epidemiology. https://doi.org/10.1093/ije/dyr099
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.