Why Most Discovered True Associations Are Inflated
The DOI matches its Crossref record; 112 of the 112 references it cites were found in a registry or an index.
Abstract
Newly discovered true (non-null) associations often have inflated effects compared with the true effect sizes. I discuss here the main reasons for this inflation. First, theoretical considerations prove that when true discovery is claimed based on crossing a threshold of statistical significance and the discovery study is underpowered, the observed effects are expected to be inflated. This has been demonstrated in various fields ranging from early stopped clinical trials to genome-wide associations. Second, flexible analyses coupled with selective reporting may inflate the published discovered effects. The vibration ratio (the ratio of the largest vs. smallest effect on the same association approached with different analytic choices) can be very large. Third, effects may be inflated at the stage of interpretation due to diverse conflicts of interest. Discovered effects are not always inflated, and under some circumstances may be deflated-for example, in the setting of late discovery of associations in sequentially accumulated overpowered evidence, in some types of misclassification from measurement error, and in conflicts causing reverse biases. Finally, I discuss potential approaches to this problem. These include being cautious about newly discovered effect sizes, considering some rational down-adjustment, using analytical methods that correct for the anticipated inflation, ignoring the magnitude of the effect (if not necessary), conducting large studies in the discovery ph…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Given that initial effect sizes are often inflated 61 and that we sought power to detect a difference between two groups, we doubled this estimate and aimed to test ~28 participants in each group with some attrition expected.”
From Adults with autism overestimate the volatility of the sensory environment (2017) · cites this paper as [61]
In the citing paper
“While this may reflect exaggeration of original effect sizes or publication bias (Ioannidis 2008), some degree of inflation is expected even in its absence, as our selection process filtered for original experiments with significant differences.”
From Estimating the replicability of Brazilian biomedical science (2025) · cites this paper as Ioannidis 2008
In the citing paper
“Further, in situations when an underpowered study detects a true effect, the estimate of this effect size is likely to be highly exaggerated, a phenomenon often referred to as the “winner’s curse” (15).”
From Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies (2016) · cites this paper as [15]
In the citing paper
“Researchers have strong incentives to interpret their results as supporting or extending the Bayesian framework, rather than challenging it—leading to a literature that systematically over-represents supportive evidence (Ioannidis 2008; Nosek et al. 2012).”
From The myth of the Bayesian brain (2025) · cites this paper as Ioannidis 2008
In the citing paper
“This flexibility in the synthesis of the evidence provides the assessors with the opportunity to inflate or disinflate the efficacy of a given technology, a phenomenon that was observed in scientific publications.[24]”
From Health technology assessments conducted in health care facilities: A strategic practice? Findings from a content analysis of HTA reports (2017) · cites this paper as [24]
8 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationGene expression profiling predicts clinical outcome of breast cancer
van 't Veer, L. J., Dai, H., van de Vijver, M. J., He, Y. D., Hart, A. A. M., Mao, M., Peterse, H. L., van der Kooy, K., Marton, M. J., Witteveen, A. T., et al. (2002) · Nature · doi:10.1038/415530a · Cited by 9,536
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040297 · Cited by 8,613
Jeffreys, H., Moran, P. A. P., & Chatfield, C. (1922) · Nature · doi:10.1093/oso/9780198503682.001.0001 · Cited by 7,878
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“This article will not deal with the debate on whether research findings are credible.(1) I will focus instead on the interesting subset of research findings that are true.”
BackgroundSelf-citationEffect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes
Nissen, S. E., & Wolski, K. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa072761 · Cited by 4,675
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., & Egger, M. (2007) · Epidemiology · doi:10.1097/ede.0b013e3181577511 · Cited by 2,779
Internal validation of predictive models
Steyerberg, E. W., Harrell, F. E., Borsboom, G. J. J. M., Eijkemans, M. J. C., Vergouwe, Y., & Habbema, J. D. F. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00341-9 · Cited by 2,746
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544
Simulation Study of Confounder-Selection Strategies
Maldonado, G., & Greenland, S. (1993) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a116813 · Cited by 2,491 · cited 1×
“One may ask the following questions: does the research community in this field adopt widely statistical significance or similar selection thresholds for claiming research findings? Did the discovery arise from a small study? Is there room for large flexibility in the analyses? Are we unprotected from selective reporti…”
MethodReplication of Genome-Wide Association Signals in UK Samples Reveals Risk Loci for Type 2 Diabetes
Zeggini, E., Weedon, M. N., Lindgren, C. M., Frayling, T. M., Elliott, K. S., Lango, H., Timpson, N. J., Perry, J. R. B., Rayner, N. W., Freathy, R. M., et al. (2007) · Science · doi:10.1126/science.1142364 · Cited by 2,194
Pharmaceutical industry sponsorship and research outcome and quality: systematic review
Lexchin, J., Bero, L. A., Djulbegovic, B., & Clark, O. (2003) · BMJ · doi:10.1136/bmj.326.7400.1167 · Cited by 2,147
Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis
Wetterslev, J., Thorlund, K., Brok, J., & Gluud, C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.013 · Cited by 1,881 · cited 1×
“Most meta-analyses remain largely underpowered for small-to-modest effects.(96) Superimposed selective reporting can also be operating.”
BackgroundChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
Replication validity of genetic association studies
Ioannidis, J. P. A., Ntzani, E. E., Trikalinos, T. A., & Contopoulos-Ioannidis, D. G. (2001) · Nature Genetics · doi:10.1038/ng749 · Cited by 1,832
Self-citationBetter Reporting of Harms in Randomized Trials: An Extension of the CONSORT Statement
Ioannidis, J. P. A., Evans, S. J. W., Gøtzsche, P. C., O'Neill, R. T., Altman, D. G., Schulz, K., Moher, D., & for the CONSORT Group* (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-141-10-200411160-00009 · Cited by 1,593
Self-citationContradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465
Self-citationUncertainty in heterogeneity estimates in meta-analyses
Ioannidis, J. P. A., Patsopoulos, N. A., & Evangelou, E. (2007) · BMJ · doi:10.1136/bmj.39343.408449.80 · Cited by 1,254 · cited 1×
“problem, an evaluation of the whole Cochrane Library shows 1011 systematic reviews that have at least one meta-analysis with at least 4 studies.(95) Selecting the largest meta-analysis in each of these reviews, 256 of the 1011 meta-analyses have”
BackgroundSelf-citationClinical Trial Registration: A Statement from the International Committee of Medical Journal Editors
De Angelis, C., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., Kotzin, S., Laine, C., Marusic, A., Overbeke, A. J. P. M., et al. (2004) · New England Journal of Medicine · doi:10.1056/nejme048225 · Cited by 1,241
Buyse, M., Loi, S., van't Veer, L., Viale, G., Delorenzi, M., Glas, A. M., Saghatchian d'Assignies, M., Bergh, J., Lidereau, R., Ellis, P., et al. (2006) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djj329 · Cited by 1,238 · cited 1×
“648).(37) Prognostic ability is Submitted 17 March 2008; accepted 27 May 2008; posted 14 July.”
BackgroundJEFFREYS, H. (1922) · Nature · doi:10.1038/109132a0 · Cited by 1,217
Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy
Goodman, S. N. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-12-199906150-00008 · Cited by 1,185
The NCBI dbGaP database of genotypes and phenotypes
Mailman, M. D., Feolo, M., Jin, Y., Kimura, M., Tryka, K., Bagoutdinov, R., Hao, L., Kiang, A., Paschall, J., Phan, L., et al. (2007) · Nature Genetics · doi:10.1038/ng1007-1181 · Cited by 1,105 · cited 1×
“(eg, P 10 (7)), report all results in a single paper, and then often make the data publicly available.(87,88) Conversely, in”
BackgroundModel uncertainty, data mining and statistical inference
Chatfield, C. (1995) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2983440 · Cited by 1,103
Meta-analysis Spurious precision? Meta-analysis of observational studies
Egger, M., Schneider, M., & Smith, G. D. (1998) · BMJ · doi:10.1136/bmj.316.7125.140 · Cited by 997
Toward Evidence-Based Medical Statistics. 2: The Bayes Factor
Goodman, S. N. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-12-199906150-00019 · Cited by 912
Steyerberg, E. W., Eijkemans, M. J. C., Harrell, F. E., & Habbema, J. D. F. (2000) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(20000430)19:8<1059::aid-sim412>3.0.co;2-0 · Cited by 785
An exploratory test for an excess of significant findings
Ioannidis, J. P., & Trikalinos, T. A. (2007) · Clinical Trials · doi:10.1177/1740774507079441 · Cited by 751 · cited 1×
“Recently, a test was proposed to examine whether the number of reported study results that pass certain levels of statistical significance is reasonable or larger than what one would expect, even if the effect sizes for the proposed associations (eg, as suggested by meta-analyses of all relevant studies) were true.(47…”
BackgroundSelf-citationMaxwell, S. E. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.2.147 · Cited by 654
Chan, A.-W., & Altman, D. G. (2005) · BMJ · doi:10.1136/bmj.38356.424606.8f · Cited by 613
Statistical aspects of prognostic factor studies in oncology
Simon, R., & Altman, D. (1994) · British Journal of Cancer · doi:10.1038/bjc.1994.192 · Cited by 582
Genetic associations in large versus small studies: an empirical assessment
Ioannidis, J. P., Trikalinos, T. A., Ntzani, E. E., & Contopoulos-Ioannidis, D. G. (2003) · The Lancet · doi:10.1016/s0140-6736(03)12516-0 · Cited by 579
Self-citationSmith, G. D., Lawlor, D. A., Harbord, R., Timpson, N., Day, I., & Ebrahim, S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040352 · Cited by 569
Melchinger, A. E., Utz, H. F., & Schön, C. C. (1998) · Genetics · doi:10.1093/genetics/149.1.383 · Cited by 531
Stepwise Selection in Small Data Sets A Simulation Study of Bias in Logistic Regression Analysis
Steyerberg, E. (1999) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00103-1 · Cited by 495
QTL analysis in plants; where are we now?
Kearsey, M. J., & Farquhar, A. G. L. (1998) · Heredity · doi:10.1046/j.1365-2540.1998.00500.x · Cited by 492
Large Upward Bias in Estimation of Locus-Specific Effects from Genomewide Scans
Göring, H. H. H., Terwilliger, J. D., & Blangero, J. (2001) · The American Journal of Human Genetics · doi:10.1086/324471 · Cited by 483
Epidemiology and reporting of randomised trials published in PubMed journals
Chan, A.-W., & Altman, D. G. (2005) · The Lancet · doi:10.1016/s0140-6736(05)71879-1 · Cited by 474
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2019) · CHANCE · doi:10.1080/09332480.2019.1579573 · Cited by 463
Self-citationOutcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research
Chan, A.-W. (2004) · Canadian Medical Association Journal · doi:10.1503/cmaj.1041086 · Cited by 444
Observational research, randomised trials, and two views of medical science
Vandenbroucke, J. P. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050067 · Cited by 416 · cited 2×
“Discovery may sometimes arise from small investigations or even unanticipated case observations.(70) However, even if many discoveries in the past arose out of haphazard encounters of scientists with phenomena, this does not mean that we cannot improve in the future by running larger discovery-oriented studies.”
BackgroundClinical trial registration: a statement from the International Committee of Medical Journal Editors
De Angelis, C., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., Kotzin, S., Laine, C., Marusic, A., Overbeke, A. J. P., et al. (2004) · The Lancet · doi:10.1016/s0140-6736(04)17034-7 · Cited by 408
Meta-analysis of randomised controlled trials
Kelly, C. P. (1997) · The Lancet · doi:10.1016/s0140-6736(05)63834-2 · Cited by 382
Overcoming the Winner’s Curse: Estimating Penetrance Parameters from Case-Control Data
Zöllner, S., & Pritchard, J. K. (2007) · The American Journal of Human Genetics · doi:10.1086/512821 · Cited by 382 · cited 1×
“Although the replication record is better with genome-wide association studies, theoretical considerations again show the early discovered effects are inflated.(23,24) Furthermore, if the observed effects are used as estimates in designing replication studies, these subsequent studies will be underpowered, and genuine…”
BackgroundUtz, H. F., Melchinger, A. E., & Schön, C. C. (2000) · Genetics · doi:10.1093/genetics/154.4.1839 · Cited by 362
Molecular Dissection of Complex Traits
Paterson, A. H. (2019) · CRC Press eBooks · doi:10.1201/9781420049381 · Cited by 336
Translation of highly promising basic science research into clinical applications
Contopoulos-Ioannidis, D. G., Ntzani, E. E., & Ioannidis, J. P. A. (2003) · The American Journal of Medicine · doi:10.1016/s0002-9343(03)00013-5 · Cited by 323 · cited 1×
“The sparse successful clinical translation of major promises made in the most high-profile basic science journals shows that this overinterpretation is common.(54)”
BackgroundSelf-citationThe GAIN Collaborative Research Group (2007) · Nature Genetics · doi:10.1038/ng2127 · Cited by 318 · cited 1×
“(eg, P 10 (7)), report all results in a single paper, and then often make the data publicly available.(87,88) Conversely, in”
BackgroundJennions, M. D., & Møller, A. P. (2002) · Proceedings of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rspb.2001.1832 · Cited by 255
Ioannidis, J. P. A., Trikalinos, T. A., & Khoury, M. J. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj259 · Cited by 246
Self-citationNon-Replication and Inconsistency in the Genome-Wide Association Setting
Ioannidis, J. P. A. (2007) · Human Heredity · doi:10.1159/000103512 · Cited by 245 · cited 1×
“Furthermore, if the observed effects are used as estimates in designing replication studies, these subsequent studies will be underpowered, and genuine effects will be falsely nonreplicated.(38)”
BackgroundSelf-citationComparison of evidence on harms of medical interventions in randomized and nonrandomized studies
Papanikolaou, P. N. (2006) · Canadian Medical Association Journal · doi:10.1503/cmaj.050873 · Cited by 229 · cited 1×
“However, the literature on adverse events of interventions is small compared with the literature on effectiveness.(74) Most harms probably remain unknown rather than silenced.”
ResultPersistence of Contradicted Claims in the Literature
Tatsioni, A., Bonitsis, N. G., & Ioannidis, J. P. A. (2007) · JAMA · doi:10.1001/jama.298.21.2517 · Cited by 225 · cited 1×
“For example, even the most strongly refuted associations continue to have supporters many years after the refutation.(76) Investigators may suppress new findings when they do not suit their beliefs.”
BackgroundSelf-citationTHE EFFECTS OF MEASUREMENT ERRORS ON RELATWE RISK REGRESSIONS
ARMSTRONG, B. G. (1990) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a115761 · Cited by 206
THE PROBLEM OF MULTIPLE INFERENCE IN STUDIES DESIGNED TO GENERATE HYPOTHESES
THOMAS, D. C., SIEMIATYCKI, J., DEWAR, R., ROBINS, J., GOLDBERG, M., & ARMSTRONG, B. G. (1985) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a114189 · Cited by 190
For Debate: The statistical basis of public policy: a paradigm shift is overdue
Lilford, R. J., & Braunholtz, D. (1996) · BMJ · doi:10.1136/bmj.313.7057.603 · Cited by 188
Limitations are not properly acknowledged in the scientific literature
Ioannidis, J. P. A. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.09.011 · Cited by 180
Self-citationStatistical false positive or true disease pathway?
Todd, J. A. (2006) · Nature Genetics · doi:10.1038/ng0706-731 · Cited by 180 · cited 1×
“Agnostic genome-wide associations provide such an example.(78) Using a strict protocol for the design, conduct, and analysis of a study can diminish vibration, but would this stifle creativity? Flexible analyses will not cause a problem if”
BackgroundEffect sizes in cumulative meta-analyses of mental health randomized trials evolved over time
Trikalinos, T. A., Churchill, R., Ferri, M., Leucht, S., Tuunainen, A., Wahlbeck, K., & Ioannidis, J. P. A. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.02.018 · Cited by 177
Self-citationThe Bayesian Outlook and Its Application
Cornfield, J. (1969) · Biometrics · doi:10.2307/2528565 · Cited by 173
Local Literature Bias in Genetic Epidemiology: An Empirical Evaluation of the Chinese Literature
Pan, Z., Trikalinos, T. A., Kavvoura, F. K., Lau, J., & Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020334 · Cited by 168
Self-citationPractical problems in interim analyses, with particular regard to estimation
Pocock, S. J., & Hughes, M. D. (1989) · Controlled Clinical Trials · doi:10.1016/0197-2456(89)90059-7 · Cited by 160 · cited 2×
“This is performed according to robust rules that suggest termination when a demanding threshold of statistical significance is crossed during an interim analysis.(3,4) These interventions are indeed effective (the null of “no effectiveness” is correctly rejected).”
BackgroundMethodLeimu, R., & Koricheva, J. (2004) · Proceedings of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rspb.2004.2828 · Cited by 156
Bias from Nondifferential but Dependent Misclassification of Exposure and Outcome
Kristensen, P. (1992) · Epidemiology · doi:10.1097/00001648-199205000-00005 · Cited by 156 · cited 1×
“Perhaps this is because usually nonindependent and differential misclassification has been difficult to exclude, and these can either deflate or inflate observed effects.(62,63) Measurement error has decreased over time for many fields of research in the current era.”
BackgroundBagshaw, S. M., McAlister, F. A., Manns, B. J., & Ghali, W. A. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.2.161 · Cited by 134
Stopping rules and estimation problems in clinical trials
Hughes, M. D., & Pocock, S. J. (1988) · Statistics in Medicine · doi:10.1002/sim.4780071204 · Cited by 127 · cited 2×
“This is performed according to robust rules that suggest termination when a demanding threshold of statistical significance is crossed during an interim analysis.(3,4) These interventions are indeed effective (the null of “no effectiveness” is correctly rejected).”
BackgroundMethodIs Molecular Profiling Ready for Use in Clinical Decision Making?
Ioannidis, J. P. A. (2007) · The Oncologist · doi:10.1634/theoncologist.12-3-301 · Cited by 126
Self-citationDiscussion Sections in Reports of Controlled Trials Published in General Medical Journals
Clarke, M. (2002) · JAMA · doi:10.1001/jama.287.21.2799 · Cited by 125
DNA data sharing: research participants' perspectives
McGuire, A. L., Hamilton, J. A., Lunstroth, R., McCullough, L. B., & Goldman, A. M. (2008) · Genetics in Medicine · doi:10.1097/gim.0b013e31815f1e00 · Cited by 125 · cited 1×
“easily said than done, and there are many challenges in making this a widespread practice.(85,86) Still, the antithesis of”
BackgroundKhoury, M. J., Little, J., Gwinn, M., & Ioannidis, J. P. (2007) · International Journal of Epidemiology · doi:10.1093/ije/dyl253 · Cited by 119
Self-citationSelection in Reported Epidemiological Risks: An Empirical Assessment
Kavvoura, F. K., Liberopoulos, G., & Ioannidis, J. P. A. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040079 · Cited by 115 · cited 1×
“Empirical evidence has demonstrated across a large sample of 379 epidemiologic studies that investigators selected the contrasts for continuous variables so as to show effects as being larger: more extreme contrasts were presented, when effects were inherently smaller.(46) Post hoc demonstration of selective analysis…”
BackgroundSelf-citationUpward bias in odds ratio estimates from genome‐wide association studies
Garner, C. (2007) · Genetic Epidemiology · doi:10.1002/gepi.20209 · Cited by 103 · cited 1×
“Although the replication record is better with genome-wide association studies, theoretical considerations again show the early discovered effects are inflated.(23,24) Furthermore, if the observed effects are used as estimates in designing replication studies, these subsequent studies will be underpowered, and genuine…”
BackgroundGarcía‐Closas, M., Rothman, N., & Lubin, J. H. (1999) · PubMed · Cited by 101
Clinical trial registration: looking back and moving ahead
Laine, C. M., Horton, R. C., DeAngelis, C. D., Drazen, J. M., Frizelle, F., Godlee, F., Haug, C. J., Hébert, P. C., Kotzin, S., Marušić, A., et al. (2007) · The Lancet · doi:10.1016/s0140-6736(07)60894-0 · Cited by 100 · cited 1×
“ized trials, which are further ahead in registration and reporting efforts.(83,84) Making databases publicly available is more”
BackgroundThe Emergence of Networks in Human Genome Epidemiology: Challenges and Opportunities
Seminara, D., Khoury, M. J., O??Brien, T. R., Manolio, T., Gwinn, M. L., Little, J., Higgins, J. P. T., Bernstein, J. L., Boffetta, P., Bondy, M., et al. (2007) · Epidemiology · doi:10.1097/01.ede.0000249540.17855.b7 · Cited by 92 · cited 1×
“Inclusive consortia of investigators may also help enhance transparency and completeness of reporting of results.(90)”
BackgroundSelf-citationStopping at Nothing? Some Dilemmas of Data Monitoring in Clinical Trials
Goodman, S. N. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-146-12-200706190-00010 · Cited by 90 · cited 1×
“For example, should a meta-analysis worry about including an early terminated study? In principle, the replication process, if unbiased, should correct the inflation(91) and if stopping is not very early, inflation is small regardless.(93) However, the replication process may not be unbiased, and may sometimes suffer…”
BackgroundThe fading of reported effectiveness. A meta-analysis of randomised controlled trials
Gehr, B. T., Weiss, C., & Porzsolt, F. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-25 · Cited by 88
Low power, type II errors, and other statistical problems in recent cardiovascular research
Williams, J. L., Hathaway, C. A., Kloster, K. L., & Layne, B. H. (1997) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.1997.273.1.h487 · Cited by 85
Garcia-Closas, M., Thompson, W. D., & Robins, J. M. (1998) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009467 · Cited by 84
Chokshi, D. A. (2006) · Bulletin of the World Health Organization · doi:10.2471/blt.06.029843 · Cited by 76 · cited 1×
“easily said than done, and there are many challenges in making this a widespread practice.(85,86) Still, the antithesis of”
BackgroundEstimation of magnitude in gene–environment interactions in the presence of measurement error
Wong, M. Y., Day, N. E., Luan, J. A., & Wareham, N. J. (2004) · Statistics in Medicine · doi:10.1002/sim.1662 · Cited by 71
Systematic reviewers neglect bias that results from trials stopped early for benefit
Bassler, D., González, I. F., Briel, M., Cook, D. J., Devereaux, P. J., Heels-Ansdell, D., Kirpalani, H. M., Meade, M. O., Montori, V. M., Rozenberg, A., et al. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.12.006 · Cited by 69 · cited 1×
“A crucial question is whether replication suffices to correct the inflated effects that arise in early studies.(91,92) For example, should a meta-analysis worry about including an early terminated study? In principle, the replication process, if unbiased, should correct the inflation(91) and if stopping is not very ea…”
BackgroundAdverse events: the more you search, the more you find
Ioannidis, J. P. A., Mulrow, C. D., & Goodman, S. N. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-4-200602210-00013 · Cited by 69 · cited 1×
“Most harms probably remain unknown rather than silenced.(75) Finally, conflicts may be of nonfinancial nature.”
BackgroundSelf-citationWeaver, C. S., Leonardi-Bee, J., Bath-Hextall, F. J., & Bath, P. M. W. (2004) · Stroke · doi:10.1161/01.str.0000125010.70652.93 · Cited by 67
Allison, D. B., Fernandez, J. R., Heo, M., Zhu, S., Etzel, C. J., Beasley, T. M., & Amos, C. I. (2002) · The American Journal of Human Genetics · doi:10.1086/339273 · Cited by 64
Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., & de Vet, H. C. W. (2003) · American Journal of Roentgenology · doi:10.2214/ajr.181.1.1810051 · Cited by 60
Review of On Truth and Practice.
Sorensen, A. D. (1905) · Psychological Bulletin · doi:10.1037/h0067331 · Cited by 53
Ntzani, E. E., Rizos, E. C., & Ioannidis, J. P. A. (2007) · American Journal of Epidemiology · doi:10.1093/aje/kwk085 · Cited by 52
Self-citationAre standardized mortality ratios valid for public health data analysis?
Goldman, D. A., & Brender, J. D. (2000) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(20000430)19:8<1081::aid-sim406>3.0.co;2-a · Cited by 49
Kavvoura, F. K., McQueen, M. B., Khoury, M. J., Tanzi, R. E., Bertram, L., & Ioannidis, J. P. A. (2008) · American Journal of Epidemiology · doi:10.1093/aje/kwn206 · Cited by 48
Self-citationStatistical power of articles published in three health-psychology related journals.
Maddock, J. E., & Rossi, J. S. (2001) · Health Psychology · doi:10.1037//0278-6133.20.1.76 · Cited by 46
Reporting bayesian analyses of clinical trials
Hughes, M. D. (1993) · Statistics in Medicine · doi:10.1002/sim.4780121802 · Cited by 45
Combining estimates of the odds ratio: the state of the art
Emerson, J. D. (1994) · Statistical Methods in Medical Research · doi:10.1177/096228029400300204 · Cited by 42
Strube, M. (2006) · Behavior Research Methods · doi:10.3758/bf03192746 · Cited by 41
Hernández, A. V., Walker, E., Ioannidis, J. P. A., & Kattan, M. W. (2008) · American Heart Journal · doi:10.1016/j.ahj.2008.03.002 · Cited by 40
Self-citationZhang, L., Mukherjee, B., Ghosh, M., Gruber, S. B., & Moreno, V. R. (2007) · Statistics in Medicine · doi:10.1002/sim.3044 · Cited by 40
The prevalence of underpowered randomized clinical trials in rheumatology.
Keen, H. I., Pile, K. D., & Hill, C. L. (2005) · PubMed · Cited by 38
Common Genetic Variants for Breast Cancer: 32 Largely Refuted Candidates and Larger Prospects
Ioannidis, J. P. A. (2006) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djj392 · Cited by 36
Self-citationKrum, H., & Tonkin, A. (2003) · Journal of Cardiac Failure · doi:10.1054/s1071-9164(03)00018-6 · Cited by 30
Upward Bias in Estimation of Genetic Effects
Siegmund, D. (2002) · The American Journal of Human Genetics · doi:10.1086/343819 · Cited by 27
Randomized Trials Stopped Early for Benefit: A Systematic Review
Montori, V. M., Devereaux, P. J., Adhikari, N. K. J., Burns, K. E. A., Eggert, C. H., Briel, M., Lacchetti, C., Leung, T. W., Darling, E., Bryant, D. M., et al. (2005) · JAMA · doi:10.1001/jama.294.17.2203 · Cited by 26
Methodology to Correct for Differential Misclassification
Flanders, W. D., Drews‐Botsch, C., & Kosinski, A. S. (1995) · Epidemiology · doi:10.1097/00001648-199503000-00011 · Cited by 23 · cited 1×
“Perhaps this is because usually nonindependent and differential misclassification has been difficult to exclude, and these can either deflate or inflate observed effects.(62,63) Measurement error has decreased over time for many fields of research in the current era.”
BackgroundData mining in pharmacovigilance: lessons from phantom ships
Hauben, M., Reich, L., van Puijenbroek, E., Gerrits, C. M., & Patadia, V. K. (2006) · European Journal of Clinical Pharmacology · doi:10.1007/s00228-006-0181-4 · Cited by 23
Systematic reviews are not biased by results from trials stopped early for benefit
Goodman, S. N. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.06.012 · Cited by 22
Selective discussion and transparency in microarray research findings for cancer outcomes
Ioannidis, J. P. A., Polyzos, N. P., & Trikalinos, T. A. (2007) · European Journal of Cancer · doi:10.1016/j.ejca.2007.05.019 · Cited by 20
Self-citationTung, L., Gordon, D., & Finch, S. J. (2007) · Human Heredity · doi:10.1159/000099182 · Cited by 11
Johnson, V. E., & Yuan, Y. (2007) · Clinical Trials · doi:10.1177/1740774507079437 · Cited by 11
Long-term Risk of Cardiovascular Events With Rosiglitazone: A Meta-analysis
Singh, S., Loke, Y. K., & Furberg, C. D. (2007) · JAMA · doi:10.1001/jama.298.10.1189 · Cited by 5
Harrell, F., Jr. (1996) · doi:10.1002/(sici)1097-0258(19960229)15:4<361::aid-sim168>3.0.co;2-4
Cited by
Sterne, J. A. C., Sutton, A. J., Ioannidis, J. P. A., Terrin, N., Jones, D. R., Lau, J., Carpenter, J., Rucker, G., Harbord, R. M., Schmid, C. H., et al. (2011) · BMJ · doi:10.1136/bmj.d4002 · Cited by 7,709
“and trials of lower quality also tend to show larger intervention effects.”
BackgroundSelf-citationPower failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
Self-citationThe ASA Statement on p -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306
Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181c30fb2 · Cited by 5,170
“This poses a high risk of overoptimistic expectations of the marker’s performance.(21,22) Moreover, we are interested in only the incremental value of a marker, on top of predictors that are readily accessible.”
BackgroundMoons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citation
Metadata
- Authors
- Ioannidis, J. P. A.
- Affiliations
- University of Ioannina (GR)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- Epidemiology
- Published
- 2008-09
- Volume
- 19
- Issue
- 5
- Pages
- 640-648
- ISSN
- 1044-3983 (ISSN-L)
- OpenAlex
- W2150215887
- Semantic Scholar
- 0373f18622b7b64e99c7e56340ab2011dbd00b5d
- Cited by
- 1,495
- References
- 109
- Citations by year
- 2026: 91 · 2025: 87 · 2024: 78 · 2023: 112 · 2022: 94 · 2021: 99 · 2020: 103 · 2019: 131 · 2018: 108 · 2017: 101 · 2016: 87 · 2015: 77 · 2014: 72 · 2013: 60 · 2012: 50
- FWCI
- 15.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- effect size inflation, statistical significance thresholds, underpowered studies, selective reporting, conflicts of interest, reverse bias, measurement error, replication
- MeSH terms
- Clinical Trials as Topic, Data Interpretation, Statistical, Humans, Sensitivity and Specificity, Models, Statistical, Linkage Disequilibrium, Molecular Epidemiology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.