Why Most Discovered True Associations Are Inflated
The DOI matches its Crossref record; 111 of the 111 references it cites were found in a registry or an index.
Abstract
Newly discovered true (non-null) associations often have inflated effects compared with the true effect sizes. I discuss here the main reasons for this inflation. First, theoretical considerations prove that when true discovery is claimed based on crossing a threshold of statistical significance and the discovery study is underpowered, the observed effects are expected to be inflated. This has been demonstrated in various fields ranging from early stopped clinical trials to genome-wide associations. Second, flexible analyses coupled with selective reporting may inflate the published discovered effects. The vibration ratio (the ratio of the largest vs. smallest effect on the same association approached with different analytic choices) can be very large. Third, effects may be inflated at the stage of interpretation due to diverse conflicts of interest. Discovered effects are not always inflated, and under some circumstances may be deflated-for example, in the setting of late discovery of associations in sequentially accumulated overpowered evidence, in some types of misclassification from measurement error, and in conflicts causing reverse biases. Finally, I discuss potential approaches to this problem. These include being cautious about newly discovered effect sizes, considering some rational down-adjustment, using analytical methods that correct for the anticipated inflation, ignoring the magnitude of the effect (if not necessary), conducting large studies in the discovery ph…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Given that initial effect sizes are often inflated 61 and that we sought power to detect a difference between two groups, we doubled this estimate and aimed to test ~28 participants in each group with some attrition expected.”
From Adults with autism overestimate the volatility of the sensory environment (2017) · cites this paper as [61]
In the citing paper
“While this may reflect exaggeration of original effect sizes or publication bias (Ioannidis 2008), some degree of inflation is expected even in its absence, as our selection process filtered for original experiments with significant differences.”
From Estimating the replicability of Brazilian biomedical science (2025) · cites this paper as Ioannidis 2008
In the citing paper
“Further, in situations when an underpowered study detects a true effect, the estimate of this effect size is likely to be highly exaggerated, a phenomenon often referred to as the “winner’s curse” (15).”
From Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies (2016) · cites this paper as [15]
In the citing paper
“Researchers have strong incentives to interpret their results as supporting or extending the Bayesian framework, rather than challenging it—leading to a literature that systematically over-represents supportive evidence (Ioannidis 2008; Nosek et al. 2012).”
From The myth of the Bayesian brain (2025) · cites this paper as Ioannidis 2008
In the citing paper
“This flexibility in the synthesis of the evidence provides the assessors with the opportunity to inflate or disinflate the efficacy of a given technology, a phenomenon that was observed in scientific publications.[24]”
From Health technology assessments conducted in health care facilities: A strategic practice? Findings from a content analysis of HTA reports (2017) · cites this paper as [24]
8 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationGene expression profiling predicts clinical outcome of breast cancer
van 't Veer, L. J., Dai, H., van de Vijver, M. J., He, Y. D., Hart, A. A. M., Mao, M., Peterse, H. L., van der Kooy, K., Marton, M. J., Witteveen, A. T., et al. (2002) · Nature · doi:10.1038/415530a · Cited by 9,536
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040297 · Cited by 8,613
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“This article will not deal with the debate on whether research findings are credible.(1) I will focus instead on the interesting subset of research findings that are true.”
BackgroundSelf-citationEffect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes
Nissen, S. E., & Wolski, K. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa072761 · Cited by 4,675
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., & Egger, M. (2007) · Epidemiology · doi:10.1097/ede.0b013e3181577511 · Cited by 2,779
Internal validation of predictive models
Steyerberg, E. W., Harrell, F. E., Borsboom, G. J. J. M., Eijkemans, M. J. C., Vergouwe, Y., & Habbema, J. D. F. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00341-9 · Cited by 2,746
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544
Cited by
Items 401–500 of 1,004
Panagiotou, O. A., & Ioannidis, J. P. A. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.01.008 · Cited by 53
Self-citationSpeech‐in‐noise perception in musicians and non‐musicians: A multi‐level meta-analysis
Hennessy, S., Mack, W. J., & Habibi, A. (2022) · Hearing Research · doi:10.1016/j.heares.2022.108442 · Cited by 54
Lozada-Martínez, I. D., Hernández-Páez, D. A., Moreno, O. F., Picón‐Jaimes, Y. A., & Bermúdez, V. (2025) · Publications · doi:10.3390/publications13010007 · Cited by 56
“But, even if a research discovery is true, the observed effect size would probably be inflated due to underpowered studies and statistical significance thresholds, flexible analysis and selective reporting, or interpretation bias (Ioannidis, 2008).”
Genetic variability in ABCB1, occupational pesticide exposure, and Parkinson’s disease
Narayan, S., Sinsheimer, J. S., Paul, K. C., Liew, Z., Cockburn, M., Bronstein, J. M., & Ritz, B. R. (2015) · Environmental Research · doi:10.1016/j.envres.2015.08.022 · Cited by 55
“Possible explanations for differences between the results in our study and the French study (Dutheil et al., 2010) include a ‘winner’s curse’ phenomenon (Ioannidis, 2008), differences in additional genes and population stratification, differences in co-exposures and in pesticide exposure assessment.”
ResultDillon, S. K., McEvoy, R., Baldwin, D. S., Rees, G. N., Parsons, Y. M., & Southerton, S. G. (2014) · PLoS ONE · doi:10.1371/journal.pone.0103515 · Cited by 55
“Inflated effect sizes for some loci could reflect increased variance in environmental values due to the underpowered nature of the study owing to the small number of populations and individuals sampled [67].”
BackgroundMcGinnis, D. P., Brownstein, J. S., & Patel, C. J. (2016) · Scientific Reports · doi:10.1038/srep30373 · Cited by 53
Investigating the work–family conflict and health link: Repetitive thought as a mechanism
Davis, K. D., Gere, J., & Sliwinski, M. J. (2017) · Stress and Health · doi:10.1002/smi.2711 · Cited by 45
“First, the sample size is relatively small and, therefore, the possibility exists that the statistically significant effects may be spurious or their size overestimated (e.g., Ioannidis, 2008).”
BackgroundKatsoulis, M., Dimakopoulou, K., Pedeli, X., Trichopoulos, D., Gryparis, A., Trichopoulou, A., & Katsouyanni, K. (2014) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2014.05.058 · Cited by 59
MASCC guideline: cannabis for cancer-related pain and risk of harms and adverse events
To, J., Davis, M., Sbrana, A., Alderman, B., Hui, D., Mukhopadhyay, S., Bouleuc, C., Case, A., Amano, K., Crawford, G., et al. (2023) · Supportive Care in Cancer · doi:10.1007/s00520-023-07662-1 · Cited by 38
Kumar, J. V., Moss, M. E., Liu, H., & Fisher‐Owens, S. A. (2023) · Public Health · doi:10.1016/j.puhe.2023.03.011 · Cited by 53
Sio, U. N., & Lortie‐Forgues, H. (2024) · Psychological Bulletin · doi:10.1037/bul0000432 · Cited by 62
Spalek, K., Fastenrath, M., Ackermann, S., Auschra, B., Coynel, D., Frey, J. N., Gschwind, L., Hartmann, F., van der Maarel, N., Papassotiropoulos, A., et al. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.2384-14.2015 · Cited by 65
“Small sample size has been identified as an issue undermining the reliability of findings in neuroscience (Ioannidis, 2008; Button et al., 2013).”
BackgroundBrowne, M. L., Rasmussen, S. A., Hoyt, A. T., Waller, D. K., Druschel, C. M., Caton, A. R., Canfield, M. A., Lin, A. E., Carmichael, S. L., & Romitti, P. A. (2009) · Birth Defects Research Part A Clinical and Molecular Teratology · doi:10.1002/bdra.20573 · Cited by 55
“It is possible that some of the significant associations represent false-positive findings or that the effect sizes are inflated (Ioannidis, 2008).”
BackgroundPolanin, J. R., & Terzian, M. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.10.006 · Cited by 46
Chung, M. K., Rappaport, S. M., Wheelock, C. E., Nguyen, V. K., van der Meer, T. P., Miller, G. W., Vermeulen, R. C. H., & Patel, C. J. (2021) · Environmental Health Perspectives · doi:10.1289/ehp8327 · Cited by 51
Ko, W. K. D., & Bézard, E. (2017) · Experimental Neurology · doi:10.1016/j.expneurol.2017.07.020 · Cited by 50
“…(Hackam, 2007; Martic-Kehl et al., 2012; Van Der Worp et al., 2010), which has led to exaggerated findings (Button et al., 2013; Hackam, 2007; Ioannidis, 2008; Van Der Worp and Macleod, 2011), the use of animal models in PD can benefit by strictly adhering to randomisation,…”
BackgroundTrading volume and stock returns: A meta-analysis
Bajzík, J. (2021) · International Review of Financial Analysis · doi:10.1016/j.irfa.2021.101923 · Cited by 57
Evans, D. S., Parimi, N., Nievergelt, C. M., Blackwell, T., Redline, S. S., Ancoli‐Israel, S., ORWOLL, E. S., Cummings, S. R., Stone, K. L., Tranah, G. J., et al. (2013) · SLEEP · doi:10.5665/sleep.2466 · Cited by 52
Kent, S. T., Howard, G., Crosson, W. L., Prineas, R. J., & McClure, L. A. (2011) · Environmental Health · doi:10.1186/1476-069x-10-7 · Cited by 48
“Furthermore, this study finds that the previous associations and estimates in earlier studies regarding this relationship were not likely inflated or due to Type I error as can be common with newly discovered associations [25].”
ResultEffect size and statistical power in the rodent fear conditioning literature – A systematic review
Carneiro, C. F. D., Moulin, T. C., Macleod, M. R., & Amaral, O. B. (2018) · PLoS ONE · doi:10.1371/journal.pone.0196258 · Cited by 54
“Moreover, most of the large effect sizes in our sample were found in underpowered studies, suggesting that they could represent inflated estimates [22].”
BackgroundResultResting States and Memory Consolidation: A Preregistered Replication and Meta-Analysis
Humiston, G. B., Tucker, M. A., Summer, T., & Wamsley, E. J. (2019) · Scientific Reports · doi:10.1038/s41598-019-56033-6 · Cited by 56
Effect of Weight Loss Medications on Hepatic Steatosis and Steatohepatitis: A Systematic Review
Pan, C. S., & Stanley, T. L. (2020) · Frontiers in Endocrinology · doi:10.3389/fendo.2020.00070 · Cited by 46
Hullman, J. R., Kapoor, S., Nanayakkara, P., Gelman, A., & Narayanan, A. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534196 · Cited by 59
“ognized, corrective actions should be taken, and claims discounted— especially when they cannot be externally reproduced [42, 85, 125].”
BackgroundA New Look at P Values for Randomized Clinical Trials
van Zwet, E. W., Gelman, A., Greenland, S., Imbens, G. W., Schwab, S., & Goodman, S. N. (2023) · NEJM Evidence · doi:10.1056/evidoa2300003 · Cited by 57
“Its connection to results of randomized trials has been pointed out by several authors 11 5 19 .”
BackgroundConsequences of relying on statistical significance: Some illustrations
CALSTER, B. V., Steyerberg, E. W., Collins, G. S., & Smits, T. (2018) · European Journal of Clinical Investigation · doi:10.1111/eci.12912 · Cited by 47
A Causal Replication Framework for Designing and Assessing Replication Efforts
Steiner, P. M., Wong, V. C., & Anglin, K. (2019) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000385 · Cited by 54
“Ioannidis (2008) reviewed more than 1,000 medical publications and found that only 44% of replication efforts produced results that corresponded with the original findings.”
ResultA Tutorial on Hunting Statistical Significance by Chasing N
Szucs, D. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01444 · Cited by 45
“Inthis paper I deal with special cases of what Simmons et al. (2011) called ‘researcher degrees of freedom,’ or put otherwise, special factors behind what Ioannidis (2008) termed ‘the vibration ratio.’”
BackgroundPatel, C. J., Ji, J., Sundquist, J., Ioannidis, J. P. A., & Sundquist, K. (2016) · Scientific Reports · doi:10.1038/srep31308 · Cited by 51
“We associated each of the 552 possible drug prescriptions with (1) any, (2) breast, (3) prostate, and (4) colon cancer risk using two different modeling strategies in order to see how much the results are affected by the model choice and its assumptions.”
Self-citationCharacterizing the pathogenicity of genetic variants: the consequences of context
Ciesielski, T. H., Sirugo, G., Iyengar, S. K., & Williams, S. M. (2024) · npj Genomic Medicine · doi:10.1038/s41525-023-00386-5 · Cited by 48
“…can be partly explained by the factors that drive the winner ’ s curse (i.e. in fl ated magnitude of initial associations due to low power, publication bias, model over fi tting, etc.) 24,25 , it must be added that smaller associations should be expected when the study participants are more diverse.”
BackgroundJanus kinase-1 and Janus kinase-2 inhibitors for treating myelofibrosis
Martí-Carvajal, A. J., Anand, V., & Solà, I. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010298.pub2 · Cited by 51
Interventions for treating leg ulcers in people with sickle cell disease
Martí-Carvajal, A. J., Knight‐Madden, J., & Martínez‐Zapata, M. J. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008394.pub3 · Cited by 49
Ioannidis, J. P. A., Zhou, Y., Chang, C. Q., Schully, S. D., Khoury, M. J., & Freedman, A. N. (2013) · Annals of Oncology · doi:10.1093/annonc/mdt372 · Cited by 53
“This may be due to the winner’s curse (a regression-to-the-mean phenomenon), where results selected on the basis of statistical significance are expected to have inflated effect sizes, even if some association is genuinely present [78].”
BackgroundSelf-citationQin, X., Jia, G., Zhou, X., & Yang, Z. (2022) · Advances in Nutrition · doi:10.1093/advances/nmac087 · Cited by 51
Handavu, F., Syampungani, S., Sileshi, G. W., & Chirwa, P. W. (2021) · Carbon Management · doi:10.1080/17583004.2021.1926330 · Cited by 46
Using meta-analysis for evidence synthesis: The case of incomplete neutralization in German
Nicenboim, B., Roettger, T. B., & Vasishth, S. (2018) · Journal of Phonetics · doi:10.1016/j.wocn.2018.06.001 · Cited by 50
The case for preregistering all region of interest (ROI) analyses in neuroimaging research
Gentili, C., Cecchetti, L., Handjaras, G., Lettieri, G., & Cristea, I. A. (2020) · European Journal of Neuroscience · doi:10.1111/ejn.14954 · Cited by 47
“In the functional magnetic resonance imaging (fMRI) literature, false report probability (FRP), the probability that statistically significant findings are actually not true (Ioannidis, 2005), as well as the probability that reported effects are inflated (Ioannidis, 2008), are pervasive problems.”
BackgroundWhy bother using non-human primate models of cognitive disorders in translational research?
Camus, S. M. J., Ko, W. K. D., Pioli, E. Y., & Bézard, E. (2015) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2015.06.012 · Cited by 45
Janssens, A. C. J. W., Ioannidis, J. P. A., Bedrosian, S., Boffetta, P., Dolan, S. M., Dowling, N., Fortier, I., Freedman, A. N., Grimshaw, J. M., Gulcher, J., et al. (2011) · European Journal of Clinical Investigation · doi:10.1111/j.1365-2362.2011.02493.x · Cited by 40
Self-citationDo Newborns Have the Ability to Imitate?
Slaughter, V. P. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2021.02.006 · Cited by 47
Vermeulen, I., Beukeboom, C. J., Batenburg, A., Avrămiea, A.-E., Stoyanov, D. S., van de Velde, B., & Oegema, D. (2015) · Communication Methods and Measures · doi:10.1080/19312458.2015.1096333 · Cited by 47
“Second, publication bias’s selection on p-values also yields a selection on effect sizes: Results with higher effect sizes are more likely to be published than results with lower effect sizes (Ioannidis, 2008), such that strong effects are overrepresented in the literature.”
BackgroundExchange rate regimes and FDI in developing countries: A propensity score matching approach
Cushman, D. O., & Vita, G. D. (2017) · Journal of International Money and Finance · doi:10.1016/j.jimonfin.2017.07.018 · Cited by 54
Carbine, K. A., & Larson, M. J. (2019) · Health Psychology Review · doi:10.1080/17437199.2019.1622144 · Cited by 46
Excess death estimates from multiverse analysis in 2009–2021
Levitt, M., Zonta, F., & Ioannidis, J. P. A. (2023) · European Journal of Epidemiology · doi:10.1007/s10654-023-00998-2 · Cited by 51
“Commonly used terms are “multiverse analysis” (11-14), “vibration of effects” (16-18) and “multi-analyst analysis” (19,20) (when multiple researchers are each asked to select .”
BackgroundSelf-citationGordon, M., Viganola, D., Dreber, A., Johannesson, M., & Pfeiffer, T. (2021) · PLOS ONE · doi:10.1371/journal.pone.0248780 · Cited by 44
“increased substantially for the SSRP project following concerns that effect sizes for original findings may be inflated [12, 22], which increases the chances of false negatives among the replication outcomes in the RPP and EERP projects.”
BackgroundLitvin, S., Saunders, R., Jefferies, P., Seely, H., Pössel, P., & Lüttke, S. (2023) · JMIR Mental Health · doi:10.2196/47285 · Cited by 43
“Insufficient sample sizes can promote favorable effects of particular interventions [1,2,5], reducing the likelihood that a statistically significant result reflects a true effect [26,27].”
BackgroundMethodVideo games and attitude change: A meta-analysis
Kolek, L., Ropovik, I., Šisler, V., van Oostendorp, H., & Brom, C. (2023) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2023.102225 · Cited by 45
Measuring Sex Differences and Similarities
Giudice, M. D. (2022) · Focus on sexuality research · doi:10.1007/978-3-030-84273-4_1 · Cited by 47
Digit ratio (2D:4D) and sex-role orientation: Further evidence and meta-analysis
Voracek, M., Pietschnig, J., Nader, I. W., & Stieger, S. (2010) · Personality and Individual Differences · doi:10.1016/j.paid.2010.06.009 · Cited by 43
Role of habit in treatment adherence among adults with cystic fibrosis
Hoo, Z. H., Gardner, B., Arden, M. A., Waterhouse, S., Walters, S. J., Campbell, M. J., Hind, D., Maguire, C., Dewar, J., & Wildman, M. J. (2019) · Thorax · doi:10.1136/thoraxjnl-2017-211453 · Cited by 40
The “sex gap” in COVID-19 trials: a scoping review
Schiffer, V. M. M. M., Janssen, E. B. N. J., van Bussel, B. C. T., Jorissen, L. M., Tas, J., Sels, J.-W. E. M., Bergmans, D. C. J. J., Dinh, T., van Kuijk, S. M. J., Hana, A., et al. (2020) · EClinicalMedicine · doi:10.1016/j.eclinm.2020.100652 · Cited by 43
“For already published studies, we recommend investigators to perform post-hoc sex-specific analyses, while considering potential power insufficiencies in the interpretation of the results [68,69].”
MethodDavis‐Stober, C. P., & Dana, J. D. (2013) · Behavior Research Methods · doi:10.3758/s13428-013-0342-1 · Cited by 43
“depend upon unreported researcher activities, such as publication bias (Francis, 2012a, b; Ioannidis, 2008) or researcher degrees of freedom (Simmons, Nelson, & Simonsohn, 2011).”
BackgroundA 20-Year Review of Outcome Reporting Bias in Moderated Multiple Regression
O’Boyle, E. H., Banks, G. C., Carter, K., Walter, S. L., & Yuan, Z. (2018) · Journal of Business and Psychology · doi:10.1007/s10869-018-9539-8 · Cited by 43
“ORB is at the heart of several movements underway in both the physical and social sciences to evaluate the evidentiary value of empirical research (e.g., Finkel, Eastwick, & Reis, 2015; Ioannidis, 2008; Nosek & Bar-Anan, 2012).”
BackgroundWhy replication has more scientific value than original discovery
Ioannidis, J. P. A. (2018) · Behavioral and Brain Sciences · doi:10.1017/s0140525x18000729 · Cited by 51
Self-citationHeterogeneity in banker culture and its influence on dishonesty
Rahwan, Z., Yoeli, E., & Fasolo, B. (2019) · Nature · doi:10.1038/s41586-019-1741-y · Cited by 54
Behavioral and brain signatures of substance use vulnerability in childhood
Rapuano, K. M., Rosenberg, M. D., Maza, M. T., Dennis, N. J., Dorji, M., Greene, A. S., Horien, C. L., Scheinost, D., Constable, R. T., & Casey, B. J. (2020) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2020.100878 · Cited by 43
“, 2020)—and conversely, that statistically significant effects in underpowered studies are likely to be inflated (Button et al., 2013; Ioannidis, 2008; Marek et al., 2020; Yarkoni, 2009), it is unsurprising that the effect sizes .”
BackgroundResultAntibiotics for treating acute chest syndrome in people with sickle cell disease
Martí-Carvajal, A. J., Conterno, L. D. O., & Knight‐Madden, J. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006110.pub4 · Cited by 42
Identification of and correction for publication biasPreprint
Kasy, M., & Andrews, I. (2018) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/49yst · Cited by 42
Bogg, T., & Lasecki, L. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.01589 · Cited by 41
“…the range of effect sizes located by Au et al. (2014; Hedge’s g range, −0.28 to 1.10), small studies are more likely to show a vibration of effects (Ioannidis, 2008), where inconsistencies in analytic approaches and tests (removing versus retaining outliers, controlling for covariates), as well…”
BackgroundInfluentialMallamaci, F., D’Arrigo, G., Tripepi, G. L., Lamberti, N., Torino, C., Manfredini, F., & Zoccali, C. (2022) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.03160322 · Cited by 46
“History of neoplasia, % 22 (22) 22 (18) Antihypertensive therapy, % 76 (77) 82 (70)”
MethodPublication bias in “Red, rank, and romance in women viewing men,” by Elliot et al. (2010).
Francis, G. (2013) · Journal of Experimental Psychology General · doi:10.1037/a0027923 · Cited by 50
“The biased experiments probably overestimate the true effect size (Ioannidis, 2008); so suppose the true effect of the color red on attractiveness ratings is half the two-tailed, two-sample t-test using a Type I error rate of 0.05 and power of 0.8 (Cohen, 1988), the new experiment needs around 103…”
BackgroundMethodInfluentialJoshi, A. D., Lindström, S., Hüsing, A., Barrdahl, M., VanderWeele, T. J., Campa, D., Canzian, F., Gaudet, M. M., Figueroa, J. D., Baglietto, L., et al. (2014) · American Journal of Epidemiology · doi:10.1093/aje/kwu214 · Cited by 41
Miller, J. O., & Ulrich, R. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616649170 · Cited by 43
“…the estimation of true effect sizes is much more difficult if only significant results are published (e.g., Hedges, 1984, 1992; Holman, Head, Lanfear, & Jennions, 2015; Ioannidis, 2008; Iyengar & Greenhouse, 1988; Lane & Dunlap, 1978; Simonsohn, Nelson, & Simmons, 2014a; van Assen et al., 2015).”
Backgroundvan Roekel, E. H., Trijsburg, L. E., Assi, N., Carayol, M., Achaintre, D., Murphy, N., Rinaldi, S., Schmidt, J. A., Stępień, M., Kaaks, R., et al. (2018) · Nutrients · doi:10.3390/nu10050654 · Cited by 44
“The availability of longitudinal data would clarify whether alcohol intake could modify specific metabolic responses within the same individuals over time.”
Benjamin, D. J., Laibson, D., Mischel, W., Peake, P. K., Shoda, Y., Wellsjo, A., & Wilson, N. L. (2019) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2019.08.016 · Cited by 45
“Due to Bayesian “shrinkage” toward the prior, the Bayesian posterior means do not suffer from the “statistical winner’s curse” bias ( Ioannidis, 2008 ) and can thus serve as our “best guess” effect-size estimates.”
BackgroundWhat is the optimum design for my animal experiment?
Karp, N. A., & Fry, D. (2021) · BMJ Open Science · doi:10.1136/bmjos-2020-100126 · Cited by 45
Belgers, M., van Eijndhoven, P., Markus, W., Schene, A. H., & Schellekens, A. (2022) · Journal of Clinical Medicine · doi:10.3390/jcm11040951 · Cited by 45
“Small samples increase the risk of type I/II errors and might inflate the magnitude of effect (winner’s curse) [71].”
BackgroundMeta-analyzing the multiverse: A peek under the hood of selective reporting.
Olsson-Collentine, A., van Aert, R. C. M., Bakker, M., & Wicherts, J. M. (2025) · Psychological Methods · doi:10.1037/met0000559 · Cited by 52
Sidebotham, D. A., & Barlow, C. J. (2023) · Anaesthesia · doi:10.1111/anae.16161 · Cited by 40
Spurious regressions and near-multicollinearity, with an application to aid, policies and growth
Chatelain, J., & Ralf, K. (2014) · Journal of Macroeconomics · doi:10.1016/j.jmacro.2013.11.003 · Cited by 40
Lataster, J., Collip, D., Ceccarini, J., Hernaus, D., Haas, D., Booij, L., van Os, J., Pruessner, J. C., Laere, K. V., & Myin‐Germeys, I. (2013) · Schizophrenia Bulletin · doi:10.1093/schbul/sbs187 · Cited by 43
Suarez-Gestal, M., Pérez-Pampín, E. M., Calaza, M., Gómez-Reino, J. J., & Gonzalez, A. (2010) · Arthritis Research & Therapy · doi:10.1186/ar2990 · Cited by 44
“Therefore, it seems more likely that the original strong associations were due to random variation of allele frequencies in a study including more than 300,000 SNPs and to the heavy bias characteristic of GWASs of small sample size [17,18].”
BackgroundMagnitude of effects in clinical trials published in high-impact general medical journals
Siontis, K. C., Evangelou, E., & Ioannidis, J. P. (2011) · International Journal of Epidemiology · doi:10.1093/ije/dyr095 · Cited by 38
Self-citationThe statistical properties of RCTs and a proposal for shrinkage
van Zwet, E. W., Schwab, S., & Senn, S. J. (2021) · Statistics in Medicine · doi:10.1002/sim.9173 · Cited by 42
“Overestimation of the effect size and undercoverage of the confidence interval are obviously serious issues which are part of the explanation for the phenomenon of poor replication [8, 9, 2, 1].”
BackgroundΕvangelou, Ε., Siontis, K., Pfeiffer, T., & Ioannidis, J. P. A. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.06.009 · Cited by 42
Self-citationPractical tools and strategies for researchers to increase replicability
Nuijten, M. B. (2018) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.14054 · Cited by 45
“How solid are the conclusions we can draw from scientific research? This question is asked more and more, and there is growing concern that many published findings are overly optimistic.(1) A vast amount of research has found evidence for problems such as overestimated effects, low replicability rates, and statistical…”
BackgroundMRI-based biomarkers of accelerated aging and dementia risk in midlife: how close are we?
Elliott, M. L. (2020) · Ageing Research Reviews · doi:10.1016/j.arr.2020.101075 · Cited by 44
May, R. J., Tyndall, I. T., McTiernan, A., Roderique‐Davies, G., & McLoughlin, S. (2022) · British Journal of Educational Technology · doi:10.1111/bjet.13192 · Cited by 40
Ekkekakis, P., Swinton, P. A., & Tiller, N. B. (2023) · Sports Medicine · doi:10.1007/s40279-023-01880-7 · Cited by 40
“An additional problem, named ‘winner’s curse’ [114, 115], emerges from underpowered studies.”
BackgroundReply to: Multivariate BWAS can be replicable with moderate sample sizes
Tervo‐Clemmens, B., Marek, S. A., Chauvin, R. J., Van, A. N., Kay, B. P., Laumann, T. O., Thompson, W. K., Nichols, T. E., Yeo, B. T. T., Barch, D. M., et al. (2023) · Nature · doi:10.1038/s41586-023-05746-w · Cited by 44
Does sentiment affect stock returns? A meta-analysis across survey-based measures
Gric, Z., Bajzík, J., & Badura, O. (2023) · International Review of Financial Analysis · doi:10.1016/j.irfa.2023.102773 · Cited by 42
Interpreting Randomized Controlled Trials
Msaouel, P., Lee, J., & Thall, P. F. (2023) · Cancers · doi:10.3390/cancers15194674 · Cited by 40
“Phase 3 RCTs with negative results are less likely to be accepted for publication by the editors of medical journals [29], so the estimated effect sizes in published phase 3 RCTs are biased upward and thus are likely to overstate actual treatment differences [29,58].”
BackgroundLearning design research: advancing pedagogies in the digital age
Dobozy, E. (2013) · Educational Media International · doi:10.1080/09523987.2013.777181 · Cited by 38
“Nevertheless, “ what constitutes fair interpretation of new discoveries is unavoidably subjective ” (Ioannidis, 2008, p. 645).”
BackgroundGene expression variability in human and chimpanzee populations share common determinants
Fair, B. J., Blake, L. E., Sarkar, A., Pavlovic, B. J., Cuevas, C., & Gilad, Y. (2020) · eLife · doi:10.7554/elife.59929 · Cited by 42
Mixed-effects design analysis for experimental phonetics
Kirby, J., & Sonderegger, M. (2018) · Journal of Phonetics · doi:10.1016/j.wocn.2018.05.005 · Cited by 42
“This is because the likelihood of committing a Type M or Type S error is much higher when power is low, as can be seen in Figure 5 (red lines): statistically significant results from low-powered studies are virtually guaranteed to have an effect size inflated in magnitude, and possibly with the wrong sign, a phenomeno…”
BackgroundResultInfluentialEvaluating Replication ResultsPreprint
Simonsohn, U. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2259879 · Cited by 42
Goddard, K. A. B., Knaus, W. A., Whitlock, E. P., Lyman, G. H., Feigelson, H. S., Schully, S. D., Ramsey, S. D., Tunis, S., Freedman, A. N., Khoury, M. J., et al. (2012) · Genetics in Medicine · doi:10.1038/gim.2012.16 · Cited by 40
“18-20 Further, the attributable risk may be small because of low frequency or low penetrance, or the variant may only be linked to the functional variant.”
BackgroundLee, P. H. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.02.004 · Cited by 39
Genetic model of MS severity predicts future accumulation of disability
Jackson, K. C., Sun, K., Barbour, C. R., Hernandez, D. G., Kosa, P. F., Tanigawa, M., Weideman, A. M., & Bielekova, B. (2019) · Annals of Human Genetics · doi:10.1111/ahg.12342 · Cited by 40
“Thus, the performance of CGRS, derived from a small, unvalidated cohort, represents an overfit (Ioannidis, 2008).”
BackgroundImplications of the open science era for educational psychology research syntheses
Patall, E. A. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1897009 · Cited by 38
“Second, meta-analyses that rely on a small number of studies may be particularly likely to detect a false positive effect, as indicated by research suggesting an inverse relationship between the number of studies in a research synthesis and the pooled effect size (Ioannidis, 2008c).”
BackgroundQin, X., Chen, J., Jia, G., & Yang, Z. (2023) · Advances in Nutrition · doi:10.1016/j.advnut.2023.02.004 · Cited by 39
“, case-control studies), which are likely to report inaccurate dietary consumption measures and suffer from several biases, including recall bias, thereby reducing the strength of the pooled scientific evidence [12,13].”
BackgroundSports genetics moving forward: lessons learned from medical research
Mattsson, C. M., Wheeler, M. T., Waggott, D. M., Caleshu, C. A., & Ashley, E. A. (2016) · Physiological Genomics · doi:10.1152/physiolgenomics.00109.2015 · Cited by 44
“Similarly, exaggerated effect size estimates result in an underestimation of the required replication sample size and subsequently a failure to replicate (37).”
BackgroundKrauth, D., Anglemyer, A., Philipps, R., & Bero, L. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001770 · Cited by 36
“Among clinical studies, most discovered true associations are inflated due to methodological flaws [28].”
BackgroundResultTheory of mind and psychosocial characteristics in older men
Radecki, M. A., Cox, S. R., & MacPherson, S. E. (2019) · Psychology and Aging · doi:10.1037/pag0000324 · Cited by 36
“Finally, while our study was sufficiently powered to reliably detect large- and mediumsized effects, our sample size precluded the reliable estimation of small effects and may have resulted in overfitting and inflation of effect sizes (Ioannidis, 2008).”
ResultMethods of Evolutionary Sciences
Simpson, J. A., & Campbell, L. (2015) · Wiley eBooks · doi:10.1002/9780470939376.ch4 · Cited by 36
Kim, J. H., Kim, J. Y., Lee, S., Lee, S., Stubbs, B., Koyanagi, A., Dragioti, E., Jacob, L., Carvalho, A. F., Raduà, J., et al. (2022) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2022.104761 · Cited by 39
Biomarkers of Inflammation and Immune Function and Risk of Colorectal Cancer
García-Anguita, A., Κακούρου, Α., & Tsilidis, K. K. (2015) · Current Colorectal Cancer Reports · doi:10.1007/s11888-015-0282-5 · Cited by 39
Gehrman, P., Keenan, B. T., Byrne, E. M., & Pack, A. I. (2015) · Psychiatric Clinics of North America · doi:10.1016/j.psc.2015.07.004 · Cited by 41
Oei, L., Estrada, K., Duncan, E. L., Christiansen, C., Liu, C., Langdahl, B. L., Obermayer-Pietsch, B. M., Riancho, J. A., Prince, R. L., van Schoor, N. M., et al. (2013) · Bone · doi:10.1016/j.bone.2013.10.015 · Cited by 37
“Signals in underpowered settings are likely to display inflated effects due to the “winner's curse” phenomenon, where the effect estimate observed in the first study overestimates the actual risk observed at the general population level [70–72].”
BackgroundSelf-citationOvercoming obstacles in obtaining individual participant data for meta‐analysis
Polanin, J. R., & Williams, R. T. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1208 · Cited by 36
Clark, L. V., Dzyubenko, E. A., Dzyubenko, N., Bagmet, L. V., Sabitov, A., Chebukin, P. A., Johnson, D. A., Kjeldsen, J. B., Petersen, K. K., Jørgensen, U., et al. (2016) · Annals of Botany · doi:10.1093/aob/mcw137 · Cited by 40
“Due to the small number of individuals included in our GWAS, it is possible that we have overestimated QTL effects (Beavis, 1998; Ioannidis, 2008); however, they represent testable hypotheses, and we intend to perform crosses to validate these QTLs so that they can potentially be used in breeding…”
Background
Metadata
- Authors
- Ioannidis, J. P. A.
- Affiliations
- University of Ioannina (GR)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- Epidemiology
- Published
- 2008-09
- Volume
- 19
- Issue
- 5
- Pages
- 640-648
- ISSN
- 1044-3983 (ISSN-L)
- OpenAlex
- W2150215887
- Semantic Scholar
- 0373f18622b7b64e99c7e56340ab2011dbd00b5d
- Cited by
- 1,495
- References
- 109
- Citations by year
- 2026: 91 · 2025: 87 · 2024: 78 · 2023: 112 · 2022: 94 · 2021: 99 · 2020: 103 · 2019: 131 · 2018: 108 · 2017: 101 · 2016: 87 · 2015: 77 · 2014: 72 · 2013: 60 · 2012: 50
- FWCI
- 15.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- effect size inflation, statistical significance thresholds, underpowered studies, selective reporting, conflicts of interest, reverse bias, measurement error, replication
- MeSH terms
- Clinical Trials as Topic, Data Interpretation, Statistical, Humans, Sensitivity and Specificity, Models, Statistical, Linkage Disequilibrium, Molecular Epidemiology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.