Why Most Discovered True Associations Are Inflated
The DOI matches its Crossref record; 111 of the 111 references it cites were found in a registry or an index.
Abstract
Newly discovered true (non-null) associations often have inflated effects compared with the true effect sizes. I discuss here the main reasons for this inflation. First, theoretical considerations prove that when true discovery is claimed based on crossing a threshold of statistical significance and the discovery study is underpowered, the observed effects are expected to be inflated. This has been demonstrated in various fields ranging from early stopped clinical trials to genome-wide associations. Second, flexible analyses coupled with selective reporting may inflate the published discovered effects. The vibration ratio (the ratio of the largest vs. smallest effect on the same association approached with different analytic choices) can be very large. Third, effects may be inflated at the stage of interpretation due to diverse conflicts of interest. Discovered effects are not always inflated, and under some circumstances may be deflated-for example, in the setting of late discovery of associations in sequentially accumulated overpowered evidence, in some types of misclassification from measurement error, and in conflicts causing reverse biases. Finally, I discuss potential approaches to this problem. These include being cautious about newly discovered effect sizes, considering some rational down-adjustment, using analytical methods that correct for the anticipated inflation, ignoring the magnitude of the effect (if not necessary), conducting large studies in the discovery ph…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Given that initial effect sizes are often inflated 61 and that we sought power to detect a difference between two groups, we doubled this estimate and aimed to test ~28 participants in each group with some attrition expected.”
From Adults with autism overestimate the volatility of the sensory environment (2017) · cites this paper as [61]
In the citing paper
“While this may reflect exaggeration of original effect sizes or publication bias (Ioannidis 2008), some degree of inflation is expected even in its absence, as our selection process filtered for original experiments with significant differences.”
From Estimating the replicability of Brazilian biomedical science (2025) · cites this paper as Ioannidis 2008
In the citing paper
“Further, in situations when an underpowered study detects a true effect, the estimate of this effect size is likely to be highly exaggerated, a phenomenon often referred to as the “winner’s curse” (15).”
From Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies (2016) · cites this paper as [15]
In the citing paper
“Researchers have strong incentives to interpret their results as supporting or extending the Bayesian framework, rather than challenging it—leading to a literature that systematically over-represents supportive evidence (Ioannidis 2008; Nosek et al. 2012).”
From The myth of the Bayesian brain (2025) · cites this paper as Ioannidis 2008
In the citing paper
“This flexibility in the synthesis of the evidence provides the assessors with the opportunity to inflate or disinflate the efficacy of a given technology, a phenomenon that was observed in scientific publications.[24]”
From Health technology assessments conducted in health care facilities: A strategic practice? Findings from a content analysis of HTA reports (2017) · cites this paper as [24]
8 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationGene expression profiling predicts clinical outcome of breast cancer
van 't Veer, L. J., Dai, H., van de Vijver, M. J., He, Y. D., Hart, A. A. M., Mao, M., Peterse, H. L., van der Kooy, K., Marton, M. J., Witteveen, A. T., et al. (2002) · Nature · doi:10.1038/415530a · Cited by 9,536
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040297 · Cited by 8,613
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“This article will not deal with the debate on whether research findings are credible.(1) I will focus instead on the interesting subset of research findings that are true.”
BackgroundSelf-citationEffect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes
Nissen, S. E., & Wolski, K. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa072761 · Cited by 4,675
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., & Egger, M. (2007) · Epidemiology · doi:10.1097/ede.0b013e3181577511 · Cited by 2,779
Internal validation of predictive models
Steyerberg, E. W., Harrell, F. E., Borsboom, G. J. J. M., Eijkemans, M. J. C., Vergouwe, Y., & Habbema, J. D. F. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00341-9 · Cited by 2,746
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544
Cited by
Items 301–400 of 1,004
The Replication Paradox: Combining Studies can Decrease Accuracy of Effect Size Estimates
Nuijten, M. B., van Assen, M. A. L. M., Veldkamp, C. L. S., & Wicherts, J. M. (2015) · Review of General Psychology · doi:10.1037/gpr0000034 · Cited by 69
“As our results showed, studies with high power will contain less bias in their effect size (see also Bakker et al., 2012; Button et al., 2013; Ioannidis, 2008; Kraemer et al., 1998).”
ResultCaviola, S., Colling, L. J., Mammarella, I. C., & Szűcs, D. (2020) · Developmental Science · doi:10.1111/desc.12957 · Cited by 72
“Considering that the data from Schneider et al. also includes adult studies and metaanalyses are subject to effect size exaggeration (Ioannidis, 2008; Szűcs & Ioannidis, 2017b) the larger meta-analytic estimate is not surprising.”
BackgroundMethodHeart rate variability in mental disorders: an umbrella review of meta-analyses
Wang, Z., Zou, Y., Liu, J., Peng, W., Li, M., & Zou, Z. (2025) · Translational Psychiatry · doi:10.1038/s41398-025-03339-x · Cited by 80
Clinical Trials: Discerning Hype From Substance
Fleming, T. R. (2010) · Annals of Internal Medicine · doi:10.7326/0003-4819-153-6-201009210-00008 · Cited by 72
A Compendium of Genome-Wide Associations for Cancer: Critical Synopsis and Reappraisal
Ioannidis, J. P. A., Castaldi, P. J., & Εvangelou, Ε. (2010) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djq173 · Cited by 81
“Discovered estimates may be slightly or modestly inflated because of winner’s curse (70,71).”
BackgroundSelf-citationHoffmann, S., Schönbrodt, F. D., Elsas, R., Wilson, R., Strasser, U., & Boulesteix, A. (2021) · Royal Society Open Science · doi:10.1098/rsos.201925 · Cited by 88
Zeiler, F. A., McFadyen, C., Newcombe, V., Synnot, A., Donoghue, E. L., Ripatti, S., Steyerberg, E. W., Gruen, R. L., McAllister, T. W., Rosand, J. M., et al. (2021) · Journal of Neurotrauma · doi:10.1089/neu.2017.5583 · Cited by 73
“The poor statistical power inherent in these small sample sizes not only reduces the confidence in the effect sizes reported, but is also likely to bias toward the finding of positive results.(97) This lack of statistical power is further compounded by (the often unrecognized) multiple testing in some patient populati…”
ResultPerceived crisis and reforms: Issues, explanations, and remedies.
De Boeck, P., & Jeon, M. (2018) · Psychological Bulletin · doi:10.1037/bul0000154 · Cited by 64
Zinsser, K. M., Gordon, R. A., & Xue, J. (2021) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2021.02.001 · Cited by 76
Lubieniecka, J. M., Graham, J., Heffner, D. J., Mottus, R., Reid, R. E., Hogge, D. E., Grigliatti, T. A., & Riggs, W. K. (2013) · Frontiers in Genetics · doi:10.3389/fgene.2013.00231 · Cited by 76
“These point and interval estimates of the OR are expected to be biased away from one as a result of the multiple genes queried (Ioannidis, 2008).”
BackgroundWhy Selective Publication of Statistically Significant Results Can Be Effective
de Winter, J., & Happee, R. (2013) · PLoS ONE · doi:10.1371/journal.pone.0066463 · Cited by 67
“Published research findings are often false positives [2] or gross exaggerations of the true effect [3,4], especially in domains where effect sizes and sample sizes are small and the prior probability of a hypothesis being true is low [2,5,6].”
BackgroundMagiorkinis, G., Belshaw, R., & Katzourakis, A. (2013) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2012.0504 · Cited by 68
“However, mapping individual ERVs to their genomic positions remains a major experimental and biocomputing challenge, whereas studies of human ERV (HERV) expression are small with no clear consensus [5,6].”
Ojerholm, E., Smith, A. M., Hwang, W., Baumann, B. C., Tucker, K. N., Lerner, S. P., Mamtani, R., Boursi, B., & Christodouleas, J. P. (2016) · Cancer · doi:10.1002/cncr.30422 · Cited by 71
“2 on the univariable analysis were included in a multivariable Cox regression model.(41) We used the 2-sided Wald test to determine significance.”
MethodBrini, S., Sohrabi, H. R., Hébert, J. J., Forrest, M. R. L., Laine, M., Hämäläinen, H., Karrasch, M., Peiffer, J. J., Martins, R. N., & Fairchild, T. J. (2020) · Neuropsychology Review · doi:10.1007/s11065-020-09426-8 · Cited by 71
Saquib, N., Saquib, J., & Ioannidis, J. P. A. (2013) · BMJ · doi:10.1136/bmj.f4313 · Cited by 68
Self-citationOverlapping network meta-analyses on the same topic: survey of published studies
Naudet, F., Schuit, E., & Ioannidis, J. P. A. (2017) · International Journal of Epidemiology · doi:10.1093/ije/dyx138 · Cited by 68
Self-citationClausen, N. G., Kähler, S., & Hansen, T. G. (2018) · British Journal of Anaesthesia · doi:10.1016/j.bja.2017.11.107 · Cited by 70
The role and value of replication in empirical software engineering results
Shepperd, M., Ajienka, N., & Counsell, S. (2018) · Information and Software Technology · doi:10.1016/j.infsof.2018.01.006 · Cited by 60
“1It is the high variance associated with under-powered studies, coupled with selective reporting that can lead to systematic over-estimation of effect sizes[39].”
BackgroundMaking ERP research more transparent: Guidelines for preregistration
Paul, M., Govaart, G., & Schettino, A. (2021) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2021.02.016 · Cited by 71
Milette, K., Roseman, M., & Thombs, B. D. (2010) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2010.09.015 · Cited by 71
“Comparisons of study protocols and published results have shown that significant outcomes are more likely to be reported than nonsignificant outcomes and that nonsignificant prespecified primary outcomes are sometimes replaced with statistically significant secondary outcomes that are identified as “primary” in publis…”
ResultEilertsen, E. M., Gjerde, L. C., Reichborn-Kjennerud, T., Ørstavik, R. E., Knudsen, G. P., Stoltenberg, C., Czajkowski, N., Røysamb, E., Kendler, K. S., & Ystrom, E. (2017) · International Journal of Epidemiology · doi:10.1093/ije/dyx067 · Cited by 60
“A plausible explanation is that several studies are underpowered for studying effects of this magnitude.(32) For instance, a meta-analysis found that the odds of children prenatally exposed to alcohol having ADHD were 2.”
BackgroundAn Assessment of the Cox Proportional Hazards Regression Model for Epidemiologic Studies
Moolgavkar, S. H., Chang, E. T., Watson, H. N., & Lau, E. (2017) · Risk Analysis · doi:10.1111/risa.12865 · Cited by 67
Statistical Inference for Data Adaptive Target Parameters
Hubbard, A., Kherad-Pajouh, S., & van der Laan, M. J. (2016) · The International Journal of Biostatistics · doi:10.1515/ijb-2015-0013 · Cited by 89
“Others have noted the particular dangers of high dimensional data combined with flexible methodologies to generate excessive false positive findings (Ioannidis [9], Broadhurst and Kell [10]).”
Leichtmann, B., Nitsch, V., & Mara, M. (2022) · Frontiers in Robotics and AI · doi:10.3389/frobt.2022.838116 · Cited by 66
“Even if an underpowered study observes an effect, such studies have only small predictive power and are rather overestimated, which is also known as “ winner ’ s curse ” (Ioannidis, 2008; Button et al., 2013).”
BackgroundUsing Bayes factors to evaluate evidence for no effect: examples from the SIPS project
Dienes, Z., Coulton, S., & Heather, N. (2018) · Addiction · doi:10.1111/add.14002 · Cited by 65
An environmental wide association study (EWAS) approach to the metabolic syndrome
Lind, P., Risérus, U., Salihović, S., van Bavel, B., & Lind, L. (2013) · Environment International · doi:10.1016/j.envint.2013.01.017 · Cited by 69
“…this approach will limit the number of false positive findings induced by selective reporting of significant results, as previously been discussed by Ioannidis and others (Blair et al., 2009; Boffetta et al., 2008; Fallin and Kao, 2011; Ioannidis, 2005, 2008; Ioannidis et al., 2009; Young, 2010).”
ResultA meta-analysis of coffee consumption and pancreatic cancer
Turati, F., Galeone, C., Edefonti, V. C., Ferraroni, M., Λάγιου, Π., Vecchia, C. L., & Tavani, A. (2011) · Annals of Oncology · doi:10.1093/annonc/mdr331 · Cited by 75
“over time to approach unity since the mid-1900s, is likely due to false-positive results in earlier studies [15, 16].”
ResultElective, Major Noncardiac Surgery on the Weekend
McIsaac, D. I., Bryson, G. L., & van Walraven, C. (2014) · Medical Care · doi:10.1097/mlr.0000000000000137 · Cited by 65
Jin, Z., Zhang, Q., Liu, K., Wang, S., Yan, Y., Zhang, B., Zhao, L., & Tong, X. (2024) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2024.111615 · Cited by 68
Replication and contradiction of highly cited research papers in psychiatry: 10-year follow-up
Tajika, A., Ogawa, Y., Takeshima, N., Hayasaka, Y., & Furukawa, T. A. (2015) · British Journal of Psychiatry · doi:10.1192/bjp.bp.113.143701 · Cited by 61
Kim, J.-H., Shin, Y.-J., & Joo, Y.-H. (2016) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s108537 · Cited by 53
“Given the relatively small sample size, correlation coefficients from multiple regression and mediation analysis may have tended to overestimate associations and need to be interpreted carefully.(55) A prospective study with a larger sample is warranted to confirm the stability of the relationships identified herein.”
Electroconvulsive Therapy for Agitation and Aggression in Dementia: A Systematic Review
van den Berg, J. F., Kruithof, H. C., Kok, R., Verwijk, E., & Spaans, H. (2017) · American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2017.09.023 · Cited by 71
Registered Replication Report on Fischer, Castel, Dodd, and Pratt (2003)
Colling, L. J., Szűcs, D., Marco, D. D., Cipora, K., Ulrich, R., Nuerk, H., Soltanlou, M., Bryce, D., Chen, S.-C., Schroeder, P. A., et al. (2020) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920903079 · Cited by 66
“…a study’s results typically affects whether or not the study is published, which results in a set of published studies that is not representative, and (b) meta-analytic results are biased when the set of studies analyzed is not representative (Ioannidis, 2008; McShane, Böckenholt, & Hansen, 2016).”
Incorrect results in software engineering experiments: How to improve research practices
Jørgensen, M., Dybå, T., Liestøl, K., & Sjøberg, D. I. K. (2015) · Journal of Systems and Software · doi:10.1016/j.jss.2015.03.065 · Cited by 70
“A high level of publication and researcher bias increases the proportion of incorrect research results and inflates the effect sizes [19, 20].”
BackgroundDoes Neonatal Imitation Exist? Insights From a Meta-Analysis of 336 Effect Sizes
Davis, J., Redshaw, J., Suddendorf, T., Nielsen, M., Kennedy‐Costantini, S., Oostenbroek, J., & Slaughter, V. P. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620959834 · Cited by 66
“plots can be used to diagnose publication bias where other sources of heterogeneity have already been considered (Ioannidis, 2008).”
BackgroundKroesbergen, E. H., Huijsmans, M. D. E., & den Bos, I. F. (2022) · Review of Educational Research · doi:10.3102/00346543221132773 · Cited by 69
“Next, all effect sizes that exceeded 3 were recoded as 3 because of the reported problems in replicability and reliability that may especially affect very large effect sizes associated with small samples (Ioannidis, 2008).”
BackgroundWessel, J., McDonald, S. M., Hinds, D. A., Stokowski, R. P., Javitz, H. S., Kennemer, M. I., Krasnow, R. E., Dirks, W., Hardin, J., Pitts, S. J., et al. (2010) · Neuropsychopharmacology · doi:10.1038/npp.2010.120 · Cited by 68
“Small sample sizes leading to underpowered candidate gene studies have been an often-cited reason for inconsistent results across studies (Ioannidis, 2008).”
ResultDesign and Analysis for Studying microRNAs in Human Disease: A Primer on -Omic Technologies
Nair, V. S., Pritchard, C. C., Tewari, M., & Ioannidis, J. P. A. (2014) · American Journal of Epidemiology · doi:10.1093/aje/kwu135 · Cited by 60
Self-citationDiscovery Properties of Genome-wide Association Signals From Cumulatively Combined Data Sets
Pereira, T. V., Patsopoulos, N. A., Salanti, G., & Ioannidis, J. P. A. (2009) · American Journal of Epidemiology · doi:10.1093/aje/kwp262 · Cited by 60
Self-citationHéroux, M. E., Taylor, J. L., & Gandevia, S. C. (2015) · PLoS ONE · doi:10.1371/journal.pone.0144151 · Cited by 66
Chung, M. K., Louis, G. M. B., Kannan, K., & Patel, C. J. (2018) · Environment International · doi:10.1016/j.envint.2018.11.037 · Cited by 63
“However, as measurement capacity increases and more phenotypes and exposures are measured in epidemiological and observational cohorts, findings are prone to publication bias and associations could be falsely identified [6,7].”
BackgroundPalpacuer, C., Hammas, K., Duprez, R., Laviolle, B., Ioannidis, J. P. A., & Naudet, F. (2019) · BMC Medicine · doi:10.1186/s12916-019-1409-3 · Cited by 67
Self-citationDieguez‐Gonzalez, R., Calaza, M., Pérez-Pampín, E. M., Balsa, A., Blanco, F. J., Cañete, J. D. D., Cáliz, R., Carreño, L., de la Serna, A. R., Fernández‐Gutiérrez, B., et al. (2009) · Arthritis Research & Therapy · doi:10.1186/ar2650 · Cited by 62
“tion because they are conditional on those studies being the first to detect the association [22].”
BackgroundSex is a defining feature of neuroimaging phenotypes in major brain disorders
Salminen, L. E., Tubi, M. A., Bright, J. K., Thomopoulos, S. I., Wieand, A., & Thompson, P. M. (2021) · Human Brain Mapping · doi:10.1002/hbm.25438 · Cited by 69
Are we witnessing the decline effect in the Type D personality literature? What can be learned?
Coyne, J. C., & de Voogd, J. N. (2012) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2012.09.016 · Cited by 67
Interpreting epidemiologic studies of developmental neurotoxicity: Conceptual and analytic issues
Bellinger, D. C. (2009) · Neurotoxicology and Teratology · doi:10.1016/j.ntt.2009.06.006 · Cited by 66
Putting Psychology to the Test: Rethinking Model Evaluation Through Benchmarking and Prediction
Rocca, R., & Yarkoni, T. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459211026864 · Cited by 63
“…fail to fully appreciate the countervailing costs—such as the degree to which small samples inflate effect sizes (Gelman & Weakliem, 2009; Ioannidis, 2008; Yarkoni, 2009) and how, in the presence of selection bias, false-positive rates (Button et al., 2013) are susceptible to…”
BackgroundIoannidis, J. P. A. (2014) · Biostatistics · doi:10.1093/biostatistics/kxt036 · Cited by 60
“See Ioannidis (2008) for references on this issue; Siontis and others (2011) on inflated effects in major journals; and Pereira and others (2011) on inflation of large effects in >85 000 meta-analyses.”
Self-citationGenomic evidence that resource-based trade-offs limit host-range expansion in a seed beetle
Gompert, Z., & Messina, F. J. (2016) · Evolution · doi:10.1111/evo.12933 · Cited by 63
“Third, we expect that point estimates of s are biased upwards for cases where the posterior distribution of s mostly excludes zero, as dictated by the winner’s curse phenomenon (Beavis 1998; Ioannidis 2008; Open Science Collaboration 2015).”
Daigneault, I., Vézina-Gagnon, P., Bourgeois, C., ESPOSITO, T., & Hébert, M. (2017) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2017.02.038 · Cited by 66
Marchetti, M. A., Coit, D. G., Dusza, S. W., Yu, A. M., McLean, L., Hu, Y., Nanda, J. K., Matsoukas, K., Mancebo, S. E., & Bartlett, E. K. (2020) · JAMA Dermatology · doi:10.1001/jamadermatol.2020.1731 · Cited by 64
Be positive about negatives–recommendations for the publication of negative (or null) results
Bespalov, A., Steckler, T., & Skolnick, P. (2019) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2019.10.007 · Cited by 59
Statistical power in network neuroscience
Helwegen, K., Libedinsky, I., & van den Heuvel, M. P. (2023) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2022.12.011 · Cited by 63
Panagiotou, O. A., Evangelou, E., & Ioannidis, J. P. A. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwq234 · Cited by 58
Self-citationFrone, M. R. (2018) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2018.05.001 · Cited by 66
“Large probability samples provide (a) more variation in the outcome variables, (b) adequate statistical power to detect hypothesized effects, and (c) more accurate estimates of population parameters (Ioannidis, 2005, 2008; Schmidt, 1992).”
BackgroundOrtega‐Loubon, C., Herrera‐Gómez, F., Bernuy, C., Jorge-Monjas, P., Sangrador, C. O., Bustamante‐Munguira, J., Tamayo, E., & Alvarez, E. (2019) · Journal of Clinical Medicine · doi:10.3390/jcm8122208 · Cited by 60
“Reporting bias can lead to overly optimistic conclusions in systematic reviews [45].”
BackgroundRegistration practices for observational studies on ClinicalTrials.gov indicated low adherence
Boccia, S., Rothman, K. J., Panić, N., Flacco, M. E., Rosso, A., Pastorino, R., Manzoli, L., Vecchia, C. L., Villari, P., Boffetta, P., et al. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.09.009 · Cited by 63
Self-citationStent Retrievers for the Treatment of Acute Ischemic Stroke
Touma, L., Filion, K. B., Sterling, L. H., Atallah, R., Windle, S. B., & Eisenberg, M. J. (2016) · JAMA Neurology · doi:10.1001/jamaneurol.2015.4441 · Cited by 58
Bayesian evaluation of effect size after replicating an original study
van Aert, R. C. M., & van Assen, M. A. L. M. (2017) · PLoS ONE · doi:10.1371/journal.pone.0175302 · Cited by 61
Computational and Informatic Advances for Reproducible Data Analysis in Neuroimaging
Poldrack, R. A., Gorgolewski, K. J., & Varoquaux, G. P. (2019) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-072018-021237 · Cited by 61
Frone, M. R., & Blais, A.-R. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17030719 · Cited by 65
The use and misuse of statistical methodologies in pharmacology research
Marino, M. J. (2013) · Biochemical Pharmacology · doi:10.1016/j.bcp.2013.05.017 · Cited by 61
“In the field of genome wide association studies (GWAS) millions of genetic markers measured simultaneously leads to vanishingly small a values with any of the corrections for multiple comparisons describe above, a fact that likely attributes to low rates of reproducibility [45].”
BackgroundBüttner, F., Toomey, E., McClean, S., Roe, M., & Delahunt, E. (2020) · British Journal of Sports Medicine · doi:10.1136/bjsports-2019-101863 · Cited by 47
Tierney, B. T., Tan, Y., Kostic, A. D., & Patel, C. J. (2021) · Nature Communications · doi:10.1038/s41467-021-23029-8 · Cited by 65
“(Ioannidis 2008; Patel et al. 2015; Klau et al. 2020)” “Meta-analyses are emerging in the microbiome, and have been used to discover new microbiome-disease associations in colorectal cancer.”
MethodPolygenic risk scores for predicting outcomes and treatment response in psychiatry: hope or hype?
Fusar‐Poli, L., Rutten, B. P. F., van Os, J., Aguglia, E., & Gülöksüz, S. (2022) · International Review of Psychiatry · doi:10.1080/09540261.2022.2101352 · Cited by 58
“Additionally, most studies included in our review tended to emphasize nominally significant yet small effect sizes that were less likely to be clinically relevant (Ioannidis, 2008).”
BackgroundThe Weak Spots in Contemporary Science (and How to Fix Them)
Wicherts, J. M. (2017) · Animals · doi:10.3390/ani7120090 · Cited by 60
“Many of the issues of significance testing are caused by low power [4,5,53,123].”
BackgroundInfluentialMethodological and reporting quality in laboratory studies of human eating behavior
Robinson, E., Bevelander, K. E., Field, M., & Jones, A. (2018) · Appetite · doi:10.1016/j.appet.2018.02.008 · Cited by 59
“Alongside incentive structures in science to publish frequently and improper use of statistical analyses, sub-optimal methodological and reporting practices are likely to contribute to unreliable scientific findings (Ioannidis, 2005, 2008; Simmons, Nelson, & Simonsohn, 2011).”
Effect of Statin on Cancer Incidence: An Umbrella Systematic Review and Meta-Analysis
Jeong, G. H., Lee, K. H., Kim, J. Y., Eisenhut, M., Kronbichler, A., van der Vliet, H. J., Hong, S. H., Shin, J. I., & Gamerith, G. (2019) · Journal of Clinical Medicine · doi:10.3390/jcm8060819 · Cited by 58
“In addition, observational studies tend to have more biases than RCTs [76], and some reports suggest that the outcomes of observed associations could be false positives or inflated if a large between-study heterogeneity is present [77,78].”
Work Fatigue in a Non-Deployed Military Setting: Assessment, Prevalence, Predictors, and Outcomes
Frone, M. R., & Blais, A.-R. (2019) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph16162892 · Cited by 59
Mist, S. D., & Jones, K. D. (2018) · Pain Medicine · doi:10.1093/pm/pnx322 · Cited by 55
Neglect of publication bias compromises meta-analyses of educational research
Ropovik, I., Adamkovic, M., & Greger, D. (2021) · PLoS ONE · doi:10.1371/journal.pone.0252415 · Cited by 61
Wang, X., Zhang, Z., Liu, W., Xiong, Y., Sun, W., Huang, X., Jiang, Y., Ni, G., Sun, W., Zhou, L., et al. (2013) · Gene · doi:10.1016/j.gene.2013.05.083 · Cited by 55
“When the same effects were regarded as observation indicator, the posterity will not be able to replicate the result not representing the fact as previously stated reasons (Ioannidis, 2008).”
BackgroundWarner, S. C., van Meurs, J. B. J., Schiphof, D., Bierma‐Zeinstra, S., Hofman, A., Uitterlinden, A. G., Richardson, H., Jenkins, W., Doherty, M., & Valdes, A. M. (2017) · European Journal of Human Genetics · doi:10.1038/ejhg.2016.196 · Cited by 57
Four simple ways to increase power without increasing the sample size
Lazic, S. E. (2018) · Laboratory Animals · doi:10.1177/0023677218767478 · Cited by 61
Kronenberg, F. (2019) · Clinical Research in Cardiology Supplements · doi:10.1007/s11789-019-00093-5 · Cited by 57
Pereira-Sánchez, V., Franco, A. R., Vieira, D. L., de Castro‐Manglano, P., Soutullo, C. A., Milham, M. P., & Castellanos, F. X. (2020) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2020.10.013 · Cited by 57
Tang, S., Li, T., Fang, J., Chen, R., Cha, Y., Wang, Y., Zhu, M., Zhang, Y., Chen, Y., Du, Y., et al. (2021) · Environment International · doi:10.1016/j.envint.2021.106866 · Cited by 56
Systematic review and meta-analysis of genetic studies of late-life depression
Tsang, R. S. M., Mather, K. A., Sachdev, P. S., & Reppermund, S. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.01.028 · Cited by 54
“…and mediates dopaminergic transmission (Nestler and Carlezon, 2006) CNTF 11q12 Regulates neuronal survival and involved in injury responses (Ip and Yancopoulos, 1996) COMT 22q11 Mediates dopaminergic transmission (Chen et al., 2004) CRP 1q23 Mediates immune and injury responses (Du Clos, 2000)…”
Improving the rigor of psychophysiology research
Baldwin, S. A. (2016) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2016.04.006 · Cited by 57
Fornaro, M., Dragioti, E., Prisco, M. D., Billeci, M., Mondin, A. M., Calati, R., Smith, L., Hatcher, S., Kaluzienski, M., Fiedorowicz, J. G., et al. (2022) · BMC Medicine · doi:10.1186/s12916-022-02423-z · Cited by 64
“Specifically, significant associations claimed by the original observational studies, or their pooled synthesis may be susceptible to biases such as excess significance [11], publication bias, reporting bias, and residual confounding, leading to misleading or inflated estimates of these associations [12].”
BackgroundSchielzeth, H., Kempenaers, B., Ellegren, H., & Forstmeier, W. (2011) · Evolution · doi:10.1111/j.1558-5646.2011.01431.x · Cited by 54
Hein, R., Maranian, M. J., Hopper, J. L., Kapuscinski, M. K., Southey, M. C., Park, D. J., Schmidt, M. K., Broeks, A., Hogervorst, F. B. L., Bueno-de-Mesquita, H. B., et al. (2012) · PLoS ONE · doi:10.1371/journal.pone.0042380 · Cited by 60
“the suboptimal power of the pool of Asian studies (due to the small number of Asian individuals) together with the commonly used requirement for a published association to pass a certain predefined p-value threshold may have resulted in biased SNP effect estimates [10,11].”
BackgroundQuantifying Selective Reporting and the Proteus Phenomenon for Multiple Datasets with Similar Bias
Pfeiffer, T., Bertram, L., & Ioannidis, J. P. A. (2011) · PLoS ONE · doi:10.1371/journal.pone.0018362 · Cited by 49
“Reasons and additional examples for initially inflated findings and for the Proteus phenomenon have been discussed earlier [8,13,14].”
BackgroundSelf-citationBramon, E., Pirinen, M., Strange, A., Lin, K., Freeman, C., Bellenguez, C., Su, Z., Band, G., Pearson, R. D., Vukcevic, D., et al. (2013) · Biological Psychiatry · doi:10.1016/j.biopsych.2013.03.033 · Cited by 61
“CEU, Utah residents (CEP association is a true positive but heterogeneous and/or subject to a “winner’s curse” effect (114,115) or because the association is a false positive and appears heterogeneous because of a relatively strong discovery signal created by chance effects.”
BackgroundWeintraub, M. J., Hall, D. L., Carbonella, J. Y., de Mamani, A. W., & Hooley, J. M. (2016) · Family Process · doi:10.1111/famp.12208 · Cited by 61
“Replication of a particular phenomenon helps to bolster the credibility of that finding, as repeatedly obtaining positive results decreases the chances of large “file-drawer” effects or failed attempts to replicate (Ioannidis, 2008).”
CORP: Minimizing the chances of false positives and false negatives
Curran‐Everett, D. (2016) · Journal of Applied Physiology · doi:10.1152/japplphysiol.00937.2016 · Cited by 54
“In contrast, power—with its inherent connection to the statistical benchmark of —does impact the reproducibility of point and interval estimates(4) of the magnitude of some biologic effect (18).”
BackgroundInfluentialCan the behavioral sciences self-correct? A social epistemic study
Romero, F. (2016) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2016.10.002 · Cited by 57
Genome-wide association study for claw disorders and trimming status in dairy cattle
van der Spek, D., van Arendonk, J. A. M., & Bovenhuis, H. (2014) · Journal of Dairy Science · doi:10.3168/jds.2014-8302 · Cited by 53
Magnetic Resonance Spectroscopy in Patients with Insomnia: A Repeated Measurement Study
Spiegelhalder, K., Regen, W., Nissen, C., Feige, B., Baglioni, C., Riemann, D. W., Hennig, J., & Lange, T. (2016) · PLoS ONE · doi:10.1371/journal.pone.0156771 · Cited by 53
“In addition, newly discovered effects often provide inflated effect sizes compared to true effect sizes [57].”
BackgroundTLR7 dosage polymorphism shapes interferogenesis and HIV-1 acute viremia in women
Azar, P., Mejía, J. E., Cénac, C., Shaiykova, A., Youness, A., Laffont, S., Essat, A., Izopet, J., Passaes, C., Müller-Trutwin, M. C., et al. (2020) · JCI Insight · doi:10.1172/jci.insight.136047 · Cited by 57
Nordgård, R., & Låg, T. (2021) · Frontiers in Virtual Reality · doi:10.3389/frvir.2021.699383 · Cited by 46
“This tendency leads to an over-estimation of the summary effect sizes, in particular when the population of studies being sampled from is characterized by low statistical power (Ioannidis, 2008; Button et al., 2013).”
BackgroundRelationship between the Implicit Association Test and intergroup behavior: A meta-analysisPreprint
Kurdi, B., Seitchik, A. E., Axt, J. R., Carroll, T. J., Karapetyan, A., Kaushik, N., Tomezsko, D., Greenwald, A. G., & Banaji, M. R. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/582gh · Cited by 53
Pietschnig, J., Siegel, M., Eder, J. S. N., & Gittler, G. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02874 · Cited by 56
“Such observations have led some critical voices to assert that most effects that are reported in the empirical literature are inflated (Ioannidis, 2008).”
BackgroundRisk Factors for Ovarian Cancer: An Umbrella Review of the Literature
Whelan, E., Kalliala, I. E. J., Semertzidou, A., Raglan, O., Bowden, S. J., Kechagias, K. S., Markozannes, G., Cividini, S., McNeish, I. A., Marchesi, J. R., et al. (2022) · Cancers · doi:10.3390/cancers14112708 · Cited by 64
DEBATE-statistical analysis plans for observational studies
Hiemstra, B., Keus, F., Wetterslev, J., Gluud, C. N., & van der Horst, I. C. C. (2019) · BMC Medical Research Methodology · doi:10.1186/s12874-019-0879-5 · Cited by 55
Bahadoran, Z., Mirmiran, P., Kashfi, K., & Ghasemi, A. (2020) · Journal of the American Association for Laboratory Animal Science · doi:10.30802/aalas-jaalas-19-000139 · Cited by 58
Brodeur, A., Cook, N., Hartley, J., & Heyes, A. (2024) · Journal of Political Economy Microeconomics · doi:10.1086/730455 · Cited by 75
“A further limitation is that p -hacking and publication bias have been shown to lead to inflated effect sizes (Ioannidis 2008; Gelman and Carlin 2014), which our methods do not detect or quantify.”
Prophylactic lidocaine for myocardial infarction
Martí-Carvajal, A. J., Simancas‐Racines, D., Anand, V., & Bangdiwala, S. I. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008553.pub2 · Cited by 57
Lisboa, F., Bradley, M. J., Hueman, M. T., Schobel, S., Gaucher, B. J., Styrmisdottir, E. L., Potter, B. K., Forsberg, J. A., & Elster, E. A. (2016) · Surgery · doi:10.1016/j.surg.2016.10.011 · Cited by 53
“3) Previous tobacco use (%) 22 (39) 5 (29) Wound mechanism Crush injury (%) — 1 (6) Gunshot wound (%) 8 (14) 1 (6) Explosive blast (%) 48 (86) 15 (88) Extremity wounds 86 30 Soft-tissue injuries (%) 23 (27) 8 (27) Open fractures (%) 20 (23) 3 (10) Total amputations (%) 43 (50) 19 (63) Transhumeral amputation (%) 1 (2)…”
P-values – a chronic conundrum
Gao, J. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01051-6 · Cited by 57
Metadata
- Authors
- Ioannidis, J. P. A.
- Affiliations
- University of Ioannina (GR)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- Epidemiology
- Published
- 2008-09
- Volume
- 19
- Issue
- 5
- Pages
- 640-648
- ISSN
- 1044-3983 (ISSN-L)
- OpenAlex
- W2150215887
- Semantic Scholar
- 0373f18622b7b64e99c7e56340ab2011dbd00b5d
- Cited by
- 1,495
- References
- 109
- Citations by year
- 2026: 91 · 2025: 87 · 2024: 78 · 2023: 112 · 2022: 94 · 2021: 99 · 2020: 103 · 2019: 131 · 2018: 108 · 2017: 101 · 2016: 87 · 2015: 77 · 2014: 72 · 2013: 60 · 2012: 50
- FWCI
- 15.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- effect size inflation, statistical significance thresholds, underpowered studies, selective reporting, conflicts of interest, reverse bias, measurement error, replication
- MeSH terms
- Clinical Trials as Topic, Data Interpretation, Statistical, Humans, Sensitivity and Specificity, Models, Statistical, Linkage Disequilibrium, Molecular Epidemiology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.