Why Most Discovered True Associations Are Inflated
The DOI matches its Crossref record; 111 of the 111 references it cites were found in a registry or an index.
Abstract
Newly discovered true (non-null) associations often have inflated effects compared with the true effect sizes. I discuss here the main reasons for this inflation. First, theoretical considerations prove that when true discovery is claimed based on crossing a threshold of statistical significance and the discovery study is underpowered, the observed effects are expected to be inflated. This has been demonstrated in various fields ranging from early stopped clinical trials to genome-wide associations. Second, flexible analyses coupled with selective reporting may inflate the published discovered effects. The vibration ratio (the ratio of the largest vs. smallest effect on the same association approached with different analytic choices) can be very large. Third, effects may be inflated at the stage of interpretation due to diverse conflicts of interest. Discovered effects are not always inflated, and under some circumstances may be deflated-for example, in the setting of late discovery of associations in sequentially accumulated overpowered evidence, in some types of misclassification from measurement error, and in conflicts causing reverse biases. Finally, I discuss potential approaches to this problem. These include being cautious about newly discovered effect sizes, considering some rational down-adjustment, using analytical methods that correct for the anticipated inflation, ignoring the magnitude of the effect (if not necessary), conducting large studies in the discovery ph…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Given that initial effect sizes are often inflated 61 and that we sought power to detect a difference between two groups, we doubled this estimate and aimed to test ~28 participants in each group with some attrition expected.”
From Adults with autism overestimate the volatility of the sensory environment (2017) · cites this paper as [61]
In the citing paper
“While this may reflect exaggeration of original effect sizes or publication bias (Ioannidis 2008), some degree of inflation is expected even in its absence, as our selection process filtered for original experiments with significant differences.”
From Estimating the replicability of Brazilian biomedical science (2025) · cites this paper as Ioannidis 2008
In the citing paper
“Further, in situations when an underpowered study detects a true effect, the estimate of this effect size is likely to be highly exaggerated, a phenomenon often referred to as the “winner’s curse” (15).”
From Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies (2016) · cites this paper as [15]
In the citing paper
“Researchers have strong incentives to interpret their results as supporting or extending the Bayesian framework, rather than challenging it—leading to a literature that systematically over-represents supportive evidence (Ioannidis 2008; Nosek et al. 2012).”
From The myth of the Bayesian brain (2025) · cites this paper as Ioannidis 2008
In the citing paper
“This flexibility in the synthesis of the evidence provides the assessors with the opportunity to inflate or disinflate the efficacy of a given technology, a phenomenon that was observed in scientific publications.[24]”
From Health technology assessments conducted in health care facilities: A strategic practice? Findings from a content analysis of HTA reports (2017) · cites this paper as [24]
8 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationGene expression profiling predicts clinical outcome of breast cancer
van 't Veer, L. J., Dai, H., van de Vijver, M. J., He, Y. D., Hart, A. A. M., Mao, M., Peterse, H. L., van der Kooy, K., Marton, M. J., Witteveen, A. T., et al. (2002) · Nature · doi:10.1038/415530a · Cited by 9,536
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040297 · Cited by 8,613
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“This article will not deal with the debate on whether research findings are credible.(1) I will focus instead on the interesting subset of research findings that are true.”
BackgroundSelf-citationEffect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes
Nissen, S. E., & Wolski, K. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa072761 · Cited by 4,675
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., & Egger, M. (2007) · Epidemiology · doi:10.1097/ede.0b013e3181577511 · Cited by 2,779
Internal validation of predictive models
Steyerberg, E. W., Harrell, F. E., Borsboom, G. J. J. M., Eijkemans, M. J. C., Vergouwe, Y., & Habbema, J. D. F. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00341-9 · Cited by 2,746
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544
Cited by
Items 601–700 of 1,004
Erythropoiesis-stimulating agents for anemia in rheumatoid arthritis
Martí-Carvajal, A. J., Agreda-Pérez, L. H., Solà, I., & Simancas-Racines, D. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000332.pub3 · Cited by 31
Testing oncological treatments in the era of personalized medicine
Teira, D. (2016) · Cited by 27
Power and reproducibility in the external validation of brain-phenotype predictions
Rosenblatt, M., Tejavibulya, L., Sun, H., Camp, C., Khaitova, M., Adkinson, B. D., Jiang, R., Westwater, M. L., Noble, S., & Scheinost, D. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01931-7 · Cited by 31
“29 –32 for MAE).”
Ortloff, A.-M., Martius, F., Meier, M., Sans-Raimbault, T., Geierhaas, L., & Smith, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713671 · Cited by 30
“When considering only published results, as we do in our work, publication bias has an influence on the size of reported effects [58].”
Getting to Know Pain Effect Sizes—Guidelines for Effect Size and Sample Size in Global Pain Research
Zieliński, G. (2026) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2026.01.006 · Cited by 28
Prolonged storage of packed red blood cells for blood transfusion
Martí-Carvajal, A. J., Simancas‐Racines, D., & Peña-González, B. S. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009330.pub2 · Cited by 30
Reclassification of genetic-based risk predictions as GWAS data accumulate
Krier, J. B., Barfield, R., Green, R. C., & Kraft, P. L. (2016) · Genome Medicine · doi:10.1186/s13073-016-0272-5 · Cited by 26
Scientific Uncertainty in the Press: How Newspapers Describe Initial Biomedical Findings
Dumas-Mallet, E., Smith, A., Boraud, T., & Gonon, F. (2018) · Science Communication · doi:10.1177/1075547017752166 · Cited by 29
“Yet initial results are poorly replicated (Dumas-Mallet, Button, Boraud, Munafo, & Gonon, 2016; Ioannidis, 2005a, 2008), and journalists scarcely mention these invalidations (Dumas-Mallet et al., 2017).”
ResultA Proposal for Informative Default Priors Scaled by the Standard Error of Estimates
van Zwet, E. W., & Gelman, A. (2021) · The American Statistician · doi:10.1080/00031305.2021.1938225 · Cited by 17
“The relation between overestimation of the effect size and the SNR has been demonstrated through simulation (Gelman and Carlin 2014; Ioannidis 2008), and more recently, the following theorem has been established (van Zwet and Cator 2021).”
BackgroundInfluentialOn the Gap between Adoption and Understanding in NLP
Bianchi, F., & Hovy, D. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.340 · Cited by 35
“3896 ner’s curse” (Ioannidis, 2008) of unreplicable re-”
von Hippel, P. T. (2022) · Perspectives on Psychological Science · doi:10.1177/17456916211072525 · Cited by 30
“Some failed replication studies find no effect at all; others find an effect that, though larger than zero, is much smaller than the effect reported in the original study (Ioannidis, 2008).”
ResultThe physiological study of emotional piloerection: A systematic review and guide for future research
McPhetres, J., & Zickfeld, J. (2022) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2022.06.010 · Cited by 32
“These effects are very large — much larger than the average psychological effect (Funder and Ozer, 2019; Gignac and Szodorai, 2016) — and small samples inflate effect sizes (Button et al., 2013; Ioannidis, 2008; Rochefort-Maranda, 2017).”
Associations between airborne crude oil chemicals and symptom-based asthma
Lawrence, K. G., Niehoff, N. M., Keil, A. P., Jackson, W. B., Christenbury, K. E., Stewart, P. A., Stenzel, M., Huynh, T., Groth, C. P., Ramachandran, G., et al. (2022) · Environment International · doi:10.1016/j.envint.2022.107433 · Cited by 26
Brief critical review: Statistical assessment of biomarker performance
Kampfrath, T., & Levinson, S. S. (2013) · Clinica Chimica Acta · doi:10.1016/j.cca.2013.02.006 · Cited by 26
“Thus, in a large study, the p-value is more likely to be valid but will less ably depict discrimination, while in a small study a p-value may have more meaning in terms of discrimination but the sample may be a misrepresentation of the population which is why most preliminary studies show results that are inflated [20…”
ResultLikelihood ratios for genome medicine
Morgan, A. A., Chen, R., & Butte, A. J. (2010) · Genome Medicine · doi:10.1186/gm151 · Cited by 27
“Genetic studies that com bine a discovery for a significant association with disease with an estimate of associated risk are strongly biased to overestimate the level of risk [26].”
BackgroundPreregistration and reproducibility
Strømland, E. (2019) · Journal of Economic Psychology · doi:10.1016/j.joep.2019.01.006 · Cited by 27
Spalek, K., Loos, E., Schicktanz, N., Hartmann, F., de Quervain, D. J. ‐., Stier, C., & Milnik, A. (2019) · Neuropsychopharmacology · doi:10.1038/s41386-019-0362-3 · Cited by 27
Herrera‐Gómez, F., Chimeno, M. M., Martín-García, D., Lizaraso-Soto, F., Maurtua-Briseño-Meiggs, Á., Grande-Villoría, J., Bustamante‐Munguira, J., Alamartine, É., Vilardell, M., Sangrador, C. O., et al. (2019) · Scientific Reports · doi:10.1038/s41598-019-45431-5 · Cited by 26
Εvangelou, Ε., Maraganore, D. M., Annesi, G., Brighina, L., Brice, A., Elbaz, A., Ferrarese, C., Hadjigeorgiou, G. M., Krüger, R., Lambert, J., et al. (2009) · American Journal of Medical Genetics Part B Neuropsychiatric Genetics · doi:10.1002/ajmg.b.30980 · Cited by 27
“Even for genuine associations, it is expected that effect sizes may be inflated in the discovery (GWA) phase [Zollner and Pritchard, 2007] [Ioannidis, 2008b] and this may result in heterogeneity in the data when both the GWA and replication data are considered.”
BackgroundMethodInfluentialSelf-citationStatistical Power in Plant Pathology Research
Gent, D. H., Esker, P. D., & Kriss, A. B. (2017) · Phytopathology · doi:10.1094/phyto-03-17-0098-le · Cited by 28
Degen, P. M., & Medo, M. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011630 · Cited by 28
Anticipating consequences of sharing raw data and code and of awarding badges for sharing
Ioannidis, J. P. A. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.04.015 · Cited by 25
Self-citationMarino, M. J. (2017) · Biochemical Pharmacology · doi:10.1016/j.bcp.2017.12.011 · Cited by 27
Farver-Vestergaard, I., Danielsen, J. T., Løkke, A., & Zachariae, R. H. (2022) · Psychosomatic Medicine · doi:10.1097/psy.0000000000001043 · Cited by 26
“A possible explanation is the documented general tendency for inflation of early reported effect sizes due to a higher risk of publication bias, underpowered studies, and lower study quality in emergent research fields (97).”
ResultThe theatre of depression: a role for physical therapy
Varela, A. J., & Melvin, A. (2022) · Physiotherapy Theory and Practice · doi:10.1080/09593985.2022.2041136 · Cited by 27
“Antidepressants may provide modest short-term benefits and fall short of any long-term therapeutic effect (Ioannidis, 2008; Lam et al.).”
BackgroundPatterns of polysomnography parameters in 27 neuropsychiatric diseases: an umbrella review
Zhang, Y., Ren, R., Yang, L., Zhang, H., Shi, Y., Vitiello, M. V., Sanford, L. D., & Tang, X. (2022) · Psychological Medicine · doi:10.1017/s0033291722001581 · Cited by 26
“Specifically, meta-analyses are susceptible to reporting bias, publication bias, and residual confounding bias, and other types of problems which can result in inflated estimates (Ioannidis, 2008) or false positives (Ioannidis, 2005) for examined data parameters.”
BackgroundBiomarkers for Psychosis: Are We There Yet? Umbrella Review of 1478 Biomarkers
Fuentes‐Claramonte, P., Estradé, A., Solanes, A., Ramella‐Cravaro, V., García‐León, M. Á., de Diego-Adeliño, J., Molins, C., Fung, E., Valentí, M., Anmella, G., et al. (2024) · Schizophrenia Bulletin Open · doi:10.1093/schizbullopen/sgae018 · Cited by 28
“202 In our sample, 38.8% of all factors based on a single study reported statistically significant associations.”
BackgroundInfluentialBirds of a Feather: Applied Behavior Analysis and Quality of Life
Gambrill, E. D. (2012) · Research on Social Work Practice · doi:10.1177/1049731512465775 · Cited by 29
“Ioannidis (2005) argues that most biomedical research findings are false (see also Ioannidis, 2008).”
BackgroundDorn, R. A. V., Desmarais, S. L., Grimm, K. J., Tueller, S., Johnson, K. L., Sellers, B. G., & Swartz, M. S. (2016) · Psychiatry Research · doi:10.1016/j.psychres.2016.08.029 · Cited by 26
“Because of a longstanding focus on single-study designs that are frequently underpowered, it has been argued that the majority of published research findings are false (Ioannidis, 2005); and even ‘true’ associations are frequently inflated, which affects subsequent studies when powered to the original study (Ioannidis…”
Lewis, N. A., & Earl, A. N. (2018) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000183 · Cited by 29
Does Belief in Free Will Increase Support for Economic Inequality?
Mercier, B., Wiwad, D., Piff, P. K., Aknin, L. B., Robinson, A. R., & Shariff, A. (2020) · Collabra Psychology · doi:10.1525/collabra.303 · Cited by 32
Association between cardiorespiratory fitness and colorectal cancer in the UK Biobank
Hillreiner, A., Baumeister, S. E., Sedlmeier, A. M., Finger, J. D., Schlitt, H. J., & Leitzmann, M. F. (2019) · European Journal of Epidemiology · doi:10.1007/s10654-019-00575-6 · Cited by 26
Assmann, A., Richter, A., Schütze, H., Soch, J., Barman, A., Behnisch, G., Knopf, L., Raschick, M., Schult, A., Wüstenberg, T., et al. (2020) · European Journal of Neuroscience · doi:10.1111/ejn.14872 · Cited by 26
“Moreover, a common phenomenon in genetic association studies, but not limited to these, is that effect sizes tend to be relatively large in the initial reports and turn out to be smaller in replications and meta-analyses (Ioannidis, 2008).”
BackgroundCorella, D., Sabio, J. B. R., Coltell, Ó., Ortega‐Azorín, C., Estruch, R., Martínez‐González, M. Á., Salas‐Salvadó, J., Sorlí, J. V., Castañer, O., Arós, F., et al. (2018) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2017.09.025 · Cited by 27
Mapping brain asymmetry in health and disease through the ENIGMA consortiumPreprint
Kong, X., Postema, M. C., Guadalupe, T. M., de Kovel, C. G. F., Boedhoe, P. S. W., Hoogman, M., Mathias, S. R., van Rooij, D., Schijven, D., Glahn, D. C., et al. (2019) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/ufwhp · Cited by 25
Examining the predictive validity of NIH peer review scores
Lindner, M. D., & Nakamura, R. K. (2015) · PLOS ONE · doi:10.1371/journal.pone.0126938 · Cited by 24
In Search Of True Things Worth Knowing: Considerations For A New Article Prototype
Tullett, A. M. (2015) · Social and Personality Psychology Compass · doi:10.1111/spc3.12166 · Cited by 24
“It is also important for ameliorating the filedrawer problem, thus providing more accurate meta-analytic estimates of effects (Ioannidis, 2008; Rosenthal, 1979).”
BackgroundInfluentialGard, M., & Pluim, C. (2017) · Sport Education and Society · doi:10.1080/13573322.2016.1213716 · Cited by 26
“…researchers scrutinize these results and then attempt to replicate these initial findings, the ‘effect’ of the procedure, drug or intervention tends to diminish, often to nothing (see, e.g. Ioannidis’, 2008 article ‘Why most discovered true associations are inflated’ in the journal Epidemiology).”
BackgroundLangue de publication et performance en recherche
Imbeau, L., & Ouimet, M. (2013) · Politique et Sociétés · doi:10.7202/1014959ar · Cited by 24
The Collaborative Roots of Corruption? A Replication of Weisel & Shalvi (2015)
Wouda, J., Bijlstra, G., Frankenhuis, W. E., & Wigboldus, D. H. J. (2017) · Collabra Psychology · doi:10.1525/collabra.97 · Cited by 24
“Research shows that published effect sizes tend to overestimate true effect sizes, and such overestimation tends to be greater in pioneering studies that are the first to report an effect, a ‘decline effect’ (Anderson, Kelley, & Maxwell, 2017; Ioannidis, 2008; Simonsohn, 2015).”
BackgroundPeer-Review Guidelines Promoting Replicability and Transparency in Psychological Science
Davis, W. E., Giner‐Sorolla, R., Lindsay, D. S., Lougheed, J. P., Makel, M. C., Meier, M. E., Sun, J., Vaughn, L. A., & Zelenski, J. M. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918806489 · Cited by 24
“Low sensitivity raises the likelihood that a significant result is a false positive, especially when the finding is unlikely (Ioannidis, 2005, 2008; Zöllner & Pritchard, 2007).”
BackgroundEmotional oddball: A review on memory effects
Schlüter, H., Hackländer, R. P. M., & Bermeitinger, C. (2019) · Psychonomic Bulletin & Review · doi:10.3758/s13423-019-01658-x · Cited by 25
“This small number of effect sizes is not very informative, and is likely to be inflated (e.g., Ioannidis, 2008).”
BackgroundNo association betweenCALHM1and risk for Alzheimer dementia in a Belgian population
Sleegers, K., Brouwers, N., Bettens, K., Engelborghs, S., Miegroet, H. V., Deyn, P. P. D., & Broeckhoven, C. V. (2009) · Human Mutation · doi:10.1002/humu.20990 · Cited by 25
“It is possible that the risk estimates of the original study are inflated compared to the true effect of the polymorphism [Ioannidis, 2008], which could implicate that the true effect is too small to be detected in our study population, but our data do not even reveal a trend towards association.”
ResultRodgers, E. M., & Isaza, D. F. G. (2022) · Journal of Thermal Biology · doi:10.1016/j.jtherbio.2022.103252 · Cited by 23
Εvangelou, Ε., Fellay, J., Colombo, S. F., Martínez‐Picado, J., Obel, N., Goldstein, D. B., Telenti, A., & Ioannidis, J. P. A. (2011) · American Journal of Epidemiology · doi:10.1093/aje/kwr024 · Cited by 24
Self-citationGambrill, E. D. (2016) · Journal of Social Work Education · doi:10.1080/10437797.2016.1174642 · Cited by 29
Swaen, G. M. H., Langendam, M. W., Weyler, J., Burger, H., Siesling, S., Atsma, W. J., & Bouter, L. M. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.02.010 · Cited by 26
The Relationship Between Immorality and Cleansing: A Meta-Analysis of the Macbeth Effect
Siev, J., Zuckerman, S. E., & Siev, J. J. (2018) · Social Psychology · doi:10.1027/1864-9335/a000349 · Cited by 24
“As a result of this bias, in combination with the likelihood that initial studies have small samples, initial published studies are likely to have particularly large effect sizes (e.g., Ioannidis, 2008; Levine, Asada, & Carpenter, 2009).”
ResultDark triad and impulsivity – an ecological momentary assessment approach
Malesza, M., & Kalinowski, K. (2019) · Current Psychology · doi:10.1007/s12144-019-00320-y · Cited by 23
“However, given the small sample size, results of this study should be interpreted with caution as the small and homogenous sample may have led to an overestimation of effect sizes (Ioannidis 2008).”
Adily, A., Albalawi, O., Kariminia, A., Wand, H. C., Chowdhury, N. Z., Allnutt, S. H., Schofield, P. W., Sara, G. E., Ogloff, J. R. P., O’Driscoll, C., et al. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.1255 · Cited by 24
Community-Augmented Meta-Analyses (CAMAs) in Psychology: Potentials and Current Systems
Burgard, T., Bošnjak, M., & Studtrucker, R. (2021) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000431 · Cited by 24
Flaws in animal studies exploring statins and impact on meta‐analysis
Moja, L., Pecoraro, V. L., Ciccolallo, L., Dall’Olmo, L., Virgili, G., & Garattini, S. K. (2014) · European Journal of Clinical Investigation · doi:10.1111/eci.12264 · Cited by 26
Carbon black and lung cancer—testing a novel exposure metric by multi‐model inference
Morfeld, P., & McCunney, R. J. (2009) · American Journal of Industrial Medicine · doi:10.1002/ajim.20754 · Cited by 23
Planning and Executing a Genome Wide Association Study (GWAS)
Sale, M. M., Mychaleckyj, J. C., & Chen, W. (2009) · Methods in molecular biology · doi:10.1007/978-1-60327-378-7_25 · Cited by 24
“Additionally, the initial effect estimates are likely to be inflated due to the phenomenon known as the ‘‘winner’s curse’’ (55, 56).”
Rigor and replication: Toward improved best practices in human electrophysiology research
Larson, M. J., & Moser, J. S. (2016) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2016.12.001 · Cited by 25
Neurocognitive Phenotypes and Genetic Dissection of Disorders of Brain and Behavior
Congdon, E., Poldrack, R. A., & Freimer, N. B. (2010) · Neuron · doi:10.1016/j.neuron.2010.10.007 · Cited by 22
Maternal adiposity and perinatal and offspring outcomes: an umbrella review
Yang, Z., Feng, G., Gao, X., Yan, X., Li, Y., Wang, Y., Li, S., Jiang, Y., Zhao, S., Zhao, H., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01994-6 · Cited by 22
“Further, the results from individual primary studies may be skewed by residual confounding, excess significance bias and selective reporting, potentially exaggerating the observed effect sizes 16–18 .”
Balafoutas, L., Celse, J., Karakostas, A., & Umashev, N. (2024) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2024.102327 · Cited by 24
Ramachandran, A. K., Pedley, J. S., Moeskops, S., Oliver, J. L., Myer, G. D., Hsiao, H.-I., & Lloyd, R. S. (2025) · Sports Medicine · doi:10.1007/s40279-025-02190-w · Cited by 26
Enhancing and Accelerating Social Science Via Automation: Challenges and Opportunities
Yarkoni, T., Eckles, D., Heathers, J. A. J., Levenstein, M. C., Smaldino, P. E., & Lane, J. I. (2021) · Harvard Data Science Review · doi:10.1162/99608f92.df2262f5 · Cited by 26
van Witteloostuijn, A. C., Vanderstraeten, J., Slabbinck, H., Dejardin, M., Hermans, J., & Coreynen, W. (2022) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2022.100269 · Cited by 26
Simancas-Racines, D., Franco, J. V., Guerra, C. V., Felix, M. L., Hidalgo, R., & Martinez-Zapata, M. J. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002190.pub5 · Cited by 22
Imaginary alternatives: The effects of mental simulation on powerless negotiators.
Schaerer, M., Schweinsberg, M., & Swaab, R. I. (2018) · Journal of Personality and Social Psychology · doi:10.1037/pspi0000129 · Cited by 23
Statistical analysis of longitudinal data on tumour growth in mice experiments
Zavrakidis, I., Jóźwiak, K., & Hauptmann, M. (2020) · Scientific Reports · doi:10.1038/s41598-020-65767-7 · Cited by 23
Changing the logic of replication: A case from infant studies
Margoni, F., & Shepperd, M. (2020) · Infant Behavior and Development · doi:10.1016/j.infbeh.2020.101483 · Cited by 22
Addressing Exaggeration of Effects from Single RCTs
van Zwet, E. W., Schwab, S., & Greenland, S. (2021) · Significance · doi:10.1111/1740-9713.01587 · Cited by 22
Lack of association between the 5-HTTLPR and the error-related negativity (ERN)
Olvet, D. M., Hatchwell, E., & Hajcak, G. (2010) · Biological Psychology · doi:10.1016/j.biopsycho.2010.09.012 · Cited by 22
NIH Peer Review: Scored Review Criteria and Overall Impact
Lindner, M. D., Vancea, A., Chen, M., & Chacko, G. W. (2015) · American Journal of Evaluation · doi:10.1177/1098214015582049 · Cited by 23
“They contribute to high rates of false positive results, inflated effect sizes, and poor reproducibility (Ioannidis, 2005a, 2005b, 2008; Young, Ioannidis, & Al-Ubaydli, 2008) and there is increasing evidence that many of the findings published in the literature are not reproducible (Begley & Ellis,…”
BackgroundVibration of effects in epidemiologic studies of alcohol consumption and breast cancer risk
Chu, L., Ioannidis, J. P. A., Egilman, A. C., Vasiliou, V., Ross, J. S., & Wallach, J. D. (2020) · International Journal of Epidemiology · doi:10.1093/ije/dyz271 · Cited by 24
Self-citationUsing collaboration networks to identify authorship dependence in meta‐analysis results
Moulin, T. C., & Amaral, O. B. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1430 · Cited by 25
Weber, E. J., & Hoo, Z. H. (2018) · Emergency Medicine Journal · doi:10.1136/emermed-2018-207763 · Cited by 22
Steyerberg, E. W., & CALSTER, B. V. (2020) · European Journal of Clinical Investigation · doi:10.1111/eci.13229 · Cited by 22
Götz, M., Sarma, A., & O’Boyle, E. H. (2024) · International Journal of Psychology · doi:10.1002/ijop.13229 · Cited by 22
“Most (social) scientists address research questions with one or a small number of statistical analyses to derive empirical results (e.g., Del Giudice & Gangestad, 2021; Ioannidis, 2008; Simonsohn et al., 2020).”
Statistical Analysis in Preclinical Biomedical Research
Marino, M. J. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-804725-5.00003-3 · Cited by 28
An Introduction to Calculating Bayes Factors in JASP for Speech, Language, and Hearing Research
Brydges, C. R., & Gaeta, L. (2019) · Journal of Speech Language and Hearing Research · doi:10.1044/2019_jslhr-h-19-0183 · Cited by 22
Lavigne, J. V. (2016) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsv161 · Cited by 21
An Overview of Bootstrapping Method Applicable to Survey Researches in Rehabilitation Science
Choi, B. (2016) · Physical Therapy Korea · doi:10.12674/ptk.2016.23.2.093 · Cited by 21
Greber, M., Rogenmoser, L., Elmer, S., & Jäncke, L. (2018) · eNeuro · doi:10.1523/eneuro.0333-18.2018 · Cited by 22
“Priors were not based on the effect sizes reported in the original study because small samples often result in inflated effect size estimates (Ioannidis, 2008; Button et al., 2013; Halsey et al., 2015).”
MethodMunkholm, K., Faurholt‐Jepsen, M., Ioannidis, J. P. A., & Hemkens, L. G. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.12.002 · Cited by 24
Self-citationSmalheiser, N. R., Graetz, E. E., Yu, Z., & Wang, J. (2021) · PLoS ONE · doi:10.1371/journal.pone.0243668 · Cited by 24
Iverson, G. L., Büttner, F., & Caccese, J. B. (2021) · Frontiers in Neurology · doi:10.3389/fneur.2021.727089 · Cited by 22
Martí-Carvajal, A. J., Comunián-Carrasco, G., & Peña-Martí, G. E. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008577.pub2 · Cited by 23
Jiang, W., & Yu, W. (2016) · BMC Genomics · doi:10.1186/s12864-015-2296-4 · Cited by 21
A Blood Test for Alzheimer’s Disease: Progress, Challenges, and Recommendations
Kiddle, S. J., Voyle, N., & Dobson, R. (2018) · Journal of Alzheimer s Disease · doi:10.3233/jad-179904 · Cited by 23
“Other recommendations from John Ioannidis regard institutional changes affecting the incentives for scientists which would be even harder to achieve.”
InfluentialA default prior for regression coefficients
van Zwet, E. W. (2018) · Statistical Methods in Medical Research · doi:10.1177/0962280218817792 · Cited by 24
Identification of and Correction for Publication Bias
Andrews, I., & Kasy, M. (2017) · National Bureau of Economic Research · doi:10.3386/w23298 · Cited by 33
“One leading explanation for replication failure is publication bias (cf. Ioannidis, 2005, 2008; McCrary et al., 2016; Christensen and Miguel, 2016).”
Risk factors for suicidality among college students: A systematic review and meta-analysis
Zhao, M., Xin, Y., Bai, X., Zhang, S., Liu, H., Xu, W., Duan, W., Jin, Q., Chen, Y., Luo, Y., et al. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.04.137 · Cited by 23
Carrión-Castillo, A., van Bergen, E., Vino, A., van Zuijen, T. L., de Jong, P. F., Francks, C., & Fisher, S. E. (2016) · Genes Brain & Behavior · doi:10.1111/gbb.12299 · Cited by 20
“Nevertheless, it is likely that the effect sizes in the GWAS studies where the associations were first reported are overestimates of any real effects, due to the winner’s curse (Ioannidis 2008), and hence our power to detect these effects may be only roughly 60%.”
BackgroundRisk of Dementia in Cancer Survivors: A Meta-Analysis of Population-Based Cohort Studies
Zhang, D., Ou, Y., Fu, Y., Wang, Z., Huang, L., Tan, L., & Yu, J. (2022) · Journal of Alzheimer s Disease · doi:10.3233/jad-220436 · Cited by 21
Open Science Standards at Journals that Inform Evidence-Based Policy
Grant, S., Mayo‐Wilson, E., Kianersi, S., Naaman, K., & Henschel, B. (2023) · Prevention Science · doi:10.1007/s11121-023-01543-z · Cited by 22
“Unfortunately, journals often publish articles with results that are false, exaggerated, and not replicable (Baker, 2016; Ioannidis, 2005, 2008).”
BackgroundCatalogue of bias: novelty bias
Luo, Y., Heneghan, C. J., & Persaud, N. (2023) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2022-112215 · Cited by 20
Brain morphology predicts individual sensitivity to pain: a multicenter machine learning approach
Kotikalapudi, R., Kincses, B., Zunhammer, M., Schlitt, F., Asan, L., Schmidt‐Wilcke, T., Kincses, Z. T., Bingel, U., & Spisák, T. (2023) · Pain · doi:10.1097/j.pain.0000000000002958 · Cited by 21
“39,95 Moreover, the model training procedure proved to be highly stable for hyperparameters (regularization) and model coefficients (predictive weights) when generalizing to the heterogeneous left-out data set (different scanners, sequences, research staff, etc).”
Examining the replicability of online experiments selected by a decision market
Holzmeister, F., Johannesson, M., Camerer, C. F., Chen, Y., Ho, T., Hoogeveen, S., Huber, J., Imai, N., Imai, T., Jin, L. J., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02062-9 · Cited by 26
“But also true positive findings can be expected to have exaggerated effect sizes in the published literature owing to a lack of statistical power 18,19 .”
InfluentialRASSF1A polymorphism in familial breast cancer
Bergqvist, J., Latif, A., Roberts, S. A., Hadfield, K. D., Lalloo, F. I., Howell, A., Evans, D. G., & Newman, W. G. (2010) · Familial Cancer · doi:10.1007/s10689-010-9335-8 · Cited by 20
“This suggests that the association of the magnitude reported is not replicated rather than being underpowered to establish a smaller effect, which is often seen in replication studies [11].”
Breen, M. S., Stein, D. J., & Baldwin, D. S. (2016) · Human Psychopharmacology Clinical and Experimental · doi:10.1002/hup.2546 · Cited by 22
Gaigher, A., Burri, R., San‐Jose, L. M., Roulin, A., & Fumagalli, L. (2019) · Molecular Ecology · doi:10.1111/mec.15276 · Cited by 23
“The outcome of underpowered studies is more likely to be biased owing to the use of different analytical tools across studies, resulting in the so‐called vibration of effects (i.e., wide variation in the estimates of an effect across studies; Ioannidis, 2008).”
BackgroundTagne, A. M., Cassina, N., Furgiuele, A., Storelli, E., Cosentino, M., & Marino, F. (2020) · Journal of Academic Ethics · doi:10.1007/s10805-020-09359-0 · Cited by 20
Paphawasit, B., & Wudhikarn, R. (2022) · Administrative Sciences · doi:10.3390/admsci12020071 · Cited by 21
Chen, Y., Huang, H., Liu, S., Pan, L.-A., Zhou, B., Zhang, L., & Zeng, Z. (2011) · Clinical Biochemistry · doi:10.1016/j.clinbiochem.2011.06.010 · Cited by 20
“The differences of effect size between the initial result and the following replication result have been reported in many association studies [19].”
ResultThe DRD4 Exon III VNTR, Bupropion, and Associations With Prospective Abstinence
Bergen, A. W., Javitz, H. S., Su, L., He, Y., Conti, D. V., Benowitz, N. L., Tyndale, R. F., Lerman, C. E., & Swan, G. E. (2012) · Nicotine & Tobacco Research · doi:10.1093/ntr/nts245 · Cited by 21
Metadata
- Authors
- Ioannidis, J. P. A.
- Affiliations
- University of Ioannina (GR)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- Epidemiology
- Published
- 2008-09
- Volume
- 19
- Issue
- 5
- Pages
- 640-648
- ISSN
- 1044-3983 (ISSN-L)
- OpenAlex
- W2150215887
- Semantic Scholar
- 0373f18622b7b64e99c7e56340ab2011dbd00b5d
- Cited by
- 1,495
- References
- 109
- Citations by year
- 2026: 91 · 2025: 87 · 2024: 78 · 2023: 112 · 2022: 94 · 2021: 99 · 2020: 103 · 2019: 131 · 2018: 108 · 2017: 101 · 2016: 87 · 2015: 77 · 2014: 72 · 2013: 60 · 2012: 50
- FWCI
- 15.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- effect size inflation, statistical significance thresholds, underpowered studies, selective reporting, conflicts of interest, reverse bias, measurement error, replication
- MeSH terms
- Clinical Trials as Topic, Data Interpretation, Statistical, Humans, Sensitivity and Specificity, Models, Statistical, Linkage Disequilibrium, Molecular Epidemiology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.