Why Most Discovered True Associations Are Inflated
The DOI matches its Crossref record; 111 of the 111 references it cites were found in a registry or an index.
Abstract
Newly discovered true (non-null) associations often have inflated effects compared with the true effect sizes. I discuss here the main reasons for this inflation. First, theoretical considerations prove that when true discovery is claimed based on crossing a threshold of statistical significance and the discovery study is underpowered, the observed effects are expected to be inflated. This has been demonstrated in various fields ranging from early stopped clinical trials to genome-wide associations. Second, flexible analyses coupled with selective reporting may inflate the published discovered effects. The vibration ratio (the ratio of the largest vs. smallest effect on the same association approached with different analytic choices) can be very large. Third, effects may be inflated at the stage of interpretation due to diverse conflicts of interest. Discovered effects are not always inflated, and under some circumstances may be deflated-for example, in the setting of late discovery of associations in sequentially accumulated overpowered evidence, in some types of misclassification from measurement error, and in conflicts causing reverse biases. Finally, I discuss potential approaches to this problem. These include being cautious about newly discovered effect sizes, considering some rational down-adjustment, using analytical methods that correct for the anticipated inflation, ignoring the magnitude of the effect (if not necessary), conducting large studies in the discovery ph…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Given that initial effect sizes are often inflated 61 and that we sought power to detect a difference between two groups, we doubled this estimate and aimed to test ~28 participants in each group with some attrition expected.”
From Adults with autism overestimate the volatility of the sensory environment (2017) · cites this paper as [61]
In the citing paper
“While this may reflect exaggeration of original effect sizes or publication bias (Ioannidis 2008), some degree of inflation is expected even in its absence, as our selection process filtered for original experiments with significant differences.”
From Estimating the replicability of Brazilian biomedical science (2025) · cites this paper as Ioannidis 2008
In the citing paper
“Further, in situations when an underpowered study detects a true effect, the estimate of this effect size is likely to be highly exaggerated, a phenomenon often referred to as the “winner’s curse” (15).”
From Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies (2016) · cites this paper as [15]
In the citing paper
“Researchers have strong incentives to interpret their results as supporting or extending the Bayesian framework, rather than challenging it—leading to a literature that systematically over-represents supportive evidence (Ioannidis 2008; Nosek et al. 2012).”
From The myth of the Bayesian brain (2025) · cites this paper as Ioannidis 2008
In the citing paper
“This flexibility in the synthesis of the evidence provides the assessors with the opportunity to inflate or disinflate the efficacy of a given technology, a phenomenon that was observed in scientific publications.[24]”
From Health technology assessments conducted in health care facilities: A strategic practice? Findings from a content analysis of HTA reports (2017) · cites this paper as [24]
8 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationGene expression profiling predicts clinical outcome of breast cancer
van 't Veer, L. J., Dai, H., van de Vijver, M. J., He, Y. D., Hart, A. A. M., Mao, M., Peterse, H. L., van der Kooy, K., Marton, M. J., Witteveen, A. T., et al. (2002) · Nature · doi:10.1038/415530a · Cited by 9,536
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040297 · Cited by 8,613
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“This article will not deal with the debate on whether research findings are credible.(1) I will focus instead on the interesting subset of research findings that are true.”
BackgroundSelf-citationEffect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes
Nissen, S. E., & Wolski, K. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa072761 · Cited by 4,675
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., & Egger, M. (2007) · Epidemiology · doi:10.1097/ede.0b013e3181577511 · Cited by 2,779
Internal validation of predictive models
Steyerberg, E. W., Harrell, F. E., Borsboom, G. J. J. M., Eijkemans, M. J. C., Vergouwe, Y., & Habbema, J. D. F. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00341-9 · Cited by 2,746
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544
Cited by
Items 501–600 of 1,004
Protzko, J., & Schooler, J. W. (2017) · Wiley eBooks · doi:10.1002/9781119095910.ch6 · Cited by 40
Quintana, D. S. (2020) · Comprehensive Psychoneuroendocrinology · doi:10.1016/j.cpnec.2020.100014 · Cited by 39
“14, and that it is likely that these effects are inflated due to publication bias [12], specifying effects less than δ 1⁄4 0.”
BackgroundCannabidiol (CBD) Products for Pain: Ineffective, Expensive, and With Potential Harms
Moore, A., Straube, S., Fisher, E., & Eccleston, C. (2023) · Journal of Pain · doi:10.1016/j.jpain.2023.10.009 · Cited by 42
A failed attempt to conduct an individual patient data meta-analysis
Jaspers, G. J., & Degraeuwe, P. L. J. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-97 · Cited by 36
“This argument reminded us of a paper of Ioannidis wherein the author states that databases can be used as ‘private goldmines not to be shared’ or as ‘public commodity’ [8].”
BackgroundThe Same Old New Look: Publication Bias in a Study of Wishful Seeing
Francis, G. (2012) · i-Perception · doi:10.1068/i0519ic · Cited by 35
Dimond, D., Ishaque, A. H., Chenji, S., Mah, D., Chen, Z., Seres, P., Beaulieu, C., & Kalra, S. (2016) · Human Brain Mapping · doi:10.1002/hbm.23452 · Cited by 38
Enteral lactoferrin to prevent infection for very preterm infants: the ELFIN RCT
Griffiths, J., Jenkins, P. A., Vargova, M., Bowler, U., Juszczak, E., King, A., Linsell, L., Murray, D. R. P., Partlett, C., Patel, M., et al. (2018) · Health Technology Assessment · doi:10.3310/hta22740 · Cited by 37
Conceptual and methodological flaws undermine claims of a link between the gut microbiome and autism
Mitchell, K. J., Dahly, D. L., & Bishop, D. (2025) · Neuron · doi:10.1016/j.neuron.2025.10.006 · Cited by 35
Meta-analyses triggered by previous (false-)significant findings: problems and solutions
Schuit, E., Roes, K. C. B., Mol, B. W., Kwee, A., Moons, K. G. M., & Groenwold, R. H. H. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0048-9 · Cited by 36
The Replication Crisis in Epidemiology: Snowball, Snow Job, or Winter Solstice?
Lash, T. L., Collin, L. J., & Dyke, M. E. V. (2018) · Current Epidemiology Reports · doi:10.1007/s40471-018-0148-x · Cited by 39
“The selection pressures do not apply to the replication studies, so their results should be expected to regress towards the null [19].”
Improving environmental risk assessments of chemicals: Steps towards evidence-based ecotoxicology
Martin, O., Adams, J. E., Beasley, A. M., Belanger, S. E., Breton, R. L., Brock, T. C. M., Buonsante, V. A., Galay‐Burgos, M., Green, J., Guiney, P. D., et al. (2019) · Environment International · doi:10.1016/j.envint.2019.04.053 · Cited by 37
“Nonetheless, others warn that various conflicts of interest, such as financial incentives or existing accepted paradigms, may work in the direction of censoring or diminishing effects (Ioannidis, 2008).”
Liu, W., Wang, T., Zhu, M., & Jin, G. (2023) · Nutrients · doi:10.3390/nu15061344 · Cited by 38
Serghiou, S., Patel, C. J., Tan, Y. Y., Koay, P., & Ioannidis, J. P. A. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.09.004 · Cited by 34
“Sometimes, analytical choices[36,37] may be aimed at facilitating the identification of as strong an effect as possible[39], leading to potentially inflated or widely different estimates[40].”
InfluentialSelf-citationZhang, J., Redden, D. T., McGwin, G., Callahan, L. F., Smith, E. A., Alarcón, G. S., Moreland, L. W., van der Heijde, D. M. F. M., Brown, E. E., Arnett, D. K., et al. (2010) · Arthritis & Rheumatism · doi:10.1002/art.27510 · Cited by 35
“tended to show an overinflated effect size (49), the”
Does neuroticism disrupt the psychological benefits of nostalgia? a meta-analytic test
Frankenbach, J., Wildschut, T., Juhl, J., & Sedikides, C. (2021) · European Journal of Personality · doi:10.1002/per.2276 · Cited by 34
“Meta-analyses may overestimate effects, because studies reporting small, non-significant effect sizes are less likely to be submitted to, and published by, scientific journals (Ioannidis, 2008).”
BackgroundMeta-Research Evidence for Evaluating Therapies
Fuller, J. (2018) · Philosophy of Science · doi:10.1086/699689 · Cited by 40
Schneider, J. W., Allum, N., Andersen, J. P., Petersen, M. B., Madsen, E. B., Mejlgaard, N., & Zachariae, R. H. (2024) · PLoS ONE · doi:10.1371/journal.pone.0304342 · Cited by 40
Guidelines for Translational Research in Heart Failure
Lara‐Pezzi, E., Menasché, P., Trouvin, J., Badimón, L., Ioannidis, J. P. A., Wu, J. C., Hill, J. A., Koch, W. J., Felice, A. F. D., Waele, P. D., et al. (2015) · Journal of Cardiovascular Translational Research · doi:10.1007/s12265-015-9606-8 · Cited by 36
“Moreover, when detected, estimates of false positives or false negatives are likely to be inflated in underpowered studies, even for genuine effects -[85].”
BackgroundSelf-citationKlau, S., Hoffmann, S., Patel, C. J., Ioannidis, J. P. A., & Boulesteix, A. (2020) · International Journal of Epidemiology · doi:10.1093/ije/dyaa164 · Cited by 39
“1 Model and sampling vibration We previously introduced the concept of vibration of effects to quantify the variability in results when studying an association of interest under a broad range of model specifications [7].”
BackgroundMethodResultInfluentialSelf-citationHancox, T. P. M., Skene, D. J., Dallmann, R., & Dunn, W. B. (2021) · Metabolites · doi:10.3390/metabo11050328 · Cited by 34
Brain‐wide inferiority and equivalence tests in fMRI group analyses: Selected applications
Gerchen, M. F., Kirsch, P., & Feld, G. B. (2021) · Human Brain Mapping · doi:10.1002/hbm.25664 · Cited by 36
“…since small sample sizes tend to overestimate effect sizes in clusters passing the significance threshold, which leads to unreasonable expectations towards what constitutes a meaningful effect in fMRI research (Button et al., 2013; Ioannidis, 2008; Lindquist & Mejia, 2015; Reddan et al., 2017).”
BackgroundZhao, H., Wang, M., Peng, X., Zhong, L. X., Liu, X., Shi, Y., Li, Y., Chen, Y., & Tang, S. (2023) · Annals of Translational Medicine · doi:10.21037/atm-22-6515 · Cited by 42
“Although many of the reported associations could be causal, they could also be flawed due to residual confounding, reporting bias, or other biases, which frequently over-estimate the magnitudes of the observed effects (15,16) .”
Negative Results: Why Do they Need to be Published?
Sandercock, P. A. (2011) · International Journal of Stroke · doi:10.1111/j.1747-4949.2011.00723.x · Cited by 34
Holtz, B., Murray, K. M. E., Hershey, D. S., Richman, J., Dunneback, J. K., Vyas, A. K., & Wood, M. A. (2017) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x17745470 · Cited by 36
“Additionally, while our effect sizes are somewhat larger than expected, they should be taken with extreme caution, as we believe that they are due to the small sample size and the homogeneity of the sample.(43) Due to technical difficulties, the app user data may have underestimated potential future benefit.”
BackgroundKong, X., ENIGMA Laterality Working Group, & Francks, C. (2020) · Human Brain Mapping · doi:10.1002/hbm.25154 · Cited by 35
Chioléro, A. (2013) · Epidemiology · doi:10.1097/ede.0b013e31829e46dc · Cited by 34
Platt, B., Sfärlea, A., Buhl, C., Löchner, J., Neumüller, J., Thomsen, L. A., Starman‐Wöhrle, K., Salemink, E., & Schulte‐Körne, G. (2021) · Child Psychiatry & Human Development · doi:10.1007/s10578-020-01105-2 · Cited by 34
Strengthening the reporting of genetic risk prediction studies (GRIPS): explanation and elaboration
Janssens, A. C. J., Ioannidis, J. P., Bedrosian, S., Boffetta, P., Dolan, S. M., Dowling, N., Fortier, I., Freedman, A. N., Grimshaw, J. M., Gulcher, J., et al. (2011) · European Journal of Human Genetics · doi:10.1038/ejhg.2011.27 · Cited by 34
“57 Explanation: Assessment of the risk model in the same population as that from which the model was derived generally leads to more positive conclusions than when the evaluation is conducted in an independent population”
ResultSelf-citationHow Routinely Collected Data for Randomized Trials Provide Long-term Randomized Real-World Evidence
Hemkens, L. G. (2018) · JAMA Network Open · doi:10.1001/jamanetworkopen.2018.6014 · Cited by 33
An analysis of the quality of experimental design and reliability of results in tribology research
Watson, M., Christoforou, P., Herrera, P., Preece, D., Carrell, J., Harmon, M., Krier, P., Lewis, S. E., Maiti, R., Skipper, W. A., et al. (2019) · Wear · doi:10.1016/j.wear.2018.12.028 · Cited by 34
“While the norm is to publish only positive results, a change with no overall benefit, or even a detriment to performance can appear effective even to well carried out systematic reviews [24, 25, 6].”
BackgroundP‐Hacking , P‐Curves , and the PSM –Performance Relationship: Is There Evidential Value?
Vogel, D., & Homberg, F. (2020) · Public Administration Review · doi:10.1111/puar.13273 · Cited by 37
Measuring affect dynamics: An empirical framework
Pirla, S., Taquet, M., & Quoidbach, J. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01829-0 · Cited by 42
“The current lack of evidence to guide sampling decisions might result in underpowered studies, leading to missed opportunities to discover true effects and inflated effect sizes of discovered effects (Ioannidis, 2008).”
BackgroundBeets, M. W., Weaver, R. G., Ioannidis, J. P. A., Pfledderer, C. D., Jones, A., von Klinggraeff, L., & Armstrong, B. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02184-7 · Cited by 35
“from meta‐analyses For the purpose of this study, we defined the vibration of effects (VoE) as the difference between the originally reported summary meta-analytical SMD and the re-estimated summary SMD restricting analyses to trials based upon the classifications as defined above [15].”
MethodSelf-citationGroen‐Blokhuis, M. M., Franić, S., van Beijsterveldt, C. E. M., de Geus, E. J. C., Bartels, M., Davies, G. E., Ehli, E. A., Xiao, X., Scheet, P. A., Althoff, R. R., et al. (2013) · American Journal of Medical Genetics Part B Neuropsychiatric Genetics · doi:10.1002/ajmg.b.32175 · Cited by 41
“In a context of low power in which claims of discovery are based on thresholds of statistical significance, newly discovered associations are prone to overestimate true effect sizes [Ioannidis, 2008].”
BackgroundPunishment sensitivity modulates the processing of negative feedback but not error-induced learning
Unger, K., Heintz, S., & Kray, J. (2012) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2012.00186 · Cited by 40
“In contrast to previous studies employing simple motor tasks, such as the Flankers and Go/No-Go task (Boksem et al., 2006, 2008; Amodio et al., 2008), we found no relation between punishment sensitivity and the Ne/ERN.”
BackgroundResultInfluentialAll in the Family: Work–Family Enrichment and Crossover Among Farm Couples
Sprung, J., & Jex, S. (2017) · Journal of Occupational Health Psychology · doi:10.1037/ocp0000033 · Cited by 23
“On a related note, because of the relatively small sample size, the current findings may potentially represent an overestimate of the actual population parameters (Ioannidis, 2008).”
BackgroundGenetic Polymorphisms of Infectious Diseases in Case-Control Studies
Pacheco, A. G., & Moraes, M. O. (2009) · Disease Markers · doi:10.1155/2009/685618 · Cited by 39
Ding, E., Deng, F., Fang, J., Liu, J., Yan, W., Yao, Q., Miao, K., Wang, Y., Sun, P., Li, C., et al. (2024) · Environmental Health Perspectives · doi:10.1289/ehp13864 · Cited by 36
Brysbaert, M., & Debeer, D. (2025) · Journal of Cognition · doi:10.5334/joc.409 · Cited by 40
No Association Between Polygenic Risk for Schizophrenia and Brain Volume in the General Population
der Auwera, S. V., Wittfeld, K., Homuth, G., Teumer, A., Hegenscheid, K., & Grabe, H. J. (2015) · Biological Psychiatry · doi:10.1016/j.biopsych.2015.02.038 · Cited by 34
Ego Depletion May Disappear by 2020
Vadillo, M. A. (2019) · Social Psychology · doi:10.1027/1864-9335/a000375 · Cited by 33
Kirlić, N., Colaizzi, J. M., Cosgrove, K. T., Cohen, Z. P., Yeh, H., Breslin, F. J., Morris, A. S., Aupperle, R. L., Singh, M. K., & Paulus, M. P. (2021) · Child Development · doi:10.1111/cdev.13578 · Cited by 32
“Furthermore, there are also findings that suggest that the strength of many previously identified neurodevelopmental associations stemming from smaller samples have been significantly inflated (Button et al., 2013; Ioannidis, 2008).”
BackgroundCorgnet, B., DECK, C. A., DeSantis, M., Hampton, K., & Kimbrough, E. O. (2019) · Management Science · doi:10.1287/mnsc.2022.4463 · Cited by 35
Scientific misconduct: the dark side of science
Carafoli, E. (2015) · RENDICONTI LINCEI · doi:10.1007/s12210-015-0415-4 · Cited by 38
“Science 342:1035–1039 Ioannidis JPA (2005) Why most published research findings are false.”
InfluentialMorfeld, P., & McCunney, R. J. (2010) · Journal of Occupational Medicine and Toxicology · doi:10.1186/1745-6673-5-23 · Cited by 34
“This method is especially valuable when observational studies are performed with a large number of subjects (high precision) to quantify small effects sizes (low risk) as often in genetic or environmental epidemiology [61].”
BackgroundAn Association Between ICP-Derived Data and Outcome in TBI Patients: The Role of Sample Size
Cabella, B. C. T., Donnelly, J., Cardim, D., Liu, X., Cabeleira, M. T., Smielewski, P. S., Haubrich, C., Hutchinson, P. J., Kim, D., & Czosnyka, M. (2016) · Neurocritical Care · doi:10.1007/s12028-016-0319-x · Cited by 32
Absolute and relative pitch processing in the human brain: neural and behavioral evidence
Leipold, S., Brauchli, C., Greber, M., & Jäncke, L. (2019) · Brain Structure and Function · doi:10.1007/s00429-019-01872-2 · Cited by 35
Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/umq8d · Cited by 37
Danielsen, A. S., Franconeri, L., Page, S., Myhre, A. E., Tornes, R. A., Kacelnik, O., & Bjørnholt, J. V. (2023) · BMC Infectious Diseases · doi:10.1186/s12879-023-08182-3 · Cited by 33
Limits to forecasting in personalized medicine: An overview
Ioannidis, J. P. A. (2009) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2009.05.003 · Cited by 35
Self-citationFunctional Analysis of Variance for Association Studies
Vsevolozhskaya, O. A., Zaykin, D. V., Greenwood, M., Wei, C., & Lu, Q. L. (2014) · PLoS ONE · doi:10.1371/journal.pone.0105074 · Cited by 33
“The traditional single-marker association studies can detect variants that explain a relatively large fraction of the phenotypic difference, however, the effect size of newly discovered variants is typically overestimated [4].”
BackgroundCunnington, M. S., Kay, C., Avery, P. J., Mayosi, B. M., Santibanez‐Koref, M., & Keavney, B. D. (2009) · BMC Medical Genetics · doi:10.1186/1471-2350-10-135 · Cited by 33
“It is a frequently observed phenomenon in genetic epidemiology that studies first reporting a novel association often show a more extreme odds ratio than subsequent replication studies [22]; if the effect size estimated by Wang et al is an upwardly biased estimate this might explain the negative association in our stu…”
Pharmacological interventions for treating heart failure in patients with Chagas cardiomyopathy
Martí-Carvajal, A. J., & Kwong, J. S. (2016) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009077.pub3 · Cited by 35
Pennington, C. R., Litchfield, D., McLatchie, N. M., & Heim, D. (2018) · European Journal of Social Psychology · doi:10.1002/ejsp.2540 · Cited by 33
“This may not be the best approach, however, because it is likely that some effect sizes reported in previous research are inflated due to small sample sizes or publication bias (Ioannidis, 2008; Nuijten, van Assen, Veldkamp, & Wicherts, 2015; Szucs & Ioannidis, 2017).”
ResultBracken, M. B. (2009) · Annals of Epidemiology · doi:10.1016/j.annepidem.2008.11.006 · Cited by 33
Gargano, J. W., Holzman, C., Senagore, P. K., Reuss, M. L., Pathak, D. R., Friderici, K. H., Jernigan, K. A., & Fisher, R. A. (2009) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2009.05.060 · Cited by 35
Schanze, D., Ekici, A. B., Gawlik, M., Pfuhlmann, B., Reis, A., & Stöber, G. (2011) · American Journal of Medical Genetics Part B Neuropsychiatric Genetics · doi:10.1002/ajmg.b.31156 · Cited by 30
One-year dietary supplementation with walnuts modifies exosomal miRNA in elderly subjects
Hazas, M. L. D. L., Gil‐Zamorano, J., Cofán, M., Mantilla-Escalante, D. C., García‐Ruiz, A., del Pozo‐Acebo, L., Pastor, Ó., Yáñez‐Mó, M., Mazzeo, C., Serra‐Mir, M., et al. (2020) · European Journal of Nutrition · doi:10.1007/s00394-020-02390-2 · Cited by 30
Rossetti, M. G., Patalay, P., Mackey, S., Allen, N. B., Batalla, A., Bellani, M., Chye, Y., Cousijn, J., Goudriaan, A. E., Hester, R., et al. (2021) · NeuroImage Clinical · doi:10.1016/j.nicl.2021.102636 · Cited by 33
“findings may have been false positive (Button et al., 2013; Ioannidis, 2008).”
Which Findings Should Be Published?
Frankel, A. P., & Kasy, M. (2022) · American Economic Journal Microeconomics · doi:10.1257/mic.20190133 · Cited by 55
Quantifying “Promising Trials Bias” in Randomized Controlled Trials in Education
Sims, S., Anders, J., Inglis, M., & Lortie‐Forgues, H. (2022) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2022.2090470 · Cited by 32
“would expect Type M errors to systematically inflate effect sizes, particularly when those studies have lower power (Ioannidis, 2008; Gelman & Carlin, 2014).”
Social Work Education and Avoidable Ignorance
Gambrill, E. D. (2014) · Journal of Social Work Education · doi:10.1080/10437797.2014.916937 · Cited by 42
“Ioannidis (2005) has argued that most published research findings are false and that most studies describing causal relationships cannot be trusted (Ioannidis, 2008).”
BackgroundTreatment for disseminated intravascular coagulation in patients with acute and chronic leukemia
Martí-Carvajal, A. J., Anand, V., & Solà, I. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008562.pub3 · Cited by 31
Associations of PER3 and RORA Circadian Gene Polymorphisms and Depressive Symptoms in Older Adults
Maglione, J. E., Nievergelt, C. M., Parimi, N., Evans, D. S., Ancoli‐Israel, S., Stone, K. L., Yaffe, K. S., Redline, S. S., & Tranah, G. J. (2015) · American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2015.03.002 · Cited by 31
Pedroza, C., Han, W., Truong, V. T. T., Green, C., & Tyson, J. E. (2015) · Statistical Methods in Medical Research · doi:10.1177/0962280215620828 · Cited by 32
Dumas-Mallet, E., Button, K. S., Boraud, T., Munafo, M. R., & Gonon, F. (2016) · PLoS ONE · doi:10.1371/journal.pone.0158064 · Cited by 31
“Likewise, using a Bayesian approach, Ioannidis (2008) tested the effect of formal statistical significance on the PLOS ONE | DOI:10.1371/journal.pone.0158064 June 23, 2016 16 / 20 credibility of 50 meta-analyses reporting a significant genetic association at p<0.05 [22].”
BackgroundMethodInfluentialConducting an Eyewitness Lineup: How the Research Got It Wrong
Gronlund, S. D., Mickes, L., Wixted, J. T., & Clark, S. E. (2015) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/bs.plm.2015.03.003 · Cited by 35
Veatch, O. J., Bauer, C. R., Keenan, B. T., Josyula, N. S., Mazzotti, D. R., Bagai, K., Malow, B. A., Robishaw, J. D., Pack, A. I., & Pendergrass, S. A. (2020) · BMC Medical Genomics · doi:10.1186/s12920-020-00755-4 · Cited by 29
Shinohara, K., Tajika, A., Imai, H., Takeshima, N., Hayasaka, Y., & Furukawa, T. A. (2015) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12502 · Cited by 29
“Academia has not remained passive and silent against these biases that seriously undermine its own credibility (11, 12).”
What Makes Everyday Scientific Reasoning So Challenging?
(2017) · Psychology of Learning and Motivation · doi:10.1016/bs.plm.2016.11.006 · Cited by 29
“Science Collaboration, 2015; Pashler & Harris, 2012)? Thousands of fMRI studies have errors in data analysis (Eklund, Nichols, & Knutsson, 2016), scientists are accused of “p-hacking” or fishing for publishable results (Ioannidis, 2008; Simmons, Nelson, & Simonsohn, 2011), and for many scientific concepts (like brain…”
BackgroundSrinivasan, A., Berkowitz, J. S., Friedrich, N. A., Kivelson, S., & Tatonetti, N. P. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.29418 · Cited by 37
The shrinking or disappearing observed treatment effect
Chuang‐Stein, C., & Kirby, S. (2014) · Pharmaceutical Statistics · doi:10.1002/pst.1633 · Cited by 31
Olfson, M., & Marcus, S. C. (2013) · Health Affairs · doi:10.1377/hlthaff.2012.1353 · Cited by 32
Evolution of the empirical and theoretical foundations of eyewitness identification reform
Clark, S. E., Moreland, M. B., & Gronlund, S. D. (2013) · Psychonomic Bulletin & Review · doi:10.3758/s13423-013-0516-y · Cited by 31
How Not to Fool Ourselves About Heterogeneity of Treatment Effects
von Hippel, P. T., & Schuetze, B. A. (2025) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459241304347 · Cited by 37
Neuhaus, A. H., Popescu, F. C., Rentzsch, J., & Gallinat, J. (2013) · Schizophrenia Bulletin · doi:10.1093/schbul/sbt151 · Cited by 29
Schumm, W. R. (2015) · Comprehensive Psychology · doi:10.2466/17.cp.4.24 · Cited by 32
“There are some who think that most early scientifi c fi ndings are incorrect or infl ated ( Ioannidis, 2005 , 2008 , 2012 ).”
BackgroundLadanie, A., Speich, B., Briel, M., Sclafani, F., Bucher, H. C., Agarwal, A., Ioannidis, J. P. A., Pereira, T., Kasenda, B., & Hemkens, L. G. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.05.033 · Cited by 32
Self-citationThe Genomic Architecture of Flowering Time Varies Across Space and Time in Mimulus guttatus
Monnahan, P. J., & Kelly, J. K. (2017) · Genetics · doi:10.1534/genetics.117.201483 · Cited by 29
Population-Wide Generalizability of Genome-Wide Discovered Associations
Ioannidis, J. P. A. (2009) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djp298 · Cited by 29
Self-citationSaarela, M., Gunasekara, S., & Kumarage, P. M. (2026) · Discover Computing · doi:10.1007/s10791-026-10015-1 · Cited by 35
“While publication bias analyses indicated robust findings (see Section 4.3 for details), indicated robust findings, the inverse relationship between sample size and effect size suggests potential small-study effects [104].”
InfluentialLipsey, M. W. (2020) · Criminology & Public Policy · doi:10.1111/1745-9133.12530 · Cited by 30
Recognising the heterogeneity of autism
Lord, C. (2019) · The Lancet Psychiatry · doi:10.1016/s2215-0366(19)30220-2 · Cited by 30
Luijken, K., Dekkers, O. M., Rosendaal, F. R., & Groenwold, R. H. H. (2022) · BMJ · doi:10.1136/bmj-2021-070113 · Cited by 31
Results of Observational Studies: Analysis of Findings from the Nurses’ Health Study
Tai, V., Grey, A. B., & Bolland, M. J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0110403 · Cited by 28
“The strength of associations reported in observational studies is often viewed as an indicator of the credibility of the association [8,69–71].”
BackgroundInterventions for preventing high altitude illness: Part 2. Less commonly-used drugs
Garay, A. G. G., Molano-Franco, D., Estrada, V. H. N., Martí-Carvajal, A. J., & Arévalo-Rodríguez, I. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012983 · Cited by 31
Connolly, H. L., Young, A. W., & Lewis, G. J. (2018) · Cognition & Emotion · doi:10.1080/02699931.2018.1535425 · Cited by 28
Who Should Do Replication Labor?
Romero, F. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918803619 · Cited by 29
“Also, when p values determine publication, published experiments with small samples tend to overestimate effect sizes (Button et al., 2013; Ioannidis, 2008)—and these overestimations could have been prevented with large samples.”
BackgroundLukas, C. A., Trevisi Fuentes, H., & Berking, M. (2019) · Internet Interventions · doi:10.1016/j.invent.2019.100250 · Cited by 27
Published correlational effect sizes in social and developmental psychology
Weinerová, J., Szűcs, D., & Ioannidis, J. P. A. (2022) · Royal Society Open Science · doi:10.1098/rsos.220311 · Cited by 28
“Nevertheless, field-specific published effect size distributions may also misrepresent true effect sizes because published effect sizes may be highly exaggerated [5,6].”
BackgroundSelf-citationReprint of "Methodological and reporting quality in laboratory studies of human eating behavior"
Robinson, E., Bevelander, K. E., Field, M., & Jones, A. (2018) · Appetite · doi:10.1016/j.appet.2018.06.037 · Cited by 29
Herrera‐Gómez, F., del Aguila, W., Pedregosa, A. T., Adler, M., Padilla-Berdugo, R., Maurtua-Briseño-Meiggs, Á., Pascual, J., Pascual, M. A., Segundo, D. S., Heidt, S., et al. (2018) · International Immunopharmacology · doi:10.1016/j.intimp.2018.10.028 · Cited by 33
Accumulation Bias in meta-analysis: the need to consider time in error control
ter Schure, J., & Grünwald, P. (2019) · F1000Research · doi:10.12688/f1000research.19375.1 · Cited by 40
“…that early studies are unreliable predictors of later replications in a study series (Roberts and Ker, 2015; Chalmers and Glasziou, 2016) (and see references 6-34 in Ioannidis (2008) and references 33-49 in Pereira and Ioannidis (2011)), which is also an indication of early study Accumulation Bias.”
BackgroundTowards a model of language neurobiology in early development
Sanchez–Alonso, S., & Aslin, R. Ν. (2021) · Brain and Language · doi:10.1016/j.bandl.2021.105047 · Cited by 30
“Finally, a related consideration is sample size, which will affect the reproducibility of the effect since small sample sizes often have large variability in estimates, which may lead to model overfitting (Gelman, 2010; Ioannidis, 2008; Yarkoni & Westfall, 2017).”
BackgroundLiao, Z., Fang, Z., Gou, S., Luo, Y., Liu, Y., He, Z., Li, X., Peng, Y., Fu, Z., Li, D., et al. (2022) · BMC Medicine · doi:10.1186/s12916-021-02229-5 · Cited by 30
Parry, D. (2023) · Media Psychology · doi:10.1080/15213269.2023.2286647 · Cited by 16
“Moreover, as Quintana (2022) notes, studies that cannot reliably detect a wide range of effect sizes are less likely to discover true effects and more likely to identify exaggerated effect sizes (Button et al., 2013; Ioannidis, 2008; Rochefort-Maranda, 2021).”
New developments in research methods
Ledgerwood, A. (2019) · eScholarship (California Digital Library) · Cited by 28
Evaluation of a meta-analysis of air quality and heart attacks, a case study
Young, S. S., & Kindzierski, W. B. (2019) · Critical Reviews in Toxicology · doi:10.1080/10408444.2019.1576587 · Cited by 28
“Collectively, efforts chasing after relatively weak risk factors like ambient air quality mostly lead to outcomes that vibrate between showing effects/no effects dependent upon how researchers design studies, use data, analyze and report results (called ‘vibration of effects’ after Ioannidis 2008).”
BackgroundMitra, S., Mehta, U. M., Bhaskarapillai, B., Venkatasubramanian, G., & Thirthalli, J. (2019) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2019.07.006 · Cited by 29
“Additionally, in pursuance of statistical significance, a study with a low power inflates the magnitude of the effect which it claims to be measuring (winner’s curse; Ioannidis, 2008).”
BackgroundDo Monetary Incentives Undermine Performance on Intrinsically Enjoyable Tasks? A Field Test
Esteves‐Sorenson, C., & Broce, R. (2022) · The Review of Economics and Statistics · doi:10.1162/rest_a_00947 · Cited by 40
“As a result, subsequent replications yield no or smaller effects (Ioannidis, 2008).”
Background
Metadata
- Authors
- Ioannidis, J. P. A.
- Affiliations
- University of Ioannina (GR)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- Epidemiology
- Published
- 2008-09
- Volume
- 19
- Issue
- 5
- Pages
- 640-648
- ISSN
- 1044-3983 (ISSN-L)
- OpenAlex
- W2150215887
- Semantic Scholar
- 0373f18622b7b64e99c7e56340ab2011dbd00b5d
- Cited by
- 1,495
- References
- 109
- Citations by year
- 2026: 91 · 2025: 87 · 2024: 78 · 2023: 112 · 2022: 94 · 2021: 99 · 2020: 103 · 2019: 131 · 2018: 108 · 2017: 101 · 2016: 87 · 2015: 77 · 2014: 72 · 2013: 60 · 2012: 50
- FWCI
- 15.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- effect size inflation, statistical significance thresholds, underpowered studies, selective reporting, conflicts of interest, reverse bias, measurement error, replication
- MeSH terms
- Clinical Trials as Topic, Data Interpretation, Statistical, Humans, Sensitivity and Specificity, Models, Statistical, Linkage Disequilibrium, Molecular Epidemiology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.