Why Most Discovered True Associations Are Inflated
The DOI matches its Crossref record; 111 of the 111 references it cites were found in a registry or an index.
Abstract
Newly discovered true (non-null) associations often have inflated effects compared with the true effect sizes. I discuss here the main reasons for this inflation. First, theoretical considerations prove that when true discovery is claimed based on crossing a threshold of statistical significance and the discovery study is underpowered, the observed effects are expected to be inflated. This has been demonstrated in various fields ranging from early stopped clinical trials to genome-wide associations. Second, flexible analyses coupled with selective reporting may inflate the published discovered effects. The vibration ratio (the ratio of the largest vs. smallest effect on the same association approached with different analytic choices) can be very large. Third, effects may be inflated at the stage of interpretation due to diverse conflicts of interest. Discovered effects are not always inflated, and under some circumstances may be deflated-for example, in the setting of late discovery of associations in sequentially accumulated overpowered evidence, in some types of misclassification from measurement error, and in conflicts causing reverse biases. Finally, I discuss potential approaches to this problem. These include being cautious about newly discovered effect sizes, considering some rational down-adjustment, using analytical methods that correct for the anticipated inflation, ignoring the magnitude of the effect (if not necessary), conducting large studies in the discovery ph…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Given that initial effect sizes are often inflated 61 and that we sought power to detect a difference between two groups, we doubled this estimate and aimed to test ~28 participants in each group with some attrition expected.”
From Adults with autism overestimate the volatility of the sensory environment (2017) · cites this paper as [61]
In the citing paper
“While this may reflect exaggeration of original effect sizes or publication bias (Ioannidis 2008), some degree of inflation is expected even in its absence, as our selection process filtered for original experiments with significant differences.”
From Estimating the replicability of Brazilian biomedical science (2025) · cites this paper as Ioannidis 2008
In the citing paper
“Further, in situations when an underpowered study detects a true effect, the estimate of this effect size is likely to be highly exaggerated, a phenomenon often referred to as the “winner’s curse” (15).”
From Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies (2016) · cites this paper as [15]
In the citing paper
“Researchers have strong incentives to interpret their results as supporting or extending the Bayesian framework, rather than challenging it—leading to a literature that systematically over-represents supportive evidence (Ioannidis 2008; Nosek et al. 2012).”
From The myth of the Bayesian brain (2025) · cites this paper as Ioannidis 2008
In the citing paper
“This flexibility in the synthesis of the evidence provides the assessors with the opportunity to inflate or disinflate the efficacy of a given technology, a phenomenon that was observed in scientific publications.[24]”
From Health technology assessments conducted in health care facilities: A strategic practice? Findings from a content analysis of HTA reports (2017) · cites this paper as [24]
8 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationGene expression profiling predicts clinical outcome of breast cancer
van 't Veer, L. J., Dai, H., van de Vijver, M. J., He, Y. D., Hart, A. A. M., Mao, M., Peterse, H. L., van der Kooy, K., Marton, M. J., Witteveen, A. T., et al. (2002) · Nature · doi:10.1038/415530a · Cited by 9,536
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040297 · Cited by 8,613
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“This article will not deal with the debate on whether research findings are credible.(1) I will focus instead on the interesting subset of research findings that are true.”
BackgroundSelf-citationEffect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes
Nissen, S. E., & Wolski, K. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa072761 · Cited by 4,675
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., & Egger, M. (2007) · Epidemiology · doi:10.1097/ede.0b013e3181577511 · Cited by 2,779
Internal validation of predictive models
Steyerberg, E. W., Harrell, F. E., Borsboom, G. J. J. M., Eijkemans, M. J. C., Vergouwe, Y., & Habbema, J. D. F. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00341-9 · Cited by 2,746
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544
Cited by
Items 801–900 of 1,004
Clarifying the reliability paradox: poor measurement reliability attenuates group differences
Karvelis, P., & Diaconescu, A. O. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1592658 · Cited by 16
“While low measurement reliability attenuates observed effect , small samples produce unstable estimates that are often selectively reported, leading to systematic inflation of reported effects—known as the winner’s curse (Sidebotham and Barlow, 2024; Button et al., 2013; Ioannidis, 2008).”
Wallach, J. D., Ciani, O., Woods, A. P., Gonsalves, G., Krumholz, H. M., Taylor, R. S., & Ross, J. S. (2018) · BMC Medicine · doi:10.1186/s12916-018-1023-9 · Cited by 15
Predictive modeling of neurobehavioral state and trait variation across development
Sanchez–Alonso, S., & Aslin, R. Ν. (2020) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2020.100855 · Cited by 13
Lukas, C. A., & Berking, M. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-021-00799-y · Cited by 14
“First, due to the small sample size used in this pilot study, results should be interpreted with caution as they are limited in generalizability [18, 19].”
MethodElosúa, R., Lluís-Ganella, C., & Lucas, G. (2009) · Revista Española de Cardiología Suplementos · doi:10.1016/s1131-3587(09)71502-x · Cited by 16
Liu, Y., Zhang, L., Roy‐Engel, A. M., Saito, S., Lasky, J. A., Wang, G., & Wang, H. (2016) · Gene · doi:10.1016/j.gene.2016.11.028 · Cited by 13
“Statistically speaking, it can also alleviate effectively the common problem of multiple testing and “winner’s curse” (Ioannidis 2008) in a genomic study.”
BackgroundRehkopf, D. H., Modrek, S., Cantley, L. F., & Cullen, M. R. (2017) · Health Affairs · doi:10.1377/hlthaff.2016.1186 · Cited by 14
Prioritering, styring og likebehandling: Utfordringer i norsk helsetjeneste
Bjorvatn, A., Carlsen, B., Hofmann, B., Kaur, J. G., Magelssen, M., Nilssen, E., Radhakrishnan, K., Hodne, S., Sudmann, T. T., Teig, I. L., et al. (2018) · doi:10.23865/noasp.33 · Cited by 14
Leipold, S., Oderbolz, C., Greber, M., & Jäncke, L. (2019) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2018.12.016 · Cited by 13
Potential Diagnostic and Prognostic Biomarkers of Circular RNAs for Lung Cancer in China
Wang, C., Jiang, Y., Lei, Q., Wu, Y., Shao, J., Pu, D., & Li, W. (2019) · BioMed Research International · doi:10.1155/2019/8023541 · Cited by 13
“First, the majority of sample size of subjects is small [68, 69].”
BackgroundElaboration of countertransference experience and the workings of the working alliance.
Barreto, J. F., Nata, G., & Matos, P. M. (2020) · Psychotherapy · doi:10.1037/pst0000250 · Cited by 15
Evaluation of a Meta-Analysis of Ambient Air Quality as a Risk Factor for Asthma Exacerbation
Kindzierski, W. B., Young, S. S., Meyer, T. G., & Dunn, J. (2021) · Journal of Respiration · doi:10.3390/jor1030017 · Cited by 13
“meta-analysis as a possible example of what is known as ‘vibration of effects’ [39,113,120]—wherein summary statistics for a weak risk factor (i.”
BackgroundMethodThe Search for Truth Must Go Beyond Statistics
Willett, W. C. (2008) · Epidemiology · doi:10.1097/ede.0b013e318181b877 · Cited by 13
Truths, Associations, and Hypotheses
Kirpalani, H. M., & Whyte, R. K. (2011) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2011.05.002 · Cited by 13
Antibiotics for treating acute chest syndrome in people with sickle cell disease
Martí-Carvajal, A. J., Conterno, L. D. O., & Knight‐Madden, J. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006110.pub2 · Cited by 17
Causal Analysis of Policy Effects on Fertility
Hart, R., Bergsvik, J., Fauske, A., & Kim, W. (2023) · doi:10.2139/ssrn.4608812 · Cited by 13
Epidemiological fallacies of modern psychiatric research
Newson, R. S., Karlsson, H., & Tiemeier, H. (2011) · Nordic Journal of Psychiatry · doi:10.3109/08039488.2010.539268 · Cited by 12
The universe is asymmetric, the mouse brain too
Rivera-Olvera, A., Houwing, D. J., Ellegood, J., Masifi, S., Martina, S. L., Silberfeld, A., Pourquié, O., Lerch, J. P., Francks, C., Homberg, J. R., et al. (2024) · Molecular Psychiatry · doi:10.1038/s41380-024-02687-2 · Cited by 14
“Indeed, small sample sizes generally lead to falsely in fl ated effects [56, 57].”
Iacono, W. G. (2014) · Psychophysiology · doi:10.1111/psyp.12340 · Cited by 12
“P A A uthor M anuscript Pollastri, & Smoller, 2014; Ioannidis, 2008, 2011).”
BackgroundWeakening the ideological immune system: can debiasing techniques reduce confirmation bias?
Whitt, C. M., Tullett, A. M., Bowes, S. M., McDiarmid, A., Lilienfeld, S. O., & Hart, W. (2023) · Comprehensive Results in Social Psychology · doi:10.1080/23743603.2023.2196755 · Cited by 12
False Positives in Cancer Epidemiology
McLaughlin, J. K., & Tarone, R. E. (2012) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-12-0995 · Cited by 12
Campbell, H., & Gustafson, P. (2019) · The American Statistician · doi:10.1080/00031305.2018.1555101 · Cited by 14
Publication Bias in Meta-Analyses of Posttraumatic Stress Disorder Interventions
Niemeyer, H., van Aert, R. C. M., Schmid, S., Ülsmann, D., Knaevelsrud, C., & Schulte‐Herbrüggen, O. (2020) · Meta-Psychology · doi:10.15626/mp.2018.884 · Cited by 12
Empirical Bayes and semi-Bayes adjustments for a vast number of estimations
Strömberg, U. (2009) · European Journal of Epidemiology · doi:10.1007/s10654-009-9393-0 · Cited by 13
Data-Adaptive Target Parameters
Hubbard, A., Kennedy, C. J., & van der Laan, M. J. (2018) · Springer series in statistics · doi:10.1007/978-3-319-65304-4_9 · Cited by 16
Fogal, G. G. (2015) · TSpace (University of Toronto) · Cited by 26
Reply: POLR3A variants in hereditary spastic paraplegia and ataxia
Minnerop, M., Kurzwelly, D., Rattay, T. W., Timmann, D., Hengel, H., Synofzik, M., Stendel, C., Horváth, R., Schüle, R., & Ramírez, A. (2017) · Brain · doi:10.1093/brain/awx291 · Cited by 13
Sparacio, A., Ropovik, I., Jiga‐Boy, G. M., Forscher, P. S., Paris, B., & IJzerman, H. R. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/zpw28 · Cited by 14
Multicurious: A Multidisciplinary Guide to Multiverse Analysis
Short, C. A., Breznau, N., Bruntsch, M., Burkhardt, M., Busch, N. A., Čėsnaitė, E., Frank, M., Gießing, C., Krähmer, D., Kristanto, D., et al. (2026) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459261434881 · Cited by 12
“…approaches include “sensitivity analysis” (e.g., Saltelli et al., 2019), “robustness checks” (e.g., Neumayer & Plümper, 2017), “vibration of effects” (Ioannidis, 2008), “specification-curve analysis” (Simonsohn et al., 2020), “manyverse analysis” (Kuhn et al., 2022), “many-analysts approach”…”
Yao, T., Xi, X., & Prieto, P. (2025) · System · doi:10.1016/j.system.2025.103835 · Cited by 12
Davis, K. E., & Maiden, R. (2021) · Studies in Social Science Research · doi:10.22158/sssr.v2n2p1 · Cited by 11
“The limitations of NHST are well known across diverse fields such as psychology (Cohen, 1990, 1994; Schmidt, 1996), education (Robinson & Levin, 1997; Thompson, 1996), and medicine (Ioannidis, 2008).”
BackgroundExploratory analyses: How to meaningfully interpret and report them
Roberts, A., Roche, M., & Sainani, K. L. (2023) · PM&R · doi:10.1002/pmrj.12980 · Cited by 5
Evaluating a shrinkage estimator for the treatment effect in clinical trials
van Zwet, E. W., Tian, L., & Tibshirani, R. J. (2023) · Statistics in Medicine · doi:10.1002/sim.9992 · Cited by 11
Zhang, P., Peng, B., Lin, Z., & Du, H. (2024) · Journal of Environmental Management · doi:10.1016/j.jenvman.2024.122549 · Cited by 11
Why prudence is needed when interpreting articles reporting clinical trial results in mental health
Dal‐Ré, R., Bobes, J., & Cuijpers, P. J. (2017) · Trials · doi:10.1186/s13063-017-1899-2 · Cited by 12
“these cases, in a trial with a small sample size, the magnitude of the estimated effect could vary (the so-called ‘vibration effect’) and will depend on factors such as the principal endpoint (psychometric scale) chosen, the use of adjustments for certain confounders and the availability of alternative options in stat…”
BackgroundBiological Evidence in Adult and Adolescent Sexual Assault Cases: Timing and Relationship to Arrest
Cross, T. P., Alderden, M., Wagner, A., Sampson, L., Peters, B., & Lounsbury, K. (2017) · Journal of Interpersonal Violence · doi:10.1177/0886260517704229 · Cited by 11
“However, these results should be interpreted very cautiously, because statistically significant results in early, small studies can have inflated effect sizes and often do not replicate in future studies (see, for example, Ioannadis, 2008).”
ResultThe MRC Framework for the Development, Design and Analysis of Stratified Medicine Research
Isaacs, J. D., & Matthews, F. E. (2017) · Cited by 15
Médecine de précision et Evidence-Based Medicine : quelle articulation ?
Giroux, É. (2017) · Lato Sensu Revue de la Société de philosophie des sciences · doi:10.20416/lsrsps.v4i2.683 · Cited by 13
Borowiak, K., & von Kriegstein, K. (2020) · Translational Psychiatry · doi:10.1038/s41398-020-00903-5 · Cited by 13
Neves, K., Carneiro, C. F. D., Wasilewska‐Sampaio, A. P., de Abreu, M. B., Valério‐Gomes, B., Tan, P. B., & Amaral, O. B. (2020) · Memórias do Instituto Oswaldo Cruz · doi:10.1590/0074-02760200328 · Cited by 11
Meiering, M. S., Weigner, D., Gärtner, M., Schäfer, T., & Grimm, S. (2022) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2022.10.062 · Cited by 13
D’abord les données, ensuite la méthode ? Big data et déterminisme en sciences sociales*
Plantin, J., & Federica Russo, B. (2016) · Socio · doi:10.4000/socio.2328 · Cited by 11
Steele, J., Fisher, J. P., & Crawford, D. A. (2020) · Journal of Trainology · doi:10.17338/trainology.9.1_20 · Cited by 13
Correspondence measures for assessing replication success.
Steiner, P. M., Sheehan, P., & Wong, V. C. (2023) · Psychological Methods · doi:10.1037/met0000597 · Cited by 13
The Effectiveness of Acupuncture for Low Back Pain: An Umbrella Review and Meta-Analysis
Wu, M., Fan, C., Liu, H., Chen, X., Zhen, G., Zhao, X., Zhou, J., & Jiang, Z. (2024) · The American Journal of Chinese Medicine · doi:10.1142/s0192415x2450037x · Cited by 10
p-Hacking: Its Costs and When It Is Warranted
Erasmus, A. (2024) · Erkenntnis · doi:10.1007/s10670-024-00834-3 · Cited by 11
“The most common arguments against p-hacking operate on the logic that the chance of discovering a statistically significant result from numerous analyses is higher than if just one analysis is performed (Ioannidis, 2008; Simmons et al., 2011; Nuzzo, 2014).”
Leslie, K., & Short, T. G. (2014) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000000210 · Cited by 10
P-values and significance: The null hypothesis that they are not related is correct
Einav, S., & O'Connor, M. (2019) · Journal of Critical Care · doi:10.1016/j.jcrc.2019.08.020 · Cited by 10
Graßmann, F., Mälarstig, A., Dahl, L., Bendes, A., Dale, M., Thomas, C. E., Gabrielson, M., Hedman, Å. K., Eriksson, M., Margolin, S., et al. (2023) · British Journal of Cancer · doi:10.1038/s41416-023-02541-2 · Cited by 12
What is the vibration of effects?
Vinatier, C., Hoffmann, S., Patel, C. J., DeVito, N., Cristea, I. A., Tierney, B. T., Ioannidis, J. P. A., & Naudet, F. (2024) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2023-112747 · Cited by 13
Self-citationGelman, A., & Brown, N. J. L. (2024) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459241276398 · Cited by 10
“It is well known, although perhaps not so well understood, that published results tend to be overly optimistic about effect sizes as a result of low power and selection on statistical significance (Ioannidis, 2008).”
A bibliometric analysis of work–family balance: trends, themes, and future directions (2000–2024)
Fang, Y., Li, X., Mohtar, T. M., & Chekima, B. (2025) · Cogent Business & Management · doi:10.1080/23311975.2025.2541041 · Cited by 11
“…effects (Bornmann & Daniel, 2008; schmidt & Hunter, 2015). moreover, the applicability of meta-analysis is constrained by data availability, , 2008). narrative reviews, lacking systematic quantitative analysis, often rely on subjective judgment and are prone to selection bias (tranfield et…”
Chou, R. (2011) · Pain Physician · doi:10.36076/ppj.2011/14/e69 · Cited by 10
van der Staay, F. J., Nordquist, R. E., & Arndt, S. S. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-809468-6.00003-6 · Cited by 11
“…good efficacy and safety in animal models (Gribkoff and Kaczmarek, in press; Macleod, 2011; van der Worp et al., 2010). consequently, the translatability of result of animal studies to humans has been described as poor (Garner, 2014; ioannidis, 2008; Macleod, 2011; van der Worp et al., 2010).”
Hiew, L. F., Khairuddin, S., Aquili, L., Koh, J., Fung, M., Lim, W. L., & Lim, L. W. (2020) · Behavioural Brain Research · doi:10.1016/j.bbr.2020.112559 · Cited by 10
Adler, M., Herrera‐Gómez, F., Martín-García, D., Gavid, M., Alvarez, E., & Sangrador, C. O. (2020) · Pharmaceuticals · doi:10.3390/ph13050085 · Cited by 10
“Finally, this meta-analysis includes trials of less than 1000 participants, so our findings contribute to clarify false substantial effects reported by such small trials [39]: probably more research is needed to deny the absence of effects from 1500 mg or”
MethodExamining replicability in addictions research: How to assess and ways forward.
Pearson, M. R., Schwebel, F. J., Richards, D. K., & Witkiewitz, K. A. (2022) · Psychology of Addictive Behaviors · doi:10.1037/adb0000730 · Cited by 12
Current Status and Future Opportunities in Modeling Clinical Characteristics of Multiple Sclerosis
Liu, J., Kelly, E., & Bielekova, B. (2022) · Frontiers in Neurology · doi:10.3389/fneur.2022.884089 · Cited by 11
“Validation of Published Inverse Relationships Between Study Design Quality and Reported Effect Sizes Previous studies on non-MS fields showed that (1) small cohort studies; (2) studies of low experimental quality; and (3) studies performed only in the training cohort, all significantly overestimate true effect sizes (…”
BackgroundResultInfluentialMoons, K. G. M., Altman, D. G., Reitsma, J. B., loannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2022) · Digital Diagnostics · doi:10.17816/dd110794 · Cited by 10
Randomized clinical trials and observational studies in the assessment of drug safety
Sawchik, J., Hamdani, J., & Vanhaeverbeek, M. (2018) · Revue d Épidémiologie et de Santé Publique · doi:10.1016/j.respe.2018.03.133 · Cited by 12
Nudel, R., Christiani, C. J., Ohland, J., Uddin, M. J., Hemager, N., Ellersgaard, D., Spang, K. S., Burton, B. K., Greve, A. N., Gantriis, D. L., et al. (2020) · BMC Neuroscience · doi:10.1186/s12868-020-00581-5 · Cited by 10
Rodgers, E. M., Franklin, C. E., & Noble, D. W. A. (2021) · Journal of Experimental Biology · doi:10.1242/jeb.228213 · Cited by 10
A note regarding meta‐analysis of sequential trials with stopping for efficacy
Senn, S. J. (2014) · Pharmaceutical Statistics · doi:10.1002/pst.1639 · Cited by 10
“Ioannidis has suggested that it could be one amongst many explanations as to why research findings are exaggerated [1].”
BackgroundСивак, Е., Pankowska, P., Mendrik, A. M., Emery, T., Garcia‐Bernardo, J., Hocuk, S., Karpińska, K., Maineri, A., Mulder, J., Nissim, M., et al. (2024) · Journal of Computational Social Science · doi:10.1007/s42001-024-00275-6 · Cited by 11
Herold, R., Gevorgyan, H., Damerau, L., Hartmann, U., Friemert, D., Ellegast, R. P., Schiefer, C., Karamanidis, K., Harth, V., & Terschüren, C. (2024) · Sensors · doi:10.3390/s24206515 · Cited by 9
“Fewer participants ( n = 43) than the a priori target ( n = 58) took part in the baseline study, which could result in an inflation of the discovered associations [31].”
Backgroundde Oliveira, A. D. S., Graciliano, N. G., Praxedes, D. R. S., Macêna, M. D. L., Júnior, A. E. D. S., Pereira, M. R., dos Santos, J. V. L., Galdino-Silva, M. B., Almeida, K. M. M., Tenório, D. M. C., et al. (2025) · Food & Function · doi:10.1039/d5fo02449h · Cited by 10
Dick, A. S., Comer, J. S., Bayat, M., Curtis, M., Hayes, T., Pruden, S. M., Hawes, S. W., González, R., Laird, A. R., & Graziano, P. A. (2025) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2025.101630 · Cited by 9
“Analyses of existing literature lend weight to this concern (Bakker et al., 2012), suggesting that numerous published neurodevelopmental associations could potentially exhibit significantly exaggerated effect sizes (Button et al., 2013; Ioannidis, 2008).”
The Best Words: Linguistic Indicators of Grandiose Narcissism in Politics
Underberg, J. E., Gollwitzer, A., Oettingen, G., & Gollwitzer, P. M. (2019) · Journal of Language and Social Psychology · doi:10.1177/0261927x19883898 · Cited by 9
“Our comparatively large effect sizes are likely due to the small sample size of our study (n = 35); small samples commonly lead to inflated effect sizes, also known as the “winner’s curse” (Ioannidis, 2008).”
BackgroundFox, M. P., McGowan, L. D., James, B. D., Lessler, J., Mehta, S. H., & Murray, E. J. (2020) · American Journal of Epidemiology · doi:10.1093/aje/kwaa189 · Cited by 9
Kouraki, A., Doherty, M., Fernandes, G., Zhang, W., Walsh, D. A., Kelly, A., & Valdes, A. M. (2022) · European Journal of Pain · doi:10.1002/ejp.1902 · Cited by 9
“Another limitation with the use of GWAS is the possibility of inflated effect sizes (Ioannidis, 2008).”
BackgroundTeshome, M., Torres, C. M. M. E., Sileshi, G. W., de Mattos, P. P., Braz, E. M., Temesgen, H., da Rocha, S. J. S. S., & Alebachew, M. (2022) · Open Journal of Forestry · doi:10.4236/ojf.2022.123015 · Cited by 9
Buick, J. E., Austin, P. C., Cheskes, S., Ko, D. T., & Atzema, C. L. (2023) · Academic Emergency Medicine · doi:10.1111/acem.14756 · Cited by 10
A tutorial for calculating field-specific effect size distributions
Glaser, B. D., Kang, H., Audunsdottir, K., Sartorius, A. M., & Quintana, D. S. (2026) · Behavior Research Methods · doi:10.3758/s13428-026-03003-2 · Cited by 11
Matvienko‐Sikar, K., O'Shea, J., Kennedy, S., Thomas, S. D., AVERY, K. N. L., Byrne, M., McHugh, S., Connor, D. B. O., Saldanha, I. J., Smith, V. A., et al. (2024) · Health Psychology Review · doi:10.1080/17437199.2024.2367613 · Cited by 11
“Further, selective outcome reporting distorts pooled treatment e ff ects (Ioannidis, 2008; Macura et al., 2010; Shah et al., 2020; Thomas & Heneghan, 2022), which can lead to mis-informed decision making, with important implications for healthcare professionals and for patient, community, and…”
Thomas, E., Juliano, A., Owens, M. M., Cupertino, R. B., Mackey, S., Hermosillo, R. J. M., Miranda-Domínguez, Ó., Conan, G., Ahmed, M., Fair, D. A., et al. (2024) · Psychiatry Research Neuroimaging · doi:10.1016/j.pscychresns.2024.111877 · Cited by 10
“Many published neurodevelopmental associations may represent severely inflated effect sizes (Button et al., 2013; Ioan-nidis, 2008).”
BackgroundThe power of sample size calculations
Jansen, M. S., Groenwold, R. H. H., & Dekkers, O. M. (2024) · European Journal of Endocrinology · doi:10.1093/ejendo/lvae129 · Cited by 11
Müller, D. X., Sessler, D. I., & Saugel, B. (2025) · British Journal of Anaesthesia · doi:10.1016/j.bja.2024.12.022 · Cited by 9
Strengthening the reporting of Genetic RIsk Prediction Studies (GRIPS): explanation and elaboration
Janssens, A. C. J. W., Ioannidis, J. P. A., Bedrosian, S. R., Boffetta, P., Dolan, S. M., Dowling, N. F., Fortier, I., Freedman, A. N., Grimshaw, J., Gulcher, J. R., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.02.003 · Cited by 12
Self-citationXu, B., Moradi, M., Kuplicki, R., Stewart, J. L., McKinney, B. A., Sen, S., & Paulus, M. P. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.503248 · Cited by 9
“In the long run, regression toward the mean would be a more achievable result expected to be observed (58).”
BackgroundA Biased Perspective on Brain Imaging of ADHD
Castellanos, F. X. (2021) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2021.21060609 · Cited by 11
ter Hark, S. E., Vos, C. F., Aarnoutse, R. E., Schene, A. H., Coenen, M. J. H., & Janzing, J. (2022) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2022.03.057 · Cited by 10
Munafo, M. R. (2015) · International Journal of Epidemiology · doi:10.1093/ije/dyv056 · Cited by 10
Dai, J., Su, Y., Bartell, S. M., Le, N., Ling, W., Yi-quan, L., Gao, L., Wu, H., Veledar, E., & Vaccarino, V. (2009) · Metabolism · doi:10.1016/j.metabol.2008.11.010 · Cited by 10
“Model selection procedures such as backward elimination are widely used to develop parsimonious statistical models, but may produce biased estimation particularly for studies with small sample sizes [24].”
BackgroundStatistical evidence and the reliability of medical research
Andreoletti, M., & Teira, D. (2016) · UNED repository · doi:10.4324/9781315720739.ch20 · Cited by 10
Exercise reduces depressive symptoms in adults with arthritis: Evidential value
Kelley, G. A., & Kelley, K. S. (2016) · World Journal of Rheumatology · doi:10.5499/wjr.v6.i2.23 · Cited by 10
“While encouraging, all investigations appeared in peer-reviewed academic journals, a potential problem given that publications in academic journals yield an overly excessive number of statistically significant results[6].”
BackgroundKim, J. Y., Steingroever, J., Lee, K. H., Oh, J. S., Choi, M., Lee, J. M., Larkins, N., Schaefer, F. S., Hong, S. H., Jeong, G. H., et al. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9020394 · Cited by 10
“In addition, to assess potential biases such as reporting biases and confounding biases in statistically significant findings that might lead to false positives or inflated estimate of the association [13,14], we performed re-analyses of the identified meta-analyses including statistical tests for various biases and g…”
MethodThe Problem of New Evidence: P-Hacking and Pre-Analysis Plans
Hitzig, Z., & Stegenga, J. (2020) · Diametros · doi:10.33392/diam.1587 · Cited by 11
Novelli, M., Antognini, A. B., Boffetta, P., Ioannidis, J. P. A., Spatari, G., & Violante, F. S. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.04.006 · Cited by 9
Self-citationThe Meta-Plot: A Graphical Tool for Interpreting the Results of a Meta-Analysis
van Assen, M. A. L. M., van den Akker, O. R., Augusteijn, H. E. M., Bakker, M., Nuijten, M. B., Olsson-Collentine, A., Stoevenbelt, A. H., Wicherts, J. M., & van Aert, R. C. M. (2023) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000513 · Cited by 9
Morfeld, P. (2008) · Archives of Toxicology · doi:10.1007/s00204-008-0398-3 · Cited by 9
“The warning by Ioannidis (2008) that empirical evidence indicates that some seasoned epidemiologists distort the presentation of their results intentionally should be taken more seriously by editors, reviewers and readers.”
BackgroundStatistical perspectives: all together NOT
Hopkins, W. G., Batterham, A. M., Impellizzeri, F. M., Pyne, D. B., & ROWLANDS, D. S. (2011) · The Journal of Physiology · doi:10.1113/jphysiol.2011.218529 · Cited by 9
“The authors have ignored the problems and discontent with this traditional approach to inference (e.g. Rozeboom, 1960; Cohen, 1994; Sterne & Davey Smith, 2001; Ioannidis, 2008; Ziliak & McCloskey, 2008; Stang et al. 2010).”
BackgroundPrivacy fit in open-plan offices : its appraisal, associated outcomes & contextual factors
Weber, C. (2019) · Zürcher Hochschule für Angewandte Wissenschaften digital collection (Zurich University of Applied Sciences) · doi:10.15126/thesis.00850409 · Cited by 15
“the risk that the statistical power to detect the hypothesised effects and accurate effect sizes was limited (Ioannidis, 2008).”
BackgroundAntibiotics for treating acute chest syndrome in people with sickle cell disease
Martí-Carvajal, A. J., Conterno, L. D. O., & Knight‐Madden, J. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006110.pub3 · Cited by 10
Using meta-analysis for evidence synthesis: The case of incomplete neutralization in GermanPreprint
Nicenboim, B., Roettger, T. B., & Vasishth, S. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/p5a4z · Cited by 9
Learning from the past: refining the way we study treatments
Viechtbauer, W. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.004 · Cited by 8
Three Pre-registered Null Results for EEG Binary Transition Rate as an Aging Biomarker
Tkemaladze, J. (2026) · Longevity Horizon · doi:10.65649/a184qf96 · Cited by 8
Using routinely collected data for research purposes: challenges and mitigation strategies
Hoffmann, S., Morris, T. P., Herrmann, M., Heinze, G., Wynants, L., CALSTER, B. V., Bischl, B., Schmid, M., Shaw, P. A., Mathes, T., et al. (2026) · BMJ · doi:10.1136/bmj-2025-087812 · Cited by 12
Prone position for acute respiratory distress syndrome and the hazards of meta-analysis
Poole, D., Pisa, A., & Fumagalli, R. (2024) · Pulmonology · doi:10.1016/j.pulmoe.2022.12.005 · Cited by 9
“…one hundred patients, to avoid the small-study bias also known as the “ winner curse ” (i.e., an exaggerated effect in favour of intervention studied in the trial), 27,28 which is dif fi cult to detect with Egger ’ s regression, as this test is underpowered when performed on fewer than ten studies.”
Assessing the effectiveness of online emotion recognition training in healthy volunteers
Reed, Z. E., Suddell, S. F., Eastwood, A., Thomas, L., Dwyer, I., Penton‐Voak, I. S., Jarrold, C., Munafo, M. R., & Attwood, A. S. (2023) · Royal Society Open Science · doi:10.1098/rsos.230372 · Cited by 8
Efficacy and Safety of Low Dose Naltrexone for Chronic Pain
Irwin, M. N., Cooke, D., Berland, D., Marshall, V. D., & Smith, M. A. (2024) · Journal of Pain & Palliative Care Pharmacotherapy · doi:10.1080/15360288.2024.2302550 · Cited by 8
“In the setting of an underpowered study, the effect size is likely exaggerated and needs further examination with adequately powered studies (27).”
Background
Metadata
- Authors
- Ioannidis, J. P. A.
- Affiliations
- University of Ioannina (GR)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- Epidemiology
- Published
- 2008-09
- Volume
- 19
- Issue
- 5
- Pages
- 640-648
- ISSN
- 1044-3983 (ISSN-L)
- OpenAlex
- W2150215887
- Semantic Scholar
- 0373f18622b7b64e99c7e56340ab2011dbd00b5d
- Cited by
- 1,495
- References
- 109
- Citations by year
- 2026: 91 · 2025: 87 · 2024: 78 · 2023: 112 · 2022: 94 · 2021: 99 · 2020: 103 · 2019: 131 · 2018: 108 · 2017: 101 · 2016: 87 · 2015: 77 · 2014: 72 · 2013: 60 · 2012: 50
- FWCI
- 15.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- effect size inflation, statistical significance thresholds, underpowered studies, selective reporting, conflicts of interest, reverse bias, measurement error, replication
- MeSH terms
- Clinical Trials as Topic, Data Interpretation, Statistical, Humans, Sensitivity and Specificity, Models, Statistical, Linkage Disequilibrium, Molecular Epidemiology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.