Why Most Discovered True Associations Are Inflated
The DOI matches its Crossref record; 111 of the 111 references it cites were found in a registry or an index.
Abstract
Newly discovered true (non-null) associations often have inflated effects compared with the true effect sizes. I discuss here the main reasons for this inflation. First, theoretical considerations prove that when true discovery is claimed based on crossing a threshold of statistical significance and the discovery study is underpowered, the observed effects are expected to be inflated. This has been demonstrated in various fields ranging from early stopped clinical trials to genome-wide associations. Second, flexible analyses coupled with selective reporting may inflate the published discovered effects. The vibration ratio (the ratio of the largest vs. smallest effect on the same association approached with different analytic choices) can be very large. Third, effects may be inflated at the stage of interpretation due to diverse conflicts of interest. Discovered effects are not always inflated, and under some circumstances may be deflated-for example, in the setting of late discovery of associations in sequentially accumulated overpowered evidence, in some types of misclassification from measurement error, and in conflicts causing reverse biases. Finally, I discuss potential approaches to this problem. These include being cautious about newly discovered effect sizes, considering some rational down-adjustment, using analytical methods that correct for the anticipated inflation, ignoring the magnitude of the effect (if not necessary), conducting large studies in the discovery ph…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Given that initial effect sizes are often inflated 61 and that we sought power to detect a difference between two groups, we doubled this estimate and aimed to test ~28 participants in each group with some attrition expected.”
From Adults with autism overestimate the volatility of the sensory environment (2017) · cites this paper as [61]
In the citing paper
“While this may reflect exaggeration of original effect sizes or publication bias (Ioannidis 2008), some degree of inflation is expected even in its absence, as our selection process filtered for original experiments with significant differences.”
From Estimating the replicability of Brazilian biomedical science (2025) · cites this paper as Ioannidis 2008
In the citing paper
“Further, in situations when an underpowered study detects a true effect, the estimate of this effect size is likely to be highly exaggerated, a phenomenon often referred to as the “winner’s curse” (15).”
From Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies (2016) · cites this paper as [15]
In the citing paper
“Researchers have strong incentives to interpret their results as supporting or extending the Bayesian framework, rather than challenging it—leading to a literature that systematically over-represents supportive evidence (Ioannidis 2008; Nosek et al. 2012).”
From The myth of the Bayesian brain (2025) · cites this paper as Ioannidis 2008
In the citing paper
“This flexibility in the synthesis of the evidence provides the assessors with the opportunity to inflate or disinflate the efficacy of a given technology, a phenomenon that was observed in scientific publications.[24]”
From Health technology assessments conducted in health care facilities: A strategic practice? Findings from a content analysis of HTA reports (2017) · cites this paper as [24]
8 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationGene expression profiling predicts clinical outcome of breast cancer
van 't Veer, L. J., Dai, H., van de Vijver, M. J., He, Y. D., Hart, A. A. M., Mao, M., Peterse, H. L., van der Kooy, K., Marton, M. J., Witteveen, A. T., et al. (2002) · Nature · doi:10.1038/415530a · Cited by 9,536
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040297 · Cited by 8,613
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“This article will not deal with the debate on whether research findings are credible.(1) I will focus instead on the interesting subset of research findings that are true.”
BackgroundSelf-citationEffect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes
Nissen, S. E., & Wolski, K. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa072761 · Cited by 4,675
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., & Egger, M. (2007) · Epidemiology · doi:10.1097/ede.0b013e3181577511 · Cited by 2,779
Internal validation of predictive models
Steyerberg, E. W., Harrell, F. E., Borsboom, G. J. J. M., Eijkemans, M. J. C., Vergouwe, Y., & Habbema, J. D. F. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00341-9 · Cited by 2,746
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544
Cited by
Items 901–1,000 of 1,004
Colchicine for the primary prevention of cardiovascular events
Martí-Carvajal, A. J., Gemmato-Valecillos, M. A., Martín, D. M., Sanctis, J. B. D., Martí-Amarista, C. E., Hidalgo, R., Alegría‐Barrero, E., Lizardo, R. J. R., & Correa-Pérez, A. (2025) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd015003.pub2 · Cited by 15
Bias in data-driven replicability analysis of univariate brain-wide association studies
Burns, C. D. G., Fracasso, A., & Rousselet, G. A. (2025) · Scientific Reports · doi:10.1038/s41598-025-89257-w · Cited by 9
“Issues with NHST are well documented 26,63,64 and combined with small sample sizes it leads to underpowered studies that report distorted effect sizes 8,11,22,65 .”
Research Into the Genetic Component of Heart Disease: From Linkage Studies to Genome-Wide Genotyping
Elosúa, R., Lluís-Ganella, C., & Lucas, G. (2009) · Revista Española de Cardiología · Cited by 9
Pemahaman Peneliti Psikologi mengenai Besaran Sampel: Data dan Simulasi
Wiradhany, W., Adiasto, K., Yulianto, J. E., & Kiling, I. Y. (2019) · Jurnal Psikologi · doi:10.22146/jpsi.24260 · Cited by 8
A call to implement preclinical study registration in animal ethics review
Jeffers, M. S., MacLellan, A., Avey, M. T., Menon, J. M. L., Sunohara-Neilson, J., Fergusson, D. A., & Lalu, M. M. (2023) · PLoS Biology · doi:10.1371/journal.pbio.3002293 · Cited by 8
Cost-efficiency of knowledge creation: randomized controlled trials vs. observational studies
Struck, R., Baumgarten, G., & Wittmann, M. (2014) · Current Opinion in Anaesthesiology · doi:10.1097/aco.0000000000000060 · Cited by 8
“Study results are often published too slowly [8], not at all [9,10 && ,11] or reported inappropriately [6], and are likely to be false or have inflated results [12,13].”
BackgroundResultGenetics and Genomic Basis of Sleep Disorders in Humans
Pack, A. I., Keenan, B. T., Byrne, E. M., & Gehrman, P. (2016) · Elsevier eBooks · doi:10.1016/b978-0-323-24288-2.00031-3 · Cited by 8
Fuzzy General Linear Modeling for Functional Magnetic Resonance Imaging Analysis
Veloz, A., Moraga, C., Weinstein, A., Hernández-García, L., Chabert, S., Salas, R., Riveros, R., Bennett, C., & Allende, H. (2019) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2019.2936807 · Cited by 8
“techniques developed for brain activation detection [30], [31], [63]–[67].”
MethodProbing Word and Sentence Embeddings for Long-distance Dependencies Effects in French and English
Merlo, P. (2019) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w19-4817 · Cited by 10
“Meta-research and simulations show that first publication of results have a considerably higher chance of being inflated (Ioannidis, 2008), and that small studies have a higher chance of being false positives (Bertamini and Munafò, 2012).”
Heterogeneity of EEG resting-state brain networks in absolute pitch
Greber, M., Klein, C., Leipold, S., Sele, S., & Jäncke, L. (2020) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2020.07.007 · Cited by 8
Adjusting for bias in the mean for primary and secondary outcomes when trials are in sequence
Rothwell, J. C., Julious, S. A., & Cooper, C. L. (2021) · Pharmaceutical Statistics · doi:10.1002/pst.2180 · Cited by 8
“For example, when replicating a small trial with a larger sample size the effect seen in the second trial may decrease in size.(3) Clinical programmes where small or early trials report observing very large treatment effects often progress to larger trials or later stage trials.”
BackgroundGouraud, H., Wallach, J. D., Boussageon, R., Ross, J. S., & Naudet, F. (2022) · BMJ Medicine · doi:10.1136/bmjmed-2022-000154 · Cited by 8
Chronic Cocaine Use and White Matter Coherence: A Diffusion Tensor Imaging Study
Hodges, C. B., Steinberg, J. L., Zuniga, E. A., Ma, L., Bjork, J. M., & Moeller, F. G. (2023) · Journal of Studies on Alcohol and Drugs · doi:10.15288/jsad.21-00410 · Cited by 9
Stress Regulation via Being in Nature and Social Support in Adults, a Meta-analysis
Sparacio, A., Ropovik, I., Jiga‐Boy, G. M., Cem, A., & IJzerman, H. R. (2023) · Collabra Psychology · doi:10.1525/collabra.77343 · Cited by 9
“Such publication bias tends to lead to an inflation of the observed mean effect sizes and the Type I error rate (E. C. Carter et al., 2019; Hong & Reed, 2021; Ioannidis, 2008).”
BackgroundEthical Aspects of Outcome Studies in Social, Behavioral, and Educational Interventions
Gambrill, E. D. (2011) · Research on Social Work Practice · doi:10.1177/1049731511404437 · Cited by 11
“Ioannidis (2005) argues that most published research findings are false (see also Ioannidis, 2008).”
BackgroundLord, E. M., Weir, I. R., & Trinquart, L. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.06.012 · Cited by 8
The Division of Replication Labor
Romero, F. (2020) · Philosophy of Science · doi:10.1086/710625 · Cited by 13
“Researchers have made progress identifying statistical and methodological causes of the crisis (Ioannidis 2008; Simmons, Nelson, and Simonsohn 2011; John, Loewenstein, and Prelec 2012; see Romero 2019, for review) and forcefully defended statistical and methodological reforms (Cumming 2012;…”
BackgroundEmergence of Large Treatment Effects From Small Trials—Reply
Ioannidis, J. P. A., Pereira, T., & Horwitz, R. I. (2013) · JAMA · doi:10.1001/jama.2012.208831 · Cited by 8
Self-citationA systematic review of sample size and power in leading neuroscience journalsPreprint
Carter, A. R., Tilling, K. M., & Munafo, M. R. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/217596 · Cited by 9
“Low statistical power of a study increases the chance that a statistically significant result represents a false positive, and that for true positives the effect size will be exaggerated from the true value (6, 7).”
BackgroundHoffmann, S., Schönbrodt, F. D., Elsas, R., Wilson, R., Strasser, U., & Boulesteix, A. (2020) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/afb9p · Cited by 8
Estimating the replicability of highly cited clinical research (2004–2018)
da Costa, G. G., Neves, K., & Amaral, O. B. (2024) · PLoS ONE · doi:10.1371/journal.pone.0307145 · Cited by 8
“Many studies analyzing replications have found evidence that published effects are systematically inflated, a fact that is expected when statistical significance thresholds are used as a criterion for publication [85].”
BackgroundMechan, F., Praulins, G., Gillespie, J., Gleave, K., Murphy-Fegan, A., McDermott, D. P., Weetman, D., & Lees, R. S. (2024) · Gates Open Research · doi:10.12688/gatesopenres.16123.1 · Cited by 7
“…underpowered sample sizes is not just a risk that the effect will not be detected but that comparisons that are detected (p 0.05) by underpowered sample sizes will have an inherent risk of overestimating the effect size, as has been demonstrated in previous simulation studies (Ioannidis, 2008).”
InfluentialOrtloff, A.-M., Grohs, J. A., Lenau, S., & Smith, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714022 · Cited by 7
“Where effects are at risk to be overvalued, such as in research communities generally suffering from publication bias [35], this may be less desirable.”
Cantone, G. G., & Tomaselli, V. (2025) · Socio-Economic Planning Sciences · doi:10.1016/j.seps.2025.102314 · Cited by 8
Transglutaminase mediated crosslinking of pea protein: from rheology to proteomics
Mukherjee, A., Pee, J. V., Duijsens, D., Grauwet, T., Voorde, I. V. D., Fraeye, I., & Weiland, F. (2025) · Food Hydrocolloids · doi:10.1016/j.foodhyd.2025.112233 · Cited by 10
The reliability gap: Why high predictive accuracy doesn't guarantee stable feature importance
Takefuji, Y. (2026) · Marine Pollution Bulletin · doi:10.1016/j.marpolbul.2026.119398 · Cited by 7
Skeen, S. J., Tokarz, S., Gasik, R. E., Solano, C. M., Smith, E. A., Sagoe, M. B., Hudson, L. V., Steele, K., Theall, K. P., & Clum, G. A. (2023) · JMIR Research Protocols · doi:10.2196/47151 · Cited by 11
“…for mHealth trials remains an area of active investigation and debate [143], a passive GEMA-alone control may inflate preliminary estimates of NOLA GEM (incorporating GEMA+LIFT coping skills) efficacy on primary outcomes, an issue that is potentially exacerbated by the small sample size [144].”
ResultLhomme, É., Hejblum, B. P., Lacabaratz, C., Wiedemann, A., Lelièvre, J., Lévy, Y., Thiébaut, R., & Richert, L. E. (2019) · Journal of Immunological Methods · doi:10.1016/j.jim.2019.112711 · Cited by 7
“This is one of the determinants of the very high prevalence of false-positive results (20).”
BackgroundOpen Science: Recommendations for Research on School Bullying
Noret, N. F., Hunter, S. C., Pimenta, S. M., Taylor, R., & Johnson, R. (2022) · International Journal of Bullying Prevention · doi:10.1007/s42380-022-00130-0 · Cited by 7
“However, this early and detailed planning (especially concerning analytical procedures) can improve the focus on the quality of the research process (Ioannidis, 2008; Munafò et al., 2017).”
Sen, U., & Gredebäck, G. (2022) · Child Development · doi:10.1111/cdev.13850 · Cited by 7
Chen, C., Chung, C., Tsai, M.-C., Wu, E. H., Chiu, P.-R., Tsai, P., & Yeh, S. (2022) · IEEE Journal of Translational Engineering in Health and Medicine · doi:10.1109/jtehm.2022.3206333 · Cited by 7
“However, this does not automatically prevent the probability of finding false results [52], [53], [54].”
BackgroundSidebotham, D. A., Dominick, F., Deng, C., Barlow, C. J., & Jones, P. M. (2023) · British Journal of Anaesthesia · doi:10.1016/j.bja.2023.10.036 · Cited by 8
Krefft, A., Marcus, B., & Weigelt, O. (2024) · Journal of Business and Psychology · doi:10.1007/s10869-024-09932-z · Cited by 7
“A particular strength of “Study 4” was that it used a broad probability sample of employed adults, which was also large enough to provide adequate statistical power to detect effects and accurately estimate effect sizes (Ioannidis, 2008; Muthén & Muthén, 2002).”
Rhees, B. K., & Wingrove, J. A. (2015) · Journal of Cardiovascular Translational Research · doi:10.1007/s12265-015-9641-5 · Cited by 8
Zhang, L., Shen, Y., Hu, W., Ran, S., Lin, Y., Lei, S., Zhang, Y.-H., Papasian, C. J., Yi, N., & Pei, Y. (2015) · Human Genetics · doi:10.1007/s00439-015-1621-y · Cited by 7
“For heterogeneity effects, I2 (Higgins and Thompson 2002) or Q p values can be used as indicator.”
MethodHeterogeneity and law: toward a cognitive legal theory
Ambrosino, A. (2015) · Journal of Institutional Economics · doi:10.1017/s1744137415000351 · Cited by 11
“bias is that we can trust a result if it is supported by many different studies (Ioannidis, 2008).”
ResultP -curve accurately rejects evidence for homeopathic ultramolecular dilutions
Reisman, S., Balboul, M., & Jones, T. (2019) · PeerJ · doi:10.7717/peerj.6318 · Cited by 7
“Researchers can increase their odds of obtaining a statistically significant result by performing many studies (Rosenthal, 1979) or many analyses (Ioannidis, 2008).”
BackgroundLu, H., Yi, L., & Zhang, H. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006964 · Cited by 8
The Rollercoaster of Paclitaxel in the Lower Limbs and Skeletons in the Closet: An Opinion Review
Katsanos, K. H., Kitrou, P. M., & Spiliοpoulos, S. (2021) · Journal of Vascular and Interventional Radiology · doi:10.1016/j.jvir.2021.03.537 · Cited by 7
Senn, S. J. (2013) · Significance · doi:10.1111/j.1740-9713.2013.00652.x · Cited by 7
Kelly, E., Varosanec, M., Kosa, P. F., Sandford, M., Prchkovska, V., Moreno‐Dominguez, D., & Bielekova, B. (2022) · Frontiers in Radiology · doi:10.3389/fradi.2022.1026442 · Cited by 8
Trials and tribulations of statistical significance in biochemistry and omics
Montero, O., Hedeland, M., & Balgoma, D. (2023) · Trends in Biochemical Sciences · doi:10.1016/j.tibs.2023.01.009 · Cited by 7
Novel prediction and the problem of low-quality accommodation
Syrjänen, P. (2023) · Synthese · doi:10.1007/s11229-023-04400-2 · Cited by 7
“This issue (in association with the preferential publication of larger effects) has contributed to a common finding in the sciences, where as sample sizes have grown, effect sizes have tended to shrink (see Ioannidis, 2008; Yarkoni & Westfall, 2017, pp. 1109–1110).”
InfluentialRotelli, M. D. (2015) · Therapeutic Innovation & Regulatory Science · doi:10.1177/2168479015578155 · Cited by 8
Vilander, L. M., Vaara, S. T., Kaunisto, M. A., Pettilä, V., & The FINNAKI Study Group (2019) · Journal of Clinical Medicine · doi:10.3390/jcm8030342 · Cited by 7
“However, the true effects of associations are known to be smaller than the ones reported by first authors [40,41].”
ResultMethylation of OXT and OXTR genes, central oxytocin, and social behavior in female macaques
Leon, D. D., Nishitani, S., Walum, H., McCormack, K. M., Wilson, M. E., Smith, A. K., Young, L. J., & Sánchez, M. M. (2020) · Hormones and Behavior · doi:10.1016/j.yhbeh.2020.104856 · Cited by 7
“In fact, it has been shown across many scientific disciplines that early studies often report highly-inflated effects and suboptimal power (Ioannidis, 2008).”
BackgroundEarly Optimistic Effect in Periodontology and Implant Dentistry Trials
Menne, M. C., Seitidis, G., Faggion, C. M., Mavridis, D., & Pandis, N. (2021) · Journal of Dental Research · doi:10.1177/00220345211025242 · Cited by 9
Chatelain, J., & Ralf, K. (2012) · Revue économique · doi:10.3917/reco.633.0557 · Cited by 7
Visual feature vs. conjunction searches as a robust manipulation of perceptual load
Murphy, G., & Greene, C. M. (2016) · Journal of Cognitive Psychology · doi:10.1080/20445911.2016.1256888 · Cited by 7
“It is possible that the effect sizes reported may be over-estimated (Ioannidis, 2008) but this is”
BackgroundTranscranial direct current brain stimulation for chronic pain
O'Connell, N. E., & Wand, B. M. (2015) · BMJ · doi:10.1136/bmj.h1774 · Cited by 7
Pharmacological interventions for treating heart failure in patients with Chagas cardiomyopathy
Hidalgo, R., Martí-Carvajal, A. J., Kwong, J. S., Simancas‐Racines, D., & Nicola, S. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009077.pub2 · Cited by 10
Assessing transparency practices in dental randomized controlled trials
Prado, M. C., Dotto, L., Agostini, B., & Sarkis-Onofre, R. (2024) · BMC Medical Research Methodology · doi:10.1186/s12874-024-02316-0 · Cited by 6
Addressing low statistical power in computational modelling studies in psychology and neuroscience
Piray, P. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02348-6 · Cited by 8
“This lack of statistical power can lead to both type I and type II errors 33,34 , undermining the reliability and reproducibility of research findings.”
InfluentialExploring the proper use of p-values and confidence intervals in leading epidemiology journals
Hunter, M. K., Russell, A. J., Maldonado, G., & Burstyn, I. (2026) · Global Epidemiology · doi:10.1016/j.gloepi.2026.100247 · Cited by 8
Woimant, A., Steils, N., & Collin‐Lachaud, I. (2026) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2026.104747 · Cited by 5
Takefuji, Y. (2026) · Gastroenterology · doi:10.1053/j.gastro.2026.04.011 · Cited by 6
Characterisation and calibration of multiversal methods
Cantone, G. G., & Tomaselli, V. (2024) · Advances in Data Analysis and Classification · doi:10.1007/s11634-024-00610-9 · Cited by 8
“…it is still problematic that novel (and exotic) findings are more published than failed attempts to replicate old findings because this incentivises many misspractices in applied research (Rosenthal 1979; Thornton and Lee 2000; Ioannidis 2005, 2008; Fanelli 2010; West and Bergstrom 2021).”
Alternative to the statistical mass confusion of testing for “no effect”
Morgan, J. (2025) · The Journal of Cell Biology · doi:10.1083/jcb.202403034 · Cited by 6
Maximizing the Quality and Reporting Standards of Autism Intervention Science
LaPoint, S., Klein, C. B., Sandbank, M. P., Bottema‐Beutel, K., Fletcher‐Watson, S., Divan, G., Dimitriou, D., Anagnostou, E. A., Andersen, M. E., Binns, A. V., et al. (2025) · Autism Research · doi:10.1002/aur.70126 · Cited by 6
“The quality of trial reporting has also emerged as important to concerns on ‘reproducibility’ in psychological and health care research generally and as a safeguard against misleading clinical claims or investigator conflicts of interest (Ioannidis 2005, 2008).”
Kelley, G. A., & Kelley, K. S. (2015) · BioMed Research International · doi:10.1155/2015/151985 · Cited by 11
Miller, D. J., Nguyen, J., & Bottai, M. (2020) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x20953567 · Cited by 6
Decoding Randomized Controlled Trials: An Information Science Perspective
Msaouel, P. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.49e3c961 · Cited by 7
Genetics, neuroscience, and psychopathology: Clothing the emperor
Patrick, C. J. (2014) · Psychophysiology · doi:10.1111/psyp.12356 · Cited by 6
Big data is not about size: when data transform scholarship
Plantin, J., Lagoze, C., Edwards, P. N., & Sandvig, C. (2017) · Éditions de la Maison des sciences de l’homme eBooks · doi:10.4000/books.editionsmsh.9103 · Cited by 6
“In Ioannidis's terms, the " aggressive discoverer " in medicine conceptualizes datasets as " private goldmines not to be shared " while the successful practice of research demands that databases be public (Ioannidis, 2008b, 646) so that suspect findings can be checked.”
BackgroundThe Influence of Co-action on a Simple Attention Task: A Shift Back to the Status Quo
Dosso, J. A., Roberts, K. H., DiGiacomo, A., & Kingstone, A. F. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00874 · Cited by 6
“…current and previous work, is that the attentional withdrawal phenomenon is real but, due to the small power of the original study, the original effect size estimate was inflated and thus the present study was underpowered (Ioannidis, 2008; Button et al., 2013; Open Science Collaboration, 2015).”
ResultWhat’s Going to Happen to Me? Prognosis in the Face of Uncertainty
Chiffi, D., & Andreoletti, M. (2019) · Topoi · doi:10.1007/s11245-019-09684-z · Cited by 6
“While, more pragmatically, the leading meta-research scholar Ioannidis (2008) advocates for “rational down-adjustment of effect sizes” usually found in the medical literature.”
BackgroundNext Steps in Pneumonia Severity Scores
Guillamet, C. V., & Kollef, M. H. (2020) · Clinical Infectious Diseases · doi:10.1093/cid/ciaa184 · Cited by 6
Problematic methodology in a systematic review and meta-analysis of DecisionDx-Melanoma
Marchetti, M. A., Dusza, S. W., & Bartlett, E. K. (2020) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2020.04.186 · Cited by 6
Functional Brain Imaging of Intelligence
Basten, U., & Fiebach, C. J. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108635462.016 · Cited by 6
Seehra, J., Stonehouse‐Smith, D., & Pandis, N. (2021) · European Journal of Orthodontics · doi:10.1093/ejo/cjab042 · Cited by 6
“Replication of studies should be encouraged (22).”
BackgroundInfluentialMultiverse-style analysis: Introduction and application
Shunsen, H., Chen, H., Xiaoxiong, L., Xinran, D., & Yun, W. (2022) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2023.00196 · Cited by 7
“…学体系的局限和人为偏差)的差异,发现原研究的阳性结果中,真阳性结果不到三分之一, 相当部分的结果,极有可能是人为偏差造成的(骆大森, 2017),例如 p 值操纵(P-hacking)、研 究者自由度(researcher degree of freedom)、“小径分叉的花园”(garden of forking paths) (Gelman & Loken, 2014; Simmons et al., 2011; 胡传鹏等, 2016)。这种人为偏差主要体现在研究者设计、…”
Thibault, R. T., Kovács, M., Hardwicke, T. E., Sarafoglou, A., Ioannidis, J. P. A., & Munafo, M. R. (2023) · Royal Society Open Science · doi:10.1098/rsos.230568 · Cited by 6
Self-citationConstantinescu, A., Hughes, D. A., Bull, C. J., Fleming, K., Mitchell, R. E., Zheng, J., Kar, S., Timpson, N. J., Amulic, B., & Vincent, E. E. (2024) · Human Genomics · doi:10.1186/s40246-024-00585-w · Cited by 8
“Firstly, the novel genetic variants identified here may be a result of Win-ner’s curse [94]—SNPs can pass the “significance” threshold (commonly set at 5e-8 [95, 96]) in GWAS by chance in the first discovery study, which is then not replicated in subsequent studies [97, 98].”
BackgroundAssessing accident phobia in mild traumatic brain injury: The Accident Fear Questionnaire.
Sutherland, J., Middleton, J. W., Ornstein, T. J., Lawson, K., & Vickers, K. (2016) · Rehabilitation Psychology · doi:10.1037/rep0000090 · Cited by 6
Robledo‐Arnuncio, J. J., & Unger, G. M. (2018) · Evolutionary Applications · doi:10.1111/eva.12729 · Cited by 6
“…they corre‐ spond to a small subset of sampled loci with the strongest evidence of selection, and estimates of locus‐specific size effects based on crossing significance thresholds are expected to be upwardly biased (Göring, Terwilliger, & Blangero, 2001; Ioannidis, 2008; Williams & Haines, 2011).”
BackgroundHubbard, A., Trostle, J., Cangemi, I., & Eisenberg, J. N. S. (2019) · Epidemiology · doi:10.1097/ede.0000000000001025 · Cited by 6
Luo, Y., Chaimani, A., Furukawa, T. A., Kataoka, Y., Ogawa, Y., Cipriani, A., & Salanti, G. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1413 · Cited by 6
Is the Fisher effect asymmetric? Cointegration analysis and expectations measurement
Cushman, D. O., Vita, G. D., & Trachanas, E. (2022) · International Journal of Finance & Economics · doi:10.1002/ijfe.2616 · Cited by 7
Mello, A. T., Kammer, P. V., Nascimento, G. M., de Lima, L. P., Pessini, J., Valmorbida, A., Page, M. J., & Trindade, E. B. S. D. M. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.06.021 · Cited by 7
Assessment of systematic effects of methodological characteristics on candidate genetic associations
Aljasir, B., Ioannidis, J. P. A., Yurkiewich, A., Moher, D., Higgins, J. P. T., Arora, P., & Little, J. (2012) · Human Genetics · doi:10.1007/s00439-012-1237-4 · Cited by 6
“2011) and many effects that may be true-positives may still be inflated in magnitude compared to their true size (Ioannidis 2008).”
ResultSelf-citationCalin-Jageman, R. J. (2018) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.12.008 · Cited by 6
Interventions for preventing high altitude illness
Martí-Carvajal, A. J., Hidalgo, R., & Simancas‐Racines, D. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009761 · Cited by 6
Science for Precautionary Decision-Making
Grandjean, P. (2013) · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · Cited by 10
Veldkamp, C. L. S. (2017) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/g8cjq · Cited by 6
When and Why to Replicate: As Easy as 1, 2, 3?Preprint
Field, S. M., Hoekstra, R., Bringmann, L. F., & van Ravenzwaaij, D. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/h7js8 · Cited by 6
Comparing Meta-Analyses and Pre-Registered Multiple Labs Replication ProjectsPreprint
Kvarven, A., Strømland, E., & Johannesson, M. (2019) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/brzwt · Cited by 6
Treatment for disseminated intravascular coagulation in patients with acute and chronic leukemia
Martí-Carvajal, A. J., Simancas‐Racines, D., & Cardona, A. F. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008562.pub2 · Cited by 7
Genetics of intracerebral hemorrhage
Kimball, T. N., Tack, R. W. P., Chen, A., Prapiadou, S., Senff, J. R., Tan, B. Y., Singh, S., van Veluw, S. J., Greenberg, S. M., Rosand, J. M., et al. (2025) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1177/0271678x241310401 · Cited by 5
“73 Given the different pathophysiology and with that, genetic underpinning of lobar and deep ICH, genetic discovery in ICH has lagged behind other stroke sub-types and similar traits.”
Statistical power analysis in exercise science
Uehata, A., & Rein, R. (2025) · Journal of Sports Sciences · doi:10.1080/02640414.2025.2571841 · Cited by 6
“As a result, small samples studies tend to report effect sizes that are inflated relative to the true effect sizes (Ellis, 2010; Ioannidis, 2008; Mesquida et al., 2023).”
Schäfer, T. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-06436-w · Cited by 5
“…across psychological subdisciplines that comparability is impossible (e.g., Cristea et al., 2022; Funder & Ozer, 2019; Gignac & Szodorai, 2016; Ioannidis, 2008; Lovakov & Agadullina, 2021; Nordahl-Hansen et al., 2024; Richard et al., 2003; Schäfer & Schwarz, 2019; Szucs & Ioannidis, 2017;…”
Denkinger, C. M., Kalantri, Y., Schumacher, S. G., Michael, J. S., Shankar, D., Saxena, A., SRIRAM, N., Thangakunam, B., Luo, R. F., Pollock, N. R., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0085447 · Cited by 6
“Furthermore, publication bias is likely contributing to the overoptimistic summary estimates in the meta-analyses [21,25].”
BackgroundAssessment of temporal instability in the applied ecology and conservation evidence base
Brisco‐McCann, E. I., Kulinskaya, E., & Koricheva, J. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1691 · Cited by 5
“…range of temporal trends observed in other disciplines which may include gross overestimation of the effect in the initial studies on the topic, 11 reversal of effect sign over a period of time, 5 and the so-called Proteus phenomenon, where early replications of work frequently contradict…”
BackgroundThe response of Canada’s clinical health research ecosystem to the COVID-19 pandemic
Lamontagne, F., Masse, M.-H., Yarnell, C. J., Lemyre, F. C., Lévesque, S., Domingue, M., O’Hearn, K., Watpool, I., Hoogenes, J., Sprague, S., et al. (2024) · Canadian Medical Association Journal · doi:10.1503/cmaj.230760 · Cited by 5
Somatic Variants Acquired Later in Life Associated with Thoracic Aortic Aneurysms: JAK2 V617F
Waldron, C. J., Zafar, M. A., Ma, D., Zhang, H., Dykas, D. J., Ziganshin, B. A., Popa, A., Jha, A., Kwan, J. M., & Elefteriades, J. A. (2024) · Genes · doi:10.3390/genes15070883 · Cited by 5
“Small effect sizes can indicate subtle but meaningful contributions to disease risk, especially when considering multifactorial etiologies where multiple small effect sizes collectively contribute to a clinically significant outcomes [37–41].”
BackgroundGermani, É., Bhagwat, N., Dugré, M., Gau, R., Montillo, A., Nguyen, K. P., Sokolowski, A., Sharp, M. E., Poline, J., & Glatard, T. (2025) · PLoS ONE · doi:10.1371/journal.pone.0317566 · Cited by 7
Patricio, P., & Massé‐Alarie, H. (2025) · Brain Sciences · doi:10.3390/brainsci15070760 · Cited by 5
“This will probably diminish the effect size inflation caused by small studies [98], as shown in tDCS for CLBP.”
Does bad inference drive out good?
Marozzi, M. (2015) · Clinical and Experimental Pharmacology and Physiology · doi:10.1111/1440-1681.12422 · Cited by 6
A Dirichlet process functional approach to heteroscedastic-consistent covariance estimation
Karabatsos, G. (2016) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2016.07.008 · Cited by 8
“Generally speaking, the MDP/OLS methodology can incorporate a vibration of effects (VoE) analysis (Ioannidis, 2008) in order to assess how much the effect size differs (or vibrates) over different ways that the analysis can be performed, for e.g., with respect to different: variables that are…”
Statistical Power and Swallowing Rehabilitation Research: Current Landscape and Next Steps
Borders, J. C., Grande, A. A., & Troche, M. S. (2022) · Dysphagia · doi:10.1007/s00455-022-10428-2 · Cited by 5
Girard, J., Zaouter, C., Moore, A., Carrier, F. M., & Girard, M. (2023) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-023-02533-z · Cited by 5
“34 Assessing PPCs is fraught with pitfalls and multiple trials have used various definitions, illustrating a lack of consensus.”
Background
Metadata
- Authors
- Ioannidis, J. P. A.
- Affiliations
- University of Ioannina (GR)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- Epidemiology
- Published
- 2008-09
- Volume
- 19
- Issue
- 5
- Pages
- 640-648
- ISSN
- 1044-3983 (ISSN-L)
- OpenAlex
- W2150215887
- Semantic Scholar
- 0373f18622b7b64e99c7e56340ab2011dbd00b5d
- Cited by
- 1,495
- References
- 109
- Citations by year
- 2026: 91 · 2025: 87 · 2024: 78 · 2023: 112 · 2022: 94 · 2021: 99 · 2020: 103 · 2019: 131 · 2018: 108 · 2017: 101 · 2016: 87 · 2015: 77 · 2014: 72 · 2013: 60 · 2012: 50
- FWCI
- 15.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- effect size inflation, statistical significance thresholds, underpowered studies, selective reporting, conflicts of interest, reverse bias, measurement error, replication
- MeSH terms
- Clinical Trials as Topic, Data Interpretation, Statistical, Humans, Sensitivity and Specificity, Models, Statistical, Linkage Disequilibrium, Molecular Epidemiology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.