How to Make More Published Research True
Open accessThe DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.
Abstract
Currently, many published research findings are false or exaggerated, and an estimated 85% of research resources are wasted.N To make more published research true, practices that have improved credibility and efficiency in specific fields may be transplanted to others which would benefit from them-possibilities include the adoption of large-scale collaborative research; replication culture; registration; sharing; reproducibility practices; better statistical methods; standardization of definitions and analyses; more appropriate (usually more stringent) statistical thresholds; and improvement in study design standards, peer review, reporting and dissemination of research, and training of the scientific workforce. N Selection of interventions to improve research practices requires rigorous examination and experimental testing whenever feasible.N Optimal interventions need to understand and harness the motives of various stakeholders who operate in scientific research and who differ on the extent to which they are interested in promoting publishable, fundable, translatable, or profitable results.N Modifications need to be made in the reward system for science, affecting the exchange rates for currencies (e.g., publications and grants) and purchased academic goods (e.g., promotion and other academic or administrative power) and introducing currencies that are better aligned with translatable and reproducible research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Accordingly, there is growing recognition that data availability is a critical feature of an efficient, progressive, and ultimately self-correcting scientific ecosystem that generates credible findings [1–3].”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Some journals have heeded the growing calls for adoption of transparent and reproducible research practices and introduced new policies to implement change [1,2].”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, protocol registration is applicable to and recommended for all studies and is supported by various platforms, such as the Open Science Framework [3, 29, 35–37].”
From Assessing transparency practices in dental randomized controlled trials (2024) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Well-planned, well-executed, and well-reported RCTs, which embrace transparent and open science practices, enhance the likelihood that results are accurate, reliable, and applicable [2, 3].”
From Assessing transparency practices in dental randomized controlled trials (2024) · cites this paper as [3]
Experimental — accuracy not yet measuredIn the citing paper
“Consider that the stakeholders who fund and support science may have incentives that are not always aligned with those of active researchers [82].”
From The natural selection of bad science (2016) · cites this paper as [82]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationConfirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267 · cited 2×
“powerful in scientific processes [8,9].”
BackgroundSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 2×
“Many new proposed associations and/or effects are false or grossly exaggerated [2,3], and translation of knowledge into useful applications is often slow and potentially inefficient [4].”
BackgroundSelf-citationClinical Practice Guidelines We Can Trust
Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Board on Health Care Services, & Institute of Medicine (2011) · National Academies Press eBooks · doi:10.17226/13058 · Cited by 3,081
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 1×
“Pharmaceutical industry teams have championed reproducibility in preclinical research [56,57] because they depend on pre-clinical academic investigations accurately pinpointing useful drug targets.”
BackgroundBelieve it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · cited 1×
“Pharmaceutical industry teams have championed reproducibility in preclinical research [56,57] because they depend on pre-clinical academic investigations accurately pinpointing useful drug targets.”
BackgroundEvidence based medicine: a movement in crisis?
Greenhalgh, T., Howick, J., Maskrey, N., & for the Evidence Based Medicine Renaissance Group (2014) · BMJ · doi:10.1136/bmj.g3725 · Cited by 1,766
Scientific method: Statistical errors
Nuzzo, R. (2014) · Nature · doi:10.1038/506150a · Cited by 1,536
Why Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495 · cited 1×
“Many new proposed associations and/or effects are false or grossly exaggerated [2,3], and translation of knowledge into useful applications is often slow and potentially inefficient [4].”
BackgroundSelf-citation
Research works printed but not linked: 1
- R Al-Shahi Salman, 2014, Increasing value and reducing waste in biomedical research regulation and management, Lancet, 383, 176, 10.1016/s0140-6736(13)62297-7 · DOI · Paper page
Cited by
Items 301–400 of 802
Scientific Reporting: Raising the Standards
McLeroy, K. R., Garney, W. R., Mayo‐Wilson, E., & Grant, S. (2016) · Health Education & Behavior · doi:10.1177/1090198116668522 · Cited by 19
“null study findings, thus selecting what is published based on the study results (Ioannidis, 2014).”
BackgroundBampoe, S., Cook, T. M., Fleisher, L. A., Grocott, M. P. W., Neuman, M. D., Story, D., Myles, P. S., & Haller, G. (2018) · BMJ Open · doi:10.1136/bmjopen-2018-023427 · Cited by 18
Hasson, H., Nilsén, P., Augustsson, H., Ingvarsson, S., Korlén, S., & Schwarz, U. V. T. (2019) · Implementation Science · doi:10.1186/s13012-019-0919-x · Cited by 18
Four erroneous beliefs thwarting more trustworthy research
Yarborough, M. A., Nadon, R., & Karlin, D. (2019) · eLife · doi:10.7554/elife.45261 · Cited by 17
“In consequence, we could determine which reforms are most capable of strengthening the overall health of biomedical research (Ioannidis, 2014).”
BackgroundLies, damned lies and statistics: Clinical importance versus statistical significance in research
Mellis, C. (2017) · Paediatric Respiratory Reviews · doi:10.1016/j.prrv.2017.02.002 · Cited by 19
Beyond Hat in Hand: Science Advocacy Is Foundational for Policy Decisions
Kendall‐Taylor, N., & Levitt, P. R. (2017) · Neuron · doi:10.1016/j.neuron.2017.04.039 · Cited by 17
“These are discussions that we hold among ourselves and that are meant to spur critical thinking about improving the scientific process (Ioannidis, 2014; Landis et al., 2012; Pike, 1933).”
BackgroundAre You Sure You Want to Do That? Fostering the Responsible Conduct of Medical Education Research
Maggio, L. A., Artino, A. R., Picho, K., & Driessen, E. W. (2017) · Academic Medicine · doi:10.1097/acm.0000000000001805 · Cited by 19
Reorganizing Biomedical Research: Biobanks as Conditions of Possibility for Personalized Medicine
Tarkkala, H. (2019) · Helda - University of Helsinki Institutional Repository · Cited by 24
Estudio de Seguimiento de las Recomendaciones sobre Análisis Factorial Exploratorio en RIDEP
Paniagua, D., Alvarado, J. M., Olivares, M., Ruiz, I., Romero-Suárez, M., & Aguayo‐Estremera, R. (2022) · Revista Iberoamericana de Diagnóstico y Evaluación - e Avaliação Psicológica · doi:10.21865/ridep66.5.10 · Cited by 16
Chioléro, A., Tancredi, S., & Ioannidis, J. P. A. (2023) · European Journal of Epidemiology · doi:10.1007/s10654-023-01049-6 · Cited by 17
“However, some features of these methods (e.g., flexible data analysis, multiplicity of options, and lack of prespecified hypotheses) increase the probability of false findings [14].”
BackgroundSelf-citationLi, T., Ihanus, A., Ohukainen, P., Järvelin, M., Kähönen, M. A. P., Kettunen, J., Raitakari, O. T., Lehtimäki, T. J., Mäkinen, V., Tynkkynen, T., et al. (2023) · International Journal of Epidemiology · doi:10.1093/ije/dyad162 · Cited by 17
“50 Even though urinary metabolites intrinsically disclose what we eat and drink, the search for discriminatory molecular signals for individual nutrients or even dietary patterns has proved to be futile, 41 though some systemic metabolomics studies have been able to link multi-metabolic profiles of…”
Backgroundde Lorgeril, M., & Rabaeus, M. (2016) · Journal of Controversies in Biomedical Research · doi:10.15586/jcbmr.2015.11 · Cited by 18
“Prestigious scientists recently stated that modern medical scientific data are generally irreproducible (31-33), often wrong (34-38), going as far as claiming that “an estimated 85% of research resources are wasted” (31, 32).”
BackgroundPasterkamp, G., Hoefer, I. E., & Prakken, B. J. (2016) · Nature Biotechnology · doi:10.1038/nbt.3691 · Cited by 16
Moore, J. E., Liu, B., Khan, S., Harris, C., Ewusie, J. E., Hamid, J. S., & Straus, S. E. (2019) · BMC Geriatrics · doi:10.1186/s12877-019-1124-0 · Cited by 21
Plyusnin, I., Holm, L., & Törönen, P. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007419 · Cited by 17
“Indeed, the reliability of scientific findings [30, 31] and over-optimistic results [18, 32] have generated much discussion recently.”
BackgroundStatistical reproducibility for pairwise t-tests in pharmaceutical research
Simkus, A., Coolen, F. P. A., Coolen‐Maturi, T., Karp, N. A., & Bendtsen, C. (2021) · Statistical Methods in Medical Research · doi:10.1177/09622802211041765 · Cited by 23
Whole-Blood Gene Expression Profiles in Large-Scale Epidemiological Studies: What Do They Tell?
Olsen, K. S., Skeie, G., & Lund, E. (2015) · Current Nutrition Reports · doi:10.1007/s13668-015-0143-5 · Cited by 18
We can shift academic culture through publishing choices
Logan, C. (2017) · F1000Research · doi:10.12688/f1000research.11415.1 · Cited by 22
Scuderi, L. A., Weissmann, G. S., Hartley, A. J., Yang, X., & Lancaster, N. (2017) · Aeolian Research · doi:10.1016/j.aeolia.2017.05.004 · Cited by 18
Why Lighting Claims Might Well Be Wrong
Marchant, P. (2017) · International Journal of Sustainable Lighting · doi:10.26607/ijsl.v19i1.71 · Cited by 19
The need for a formalised system of Quality Control for environmental policy-science
Larcombe, P., & Ridd, P. (2017) · Marine Pollution Bulletin · doi:10.1016/j.marpolbul.2017.11.038 · Cited by 16
“Despite replication studies being fundamental to establishing science reliability, such studies are rarely funded, and are not generally seen as a way of advancing a scientific career (Ioannidis, 2014).”
BackgroundInfluentialOf mice and men: long-term safety of assisted reproduction treatments
Sharpe, R. M. (2018) · Human Reproduction · doi:10.1093/humrep/dey071 · Cited by 17
Needs and Challenges for Radiation Oncology in the Era of Precision Medicine
Quon, H., McNutt, T., Lee, J., Bowers, M., Jiang, W., Lakshminarayanan, P., Cheng, Z., Han, P., Hui, X., Shah, V., et al. (2018) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2018.11.017 · Cited by 17
Greenfeld, M., van de Meent, J.-W., Pavlichin, D. S., Mabuchi, H., Wiggins, C. H., Gonzalez, R. L., & Herschlag, D. (2015) · BMC Bioinformatics · doi:10.1186/s12859-014-0429-4 · Cited by 15
Tyranny of the randomised clinical trial
Rosenbek, J. C. (2016) · International Journal of Speech-Language Pathology · doi:10.3109/17549507.2015.1126644 · Cited by 13
“Ioannidis (2005, 2014) argued that findings for most published research are wrong or exaggerated.”
BackgroundGrants: The Good, the Bad, the Ugly, and the Puzzling
Martin, J. J. (2020) · Kinesiology Review · doi:10.1123/kr.2020-0013 · Cited by 15
Pearson, A., Groves, C. P., & Cardini, A. (2015) · Journal of Human Evolution · doi:10.1016/j.jhevol.2015.06.012 · Cited by 15
Smaldino, P. E. (2016) · Behavioral and Brain Sciences · doi:10.1017/s0140525x1500151x · Cited by 15
Salandra, R., Criscuolo, P., & Salter, A. J. (2020) · Research Policy · doi:10.1016/j.respol.2020.104130 · Cited by 16
Digital transformations: an introduction
Olleros, F. X., & Zhegu, M. (2016) · Edward Elgar Publishing eBooks · doi:10.4337/9781784717766.00006 · Cited by 18
“Many authors have argued that the current regime of academic research and publication is suboptimal (Nielsen, 2012; Ioannidis, 2014).”
The language network is not engaged in object categorization
Benn, Y., Ivanova, A. A., Clark, O., Mineroff, Z., Seikus, C., Silva, J. S., Varley, R., & Fedorenko, E. (2023) · Cerebral Cortex · doi:10.1093/cercor/bhad289 · Cited by 14
Belcher, P. E., Moberg, A., & Murphy, M. B. (2024) · SLAS DISCOVERY · doi:10.1016/j.slasd.2024.100192 · Cited by 14
The Credibility Chasm in Policy Research from Academics, Think Tanks, and Advocacy Organizations
Doberstein, C. (2017) · Canadian Public Policy · doi:10.3138/cpp.2016-067 · Cited by 23
“‘‘wild west’’ of some open-access journals (Bohannon 2013; Ioannidis 2014).”
InfluentialAn assessment of transparency and reproducibility‐related research practices in otolaryngology
Johnson, A., Torgerson, T., Skinner, M., Hamilton, T., Tritz, D., & Vassar, M. B. (2019) · The Laryngoscope · doi:10.1002/lary.28322 · Cited by 14
A practical guide to data management and sharing for biomedical laboratory researchers
Fouad, K., Vavrek, R., Surles-Zeigler, M. C., Huie, J. R., Radabaugh, H. L., Gurkoff, G. G., Visser, U., Grethe, J. S., Martone, M. E., Ferguson, A. R., et al. (2024) · Experimental Neurology · doi:10.1016/j.expneurol.2024.114815 · Cited by 15
Association of serum amyloid A and oxidative stress with paraoxonase 1 in sarcoidosis patients
Ivanišević, J., Kotur‐Stevuljević, J., Stefanović, A., Spasić, S., Mihailović, V. V., Ivanov, J. V., & Jelić‐Ivanović, Z. (2016) · European Journal of Clinical Investigation · doi:10.1111/eci.12610 · Cited by 14
Probandari, A. N., Widjanarko, B., Mahendradhata, Y., Sanjoto, H., Cerisha, A., Nungky, S., Riono, P., Simon, S., Farid, M. N., Giriputra, S., et al. (2016) · Global Health Action · doi:10.3402/gha.v9.29866 · Cited by 14
Carpenter, C. R., & Pinnock, H. (2017) · Journal of the American Geriatrics Society · doi:10.1111/jgs.14938 · Cited by 15
A Systems Approach to Understanding and Improving Research Integrity
Gorman, D. M., Elkins, A. D., & Lawley, M. (2017) · Science and Engineering Ethics · doi:10.1007/s11948-017-9986-z · Cited by 14
Transparent reporting of research results in eLife
Teare, M. D. (2016) · eLife · doi:10.7554/elife.21070 · Cited by 14
“G rowing concerns about a lack of reproducibility in certain areas of biomedical research have led to several initiatives to improve the design of experiments, the analysis of data, and the reporting of methods and results (Ioannidis, 2014).”
BackgroundIntroducing New Open Science Practices at EJPA
Greiff, S., van der Westhuizen, L., Mund, M., Rauthmann, J. F., & Wetzel, E. (2020) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000628 · Cited by 14
Büttner, F., Ardern, C. L., Blazey, P., Dastouri, S., McKay, H. A., Moher, D., & Khan, K. M. (2020) · British Journal of Sports Medicine · doi:10.1136/bjsports-2020-103146 · Cited by 16
Research Portfolios in Science Policy: Moving from Financial Returns to Societal BenefitsPreprint
Wallace, M. L., & Ràfols, I. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2743220 · Cited by 14
Open and transparent sports science research: the role of journals to move the field forward
Hansford, H. J., Cashin, A. G., Wewege, M. A., Ferraro, M. C., McAuley, J. H., Jones, M. D., & Collaborators, T. T. S. S. (2022) · Knee Surgery Sports Traumatology Arthroscopy · doi:10.1007/s00167-022-06893-9 · Cited by 14
Yang, M., Zhang, P., Halladay, J. E., Zou, K., Choonara, I., Ji, X., Zhang, S., Yan, W., Huang, L., Lu, X., et al. (2024) · BMC Medicine · doi:10.1186/s12916-024-03560-3 · Cited by 14
“%) did not clearly report the process of cognitive interviews or other pilot tests of the PROMs damaging the transparency and rigor of the development process [146–148].”
BackgroundHe, Y., Yuan, C., Chen, L., Liu, Y., Zhou, H., Xu, N., & Liao, D. J. (2018) · International Journal of Medical Sciences · doi:10.7150/ijms.23215 · Cited by 14
Uddin, S., Choudhury, N., & Hossain, M. E. (2019) · Scientometrics · doi:10.1007/s11192-019-03057-4 · Cited by 13
“In recent years, it is observable that collaborative research has gained popularity and researchers therefore suggest large scale research collaboration to develop new and innovative method that improves credibility and efficiency of research (Ioannidis 2014).”
Machine intelligence identifies soluble TNFa as a therapeutic target for spinal cord injury
Huie, J. R., Ferguson, A. R., Kyritsis, N., Pan, J. Z., Irvine, K., Nielson, J. L., Schupp, P. G., Oldham, M. C., Gensel, J. C., Lin, A., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-82951-5 · Cited by 14
Stakeholders’ views on an institutional dashboard with metrics for responsible research
Haven, T. L., Holst, M., & Strech, D. (2022) · PLoS ONE · doi:10.1371/journal.pone.0269492 · Cited by 15
Assessing the reproducibility of discriminant function analyses
Andrew, R. L., Albert, A., Renaut, S., Rennison, D. J., Bock, D. G., & Vines, T. (2015) · PeerJ · doi:10.7717/peerj.1137 · Cited by 14
“Shared data is an important substrate for science and is one of the levers that may be used to improve the reliability of research (Ioannidis, 2014).”
BackgroundInfluentialTeaching medical discourse in higher education: An introduction
Gotti, M., & Salager‐Meyer, F. (2016) · Language Learning in Higher Education · doi:10.1515/cercles-2016-5001 · Cited by 13
Whitlock, E. P., Dunham, K. M., DiGioia, K., Lazowick, E., Gleason, T., & Atkins, D. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2018.7498 · Cited by 16
Scandola, M., Aglioti, S. M., Avesani, R., Bertagnoni, G., Marangoni, A., & Moro, V. (2019) · PLoS ONE · doi:10.1371/journal.pone.0213838 · Cited by 13
Brain‐wide versus genome‐wide vulnerability biomarkers for severe mental illnesses
Kochunov, P., Ma, Y., Hatch, K. S., Gao, S., Jahanshad, N., Thompson, P. M., Adhikari, B. M., Bruce, H. A., van der Vaart, A. D., Goldwaser, E. L., et al. (2022) · Human Brain Mapping · doi:10.1002/hbm.26056 · Cited by 14
“ibility, but more recent large-scale studies have derived replicable patterns of deficits (Ioannidis, 2014; Kochunov, Hong, et al., 2020; Kochunov, Thompson, & Hong, 2019).”
BackgroundMethods used to assess outcome consistency in clinical studies: A literature-based evaluation
Rogozińska, E., Gargon, E., Olmedo‐Requena, R., Asour, A., Cooper, N. A. M., Vale, C. L., & Hooft, J. V. ’. (2020) · PLoS ONE · doi:10.1371/journal.pone.0235485 · Cited by 14
“[32, 33] However, we discourage labelling this type of evaluation, as a “systematic review”.”
MethodPolgár, S., Schofield, P. N., & Madas, B. G. (2022) · Scientific Data · doi:10.1038/s41597-022-01653-3 · Cited by 14
Women's participation in the research development of a country
Herrera-Franco, G., Peña-Villacreses, G., & Bravo-Montero, L. (2024) · International Journal of Educational Research Open · doi:10.1016/j.ijedro.2024.100413 · Cited by 15
Esterhuizen, T. M., Mbuagbaw, L. C. E., Rehman, N., Yanwou, N., Swaby, D. J., Kittle, E., Licht, J.-C., & Thabane, L. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-100411 · Cited by 13
Gorman, D. M., & Ferdinand, A. O. (2020) · Research Integrity and Peer Review · doi:10.1186/s41073-020-00098-9 · Cited by 13
Bottino, F., Lucignani, M., Pasquini, L., Mastrogiovanni, M., Gazzellini, S., Ritrovato, M., Longo, D., Figà-Talamanca, L., Rossi‐Espagnet, M. C., & Napolitano, A. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.736524 · Cited by 13
“Most of previous studies were performed on a limited number of subjects (Button et al., 2013; Ioannidis, 2014), thus raising the issue that small sample size could be responsible for poorly reliable results.”
ResultA critical analysis of cancer biobank practices in relation to biospecimen quality
Rush, A., Spring, K. J., & Byrne, J. A. (2015) · Biophysical Reviews · doi:10.1007/s12551-015-0178-2 · Cited by 13
Reporting in Experimental Philosophy: Current Standards and Recommendations for Future Practice
Polonioli, A., Vega‐Mendoza, M., Blankinship, B., & Carmel, D. (2018) · Review of Philosophy and Psychology · doi:10.1007/s13164-018-0414-3 · Cited by 12
“…selection biases), analysis (e.g., questionable research practices such as p-hacking) and publication (e.g., a preference for the publication of statistically significant findings) (Nosek et al. 2015; Ioannidis 2005, 2014; Ioannidis et al. 2014; Fanelli 2010; Simmons et al. 2011; John et al. 2012).”
BackgroundCultural obstacles to research data management and sharing at TU Delft
Plomp, E., Dintzner, N., Teperek, M., & Dunning, A. (2019) · Insights the UKSG journal · doi:10.1629/uksg.484 · Cited by 16
Implementation science and periodontal practice: Translation of evidence into periodontology
Frantsve‐Hawley, J., Kumar, S., Rindal, D. B., & Weyant, R. J. (2020) · Periodontology 2000 · doi:10.1111/prd.12336 · Cited by 12
Uddén, A., Carlsson, Y., Karlsson, O., Peeker, R., & Svanvik, T. (2022) · International Journal of Gynecology & Obstetrics · doi:10.1002/ijgo.14285 · Cited by 13
Hansford, H. J., Cashin, A. G., Bagg, M. K., Wewege, M. A., Ferraro, M. C., Kianersi, S., Mayo-Wilson, E., Grant, S., Toomey, E. C., Skinner, I., et al. (2022) · Sports Medicine - Open · doi:10.1186/s40798-022-00496-x · Cited by 14
Statistics Education in Undergraduate Psychology: A Survey of UK Curricula
. TARG Meta-Research Group (2022) · Collabra Psychology · doi:10.1525/collabra.38037 · Cited by 12
Reviews in Research on Adolescence: Genres, Trends, and Challenges
Levesque, R. J. R. (2015) · Adolescent Research Review · doi:10.1007/s40894-015-0022-8 · Cited by 12
“…concerned about replicability issues conclude that ‘‘research on research’’ (i.e., meta-research) now serves as a key way to determine the inadequacies of findings (Ioannidis 2014, p. 3), which is a reasonable argument given that a meta-analysis actually is an explicit form of external replication.”
BackgroundPitfalls of clinical practice guidelines in the era of broken science: Let's raise the standards
Afshari, A., & Hert, S. G. D. (2018) · European Journal of Anaesthesiology · doi:10.1097/eja.0000000000000892 · Cited by 12
WHO, WHAT, WHEN, WHERE, AND WHY? QUANTIFYING AND UNDERSTANDING BIOMEDICAL DATA REUSE
Federer, L. (2019) · University Libraries (University of Maryland) · doi:10.13016/60jd-9hux · Cited by 18
The changing landscape of scientific publishing
Chiriboga, L. A. (2019) · Journal of Histotechnology · doi:10.1080/01478885.2019.1636554 · Cited by 12
“This, combined with publication bias and compromised research, has greatly contributed to the ‘reproducibility crisis’ that has garnered public attention as of late [5,6].”
BackgroundThe Zoltar forecast archive, a tool to standardize and store interdisciplinary prediction research
Reich, N. G., Cornell, M., Ray, E. L., House, K., & Le, K. (2021) · Scientific Data · doi:10.1038/s41597-021-00839-5 · Cited by 12
Good practices and common pitfalls of machine learning in nutrition research
Kirk, D. (2024) · Proceedings of The Nutrition Society · doi:10.1017/s0029665124007638 · Cited by 14
Lipp, H., & Wolfer, D. P. (2022) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2022.958067 · Cited by 12
“…sustainable research data much better (du Sert et al., 2020), while the use of large test batteries, complex operant conditioning or complex mazes would seem less important because they lower reproducibility of behavioral studies—a most important point for acceptance in science (Ioannidis, 2014).”
BackgroundOpenness and Computational Reproducibility in Plant Pathology: Where We Stand and a Way Forward
Sparks, A., Ponte, E. M. D., Alves, K. S., Foster, Z., & Grünwald, N. J. (2023) · Phytopathology · doi:10.1094/phyto-10-21-0430-per · Cited by 11
Hoekstra, D., Gerhardus, A., & Lhachimi, S. K. (2023) · Health Research Policy and Systems · doi:10.1186/s12961-023-01039-w · Cited by 12
“However, decision-making about which research should be conducted first is often not evidence-based [19].”
BackgroundDifferences between protocols for randomized controlled trials and systematic reviews
Pieper, D., & Allers, K. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.11.027 · Cited by 12
Lack of Regulations and Conflict of Interest Transparency of New Hernia Surgery Technologies
Olavarria, O. A., Shah, P. M., Bernardi, K., Lyons, N. B., Holihan, J. L., Ko, T. C., Kao, L. S., & Liang, M. K. (2019) · Journal of Surgical Research · doi:10.1016/j.jss.2019.09.061 · Cited by 12
Design, challenges, and the potential of transcriptomics to understand social behavior
Lim, W. K., & Mathuru, A. S. (2020) · Current Zoology · doi:10.1093/cz/zoaa007 · Cited by 13
“The number of biological replicates is particularly important given the reproducibility crisis due to underpowered studies highlighted in psychology and neuroscience (Button et al. 2013; Ioannidis 2014; Cumming and Calin-Jageman 2016).”
BackgroundBang, J. Y., Sharda, M., & Nadig, A. (2020) · Journal of Neurodevelopmental Disorders · doi:10.1186/s11689-020-09321-6 · Cited by 11
Replicability, Reproducibility, and Fragility of Research Findings—Ultimately, Caveat Emptor
Vetter, T. R., McGwin, G., & Pittet, J. (2016) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000001398 · Cited by 11
Observational Research Using Propensity Scores
Raghunathan, K., Layton, J. B., Ohnuma, T., & Shaw, A. (2016) · Advances in Chronic Kidney Disease · doi:10.1053/j.ackd.2016.11.010 · Cited by 12
Otte, W. M., Tijdink, J. K., Weerheim, P. L., Lamberink, H. J., & Vinkers, C. H. (2018) · eLife · doi:10.7554/elife.34412 · Cited by 12
“Even though adequate power in clinical trials is of vital importance, (Ioannidis, 2014) future studies on gender collaborations should also Table 2 continued Variables Odds ratio 95% CI Z value P value”
BackgroundA practical approach to evidence-based dentistry: X
Carrasco‐Labra, A., Brignardello‐Petersen, R., Azarpazhooh, A., Glick, M., & Guyatt, G. H. (2015) · The Journal of the American Dental Association · doi:10.1016/j.adaj.2015.08.008 · Cited by 13
The great debate: Should Osteoarthritis Research Focus on “Mice” or “Men”?
Hunter, D. J., & Little, C. B. (2015) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2015.07.029 · Cited by 13
Open Science in Applied Linguistics: An Introduction to MetasciencePreprint
Al‐Hoorie, A. H., & Hiver, P. V. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/76dak · Cited by 12
Analysis of Scientific Production in Family Medicine in Mexico
Mendiola-Pastrana, I. R., López‐Ortiz, E., Hernández‐López, R., Henríquez, L. F. R., Dávila-Mendoza, R., & López-Ortiz, G. (2024) · Publications · doi:10.3390/publications12040031 · Cited by 74
Marone, L., de Casenave, J. N. L., & del Solar, R. G. (2019) · Revista de Humanidades de Valparaíso · doi:10.22370/rhv2019iss14pp363-376 · Cited by 10
“More recently, Ioannidis (2014) summarized the research Revista de Humanidades de Valparaíso, 2019, No 14, 363-376 CC BY-NC-ND practices that may help increase the proportion of true research findings: the adoption of reproducibility practices, standardization of definitions and analyses, better…”
BackgroundPro-Con Debate: Interdisciplinary Perspectives on Industry-Sponsored Research
Sessler, D. I., Alman, B. A., Treggiari, M. M., & Mont, M. A. (2023) · The Journal of Arthroplasty · doi:10.1016/j.arth.2023.02.018 · Cited by 10
Williams, M. J., Mullane, K., & Curtis, M. J. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-804725-5.00005-7 · Cited by 13
The Problem of Irreproducible Bioscience Research
Flier, J. S. (2022) · Perspectives in biology and medicine · doi:10.1353/pbm.2022.0032 · Cited by 12
“Once past these objections and limitations to action, corrective steps fall into several broad categories (Ioannidis 2014).”
Campbell, S. E., & Simberloff, D. (2022) · BioScience · doi:10.1093/biosci/biac082 · Cited by 12
METHODS TO ENSURE THE REPRODUCIBILITY OF BIOMEDICAL RESEARCH
Karczewski, K. J., Tatonetti, N. P., Manrai, A. K., Patel, C. J., Brown, C. T., & Ioannidis, J. P. A. (2016) · Pacific Symposium on Biocomputing (PSB) · doi:10.1142/9789813207813_0012 · Cited by 10
Self-citationHealth Psychology Bulletin: Improving Publication Practices to Accelerate Scientific Progress
Peters, G.-J., Kok, G., Crutzen, R., & Sanderman, R. (2017) · Health Psychology Bulletin · doi:10.5334/hpb.2 · Cited by 10
“Despite many articles diagnosing these problems and pleading for improvement (Ioannidis, 2005, 2014; Nosek, Spies, & Motyl, 2012), either authors remain reluctant to submit for example null findings and reports of failed manipulations, or publishers and journals remain reluctant to start accepting…”
BackgroundKrefeld-Schwalb, A., Witte, E. H., & Zenker, F. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00460 · Cited by 17
“It adds severity that this crisis extends beyond psychology— to medicine and health care (Ioannidis, 2014, 2016), genetics (Alfaro and Holder, 2006), sociology (Freese and Peterson, 2017), and political science (Clinton, 2012), among other fields (Fanelli, 2009)—and affects each field as a whole.”
BackgroundEffect of right censoring bias on survival analysisPreprint
Barrajón, E., & Barrajón, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.08649 · Cited by 10
“he Food and Drug Administration (FDA) show some kind of discordance to the published articles [4], and, for this reason, it has been claimed that most of research resources are wasted for this reason [5]. It is worrisome that, in an age of precision medicine, clinical research is threatened by different kinds of biase…”
BackgroundGeneralizability is not optional: Insights from a cross-cultural study of social discountingPreprint
Tiokhin, L., Hackman, J., Munira, S., Jesmin, K., & Hruschka, D. J. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/f5s84 · Cited by 10
The Value of Establishing the Quantitative Accuracy of PET/CT Imaging
Kinahan, P. E., Mankoff, D. A., & Linden, H. M. (2015) · Journal of Nuclear Medicine · doi:10.2967/jnumed.115.159178 · Cited by 9
“Given the challenges with reproducibility and the pooling of data for meta-analysis in biomedical research (17), the sharing of data by Merck and ACRIN is a welcome model of progress in medical imaging research and a foundation for further progress in the development of the precision medicine paradigm.”
MethodKey Issues in the Statistical Interpretation of Randomized Clinical Trials
Brophy, J. M. (2021) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2020.12.014 · Cited by 9
Battle of the primes – The effect and interplay of health and hedonic primes on food choice
Bauer, J. M., van der Laan, L. N., de Bruijn, G., & Reisch, L. A. (2022) · Appetite · doi:10.1016/j.appet.2022.105956 · Cited by 9
“, 2016) showed that goal priming effects are robust and that this literature shows little evidence for publication bias, the priming literature is at the core of the ongoing replication crisis (Doyen et al., 2012; Ioannidis, 2014; Simmons et al., 2011).”
Background
Metadata
- Authors
- Ioannidis, J. P. A.
- Affiliations
- Stanford Medicine (US)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-10-21
- Volume
- 11
- Issue
- 10
- Pages
- e1001747
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC4204808
- OpenAlex
- W2077181076
- Cited by
- 760
- References
- 75
- Citations by year
- 2026: 26 · 2025: 31 · 2024: 37 · 2023: 50 · 2022: 51 · 2021: 44 · 2020: 51 · 2019: 68 · 2018: 93 · 2017: 108 · 2016: 111 · 2015: 80 · 2014: 10
- FWCI
- 192.35
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Laura and John Arnold Foundation
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- research integrity, reproducibility, research credibility, research registration, statistical thresholds, peer review, research reporting, research incentives, publication incentives, research waste
- MeSH terms
- Publications, Publishing, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.