How to Make More Published Research True
Open accessThe DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.
Abstract
Currently, many published research findings are false or exaggerated, and an estimated 85% of research resources are wasted.N To make more published research true, practices that have improved credibility and efficiency in specific fields may be transplanted to others which would benefit from them-possibilities include the adoption of large-scale collaborative research; replication culture; registration; sharing; reproducibility practices; better statistical methods; standardization of definitions and analyses; more appropriate (usually more stringent) statistical thresholds; and improvement in study design standards, peer review, reporting and dissemination of research, and training of the scientific workforce. N Selection of interventions to improve research practices requires rigorous examination and experimental testing whenever feasible.N Optimal interventions need to understand and harness the motives of various stakeholders who operate in scientific research and who differ on the extent to which they are interested in promoting publishable, fundable, translatable, or profitable results.N Modifications need to be made in the reward system for science, affecting the exchange rates for currencies (e.g., publications and grants) and purchased academic goods (e.g., promotion and other academic or administrative power) and introducing currencies that are better aligned with translatable and reproducible research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Accordingly, there is growing recognition that data availability is a critical feature of an efficient, progressive, and ultimately self-correcting scientific ecosystem that generates credible findings [1–3].”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Some journals have heeded the growing calls for adoption of transparent and reproducible research practices and introduced new policies to implement change [1,2].”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, protocol registration is applicable to and recommended for all studies and is supported by various platforms, such as the Open Science Framework [3, 29, 35–37].”
From Assessing transparency practices in dental randomized controlled trials (2024) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Well-planned, well-executed, and well-reported RCTs, which embrace transparent and open science practices, enhance the likelihood that results are accurate, reliable, and applicable [2, 3].”
From Assessing transparency practices in dental randomized controlled trials (2024) · cites this paper as [3]
Experimental — accuracy not yet measuredIn the citing paper
“Consider that the stakeholders who fund and support science may have incentives that are not always aligned with those of active researchers [82].”
From The natural selection of bad science (2016) · cites this paper as [82]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationConfirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267 · cited 2×
“powerful in scientific processes [8,9].”
BackgroundSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 2×
“Many new proposed associations and/or effects are false or grossly exaggerated [2,3], and translation of knowledge into useful applications is often slow and potentially inefficient [4].”
BackgroundSelf-citationClinical Practice Guidelines We Can Trust
Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Board on Health Care Services, & Institute of Medicine (2011) · National Academies Press eBooks · doi:10.17226/13058 · Cited by 3,081
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 1×
“Pharmaceutical industry teams have championed reproducibility in preclinical research [56,57] because they depend on pre-clinical academic investigations accurately pinpointing useful drug targets.”
BackgroundBelieve it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · cited 1×
“Pharmaceutical industry teams have championed reproducibility in preclinical research [56,57] because they depend on pre-clinical academic investigations accurately pinpointing useful drug targets.”
BackgroundEvidence based medicine: a movement in crisis?
Greenhalgh, T., Howick, J., Maskrey, N., & for the Evidence Based Medicine Renaissance Group (2014) · BMJ · doi:10.1136/bmj.g3725 · Cited by 1,766
Scientific method: Statistical errors
Nuzzo, R. (2014) · Nature · doi:10.1038/506150a · Cited by 1,536
Why Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495 · cited 1×
“Many new proposed associations and/or effects are false or grossly exaggerated [2,3], and translation of knowledge into useful applications is often slow and potentially inefficient [4].”
BackgroundSelf-citation
Research works printed but not linked: 1
- R Al-Shahi Salman, 2014, Increasing value and reducing waste in biomedical research regulation and management, Lancet, 383, 176, 10.1016/s0140-6736(13)62297-7 · DOI · Paper page
Cited by
Items 201–300 of 802
Blockchain for Science and Knowledge Creation
Bartling, S. H. (2018) · Springer eBooks · doi:10.1007/978-3-662-57611-3_10 · Cited by 35
Global, Yet Incomplete Overview of Cohort Studies in Parkinson’s disease
Heinzel, S., Lerche, S., Maetzler, W., & Berg, D. (2017) · Journal of Parkinson s Disease · doi:10.3233/jpd-171100 · Cited by 32
Independent discussion sections for improving inferential reproducibility in published research
Avidan, M. S., Ioannidis, J. P. A., & Mashour, G. A. (2019) · British Journal of Anaesthesia · doi:10.1016/j.bja.2018.12.010 · Cited by 34
Self-citationStieglitz, S., Meske, C., Roß, B., & Mirbabaie, M. (2018) · Information Systems Frontiers · doi:10.1007/s10796-018-9867-2 · Cited by 34
“There have been prominent calls for showing the replicability of results in several disciplines, from medicine (Ioannidis 2014) to the social sciences (Schmidt 2009), after several previously accepted findings were discovered to be spurious (perhaps most famously, the suggested link between MMR…”
BackgroundAll science should inform policy and regulation
Ioannidis, J. P. A. (2018) · PLoS Medicine · doi:10.1371/journal.pmed.1002576 · Cited by 33
Self-citationWhy science’s crisis should not become a political battling ground
Saltelli, A. (2018) · Futures · doi:10.1016/j.futures.2018.07.006 · Cited by 34
A Future for Observational Epidemiology: Clarity, Credibility, Transparency
Harper, S. B. (2019) · American Journal of Epidemiology · doi:10.1093/aje/kwy280 · Cited by 34
From Discovery to Justification: Outline of an Ideal Research Program in Empirical Psychology
Witte, E. H., & Zenker, F. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01847 · Cited by 38
“1 In our view, the ongoing replicability crisis reflects a goal-conflict between publishing statistically significant results as an individual researcher and increasing the trustworthiness of scientific knowledge as a community (Bakker et al., 2012; Nosek et al., 2012; Ioannidis, 2014).”
BackgroundSigmoidal Acquisition Curves Are Good Indicators of Conformist Transmission
Smaldino, P. E., Aplin, L. M., & Farine, D. R. (2018) · Scientific Reports · doi:10.1038/s41598-018-30248-5 · Cited by 35
Responsible Universities in Context
Geschwind, L., Kekäle, J., Pinheiro, R. M., & Sørensen, M. P. (2019) · Springer eBooks · doi:10.1007/978-3-030-25646-3_1 · Cited by 31
Zhou, L., Deng, X., Guo, K., Hou, L., Xu, H., Wu, Y., Xu, M., Wang, Y., Liang, S., Yang, K., et al. (2023) · JMIR Public Health and Surveillance · doi:10.2196/44745 · Cited by 36
“Furthermore, the preregistration of studies and publishing research protocols can XSL • FO RenderX be beneficial for improving the transparency and trustworthiness of the evidence generated [50,51].”
Howard, M. O., & Garland, E. L. (2015) · Journal of the Society for Social Work and Research · doi:10.1086/681099 · Cited by 30
“Many proposals have been made to enhance the reproducibility of scientific findings (Collins & Tabak, 2014; Ioannidis, 2005b, 2014; Landis et al., 2012).”
BackgroundAdoption of protocols to improve quality of medical research
Montagna, E., Zaia, V., & Laporta, G. Z. (2019) · Einstein (São Paulo) · doi:10.31744/einstein_journal/2020ed5316 · Cited by 36
Maniadis, Z., Tufano, F., & List, J. A. (2015) · Emerald Group Publishing Limited eBooks · doi:10.1108/s0193-230620150000018008 · Cited by 33
Generating FAIR research data in experimental tribology
Garabedian, N., Schreiber, P., Brandt, N., Zschumme, P., Blatter, I., Dollmann, A., Haug, C., Kümmel, D., Li, Y., Meyer, F. I., et al. (2022) · Scientific Data · doi:10.1038/s41597-022-01429-9 · Cited by 34
Enhancing reproducibility: Failures from Reproducibility Initiatives underline core challenges
Mullane, K., & Williams, M. (2017) · Biochemical Pharmacology · doi:10.1016/j.bcp.2017.04.008 · Cited by 28
The Research Reproducibility Crisis and Economics of Science
Maniadis, Z., & Tufano, F. (2017) · The Economic Journal · doi:10.1111/ecoj.12526 · Cited by 32
“The accumulated empirical evidence has led researchers in these disciplines to pay increasing attention to researcher incentives (e.g., Bakker et al., 2012; Nosek et al., 2012; Ioannidis, 2014), with a focus on how to improve research practices.”
Should We Strive to Make Science Bias-Free? A Philosophical Assessment of the Reproducibility Crisis
Hudson, R. G. S. (2021) · Journal for General Philosophy of Science · doi:10.1007/s10838-020-09548-w · Cited by 33
“2012, 617, and Ioannidis 2014, 3–5).”
Krefeld-Schwalb, A., & Scheibehenne, B. (2022) · Marketing Letters · doi:10.1007/s11002-022-09662-3 · Cited by 31
“In addition, to the social sciences, similar efforts revealed issues in medicine (Ioannidis, 2014, 2016) and the natural sciences (Alfaro & Holder, 2015) as well.”
BackgroundYarborough, M. A., Bredenoord, A. L., D’Abramo, F., Joyce, N. C., Kimmelman, J., Ogbogu, U., Sena, E. S., Strech, D., & Dirnagl, U. (2018) · PLoS Biology · doi:10.1371/journal.pbio.2006343 · Cited by 30
Auditory Brainstem Pathology in Autism Spectrum Disorder: A Review
Pillion, J. P., Boatman‐Reich, D., & Gordon, B. (2018) · Cognitive and Behavioral Neurology · doi:10.1097/wnn.0000000000000154 · Cited by 30
Why is reporting quality improvement so hard? A qualitative study in perioperative care
Jones, E. L., Dixon‐Woods, M., & Martin, G. P. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-030269 · Cited by 30
Koçak, B., Yardımcı, A. H., Yüzkan, S., Keles, A., Altun, O., Bulut, E., Bayrak, O. N., & Okumus, A. A. (2022) · Academic Radiology · doi:10.1016/j.acra.2022.11.030 · Cited by 32
Feasibility of individual patient data meta-analyses in orthopaedic surgery
Villain, B., Dechartres, A., Boyer, P. J., & Ravaud, P. (2015) · BMC Medicine · doi:10.1186/s12916-015-0376-6 · Cited by 27
“Finally, as recently outlined by Ioannidis, the current system does not reward enough data sharing [43].”
BackgroundDecision Aids to Increase Living Donor Kidney Transplantation
Gander, J. C., Gordon, E. J., & Patzer, R. E. (2017) · Current Transplantation Reports · doi:10.1007/s40472-017-0133-1 · Cited by 31
Kochunov, P., Patel, B., Ganjgahi, H., Donohue, B., Ryan, M. C., Hong, L. E., Chen, X., Adhikari, B. M., Jahanshad, N., Thompson, P. M., et al. (2019) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2019.00016 · Cited by 28
“Recent reports on low reproducibility in biomedical research are raising concerns that have to be addressed within the scientific community (Ioannidis, 2014).”
BackgroundThe GalNAc-T Activation (GALA) Pathway: Drivers and markers
Chia, J., Tay, F. P., & Bard, F. (2019) · PLoS ONE · doi:10.1371/journal.pone.0214118 · Cited by 27
A Path to Greater Credibility: Large-Scale Collaborative Education Research
Makel, M. C., Smith, K. N., McBee, M. T., Peters, S. J., & Miller, E. M. (2019) · AERA Open · doi:10.1177/2332858419891963 · Cited by 31
Über das Gehirn hinaus: Aktiver Externalismus und die Natur des Mentalen
Walter, H. (2018) · Nervenheilkunde · doi:10.1055/s-0038-1668314 · Cited by 28
Do citations and readership identify seminal publications?
Herrmannová, D., Patton, R. M., Knoth, P., & Stahl, C. G. (2018) · Scientometrics · doi:10.1007/s11192-018-2669-y · Cited by 29
Mechanotransductive Receptor Piezo1 as a Promising Target in the Treatment of Neurological Diseases
Bryniarska, N., Kubiak, A., & Basta‐Kaim, A. (2022) · Current Neuropharmacology · doi:10.2174/1570159x20666220927103454 · Cited by 26
“On the other hand, Piezo1/2 boom should lead researchers to pru-dence about results presented in new scientific publications – in the context of also physiological meaning, data reproducibility and their scientific merit [40, 41].”
BackgroundFostering a research integrity culture: Actionable advice for institutions
Peuter, S. D., & Conix, S. (2022) · Science and Public Policy · doi:10.1093/scipol/scac059 · Cited by 29
Gender gap among highly cited researchers, 2014–2021
Meho, L. I. (2022) · Quantitative Science Studies · doi:10.1162/qss_a_00218 · Cited by 27
“Publications represent the primary means of disseminating knowledge and the principal measure of research productivity, which influences career prospects and visibility (Holman et al., 2018; Ioannidis, 2014).”
BackgroundWucherpfennig, F., Rubel, J. A., Hollon, S. D., & Lutz, W. (2016) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.11.003 · Cited by 29
Barillot, C., Bannier, É., Commowick, O., Corouge, I., Baire, A., Fakhfakh, I., Guillaumont, J., Yao, Y., & Kain, M. (2016) · Frontiers in ICT · doi:10.3389/fict.2016.00025 · Cited by 27
“, 2009), and 4) the validation of research results based on proofed control statistical analysis of images for validation and quality control of experimental research (Button et al., 2013; Carp, 2012; Ioannidis, 2014; Ioannidis et al., 2014).”
BackgroundAnticipating consequences of sharing raw data and code and of awarding badges for sharing
Ioannidis, J. P. A. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.04.015 · Cited by 25
Self-citationLi, D., & CORNELIS, G. C. (2020) · Journal of Empirical Research on Human Research Ethics · doi:10.1177/1556264620927628 · Cited by 26
“…that research misconduct comes from both the ineffectiveness of the self-discipline system (such as peer review and the replication of published results) and the lack of outside controlling systems (Bohannon, 2013; Bouter & Hendrix, 2017; Goodstein, 1996; Ioannidis, 2014; Stroebe et al., 2012).”
BackgroundQu, S., Hu, Z., Zhang, Y., Wang, P., Li, S., Huang, S., Dong, Y., Xu, H., Rong, Y., Zhu, W., et al. (2021) · Advances in Wound Care · doi:10.1089/wound.2020.1186 · Cited by 26
Snyder, H. M., Shineman, D. W., Friedman, L. G., Hendrix, J. A., Khachaturian, A. S., Guillou, I. L., Pickett, J., Refolo, L. M., Sancho, R. M., & Ridley, S. H. (2016) · Alzheimer s & Dementia · doi:10.1016/j.jalz.2016.07.001 · Cited by 28
What confidence should we have in GRADE ?
Mercuri, M., & Baigrie, B. S. (2018) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12993 · Cited by 28
Schnell, S. D. (2018) · Bulletin of Mathematical Biology · doi:10.1007/s11538-018-0500-9 · Cited by 28
“It has been recommended to modify the reward system in science by awarding scientists who aligned with best practices for reproducibility research (Ioannidis 2014).”
BackgroundPreclinical data reproducibility for R&D—the challenge for neuroscience
Steckler, T. (2015) · Psychopharmacology · doi:10.1007/s00213-014-3836-3 · Cited by 25
Academic criteria for appointment, promotion and rewards in medical research: where's the evidence?
Moher, D., Goodman, S. N., & Ioannidis, J. P. A. (2016) · European Journal of Clinical Investigation · doi:10.1111/eci.12612 · Cited by 25
Self-citationGard, M., & Pluim, C. (2017) · Sport Education and Society · doi:10.1080/13573322.2016.1213716 · Cited by 26
“First, not only are most published findings of dubious worth, but that ‘many published research findings are false or exaggerated, and an estimated 85% of research resources are wasted’ (Ioannidis, 2014, p. 1).”
Backgroundde Ridder, J. (2022) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2022.02.003 · Cited by 26
Draht, M. X. G., Smits, K. M., Jooste, V., Tournier, B., Vervoort, M., Ramaekers, C. H. M. A., Chapusot, C., Weijenberg, M. P., van Engeland, M., & Melotte, V. (2016) · Clinical Epigenetics · doi:10.1186/s13148-016-0211-8 · Cited by 27
Random responses inflate statistical estimates in heavily skewed addictions data
King, K. M., Kim, D. S., & McCabe, C. J. (2017) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2017.10.033 · Cited by 25
Liang, M., Chen, Q., Zhang, Y., He, L., Wang, J., Cai, Y., & Li, L. (2016) · Oncotarget · doi:10.18632/oncotarget.14312 · Cited by 24
Transparency, bias, and reproducibility across science: a meta-research view
Ioannidis, J. P. A. (2024) · Journal of Clinical Investigation · doi:10.1172/jci181923 · Cited by 26
“In assessing worked at least in some fields and/or hold high promise (37): conduct of large-scale, collaborative research; adoption of a replication culture; registration (of studies, protocols, analysis codes, datasets, raw data, and results); sharing (of data, protocols, materials, software, and…”
Self-citationFunctional vs Structural Cortical Deficit Pattern Biomarkers for Major Depressive Disorder
Kochunov, P., Adhikari, B. M., Keator, D. B., Amen, D., Gao, S., Karcher, N. R., Labate, D., Azencott, R., Huang, Y., Syed, H., et al. (2025) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2025.0192 · Cited by 25
Habituation of laser-evoked potentials by migraine phase: a blinded longitudinal study
Uglem, M., Omland, P. M., Stjern, M., Gravdahl, G. B., & Sand, T. (2017) · The Journal of Headache and Pain · doi:10.1186/s10194-017-0810-6 · Cited by 24
Risk of bias in exercise science: A systematic review of 340 studies
Preobrazenski, N., McCaig, A., Turner, A., Kushner, M., Pacitti, L. J., Mendolia, P., MacDonald, B., Storoschuk, K. L., Bouck, T., Zaza, Y., et al. (2024) · iScience · doi:10.1016/j.isci.2024.109010 · Cited by 25
Gambrill, E. D. (2016) · Journal of Social Work Education · doi:10.1080/10437797.2016.1174642 · Cited by 29
“Ioannidis (2014) estimates that billions of dollars have been wasted on research that cannot answer questions pursued.”
BackgroundReproducibility Issues: Avoiding Pitfalls in Animal Inflammation Models
Laman, J. D., Kooistra, S. M., & Clausen, B. E. (2017) · Methods in molecular biology · doi:10.1007/978-1-4939-6786-5_1 · Cited by 24
“models read at least the following key publications: On being a scientist [37], the ARRIVE guidelines [38, 39], Reproducibility in science [7], A call for transparent reporting to optimize the predictive value of preclinical research [40], How to make more published research true [8], The Leiden Manifesto [41], and Th…”
BackgroundMethodA data-driven approach for evaluating multi-modal therapy in traumatic brain injury
Haefeli, J., Ferguson, A. R., Bingham, D., Orr, A., Won, S. J., Lam, T. I., Shi, J., Hawley, S., Liu, J., Swanson, R. A., et al. (2017) · Scientific Reports · doi:10.1038/srep42474 · Cited by 25
“, number of principal components) was defined based on the following criteria: (1) Kaiser rule, retaining principal components with eigenvalues > 1; (2) Cattell rule, retaining principal components above the elbow in the scree plot; (3) PC over-determination, retaining components with at least four loadings above 0.”
Method¡Enjaulad a Dodo, por favor! El cuento de que todas las psicoterapias son igual de eficaces
González‐Blanch, C., & Carral-Fernández, L. (2017) · Papeles del Psicólogo - Psychologist Papers · doi:10.23923/pap.psicol2017.2828 · Cited by 29
“La Psicología (junto con la Psiquiatría) es la más propensa a publicar resultados positivos, hasta 5 veces más en la comparación con las ciencias del espacio, que están en el otro extremo de la lista (Fanelli, 2010).”
BackgroundInfluentialMongeon, P., Robinson‐García, N., Jeng, W., & Costas, R. (2017) · Aslib Journal of Information Management · doi:10.1108/ajim-01-2017-0024 · Cited by 30
“Data sharing also promotes transparency (Lyon, 2016), and makes research more valid and rigorous by facilitating reproducibility and promoting a replication culture (Ioannidis, 2014).”
BackgroundResultHinkelbein, J., Iovino, I. Z., ROBERTIS, E. D., & Kranke, P. (2019) · BMC Anesthesiology · doi:10.1186/s12871-019-0716-8 · Cited by 25
Rigor and replication: Toward improved best practices in human electrophysiology research
Larson, M. J., & Moser, J. S. (2016) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2016.12.001 · Cited by 25
Balafoutas, L., Celse, J., Karakostas, A., & Umashev, N. (2024) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2024.102327 · Cited by 24
Anthropologies of Global Maternal and Reproductive Health: From Policy Spaces to Sites of Practice
Wallace, L. J., MacDonald, M. E., & Storeng, K. T. (2022) · Global maternal and child health · doi:10.1007/978-3-030-84514-8 · Cited by 27
Schary, W., Paskali, F., Rentschler, S., Ruppert, C., Wagner, G. E., Steinmetz, I., Deigner, H., & Kohl, M. (2022) · Diagnostics · doi:10.3390/diagnostics12030589 · Cited by 25
“Also, the automatic documentation of all analysis steps, implemented in the application via R Markdown [19], allows for accurate reproducibility which is of high importance in biomedical research and clinical practice [20,21].”
MethodOpen access meta-analysis for psychotherapy research.
Baldwin, S. A., & Re, A. C. D. (2016) · Journal of Counseling Psychology · doi:10.1037/cou0000091 · Cited by 22
“Anything that appears to threaten a researcher’s opportunity to publish and receive funding, is likely to be met with resistance (Ioannidis, 2014).”
BackgroundResearch waste‐related editorial policies of leading dental journals: Situation 2018
Almaqrami, B. S., Hua, F., Liu, Y., & He, H. (2020) · Oral Diseases · doi:10.1111/odi.13257 · Cited by 22
“The reasonable updating of an existing systematic review with the results of the latest primary study may be justified in confirming previous results or to assess the generalizability of novel findings (Ioannidis, 2014).”
ResultPost-normal institutional identities: Quality assurance, reflexivity and ethos of care
Pereira, Â. G., & Saltelli, A. (2017) · Futures · doi:10.1016/j.futures.2016.11.009 · Cited by 27
Embracing Open Science and Transparency in Health Psychology
Hagger, M. S. (2019) · Health Psychology Review · doi:10.1080/17437199.2019.1605614 · Cited by 22
“The issues around replication and reproducibility of research findings has led to calls for greater transparency from researchers in the conduct, reporting, and dissemination of their results (Chambers, Feredoes, Muthukumaraswamy, & Etchells, 2014; Ioannidis, 2014; Laraway, Snycerski, Pradhan, & Huitema, 2019; Munafò…”
BackgroundMethodScience’s moral economy of repair: Replication and the circulation of reference
Penders, B., de Rijcke, S., & Holbrook, J. B. (2020) · Accountability in Research · doi:10.1080/08989621.2020.1720659 · Cited by 24
“The dominant response has been to identify, expose and expunge asmuch bias as possible, and to continue to uphold objectivity as the hallmark for proper science (Ioannidis 2005, 2014).”
BackgroundCharting the Constellation of Science Reform
Field, S. M. (2022) · University of Groningen research database (University of Groningen / Centre for Information Technology) · doi:10.33612/diss.229114775 · Cited by 31
Guidelines for standardizing and increasing the transparency in the reporting of biomedical research
Khan, A. M. (2017) · Journal of Thoracic Disease · doi:10.21037/jtd.2017.07.30 · Cited by 22
“Among all the research practices suggested to improve the reliability of medical research, improving and standardizing reporting is one of the key elements (4).”
BackgroundSteckler, T., Brose, K., Haas, M., Kas, M. J., Koustova, E., & Bespalov, A. (2015) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2015.05.011 · Cited by 22
“While those are issues that have a high impact on data reproducibility, they have the added advantage that they can be identi fi ed and potentially corrected by appropriate actions such as training and education (Ioannidis, 2014).”
Schuerbeek, P. V., Baeken, C., & Mey, J. D. (2016) · PLoS ONE · doi:10.1371/journal.pone.0153865 · Cited by 22
Research, regulatory, and clinical decision-making: the importance of scientific integrity
Wallach, J. D., Gonsalves, G., & Ross, J. S. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.021 · Cited by 25
Catalá-López, F., Alonso‐Arroyo, A., Page, M. J., Hutton, B., Ridao-López, M., Tabarés‐Seisdedos, R., Aleixandre‐Benavent, R., & Moher, D. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-022769 · Cited by 21
Signals Among Signals: Prioritizing Nongenetic Associations in Massive Data Sets
Manrai, A. K., Ioannidis, J. P. A., & Patel, C. J. (2019) · American Journal of Epidemiology · doi:10.1093/aje/kwz031 · Cited by 26
“example, to devise ways to enhance data and code sharing, as well as collaborative team science (23).”
BackgroundSelf-citationWrangling with Non-Standard Data
Eetu Mäkelä, A., Lagus, K., Lahti, L., Säily, T., Tolonen, M., Hämäläinen, M. K., Kaislaniemi, S., & Nevalainen, T. (2020) · Digital Humanities in the Nordic and Baltic Countries Publications · doi:10.5617/dhnbpub.11180 · Cited by 21
“In practical terms, we wish to highlight open science as a useful approach to solving the data problems described here [14,29].”
BackgroundSmalheiser, N. R., Graetz, E. E., Yu, Z., & Wang, J. (2021) · PLoS ONE · doi:10.1371/journal.pone.0243668 · Cited by 24
A Blood Test for Alzheimer’s Disease: Progress, Challenges, and Recommendations
Kiddle, S. J., Voyle, N., & Dobson, R. (2018) · Journal of Alzheimer s Disease · doi:10.3233/jad-179904 · Cited by 23
“12 John Ioannidis has provided general recommendations for researchers to improve the chance of true positive findings that we think are relevant: “large-scale collaborative research, replication culture, registration, sharing, reproducibility practices, better statistical methods, standardization of definitions and a…”
Failure to Replicate: Sound the Alarm.
Ioannidis, J. P. A. (2015) · PubMed Central · Cited by 22
Self-citationThe Conceptual, Cunning, and Conclusive Experiment in Psychology
Debrouwere, S., & Rosseel, Y. (2021) · Perspectives on Psychological Science · doi:10.1177/17456916211026947 · Cited by 23
“005), a bit more honest (preregistration), and a bit more diligent (replication, meta-analysis; Ioannidis, 2014; Nosek et al., 2018).”
Theater as Data: Computational Journeys into Theater Research
Varela, M. E. (2020) · University of Michigan Press eBooks · doi:10.3998/mpub.11667458 · Cited by 23
Estradiol does not affect spasms in the betamethasone‐ NMDA rat model of infantile spasms
Chachua, T., Grazia, P. D., Chern, C., Johnkutty, M., Hellman, B., Lau, H. A., Shakil, F., Daniel, M., Goletiani, C. J., Velı́šková, J., et al. (2016) · Epilepsia · doi:10.1111/epi.13434 · Cited by 20
Ulrich, R., Miller, J. O., & Erdfelder, E. (2018) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000319 · Cited by 21
“Moreover, this bias may be especially harmful in clinical and other applied research domains, and may promote the waste of public resources (Ioannidis, 2014).”
BackgroundComparison of veterinary drugs and veterinary homeopathy: part 2
Lees, P., Pelligand, L., Whiting, M. C., Chambers, D., Toutain, P., & Whitehead, M. L. (2017) · Veterinary Record · doi:10.1136/vr.104279 · Cited by 27
Meta-research: Evaluation and Improvement of Research Methods and Practices
Ioannidis, J. P. A. (2020) · Böhlau Verlag eBooks · doi:10.7767/9783205233008.101 · Cited by 20
Self-citationAbove a swamp: A theory of high-quality scientific production
Kiri, B., Lacetera, N., & Zirulia, L. (2018) · Research Policy · doi:10.1016/j.respol.2018.01.011 · Cited by 20
“The concerns that the excessive incentives toward producing compromise depth and reliability are often expressed together with a recommendation to provide more recognition for incremental and replicative work (Ioannidis, 2014; Schekman, 2013).”
Mouchabac, S., Adrien, V., Falala-Séchet, C., Bonnot, O., Maatoug, R., Millet, B., Peretti, C.-S., Bourla, A., & Ferreri, F. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.622506 · Cited by 21
Effects of psychedelics on neurogenesis and broader neuroplasticity: a systematic review
da Cruz, R. V. L., Leão, R. N., & Moulin, T. C. (2024) · Molecular Medicine · doi:10.1186/s10020-024-01013-4 · Cited by 22
“These results highlight the importance of improving quality of reporting for increasing reproducibility in biological research (Begley & Ioannidis, 2015; Ioannidis, 2014).”
Understanding of researcher behavior is required to improve data reliability
Wass, M. N., Ray, L., & Michaelis, M. (2019) · GigaScience · doi:10.1093/gigascience/giz017 · Cited by 22
Why Study the History of Neuroscience?
Brown, R. E. (2019) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2019.00082 · Cited by 20
“Ioannidis (2014) has presented 12 steps for improving the reliability and validity of published research.”
BackgroundNational priorities for perioperative research in South Africa
Biccard, B. M., Alphonsus, C. S., Bishop, D. G., Cronjé, L., Kluyts, H., Kusel, B., Maswime, S., Oodit, R., Reed, A., Torborg, A. M., et al. (2016) · South African Medical Journal · doi:10.7196/samj.2016.v106i5.10269 · Cited by 21
Null Findings, Replications and Preregistered Studies in Business Ethics Research
Roloff, J., & Zyphur, M. J. (2018) · Journal of Business Ethics · doi:10.1007/s10551-018-3864-8 · Cited by 20
“The result is that some academics estimate that 85% of research resources across the sciences are wasted on studies that provide unreliable results (Ioannidis 2014), contributing to a crisis of trust in science.”
BackgroundParkinson’s disease research: adopting a more human perspective to accelerate advances
Marshall, L. J., & Willett, C. (2018) · Drug Discovery Today · doi:10.1016/j.drudis.2018.09.010 · Cited by 19
Availability of Research Data in High-Impact Addiction Journals with Data Sharing Policies
Gorman, D. M. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-020-00203-7 · Cited by 20
Pandis, N., Fleming, P. S., Koletsi, D., & Hopewell, S. (2016) · Trials · doi:10.1186/s13063-016-1713-6 · Cited by 18
Experiences in integrated data and research object publishing using GigaDB
Edmunds, S., Li, P., Hunter, C., Zhe, X. S., Davidson, R., Nogoy, N., & Goodman, L. (2016) · International Journal on Digital Libraries · doi:10.1007/s00799-016-0174-6 · Cited by 20
“Systematically testing the reproducibility of papers across various research fields, Ioannidis and others determined that the proportion of published research findings are false or exaggerated, and an estimated 85% of research resources are wasted because of this [33].”
BackgroundData Citation in Neuroimaging: Proposed Best Practices for Data Identification and Attribution
Honor, L. B., Haselgrove, C., Frazier, J. A., & Kennedy, D. N. (2016) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2016.00034 · Cited by 19
“With burgeoning concerns about the reproducibility of science, the urgency for developing “best practices” for generating and including the data sources associated with a scientific publication has grown dramatically in the recent years (Ioannidis, 2005, 2014; Button et al., 2013)1.”
BackgroundA Systematic Review of In Vitro and In Vivo Radio Frequency Exposure Methods
Hansen, J. W., Swartz, E., Cleveland, J., Asif, S. M., Brooks, B. D., Braaten, B. D., & Ewert, D. L. (2019) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2019.2912023 · Cited by 18
Kochunov, P., Dickie, E. W., Viviano, J. D., Turner, J. A., Kingsley, P. B., Jahanshad, N., Thompson, P. M., Ryan, M. C., Fieremans, E., Novikov, D. S., et al. (2017) · Human Brain Mapping · doi:10.1002/hbm.23900 · Cited by 18
“A challenge for evaluating regional WM deficits is the need for statistically powerful and representative samples that can be difficult to collect at a single site (Ioannidis 2014; Jahanshad et al., 2013).”
Ogino, S., Jhun, I., Mata, D. A., Soong, T. R., Hamada, T., Liu, L., Nishihara, R., Giannakis, M., Cao, Y., Manson, J. E., et al. (2017) · npj Precision Oncology · doi:10.1038/s41698-017-0042-x · Cited by 18
McCrae, N., & Purssell, E. (2015) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12448 · Cited by 17
Metadata
- Authors
- Ioannidis, J. P. A.
- Affiliations
- Stanford Medicine (US)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-10-21
- Volume
- 11
- Issue
- 10
- Pages
- e1001747
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC4204808
- OpenAlex
- W2077181076
- Cited by
- 760
- References
- 75
- Citations by year
- 2026: 26 · 2025: 31 · 2024: 37 · 2023: 50 · 2022: 51 · 2021: 44 · 2020: 51 · 2019: 68 · 2018: 93 · 2017: 108 · 2016: 111 · 2015: 80 · 2014: 10
- FWCI
- 192.35
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Laura and John Arnold Foundation
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- research integrity, reproducibility, research credibility, research registration, statistical thresholds, peer review, research reporting, research incentives, publication incentives, research waste
- MeSH terms
- Publications, Publishing, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.