How to Make More Published Research True
Open accessThe DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.
Abstract
Currently, many published research findings are false or exaggerated, and an estimated 85% of research resources are wasted.N To make more published research true, practices that have improved credibility and efficiency in specific fields may be transplanted to others which would benefit from them-possibilities include the adoption of large-scale collaborative research; replication culture; registration; sharing; reproducibility practices; better statistical methods; standardization of definitions and analyses; more appropriate (usually more stringent) statistical thresholds; and improvement in study design standards, peer review, reporting and dissemination of research, and training of the scientific workforce. N Selection of interventions to improve research practices requires rigorous examination and experimental testing whenever feasible.N Optimal interventions need to understand and harness the motives of various stakeholders who operate in scientific research and who differ on the extent to which they are interested in promoting publishable, fundable, translatable, or profitable results.N Modifications need to be made in the reward system for science, affecting the exchange rates for currencies (e.g., publications and grants) and purchased academic goods (e.g., promotion and other academic or administrative power) and introducing currencies that are better aligned with translatable and reproducible research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Accordingly, there is growing recognition that data availability is a critical feature of an efficient, progressive, and ultimately self-correcting scientific ecosystem that generates credible findings [1–3].”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Some journals have heeded the growing calls for adoption of transparent and reproducible research practices and introduced new policies to implement change [1,2].”
From Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition (2018) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, protocol registration is applicable to and recommended for all studies and is supported by various platforms, such as the Open Science Framework [3, 29, 35–37].”
From Assessing transparency practices in dental randomized controlled trials (2024) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Well-planned, well-executed, and well-reported RCTs, which embrace transparent and open science practices, enhance the likelihood that results are accurate, reliable, and applicable [2, 3].”
From Assessing transparency practices in dental randomized controlled trials (2024) · cites this paper as [3]
Experimental — accuracy not yet measuredIn the citing paper
“Consider that the stakeholders who fund and support science may have incentives that are not always aligned with those of active researchers [82].”
From The natural selection of bad science (2016) · cites this paper as [82]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Self-citationConfirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267 · cited 2×
“powerful in scientific processes [8,9].”
BackgroundSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 2×
“Many new proposed associations and/or effects are false or grossly exaggerated [2,3], and translation of knowledge into useful applications is often slow and potentially inefficient [4].”
BackgroundSelf-citationClinical Practice Guidelines We Can Trust
Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Board on Health Care Services, & Institute of Medicine (2011) · National Academies Press eBooks · doi:10.17226/13058 · Cited by 3,081
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 1×
“Pharmaceutical industry teams have championed reproducibility in preclinical research [56,57] because they depend on pre-clinical academic investigations accurately pinpointing useful drug targets.”
BackgroundBelieve it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · cited 1×
“Pharmaceutical industry teams have championed reproducibility in preclinical research [56,57] because they depend on pre-clinical academic investigations accurately pinpointing useful drug targets.”
BackgroundEvidence based medicine: a movement in crisis?
Greenhalgh, T., Howick, J., Maskrey, N., & for the Evidence Based Medicine Renaissance Group (2014) · BMJ · doi:10.1136/bmj.g3725 · Cited by 1,766
Scientific method: Statistical errors
Nuzzo, R. (2014) · Nature · doi:10.1038/506150a · Cited by 1,536
Why Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495 · cited 1×
“Many new proposed associations and/or effects are false or grossly exaggerated [2,3], and translation of knowledge into useful applications is often slow and potentially inefficient [4].”
BackgroundSelf-citation
Research works printed but not linked: 1
- R Al-Shahi Salman, 2014, Increasing value and reducing waste in biomedical research regulation and management, Lancet, 383, 176, 10.1016/s0140-6736(13)62297-7 · DOI · Paper page
Cited by
Items 101–200 of 802
Confidence crisis of results in biomechanics research
Knudson, D. V. (2017) · Sports Biomechanics · doi:10.1080/14763141.2016.1246603 · Cited by 87
“The following four solutions are based on this research and recent recommendations (Button et al., 2013; Ioannidis, 2014; Nosek et al., 2012; Simmons et al., 2011).”
MethodA “datathon” model to support cross-disciplinary collaboration
Aboab, J., Celi, L. A., Charlton, P., Feng, M., Ghassemi, M., Marshall, D. C., Mayaud, L., Naumann, T., McCague, N., Paik, K. E., et al. (2016) · Science Translational Medicine · doi:10.1126/scitranslmed.aad9072 · Cited by 82
“If half of reported research is indeed unreliable (3), those who read the literature are not only wasting half their time, they are actually poisoning half their time with falsehoods.”
BackgroundRenn, O., Baghramian, M., Capaccioli, M., de Rijcke, S., Drotner, K., Dubertret, L., Irwin, A., Luty, T., Makarow, M., Moberg, C., et al. (2019) · Leiden Repository (Leiden University) · doi:10.26356/masos · Cited by 85
“…institutional and socio-cultural dimensions of evidence have become more evident since the 2000s, as also basic disciplinary science was seen by many as becoming mired in problems of reproducibility (Ioannidis, 2014), fraud (Benessia et al., 2016) and lack of trust (Benessia et al., 2016).”
BackgroundHardwicke, T. E., & Ioannidis, J. P. A. (2018) · PLOS ONE · doi:10.1371/journal.pone.0201856 · Cited by 78
Self-citationIs Most Published Research Really False?
Leek, J. T., & Jager, L. R. (2016) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-060116-054104 · Cited by 90
Expanding the CONSORT Figure: Increasing Transparency in Reporting on External Validity
Glasgow, R. E., Huebschmann, A. G., & Brownson, R. C. (2018) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2018.04.044 · Cited by 76
Routine outcome monitoring: Coming of age—With the usual developmental challenges.
Wampold, B. E. (2015) · Psychotherapy · doi:10.1037/pst0000037 · Cited by 83
“…in the future, given how most effects decline over years in the sciences, and in psychology in particular (Yong, 2012), including the effect of CBT as a treatment for depression (Johnsen & Friborg, 2015), may not represent “truth” (Ioannidis, 2014), and may not be applicable in practice settings.”
BackgroundPhase II clinical trials of anti–amyloid β antibodies: When is enough, enough?
Gold, M. (2017) · Alzheimer's & Dementia: Translational Research & Clinical Interventions · doi:10.1016/j.trci.2017.04.005 · Cited by 75
Prevention of orthodontic enamel demineralization: A systematic review with meta‐analyses
Tasios, T., Papageorgiou, S. N., Papadopoulos, M. A. A. P., Τσάπας, Α., & Haidich, A. (2019) · Orthodontics and Craniofacial Research · doi:10.1111/ocr.12322 · Cited by 80
Striving for transparent and credible research: practical guidelines for behavioral ecologists
Ihle, M., Winney, I. S., Krystalli, A., & Croucher, M. (2017) · Behavioral Ecology · doi:10.1093/beheco/arx003 · Cited by 76
Ten Simple Rules for Reducing Overoptimistic Reporting in Methodological Computational Research
Boulesteix, A.-L. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004191 · Cited by 74
Hoffmann, S., Schönbrodt, F. D., Elsas, R., Wilson, R., Strasser, U., & Boulesteix, A. (2021) · Royal Society Open Science · doi:10.1098/rsos.201925 · Cited by 88
“A focus on cumulative evidence can be found in calls for replications as postpublication quality control [96, 97, 17] and in the proposal of Benjamin et al.”
BackgroundMeta‐analysis of post‐release fishing mortality in apex predatory pelagic sharks and white marlin
Musyl, M. K., & Gilman, E. L. (2019) · Fish and Fisheries · doi:10.1111/faf.12358 · Cited by 82
Psychedelic Research and the Need for Transparency: Polishing Alice’s Looking Glass
Petranker, R., Anderson, T., & Farb, N. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.01681 · Cited by 69
“Describing all the ways in which long-term research goals can be undermined by short-term Questionable Research Practices (QRPs) is outside the scope of this paper; we recommend the work of others (Ioannidis, 2005, 2012, 2014, 2016; Munafò et al., 2017) as a primer on the greatest concerns.”
BackgroundDirnagl, U., Duda, G. N., Grainger, D. W., Reinke, P., & Roubenoff, R. (2022) · Advanced Drug Delivery Reviews · doi:10.1016/j.addr.2022.114118 · Cited by 72
“Despite obvious ethical implications involving human quality of life, clinical research also has questionable quality, reliability and replicability challenges [24,33,104].”
BackgroundHuman Challenge Studies in Endemic Settings: Ethical and Regulatory Issues
Jamrozik, E., & Selgelid, M. J. (2020) · SpringerBriefs in ethics · doi:10.1007/978-3-030-41480-1 · Cited by 68
Generalizability is not optional: insights from a cross-cultural study of social discounting
Tiokhin, L., Hackman, J., Munira, S., Jesmin, K., & Hruschka, D. (2019) · Royal Society Open Science · doi:10.1098/rsos.181386 · Cited by 57
The evolution of instrument selection for inclusion in core outcome sets at OMERACT: Filter 2.2
Maxwell, L. J., Beaton, D., Boers, M., D’Agostino, M. A., Conaghan, P. G., Grosskleg, S., Shea, B., Bingham, C. O., Boonen, A. E. R. C. H., Christensen, R., et al. (2021) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2021.08.011 · Cited by 71
Tamagusko, T., & Ferreira, A. (2023) · Infrastructures · doi:10.3390/infrastructures8120170 · Cited by 72
An Assessment of the Cox Proportional Hazards Regression Model for Epidemiologic Studies
Moolgavkar, S. H., Chang, E. T., Watson, H. N., & Lau, E. (2017) · Risk Analysis · doi:10.1111/risa.12865 · Cited by 67
Diffusion-Weighted Imaging in Mild Traumatic Brain Injury: A Systematic Review of the Literature
Lindsey, H. M., Hodges, C. B., Greer, K. M., Wilde, E. A., & Merkley, T. L. (2021) · Neuropsychology Review · doi:10.1007/s11065-021-09485-5 · Cited by 75
Transgenerational inheritance of metabolic disease
Stegemann, R., & Buchner, D. A. (2015) · Seminars in Cell & Developmental Biology · doi:10.1016/j.semcdb.2015.04.007 · Cited by 59
“These apparently positive outcomes may instead be due to chance observations as the result of underpowered experimental designs [122].”
BackgroundIoannidis, J. P. A. (2015) · JAMA · doi:10.1001/jama.2014.17662 · Cited by 64
Self-citationGoing from evidence to recommendations: Can GRADE get us there?
Mercuri, M., Baigrie, B. S., & Upshur, R. E. G. (2018) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12857 · Cited by 71
Doberstein, C. (2016) · Policy Studies Journal · doi:10.1111/psj.12166 · Cited by 66
A Stronger Post-Publication Culture Is Needed for Better Science
Bastian, H. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001772 · Cited by 66
“Both improving research quality [1,2] and reducing waste in science [3] require a”
BackgroundHardwicke, T. E., Mathur, M. B., MacDonald, K. E., Nilsonne, G., Banks, G. C., Kidwell, M., Hofelich Mohr, A., Clayton, E., Yoon, E. J., Tessler, M. H., et al. (2018) · doi:10.31222/osf.io/39cfb · Cited by 52
Antunes, B. A., & Hill, D. R. C. (2024) · Computer Science Review · doi:10.1016/j.cosrev.2024.100655 · Cited by 78
“This rate may seem relatively optimistic considering (Ioannidis, 2015), who claimed that 85% of research funding was being wasted.”
BackgroundEcological brain: reframing the study of human behaviour and cognition
Vigliocco, G., Convertino, L., Felice, S. D., Gregorians, L., Kewenig, V. N., Mueller, M. A. E., Veselič, S., Musolesi, M., Hudson‐Smith, A., Tyler, N., et al. (2024) · Royal Society Open Science · doi:10.1098/rsos.240762 · Cited by 71
The reproducibility of biomedical research: Sleepers awake!
Bustin, S. A. (2014) · Biomolecular Detection and Quantification · doi:10.1016/j.bdq.2015.01.002 · Cited by 56
Advancing Science with VGI: Reproducibility and Replicability of Recent Studies using VGI
Ostermann, F., & Granell, C. (2016) · Transactions in GIS · doi:10.1111/tgis.12195 · Cited by 61
“With an increased level of competitiveness in the academic world, there are voices (Ioannidis 2014) that express concern over the possible increase of such approaches.”
Protect us from poor-quality medical research
ESHRE Capri Workshop Group, Ioannidis, J. P. A., Bhattacharya, S., Evers, J. L. H., der Veen, F. V., Somigliana, E., Barratt, C. L. R., Bontempi, G., Baird, D. T., Crosignani, P., et al. (2018) · Human Reproduction · doi:10.1093/humrep/dey056 · Cited by 59
Self-citationRIPOSTE: a framework for improving the design and analysis of laboratory-based research
Masca, N. G., Hensor, E. M., Cornelius, V. R., Buffa, F. M., Marriott, H. M., Eales, J. M., Messenger, M. P., Anderson, A. E., Boot, C., Bunce, C., et al. (2015) · eLife · doi:10.7554/elife.05519 · Cited by 56
Citicoline for treating people with acute ischemic stroke
Martí-Carvajal, A. J., Valli, C., Martí-Amarista, C. E., Solà, I., Martí‐Fábregas, J., & Bonfill, X. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013066.pub2 · Cited by 68
“This could be for many reasons: a lack of standardization of definitions and analyses of the adverse events (Ioannidis 2014), the shortage of homogeneity and plenty of variability in reporting harm data (Ioannidis 2009; Pitrou 2009), the diQerent times for conducting the trials (1988 to 2017), and…”
BackgroundCashin, A. G., Bagg, M. K., Richards, G. C., Toomey, E. C., McAuley, J. H., & Lee, H. (2020) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2019-111296 · Cited by 66
Murtagh, M. J., Blell, M. T., Butters, O. W., Cowley, L., Dove, E. S., Goodman, A. T., Griggs, R. L., Hall, A., Hallowell, N., Kumari, M., et al. (2018) · Human Genomics · doi:10.1186/s40246-018-0154-6 · Cited by 64
The Brazilian Reproducibility Initiative
Amaral, O. B., Neves, K., Wasilewska-Sampaio, A. P., & Carneiro, C. F. (2019) · eLife · doi:10.7554/elife.41602 · Cited by 49
The Weak Spots in Contemporary Science (and How to Fix Them)
Wicherts, J. M. (2017) · Animals · doi:10.3390/ani7120090 · Cited by 60
Lozada-Martínez, I. D., Rodado, D. N., Guevara, D. M., Cruz-Soto, H. S., Sanchez-Echeverry, M. P., & Liscano, Y. (2025) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2025.1497280 · Cited by 58
“…combined with concerns regarding low-quality research, necessitates institutional engagement in meta-research to uphold the credibility of scientific outputs and reinforce the social responsibility of academia (Ioannidis, 2018; Ioannidis et al., 2015; Ioannidis, 2014; Begley and Ioannidis, 2015).”
InfluentialImproving the rigor of psychophysiology research
Baldwin, S. A. (2016) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2016.04.006 · Cited by 57
Open science and modified funding lotteries can impede the natural selection of bad science
Smaldino, P. E., Turner, M. A., & Kallens, P. C. (2019) · Royal Society Open Science · doi:10.1098/rsos.190194 · Cited by 66
Smith, M. M., Clarke, E., & Little, C. B. (2016) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2016.10.016 · Cited by 55
Outcome reporting bias in randomized-controlled trials investigating antipsychotic drugs
Lancee, M., Lemmens, C. M. C., Kahn, R. S., Vinkers, C. H., & Luykx, J. J. (2017) · Translational Psychiatry · doi:10.1038/tp.2017.203 · Cited by 51
Seshia, S. S., Young, G. B., Makhinson, M., Smith, P. A., Stobart, K., & Croskerry, P. (2017) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12847 · Cited by 61
Harnessing QbD, Programming Languages, and Automation for Reproducible Biology
Sadowski, M. I., Grant, C., & Fell, T. S. (2015) · Trends in biotechnology · doi:10.1016/j.tibtech.2015.11.006 · Cited by 61
A Replication by Any Other Name: A Systematic Review of Replicative Intervention Studies
Cook, B. G., Collins, L. W., Cook, S. C., & Cook, L. (2016) · Remedial and Special Education · doi:10.1177/0741932516637198 · Cited by 56
“Perhaps foremost, special education researchers should work toward adopting a replication culture that clearly values and rewards replication research (Ioannidis, 2014).”
BackgroundAdhikari, B. M., Jahanshad, N., Shukla, D., Glahn, D. C., Blangero, J., Fox, P. T., Reynolds, R. C., Cox, R. W., Fieremans, E., Veraart, J., et al. (2018) · Human Brain Mapping · doi:10.1002/hbm.24331 · Cited by 55
“As has been noted for other phenotypes from structure MRI and diffusion MRI, very large, highly powered, and representative samples are difficult to collect at a single site (Ioannidis, 2014; Jahanshad et al., 2013), although there are some notable exceptions, for example, the UK Biobank (Elliott et al.”
BackgroundPerception and emotions: On the relationships between stress and olfaction
Bombail, V. (2018) · Applied Animal Behaviour Science · doi:10.1016/j.applanim.2018.12.013 · Cited by 56
Zebrafish (Danio rerio) meets bioethics: the 10Rs ethical principles in research
Canedo, A., Saiki, P. T. O., Santos, A. L., Carneiro, K. D. S., de Souza, A. M., Qualhato, G., Brito, R. D. S., Mello-Andrade, F., & Rocha, T. L. (2022) · Ciência Animal Brasileira · doi:10.1590/1809-6891v22e-70884 · Cited by 58
Hodges, C. B., Stone, B. M., Johnson, P. K., Carter, J. H., Sawyers, C. K., Roby, P. R., & Lindsey, H. M. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01932-2 · Cited by 60
“To address recent concerns over replication (Open Science Collaboration, 2015; 2012), the scientific community is calling for transparency in research through open science, which may aid in both reducing the number of false positives published in the literature and increasing reproducibility and replicability by encou…”
Lozada-Martínez, I. D., Hernández-Páez, D. A., Moreno, O. F., Picón‐Jaimes, Y. A., & Bermúdez, V. (2025) · Publications · doi:10.3390/publications13010007 · Cited by 57
“Science has become a global industry, with over 1.5 million scientific articles published only on biomedicine and life sciences yearly (González-Márquez et al., 2024); however, the certainty and applicability of major discoveries are still a concern (Ioannidis, 2014).”
InfluentialQueen pheromones and reproductive division of labor: a meta-analysis
Holman, L. (2018) · Behavioral Ecology · doi:10.1093/beheco/ary023 · Cited by 74
Waltemath, D., & Wolkenhauer, O. (2016) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2016.2555481 · Cited by 49
“training of the scientific workforce can help [86].”
BackgroundAssessment of automatic segmentation of teeth using a watershed-based method
Galibourg, A., Dumoncel, J., Telmon, N., Calvet, A., Michetti, J., & Maret, D. (2017) · Dentomaxillofacial Radiology · doi:10.1259/dmfr.20170220 · Cited by 49
Buljan, I., Barać, L., & Marušić, A. (2018) · Accountability in Research · doi:10.1080/08989621.2018.1463162 · Cited by 52
“…research are already available (e.g., data sharing, trial registration, improvement in research training, and redefinition of excellence criteria) (Ioannidis 2014), currently there is little evidence that interventions aiming to resolve the issue of research misconduct are effective (Marusic et…”
BackgroundChhin, C. S., Taylor, K. A., & Wei, W. S. (2018) · Educational Researcher · doi:10.3102/0013189x18788047 · Cited by 54
“Journal editors, peer reviewers, professional and scientific societies, and funding agencies are considered to be significant gatekeepers and stakeholders that can help or hinder replication research (Ioannidis, 2014).”
BackgroundCompetition for priority harms the reliability of science, but reforms can help
Tiokhin, L., Yan, M., & Morgan, T. J. H. (2021) · Nature Human Behaviour · doi:10.1038/s41562-020-01040-1 · Cited by 53
Support Your Data: A Research Data Management Guide for Researchers
Borghi, J., Abrams, S. L., Lowenberg, D., Simms, S., & Chodacki, J. (2018) · Research Ideas and Outcomes · doi:10.3897/rio.4.e26439 · Cited by 56
“Complementing calls for improved data management and more widespread data sharing by transparency and reproducibility-related initiatives within the research community (Ioannidis 2014, Munafò et al. 2017), RDM has increasingly become a focus for academic libraries.”
BackgroundKasiri-Martino, H., & Bright, P. (2015) · Manual Therapy · doi:10.1016/j.math.2015.09.003 · Cited by 47
Gammon, K. (2018) · Nature Medicine · doi:10.1038/nm0418-378 · Cited by 46
Effects of MVA85A vaccine on tuberculosis challenge in animals: systematic review
Kashangura, R., Sena, E. S., Young, T., & Garner, P. (2015) · International Journal of Epidemiology · doi:10.1093/ije/dyv142 · Cited by 46
“As we set up to carry out an independent systematic review of MVA85A vaccine trials in children, there was considerable debate in the literature that highlighted the potential gains from optimizing the design, conduct and analysis of biomedical research to avoid misleading findings and wasting resources.(3,4) We there…”
MethodDyadic Interdependence in Non-spousal Caregiving Dyads’ Wellbeing: A Systematic Review
Ferraris, G., Dang, S., Woodford, J., & Hagedoorn, M. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.882389 · Cited by 45
Who Believes in the Storybook Image of the Scientist?
Veldkamp, C. L. S., Hartgerink, C. H. J., van Assen, M. A. L. M., & Wicherts, J. M. (2017) · Accountability in Research · doi:10.1080/08989621.2016.1268922 · Cited by 47
“For example, the scientific community was found to be the second most trusted institution in the United States (Smith and Son 2013), and in the United Kingdom, the general public believed that scientists meet the expectations of honesty, ethical behavior, and open-mindedness (Ipsos MORI 2014).”
BackgroundModelling plankton ecosystems in the meta-omics era. Are we ready?
Stec, K., Caputi, L., Buttigieg, P. L., D’Alelio, D., Ibarbalz, F. M., Sullivan, M. B., Chaffron, S., Bowler, C., d’Alcalà, M. R., & Iudicone, D. (2017) · Marine Genomics · doi:10.1016/j.margen.2017.02.006 · Cited by 49
Hollander, Z., DeMarco, M. L., Sadatsafavi, M., McManus, B. M., Ng, R. T., & Sin, D. D. (2016) · CHEST Journal · doi:10.1016/j.chest.2016.09.012 · Cited by 44
Wucherpfennig, F., Rubel, J. A., Hofmann, S. G., & Lutz, W. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000263 · Cited by 46
Kapadia, M., Xu, J., & Šakić, B. (2016) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2016.05.016 · Cited by 47
QInfer: Statistical inference software for quantum applications
Granade, C., Ferrie, C., Hincks, I., Casagrande, S., Alexander, T., Gross, J. A., Kononenko, M., & Sanders, Y. R. (2017) · Quantum · doi:10.22331/q-2017-04-25-5 · Cited by 56
“Ensuring the reproducibility of research is critical for evaluating the extent to which statistical and methodological errors undermine the credibility of published research [11].”
BackgroundEnzymeML—a data exchange format for biocatalysis and enzymology
Range, J., Halupczok, C., Lohmann, J., Swainston, N., Kettner, C., Bergmann, F., Weidemann, A., Wittig, U., Schnell, S. D., & Pleiss, J. (2021) · FEBS Journal · doi:10.1111/febs.16318 · Cited by 44
“Scientific journals are responding accordingly [12], and there has been a recommendation to modify the academic reward system by recognizing scientists who aligned with best practices for reproducible research [13].”
BackgroundVable, A. M., Diehl, S. F., & Glymour, M. M. (2021) · American Journal of Epidemiology · doi:10.1093/aje/kwab092 · Cited by 46
Ehring, T. W., Limburg, K., Kunze, A. E., Wittekind, C. E., Werner, G. G., Wolkenstein, L., Guzey, M., & Cludius, B. (2022) · Clinical Psychology Review · doi:10.1016/j.cpr.2022.102163 · Cited by 49
Aligning physiology with psychology: Translational neuroscience in neuropsychiatric drug discovery
McArthur, R. A. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.02.004 · Cited by 42
“…contributing the lack of reproducibility, perhaps he most pernicious source of variance is the level of statistial expertise by many biological investigators compounded by he widespread use of automated statistical programs (Ioannidis, 005, 2007; Ioannidis, 2014; Ioannidis et al., 2014).”
A manifesto for reproducible science.
Munafo, M. R., Nosek, B. A., Bishop, D., Button, K. S., Chambers, C., du Sert, N. P., Simonsohn, U., Wagenmakers, E., Ware, J. J., & Ioannidis, J. P. A. (2024) · American Psychological Association eBooks · doi:10.1037/0000409-041 · Cited by 42
Self-citationMiller, J. O., & Ulrich, R. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616649170 · Cited by 43
“…rewards for their productivity using a currency system that accurately reflected the value of their research outputs to the field as a whole (e.g., Ioannidis, 2014, Table 2; Kreiman & Maunsell, 2011), but there is broad agreement that this is not the case at present (e.g., Bakker et al., 2012;…”
BackgroundMethodInfluentialTransparent, Open, and Reproducible Prevention Science
Grant, S., Wendt, K. E., Leadbeater, B. J. R., Supplee, L. H., Mayo‐Wilson, E., Gardner, F. E. M., & Bradshaw, C. P. (2022) · Prevention Science · doi:10.1007/s11121-022-01336-w · Cited by 47
“Teams should respect Indigenous Peoples’ rights to control, access, and govern their data 1 3 social determinants of this issue are selective non-reporting, research misconduct, and misaligned incentives in the scientific ecosystem (Ioannidis, 2014).”
BackgroundGoodman, K. W. (2015) · Cambridge University Press eBooks · doi:10.1017/cbo9781139600330 · Cited by 53
FAIR SCI Ahead: The Evolution of the Open Data Commons for Pre-Clinical Spinal Cord Injury Research
Fouad, K., Bixby, J. L., Callahan, A., Grethe, J. S., Jakeman, L. B., Lemmon, V., Magnuson, D. S. K., Martone, M. E., Nielson, J. L., Schwab, J. M., et al. (2020) · Journal of Neurotrauma · doi:10.1089/neu.2019.6674 · Cited by 45
Sample size estimation revisited
Abt, G., Boreham, C. A., Davison, G. W., Jackson, R. C., Jobson, S., Wallace, E. S., & Williams, M. R. F. (2025) · Journal of Sports Sciences · doi:10.1080/02640414.2025.2499403 · Cited by 46
“Many have outlined the hypercompetitive nature and perverse incentives that now drive researchers in all fields (Edwards & Roy, 2017; Ioannidis, 2014; Munafò et al., 2017), and it is likely that if we are to improve research practice and reporting in our discipline, that change will be required…”
Advances in Exercise, Fitness, and Performance Genomics in 2014
Loos, R. J. F., Hagberg, J. M., Përusse, L., Roth, S. M., Sarzynski, M. A., Wolfarth, B., Rankinen, T., & Bouchard, C. (2015) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000000645 · Cited by 51
Shifting the Level of Selection in Science
Tiokhin, L., Panchanathan, K., Smaldino, P. E., & Lakens, D. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916231182568 · Cited by 44
“…this point, noting that “the extent to which these [reforms] can be expected to improve the efficiency and reliability of science remains unknown” (Ioannidis, 2014, p. 5) and that new evaluation criteria “need to be studied in terms of . . . the kind of systems needed to implement them their…”
BackgroundEkkekakis, P., Swinton, P. A., & Tiller, N. B. (2023) · Sports Medicine · doi:10.1007/s40279-023-01880-7 · Cited by 40
Scandola, M., Aglioti, S. M., Lazzeri, G., Avesani, R., Ionta, S., & Moro, V. (2020) · Scientific Reports · doi:10.1038/s41598-020-62080-1 · Cited by 38
Semahegn, A., Manyazewal, T., Hanlon, C., Getachew, E., Fekadu, B., Assefa, E., Kassa, M., Hopkins, M. M., Woldehanna, T., Davey, G., et al. (2023) · Health Research Policy and Systems · doi:10.1186/s12961-023-01084-5 · Cited by 37
The Ethical Challenges of Animal Research: Honoring Henry Beecher’s Approach to Moral Problems
Ferdowsian, H. R., & Gluck, J. P. (2015) · Cambridge Quarterly of Healthcare Ethics · doi:10.1017/s0963180115000067 · Cited by 45
Smith, S. M., Fava, M., Jensen, M. P., Mbowe, O. B., McDermott, M. P., Turk, D. C., & Dworkin, R. H. (2020) · Pain · doi:10.1097/j.pain.0000000000001849 · Cited by 42
Sports genetics moving forward: lessons learned from medical research
Mattsson, C. M., Wheeler, M. T., Waggott, D. M., Caleshu, C. A., & Ashley, E. A. (2016) · Physiological Genomics · doi:10.1152/physiolgenomics.00109.2015 · Cited by 44
CaReL Guidelines: A Consensus-Based Guideline on Case Reports and Literature Review (CaReL)
Prasad, S., Nassar, M., Azzam, A. Y., José, F. G.-M.-S., Jamee, M., Sliman, R. K. A., Evola, G., Mustafa, A. M., Abdullah, H. O., Abdalla, B. A., et al. (2024) · Barw Medical Journal · doi:10.58742/bmj.v2i2.89 · Cited by 95
The credibility crisis in research: Can economics tools help?
Gall, T., Ioannidis, J. P. A., & Maniadis, Z. (2017) · PLoS Biology · doi:10.1371/journal.pbio.2001846 · Cited by 38
Self-citationMonsarrat, P., & Vergnes, J. (2017) · GigaScience · doi:10.1093/gigascience/gix121 · Cited by 37
Boswell, J. F. (2019) · Psychotherapy Research · doi:10.1080/10503307.2019.1686192 · Cited by 35
Publication Bias in Methodological Computational Research
Boulesteix, A., Stierle, V., & Hapfelmeier, A. (2015) · Cancer Informatics · doi:10.4137/cin.s30747 · Cited by 42
Neural Adaptations Associated with Interlimb Transfer in a Ballistic Wrist Flexion Task
Ruddy, K. L., Rudolf, A. K., Kalkman, B. M., King, M. B., Daffertshofer, A., Carroll, T. J., & Carson, R. G. (2016) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2016.00204 · Cited by 36
“Motivated by the benefits of replication in decreasing the likelihood of false positive findings (Ioannidis, 2014) the latter experiment included the two visual feedback conditions from Experiment 1 for which the largest difference in behavioral outcomes was obtained.”
BackgroundMethodThe Replication Crisis in Epidemiology: Snowball, Snow Job, or Winter Solstice?
Lash, T. L., Collin, L. J., & Dyke, M. E. V. (2018) · Current Epidemiology Reports · doi:10.1007/s40471-018-0148-x · Cited by 39
Marton, R. M., & Ioannidis, J. P. A. (2018) · Stem Cells Translational Medicine · doi:10.1002/sctm.18-0088 · Cited by 36
Self-citationde Arruda, G. T., Terwee, C. B., Elsman, E. B. M., Ávila, M. A., Gagnier, J. J., Mokkink, L. B., PROM Reporting Group, Dibai-Filho, A. V., Firth, A. D., Mehdipour, A., et al. (2025) · Quality of Life Research · doi:10.1007/s11136-025-03949-4 · Cited by 38
“Adequate reporting is essential because it ensures that studies provide sufficient information, enabling end-users (e.g., researchers, clinicians, patients, and policy makers) to understand the methods used and to interpret the results [2].”
Repeat: a framework to assess empirical reproducibility in biomedical research
McIntosh, L. D., Juehne, A., Hudson‐Vitale, C., Liu, X., Alcoser, R., Lukas, J. C., & Evanoff, B. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0377-6 · Cited by 35
Reproducibility of antimicrobial test methods
Parker, A. E., Hamilton, M. A., & Goeres, D. M. (2018) · Scientific Reports · doi:10.1038/s41598-018-30282-3 · Cited by 33
Planning and Conducting Clinical Research: The Whole Process
Chew, B. (2019) · Cureus · doi:10.7759/cureus.4112 · Cited by 39
Methods for training collaborative biostatisticians
Pomann, G.-M., Boulware, L. E., Cayetano, S. M., Desai, M., Enders, F. T., Gallis, J. A., Gelfond, J., Grambow, S. C., Hanlon, A. L., Hendrix, A., et al. (2021) · Journal of Clinical and Translational Science · doi:10.1017/cts.2020.518 · Cited by 32
Synthetic Biology: fostering the cyber-biological revolution
Peccoud, J. (2016) · Synthetic Biology · doi:10.1093/synbio/ysw001 · Cited by 37
Metadata
- Authors
- Ioannidis, J. P. A.
- Affiliations
- Stanford Medicine (US)
- Corresponding author
- Ioannidis, John P. A.
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-10-21
- Volume
- 11
- Issue
- 10
- Pages
- e1001747
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC4204808
- OpenAlex
- W2077181076
- Cited by
- 760
- References
- 75
- Citations by year
- 2026: 26 · 2025: 31 · 2024: 37 · 2023: 50 · 2022: 51 · 2021: 44 · 2020: 51 · 2019: 68 · 2018: 93 · 2017: 108 · 2016: 111 · 2015: 80 · 2014: 10
- FWCI
- 192.35
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Laura and John Arnold Foundation
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- research integrity, reproducibility, research credibility, research registration, statistical thresholds, peer review, research reporting, research incentives, publication incentives, research waste
- MeSH terms
- Publications, Publishing, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.