The Economics of Reproducibility in Preclinical Research
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Low reproducibility rates within life science research undermine cumulative knowledge production and contribute to both delays and costs of therapeutic drug development. An analysis of past studies indicates that the cumulative (total) prevalence of irreproducible preclinical research exceeds 50%, resulting in approximately US$28,000,000,000 (US$28B)/year spent on preclinical research that is not reproducible-in the United States alone. We outline a framework for solutions and a plan for long-term improvements in reproducibility rates that will help to accelerate the discovery of life-saving therapies and cures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Fig 1 Studies reporting the prevalence of irreproducibility. Source: Begley and Ellis [6], Prinz et al. [7], Vasilevsky [8], Hartshorne and Schachner [5], and Glasziou et al. [9].](https://figures.cochat.ai/paper-figures/a30bd9bd65dc4bedb5db19b413eb814f/the-economics-of-reproducibility-in-preclinical-research-figure-1.jpg)
![Fig 2 Estimated US preclinical research spend and categories of errors that contribute to irreproducibility. Note that the percentage value of error for each category is the midpoint of the high and low prevalence estimates for that category divided (weighted) by the sum of all midpoint error rates (see S1 Dataset). Source: Chakma et al. [18] and the American Association for the Advancement of Science (AAAS) [19].](https://figures.cochat.ai/paper-figures/207a0a58df5c4620a819d7aa1f4aa960/the-economics-of-reproducibility-in-preclinical-research-figure-2.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Besides, reliably determining actual translational success rates is unmanageable as long as the current status of reporting of preclinical research leaves room for improvement [89], and non-reproducibility is such a critical issue in both animal [1] and human [90] studies.”
From Animal to human translation: a systematic scoping review of reported concordance rates (2019) · cites this paper as [1]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In the field of biomedical research, the majority of incorrect results are estimated to derive from the incorrect use of material standards and experimental reagents [2, 3].”
From Semi-automated fact-checking of nucleotide sequence reagents in biomedical research publications: The Seek & Blastn tool (2019) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The S&B tool therefore directly addresses the larger problem of material reagents and standards representing the major source of incorrect published results from pre-clinical research [3, 4].”
From Semi-automated fact-checking of nucleotide sequence reagents in biomedical research publications: The Seek & Blastn tool (2019) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current drug development is handicapped by high attrition rates; many molecules that were promising during preclinical development fail during subsequent clinical testing [1].”
From Animal to human translation: a systematic scoping review of reported concordance rates (2019) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This observation has been followed by a number of additional reports, the summary of which is, in essence, that 77-90% of biomedical experiments are not reproducible [14-17].”
From Guidelines for the use of flow cytometry and cell sorting in immunological studies (third edition) (2021) · cites this paper as [16]
Experimental — accuracy not yet measured
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Governing the Commons: The Evolution of Institutions for Collective Action
Field, B. C., & Ostrom, E. (1992) · Land Economics · doi:10.2307/3146384 · Cited by 18,859
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 2×
“Source: Begley and Ellis [6], Prinz et al.”
BackgroundBelieve it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · cited 1×
“[7], Vasilevsky [8], Hartshorne and Schachner [5], and Glasziou et al.”
BackgroundIncreasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597 · cited 1×
“However, it is reasonable to state that cumulative errors in the following broad categories—as well as underlying biases that could contribute to each problem area [14] or even result in entire studies never being published or reported [15]—are the primary causes of irreproducibility [16]: (1) study design, (2) biolog…”
BackgroundPolicy: NIH plans to enhance reproducibility
Collins, F. S., & Tabak, L. A. (2014) · Nature · doi:10.1038/505612a · Cited by 1,247
A call for transparent reporting to optimize the predictive value of preclinical research
Landis, S. C., Amara, S. G., Asadullah, K., Austin, C. P., Blumenstein, R., Bradley, E. W., Crystal, R. G., Darnell, R. B., Ferrante, R. J., Fillit, H., et al. (2012) · Nature · doi:10.1038/nature11556 · Cited by 1,196
Reproducibility in Science: Improving the Standard for Basic and Preclinical Research
Begley, C. G., & Ioannidis, J. P. A. (2015) · Circulation Research · doi:10.1161/circresaha.114.303819 · Cited by 1,035 · cited 1×
“Much has been written about the alarming number of preclinical studies that were later found to be irreproducible [1,2].”
BackgroundBiomedical research: increasing value, reducing waste.
Macleod, M. R., Michie, S., Roberts, I., Dirnagl, U., Chalmers, I., Ioannidis, J. P. A., Salman, R. A.-S., Chan, A.-W., & Glasziou, P. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62329-6 · Cited by 1,003
Partial Identification of Probability Distributions
(2003) · Springer Series in Statistics · doi:10.1007/b97478 · Cited by 698
What is missing from descriptions of treatment in trials and reviews?
Glasziou, P., Meats, E., Heneghan, C., & Shepperd, S. (2008) · BMJ · doi:10.1136/bmj.39590.732037.47 · Cited by 614
Research works printed but not linked: 1
- CF Manski, 2003, Springer Series in Statistics, 178
Cited by
Items 401–500 of 1,003
Biospecimen Data Reporting in the Research Literature
Meredith, A., Simeon‐Dubach, D., Matzke, L. A. M., Cheah, S., & Watson, P. H. (2019) · Biopreservation and Biobanking · doi:10.1089/bio.2018.0143 · Cited by 13
Introducing the Open Science Chain: Protecting Integrity and Provenance of Research Data
Sivagnanam, S., Nandigam, V., & Lin, K. (2019) · Practice and Experience in Advanced Research Computing · doi:10.1145/3332186.3332203 · Cited by 23
Hummel, M., & Specht, C. (2019) · Journal of Laboratory Medicine · doi:10.1515/labmed-2019-0106 · Cited by 13
Lehmkuhl, M., & Promies, N. (2020) · PLoS ONE · doi:10.1371/journal.pone.0241376 · Cited by 23
Zucchelli, P., Horak, G., & Skinner, N. (2020) · SLAS TECHNOLOGY · doi:10.1177/2472630320971218 · Cited by 13
“There has been considerable debate about the causes, although common factors include basic flaws in experimental practice, 4,12,13 variability in antibody-based methods and assays, 14 and article retractions reported in PubMed.”
BackgroundPang, L., Liu, Z., Feng, W., Cai, C., & Yang, X. (2020) · Archives of Toxicology · doi:10.1007/s00204-020-02952-7 · Cited by 15
“Previous publications have discussed the factors commonly contributing to lack of reproducibility in nonclinical research, such as unblinded study design, small sample size, and unrealistically high doses that cause clinically irrelevant effects (Leonard 2015; Manolagas and Kronenberg 2014).”
BackgroundRecommendations for a Dutch Sustainable Biobanking Environment
van der Stijl, R., Manders, P., & Eijdems, E. W. H. M. (2021) · Biopreservation and Biobanking · doi:10.1089/bio.2021.0011 · Cited by 14
“Both the missing meta-data and the divergent standards can negatively impact the reproducibility of research results, influencing the trust stakeholders have in biobanks and biomedical research.(25,42) Next to variation between biobanks, there is also a discrepancy between academic, clinical, and industrial environmen…”
BackgroundImproving Reproducibility in Research: The Role of Measurement Science
Hanisch, R. J., Gilmore, I. S., & Plant, A. L. (2019) · Journal of Research of the National Institute of Standards and Technology · doi:10.6028/jres.124.024 · Cited by 15
“, which estimated that over half of the investment in preclinical research, around $28 billion, in the United States is not reproducible [12].”
BackgroundGenetic quality: a complex issue for experimental study reproducibility
Yoshiki, A., Ballard, G., & Perez, A. V. (2022) · Transgenic Research · doi:10.1007/s11248-022-00314-w · Cited by 17
“It has been estimated that 36% of the cost of preclinical research is spent on irreproducible experiments due to errors of reagents used and materials including animals (Freedman et al. 2015).”
BackgroundImproving Science That Uses Code
Thimbleby, H. (2023) · The Computer Journal · doi:10.1093/comjnl/bxad067 · Cited by 18
Shaping Science for Increasing Interdependence and Specialization
Utzerath, C., & Fernández, G. (2017) · Trends in Neurosciences · doi:10.1016/j.tins.2016.12.005 · Cited by 16
An assessment of transparency and reproducibility‐related research practices in otolaryngology
Johnson, A. L., Torgerson, T., Skinner, M., Hamilton, T., Tritz, D. J., & Vassar, M. (2019) · medRxiv · doi:10.1002/lary.28322 · Cited by 8
“Irreproducibility carries an expensive price tag; for example, in their retrospective analysis, Freedman et al.(25) estimated that $28 billion has been spent on preclinical research that is irreproducible.”
BackgroundClearing Opacity through Machine LearningPreprint
Price, W. N., & Rai, A. K. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3536983 · Cited by 13
von Kortzfleisch, V. T., & Richter, S. H. (2023) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2023.109992 · Cited by 15
Evidence-based medicine or statistically manipulated medicine? Are we slaves to the P -value?
Goel, H., Raheja, D., & Nadar, S. K. (2024) · Postgraduate Medical Journal · doi:10.1093/postmj/qgae012 · Cited by 13
Filamented hydrogels as tunable conduits for guiding neurite outgrowth
Liu, H., Puiggalí‐Jou, A., Chansoria, P., Janiak, J., & Zenobi‐Wong, M. (2025) · Materials Today Bio · doi:10.1016/j.mtbio.2025.101471 · Cited by 16
Selby, P. J., Banks, R. E., Gregory, W. M., Hewison, J., Rosenberg, W. M., Altman, D. G., Deeks, J. J., McCabe, C. J., Parkes, J., Sturgeon, C. M., et al. (2018) · Programme Grants for Applied Research · doi:10.3310/pgfar06030 · Cited by 16
“More broadly, the irreproducibility in preclinical research in the life sciences generally has been estimated to cost > $28M per year in the USA alone.(732) In the clinical environment, failure to standardise pre-analytical conditions can have a critical impact on patient safety, for example in terms of a wrong or del…”
BackgroundStumptner, C., Stadlbauer, V., O’Neil, D., Gessner, A., Hiergeist, A., Zatloukal, K., & Abuja, P. M. (2022) · Nutrients · doi:10.3390/nu14091976 · Cited by 14
Chiriboga, L. A., Callis, G. M., Wang, Y., & Chlipala, E. A. (2022) · Journal of Histotechnology · doi:10.1080/01478885.2022.2134022 · Cited by 14
“Freedman et al. defined ‘irreproducibility’ as the ‘exis-tence and propagation of one or more errors, flaws, inadequacies or omissions that prevent the replication of results’ [4].”
BackgroundInfluentialApproaches for the isolation and long-term expansion of pericytes from human and animal tissues
Alvino, V. V., Mohammed, K. A. K., Gu, Y., & Madeddu, P. R. (2023) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.1095141 · Cited by 17
A critical analysis of cancer biobank practices in relation to biospecimen quality
Rush, A., Spring, K. J., & Byrne, J. A. (2015) · Biophysical Reviews · doi:10.1007/s12551-015-0178-2 · Cited by 13
Making the most of peer review
Slavov, N. (2015) · eLife · doi:10.7554/elife.12708 · Cited by 12
“By neglecting these comments we are missing an opportunity to reduce the time and resources that are spent trying to repeat and build on experiments that are not reproducible (Freedman et al., 2015).”
BackgroundOpen for collaboration: an academic platform for drug discovery and development at SciLifeLab
Arvidsson, P. I., Sandberg, K., & Forsberg‐Nilsson, K. (2016) · Drug Discovery Today · doi:10.1016/j.drudis.2016.06.026 · Cited by 12
Zozus, M. N., Richesson, R., Walden, A., Tenenbaum, J., & Hammond, W. E. (2016) · PubMed Central · Cited by 14
Valdez, J., Kim, M. J., Rueschman, M., Socrates, V., Redline, S. S., & Sahoo, S. S. (2017) · PubMed · Cited by 17
What's in a sample? Increasing transparency in biospecimen procurement methods
LaBaer, J., Miceli, J. F., & Freedman, L. P. (2018) · Nature Methods · doi:10.1038/nmeth.4684 · Cited by 12
Self-citationKoshiyama, A., & Firoozye, N. B. (2018) · The Journal of Financial Data Science · doi:10.3905/jfds.2019.1.013 · Cited by 16
“, [35], [26], [7], [13]), and [24]) although it may not always be willful ([16]), and is pernicious in finance (see for instance, [44]).”
BackgroundTumor Banks: A Quality Control Scheme Proposal
Craciun, L., Spinette, S. A., Rassy, M., Salgado, R., de Wind, A., Demetter, P., Verset, L., Gomez-Galdón, M., Chintinne, M., Sirtaine, N., et al. (2019) · Frontiers in Medicine · doi:10.3389/fmed.2019.00225 · Cited by 12
“The lack of reproducibility in gene signatures is often associated with tissue heterogeneity due to, among other things, the lack of standardization of collection procedures (2).”
BackgroundLife Sciences for Philosophers and Philosophy for Life Scientists: What Should We Teach?
Boniolo, G., & Campaner, R. (2019) · Biological Theory · doi:10.1007/s13752-019-00333-7 · Cited by 17
Beyond Standardization: Improving External Validity and Reproducibility in Experimental Evolution
Desjardins, E., Kurtz, J., Kranke, N., Lindeza, A. S., & Richter, S. H. (2021) · BioScience · doi:10.1093/biosci/biab008 · Cited by 12
Quality of histone modification antibodies undermines chromatin biology research
Kungulovski, G., & Jeltsch, A. (2015) · F1000Research · doi:10.12688/f1000research.7265.1 · Cited by 12
Striving for Reproducible Science
Sweedler, J. V. (2015) · Analytical Chemistry · doi:10.1021/acs.analchem.5b04300 · Cited by 12
The Somatic Mutation Working Group of the SEQC-II Consortium (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/626440 · Cited by 12
Johnson, A. L., Anderson, J. M., Bouvette, M., Pinero, I., Rauh, S., Johnson, B., Kee, M., Heigle, B., Tricco, A. C., Page, M. J., et al. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.11.009 · Cited by 11
Wasted Money in United States Biomedical and Agricultural Animal Research
Keen, J. (2019) · Animal Experimentation: Working Towards a Paradigm Change · doi:10.1163/9789004391192_011 · Cited by 11
Recommendations to improve use and reporting of statistics in animal experiments
Rowe, A. (2022) · Laboratory Animals · doi:10.1177/00236772221140669 · Cited by 15
“Using an estimate that 50% of preclinical life sciences research is not reproducible, one study found this wastes $28 billion per annum in the USA alone.(20) Reporting of methods and statistics is measurably improved by using checklists.”
BackgroundAhmadzadeh, K., Dizaji, S. R., & Yousefifard, M. (2023) · International Journal of Surgery · doi:10.1097/js9.0000000000000475 · Cited by 11
“Only in the United States has irreproducible preclinical research been estimated to cause a waste of 28 billion dollars annually([1]).”
BackgroundBiobank of fungi from marine and terrestrial Antarctic environments
NARDO, V. G., Otero, I. V. R., Giovanella, P., dos Santos, J. A., Pellizzer, E. P., DOVIGO, D. R., PAES, E. C. P., & Sette, L. D. (2023) · Anais da Academia Brasileira de Ciências · doi:10.1590/0001-3765202320230603 · Cited by 11
Improving Reliability of Immunological Assays by Defining Minimal Criteria for Cell Fitness
Ivison, S. M., Boucher, G., Zheng, G., García, R., Kohen, R., Bitton, A., Rioux, J. D., & Levings, M. K. (2024) · ImmunoHorizons · doi:10.4049/immunohorizons.2300095 · Cited by 11
Townsend, H. G. G., Osterrieder, K., Jelinski, M. D., Morck, D. W., Waldner, C., Cox, W. R., Gerdts, V., Potter, A. A., Babiuk, L. A., & Cross, J. C. (2025) · Scientific Reports · doi:10.1038/s41598-025-15935-4 · Cited by 17
Nussbeck, S. Y., Rabone, M., Benson, E. E., Droege, G., Mackenzie-Dodds, J., & Lawlor, R. T. (2016) · Biopreservation and Biobanking · doi:10.1089/bio.2015.0061 · Cited by 15
“More than one-third of irreproducibility in pre-clinical research results is reportedly due to the biological reagents and reference materials used, and about 25% is due to data analysis and reporting issues.(30) It would have been valuable to have asked a question on the state of informatics at the organizations repr…”
MethodGuiding Principles for Human and Animal Research During Pharmaceutical Product Development
Maheshwari, R., Gupta, A., Ganeshpurkar, A., Chourasiya, Y., Tekade, M., & Tekade, R. K. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-814421-3.00018-x · Cited by 14
Woodard, C. L., Nassrallah, W. B., Samiei, B. V., Murphy, T. H., & Raymond, L. A. (2020) · Scientific Reports · doi:10.1038/s41598-020-68477-2 · Cited by 12
Quality Considerations When Using Tissue Samples for Biomarker Studies in Cancer Research
Speirs, V. (2021) · Biomarker Insights · doi:10.1177/11772719211009513 · Cited by 12
UThwigl — An R package for closed- and open-system uranium–thorium dating
Dosseto, A., & Marwick, B. (2021) · Quaternary Geochronology · doi:10.1016/j.quageo.2021.101235 · Cited by 11
“High rates of irreproducibility of research results have been estimated in several fields and disciplines (Camerer et al., 2018, 2016; Freedman et al., 2015; Institute, 2013; Ioannidis, 2005; Open Science Collaboration, 2015).”
BackgroundSoni, S. S., D’Elia, A. M., & Rodell, C. B. (2023) · Drug Delivery and Translational Research · doi:10.1007/s13346-023-01290-2 · Cited by 11
“Notably, these models do face a degree of logistical challenges, including relatively high costs, as well as greater demands for maintenance and care in laboratory settings [97, 98].”
BackgroundHernández‐Arteaga, E., & Ågmo, A. (2023) · Frontiers in Behavioral Neuroscience · doi:10.3389/fnbeh.2023.1192213 · Cited by 17
“It has been claimed that about 28 billion US$ are spent on irreproducible research every year in the United States alone (Freedman et al., 2015).”
BackgroundSahoo, S. S., Valdez, J., & Rueschman, M. (2016) · PubMed Central · Cited by 11
The MRC Framework for the Development, Design and Analysis of Stratified Medicine Research
Isaacs, J. D., & Matthews, F. E. (2017) · Cited by 15
Campbell, H., & Gustafson, P. (2019) · The American Statistician · doi:10.1080/00031305.2018.1555101 · Cited by 14
“Unreliable research not only reduces the credibility of science, but is also very costly (Freedman, Cockburn, and Simcoe 2015) and as such, addressing the underlying issues is of “vital importance” (Spiegelhalter 2017).”
BackgroundLalu, M. M., Leung, G. J., Dong, Y., Montroy, J. R., Butler, C., Auer, R. C., & Fergusson, D. A. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-029475 · Cited by 11
Ineichen, B. V., Sati, P., Granberg, T., Absinta, M., Lee, N. J., Lefeuvre, J., & Reich, D. S. (2020) · NeuroImage Clinical · doi:10.1016/j.nicl.2020.102371 · Cited by 11
“A recent report indicates that, in the United States alone, the cumulative prevalence of irreproducible preclinical research exceeds 50% with an approximate cost of $28 billion/year (Freedman et al., 2015).”
BackgroundHarmonization of Biobank Education for Biobank Technicians: Identification of Learning Objectives
Hartung, M. L., Baber, R., Herpel, E., Specht, C., Brucker, D. P., Schoneberg, A., Winter, T., & Nussbeck, S. Y. (2021) · BioTech · doi:10.3390/biotech10020007 · Cited by 11
“By now, an alarming number of irreproducible preclinical studies are a consequence of improper biospecimens or reference material, wasting approximately US$ 28 billion per year in the United Stated of America [4,5].”
BackgroundBiobanks in personalized medicine
Zatloukal, K., Stumptner, C., Kungl, P., & Mueller, H. (2018) · Expert Review of Precision Medicine and Drug Development · doi:10.1080/23808993.2018.1493921 · Cited by 14
Combinatorial cassettes to systematically evaluate tissue-engineered constructs in recipient mice
Bodhak, S., de Castro, L. F., Kuznetsov, S. A., Maeda, A., Bonfim, D. C., Robey, P. G., & Simon, C. G. (2018) · Biomaterials · doi:10.1016/j.biomaterials.2018.09.035 · Cited by 12
Carter, M. J. (2017) · Wound Repair and Regeneration · doi:10.1111/wrr.12565 · Cited by 11
Engineered Ovarian Cancer Cell Lines for Validation of CAR T Cell Function
Repellin, C. E., Ganesan, P., Alcudia, J. F., Lakshmireddy, H. K. D., Patel, P. D., Beviglia, L., Javitz, H. S., Sambucetti, L. C., & Bhatnagar, P. (2019) · Advanced Biosystems · doi:10.1002/adbi.201900224 · Cited by 11
Better reporting for better research: a checklist for reproducibility
Kenall, A., Edmunds, S., Goodman, L., Bal, L., Flintoft, L., Shanahan, D. R., & Shipley, T. (2015) · BMC Neuroscience · doi:10.1186/s12868-015-0177-z · Cited by 13
“estimated this at approximately $28 billion a year for preclinical research in the United States alone [10].”
BackgroundWilliams, R. (2015) · Circulation Research · doi:10.1161/circresaha.115.307532 · Cited by 13
Juergens, H., Niemeijer, M. S., Jennings, L. D., Mans, R., Morel, J., van Maris, A. J. A., Pronk, J. T., & Gardner, T. S. (2018) · Scientific Data · doi:10.1038/sdata.2018.195 · Cited by 11
Lucas, F., Gil‐Pulido, J., LaMacchia, J., Preffer, F., Wallace, P. K., & Lopez, P. (2020) · Cytometry Part A · doi:10.1002/cyto.a.23940 · Cited by 10
“In 2015, it was estimated that irreproducibility of published scientific articles ranges from 50% to 89%, with an economic impact of around $28B/year in the United States alone (4).”
BackgroundThe learning curve of murine subretinal injection among clinically trained ophthalmic surgeons
Huang, P., Narendran, S., Pereira, F., Fukuda, S., Nagasaka, Y., Apicella, I., Yerramothu, P., Marion, K. M., Cai, X., Sadda, S. R., et al. (2022) · Translational Vision Science & Technology · doi:10.1167/tvst.11.3.13 · Cited by 10
“Low reproducibility rates can plague preclinical studies, with reports that cumulative (total) prevalence of irreproducible preclinical research exceeds 50% (ranging from 51 - 89%),31-35 resulting in around $28 billion wasted annually in the United States.36 Lack of training, along with scientific culture and incentiv…”
BackgroundGenome editing to 're-write' wrongs
Perry, M. E., Valdes, K. M., Wilder, E. L., Austin, C. P., & Brooks, P. J. (2018) · Nature Reviews Drug Discovery · doi:10.1038/nrd.2018.91 · Cited by 10
The future of protein biomarker research in type 2 diabetes mellitus
Tans, R., Verschuren, L., Wessels, H. J. C. T., Bakker, S. J. L., Tack, C. J., Gloerich, J., & van Gool, A. J. (2018) · Expert Review of Proteomics · doi:10.1080/14789450.2018.1551134 · Cited by 10
Predictors of Success of Phase II Pediatric Oncology Clinical Trials
Franshaw, L., Tsoli, M., Byrne, J. A., Mayoh, C., Sivarajasingam, S., Norris, M. D., Marshall, G. M., & Ziegler, D. S. (2019) · The Oncologist · doi:10.1634/theoncologist.2017-0666 · Cited by 11
“Negative preclinical results are usually not published, and it has also been proposed that up to 50% of preclinical findings cannot be reproduced [31, 36, 37].”
BackgroundTracking cell lineages to improve research reproducibility
Zaaijer, S., Groen, S. C., & Sanjana, N. E. (2021) · Nature Biotechnology · doi:10.1038/s41587-021-00928-1 · Cited by 13
Munro, B. A., Bergen, P., & Pang, D. (2021) · Veterinary Anaesthesia and Analgesia · doi:10.1016/j.vaa.2021.09.004 · Cited by 11
Developmental origins of Parkinson disease: Improving the rodent models
Jiménez-Salvador, I., Meade, P., Iglesias, E., Bayona‐Bafaluy, M. P., & Ruiz‐Pesini, E. (2023) · Ageing Research Reviews · doi:10.1016/j.arr.2023.101880 · Cited by 12
“Although animal models allow strong control over confounding factors, according to one estimate, as much as $28 billion is wasted annually in the United States alone in pre-clinical experiments involving laboratory animals due to the lack of repeatability of these studies (Freedman et al., 2015).”
BackgroundChittiboyina, A. G., & Khan, I. A. (2024) · Journal of Ethnopharmacology · doi:10.1016/j.jep.2024.117871 · Cited by 11
Publishing code is essential for reproducible flow cytometry bioinformatics
O’Neill, K., & Brinkman, R. R. (2016) · Cytometry Part A · doi:10.1002/cyto.a.22805 · Cited by 10
“In a larger economic study, 50% of all preclinical research findings were found to be irreproducible, representing $28.2 billion in wasted funding in the USA alone (7).”
BackgroundInfluentialWilliams, M. J., Mullane, K., & Curtis, M. J. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-804725-5.00005-7 · Cited by 13
Reagent Validation to Facilitate Experimental Reproducibility
Williams, M. J. (2018) · Current Protocols in Pharmacology · doi:10.1002/cpph.40 · Cited by 11
Scherz, P. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108593694 · Cited by 12
Towards Reproducible Bioinformatics: The OpenBio-C Scientific Workflow Environment
Kanterakis, A., Iatraki, G., Pityanou, K., Koumakis, L., Kanakaris, N., Karacapilidis, N. I., & Potamias, G. (2019) · IEEE International Conference on Bioinformatics and Bioengineering (BIBE) · doi:10.1109/bibe.2019.00047 · Cited by 10
“In addition, there are estimates that preclinical research is dominated by more than 50% of irreproducible results at a cost of about 28 billion dollars [5].”
BackgroundMouse model systems of autism spectrum disorder: Replicability and informatics signature
Kabitzke, P., Morales, D. V., He, D., Cox, K., Sutphen, J. C., Thiede, L., Sabath, E. V., Hanania, T., Biemans, B., & Brunner, D. (2020) · Genes Brain & Behavior · doi:10.1111/gbb.12676 · Cited by 12
Liu, H., Gielen, M. J. C. A. M., Bosmans, J. W. A. M., Winkens, B., & Bouvy, N. D. (2021) · Hernia · doi:10.1007/s10029-020-02351-y · Cited by 11
Current Status and Future Opportunities in Modeling Clinical Characteristics of Multiple Sclerosis
Liu, J., Kelly, E., & Bielekova, B. (2022) · Frontiers in Neurology · doi:10.3389/fneur.2022.884089 · Cited by 11
“It is estimated that between 51% and 89% of the published literature in biomedical sciences is not reproducible ( 15, 23, 24) and poor study design, based on small sample sizes (11, 13) and the failure to prevent bias (25–27) is the major contributor to this reproducibility crisis.”
BackgroundGuru, S. M., Hanigan, I., Nguyen, H. A., Burns, E., Stein, J., Blanchard, W., Lindenmayer, D. B., & Clancy, T. (2016) · Ecological Informatics · doi:10.1016/j.ecoinf.2016.08.003 · Cited by 10
“According to Pritsker (2012), it is estimated that only 10%–30% of published studies are repeatable, although Freedman et al. (2015) claim that 51%–89% of all published research is not reproducible, and that one in four data r B.V.”
BackgroundLucanic, M., Plummer, W. T., Harke, J., Lucanic, M., Chen, E., Bhaumik, D., Harinath, G., Coleman-Hulbert, A. L., Dumas, K. J., Onken, B., et al. (2017) · Protocol Exchange · doi:10.1038/protex.2016.086 · Cited by 12
Fontoura-Andrade, J. L., de Amorim, R. F. B., & de Sousa, J. B. (2017) · Acta Cirúrgica Brasileira · doi:10.1590/s0102-865020170030000010 · Cited by 11
Pre- and Post-analytical Factors in Biomarker Discovery
Klont, F., Horvatovich, P. L., Govorukhina, N. I., & Bischoff, R. (2019) · Methods in molecular biology · doi:10.1007/978-1-4939-9164-8_1 · Cited by 12
Soft windowing application to improve analysis of high-throughput phenotyping data
Haselimashhadi, H., Mason, J., Muñoz‐Fuentes, V., López, F., Babalola, K., Acar, E. F., Kumar, V., White, J. K., Flenniken, A. M., King, R., et al. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz744 · Cited by 10
Rižner, T. L., & Adamski, J. (2017) · The Journal of Steroid Biochemistry and Molecular Biology · doi:10.1016/j.jsbmb.2017.12.017 · Cited by 11
Assay Guidance Manual for Drug Discovery: Robust or Go Bust
Markossian, S., Coussens, N. P., Dahlin, J. L., & Sittampalam, G. S. (2021) · SLAS DISCOVERY · doi:10.1177/24725552211054044 · Cited by 10
The Australian academic STEMM workplace post-COVID: a picture of disarrayPreprint
Christian, K., Larkins, J., & Doran, M. R. (2022) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2022.12.06.519378 · Cited by 12
“billion per year [37]; there is a risk that the growth in publication output may put downward pressure on 26”
BackgroundAssessment of inverse publication bias in safety outcomes: an empirical analysis
Xing, X., Zhu, J., Shi, L., Xu, C., & Lin, L. (2024) · BMC Medicine · doi:10.1186/s12916-024-03707-2 · Cited by 9
“it generally refers to the preference for publishing studies based on the nature and direction of their findings, for example, those with significant results or large effect sizes [3, 4].”
BackgroundRapid Deep Vat Printing Using Photoclickable Collagen‐Based Bioresins
Winkelbauer, M., Hasenauer, A., Rütsche, D., Liu, H., Janiak, J., Nguyen, M. B., Christman, K. L., Zenobi‐Wong, M., & Chansoria, P. (2025) · Advanced Healthcare Materials · doi:10.1002/adhm.202405105 · Cited by 10
flowIO: Flow cytometry standard conformance testing, editing, and export tool
Koblížek, M., Lebedeva, A., & Fišer, K. (2018) · Cytometry Part A · doi:10.1002/cyto.a.23563 · Cited by 9
Relative effects of age on implant integration in a rat model: A longitudinal in vivo microct study
Freitag, L., Günther, C., Eberli, U., Fürst, A., Zeiter, S., & Stadelmann, V. A. (2018) · Journal of Orthopaedic Research® · doi:10.1002/jor.24210 · Cited by 10
“have reported that up to 50% of preclinical research performed in the United State is irreproducible.(3) Numerous factors were identified as main causes leading to this reproducibility crisis: Wrong choice of biological reagents or laboratory protocols, flawed study design or data analysis, and incomplete reporting.”
BackgroundWillers, C., Lynch, T., Chand, V., Islam, M. S., Lassere, M. N., & MARCH, L. M. (2021) · Biopreservation and Biobanking · doi:10.1089/bio.2021.0098 · Cited by 9
Role of Biobanks for Cancer Research and Precision Medicine in Hepatocellular Carcinoma
Korhan, P., Avcı, S. T., Yılmaz, Y., Islakoğlu, Y. Ö., & Atabey, N. (2021) · Journal of Gastrointestinal Cancer · doi:10.1007/s12029-021-00759-y · Cited by 10
“in 2015, the cumulative (total) prevalence of non-reproducible preclinical studies exceeded 50%, resulting in approximately US$28 billion/ year for non-replicable preclinical trials in the USA alone was spent [44].”
BackgroundMisidentified biomedical resources: Journal guidelines are not a quick fix
Hepkema, W. M., Horbach, S. P. J. M., Hoek, J. M., & Halffman, W. (2021) · International Journal of Cancer · doi:10.1002/ijc.33882 · Cited by 10
“A Nature survey revealed that over 70% of researchers have tried and failed to reproduce another scientist's experiment in biomedical research,(9) in which inadequately identified research materials are a major factor of concern.(5) Even though the alarmist term ‘reproducibility crisis’ may wrongly suggest a recent ou…”
BackgroundFrom viability to cell death: Claims with insufficient evidence in high-impact cell culture studies
Özkaya, A. B., & Geyik, C. (2022) · PLoS ONE · doi:10.1371/journal.pone.0250754 · Cited by 11
“Therefore, many aspects of the preclinical study design have been tackled by various studies over the years [7, 8].”
BackgroundConfirmation by Robustness Analysis: A Bayesian Account
Casini, L., & Landes, J. (2022) · Erkenntnis · doi:10.1007/s10670-022-00537-7 · Cited by 9
“Finally,thereisanongoingmethodologicaldebateonwhatcountsas asuccessfulreplicationingeneral,(see,e.g.,Atmanspacher,2016;Fletcher,2021; Freedman,2015;Rubin,2021).”
McGranahan, M. J., Kibildis, S. W., McCully, K. K., & O’Connor, P. J. (2023) · Microvascular Research · doi:10.1016/j.mvr.2023.104532 · Cited by 9
Teng, M. Z., Merenick, D. R., Jessel, A., Ganshorn, H., & Pang, D. (2024) · Comparative Medicine · doi:10.30802/aalas-cm-23-000063 · Cited by 11
Quality and reporting practices in an Australian cancer biobank cohort
Rush, A., & Byrne, J. A. (2015) · Clinical Biochemistry · doi:10.1016/j.clinbiochem.2015.12.007 · Cited by 9
Ready display of antigenic peptides in a protein ‘mimogen’
Vallée, M. R. J., Schombs, M. W., Balaban, Z. J., Colyer, J., & Davis, B. G. (2016) · Chemical Communications · doi:10.1039/c5cc08892e · Cited by 9
Novel or reproducible: That is the question
Hagen, K. G. T. (2016) · Glycobiology · doi:10.1093/glycob/cww036 · Cited by 12
Wolf, J. C. (2018) · Environmental Toxicology and Chemistry · doi:10.1002/etc.4117 · Cited by 9
“2017), and the reproducibility of scientific results in general is becoming a more frequent topic of academic discussion (Begley and Ioannidis 2015; Freedman et al. 2015).”
Background
Metadata
- Authors
- Freedman, L. P., Cockburn, I. M., & Simcoe, T. S.
- Affiliations
- Global Biological Standards Institute (US)
- Boston University (US)
- Corresponding author
- Freedman, Leonard P.
- Venue
- PLOS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-06-09
- Volume
- 13
- Issue
- 6
- Pages
- e1002165
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC4461318
- OpenAlex
- W585341442
- Cited by
- 1,055
- References
- 35
- Citations by year
- 2026: 48 · 2025: 45 · 2024: 69 · 2023: 92 · 2022: 93 · 2021: 96 · 2020: 142 · 2019: 116 · 2018: 109 · 2017: 93 · 2016: 103 · 2015: 49
- FWCI
- 188.58
- Type
- journal article
- Language
- en
- Update notices
- correction (2018-04-10)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- OpenBU (Boston University) (submittedVersion, cc-by)
- Topics
- Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health and Medical Research Impacts
- Keywords
- reproducibility, preclinical research, research costs, research waste, reproducibility framework
- MeSH terms
- Reproducibility of Results, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.