The Economics of Reproducibility in Preclinical Research
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Low reproducibility rates within life science research undermine cumulative knowledge production and contribute to both delays and costs of therapeutic drug development. An analysis of past studies indicates that the cumulative (total) prevalence of irreproducible preclinical research exceeds 50%, resulting in approximately US$28,000,000,000 (US$28B)/year spent on preclinical research that is not reproducible-in the United States alone. We outline a framework for solutions and a plan for long-term improvements in reproducibility rates that will help to accelerate the discovery of life-saving therapies and cures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Fig 1 Studies reporting the prevalence of irreproducibility. Source: Begley and Ellis [6], Prinz et al. [7], Vasilevsky [8], Hartshorne and Schachner [5], and Glasziou et al. [9].](https://figures.cochat.ai/paper-figures/a30bd9bd65dc4bedb5db19b413eb814f/the-economics-of-reproducibility-in-preclinical-research-figure-1.jpg)
![Fig 2 Estimated US preclinical research spend and categories of errors that contribute to irreproducibility. Note that the percentage value of error for each category is the midpoint of the high and low prevalence estimates for that category divided (weighted) by the sum of all midpoint error rates (see S1 Dataset). Source: Chakma et al. [18] and the American Association for the Advancement of Science (AAAS) [19].](https://figures.cochat.ai/paper-figures/207a0a58df5c4620a819d7aa1f4aa960/the-economics-of-reproducibility-in-preclinical-research-figure-2.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Besides, reliably determining actual translational success rates is unmanageable as long as the current status of reporting of preclinical research leaves room for improvement [89], and non-reproducibility is such a critical issue in both animal [1] and human [90] studies.”
From Animal to human translation: a systematic scoping review of reported concordance rates (2019) · cites this paper as [1]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In the field of biomedical research, the majority of incorrect results are estimated to derive from the incorrect use of material standards and experimental reagents [2, 3].”
From Semi-automated fact-checking of nucleotide sequence reagents in biomedical research publications: The Seek & Blastn tool (2019) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The S&B tool therefore directly addresses the larger problem of material reagents and standards representing the major source of incorrect published results from pre-clinical research [3, 4].”
From Semi-automated fact-checking of nucleotide sequence reagents in biomedical research publications: The Seek & Blastn tool (2019) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current drug development is handicapped by high attrition rates; many molecules that were promising during preclinical development fail during subsequent clinical testing [1].”
From Animal to human translation: a systematic scoping review of reported concordance rates (2019) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This observation has been followed by a number of additional reports, the summary of which is, in essence, that 77-90% of biomedical experiments are not reproducible [14-17].”
From Guidelines for the use of flow cytometry and cell sorting in immunological studies (third edition) (2021) · cites this paper as [16]
Experimental — accuracy not yet measured
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Governing the Commons: The Evolution of Institutions for Collective Action
Field, B. C., & Ostrom, E. (1992) · Land Economics · doi:10.2307/3146384 · Cited by 18,859
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 2×
“Source: Begley and Ellis [6], Prinz et al.”
BackgroundBelieve it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · cited 1×
“[7], Vasilevsky [8], Hartshorne and Schachner [5], and Glasziou et al.”
BackgroundIncreasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597 · cited 1×
“However, it is reasonable to state that cumulative errors in the following broad categories—as well as underlying biases that could contribute to each problem area [14] or even result in entire studies never being published or reported [15]—are the primary causes of irreproducibility [16]: (1) study design, (2) biolog…”
BackgroundPolicy: NIH plans to enhance reproducibility
Collins, F. S., & Tabak, L. A. (2014) · Nature · doi:10.1038/505612a · Cited by 1,247
A call for transparent reporting to optimize the predictive value of preclinical research
Landis, S. C., Amara, S. G., Asadullah, K., Austin, C. P., Blumenstein, R., Bradley, E. W., Crystal, R. G., Darnell, R. B., Ferrante, R. J., Fillit, H., et al. (2012) · Nature · doi:10.1038/nature11556 · Cited by 1,196
Reproducibility in Science: Improving the Standard for Basic and Preclinical Research
Begley, C. G., & Ioannidis, J. P. A. (2015) · Circulation Research · doi:10.1161/circresaha.114.303819 · Cited by 1,035 · cited 1×
“Much has been written about the alarming number of preclinical studies that were later found to be irreproducible [1,2].”
BackgroundBiomedical research: increasing value, reducing waste.
Macleod, M. R., Michie, S., Roberts, I., Dirnagl, U., Chalmers, I., Ioannidis, J. P. A., Salman, R. A.-S., Chan, A.-W., & Glasziou, P. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62329-6 · Cited by 1,003
Partial Identification of Probability Distributions
(2003) · Springer Series in Statistics · doi:10.1007/b97478 · Cited by 698
What is missing from descriptions of treatment in trials and reviews?
Glasziou, P., Meats, E., Heneghan, C., & Shepperd, S. (2008) · BMJ · doi:10.1136/bmj.39590.732037.47 · Cited by 614
Research works printed but not linked: 1
- CF Manski, 2003, Springer Series in Statistics, 178
Cited by
Items 501–600 of 1,003
Towards a methodology for experimental evaluation in low-power wireless networking
Jacob, R., Boano, C. A., Raza, U., Zimmerling, M., & Thiele, L. (2019) · Workshop on Benchmarking Cyber-Physical Systems and Internet of Things (CPS-IoTBench) · doi:10.1145/3312480.3313173 · Cited by 9
“, clinical studies [6], where reproducibility has been a major concern for a long time.”
BackgroundHaslacher, H., Bayer, M., Fiegl, H., Gerner, M. C., Hofer, P., Korb, M., Neururer, S. B., Perkmann, T., Plattner, K., Sargsyan, K., et al. (2019) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2019-0491 · Cited by 9
“Recent investigations have revealed that unreliable results are to a large extent caused by poor biological reagents and reference materials [1].”
BackgroundCordell, G. A. (2019) · Journal of Ethnopharmacology · doi:10.1016/j.jep.2019.112134 · Cited by 11
Good science requires better animal welfare
Neville, V., Lecorps, B., & Mendl, M. (2022) · Science · doi:10.1126/science.abq7198 · Cited by 9
Author Country of Origin and Attention on Open Science Platforms: Evidence from COVID-19 Preprints
Fry, C., & MacGarvie, M. J. (2023) · Management Science · doi:10.1287/mnsc.2023.4936 · Cited by 13
DOSCATs: Double standards for protein quantification
Bennett, R. J., Simpson, D. M., Holman, S. W., Ryan, S., Brownridge, P., Eyers, C. E., Colyer, J., & Beynon, R. J. (2017) · Scientific Reports · doi:10.1038/srep45570 · Cited by 9
Spurlock, D. R. (2017) · Journal of Nursing Education · doi:10.3928/01484834-20170712-02 · Cited by 10
Hofman, P. M., & Barlési, F. (2019) · Expert Review of Molecular Diagnostics · doi:10.1080/14737159.2019.1611426 · Cited by 11
Fast Confirmation of Antibody Identity by MALDI-TOF MS Fingerprints
Tscheuschner, G., Schwaar, T., & Weller, M. G. (2020) · Antibodies · doi:10.3390/antib9020008 · Cited by 10
What’s wrong with Science? Modeling the collective discovery processes with the Nobel game
Chavalarias, D. (2016) · Scientometrics · doi:10.1007/s11192-016-2109-9 · Cited by 15
Prospects for a transparency revolution in the field of business history
Smith, A. D. A., & Umemura, M. (2018) · Business History · doi:10.1080/00076791.2018.1439019 · Cited by 12
Importance of Replication, The
Holden, C. J., & Goodwin, G. S. (2020) · Springer eBooks · doi:10.1007/978-3-319-24612-3_1352 · Cited by 9
The history, landscape, and outlook of human cell line authentication and security
Harbut, E., Makris, Y., Pertsemlidis, A., & Bleris, L. (2024) · SLAS Discovery · doi:10.1016/j.slasd.2024.100194 · Cited by 8
Ahuis, T. P., Smyk, M. K., Laloux, C., Aulehner, K., Bray, J. G., Waldron, A., Miljanovic, N., Seiffert, I., Song, D., Boulanger, B., et al. (2024) · PLoS ONE · doi:10.1371/journal.pone.0309521 · Cited by 10
Mundinger, C., Schulz, N. K. E., Singh, P., Janz, S., Schurig, M., Seidemann, J., Kurtz, J., Müller, C., Schielzeth, H., von Kortzfleisch, V. T., et al. (2025) · PLoS Biology · doi:10.1371/journal.pbio.3003019 · Cited by 9
State of FAIRness in ESFRI Projects
Wittenburg, P., de Jong, F. M. G., Uytvanck, D. V., Cocco, M., Jeffery, K., Lautenschlager, M., Thiemann, H., Hellström, M., Asmi, A., & Holub, P. (2019) · Data Intelligence · doi:10.1162/dint_a_00045 · Cited by 10
“· Reproducibility and reusability: Having biomedical research facing dramatic reproducibility challenges for more than a decade [13-20], BBMRI-ERIC promotes quality management in biobanks and leads development of provenance information management standard in ISO TC276 (PWI23494-1).”
BackgroundResearch Replication Prediction Using Weakly Supervised Learning
Luo, T., Li, X., Wang, H., & Liu, Y. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.132 · Cited by 12
“…the downside is that the average replication expense of each research project (which often consists a number of research studies) can go up to $500,000 (Freedman et al., 2015) 1 , which is hardly affordable to replicate each research finding, with an exponentially increasing number of publications.”
BackgroundTran, V. P., Kim, R., Maertens, M., Härtung, T., & Maertens, A. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.674370 · Cited by 16
“This leads to wasteful use of financial resources and labor, with an estimation of 28 billion dollars a year spent on irreproducible research (Freedman et al., 2015).”
BackgroundBorba, J. V., Canzian, J., Resmim, C. M., da Silva, R. M., Duarte, M. C. F., Mohammed, K. A., Schoenau, W., Adedara, I. A., & Rosemberg, D. B. (2024) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2024.105715 · Cited by 10
Noise as an Extrinsic Variable in the Animal Research Facility
Turner, J. G., & Manker, J. R. (2024) · Journal of the American Association for Laboratory Animal Science · doi:10.30802/aalas-jaalas-24-000008 · Cited by 8
Show and tell: disclosure and data sharing in experimental pathology
Schofield, P. N., Ward, J. M., & Sundberg, J. P. (2016) · Disease Models & Mechanisms · doi:10.1242/dmm.026054 · Cited by 12
“…been much discussion in recent years about the reported irreproducibility of preclinical data obtained using animal models (Begley and Ioannidis, 2015; Collins and Tabak, 2014; Freedman et al., 2015; Mak et al., 2014) and the cost to the success of both translational research and the public purse.”
Research Data Management Status of Science and Technology Research Institutes in Korea
Choi, M.-S., & Lee, S. (2020) · Data Science Journal · doi:10.5334/dsj-2020-029 · Cited by 8
Make like a glass frog: In support of increased transparency in herpetology
Marshall, B. M. (2021) · Herpetological Journal · doi:10.33256/31.1.3545 · Cited by 10
A Peer-To-Peer Publication Model on Blockchain
Khan, I., & Shahaab, A. (2021) · Frontiers in Blockchain · doi:10.3389/fbloc.2021.615726 · Cited by 8
“The economic cost of irreproducibility is as high as $28B per annum in the USA alone, where at least 50% of preclinical research remains irreproducible (Freedman et al., 2015).”
BackgroundPromoting Sustainability through Next-Generation Biologics Drug Development
Paulick, K., Seidel, S., Lange, C., Kemmer, A., Bournazou, M. N. C., Baier, A., & Haehn, D. (2022) · Sustainability · doi:10.3390/su14084401 · Cited by 11
“Machine learning can predict the efficacy of substances before they are produced [19,47,48].”
BackgroundClinical applications and challenges in the field of extracellular vesicles
Nieuwland, R., Enciso‐Martinez, A., & Bracht, J. W. P. (2023) · Medizinische Genetik · doi:10.1515/medgen-2023-2062 · Cited by 12
“28 Secondly, biomarker development requires robust standardization and reproducibility.”
BackgroundLogical Experimental Design and Execution in the Biomedical Sciences
Holder, D., & Marino, M. J. (2017) · Current Protocols in Pharmacology · doi:10.1002/cpph.20 · Cited by 9
Chemical Disarmament in a Technologically Evolving World
Forman, J. E., & Timperley, C. M. (2018) · ACS symposium series · doi:10.1021/bk-2018-1288.ch001 · Cited by 8
Reproduction and Replication: A Case Study with Automatic Essay Scoring.
Huber, E., & Çöltekin, Ç. (2020) · Language Resources and Evaluation · Cited by 8
Choice of y-axis can mislead readers
Erdoğan, B. R., Vollert, J., & Michel, M. C. (2020) · Naunyn-Schmiedeberg s Archives of Pharmacology · doi:10.1007/s00210-020-01926-x · Cited by 8
“%) in the life sciences, including pharmacology, is not reproducible (Freedman et al. 2015).”
BackgroundStatisticians, Roll Up Your Sleeves! There's A Crisis to be Solved
Seibold, H., Charlton, A., Boulesteix, A., & Hoffmann, S. (2021) · Significance · doi:10.1111/1740-9713.01554 · Cited by 12
“Applied Social Psychology, 37(1), 1–2.”
Eggert, S., Kahl, M., Kent, R., Gaats, L., Bock, N., Meinert, C., & Hutmacher, D. W. (2022) · Journal of Visualized Experiments · doi:10.3791/61261 · Cited by 8
Transparent and Reproducible Research Practices in the Surgical Literature
Hughes, T., Niemann, A., Tritz, D., Boyer, K., Robbins, H. I., & Vassar, M. B. (2022) · Journal of Surgical Research · doi:10.1016/j.jss.2021.09.024 · Cited by 8
Pauthner, M. G., Yeung, J., Ullman, C., Bakker, J., Wurch, T., Reichert, J. M., Lund‐Johansen, F., Bradbury, A. M., Carter, P. J., & Melis, J. P. M. (2016) · mAbs · doi:10.1080/19420862.2016.1153211 · Cited by 9
“, antibodies, cell lines) and broad utilization of these reagents by principle investigators; ensuring that research funder policies require documented use of validated and non-contaminated reagents and adequate funding to cover these additional costs; ensuring that procedures to document reagent validation are requir…”
BackgroundBetter reporting for better research: a checklist for reproducibility
Kenall, A., Edmunds, S., Goodman, L., Bal, L., Flintoft, L., Shanahan, D. R., & Shipley, T. (2015) · Genome biology · doi:10.1186/s13059-015-0710-5 · Cited by 9
“estimated this at approximately $28 billion a year for preclinical research in the United States alone [10].”
BackgroundKnow Thy Cells: Improving Biomedical Research Reproducibility
Freedman, L. P. (2015) · Science Translational Medicine · doi:10.1126/scitranslmed.aac7112 · Cited by 8
Self-citationHow far have we come with contextual data integration in drug discovery?
Κάτσιλα, Θ., & Matsoukas, M. (2018) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2018.1504767 · Cited by 8
“There is still an unmet need for biases, data reliability and reproducibility issues [17 – 19], resulting in very high costs (> 1 billion € per launched drug) and long duration (up to > 15 years) of the drug discovery, development and regulatory processes [17 – 20], which become gigantic if…”
BackgroundMake like a glass frog: In support of increased transparency in herpetologyPreprint
Marshall, B. M., & Strine, C. T. (2020) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/74frd · Cited by 8
“Across scientific disciplines replication efforts have revealed marked deviation from previously observed results (Freedman et al., 2015; Kelly, 2019; Open Science Collaboration, 2015); this lack of replicability, as shown in medical fields, can incur huge costs and impact human health (Freedman et…”
ResultDeSci-Decentralized SciencePreprint
Ducrée, J., Codyre, M., Walshe, R., & Barting, S. (2022) · Preprints.org · doi:10.20944/preprints202205.0223.v1 · Cited by 10
Statistical experiment design for animal researchPreprint
Sorzano, C. Ó. S. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/e9s25 · Cited by 11
Artificial intelligence: help or hindrance in solving the reproducibility crisis?
Straiton, J. (2024) · BioTechniques · doi:10.1080/07366205.2024.2355776 · Cited by 9
Gustafson, D., Nieuwland, R., & Lucien, F. (2024) · Biopreservation and Biobanking · doi:10.1089/bio.2024.0083 · Cited by 7
Setianegara, J., Wang, A., Gerard, N., Nys, J., Li, H. H., Chen, R. C., Gao, H., & Lin, Y. (2025) · Medical Physics · doi:10.1002/mp.17847 · Cited by 7
Denani, C. B., Horbach, I. S., Setatino, B. P., Azevedo, A. D. S., de Lima, S. M. B., Schwarcz, W. D., & Junior, I. P. D. S. (2025) · Human Vaccines & Immunotherapeutics · doi:10.1080/21645515.2025.2528368 · Cited by 8
“Freedman et al. 29 indicate that the total prevalence of irreproducible preclinical research exceeds 50%, resulting in an annual cost of $28 billion in the USA.”
Identity crisis—Rigor and reproducibility in human cell lines
Eckers, J. C., Swick, A. D., & Kimple, R. J. (2018) · Radiation Research · doi:10.1667/rr15086.1 · Cited by 7
“as much as $20 billion per year through publication of irreproducible results (8).”
Saadé, R. G. (2020) · Journal of Digital Innovation for Humanity · doi:10.31355/58 · Cited by 8
Shah, P., Vaughan, G., & Ledley, F. D. (2023) · PLoS ONE · doi:10.1371/journal.pone.0283887 · Cited by 7
Ankeny, R. A., Whittaker, A. L., Ryan, M., Boer, J. C., Plebanski, M., Tuke, J., & Spencer, S. J. (2023) · Brain Behavior and Immunity · doi:10.1016/j.bbi.2023.06.012 · Cited by 7
“This blind spot is also likely to have its analogue at the pre-clinical level, with important implications for how we should conduct BBI-related research from its earliest laboratory stages: for instance, a 2015 analysis (Freedman et al., 2015) showed that more than 50% of preclinical animal studies were not reproduci…”
BackgroundGood machine learning practices: Learnings from the modern pharmaceutical discovery enterprise
Makarov, V. A., Chabbert, C. D., Koletou, E., Psomopoulos, F. E., Kurbatova, N., Ramirez, S. A., Nelson, C., Natarajan, P., & Neupane, B. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108632 · Cited by 9
Good Practice Guideline for Preclinical Alcohol Research: The STRINGENCY Framework
Meinhardt, M. W., Gerlach, B., & Spanagel, R. (2024) · Current topics in behavioral neurosciences · doi:10.1007/7854_2024_484 · Cited by 7
Provide proactive reproducible analysis transparency with every publication
Meijer, P., Howard, N., Liang, J., Kelsey, A., Subramanian, S., Johnson, E., Mariz, P., Harvey, J. F., Ambrose, M., Tereshchenko, V., et al. (2025) · Royal Society Open Science · doi:10.1098/rsos.241936 · Cited by 11
“In the Freedman et al (2015) study, the authors argue that although implementing measures to improve reproducibility would raise the cost of individual studies, reducing the percentage of non-reproducible studies will yield significant cost savings.”
Büchler, R., Wendler, S., Muckova, P., Großkreutz, J., & Rhode, H. (2016) · PROTEOMICS - CLINICAL APPLICATIONS · doi:10.1002/prca.201600067 · Cited by 7
Hodgson, T., Magrabi, F., & Coiera, E. W. (2018) · Applied Clinical Informatics · doi:10.1055/s-0038-1649509 · Cited by 9
Argento, N. (2020) · EMBO Reports · doi:10.15252/embr.201949862 · Cited by 10
“More generally, the analysis highlighted difficulties in managing data related to laboratory SOPs and samples, which are crucial for data reusability and experimental reproducibility in biomedical research [7].”
BackgroundFugazzola, M. C., Wever, K. E., van de Lest, C. H. A., de Grauw, J. C., & Salvatori, D. C. F. (2022) · Osteoarthritis and Cartilage Open · doi:10.1016/j.ocarto.2022.100261 · Cited by 8
Peynshaert, K., Devoldere, J., Smedt, S. C. D., & Remaut, K. (2023) · Expert Opinion on Drug Delivery · doi:10.1080/17425247.2023.2167979 · Cited by 7
“indicates that the primary roots of irreproducibility are 1) study design, 2) biological reagents and reference materials, 3) laboratory protocols, and 4) data analysis and reporting[79].”
BackgroundAnalytic transparency is key for reproducibility of agricultural research
Sileshi, G. W. (2023) · CABI Agriculture and Bioscience · doi:10.1186/s43170-023-00144-8 · Cited by 8
Chatterjee, S., Abadie, T., Wang, M., Matar, O. K., & Ruoff, R. S. (2024) · Chemistry of Materials · doi:10.1021/acs.chemmater.3c02361 · Cited by 9
Triggle, C. R., & Williams, M. J. (2015) · Biochemical Pharmacology · doi:10.1016/j.bcp.2015.08.089 · Cited by 7
Cooper, D. R., Grabowski, M., Zimmerman, M., Porebski, P. J., Shabalin, I. G., Woińska, M., Domagalski, M. J., Zheng, H., Sroka, P., Cymborowski, M., et al. (2020) · Methods in molecular biology · doi:10.1007/978-1-0716-0892-0_13 · Cited by 9
Jasińska-Stroschein, M. (2020) · Cardiovascular Drugs and Therapy · doi:10.1007/s10557-020-07109-3 · Cited by 7
Importing genetically altered animals: ensuring quality
Birling, M., Fray, M., Kašpárek, P., Kopkanova, J., Massimi, M., Matteoni, R., Montoliu, L., Nutter, L. M. J., Raspa, M., Rozman, J., et al. (2021) · Mammalian Genome · doi:10.1007/s00335-021-09908-x · Cited by 7
An Open-Source Framework for Automated High-Throughput Cell Biology Experiments
Katunin, P., Zhou, J., Shehata, O. M., Peden, A. A., Cadby, A. J., & Nikolaev, A. (2021) · Frontiers in Cell and Developmental Biology · doi:10.3389/fcell.2021.697584 · Cited by 7
“Deep learning also helps to detect transformed cells in human tissues (Van Valen et al., 2016; Coudray et al., 2018), optimize treatment conditions (Kusumoto and Yuasa, 2019), and explain animal behavior (Heras et al., 2019).”
BackgroundTracing the footsteps of open research data in China
Li, C., Zhou, Y., Xiaohuan, Z., Zhang, Z., Jiang, L., Li, Z., Wang, P., Li, J., Xu, S., & Wang, Z. (2022) · Learned Publishing · doi:10.1002/leap.1439 · Cited by 12
“A 2015 meta-analysis five of past studies regarding the cost of non-reproducible research estimated that $28 billion per year is spent on preclinical research that is not reproducible (Freedman et al., 2015).”
BackgroundWei, B., Peer, C. J., Richardson, W. J., Hewitt, S. M., Figg, W. D., & Simpson, R. M. (2022) · Frontiers in Veterinary Science · doi:10.3389/fvets.2022.1056408 · Cited by 7
Quality-Assured Biobanking: The Leiden University Medical Center Model
Haumann, R., & Verspaget, H. W. (2018) · Methods in molecular biology · doi:10.1007/978-1-4939-7592-1_27 · Cited by 7
Bleakley, C., Reijgers, J., & Smoliga, J. M. (2020) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2020.9264 · Cited by 9
Biomedical Research: A House of Cards?
Lushington, G. H., & Chaguturu, R. (2015) · Future Medicinal Chemistry · doi:10.4155/fmc.15.171 · Cited by 7
Peer review: (r)evolution needed
Sipido, K. R., Gal, D., Luttun, A., Janssens, S. P., Sampaolesi, M., & Holvoet, P. (2017) · Cardiovascular Research · doi:10.1093/cvr/cvx191 · Cited by 7
Correction: The Economics of Reproducibility in Preclinical Research
Staff, T. P. B. (2018) · PLOS Biology · doi:10.1371/journal.pbio.1002626 · Cited by 7
Hoffmann, S., Schönbrodt, F. D., Elsas, R., Wilson, R., Strasser, U., & Boulesteix, A. (2020) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/afb9p · Cited by 8
Aggarwal, P., Dabo, A., Sun, C., Antonucci, V., Colsman, W., Fessenmayr, H., Wells, K. M., McComas, J., Nölken, G., & Nielsen, B. V. (2025) · Journal of Pharmaceutical and Biomedical Analysis · doi:10.1016/j.jpba.2025.116907 · Cited by 6
Lifespan Development, Instability, and Waddington’s Epigenetic Landscape
Hollar, D. (2016) · Springer eBooks · doi:10.1007/978-3-319-25325-1_16 · Cited by 6
Bergen, P., Munro, B. A., & Pang, D. (2023) · Journal of Veterinary Emergency and Critical Care · doi:10.1111/vec.13312 · Cited by 7
Data Management: The First Step in Reproducible Research
Soundararajan, S., & Mishra, S. (2023) · Indian Journal of Occupational and Environmental Medicine · doi:10.4103/ijoem.ijoem_342_22 · Cited by 6
The future of research dissemination: Innovation in publishing formats
Trial, H., & Einsiedler, J. (2024) · MIT Science Policy Review · doi:10.38105/spr.ex3016cqt0 · Cited by 6
Waldron, D., Levins, K. J., Murphy, D., McCarthy, M., Gorey, D., Ryan, K., Mannion, E. M., Nicholas, B. M., Brannagáin, D. Ó., Reilly, L., et al. (2025) · Current Oncology Reports · doi:10.1007/s11912-025-01642-w · Cited by 7
“To address strategies to minimize this issue, Freedman et al. [102], emphasized the identification of a primary hypothesis and outcome measure for future research and quotes, “the challenge of increasing reproducibility and addressing the costs associated with the lack of reproducibility in life…”
Rafiq, Q. A., Asencio, Í. O., Jenkins, S. I., Wilson, S. L., Patel, A. K., Barnes, A. L., Adams, C. F., Delcassian, D., & Smith, D. S. (2015) · Regenerative Medicine · doi:10.2217/rme.15.56 · Cited by 6
Les biobanques : quels enjeux en 2017 ?
Washetine, K., Ilié, M., Hofman, V., Butori, C., Long-Mira, É., Lassalle, S., Selva, É., Tanga, V., Bonneteau, C., Allégra, M., et al. (2016) · Revue Francophone des Laboratoires · doi:10.1016/s1773-035x(16)30416-6 · Cited by 6
Feygelman, V. M., Lohr, F., & Orton, C. G. (2017) · Medical Physics · doi:10.1002/mp.12090 · Cited by 6
“While the long-standing, but yet to be fulfilled, promise of using functional MR imaging to individualize radiation therapy is great, so are the challenges,(6) one of the most insidious being reproducibility.(7) Whether reliable and practical functional MRI biomarkers are ever found (and that is, statistically speakin…”
BackgroundUrsić, L., Bralić, N., Žuljević, M., Puljak, L., & Buljan, I. (2024) · Accountability in Research · doi:10.1080/08989621.2024.2345723 · Cited by 6
“However, studies have reported the costs of irreproducible preclinical research to be vastly higher (Freedman, Cockburn, and Another notable finding is our participants’ perception of the “confusion” of researchers when attempting to engage in reproducibility.”
Binary and Nonbinary Measures of Successful Aging: Do They Yield Comparable Conclusions?
Manierre, M. J. (2018) · Research on Aging · doi:10.1177/0164027518819208 · Cited by 8
The effect of journal guidelines on the reporting of antibody validation
Hoek, J. M., Hepkema, W. M., & Halffman, W. (2020) · PeerJ · doi:10.7717/peerj.9300 · Cited by 6
Reproducibility Challenges for Biomarker Detection with Uncertain But Informative Experimental Data
Zhuang, W. V., Camacho, L., Silva, C. S., & Hong, H. (2020) · Biomarkers in Medicine · doi:10.2217/bmm-2019-0599 · Cited by 6
ISO 23494: Biotechnology – Provenance Information Model for Biological Specimen And Data
Wittner, R., Holub, P., Müller, H., Geiger, J., Goble, C., Soiland‐Reyes, S., Pireddu, L., Frexia, F., Mascia, C., Fairweather, E., et al. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-80960-7_16 · Cited by 6
Sansbury, B. E., Nystoriak, M. A., Uchida, S., Wysoczynski, M., & Moore, J. B. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.913612 · Cited by 9
“In fact, it was previously estimated that between 51 and 89% of animal studies cannot be reproduced (33).”
BackgroundA Database of Accurate Electrophoretic Migration Patterns for Human Proteins
Mylonas, R., Potts, A., Waridel, P., Barblan, J., Rubio, M. D. C. C., Widmann, C., & Quadroni, M. (2022) · Journal of Molecular Biology · doi:10.1016/j.jmb.2022.167933 · Cited by 8
Reproducibility and rigor in rheumatology research
Alnaimat, F., Sweis, N. J., Sweis, J. J. G., Ascoli, C., Korsten, P., Rubinstein, I., & Sweiss, N. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2022.1073551 · Cited by 9
“Previous studies have shown that the prevalence of irreproducible preclinical research in the US alone exceeds 50%, with ∼28 billion US dollars spent yearly on preclinical research that is not reproducible (44).”
BackgroundReproducibility in Biomedical Research
Mullane, K., Curtis, M. J., & Williams, M. J. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-804725-5.00001-x · Cited by 11
A decentralized framework for cultivating research lifecycle transparency
Jeng, W., Wang, S.-H., Chen, H., Huang, P., Chen, Y., & Hsiao, H. (2020) · PLoS ONE · doi:10.1371/journal.pone.0241496 · Cited by 6
“It has been reported that only 39% of psychology articles in Nature could be replicated [1], and the direct cost of irreproducible preclinical research is about US$28 billion per year [2].”
BackgroundHaslberger, M., Schorr, S. G., Strech, D., & Haven, T. L. (2022) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15503 · Cited by 7
Risk mitigation in academic drug discovery
Ellinger, B., & Gribbon, P. M. (2016) · Expert Opinion on Drug Discovery · doi:10.1517/17460441.2016.1152257 · Cited by 6
“substantial proportion (up to 60%) of published preclinical findings are not reproducible when taken up by industry.[10] Here academic drug discovery centers can support risk mitiga-”
BackgroundHalvorson, T., Boucher, G., Yokosawa, D., Huang, J., García, R., Sanderink, L., Chen, L., Liu, L., Fajardo-Despaigne, J. E., Halvorson, J., et al. (2026) · Cell Reports Methods · doi:10.1016/j.crmeth.2026.101458 · Cited by 9
Jansen, J. A. W., van den Brand, M. A. M., de Vegt, F., & Swinkels, D. W. (2025) · Biopreservation and Biobanking · doi:10.1177/19475535251378202 · Cited by 6
Romero-Becerra, R., Zhao, Z. Q., Nebdal, D. J. H., Müller, E., Bergholtz, H., Norum, J. H., & Aittokallio, T. A. (2025) · Nature Communications · doi:10.1038/s41467-025-65218-9 · Cited by 8
Schlögl, M., Waltersdorfer, L., Regner, P., Siposova, A., & Brenning, A. (2026) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2026.106962 · Cited by 6
Fontana, B. D., Saccol, E. T., Borba, J. V., Resmim, C. M., de Abreu, M. S., Kalueff, A. V., & Rosemberg, D. B. (2026) · Reviews in the Neurosciences · doi:10.1515/revneuro-2026-0056 · Cited by 6
“Moreover, broader reproducibility and reliability challenges across behavioral and translational research highlight the relevance of improving methodological transparency and standardization in experimental procedures (Freedman et al. 2015; Frommlet 2020).”
The temporal value of local scientific expertise
Feldman, M. P., Dïbiaggio, L., & Tran, T. (2023) · Industrial and Corporate Change · doi:10.1093/icc/dtac047 · Cited by 5
Shajib, M. S., Futrega, K., Davies, A. M., Franco, R. A., McKenna, E., Guillesser, B., Klein, T. J., Crawford, R., & Doran, M. R. (2023) · Journal of The Royal Society Interface · doi:10.1098/rsif.2023.0468 · Cited by 7
“conducted an economic analysis where they estimated that irreproducible preclinical research costs the USA $28 billion annually [9].”
BackgroundSion, S. I., Nguyen-Phan, T., Fortin, M., Mes‐Masson, A., & Zhang, K. (2025) · Biopreservation and Biobanking · doi:10.1177/19475535251366364 · Cited by 6
“41 This situation is compounded by the structural challenges within the research community.”
Metadata
- Authors
- Freedman, L. P., Cockburn, I. M., & Simcoe, T. S.
- Affiliations
- Global Biological Standards Institute (US)
- Boston University (US)
- Corresponding author
- Freedman, Leonard P.
- Venue
- PLOS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-06-09
- Volume
- 13
- Issue
- 6
- Pages
- e1002165
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC4461318
- OpenAlex
- W585341442
- Cited by
- 1,055
- References
- 35
- Citations by year
- 2026: 48 · 2025: 45 · 2024: 69 · 2023: 92 · 2022: 93 · 2021: 96 · 2020: 142 · 2019: 116 · 2018: 109 · 2017: 93 · 2016: 103 · 2015: 49
- FWCI
- 188.58
- Type
- journal article
- Language
- en
- Update notices
- correction (2018-04-10)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- OpenBU (Boston University) (submittedVersion, cc-by)
- Topics
- Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health and Medical Research Impacts
- Keywords
- reproducibility, preclinical research, research costs, research waste, reproducibility framework
- MeSH terms
- Reproducibility of Results, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.