The Economics of Reproducibility in Preclinical Research
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Low reproducibility rates within life science research undermine cumulative knowledge production and contribute to both delays and costs of therapeutic drug development. An analysis of past studies indicates that the cumulative (total) prevalence of irreproducible preclinical research exceeds 50%, resulting in approximately US$28,000,000,000 (US$28B)/year spent on preclinical research that is not reproducible-in the United States alone. We outline a framework for solutions and a plan for long-term improvements in reproducibility rates that will help to accelerate the discovery of life-saving therapies and cures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Fig 1 Studies reporting the prevalence of irreproducibility. Source: Begley and Ellis [6], Prinz et al. [7], Vasilevsky [8], Hartshorne and Schachner [5], and Glasziou et al. [9].](https://figures.cochat.ai/paper-figures/a30bd9bd65dc4bedb5db19b413eb814f/the-economics-of-reproducibility-in-preclinical-research-figure-1.jpg)
![Fig 2 Estimated US preclinical research spend and categories of errors that contribute to irreproducibility. Note that the percentage value of error for each category is the midpoint of the high and low prevalence estimates for that category divided (weighted) by the sum of all midpoint error rates (see S1 Dataset). Source: Chakma et al. [18] and the American Association for the Advancement of Science (AAAS) [19].](https://figures.cochat.ai/paper-figures/207a0a58df5c4620a819d7aa1f4aa960/the-economics-of-reproducibility-in-preclinical-research-figure-2.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Besides, reliably determining actual translational success rates is unmanageable as long as the current status of reporting of preclinical research leaves room for improvement [89], and non-reproducibility is such a critical issue in both animal [1] and human [90] studies.”
From Animal to human translation: a systematic scoping review of reported concordance rates (2019) · cites this paper as [1]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In the field of biomedical research, the majority of incorrect results are estimated to derive from the incorrect use of material standards and experimental reagents [2, 3].”
From Semi-automated fact-checking of nucleotide sequence reagents in biomedical research publications: The Seek & Blastn tool (2019) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The S&B tool therefore directly addresses the larger problem of material reagents and standards representing the major source of incorrect published results from pre-clinical research [3, 4].”
From Semi-automated fact-checking of nucleotide sequence reagents in biomedical research publications: The Seek & Blastn tool (2019) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current drug development is handicapped by high attrition rates; many molecules that were promising during preclinical development fail during subsequent clinical testing [1].”
From Animal to human translation: a systematic scoping review of reported concordance rates (2019) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This observation has been followed by a number of additional reports, the summary of which is, in essence, that 77-90% of biomedical experiments are not reproducible [14-17].”
From Guidelines for the use of flow cytometry and cell sorting in immunological studies (third edition) (2021) · cites this paper as [16]
Experimental — accuracy not yet measured
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Governing the Commons: The Evolution of Institutions for Collective Action
Field, B. C., & Ostrom, E. (1992) · Land Economics · doi:10.2307/3146384 · Cited by 18,859
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 2×
“Source: Begley and Ellis [6], Prinz et al.”
BackgroundBelieve it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · cited 1×
“[7], Vasilevsky [8], Hartshorne and Schachner [5], and Glasziou et al.”
BackgroundIncreasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597 · cited 1×
“However, it is reasonable to state that cumulative errors in the following broad categories—as well as underlying biases that could contribute to each problem area [14] or even result in entire studies never being published or reported [15]—are the primary causes of irreproducibility [16]: (1) study design, (2) biolog…”
BackgroundPolicy: NIH plans to enhance reproducibility
Collins, F. S., & Tabak, L. A. (2014) · Nature · doi:10.1038/505612a · Cited by 1,247
A call for transparent reporting to optimize the predictive value of preclinical research
Landis, S. C., Amara, S. G., Asadullah, K., Austin, C. P., Blumenstein, R., Bradley, E. W., Crystal, R. G., Darnell, R. B., Ferrante, R. J., Fillit, H., et al. (2012) · Nature · doi:10.1038/nature11556 · Cited by 1,196
Reproducibility in Science: Improving the Standard for Basic and Preclinical Research
Begley, C. G., & Ioannidis, J. P. A. (2015) · Circulation Research · doi:10.1161/circresaha.114.303819 · Cited by 1,035 · cited 1×
“Much has been written about the alarming number of preclinical studies that were later found to be irreproducible [1,2].”
BackgroundBiomedical research: increasing value, reducing waste.
Macleod, M. R., Michie, S., Roberts, I., Dirnagl, U., Chalmers, I., Ioannidis, J. P. A., Salman, R. A.-S., Chan, A.-W., & Glasziou, P. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62329-6 · Cited by 1,003
Partial Identification of Probability Distributions
(2003) · Springer Series in Statistics · doi:10.1007/b97478 · Cited by 698
What is missing from descriptions of treatment in trials and reviews?
Glasziou, P., Meats, E., Heneghan, C., & Shepperd, S. (2008) · BMJ · doi:10.1136/bmj.39590.732037.47 · Cited by 614
Research works printed but not linked: 1
- CF Manski, 2003, Springer Series in Statistics, 178
Cited by
Items 601–700 of 1,003
Setting Up an Automated Biomanufacturing Laboratory
Pavan, M. (2021) · Methods in molecular biology · doi:10.1007/978-1-0716-1032-9_5 · Cited by 6
Statistics, statistical thinking, and the IACUC
Reynolds, P. S. (2021) · Lab Animal · doi:10.1038/s41684-021-00832-w · Cited by 6
A framework for traceble storage and curation of measurement data
Thomas, S. A., & Brochu, F. M. (2021) · Measurement Sensors · doi:10.1016/j.measen.2021.100201 · Cited by 5
Kim, S., & Hwang, S. (2021) · Pharmaceuticals · doi:10.3390/ph14121324 · Cited by 5
“Concerns regarding reproducibility in drug response studies are a practical issue raised repeatedly in preclinical studies (Haibe-Kains et al., 2013; Collins and Tabak, 2014; Freedman et al., 2015).”
BackgroundBirmeta, G., Jarso, T. S., Geleta, M., & Bekele, E. (2022) · Advances in Agriculture · doi:10.1155/2022/5874899 · Cited by 5
TIER2: enhancing Trust, Integrity and Efficiency in Research through next-level Reproducibility
Ross‐Hellauer, T., Klebel, T., Bannach‐Brown, A., Horbach, S. P. J. M., Jabeen, H., Manola, N., Metodiev, T., Papageorgiou, H., Reczko, M., Sansone, S., et al. (2022) · Research Ideas and Outcomes · doi:10.3897/rio.8.e98457 · Cited by 10
Cooper, T. (2023) · FEMS Yeast Research · doi:10.1093/femsyr/foac065 · Cited by 5
Tang, B. L. (2024) · Accountability in Research · doi:10.1080/08989621.2024.2345714 · Cited by 6
“Reporting such irreproducible or invalid results and findings is a disservice to other researchers who would try to build on or further the research (Hannun 2021), and irreproducible research has incurred a significant economic burden (Freedman, Cockburn, and Simcoe 2015).”
Li, Y., Zhou, X., Chen, R., Zhang, X., & Cao, H. (2024) · PLoS Genetics · doi:10.1371/journal.pgen.1011423 · Cited by 6
“Approximately $28 billion is spent annually on preclinical research that is not replicable in the United States alone [2].”
Blockchain technology: driving change in the scientific research workflow
Lawlor, B., Chalk, S. J., Frey, J. G., Hayashi, K., Kochalko, D., Shute, R. E., & Sopek, M. (2025) · Pure and Applied Chemistry · doi:10.1515/pac-2023-1204 · Cited by 6
Incremental Science: Papers and Grants, Yes; Discoveries, No
Sverdlov, E. D. (2018) · Molecular Genetics Microbiology and Virology · doi:10.3103/s0891416818040079 · Cited by 5
Problem Überversorgung: Definition und Lösungsansätze
Gamstätter, T. (2021) · Der Internist · doi:10.1007/s00108-021-00957-7 · Cited by 5
MALDI-TOF-MS-Based Identification of Monoclonal Murine Anti-SARS-CoV-2 Antibodies within One Hour
Tscheuschner, G., Kaiser, M. N., Lisec, J., Beslic, D., Muth, T., Krüger, M., Mages, H. W., Dorner, B. G., Knospe, J., Schenk, J. A., et al. (2022) · Antibodies · doi:10.3390/antib11020027 · Cited by 5
Mistry, H. B., & Orrell, D. (2020) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1770 · Cited by 6
“4 Therefore, not only is the input and output noise of PERSPECTIVES the data quite high, but so is the noise in our prior knowledge!”
BackgroundMcClenahan, B. J., Lojacono, M., Young, J. L., Schenk, R. J., & Rhon, D. I. (2023) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.13810 · Cited by 5
Two Cultures in Modern Science and Technology: For Safety and Validity Does Medicine Have to Update?
Becker, R. E. (2016) · Journal of Patient Safety · doi:10.1097/pts.0000000000000260 · Cited by 5
“An estimated $28 billion is wasted each year on flawed research.(8) Drug developers fail to replicate breakthrough reports in major medical journals.”
BackgroundAn Assessment of Retractions as a Measure of Scientific Misconduct and Impact on Public Health Risks
Abritis, A. J. (2015) · Digital Commons - University of South Florida (University of South Florida) · Cited by 6
Reply to: Improving Research Standards to Restore Trust in Intranasal Oxytocin
Leng, G., & Ludwig, M. (2015) · Biological Psychiatry · doi:10.1016/j.biopsych.2015.08.030 · Cited by 7
Genetic Insights into the Economic Toll of Cell Line Misidentification: A Comprehensive Review
Weiskirchen, R. (2026) · Medical Sciences · doi:10.3390/medsci14010025 · Cited by 4
“Moreover, further calculations have estimated that the costs resulting from the usage of contaminated or misidentified cells could cost the US over $28 billion/year from which 36.1% are believed to be linked to biological reagents and reference materials [15,140].”
InfluentialPark, S., Cho, S.-J., Kim, C., Chung, H. M., & Lee, S.-G. (2026) · Scientific Reports · doi:10.1038/s41598-025-34504-3 · Cited by 5
“In these settings, strict control and clear definition of environmental factors are critical to producing results that are both scientifically sound and applicable to clinical or regulatory decision-making 32,33 .”
The importance of rigor in pharmacy research: Challenges and solutions
Gionfriddo, M. R., McClendon, C., Nolfi, D., Kalarchian, M. A., & Covvey, J. R. (2025) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2025.02.005 · Cited by 4
Housing matters: Experimental variables shaping metabolism in obese mice
Choi, B. S.-Y., Holm, J. B., Brejnrod, A., Fjære, E., Xia, Z., Allingbjerg, M.-L., Larsen, I., Kristensen, D. M., Dall, M., Myrmel, L. S., et al. (2025) · Molecular Metabolism · doi:10.1016/j.molmet.2025.102190 · Cited by 5
“This challenge is exacerbated by the low reproducibility rates in biomedical research, often cited as around 50%, which signi fi cantly delays the development of potentially lifesaving therapies [4].”
Strengthening translational preclinical research through confirmatory multi-laboratory studies
Rotter, S. C., Arroyo-Araujo, M., Drude, N., Pellegrini, P., Kobold, S., Richter, G. H. S., Müller, O. J., Bruder, D., Riecken, L. B., Gerlach, B., et al. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2025.1715361 · Cited by 5
“These setbacks do not only pose serious health risks to patients who may receive new treatments with little to no benefit or worse, unforeseen harmful side effects (1), but also strain financial and healthcare systems (2, 5, 6).”
Bridging statistics and life sciences undergraduate education
Tong, L., White, B. J. G., & Singh, J. (2022) · Journal of Biological Education · doi:10.1080/00219266.2022.2118810 · Cited by 9
“Poor quality research, on the other hand, can produce misleading and false scientific conclusions, waste resources, and hinder scientific progress (Freedman, Cockburn, and Simcoe 2015).”
A powerful approach to identify replicable variants in genome-wide association studies
Li, Y., Lei, H., Wen, X., & Cao, H. (2024) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2024.04.004 · Cited by 4
Simulation methodologies to determine statistical power in laboratory animal research studies
Jeffers, A., Konrad, K., Larson, G. J., Allen-Moyer, K. N., CUNNY, H. C., & Shockley, K. R. (2024) · Laboratory Animals · doi:10.1177/00236772241273002 · Cited by 7
“5 Freedman et al. 6 estimate that the cumulative irrepro-ducibility rate in preclinical research is centered around 50%, and inadequate study design and data analysis contribute to this figure.”
Connecting the dots in neuroscience research: The future of evidence synthesis
Hair, K., Arroyo-Araujo, M., Vojvodić, S., Economou, M., Wong, C., Tinsdeall, F., Smith, S., Rackoll, T., Sena, E. S., & McCann, S. (2024) · Experimental Neurology · doi:10.1016/j.expneurol.2024.115047 · Cited by 5
“…(S.K. McCann). investigate factors affecting the reproducibility and replicability of published findings, an established and frequent concern across biomedical research, including the neurosciences (Begley and Ellis, 2012; Freedman et al., 2015; Steckler, 2015; Voelkl et al., 2020).”
Evaluating Data-Sharing Policies and Author Compliance in Leading Orthopaedic Journals
Major, J., Wild, K., Elfar, A., Dennis, B., Oldham, E. P., Hagood, A., Paul, E., Checketts, J. X., Ford, A. I., & Vassar, M. B. (2025) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.24.00955 · Cited by 4
“annual expenditure of $28 billion on nonreproducible research in the United States alone 4 .”
Introduction to the Rosetta Special Collection
Khare, S. D., & Whitehead, T. A. (2015) · PLoS ONE · doi:10.1371/journal.pone.0144326 · Cited by 4
“A recent report extrapolates that fully 50% of biological research is not reproducible [10].”
BackgroundBiospecimen User Fees: Global Feedback on a Calculator Tool
Matzke, L. A. M., Babinszky, S., Slotty, A., Meredith, A., Castillo-Pelayo, T., Henderson, M. K., Simeon‐Dubach, D., Schacter, B., & Watson, P. H. (2016) · Biopreservation and Biobanking · doi:10.1089/bio.2016.0027 · Cited by 4
At the Crossroads: Lessons and Challenges in Computational Social Science
Borge-Holthoefer, J., Moreno, Y., & Yasseri, T. (2016) · Frontiers research topics · doi:10.3389/978-2-88945-021-3 · Cited by 4
What’s New in Orthopaedic Basic Science
Allen, M. J. (2016) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.16.01120 · Cited by 5
How Will Open Science Impact on University/Industry Collaborations?
Chataway, J., Parks, S., & Smith, E. (2018) · Springer eBooks · doi:10.1007/978-3-319-62649-9_12 · Cited by 4
Considerations on Using Antibodies for Studying the Dynorphins/Kappa Opioid Receptor System
Chen, C., Widmann, M., Schwarzer, C., & Liu‐Chen, L. (2021) · Handbook of experimental pharmacology · doi:10.1007/164_2021_467 · Cited by 5
Power(ful) myths: misconceptions regarding sample size in quality of life research
Anderson, S. F. (2021) · Quality of Life Research · doi:10.1007/s11136-021-03020-y · Cited by 5
Only seeing is believing – the power of evidence and reason
Rupp, B. (2016) · Postępy Biochemii · doi:10.18388/pb.2016_2 · Cited by 4
Uhlmann, C., Nickel, A., Picard, D. J., Rossi, A., Li, G., Hildebrandt, B., Brockerhoff, G., Bendt, F., Hübenthal, U., Hewera, M., et al. (2022) · Biotechnology Journal · doi:10.1002/biot.202100693 · Cited by 4
Longworth, S., & Chalmers, A. D. (2022) · Information Services & Use · doi:10.3233/isu-220158 · Cited by 4
“The difficulty in selecting the right antibody has contributed to the reproducibility crisis in the life science industry, as well as a massive waste in time and money in research globally [2].”
Johnson, A., Bouvette, M. J., Rangu, N., Morley, T., Schultz, A., Torgerson, T., & Vassar, M. B. (2023) · Annals of Otology Rhinology & Laryngology · doi:10.1177/00034894231185642 · Cited by 4
Hamilton, D. G., Page, M. J., Everitt, S., Fraser, H. S., & Fidler, F. M. (2024) · Accountability in Research · doi:10.1080/08989621.2024.2308606 · Cited by 5
“Irreproducible and irreplicable medical research is a major cause of research waste and has been previously estimated to cost the United States alone more than USD $28 billion each year (Freedman, Cockburn, and Simcoe 2015).”
Petiti, J., Revel, L., & Divieto, C. (2025) · BMC Research Notes · doi:10.1186/s13104-025-07298-w · Cited by 4
“Over recent years, a worrying increase in the number of irreproducible preclinical studies has been observed, with laboratory protocols identified as the primary cause in 10.8% of cases [4].”
InfluentialTriggle, C. R. (2015) · Biochemical Pharmacology · doi:10.1016/j.bcp.2015.06.029 · Cited by 4
Challenges and opportunities for system biology standards and tools in medical research
König, M., Oellrich, A., Waltemath, D., Dobson, R., Hubbard, T., & Olaf Wolkenhauer, A. (2016) · Research Portal (King's College London) · Cited by 4
Experimental Biological Protocols with Formal Semantics
Abate, A., Cardelli, L., Kwiatkowska, M., Laurenti, L., & Yordanov, B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99429-1_10 · Cited by 5
More Data, Less Process? The Applicability of MPLP to Research Data
Lafferty-Hess, S., & Christian, T.-M. L. (2017) · IASSIST Quarterly · doi:10.29173/iq907 · Cited by 4
“To some degree, this standard also acknowledges the recent scrutiny of published scientific studies, a concerning number of which were reported to have failed to meet the reproducibility benchmark of scientific integrity (Chang & Li, 2015; Freedman et al., 2015; Open Science Collaboration, 2015).”
BackgroundComputational Reproducibility: The Elephant in the Room
Hatton, L., & van Genuchten, M. (2019) · IEEE Software · doi:10.1109/ms.2018.2883805 · Cited by 5
Considerations on Preclinical Neuromuscular Disease Gene Therapy Studies
Duan, D. (2019) · Springer eBooks · doi:10.1007/978-3-030-03095-7_17 · Cited by 6
Costs of Implementing Quality in Research Practice
Littrell, O. M., Stoeger, C., Maier, H., Fuchs, H., de Angelis, M. H., Cassis, L. A., Gerhardt, G. A., Grondin, R., & Gailus-Durner, V. (2019) · Handbook of experimental pharmacology · doi:10.1007/164_2019_294 · Cited by 5
Information Content of the Model for Calculating the Finite Precision of Measurements
Menin, B. M. (2020) · Physical Science International Journal · doi:10.9734/psij/2020/v24i730201 · Cited by 4
Demmrich, S., & Huber, S. (2020) · Journal of Religion in Europe · doi:10.1163/18748929-13010008 · Cited by 4
A UCSD view on replication and reproducibility for CPS & IoT
Yen, A., Flowers, B., Luo, W., Nagesh, N., Tueller, P., Kastner, R., & Pannuto, P. (2021) · Proceedings of the Workshop on Benchmarking Cyber-Physical Systems and Internet of Things · doi:10.1145/3458473.3458821 · Cited by 4
“ducible pre-clinical research exceeds 50%, which results in approximately $28B spent per year on pre-clinical, irreproducible research in the United States alone [18].”
BackgroundFAIRifying Clinical Studies Metadata: A Registry for the Biomedical Research
Meloni, V., Sulis, A., Mascia, C., & Frexia, F. (2021) · Studies in health technology and informatics · doi:10.3233/shti210281 · Cited by 4
“The difficulties of using in other contexts the datasets created during a clinical trial [1], as well as the low reproducibility of the results obtained [2], have been widely analysed in order to overcome the barriers preventing an efficient reuse of quality data [3, 4].”
BackgroundMarkossian, S., Brimacombe, K. R., Sittampalam, G. S., Grossman, A., Coussens, N. P., Inglese, J., Auld, D. S., Chin, W. W., Austin, C. P., & Hall, M. D. (2022) · Nature Reviews Drug Discovery · doi:10.1038/d41573-022-00170-7 · Cited by 4
Bickel, D. R. (2023) · Statistics · doi:10.1080/02331888.2023.2232912 · Cited by 4
“The costs may include an estimated 28 billion US dollars per year in pre-clinical research that cannot be replicated [11].”
BackgroundJanero, D. R. (2016) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2016.1212014 · Cited by 6
“billion dollars in the United States alone [8,9].”
BackgroundBannach‐Brown, A., Tillmann, S., Macleod, M. R., & Wegener, G. (2019) · BMJ Open Science · doi:10.1136/bmjos-2018-000017 · Cited by 4
“The application of the registered reports publishing framework to animal experiments can improve poor reporting, which threatens reproducibility.(27) However, the framework may need amending to fully reap the benefits.”
BackgroundLim, M. S., Beyer, T., Babayan, A., Bergmann, M. M., Brehme, M., Buyx, A., Czernin, J. G., Egger, G., Elenitoba‐Johnson, K. S. J., Gückel, B., et al. (2019) · Molecular Imaging and Biology · doi:10.1007/s11307-019-01361-2 · Cited by 4
Farrell, B., & Bengtson, J. (2019) · PLoS ONE · doi:10.1371/journal.pone.0223984 · Cited by 5
“This is particularly relevant now when the lack of reproducibility is a well-documented deficiency that results in an enormous cost to the taxpayer [14].”
BackgroundMethodMansour, N. M., Balas, E. A., Yang, F. M., & Vernon, M. (2020) · Medical Science Monitor · doi:10.12659/msm.922016 · Cited by 4
“Much preclinical research may be irreproducible, wasting, by one estimate, billions of dollars in research dollars each year [1].”
BackgroundCuration at the point of measurement and traceability of measurement workflows
Thomas, S. A., & Brochu, F. M. (2022) · Measurement Sensors · doi:10.1016/j.measen.2022.100399 · Cited by 4
Biomonitoring and Digital Data Technology as an Opportunity for Enhancing Animal Study Translation
Defensor, E. B., Lim, M. A., & Schaevitz, L. R. (2021) · ILAR Journal · doi:10.1093/ilar/ilab018 · Cited by 4
Standardization of High Content Imaging and Informatics
Trask, O. J., & Johnston, P. A. (2015) · Assay and Drug Development Technologies · doi:10.1089/adt.2015.29022.ojt · Cited by 5
“Leonard Freedman from the Global Biological Standards and colleagues recently published a commentary in PLoS Biology about the cost of irreproducibility in preclinical sciences.(3) The fact that scientific research is not always reproducible is not surprising, and to a certain extent this idea has become accepted by m…”
BackgroundIs there a robust future for research in reproduction?
Barratt, C. L. R. (2015) · Molecular Human Reproduction · doi:10.1093/molehr/gav062 · Cited by 4
“Freedman et al. (2015) calculate that US$28 000 000 000 ($28Billion) per year is spent on research in the USA that is not reproducible.”
BackgroundScience with no fiction: measuring the veracity of scientific reports by citation analysisPreprint
Grabitz, P., Lazebnik, Y. A., Nicholson, J. M., & Rife, S. C. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/172940 · Cited by 5
YAMP: a containerised workflow enabling reproducibility in metagenomics researchPreprint
Visconti, A., Martin, T. C., & Falchi, M. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/223016 · Cited by 4
Report on Data Quality in Biobanks: Problems, Issues, State-of-the-ArtPreprint
Ranasinghe, S., Pichler, H., & Eder, J. (2018) · arXiv.org · doi:10.48550/arxiv.1812.10423 · Cited by 5
“Insufficient reproducibility of data from medical studies have recently been discussed together with the highly problematic consequences for the advancement of science, the economics of research and the development of suitable drugs and therapeutic procedures (see e.g. [3,16,19]).”
BackgroundMarshall, B. M., & Duthie, A. B. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.06.19.599733 · Cited by 6
“Across many disciplines, there are examples of replication studies being unable to replicate prior research (Freedman, Cockburn & Simcoe, 2015; Open Science Collaboration, 2015; Kelly, 2019).”
BackgroundWylie, M. R., Gilbreath, J. J., Melton‐Celsa, A. R., & Merrell, D. S. (2025) · mSphere · doi:10.1128/msphere.00638-25 · Cited by 6
Epicardial patches for myocardial repair: circumventing barriers to heal ischemic heart tissue
Winkelbauer, M., Generali, M., Cheng, K., Emmert, M. Y., & Chansoria, P. (2026) · Trends in biotechnology · doi:10.1016/j.tibtech.2026.01.014 · Cited by 4
Gonzalez-Uarquin, F., Jirkof, P., Bert, B., Hawkins, P., Angelovski, L., Baumgart, J., Baumgart, N., Cevik, Ö. S., Franco, N. H., Horata, E., et al. (2026) · Laboratory Animals · doi:10.1177/00236772251400976 · Cited by 3
Fifteen challenges for generative AI applications to cell biology
Dupire, L., Khan, A. A., Karaletsos, T., Kelley, S., Lundberg, E. K., Ma, J., Paull, E., Quake, S. R., Rabadán, R., Cassius, R., et al. (2026) · Cell · doi:10.1016/j.cell.2026.07.004 · Cited by 4
Cell Line Classification Using Electric Cell-Substrate Impedance Sensing (ECIS)
Gelsinger, M. L., Tupper, L. L., & Matteson, D. S. (2019) · The International Journal of Biostatistics · doi:10.1515/ijb-2018-0083 · Cited by 3
“When such anomalies go unnoticed, erroneous results are reported, contributing to the multi-billion dollar irreproducibility problem [8].”
BackgroundBlanchard, R. C., DeWitt, T. J., Young, S., & Perkin, J. S. (2024) · Journal of Fish Biology · doi:10.1111/jfb.15665 · Cited by 3
Swan, J., Kallio, E., Magga, J. M., Mannila, J., Weber, E. M., & Törnqvist, E. (2024) · Pharmacological Research - Reports · doi:10.1016/j.prerep.2024.100020 · Cited by 5
Machine learning-assisted decoding of temporal transcriptional dynamics via fluorescent timer
Irie, N., Takeda, N., Satou, Y., Araki, K., & Lab), M. O. (. (2025) · Nature Communications · doi:10.1038/s41467-025-61279-y · Cited by 3
“These arbitrary and subjective elements undermine the transparency and rigour of data analysis, contributing to the reproducibility crisis, which is widespread in life sciences and preclinical studies 39 , including the research fi elds using fl ow cytometric analysis 27 .”
Peer Review in Transplantation
Hutchinson, J. A. (2015) · Transplantation · doi:10.1097/tp.0000000000000906 · Cited by 3
“Several recent studies estimate the prevalence of irreproducibility at 51% to 89% of published papers, a problem that is not confined to low-ranking journals.(5) Freedman and colleagues estimate the cost of irreproducible preclinical research in the USA at $28 billion/year.”
BackgroundLa reproducibilidad de las investigaciones biomédicas: Quo vadis?
Dal‐Ré, R., Bernad, A., & Garesse, R. (2016) · Medicina Clínica · doi:10.1016/j.medcli.2015.11.029 · Cited by 4
Using High Content Imaging to Quantify Target Engagement in Adherent Cells
Axelsson, H., Almqvist, H., & Seashore-Ludlow, B. A. (2018) · Journal of Visualized Experiments · doi:10.3791/58670 · Cited by 3
Kerrison, W. G. J., Ning, J., Krásný, L., Arthur, A., Guljar, N., Elms, M. L., Swain, A., Jones, R. L., Thway, K., & Huang, P. H. (2022) · Cells · doi:10.3390/cells11152418 · Cited by 5
“This has contributed to poor reproducibility in preclinical findings which is likely to have played a role in the high failure rate of translating therapeutic discoveries into oncology clinical trials and regulatory approvals [16,17].”
BackgroundNovichkov, P. S., Chandonia, J., & Arkin, A. P. (2022) · GigaScience · doi:10.1093/gigascience/giac089 · Cited by 3
“Non-reusable data are also very costly: life sciences researchers in the United States alone are estimated to spend over $28 billion each year on irreproduciblepreclinicalwork [7].”
We must improve conditions and options for Australian ECRs
Christian, K., Larkins, J., & Doran, M. R. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01621-w · Cited by 3
Botter, S. M., & Kessler, T. M. (2023) · International Journal of Molecular Sciences · doi:10.3390/ijms241814281 · Cited by 3
“Given that the prevalence of irreproducible biomedical research is substantial [72,73], standardization, especially in multicenter studies, is a key aspect that biobanks should adhere to.”
BackgroundDecatastrophizing research irreproducibility
Jarvis, M. F. (2024) · Biochemical Pharmacology · doi:10.1016/j.bcp.2024.116090 · Cited by 3
Richter, L. F., Büchele, W. R. E., Imhof, A. M., & Kühn, F. E. (2024) · HardwareX · doi:10.1016/j.ohx.2024.e00581 · Cited by 3
Villarreal, J. C., Ljungberg, E., Uy, A. G., Li, A., Kremer, S., & Borges, C. R. (2025) · Chemical Engineering Journal · doi:10.1016/j.cej.2025.160100 · Cited by 3
Effect of Four Fattening Systems on Quality of White Kołuda ® Goose Products *
Wojtycza, K., Gąsior, R., Szymczyk, B., Odrzywolska, A., Bielińska, H., & Wróblewski, W. (2025) · Annals of Animal Science · doi:10.2478/aoas-2025-0023 · Cited by 3
Festing, M. F. W. (2018) · Neuromethods · doi:10.1007/978-1-4939-8564-7_13 · Cited by 3
Novel Biomimetic Microphysiological Systems for Tissue Regeneration and Disease Modeling
Budhwani, K. I., Oliver, P. G., Buchsbaum, D. J., & Thomas, V. (2018) · Advances in experimental medicine and biology · doi:10.1007/978-981-13-0947-2_6 · Cited by 3
Liquid materials for biomedical research: a highly IT-integrated and automated biobanking solution
Popp, D., Diekmann, R., Binder, L., Asif, A. R., & Nussbeck, S. Y. (2019) · Journal of Laboratory Medicine · doi:10.1515/labmed-2017-0118 · Cited by 3
Çöltekin, Ç. (2020) · Cited by 3
Qualitätssicherung im Gewebebiobanking – Ein Überblick
Kaufhold-Wedel, C., Hummel, M., Brobeil, A., Schirmacher, P., & Schmitt, S. (2022) · Die Pathologie · doi:10.1007/s00292-022-01078-0 · Cited by 3
Girault, J. (2022) · eNeuro · doi:10.1523/eneuro.0318-22.2022 · Cited by 3
“Despite these limitations, biology moves forward at a steady pace, but poor reproducibility and limited validity are likely to decrease the yield of research, slowing progress and increasing cost (Freedman et al., 2015).”
BackgroundA Modelling Ecosystem for Prognostics
Astfalck, L., Hodkiewicz, M. R., KEATING, A. J., Cripps, E., & PECHT, M. G. (2016) · Annual Conference of the PHM Society · doi:10.36001/phmconf.2016.v8i1.2568 · Cited by 3
“Freedman, Cockburn, and Simcoe (2015) showed that approximately US$28B/year is spent on irreproducible preclinical research, an estimated 50% of research in the field.”
BackgroundExperimental Planning and Execution
Mullane, K., Curtis, M. J., & Williams, M. J. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-804725-5.00002-1 · Cited by 3
Cavero, I., & Holzgrefe, H. H. (2019) · Expert Opinion on Drug Safety · doi:10.1080/14740338.2019.1673726 · Cited by 3
Les pratiques de recherche ouvertes en psychologie
Lantian, A. (2020) · Psychologie Française · doi:10.1016/j.psfr.2020.09.001 · Cited by 6
Rethinking data and metadata in the age of machine intelligence
Bittner, M.-I. (2021) · Patterns · doi:10.1016/j.patter.2021.100208 · Cited by 4
“on irreproducible laboratory-based biomedical research in the United States alone.(3) Solving this crisis will require more than merely improving data management practices for the results of scientific inquiries.”
BackgroundSkills and education for engineering biology
Philp, J. C. (2022) · Elsevier eBooks · doi:10.1016/b978-0-12-818272-7.00005-5 · Cited by 3
Mind the gap! A survey of the challenges of biomarker commercialization
Raths, S., Parkel, S., Bredmose, J., & Daussin, V. (2019) · Drug Discovery Today · doi:10.1016/j.drudis.2019.09.015 · Cited by 3
Above a Swamp: A Theory of High-Quality Scientific Production
Kiri, B., Lacetera, N., & Zirulia, L. (2015) · National Bureau of Economic Research · doi:10.3386/w21143 · Cited by 6
Metadata
- Authors
- Freedman, L. P., Cockburn, I. M., & Simcoe, T. S.
- Affiliations
- Global Biological Standards Institute (US)
- Boston University (US)
- Corresponding author
- Freedman, Leonard P.
- Venue
- PLOS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-06-09
- Volume
- 13
- Issue
- 6
- Pages
- e1002165
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC4461318
- OpenAlex
- W585341442
- Cited by
- 1,055
- References
- 35
- Citations by year
- 2026: 48 · 2025: 45 · 2024: 69 · 2023: 92 · 2022: 93 · 2021: 96 · 2020: 142 · 2019: 116 · 2018: 109 · 2017: 93 · 2016: 103 · 2015: 49
- FWCI
- 188.58
- Type
- journal article
- Language
- en
- Update notices
- correction (2018-04-10)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- OpenBU (Boston University) (submittedVersion, cc-by)
- Topics
- Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health and Medical Research Impacts
- Keywords
- reproducibility, preclinical research, research costs, research waste, reproducibility framework
- MeSH terms
- Reproducibility of Results, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.