The Economics of Reproducibility in Preclinical Research
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Low reproducibility rates within life science research undermine cumulative knowledge production and contribute to both delays and costs of therapeutic drug development. An analysis of past studies indicates that the cumulative (total) prevalence of irreproducible preclinical research exceeds 50%, resulting in approximately US$28,000,000,000 (US$28B)/year spent on preclinical research that is not reproducible-in the United States alone. We outline a framework for solutions and a plan for long-term improvements in reproducibility rates that will help to accelerate the discovery of life-saving therapies and cures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Fig 1 Studies reporting the prevalence of irreproducibility. Source: Begley and Ellis [6], Prinz et al. [7], Vasilevsky [8], Hartshorne and Schachner [5], and Glasziou et al. [9].](https://figures.cochat.ai/paper-figures/a30bd9bd65dc4bedb5db19b413eb814f/the-economics-of-reproducibility-in-preclinical-research-figure-1.jpg)
![Fig 2 Estimated US preclinical research spend and categories of errors that contribute to irreproducibility. Note that the percentage value of error for each category is the midpoint of the high and low prevalence estimates for that category divided (weighted) by the sum of all midpoint error rates (see S1 Dataset). Source: Chakma et al. [18] and the American Association for the Advancement of Science (AAAS) [19].](https://figures.cochat.ai/paper-figures/207a0a58df5c4620a819d7aa1f4aa960/the-economics-of-reproducibility-in-preclinical-research-figure-2.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Besides, reliably determining actual translational success rates is unmanageable as long as the current status of reporting of preclinical research leaves room for improvement [89], and non-reproducibility is such a critical issue in both animal [1] and human [90] studies.”
From Animal to human translation: a systematic scoping review of reported concordance rates (2019) · cites this paper as [1]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In the field of biomedical research, the majority of incorrect results are estimated to derive from the incorrect use of material standards and experimental reagents [2, 3].”
From Semi-automated fact-checking of nucleotide sequence reagents in biomedical research publications: The Seek & Blastn tool (2019) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The S&B tool therefore directly addresses the larger problem of material reagents and standards representing the major source of incorrect published results from pre-clinical research [3, 4].”
From Semi-automated fact-checking of nucleotide sequence reagents in biomedical research publications: The Seek & Blastn tool (2019) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current drug development is handicapped by high attrition rates; many molecules that were promising during preclinical development fail during subsequent clinical testing [1].”
From Animal to human translation: a systematic scoping review of reported concordance rates (2019) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This observation has been followed by a number of additional reports, the summary of which is, in essence, that 77-90% of biomedical experiments are not reproducible [14-17].”
From Guidelines for the use of flow cytometry and cell sorting in immunological studies (third edition) (2021) · cites this paper as [16]
Experimental — accuracy not yet measured
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Governing the Commons: The Evolution of Institutions for Collective Action
Field, B. C., & Ostrom, E. (1992) · Land Economics · doi:10.2307/3146384 · Cited by 18,859
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 2×
“Source: Begley and Ellis [6], Prinz et al.”
BackgroundBelieve it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · cited 1×
“[7], Vasilevsky [8], Hartshorne and Schachner [5], and Glasziou et al.”
BackgroundIncreasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597 · cited 1×
“However, it is reasonable to state that cumulative errors in the following broad categories—as well as underlying biases that could contribute to each problem area [14] or even result in entire studies never being published or reported [15]—are the primary causes of irreproducibility [16]: (1) study design, (2) biolog…”
BackgroundPolicy: NIH plans to enhance reproducibility
Collins, F. S., & Tabak, L. A. (2014) · Nature · doi:10.1038/505612a · Cited by 1,247
A call for transparent reporting to optimize the predictive value of preclinical research
Landis, S. C., Amara, S. G., Asadullah, K., Austin, C. P., Blumenstein, R., Bradley, E. W., Crystal, R. G., Darnell, R. B., Ferrante, R. J., Fillit, H., et al. (2012) · Nature · doi:10.1038/nature11556 · Cited by 1,196
Reproducibility in Science: Improving the Standard for Basic and Preclinical Research
Begley, C. G., & Ioannidis, J. P. A. (2015) · Circulation Research · doi:10.1161/circresaha.114.303819 · Cited by 1,035 · cited 1×
“Much has been written about the alarming number of preclinical studies that were later found to be irreproducible [1,2].”
BackgroundBiomedical research: increasing value, reducing waste.
Macleod, M. R., Michie, S., Roberts, I., Dirnagl, U., Chalmers, I., Ioannidis, J. P. A., Salman, R. A.-S., Chan, A.-W., & Glasziou, P. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62329-6 · Cited by 1,003
Partial Identification of Probability Distributions
(2003) · Springer Series in Statistics · doi:10.1007/b97478 · Cited by 698
What is missing from descriptions of treatment in trials and reviews?
Glasziou, P., Meats, E., Heneghan, C., & Shepperd, S. (2008) · BMJ · doi:10.1136/bmj.39590.732037.47 · Cited by 614
Research works printed but not linked: 1
- CF Manski, 2003, Springer Series in Statistics, 178
Cited by
Items 701–800 of 1,003
A Practical Guide to Translating Nanomedical Products
Bawa, R. (2016) · Wiley eBooks · doi:10.1002/9783527800681.ch28 · Cited by 3
Mouse Model Systems of Autism Spectrum Disorder: Replicability and Informatics SignaturePreprint
Kabitzke, P., Morales, D. V., He, D., Cox, K., Sutphen, J. C., Thiede, L., Sabath, E. V., Hanania, T., Biemans, B., & Brunner, D. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/561233 · Cited by 3
Janero, D. R. (2021) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2021.1893690 · Cited by 3
“This infodemic of irreproducible published data has been estimated to encompass 50% of all published preclinical life-science research and waste around 28 USD billion annually in the United States alone [11].”
BackgroundSmith, N. A. S., Romanchikova, M., Partarrieu, I. X., Cooke, E., Lemańska, A., & Thomas, S. A. (2021) · doi:10.47120/npl.ms31 · Cited by 3
Assay Guidance Manual for Drug Discovery: Technologies That Matter
Markossian, S., Coussens, N. P., Dahlin, J. L., & Sittampalam, G. S. (2021) · SLAS TECHNOLOGY · doi:10.1177/24726303211056338 · Cited by 3
Holden, C. J., & Goodwin, G. S. (2018) · Springer eBooks · doi:10.1007/978-3-319-28099-8_1352-1 · Cited by 4
A change to Experimental Physiology’s statistics policy
Bailey, D. M., Rose, G. A., & Stewart, A. (2023) · Experimental Physiology · doi:10.1113/ep091248 · Cited by 3
Author Country of Origin and Attention on Open Science Platforms: Evidence from COVID-19 Preprints
Fry, C., & MacGarvie, M. J. (2023) · National Bureau of Economic Research · doi:10.3386/w31565 · Cited by 3
A FAIR Workflow Guide for Researchers in Human Cognitive NeurosciencePreprint
Zheng, Z., Chen, X., Brown, T., Cousijn, H., & Melloni, L. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/yhj5c · Cited by 3
Brief Reports Prepared by Students and Graduates of Master of Science in Biobanking
(2022) · Biopreservation and Biobanking · doi:10.1089/bio.2022.0064 · Cited by 2
“The workflow analysis enabled us to identify 13 main process categories and 123 requirements, which were grouped into 15 clusters and separated into 3 distinct categories: (1) functional requirements, (2) nonfunctional requirements, and (3) system operation.”
BackgroundMethodClark, J. S. C., Podsiadło, K., Sobalska‐Kwapis, M., Marciniak, B., Rydzewska, K., Ciechanowicz, A., van de Wetering, T., & Strapagiel, D. (2023) · Scientific Reports · doi:10.1038/s41598-023-43323-3 · Cited by 3
“In the medical sciences there is overwhelming evidence that result replication, which should involve an (inherent or enhanced) attempt at resetting (unforeseen, known and unknown) systematic-error values, should be an absolute requirement 30–32 .”
BackgroundDrapkina, O. M., Martsevich, S. Y., Bubnova, M. G., Gasanova, L. G., Kotova, M. B., Кутишенко, Н. П., Lukina, Y. V., Metelskaya, V. A., Леонидовна, П.-Д. А., Александровна, П. Е., et al. (2023) · doi:10.15829/ropniz-d88-2023 · Cited by 2
Utilization Patterns of Hospital Biobank Samples: Time to First Use and Material-Specific Demand
Mucher, P., Bayer, M., Flieder, I., Humer, G. T., Hou, L., Koll, A., Radakovics, A., Ristic, N., Schickbauer, T., Müller, W., et al. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15114292 · Cited by 2
“On the other hand, these facilities aim to increase research reliability by applying validated processing methods [2] and contributing their expertise regarding sample fitness for purpose, thereby addressing the problem of non-reproducibility [3–5].”
Model-as-evidence framework established using AI-integrated organoids and organs-on-chips systems
Wang, P., Santoso, B., He, S., Zhou, L., Zhang, Y., Zhang, Y., Ho, S. Y., Tuan, R. S., & Li, Z. A. (2026) · International Journal of AI for Materials and Design · doi:10.36922/ijamd026150008 · Cited by 3
Biddle, M., Cooper, J., Blades, K., Ruddy, D., Krockow, E. M., & Virk, H. S. (2026) · PLoS Biology · doi:10.1371/journal.pbio.3003984 · Cited by 2
“While Freedman and colleagues estimated approximately $28 billion annually spent on irrepro-ducible preclinical research in the United States, with biological reagents contributing 36.1% [11], the ethical dimension— measured in animal lives and human tissue donations—had remained unquantified.”
InfluentialSinha, R., & Sudhish, P. S. (2016) · IEEE International Symposium on Ethics in Engineering, Science, and Technology (ETHICS) · doi:10.1109/ethics.2016.7560050 · Cited by 2
Biomedical Research in the 21st Century: Multiple Challenges in Resolving Reproducibility Issues
Mullane, K., Curtis, M. J., & Williams, M. J. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-804725-5.00006-9 · Cited by 4
Support for Living Stock Collections: A Mammalian Stock Center Perspective
Kiaris, H. (2018) · Trends in Genetics · doi:10.1016/j.tig.2018.12.001 · Cited by 2
McKenzie, K. A., Hunt, S. L., Hulshof, G., Mudaranthakam, D. P., Meyer, K., Vidoni, E. D., Burns, J. M., & Mahnken, J. D. (2019) · JAMIA Open · doi:10.1093/jamiaopen/ooz032 · Cited by 2
Is collective nest site selection in ants influenced by the anchoring effect?
Doering, G. N., Talken, L. W., Pratt, S. C., & Sasaki, T. (2023) · Behavioural Processes · doi:10.1016/j.beproc.2023.104861 · Cited by 3
Brown, K. H., Ghita, M., Prise, K. M., & Butterworth, K. T. (2023) · F1000Research · doi:10.12688/f1000research.130883.1 · Cited by 2
“50 This study shows the feasibility of using BioXmark ® in preclinical models of RT, it ’ s use can be adapted within preclinical radiobiology centres worldwide to improve the visualisation of orthotopic tumours and the translational impact of studies.”
MethodFecal microbiota impacts development of Cryptosporidium parvum in the mouse
Widmer, G., & Creasey, H. N. (2024) · Scientific Reports · doi:10.1038/s41598-024-56184-1 · Cited by 2
“The observed fluctuation in microbiota composition, even among mice originating from the same vendor, raises research reproducibility issues as has previously been discussed 32,33 .”
Mika, T., Kalniņš, M., & Spalviņš, K. (2024) · Biology Methods and Protocols · doi:10.1093/biomethods/bpae049 · Cited by 2
“Also, we would like to mention cell-line contamination, which is not a trivial problem and brings significant experimental errors in biology and medicine [276] and is costly [277].”
BackgroundApplication of preclinical juvenile animal models
Hermans, E., Toelen, J., Ventrella, D., Schaepdrijver, L. D., Turner, K. J., Croubels, S., & Devreese, M. (2024) · Elsevier eBooks · doi:10.1016/b978-0-323-88459-4.00009-2 · Cited by 3
Esophageal adenocarcinoma models: a closer look
Bhat, N. S., Al‐Mathkour, M. M., Maacha, S., Lu, H., El-Rifai, W. M., & Ballout, F. (2024) · Frontiers in Molecular Biosciences · doi:10.3389/fmolb.2024.1440670 · Cited by 3
Nonlinear mixed-effects models to analyze actin dynamics in dendritic spines
Credico, G. D., Pelucchi, S., Pauli, F., Stringhi, R., Marcello, E. E., & Edefonti, V. C. (2025) · Scientific Reports · doi:10.1038/s41598-025-87154-w · Cited by 3
“In recent years, concerns about the low standard of statistical analysis in preclinical studies have arisen and have promoted discussion about the use of more sophisticated statistical methods 15,29 .”
The Reproducibility Crisis and Kinesiology: Confronting the Human & Statistical Limitations
Twietmeyer, G. (2025) · International Journal of Kinesiology in Higher Education · doi:10.1080/24711616.2025.2468469 · Cited by 2
“Freedman et al. (2015) suggest that in the United States alone, $28 billion a year is wasted on non-reproducible research.”
Digital publishing isn't enough: the case for ‘blueprints’ in scientific communication
Jennings, L. D., & Gardner, T. S. (2018) · Emerging Topics in Life Sciences · doi:10.1042/etls20180165 · Cited by 2
Halim, A. B. (2019) · Elsevier eBooks · doi:10.1016/b978-0-12-816121-0.00011-8 · Cited by 2
Anderson, J., Niemann, A., Johnson, A. L., Cook, C., Tritz, D. J., & Vassar, M. (2019) · JMIR Dermatology · doi:10.2196/16078 · Cited by 3
“Scientific research is currently facing a reproducibility crisis, with an estimated 50% to 90% of research having been suggested to be irreproducible [1-3].”
BackgroundMedizinische Forschung – Qualität vor Quantität
Hostettler, S., Kraft, E., & Bosshard, C. (2015) · Bulletin des Médecins Suisses · doi:10.4414/saez.2015.04171 · Cited by 3
Tchao, R. (2021) · BMC Research Notes · doi:10.1186/s13104-021-05453-7 · Cited by 2
Scholarly Publishing and Scientific Reproducibility
Bomzon, A., & Tobin, G. (2021) · Laboratory animal science and medicine · doi:10.1007/978-3-030-66147-2_9 · Cited by 3
Vergov, B., Sbirkov, Y., Minchev, D. S., Todorova, T., Baldzhieva, A., Burnusuzov, H. A., Spasova, M., & Sarafian, V. S. (2024) · Biology Methods and Protocols · doi:10.1093/biomethods/bpae042 · Cited by 2
An overview of Synlab SDN Biobank’s quality control system
Coppola, L., Grimaldi, A., Sarnacchiaro, G., di Fasano, M. S., Smaldone, G., & Salvatore, M. (2024) · Scientific Reports · doi:10.1038/s41598-024-70263-3 · Cited by 2
Bioassay protocol metadata annotation: Proposed standards adoption
Balakrishnan, R., Berg, E. L., Butler, C. C., Clark, A. M., Denker, S. P., Feierberg, I., Harris, J., Ikeda, T., Jeschonek, S. P., Makarov, V. A., et al. (2024) · SLAS DISCOVERY · doi:10.1016/j.slasd.2024.100188 · Cited by 2
“The development and general adoption of a minimal set of metadata, standardized for each class of bioassay, would improve reproducibility [5,6], interpretability, and data mining across the scientific community.”
BackgroundVaidyanathan, U., Vrieze, S. I., & Iacono, W. G. (2015) · Psychological Inquiry · doi:10.1080/1047840x.2015.1064066 · Cited by 2
Biometrology in Tissue Engineering: Thoughts and Concepts
Baptista, L. S., Silva, K. R., Beatrici, A., Fontes, G. N., Granjeiro, J. M., & Boldrini, L. C. (2017) · Journal of Scientific and Industrial Metrology · doi:10.21767/2472-1948.100008 · Cited by 2
Assessing the utility of an institutional publications officer: a pilot assessment
Cobey, K. D., Galipeau, J., Shamseer, L., & Moher, D. (2017) · PeerJ · doi:10.7717/peerj.3294 · Cited by 2
Quality and Basic Biomedical Research
Lanati, A. (2018) · Springer eBooks · doi:10.1007/978-3-319-76750-5_4 · Cited by 2
APRICOT: Advanced Platform for Reproducible Infrastructures in the Cloud via Open Tools
Giménez-Alventosa, V., Segrelles, J. D., Moltó, G., & Roca-Sogorb, M. (2020) · Methods of Information in Medicine · doi:10.1055/s-0040-1712460 · Cited by 2
“2 As a result, scienti fi c reproducibility has lately become a topic of interest for researchers and institutions which defend an open and reproducible science model.”
BackgroundProduction of Recombinant Gelonin Using an Automated Liquid Chromatography System
Berstad, M. E. B., Cheung, L. H., & Weyergang, A. (2020) · Toxins · doi:10.3390/toxins12080519 · Cited by 2
“A study published in 2015 estimates that, in the United States alone, approximately 28 billion USD are invested annually on preclinical research that is not reproducible [3].”
BackgroundA regulatory framework for the development of topical nanomedicines
Miranda, M., Cardoso, C., & Vitorino, C. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-820466-5.00003-x · Cited by 2
The imperative to find the courage to redesign the biomedical research enterprise
Bielekova, B., & Brownlee, S. (2021) · F1000Research · doi:10.12688/f1000research.53654.1 · Cited by 3
Kanterakis, A., Kanakaris, N., Koutoulakis, M., Pityanou, K., Karacapilidis, N. I., Koumakis, L., & Potamias, G. (2021) · Applied Sciences · doi:10.3390/app11209648 · Cited by 2
Krebs, A. (2019) · ALTEX · doi:10.14573/altex.1909271s · Cited by 19
“In line with reports by many others (Freedman et al., 2015; Vogt et al., 2016; Hair et al., 2019), our own survey of the scientific literature has indicated that method descriptions still show an enormous heterogeneity in quality, detail and scope.”
BackgroundResultNovel hydrogels: are they poised to transform 3D cell-based assay systems in early drug discovery?
Treherne, J. M., & Miller, A. F. (2023) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2023.2175813 · Cited by 2
“These have sparked much debate about biological data reproducibility in general [12,13].”
BackgroundUsing Primates in Captivity: Research, Conservation, and Education
Prescott, M. J. (2023) · Springer eBooks · doi:10.1007/978-3-030-82708-3_3 · Cited by 9
Yang, Z., Kubant, R., Kranenburg, E., Cho, C. E., & Anderson, G. H. (2024) · Nutrients · doi:10.3390/nu16050696 · Cited by 2
“These results support the call for the reproducibility of results in different experimental settings before conclusions are drawn and applied from preclinical research [1–3].”
BackgroundResultDietary data under fire for being unreliable
Woolston, C. (2015) · Nature · doi:10.1038/522259f · Cited by 2
The Brief Pain Inventory (BPI) – Revisited and rejuvenated?
Werner, M. U. (2016) · Scandinavian Journal of Pain · doi:10.1016/j.sjpain.2016.04.003 · Cited by 2
Visual Evoked Potential Recordings in Mice Using a Dry Non-invasive Multi-channel Scalp EEG Sensor
Yeon, C., Kim, D., Kim, K., & Chung, E. (2018) · Journal of Visualized Experiments · doi:10.3791/56927 · Cited by 2
Artificial Neural Network enabling Clinically Meaningful Biological Image Data Generation
Ha, J., Kim, S., Baik, Y., Lee, D., Lee, W., & Suh, S. (2020) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc44109.2020.9176621 · Cited by 2
Biomedical research at the crossroads and ways through
Pons, J. M. V., & Adam, P. (2018) · Medicina Clínica (English Edition) · doi:10.1016/j.medcle.2018.05.036 · Cited by 2
Aggregating Knockoffs for Controlling the False Discovery Rate with One-Shot CommunicationPreprint
Su, W., Qian, J., & Liu, L. (2015) · arXiv (Cornell University) · Cited by 2
Supporting More Reliable Results
Webb, S. (2015) · BioTechniques · doi:10.2144/000114314 · Cited by 3
Crisis in What Exactly?Preprint
Guest, O. (2016) · doi:10.15200/winn.146590.01538 · Cited by 2
Proteomics of blood plasma/serum samples stored in biobanks: insights for clinical application
Tissot, J., Currat, C., & Sprumont, D. (2017) · Expert Review of Proteomics · doi:10.1080/14789450.2017.1324301 · Cited by 2
“Thus, quality is the cornerstone of a biobank reputation and should be observed from consent to return of results, from collection to storage of samples and data, from data storage to data transfer, and from research project selection to biobank promotion [9].”
BackgroundFighting for the Future of Antibodies
Blow, N. S. (2017) · BioTechniques · doi:10.2144/000114573 · Cited by 2
An, G. C. (2018) · Preprints.org · doi:10.20944/preprints201805.0308.v2 · Cited by 2
Organizing laboratory information to analyze the reproducibility of experimental workflowsPreprint
Peccoud, J., Johnson, D., Peccoud, S. J., Setchell, J., & Zhou, W. (2022) · bioRxiv · doi:10.1101/2022.04.05.487214 · Cited by 2
Balas, E. A. (2018) · Wiley eBooks · doi:10.1002/9781119225898.ch6 · Cited by 2
An, G. C. (2018) · Preprints.org · doi:10.20944/preprints201805.0308.v1 · Cited by 2
Müller, T. (2022) · Cells · doi:10.3390/cells12010157 · Cited by 2
“The translation of promising experimental outcomes with high expectations into the clinic failed with corresponding huge financial disappointments [2,3].”
BackgroundThe Rise of "Vibe Science": How AI Threatens Scientific RigorPreprint
Rose, I., Celi, L. A., Corti, C., Ellen, J., Fiske, A., Kebede, M. A., Kwak, G. H., Lunde, T. M., Cajas, S. A., Pucher, G., et al. (2025) · doi:10.36227/techrxiv.175606229.90717623/v1 · Cited by 3
Clark, J. S. C., van de Wetering, T., Marciniak, B., Żądzińska, E., Ciechanowicz, A., Kaczmarczyk, M., Boroń, A., Rydzewska, K., Posiadło, K., & Strapagiel, D. (2022) · Scientific Reports · doi:10.1038/s41598-022-21638-x · Cited by 1
Wulczynski, M., Brooks, S. P. J., Green, J., Matias, F. M. G., Kalmokoff, M. L., Green-Johnson, J. M., & Clarke, S. T. (2025) · Animal Microbiome · doi:10.1186/s42523-024-00369-6 · Cited by 3
Evaluating Complexity in Orthopedic Tissue‐on‐a‐Chip Systems
Zlotnick, H. M., Goddard, D. N., & Burdick, J. A. (2025) · Advanced Healthcare Materials · doi:10.1002/adhm.202504275 · Cited by 1
Ehlich, H., Blease, A., Biju, R., Gustems, M., Song, F., Fessele, S., Massimi, M., Bozonelos, K., Ntafis, V., Hiltunen, A. E., et al. (2025) · Mammalian Genome · doi:10.1007/s00335-025-10174-4 · Cited by 1
“Reproducibility remains a critical issue in science, particularly in preclinical research, where irreproducibility rates range from 51 to 89% (Freedman et al. 2015 ; Baker 2016 ).”
Neuroelectromagnetic Data Archive and Repository: Open-source platform analysis
Brandmeyer, T., Riggs, A., Reddi, V., Sullivan, H., Youn, C., Sivagnanam, S., Yoshimoto, K. K., Poldrack, R. A., Hardcastle, N., Markiewicz, C. J., et al. (2026) · Aperture Neuro · doi:10.52294/001c.154162 · Cited by 1
Delivering Digital Transformation with Self-reporting Data Assets
Corte, D. D., Young, N. J., & Corte, K. W. D. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-89906-6_62 · Cited by 1
The importance of metrological tools to implementation of alternative method OECD TG 428
Geaquinto, L. R. O., de Souza, V., do Rego, E. C. P., Silva, M. L., & Balottin, L. B. L. (2022) · Toxicology in Vitro · doi:10.1016/j.tiv.2022.105425 · Cited by 1
Bontoux, C., Marcovich, A., Goffinet, S., Pesce, F., Tanga, V., Bohly, D., Salah, M., Washetine, K., Messaoudi, Z., Felix, J.-M., et al. (2023) · Journal of Personalized Medicine · doi:10.3390/jpm13071076 · Cited by 1
“The quality of the obtained samples varies greatly between structures and organizations and can lead to inconsistent and non-reproducible results that may affect the economic sustainability of the research [18].”
BackgroundData welfare is animal welfare: Building a WellFAIR research ecosystem
Petit‐Demoulière, B., & Huzard, D. (2026) · Neuroscience Applied · doi:10.1016/j.nsa.2026.106998 · Cited by 2
“The economic implications are staggering — estimated at $28 billion annually in the US alone (Freedman et al., 2015) — but the scientific consequences are even greater: we are attempting to build AI models on a foundation of fragmented, often poorly contextualised data.”
Lohse, S. (2026) · European Journal for Philosophy of Science · doi:10.1007/s13194-026-00727-y · Cited by 2
“…of this kind have, together with more general concerns regarding progress in drug develop - ment, motivated much recent work on methodological issues in preclinical research and led to the finding that much if not most animal-based research is in fact irrepro - ducible (Freedman et al., 2015).”
Designing studies for post-treatment Lyme disease and other infection-associated chronic illnesses
Arnaboldi, P. M., Becker, J., Nath, A., Coyle, P. K., Handel, A. S., Sellati, T. J., Gomes-Solecki, M. J. C., Garcet, S., Henderson, M. K., Mullins, P., et al. (2026) · Brain · doi:10.1093/brain/awag016 · Cited by 2
Schätz, M., & Sugawara, K. (2026) · Journal of Microscopy · doi:10.1111/jmi.70139 · Cited by 1
Actionable solutions to address antibody validation failures
Blades, K., Biddle, M., Froud, R., Krockow, E. M., & Virk, H. S. (2026) · PLoS Biology · doi:10.1371/journal.pbio.3003981 · Cited by 2
“An estimated $28.2 billion per year is spent in the United States on preclinical research that is not reproducible, with issues relating to biological reagents likely being the biggest contributor [11].”
InfluentialVillain of Molecular Biology: Why are we not reproducible in research?
Bhardwaj, V. (2015) · F1000Research · doi:10.12688/f1000research.6854.1 · Cited by 2
Evolvix BEST Names for semantic reproducibility across code2brain interfaces
Loewe, L., Scheuer, K. S., Keel, S. A., Vyas, V., Liblit, B., Hanlon, B. M., Ferris, M. C., Yin, J., Dutra, I., Pietsch, A., et al. (2016) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13192 · Cited by 1
Reproducibility of biomedical research: Quo vadis?
Dal‐Ré, R., Bernad, A., & Garesse, R. (2016) · Medicina Clínica (English Edition) · doi:10.1016/j.medcle.2016.06.029 · Cited by 1
Ontology Based Data Architecture to Promote Data Sharing in Electrophysiology.
Farrell, B., & Bengtson, J. (2018) · ICBO · Cited by 1
A teaching tool about the fickle p value and other statistical principles based on real-life data
Alawbathani, S., Batool, M., Fleckhaus, J., Hamad, S., Hassenrück, F., Hou, Y., Li, X., Salmanton‐García, J., Ullah, S., Wieters, F., et al. (2021) · Naunyn-Schmiedeberg s Archives of Pharmacology · doi:10.1007/s00210-020-02045-3 · Cited by 1
“An alarmingly high fraction of published research in experimental biomedicine has been found not to be reproducible or replicable (Freedman et al. 2015).”
BackgroundKane, M. (2020) · Physiology News · doi:10.36866/pn.118.7 · Cited by 1
Sex in studies: The first level of personalisation
Karp, N. A., Katial, R. K., & Thacker, K. S. (2020) · Physiology News · doi:10.36866/pn.118.32 · Cited by 2
The Changing Face of Biological Research and the Growing Role of Biosecurity
Dunaway, N., & Berger, K. M. (2021) · Advanced sciences and technologies for security applications · doi:10.1007/978-3-030-69464-7_6 · Cited by 2
Management of Metadata Types in Basic Cardiological Research
Kusch, H., Kossen, R., Suhr, M., Freckmann, L., Weber, L., Henke, C., Lehmann, C. U., Rheinländer, S., Aschenbrandt, G., Kühlborn, L. K., et al. (2021) · Studies in health technology and informatics · doi:10.3233/shti210542 · Cited by 1
“However, the problem is heterogeneous as the irreproducibility is caused by very different factors [4,5], which require diverse approaches to overcome them.”
BackgroundZhuang, W. V., Camacho, L., Silva, C. S., Thomson, M., & Snyder, K. (2022) · PLoS ONE · doi:10.1371/journal.pone.0263070 · Cited by 1
“These characteristics include the principle underlying qPCR data normalization, the utilization of a fluorescence threshold in a qPCR experiment, and the potential for a few or some observations to be censored at the end of a well-conducted qPCR experiment [32, 33].”
BackgroundMethodHoward, J. (2018) · Springer eBooks · doi:10.1007/978-3-319-93224-8_30 · Cited by 1
Reflections and images: A place for art in medical physics?
Malone, J. (2022) · Physica Medica · doi:10.1016/j.ejmp.2022.04.004 · Cited by 1
Why use a biobank? Sharing mutations, gene editing and navigating the COVID-19 pandemic.
Allen, S., Summerfield, R., & Fray, M. (2023) · Medical Research Archives · doi:10.18103/mra.v11i6.3871 · Cited by 1
Menin, B. M. (2023) · Journal of Applied Mathematics and Physics · doi:10.4236/jamp.2023.117117 · Cited by 1
Võikar, V., Casarotto, P. C., Glerean, E., Laakso, K., Saurio, K. J., Karhulahti, V., & Scherer, A. (2023) · Journal for Reproducibility in Neuroscience · doi:10.36850/jrn.2023.e47 · Cited by 1
Overview of preclinical research
Wu, S. Y., Sriram, V., Shi, B. Y., & Kremen, T. J. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-91259-4.00110-7 · Cited by 1
The replication crisis in science and its overlap into kinantropology: an introduction
Baloun, L., & Mališ, J. (2023) · Tělesná kultura · doi:10.5507/tk.2023.002 · Cited by 1
Boosting reproducible research practices with the repeat it with me: Reproducibility team challenge
Bortolotti, L., Schauman, S., & Caligiuri, M. E. (2024) · Magnetic Resonance in Medicine · doi:10.1002/mrm.30041 · Cited by 1
“29,30 The common theme revolves around a skewed reward system that prioritizes “novelty,” resulting in a push to follow “perverse incentives.””
BackgroundNeves, M. M., Klein, S. G., Silva, R. C., Bernardes, L. M. M., Malta, S. M., Vieira, T. N., Rosa, R. B., de Lima, I. L., Ferreira, F. B., & da Silva, M. V. (2024) · Frontiers in Veterinary Science · doi:10.3389/fvets.2024.1394113 · Cited by 1
“This practice significantly impacts the reproducibility of data, as it standardizes protocols and makes documentation on these procedures more robust (2) It has already been raised that the omission of data has contributed to the reproducibility crisis (3, 10).”
BackgroundZatloukal, K., & Kungl, P. (2024) · Elsevier eBooks · doi:10.1016/b978-0-323-91799-5.00021-8 · Cited by 1
Abstract Hardware Grounding Towards the Automated Design of Automation Systems
Shi, Y.-Z., Qiao, X., Meng, F., Ruan, L., & Wang, Q. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-0780-8_9 · Cited by 2
A Western Blot and Immunoprecipitation Assay to Verify Antibody Specificity
Bonnette, S. G. (2015) · BioTechniques · doi:10.2144/000114332 · Cited by 1
Metadata
- Authors
- Freedman, L. P., Cockburn, I. M., & Simcoe, T. S.
- Affiliations
- Global Biological Standards Institute (US)
- Boston University (US)
- Corresponding author
- Freedman, Leonard P.
- Venue
- PLOS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-06-09
- Volume
- 13
- Issue
- 6
- Pages
- e1002165
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC4461318
- OpenAlex
- W585341442
- Cited by
- 1,055
- References
- 35
- Citations by year
- 2026: 48 · 2025: 45 · 2024: 69 · 2023: 92 · 2022: 93 · 2021: 96 · 2020: 142 · 2019: 116 · 2018: 109 · 2017: 93 · 2016: 103 · 2015: 49
- FWCI
- 188.58
- Type
- journal article
- Language
- en
- Update notices
- correction (2018-04-10)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- OpenBU (Boston University) (submittedVersion, cc-by)
- Topics
- Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health and Medical Research Impacts
- Keywords
- reproducibility, preclinical research, research costs, research waste, reproducibility framework
- MeSH terms
- Reproducibility of Results, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.