Raise standards for preclinical cancer research
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index; 3 of the 9 sources its claims cite were checked for support.
Overview
CoChat Summary
TL;DR
This Nature Comment argues that many landmark preclinical oncology findings are not reproducible and calls for higher standards in methods, reporting, and incentives. It reports that Amgen confirmed only 6 of 53 landmark papers (11%) and cites Bayer’s finding that about 25% of published preclinical studies could be validated.
Key findings
- In Amgen’s ten-year retrospective analysis, scientific findings were confirmed in only 6 of 53 “landmark” preclinical cancer papers (11%); non-reproduced studies often lacked blinding, relied on single experiments such as one Western blot, or presented selected data.
- Bayer HealthCare reported that only about 25% of published preclinical studies could be validated to the point at which projects could continue; published cancer research represented 70% of the studies analysed in that report, with possible overlap with Amgen’s 53 papers.
- Citation data showed non-reproduced articles in journals with impact factor >20 had a mean of 248 citations (range 3–800) versus 231 (range 82–519) for reproduced articles; for impact factor 5–19, non-reproduced articles had a mean of 169 citations (range 6–1,909) versus 13 (range 3–24) for reproduced articles.
- The authors recommend reporting negative data, blinding preclinical experiments, validating reagents, repeating critical experiments, avoiding single cell lines or models, requiring patient-selection biomarkers, and changing academic incentives.
Methods
- This is a Nature Comment/perspective, not a primary experimental study; it synthesizes Amgen’s ten-year retrospective analysis of experiments performed prospectively and cites a Bayer HealthCare report.
- Amgen selected 53 “landmark” papers describing completely new approaches or alternative clinical uses, attempted confirmation, used additional models, contacted original authors, exchanged reagents, and repeated experiments under the authors’ direction, occasionally in the original investigator’s laboratory.
- The analysis reports counts of confirmed findings and citation metrics from Google Scholar, May 2011; the recommendations are proposed changes rather than tested interventions.
Limitations
- Stated: validation attempts may have failed because of technical differences or difficulties despite efforts to ensure otherwise; additional models were used; when findings could not be reproduced, authors were contacted and experiments were repeated under their direction.
- Stated: papers were deliberately selected as “landmark” and described something completely new, so some data might not hold up; Bayer’s analysed studies may overlap with Amgen’s 53 papers.
- Evident: as a Comment, the text provides no detailed systematic protocol, inclusion criteria, statistical methods, or full dataset; the citation analysis is descriptive and limited to two impact-factor strata.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 1 Not valid · 6 Not checked · 12 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Sadly, clinical trials in oncology have the highest failure rate compared with other therapeutic areas.”
What the sources say
- Partly supportedcites [1] Hutchinson, L. & Kirk, R. Nature Rev. Clin. Oncol. 8, 189–190 (2011). sourceExperimental — accuracy not yet measured
The opening passage of our copy (labelled Abstract; page unavailable) says cancer-drug attrition is much higher than in other therapeutic areas, citing 5% of anticancer agents versus 20% of cardiovascular agents reaching licensure. That comparison covers development from preclinical activity through phase III, not failure rates specifically among clinical trials; it also does not explicitly establish a highest-rate ranking. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Although we in the cancer field hoped that this would lead to more effective drugs, historically, our ability to translate cancer research to clinical success has been remarkably low 1 .”
What the sources say
- Supportedcites [1] Hutchinson, L. & Kirk, R. Nature Rev. Clin. Oncol. 8, 189–190 (2011). source
The opening passage (labeled Abstract in our copy; no page given) reports that only 5% of agents with preclinical anticancer activity are licensed after demonstrating sufficient efficacy in phase III testing. It also says cancer drug attrition is much higher than in other therapeutic areas, supporting the claim of historically low translation to clinical success. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In studies for which findings could be reproduced, authors had paid close attention to controls, reagents, investigator bias and describing the complete data set.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For results that could not be reproduced, however, data were not routinely analysed by investigators blinded to the experimental versus control groups.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Investigators frequently presented the results of one experiment, such as a single Western-blot analysis.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
12 more claims
Create a free account to see all 22 claims and every source verdict.
- It may state two separate claims as one. 2
- Probably not a substantial claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“there are other more common sources of unreliability in the literature (41)(42)(43)(44)”
From Misconduct accounts for the majority of retracted scientific publications (2012) · cites this paper as [42]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“many established scientific results in the social (49, 50) and biomedical (51-53) sciences cannot readily be replicated.”
From Misinformation in and about science (2021) · cites this paper as [51]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A landmark study conducted by Glenn Begley and his team in 2012 concluded that most cancer research studies were not reproducible, by alarmingly reporting that 47 of 53 published findings could not be replicated [13].”
From Guidelines for the use of flow cytometry and cell sorting in immunological studies (third edition) (2021) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In cell biology, two industrial laboratories reported success replicating the original results of landmark findings in only 11% and 25% of the attempted cases ( 10, 11 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Concerns about reproducibility are widespread across disciplines ( 921 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [10]
Experimental — accuracy not yet measured
13 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888
Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010271 · Cited by 805
High drug attrition rates—where are we going wrong?
Hutchinson, L., & Kirk, R. (2011) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2011.34 · Cited by 517
Drug development and clinical trials—the path to an approved cancer drug
Rubin, E. H., & Gilliland, D. G. (2012) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2012.22 · Cited by 135
Raising the bar for cancer therapy models
Francia, G., & Kerbel, R. S. (2010) · Nature Biotechnology · doi:10.1038/nbt0610-561 · Cited by 54
Cited by
Items 1,101–1,200 of 2,889
(2015) · PubMed Central · Cited by 20
Induction of pancreatic neoplasia in the KRAS / TP53 Oncopig
Mondal, P., Patel, N. S., Bailey, K., Aravind, S., Cartwright, S., Hollingsworth, M. A., Lazenby, A. J., & Carlson, M. A. (2022) · Disease Models & Mechanisms · doi:10.1242/dmm.049699 · Cited by 20
Establishing a Preclinical Multidisciplinary Board for Brain Tumors
Nimmervoll, B. V., Boulos, N., Bianski, B. M., Dapper, J. D., DeCuypere, M. G., Shelat, A. A., Terranova, S., Terhune, H. E., Gajjar, A. J., Patel, Y. T., et al. (2018) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-17-2168 · Cited by 18
“It is a hard reality that most cancer treatments that are effective in animal models fail in patients (46, 47).”
BackgroundOptimization of Cellular ELISA for Assay of Surface Antigens on Human Synoviocytes
Smith, D. D., Cohick, C. B., & Lindsley, H. B. (1997) · BioTechniques · doi:10.2144/97225rr03 · Cited by 22
Ten Simple Rules for Experiments’ Provenance
Kazic, T. (2015) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004384 · Cited by 19
Transparency, bias, and reproducibility across science: a meta-research view
Ioannidis, J. P. A. (2024) · Journal of Clinical Investigation · doi:10.1172/jci181923 · Cited by 26
“But recently a new culprit has emerged: too many basic scientific discoveries . . . are wrong” (31).”
Miedel, M. T., Varmazyad, M., Xia, M., Brooks, M. M., Gavlock, D. C., Reese, C. E., Behari, J., Soto–Gutiérrez, A., Gough, A., Taylor, D. L., et al. (2025) · Cell Reports Methods · doi:10.1016/j.crmeth.2025.101028 · Cited by 24
Khan, S. M., Gull, A., Akhtar, M., Gull, B., Najmi, A. K., Parveen, R., Ali, J., & Khan, S. I. (2025) · Expert Opinion on Drug Delivery · doi:10.1080/17425247.2025.2556979 · Cited by 23
“In 2000, Dams and colleagues investigated the bioavailability and clearance rate of repeated PEGylated liposomes and found they are cleared much faster with subsequent medication levels [122].”
Neuroprotective therapies: Preclinical reproducibility is only part of the problem
Tymianski, M. (2015) · Science Translational Medicine · doi:10.1126/scitranslmed.aac9412 · Cited by 19
Genetically Defined Strains in Drug Development and Toxicity Testing
Festing, M. F. W. (2016) · Methods in molecular biology · doi:10.1007/978-1-4939-3661-8_1 · Cited by 25
Barnett, A. G., Zardo, P., & Graves, N. (2018) · PLoS ONE · doi:10.1371/journal.pone.0195613 · Cited by 19
Machery, É. (2019) · Review of Philosophy and Psychology · doi:10.1007/s13164-019-00440-1 · Cited by 22
Haselimashhadi, H., Mason, J., Mallon, A., Smedley, D. P., Meehan, T. F., & Parkinson, H. (2020) · PLoS ONE · doi:10.1371/journal.pone.0242933 · Cited by 22
Accounting research and the significance test crisis
Johnstone, D. J. (2021) · Critical Perspectives on Accounting · doi:10.1016/j.cpa.2021.102296 · Cited by 22
Translational science: a survey of US biomedical researchers’ perspectives and practices
Walker, R. L., Saylor, K. W., Waltz, M., & Fisher, J. A. (2021) · Lab Animal · doi:10.1038/s41684-021-00890-0 · Cited by 20
Integration of Phytochemicals and Phytotherapy into Cancer Precision Medicine
Özenver, N., & Efferth, T. (2020) · Human perspectives in health sciences and technology · doi:10.1007/978-3-030-32857-3_16 · Cited by 28
An Unappreciated Challenge to Oncology Drug Discovery: Pitfalls in Preclinical Research
Begley, C. G. (2013) · American Society of Clinical Oncology Educational Book · doi:10.14694/edbook_am.2013.33.466 · Cited by 24
Self-citationThe Dilemma of Reproducibility of Gating Isotherms for Flexible MOFs
De, A., Maliuta, M., Senkovska, I., & Kaskel, S. (2022) · Langmuir · doi:10.1021/acs.langmuir.2c01999 · Cited by 22
Large inherent variability in data derived from highly standardised cell culture experiments
Reddin, I. G., Fenton, T. R., Wass, M. N., & Michaelis, M. (2023) · Pharmacological Research · doi:10.1016/j.phrs.2023.106671 · Cited by 23
“Strict experimental standardisation as suggested by many commentators to improve data quality and reliability [5,14 – 19 , 57 –”
Improving light microscopy training routines with evidence‐based education
Imreh, G., Hu, J., & Guyader, S. L. (2023) · Journal of Microscopy · doi:10.1111/jmi.13216 · Cited by 24
Best Practices and Standard Protocols as a Tool to Enhance Translation for Neuromuscular Disorders
Willmann, R., Luca, A. D., Nagaraju, K., & Rüegg, M. A. (2015) · Journal of Neuromuscular Diseases · doi:10.3233/jnd-140067 · Cited by 19
Cole, G. D., Nowbar, A. N., Mielewczik, M., Shun-Shin, M. J., & Francis, D. P. (2015) · BMJ · doi:10.1136/bmj.h4708 · Cited by 19
Cuscó, C., García, J., Nicolás, E., et al. (2016) · Polymer Chemistry · doi:10.1039/c6py01275b · Cited by 19
Modeling timelines for translational science in cancer; the impact of technological maturation
McNamee, L. M., & Ledley, F. D. (2017) · PLoS ONE · doi:10.1371/journal.pone.0174538 · Cited by 21
“Similarly, the structure of the scientific enterprise[8], the amount [41] and focus of research funding[9], connections among researchers[10, 11] and clinicians[5], the quality preclinical research[12], the involvement of industry[5, 14], the “absorptive capacity” of industry[39], and the nature of the available marke…”
BackgroundClassification and analysis of a large collection of in vivo bioassay descriptions
Zwierzyna, M., & Overington, J. P. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005641 · Cited by 17
Identification of Ligands and Translation to Clinical Applications
Haberkorn, U., Mier, W., Kopka, K., Herold‐Mende, C. C., Altmann, A., & Babich, J. W. (2017) · Journal of Nuclear Medicine · doi:10.2967/jnumed.116.186791 · Cited by 19
Time to Standardize? Time to Validate?
Meuten, D. J., Munday, J. S., & Hauck, M. (2018) · Veterinary Pathology · doi:10.1177/0300985817753869 · Cited by 19
Endogenous IQGAP1 and IQGAP3 do not functionally interact with Ras
Morgan, C. J., Hedman, A. C., Li, Z., & Sacks, D. B. (2019) · Scientific Reports · doi:10.1038/s41598-019-46677-9 · Cited by 19
Nanoparticulate Carriers Used as Vaccine Adjuvant Delivery Systems
Wang, N., Qian, R., Liu, T., Wu, T., & Wang, T. (2019) · Critical Reviews in Therapeutic Drug Carrier Systems · doi:10.1615/critrevtherdrugcarriersyst.2019027047 · Cited by 21
Bartram, I., & Jeschke, J. M. (2019) · PLoS ONE · doi:10.1371/journal.pone.0225898 · Cited by 20
“There has been a special concern for published basic cancer research after two pharmaceutical companies revealed that they could only reproduce a fraction of landmark studies [42].”
BackgroundGrüter, B. E., Croci, D. M., Schöpf, S., Nevzati, E., d’Allonzo, D., Lattmann, J., Roth, T. A., Bircher, B., Muroi, C., Dutilh, G., et al. (2020) · Translational Stroke Research · doi:10.1007/s12975-020-00801-4 · Cited by 19
The Dynamics of Retraction in Epistemic Networks
LaCroix, T., Geil, A., & O’Connor, C. (2021) · Philosophy of Science · doi:10.1086/712817 · Cited by 26
DeCorte, B. L. (2020) · Journal of Medicinal Chemistry · doi:10.1021/acs.jmedchem.0c00176 · Cited by 22
Towards FAIR protocols and workflows: the OpenPREDICT use case
Çelebi, R., Moreira, J., Hassan, A., Ayyar, S., Ridder, L., Kuhn, T., & Dumontier, M. (2020) · PeerJ Computer Science · doi:10.7717/peerj-cs.281 · Cited by 26
“, 2011), 89% 85 in cancer (Begley and Ellis, 2012), and in 66% psychology (Klein et al.”
BackgroundMethodological Problems With Online Concussion Testing
Holden, J. K., Francisco, E., Tommerdahl, A., Lensch, R., Kirsch, B., Zai, L. J., Pearce, A. J., Favorov, O. V., Dennis, R. G., & Tommerdahl, M. (2020) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2020.509091 · Cited by 23
“The replication crisis appears to span most of biomedical research, even in cancer and drug development research (Prinz et al., 2011; Begley and Ellis, 2012) in which careful replication exercises demonstrate that as much as 75 to 89% of academic research, published by the best laboratories in top…”
Background(2012) · Nature Biotechnology · doi:10.1038/nbt.2335 · Cited by 21
Seshia, S. S., Makhinson, M., Phillips, D. F., & Young, G. B. (2014) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12280 · Cited by 26
Xie, C., Pan, J., Zhang, S., Gan, J., & Liu, Z. (2014) · Neurological Sciences · doi:10.1007/s10072-014-1652-5 · Cited by 19
“The discrepancy between the results of preclinical studies and clinical trials may in part be account for the shortcomings of the clinical trials as well as due to the inadequate animal data and overoptimistic conclusions about efficacy drawn from methodologically flawed animal studies [23, 24].”
ResultAddressing the Right Targets in Oncology: Challenges and Alternative Approaches
Stock, J. K., Jones, N. P., Hammonds, T. R., Roffey, J., & Dillon, C. (2015) · SLAS DISCOVERY · doi:10.1177/1087057114564349 · Cited by 21
“Much has been written about the multitude of factors that can contribute to poor data reproducibility,(63,64,67) and many potential solutions have been proposed.(64,68,69) These include increasing opportunities to publish and present target devalidation data and linking such studies to the original”
BackgroundHussain, N. (2016) · International Journal of Pharmaceutics · doi:10.1016/j.ijpharm.2016.07.053 · Cited by 20
Scanning the Horizon: Towards transparent and reproducible neuroimaging researchPreprint
Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafo, M. R., Nichols, T. E., Poline, J., Vul, E., & Yarkoni, T. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/059188 · Cited by 74
Is biomedical research self-correcting? Modelling insights on the persistence of spurious science
Grimes, D. R. (2024) · Royal Society Open Science · doi:10.1098/rsos.231056 · Cited by 18
“In fields as vital as cancer research, only 11% – 50% of preclinical results are deemed replicable [18,19].”
BackgroundA battery-free object localization and motion sensing platform
Zhao, Y., LaMarca, A., & Smith, J. R. (2014) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2632048.2632078 · Cited by 18
“A 2012 study, for example, showed that 47 of 53 key cancer biology results could not be replicated, in some cases even with the help of the original researchers [1].”
Experiences in integrated data and research object publishing using GigaDB
Edmunds, S., Li, P., Hunter, C., Zhe, X. S., Davidson, R., Nogoy, N., & Goodman, L. (2016) · International Journal on Digital Libraries · doi:10.1007/s00799-016-0174-6 · Cited by 20
Wirkus, J., Ead, A., & Mackenzie, G. G. (2020) · Nutrition Research · doi:10.1016/j.nutres.2020.12.018 · Cited by 19
“339 Randomization is crucial to ensure that bias is minimized (66).”
BackgroundFragility index of trials supporting approval of anti-cancer drugs in common solid tumours
Desnoyers, A., Wilson, B. E., Nadler, M. B., & Amir, E. (2021) · Cancer Treatment Reviews · doi:10.1016/j.ctrv.2021.102167 · Cited by 19
Weggemans, M., ter Haar, N. M., Prakken, B. J., & ten Cate, O. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004130 · Cited by 19
Samota, E. K., & Davey, R. (2021) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2021.678554 · Cited by 23
“A 2012 study showed that 88% of drug-discovery experiments could not be reproduced even by the original authors, in some cases forcing retraction of the original work (Begley and Ellis, 2012).”
BackgroundBeyerbach, J., Stadelmaier, J., HOFFMANN, G. E., Balduzzi, S., Bröckelmann, N., & Schwingshackl, L. (2021) · Advances in Nutrition · doi:10.1093/advances/nmab095 · Cited by 19
Pyrazine Moiety: Recent Developments in Cancer Treatment
Sahu, R., Shah, K., Gautam, Y., & Sahu, K. (2023) · Current Organic Chemistry · doi:10.2174/1385272827666230816105317 · Cited by 23
Silverstein, P., Pennington, C. R., Branney, P., O'Connor, D. B., Lawlor, E., O'Brien, E., & Lynott, D. (2024) · British Journal of Psychology · doi:10.1111/bjop.12700 · Cited by 27
“…across a range of disciplines, from economics (Camerer et al., 2016) and experimental philosophy (Cova et al., 2021) to cardiovascular health (Prinz et al., 2011) and cancer biology (Begley & Ellis, 2012; Errington et al., 2021), reporting wide variability in ‘successful’ replications (30%–90%).”
BackgroundKnowledge Integration in Cancer: Current Landscape and Future Prospects
Ioannidis, J. P. A., Schully, S. D., Lam, T. K., & Khoury, M. J. (2012) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-12-1144 · Cited by 20
“Such efforts, ranging from "forensic bioinformatics" to "reproducibility checks," have shown major reproducibility problems in several research fields such as -omics signatures (51) or preclinical data on drug targets (52).”
Reproducibility and cell biology
Yamada, K. M., & Hall, A. (2015) · The Journal of Cell Biology · doi:10.1083/jcb.201503036 · Cited by 19
“These companies encountered surprisingly low reproducibility (<25%) of published work (Prinz et al., 2011; Begley and Ellis, 2012).”
BackgroundQuestioning the preclinical paradigm: natural, extreme biology as an alternative discovery platform
Buffenstein, R., Nelson, O. L., & Corbit, K. C. (2014) · Aging · doi:10.18632/aging.100704 · Cited by 19
Mattina, J., MacKinnon, N., Henderson, V. C., Fergusson, D. A., & Kimmelman, J. (2016) · Cancer Research · doi:10.1158/0008-5472.can-15-3455 · Cited by 19
Methodological Challenges and Advances in Managerial and Organizational Cognition
Galavan, R., Sund, K. J., & Hodgkinson, G. P. (2017) · Emerald Group Publishing Limited eBooks · doi:10.1108/s2397-52102017 · Cited by 21
Assessing Public Metabolomics Metadata, Towards Improving Quality
Ferreira, J. D., Inácio, B., Salek, R. M., & Couto, F. M. (2017) · Berichte aus der medizinischen Informatik und Bioinformatik/Journal of integrative bioinformatics · doi:10.1515/jib-2017-0054 · Cited by 19
Weeks, K. L., Henstridge, D. C., Salim, A., Shaw, J. E., Marwick, T. H., & McMullen, J. R. (2019) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.00327.2019 · Cited by 18
“Over the past decade, there have been increasing concerns around the reproducibility of preclinical animal research and the lack of detail provided in publications in relation to experimental design (7, 8, 28, 47).”
BackgroundVarieties of Error and Varieties of Evidence in Scientific Inference
Osimani, B., & Landes, J. (2021) · The British Journal for the Philosophy of Science · doi:10.1086/714803 · Cited by 18
“[2011]; Begley and Ellis [2012]; Open Science Collaboration [2015]; Osimani [unpublished]). The reproducibility crisis relates to repeated failures of replication studies to reproduce the results delivered by original ones (Ioannidis [2005]). Most students agree that this phenomenon is originated by the reliance on lo…”
BackgroundInfluentialHow Will Open Science Impact on University-Industry Collaboration?
Chataway, J., Parks, S., & Smith, E. (2017) · Foresight-Russia · doi:10.17323/2500-2597.2017.2.44.53 · Cited by 31
Meta-research: Evaluation and Improvement of Research Methods and Practices
Ioannidis, J. P. A. (2020) · Böhlau Verlag eBooks · doi:10.7767/9783205233008.101 · Cited by 20
Héroux, M. E., Butler, A. A., Cashin, A. G., McCaughey, E. J., Affleck, A. J., Green, M. A., Cartwright, A., Jones, M. D., Kiely, K. M., van Schooten, K. S., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-060976 · Cited by 21
Evolution of a Materials Data Infrastructure
Warren, J. A., & Ward, C. H. (2018) · JOM · doi:10.1007/s11837-018-2968-z · Cited by 17
Editorial: Do You See What I See?: Quality, Reliability, and Reproducibility in Biomedical Research
Kraus, W. L. (2014) · Molecular Endocrinology · doi:10.1210/me.2014-1036 · Cited by 18
“rate of reproducibility of landmark discoveries in biomedicine (4, 5), as well as problems with data analysis and peer review, but they fail to provide and substantiate an accurate view of the scope of the problem and its overall impact on the scientific enterprise.”
BackgroundFive Easy Ways To Make Your Research More Reproducible
Sholl, D. S. (2019) · Langmuir · doi:10.1021/acs.langmuir.9b02963 · Cited by 19
The Importance and Clinical Relevance of Surfaces in Tissue Culture
Hickman, G. J., Boocock, D. J., Pockley, A. G., & Perry, C. C. (2016) · ACS Biomaterials Science & Engineering · doi:10.1021/acsbiomaterials.5b00403 · Cited by 23
Tumkaya, T., Ott, S., & Claridge‐Chang, A. (2018) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2018.07.016 · Cited by 20
“The effects of lesions in these genes on STM are relatively modest, and have not been independently replicated: Adf1 = –21%; PQBP1 = –20%; and mbm = –33% (Figures S5–7) (DeZazzo et al. 2000; Tamura et al. 2010; de Belle and Heisenberg 1996).”
Detection and Analysis of Lung Cancer Using Radiomic Approach
Shallu, Nanglia, P., Kumar, S., & Luhach, A. K. (2019) · Springer eBooks · doi:10.1007/978-981-13-6295-8_2 · Cited by 18
Detecting multiple replicating signals using adaptive filtering procedures
Wang, J., Gui, L., Su, W., Sabatti, C., & Owen, A. B. (2022) · The Annals of Statistics · doi:10.1214/21-aos2139 · Cited by 29
Clinical relevance assessment of animal preclinical research (RAA) tool: development and explanation
Gurusamy, K. S., Moher, D., Loizidou, M., Ahmed, I., Avey, M. T., Barron, C. C., Davidson, B. R., Dwek, M. V., Gluud, C. N., Jell, G., et al. (2021) · PeerJ · doi:10.7717/peerj.10673 · Cited by 19
Bekhite, M. M., & Schulze, P. C. (2021) · Cells · doi:10.3390/cells10123483 · Cited by 25
Labuz, J. M., & Takayama, S. (2014) · Lab on a Chip · doi:10.1039/c4lc00125g · Cited by 17
Modelling Scientific Communities
O’Connor, C. (2023) · Cambridge University Press eBooks · doi:10.1017/9781009359535 · Cited by 28
The progressive trend of modeling and drug screening systems of breast cancer bone metastasis
Azar, H. K., Gharibshahian, M., Rostami, M., Mansouri, V., Sabouri, L., Beheshtizadeh, N., & Rezaei, N. (2024) · Journal of Biological Engineering · doi:10.1186/s13036-024-00408-5 · Cited by 23
“…evolved and have been able to mimic several features of tumor tissue, such as morphology, the gradient of chemical and biological factors, the expression of pro-angiogenic proteins, MDR, and the interactions of cell–cell and cell-ECM, via different 3D tissue engineering methods [73, 74, 84, 85].”
BackgroundExperts fail to reliably detect AI-generated histological data
Hartung, J., Reuter, S., Kulow, V. A., Fähling, M., Spreckelsen, C., & Mrowka, R. (2024) · Scientific Reports · doi:10.1038/s41598-024-73913-8 · Cited by 21
“…data is checked before, rather than at some distant time point after publication, the enormous follow up costs of fraudulent science which arise for fellow scientist, funding organizations, scientific journals, companies and patients alike can be reduced much more effectively 1,5,7–11 ,14 .”
Poor antibody validation is a challenge in biomedical research: a case study for detection of c-FLIP
Bucur, O., Pennarun, B., Stancu, A. L., Nadler, M. J. S., Muraru, M. S., Bertomeu, T., & Khosravi‐Far, R. (2013) · APOPTOSIS · doi:10.1007/s10495-013-0880-0 · Cited by 17
“The poor success rates have been attributed to factors such as low reproducibility of the data, insufficient number of repeats and inadequate mouse models [1, 2].”
BackgroundDrug discovery in prostate cancer mouse models
Valkenburg, K. C., & Pienta, K. J. (2015) · Expert Opinion on Drug Discovery · doi:10.1517/17460441.2015.1052790 · Cited by 24
“Several recommendations have been published to improve the quality of preclinical research publications, including allowing for publication of negative data, increasing communication between physicians and scientists and patients, and increasing access to cancer research tools (such as mouse models) among research sci…”
BackgroundWu, M., Yu, Q., Li, X., Zheng, J., Huang, J., & Kwoh, C. K. (2013) · PLoS ONE · doi:10.1371/journal.pone.0053197 · Cited by 17
“It is a growing recognition that network-based approaches are suitable to describe the complexity of human diseases and assist the development of new drugs [12].”
BackgroundA Cybernetics Update for Competitive Deep Learning System
Fiorini, R. A. (2015) · International Electronic Conference on Entropy and Its Applications (ECEA) · doi:10.3390/ecea-2-e003 · Cited by 17
“System Functional Closures for k ≡ [1, 1] with parameter m going from 0 to ∞.”
BackgroundMethodInfluentialEnhancing Scientific Foundations to Ensure Reproducibility
Hsieh, T., Vaickus, M. H., & Remick, D. G. (2017) · American Journal Of Pathology · doi:10.1016/j.ajpath.2017.08.028 · Cited by 22
“Of the nonreproducible articles, the data were routinely analyzed without blinding, among other omissions in careful study design.(7) Although all preclinical experiments may not need these steps, rigorous experimental design can aid in preventing bias.”
BackgroundAbove a swamp: A theory of high-quality scientific production
Kiri, B., Lacetera, N., & Zirulia, L. (2018) · Research Policy · doi:10.1016/j.respol.2018.01.011 · Cited by 20
“Begley and Ellis (2012) reported that they could replicate only 6 out of 53 fundamental studies in oncology and haematology.”
Scientific workflow managers in metabolomics: an overview
Verhoeven, A., Giera, M. A., & Mayboroda, O. A. (2020) · The Analyst · doi:10.1039/d0an00272k · Cited by 17
Calar, K., Plesselova, S., Bhattacharya, S., Jorgensen, M. M., & de la Puente, P. (2020) · Cancers · doi:10.3390/cancers12071722 · Cited by 18
Opinion: Independent third-party entities as a model for validation of commercial antibodies
Laflamme, C., Edwards, A. M., Bandrowski, A. E., & McPherson, P. S. (2021) · New Biotechnology · doi:10.1016/j.nbt.2021.07.001 · Cited by 34
Use of semantic workflows to enhance transparency and reproducibility in clinical omics
ZHENG, C. L., Ratnakar, V., Gil, Y., & McWeeney, S. K. (2015) · Genome Medicine · doi:10.1186/s13073-015-0202-y · Cited by 19
“This question is pertinent to clinical omics and the scientific community as a whole [20–22].”
BackgroundZuurbier, C. J., Heinen, A., Koeman, A., Stuifbergen, R., Hakvoort, T. B. M., Weber, N. C., & Hollmann, M. W. (2014) · Journal of Translational Medicine · doi:10.1186/s12967-014-0325-8 · Cited by 21
Are scientists a workforce? – Or, how Dr. Frankenstein made biomedical research sick
Lazebnik, Y. A. (2015) · EMBO Reports · doi:10.15252/embr.201541266 · Cited by 23
“Its symptoms include poor reliability (in one report [2], only six out of 53 landmark cancer research studies could be verified, with the reliability of less prominent studies also questioned [3]), insufficient funding [1], an outdated funding system [4], the scarcity of opportunities for growth in science, depression…”
BackgroundIndependent labs to verify high-profile papers
Baker, M. (2012) · Nature · doi:10.1038/nature.2012.11176 · Cited by 17
Tuyttens, F., Stadig, L. M., Heerkens, J., laer, E. V., Buijs, S., & Ampe, B. (2016) · Applied Animal Behaviour Science · doi:10.1016/j.applanim.2016.04.019 · Cited by 20
“Attempts to reproduce research findings – even if published in toptier journals – often fail (Begley and Ellis, 2012).”
BackgroundMeasuring the effects of publication bias in political science
Esarey, J., & Wu, A. (2016) · Research & Politics · doi:10.1177/2053168016665856 · Cited by 21
“…Maniadis, Tufano and List, 2014; Ioannidis et al., 2014; Open Science Collaboration, 2015; Boekel et al., 2015; Camerer et al., 2016); the “replication crisis” has plagued fields in the hard sciences as well (e.g., Prinz, Schlange and Khusru, 2011; Begley and Ellis, 2012; Steward et al., 2012).”
BackgroundScientific Misconduct: The Elephant in the Lab. A Response to Parker et al.
Clark, T. D., Binning, S. A., Raby, G. D., Speers‐Roesch, B., Sundin, J., Jutfelt, F., & Roche, D. G. (2016) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2016.09.006 · Cited by 17
Editorial: Preprints, citations and Nucleic Acids Research
Stoddard, B. L., & Fox, K. R. (2018) · Nucleic Acids Research · doi:10.1093/nar/gky1229 · Cited by 17
“More than ever, the publishing community is expected to be extremely vigilant with regard to weaknesses in submitted manuscripts (3,4).”
BackgroundSánchez‐de‐Diego, C., Yada, R. C., Sethakorn, N., Geiger, P. G., Ding, A. B., Héninger, E., Ahmed, F., Virumbrales-Muñoz, M., Lupsa, N., Bartels, E., et al. (2025) · Communications Biology · doi:10.1038/s42003-025-08384-2 · Cited by 19
“Furthermore, while in vivo animal models of cancer can provide insights into tumor growth and response to treatments, they can be cost-prohibitive anddonotrecapitulatehumanphysiology 30 .”
Reproducible Research in Vadose Zone Sciences
Skaggs, T. H., Young, M. H., & Vrugt, J. A. (2015) · Vadose Zone Journal · doi:10.2136/vzj2015.06.0088 · Cited by 17
“Reproducible research (RR) has in recent years become a prominent issue in several academic fields, including biomedical research, where the irreproducibility of a great number of pharmacology and cancer research studies has been well publicized (Prinz et al., 2011; Begley and Ellis, 2012), and in fields featuring len…”
BackgroundIs There a Replication Crisis in Medical Education Research?
Hope, D., Dewar, A., & Hay, C. (2021) · Academic Medicine · doi:10.1097/acm.0000000000004063 · Cited by 18
3D Collagen Vascular Tumor-on-a-Chip Mimetics for Dynamic Combinatorial Drug Screening
Wan, L., Yin, J., Skoko, J., Schwartz, R. S., Zhang, M., LeDuc, P. R., & Neumann, C. A. (2021) · Molecular Cancer Therapeutics · doi:10.1158/1535-7163.mct-20-0880 · Cited by 18
“Importantly, animalmodels often lack immune system functionality (19) and species-specific drug response (19, 20) thus, presenting inconsistent responses compared to clinical trials in human patients (21).”
BackgroundThe new normal? Redaction bias in biomedical science
Grimes, D. R., & Heathers, J. A. J. (2021) · Royal Society Open Science · doi:10.1098/rsos.211308 · Cited by 17
“For landmark experiments in cancer research, the replication rate was an abysmal 11% [6].”
BackgroundRitskes-Hoitinga, M. J., & Pound, P. (2022) · Journal of the Royal Society of Medicine · doi:10.1177/01410768221100970 · Cited by 17
Lu, Y., Zhang, W., Tian, Z., Liang, Q. Y., Liu, C., Wu, Y., Zhang, L., & Rong, L. (2022) · Stem Cell Research & Therapy · doi:10.1186/s13287-022-03103-8 · Cited by 19
“Therefore, it is necessary to formulate relevant policies to encourage and require journals to publish negative or neutral results in order to avoid the “drawer document” effect and reduce the impact of publication bias on its results [45].”
“Is, or is not, the two great ends of Fate”: Errors in Agronomic Research
Garland-Campbell, K. A., Thompson, Y. A., Guy, S. O., McIntosh, M. S., & Glaz, B. (2015) · Agronomy Journal · doi:10.2134/agronj14.0167 · Cited by 18
“These results are typical of environmental and biological science reporting (Anderson et al., 2000; Fidler et al., 2006; Begley and Ellis, 2012; Bosker et al., 2013).”
BackgroundResult
Metadata
- Authors
- Begley, C. G., & Ellis, L. M.
- Affiliations
- Amgen (United States) (US)
- The University of Texas MD Anderson Cancer Center (US)
- Corresponding author
- Ellis, Lee M.
- Venue
- Nature
- Published
- 2012-03-28
- Volume
- 483
- Issue
- 7391
- Pages
- 531-533
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W1987777080
- Semantic Scholar
- 1a81d3d1e8a983e0551edf485f4ac1a8cdd7fea4
- Cited by
- 2,979
- References
- 5
- Citations by year
- 2026: 106 · 2025: 116 · 2024: 127 · 2023: 169 · 2022: 179 · 2021: 206 · 2020: 236 · 2019: 255 · 2018: 262 · 2017: 271 · 2016: 283 · 2015: 262 · 2014: 226 · 2013: 133 · 2012: 62
- FWCI
- 210.63
- Type
- journal article
- Language
- en
- Update notices
- correction (2012-05-02)
- Open access
- bronze
- doi.org
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health and Medical Research Impacts
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Health Sciences › Nursing › Issues, ethics and legal aspects › Biomedical Ethics and Regulation
- Keywords
- preclinical cancer research, research standards, publication practices, research incentives, translational research, clinical trials
- MeSH terms
- Animals, Clinical Trials as Topic, Drug Evaluation, Preclinical, Humans, Neoplasms, Research Design, Reproducibility of Results, Survival Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.