Raise standards for preclinical cancer research
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index; 3 of the 9 sources its claims cite were checked for support.
Overview
CoChat Summary
TL;DR
This Nature Comment argues that many landmark preclinical oncology findings are not reproducible and calls for higher standards in methods, reporting, and incentives. It reports that Amgen confirmed only 6 of 53 landmark papers (11%) and cites Bayer’s finding that about 25% of published preclinical studies could be validated.
Key findings
- In Amgen’s ten-year retrospective analysis, scientific findings were confirmed in only 6 of 53 “landmark” preclinical cancer papers (11%); non-reproduced studies often lacked blinding, relied on single experiments such as one Western blot, or presented selected data.
- Bayer HealthCare reported that only about 25% of published preclinical studies could be validated to the point at which projects could continue; published cancer research represented 70% of the studies analysed in that report, with possible overlap with Amgen’s 53 papers.
- Citation data showed non-reproduced articles in journals with impact factor >20 had a mean of 248 citations (range 3–800) versus 231 (range 82–519) for reproduced articles; for impact factor 5–19, non-reproduced articles had a mean of 169 citations (range 6–1,909) versus 13 (range 3–24) for reproduced articles.
- The authors recommend reporting negative data, blinding preclinical experiments, validating reagents, repeating critical experiments, avoiding single cell lines or models, requiring patient-selection biomarkers, and changing academic incentives.
Methods
- This is a Nature Comment/perspective, not a primary experimental study; it synthesizes Amgen’s ten-year retrospective analysis of experiments performed prospectively and cites a Bayer HealthCare report.
- Amgen selected 53 “landmark” papers describing completely new approaches or alternative clinical uses, attempted confirmation, used additional models, contacted original authors, exchanged reagents, and repeated experiments under the authors’ direction, occasionally in the original investigator’s laboratory.
- The analysis reports counts of confirmed findings and citation metrics from Google Scholar, May 2011; the recommendations are proposed changes rather than tested interventions.
Limitations
- Stated: validation attempts may have failed because of technical differences or difficulties despite efforts to ensure otherwise; additional models were used; when findings could not be reproduced, authors were contacted and experiments were repeated under their direction.
- Stated: papers were deliberately selected as “landmark” and described something completely new, so some data might not hold up; Bayer’s analysed studies may overlap with Amgen’s 53 papers.
- Evident: as a Comment, the text provides no detailed systematic protocol, inclusion criteria, statistical methods, or full dataset; the citation analysis is descriptive and limited to two impact-factor strata.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 1 Not valid · 6 Not checked · 12 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Sadly, clinical trials in oncology have the highest failure rate compared with other therapeutic areas.”
What the sources say
- Partly supportedcites [1] Hutchinson, L. & Kirk, R. Nature Rev. Clin. Oncol. 8, 189–190 (2011). sourceExperimental — accuracy not yet measured
The opening passage of our copy (labelled Abstract; page unavailable) says cancer-drug attrition is much higher than in other therapeutic areas, citing 5% of anticancer agents versus 20% of cardiovascular agents reaching licensure. That comparison covers development from preclinical activity through phase III, not failure rates specifically among clinical trials; it also does not explicitly establish a highest-rate ranking. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Although we in the cancer field hoped that this would lead to more effective drugs, historically, our ability to translate cancer research to clinical success has been remarkably low 1 .”
What the sources say
- Supportedcites [1] Hutchinson, L. & Kirk, R. Nature Rev. Clin. Oncol. 8, 189–190 (2011). source
The opening passage (labeled Abstract in our copy; no page given) reports that only 5% of agents with preclinical anticancer activity are licensed after demonstrating sufficient efficacy in phase III testing. It also says cancer drug attrition is much higher than in other therapeutic areas, supporting the claim of historically low translation to clinical success. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In studies for which findings could be reproduced, authors had paid close attention to controls, reagents, investigator bias and describing the complete data set.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For results that could not be reproduced, however, data were not routinely analysed by investigators blinded to the experimental versus control groups.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Investigators frequently presented the results of one experiment, such as a single Western-blot analysis.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
12 more claims
Create a free account to see all 22 claims and every source verdict.
- It may state two separate claims as one. 2
- Probably not a substantial claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“there are other more common sources of unreliability in the literature (41)(42)(43)(44)”
From Misconduct accounts for the majority of retracted scientific publications (2012) · cites this paper as [42]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“many established scientific results in the social (49, 50) and biomedical (51-53) sciences cannot readily be replicated.”
From Misinformation in and about science (2021) · cites this paper as [51]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A landmark study conducted by Glenn Begley and his team in 2012 concluded that most cancer research studies were not reproducible, by alarmingly reporting that 47 of 53 published findings could not be replicated [13].”
From Guidelines for the use of flow cytometry and cell sorting in immunological studies (third edition) (2021) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In cell biology, two industrial laboratories reported success replicating the original results of landmark findings in only 11% and 25% of the attempted cases ( 10, 11 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Concerns about reproducibility are widespread across disciplines ( 921 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [10]
Experimental — accuracy not yet measured
13 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888
Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010271 · Cited by 805
High drug attrition rates—where are we going wrong?
Hutchinson, L., & Kirk, R. (2011) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2011.34 · Cited by 517
Drug development and clinical trials—the path to an approved cancer drug
Rubin, E. H., & Gilliland, D. G. (2012) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2012.22 · Cited by 135
Raising the bar for cancer therapy models
Francia, G., & Kerbel, R. S. (2010) · Nature Biotechnology · doi:10.1038/nbt0610-561 · Cited by 54
Cited by
Items 1,301–1,400 of 2,889
Võikar, V. (2020) · Journal for Reproducibility in Neuroscience · doi:10.31885/jrn.1.2020.1375 · Cited by 16
“Over the last ten years, the “reproducibility crisis” has often appeared in the headlines of scientific journals (3-6).”
BackgroundPerkel, J. M. (2014) · BioTechniques · doi:10.2144/000114143 · Cited by 14
Preserving Statistical Validity in Adaptive Data AnalysisPreprint
Dwork, C., Feldman, V., Hardt, M., Pitassi, T., Reingold, O., & Roth, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.2664 · Cited by 49
“As an instance of this problem, we propose and investigate the question of estimating the expectations of m adaptively chosen functions on an unknown distribution given n random samples.”
MethodMichel, M. C., Murphy, T. J., & Motulsky, H. J. (2019) · Drug Metabolism and Disposition · doi:10.1124/dmd.119.090027 · Cited by 14
“…published results are often not reproducible and that published statistical analyses are often not performed or interpreted properly (e.g., Prinz et al., 2011; Begley and Ellis, 2012; Collins and Tabak, 2014; Freedman and Gibson, 2015; National Academies of Sciences Engineering and Medicine, 2019).”
BackgroundEstimating the replicability of Brazilian biomedical science
Initiative, T. B. R., Amaral, O. B., Carneiro, C. F. D., Neves, K., Sampaio, A. P. W., Gomes, B. V., Abreu, M. B. D., Tan, P. B., Mota, G. P. S., Goulart, R. N., et al. (2025) · bioRxiv · doi:10.1101/2025.04.02.645026 · Cited by 13
Humanized mouse as a tool to predict immunotoxicity of human biologics
Yong, K. S. M., Her, Z., Tan, S. Y., Tan, W. W. S., Liu, M., Lai, F., Heng, S. M., Fan, Y., Chang, K. T. E., Wang, C.-I., et al. (2020) · Frontiers in Immunology · doi:10.3389/fimmu.2020.553362 · Cited by 13
“Shortcomings in these assessments, suggest that the selection of a cost-effective, validated, predictive animal model, containing human-specific cells will add substantial value in identifying in vivo humanspecific clinical toxicity, particularly for biologics with novel mechanisms of action, therefore allowing accura…”
BackgroundDeciphering permissivity of human tumor ecosystems to oncolytic viruses
Schoeps, B., Lauer, U. M., & Elbers, K. (2025) · Oncogene · doi:10.1038/s41388-025-03357-5 · Cited by 16
“…more than 10 years ago, it was highlighted that translation of pre-clinical fi ndings into the clinics in the cancer fi eld is very inef fi cient [16]: only 5% of drugs that show anticancer activity in pre-clinical development fi nally obtain approval, a dismal success rate as compared to 20% for…”
Gunn, W. (2014) · Nature · doi:10.1038/505483b · Cited by 12
Meningioma animal models: a systematic review and meta-analysis
Andersen, M. S., Kofoed, M. S., Paludan‐Müller, A. S., Pedersen, C. B., Mathiesen, T. I., Mawrin, C., Wirenfeldt, M., Kristensen, B. W., Olsen, B. B., Halle, B., et al. (2023) · Journal of Translational Medicine · doi:10.1186/s12967-023-04620-7 · Cited by 13
Introduction: Replicating John Hedley Brooke’s Work on the History of Science and Religion
Peels, R., van den Brink, G., Eyghen, H. V., & Pear, R. S. A. (2024) · Zygon® · doi:10.16995/zygon.11255 · Cited by 15
Inflammatory Modulation Effects of Probiotics: A Safe and Promising Modulator for Cancer Prevention
Mahooti, M., Abdolalipour, E., Sanami, S., & Zare, D. (2024) · Current Microbiology · doi:10.1007/s00284-024-03901-8 · Cited by 15
“However, herbal treatments have their drawbacks, ranging from a lack of supported comprehensive pre-clinical and clinical trials to some other limitations like recurrence and many side effects just like the two most highly used available cancer treatments (i.e., radiotherapy and chemotherapy) [23].”
Schatz, G. (2014) · Nature Reviews Molecular Cell Biology · doi:10.1038/nrm3807 · Cited by 13
Panchision, D. M. (2013) · Stem Cells Translational Medicine · doi:10.5966/sctm.2012-0149 · Cited by 13
Quality really matters: the need to improve specimen quality in biomedical research
Simeon‐Dubach, D., Burt, A. D., & Hall, P. A. (2012) · Histopathology · doi:10.1111/his.12060 · Cited by 13
“Therefore findings of low reproducibility of published scientific data as reported recently by two different groups are perhaps not surprising.(6,7) It will take some time and significant effort to sort this sorry situation out.”
BackgroundTranslating Stem Cell Research from the Bench to the Clinic: A Need for Better Quality Data
Plant, A. L., & Parker, G. C. (2013) · Stem Cells and Development · doi:10.1089/scd.2013.0188 · Cited by 12
“Regulators and investors alike appear to be voicing the same concerns: they see (1) insufficient high-quality data to provide confidence regarding the claims of medical benefit, (2) an insufficient understanding of the mechanism of action, and (3) a lack of identification of essential characteristics for product relea…”
BackgroundSimmonds, A., Mitrou, P. N., & Wiseman, M. J. (2014) · Nutrition Bulletin · doi:10.1111/nbu.12077 · Cited by 13
The Cancer Target Discovery and Development Network (2016) · Molecular Cancer Research · doi:10.1158/1541-7786.mcr-16-0090 · Cited by 14
Suitability of North Dakota for Conducting Effective Hygroscopic Seeding
Delene, D. J. (2016) · The Journal of Weather Modification · doi:10.54782/jwm.v48i1.549 · Cited by 13
“As scientists learn that some important results cannot be reproduced, research standards are increasing (Begley and Ellis 2012), which means projects require more care and papers greater detail.”
BackgroundThe path to VICTORy – a beginner's guide to success using commercial research antibodies
Goodman, S. L. (2018) · Journal of Cell Science · doi:10.1242/jcs.216416 · Cited by 14
“*Author for correspondence (Simgoo@gmx.net) S.L.G., 0000-0002-3480-1346 Jo u rn al o f Ce ll Sc ie n ce has been dubbed ‘the reproducibility crisis’ (Baker, 2015b; Begley and Ellis, 2012).”
BackgroundInfluentialToxicological assessment of Tobacco Heating System 2.2: Findings from an independent peer review
Boué, S., Schlage, W. K., Page, D. I., Hoeng, J., & Peitsch, M. C. (2019) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2019.03.007 · Cited by 14
THE FAIR REWARD PROBLEM: THE ILLUSION OF SUCCESS AND HOW TO SOLVE IT
Sornette, D., Wheatley, S., & Cauwels, P. P. (2019) · Advances in Complex Systems · doi:10.1142/s021952591950005x · Cited by 14
“They reproduced a mere 11% (Economist 2013, Begley and Ellis 2012, Baker 2016).”
Translating Laboratory Tests into Clinical Practice: A Conceptual Framework
Nagler, M. (2020) · Hämostaseologie · doi:10.1055/a-1227-8008 · Cited by 15
Eliciting group judgements about replicability: a technical implementation of the IDEA Protocol
Pearson, R., Fraser, H. S., Bush, M., Mody, F., Widjaja, I., Head, A., Wilkinson, D. P., Sinnott, R., Wintle, B. C., Burgman, M. A., et al. (2021) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2021.055 · Cited by 8
A primer on applying AI synergistically with domain expertise to oncology
Kim, J., Kusko, R. L., Zeskind, B. J., Zhang, J., & Escalante-Chong, R. (2021) · Biochimica et Biophysica Acta (BBA) - Reviews on Cancer · doi:10.1016/j.bbcan.2021.188548 · Cited by 13
Improving the reproducibility of findings by updating research methodology
Klein, J. (2021) · Quality & Quantity · doi:10.1007/s11135-021-01196-6 · Cited by 22
Cheleuitte-Nieves, C., & Lipman, N. S. (2019) · ILAR Journal · doi:10.1093/ilar/ilaa009 · Cited by 14
Sardanelli, F., & Colarieti, A. (2022) · La radiologia medica · doi:10.1007/s11547-022-01582-6 · Cited by 13
“According to Begley and Ellis [5], the reproducibility success rate for 53 studies considered fundamental in oncology was even lower (11%).”
BackgroundPredicting and reasoning about replicability using structured groups
Wintle, B. C., Smith, E. T., Bush, M., Mody, F., Wilkinson, D. P., Hanea, A. M., Marcoci, A., Fraser, H. S., Hemming, V., Thorn, F. S., et al. (2023) · Royal Society Open Science · doi:10.1098/rsos.221553 · Cited by 33
Lubega, N., Anderson, A., & Nelson, N. C. (2023) · PLoS ONE · doi:10.1371/journal.pone.0293584 · Cited by 16
Reporting of Results from Animal Studies
Tilson, H. A., & Schroeder, J. C. (2013) · Environmental Health Perspectives · doi:10.1289/ehp.1307676 · Cited by 13
“Many journals have endorsed the CONSORT guidelines, and there is some evidence that use of these guidelines has improved the quality of papers on clinical trials (Kane et al. 2007).”
BackgroundOpen for collaboration: an academic platform for drug discovery and development at SciLifeLab
Arvidsson, P. I., Sandberg, K., & Forsberg‐Nilsson, K. (2016) · Drug Discovery Today · doi:10.1016/j.drudis.2016.06.026 · Cited by 12
Theory and practice of clinical pharmacodynamics in oncology drug development
Parchment, R. E., & Doroshow, J. H. (2016) · Seminars in Oncology · doi:10.1053/j.seminoncol.2016.07.001 · Cited by 13
Study Design Rigor in Animal-Experimental Research Published in Anesthesia Journals
Hoerauf, J. M., Moss, A. F. D., Fernandez-Bustamante, A., & Bartels, K. (2017) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000001907 · Cited by 12
The role of the IACUC in ensuring research reproducibility
Silverman, J., Macy, J. D., & Preisig, P. A. (2017) · Lab Animal · doi:10.1038/laban.1213 · Cited by 17
Collaboration and competition: ethics in toxicology
Walker, E. S., & Roberts, R. A. (2018) · Toxicology Research · doi:10.1039/c7tx00306d · Cited by 13
Chung, S., & Fink, E. L. (2018) · Annals of the International Communication Association · doi:10.1080/23808985.2018.1425100 · Cited by 13
“Even though concerns about reproducibility have been raised in many disciplines (Begley & Ellis, 2012; Ioannidis, 2005; McNutt, 2014; Pashler & Wagenmakers, 2012; Prinz, Schlange, & Asadullah, 2011), replication studies remain rare in the behavioral sciences (Evanschitzky, Baumgarth, Hubbard, &…”
BackgroundMiękus, N., Olędzka, I., Harshkova, D., Liakh, I., Plenis, A., Kowalski, P., & Bączek, T. (2018) · International Journal of Molecular Sciences · doi:10.3390/ijms19061560 · Cited by 13
“Preclinical discoveries are often not reproducible, in part because of the inadequate cell lines and animal models applied for such studies [1].”
BackgroundGrant, M. K. O., Shapiro, S. L., Ashe, K., Liu, P., & Zahs, K. (2019) · Biological Procedures Online · doi:10.1186/s12575-019-0095-z · Cited by 13
Klau, S., Martin‐Magniette, M., Boulesteix, A., & Hoffmann, S. (2019) · Biometrical Journal · doi:10.1002/bimj.201800309 · Cited by 17
“…community and the general public have recently been rocked with the recognition that many research findings do not replicate on independent data sets (Begley & Ellis, 2012; Ioannidis et al., 2009; Open Science Collaboration, 2015), acknowledging and accounting for method uncertainty thus seem like…”
BackgroundGadaire, D. M., & Kilmer, R. P. (2020) · Journal of Evidence-Based Social Work · doi:10.1080/26408066.2020.1724226 · Cited by 14
Psychology Exceptionalism and the Multiple Discovery of the Replication Crisis
Nelson, N. C., Chung, J., Ichikawa, K., & Malik, M. M. (2021) · Review of General Psychology · doi:10.1177/10892680211046508 · Cited by 27
“These discussions are associated with discussions of the aforementioned papers by scientists at Amgen and Bayer describing their difficulties in replicating results from the preclinical cancer research literature (Begley & Ellis, 2012; Prinz et al., 2011).”
BackgroundLinshiz, G., Goldberg, A., Konry, T., & Hillson, N. J. (2012) · Perspectives in biology and medicine · doi:10.1353/pbm.2012.0044 · Cited by 13
“Pharmaceutical companies and scientists from academia report that more than half of academic research is not reproducible (Begley and Ellis 2012; Ioannidis et al. 2009; Mullard 2011).”
BackgroundFord, E., & Deye, J. A. (2016) · Seminars in Radiation Oncology · doi:10.1016/j.semradonc.2016.06.002 · Cited by 13
Striving for Reproducible Science
Sweedler, J. V. (2015) · Analytical Chemistry · doi:10.1021/acs.analchem.5b04300 · Cited by 12
Introduction to the Special Issue on Methodological Rigor and Replicability
Stangor, C., & Lemay, E. P. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.02.006 · Cited by 17
Drug Development in Endometriosis and Adenomyosis: It Takes More Than Just Good Science
Groothuis, P. G., & Guo, S.-W. (2018) · Reproductive Sciences · doi:10.1177/1933719118785767 · Cited by 13
Maximizing the Reproducibility of Your Research
Open Science Collaboration (2017) · Wiley eBooks · doi:10.1002/9781119095910.ch1 · Cited by 17
Reporting In Vitro Experiments Responsibly – the RIVER RecommendationsPreprint
The RIVER working group (2023) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/x6aut · Cited by 25
Drivas, K., Lei, Z., & Wright, B. D. (2017) · International Journal of Industrial Organization · doi:10.1016/j.ijindorg.2017.10.005 · Cited by 12
Analyzing biomarker discovery: Estimating the reproducibility of biomarker sets
Forouzandeh, A., Rutar, A., Kalmady, S., & Greiner, R. (2021) · bioRxiv · doi:10.1371/journal.pone.0252697 · Cited by 13
“Indeed, Begley and Ellis [24] report that only 6 of 53 published findings in cancer biology could be confirmed; which Wen et al. [25] notes is “a rate approaching an alarmingly low 10% of reproducibility”.”
BackgroundCan antibodies be “vegan”? A guide through the maze of today’s antibody generation methods
Dübel, S. (2024) · mAbs · doi:10.1080/19420862.2024.2343499 · Cited by 15
Advancing biofoundry development: strategies and challenges
Gupta, A., Lee, S., Sung, B. H., Lee, D., Cho, B., Kim, D., & Kim, H. (2024) · Biotechnology and Bioprocess Engineering · doi:10.1007/s12257-024-00121-6 · Cited by 12
“Biofoundries have gained attention for their potential to address the challenges of limited speed and reproducibility of conventional experimental procedures [24], which often pose substantial challenges in biological research.”
Statistical Perspectives on Reproducibility: Definitions and Challenges
Simkus, A., Coolen‐Maturi, T., Coolen, F. P. A., & Bendtsen, C. (2025) · Journal of Statistical Theory and Practice · doi:10.1007/s42519-025-00459-x · Cited by 15
“Journals should strive to accept for publication articles with negative or null results [17, 19, 33].”
Balch, C., Arias‐Pulido, H., Banerjee, S., Lancaster, A. K., Clark, K. B., Perilstein, M., Hawkins, B. T., Rhodes, J., Piotr Sliz, A., Wilkins, J. F., et al. (2014) · BioEssays · doi:10.1002/bies.201400167 · Cited by 11
“A direct result is that recent examinations of preclinical cancer findings discovered reproducibility rates of less than 50% [6].”
BackgroundInfluentialBetter reporting for better research: a checklist for reproducibility
Kenall, A., Edmunds, S., Goodman, L., Bal, L., Flintoft, L., Shanahan, D. R., & Shipley, T. (2015) · BMC Neuroscience · doi:10.1186/s12868-015-0177-z · Cited by 13
“One well-publicised example of this was brought to light in an oncology study of preclinical research findings in which researchers were able to confirm only 11% of the findings [8, 9].”
BackgroundGreenfeld, M., van de Meent, J.-W., Pavlichin, D. S., Mabuchi, H., Wiggins, C. H., Gonzalez, R. L., & Herschlag, D. (2015) · BMC Bioinformatics · doi:10.1186/s12859-014-0429-4 · Cited by 15
“try where the reproducibility of scientific results has been questioned [17-20] has raised awareness that a lack of easy access to raw data (arising from many”
BackgroundCrystallization reports are the backbone ofActa Cryst. F, but do they have any spine?
Newman, J. M., Burton, D. R., Caria, S., et al. (2013) · Acta Crystallographica Section F Structural Biology and Crystallization Communications · doi:10.1107/s1744309113014152 · Cited by 12
“Indeed, a recent commentary in Nature shows just how unlikely it is that even those skilled in the art can now reproduce an experiment (Begley & Ellis, 2012).”
BackgroundGeorge, B. J., Sobus, J. R., Phelps, L. P., Rashleigh, B., Simmons, J. E., Hines, R. N., & Groups, T. C. O. P. F. S. G. D. W. (2015) · Toxicological Sciences · doi:10.1093/toxsci/kfv020 · Cited by 15
“…were the lack of appropriate statistical considerations, including inadequate attention to power, failure to consider bias during study design, less than rigorous or simply flawed data analysis, and failure to include adequate methods descriptions (Begley and Ellis, 2012; Prinz et al., 2011).”
InfluentialPositive and negative forms of replicability in gene network analysis
Verleyen, W., Ballouz, S., & Gillis, J. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv734 · Cited by 11
Tackling the Challenges of Clinical Trials for Severe Traumatic Brain Injury in Children
Kochanek, P. M., & Bell, M. J. (2015) · Critical Care Medicine · doi:10.1097/ccm.0000000000001041 · Cited by 12
Klaper, R. D., & Niemuth, N. J. (2016) · Chemosphere · doi:10.1016/j.chemosphere.2016.08.048 · Cited by 12
The Impact of Microenvironment on the Synovial Sarcoma Transcriptome
Jin, H., Barrott, J. J., Cable, M. G., Monument, M. J., Lerman, D. M., Smith-Fry, K., Nollner, D., & Jones, K. B. (2017) · Cancer Microenvironment · doi:10.1007/s12307-017-0192-y · Cited by 12
Reproducibility: A Primer on Semantics and Implications for Research
Pellizzari, E. D., Lohr, K. N., Blatecky, A. R., & Creel, D. V. (2017) · doi:10.3768/rtipress.2017.bk.0020.1708 · Cited by 13
Fast and Accurate Metadata Authoring Using Ontology-Based Recommendations.
Martínez-Romero, M., O’Connor, M. J., Shankar, R., Panahiazar, M., Willrett, D., Egyedi, A. L., Gevaert, O., Graybeal, J. B., & Musen, M. A. (2017) · PubMed · Cited by 15
Towards Reproducible Bioinformatics: The OpenBio-C Scientific Workflow Environment
Kanterakis, A., Iatraki, G., Pityanou, K., Koumakis, L., Kanakaris, N., Karacapilidis, N. I., & Potamias, G. (2019) · IEEE International Conference on Bioinformatics and Bioengineering (BIBE) · doi:10.1109/bibe.2019.00047 · Cited by 10
Translational Nano-medicine Lab to Clinic
Khan, M. N. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-817830-0.00008-4 · Cited by 12
Neuroethics: Fostering Collaborations to Enable Neuroscientific Discovery
Farahany, N. A., & Ramos, K. M. (2020) · AJOB Neuroscience · doi:10.1080/21507740.2020.1778117 · Cited by 16
“…anthropologists, chemists, and others) toward improved models for studying the human brain and its diseases is critical, since it is well-appreciated that many model systems—including animal ones—have limitations in their approximation of the human brain (Begley and Ellis 2012; Prinz et al. 2011).”
BackgroundZucchelli, P., Horak, G., & Skinner, N. (2020) · SLAS TECHNOLOGY · doi:10.1177/2472630320971218 · Cited by 13
The potential benefit of leptin therapy against amyotrophic lateral sclerosis (ALS)
Ferrer-Donato, Á., Contreras, A., Fernández, P., & Fernández-Martos, C. M. (2021) · Brain and Behavior · doi:10.1002/brb3.2465 · Cited by 11
“Accumulating in vivo and in vitro studies have suggested that leptin has notable effects on neuroprotection (Bahor et al., 2017; Begley & Ellis, 2012; Fernandez-Martos et al., 2017; Liu et al., 2020; Marwarha et al., 2010) as well as on improving learning and cognitive function in Alzheimert’s…”
BackgroundImproving clinical trials using Bayesian adaptive designs: a breast cancer example
Hong, W., McLachlan, S., Moore, M. M., et al. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01603-y · Cited by 12
“The advantages of Bayesian adaptive designs are particularly desirable in oncology, where clinical trial failure rates are the highest among all medical specialties [7, 8].”
BackgroundApproaches for the isolation and long-term expansion of pericytes from human and animal tissues
Alvino, V. V., Mohammed, K. A. K., Gu, Y., & Madeddu, P. R. (2023) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.1095141 · Cited by 17
Mitigating the epidemic of type I error: ecology and evolution can learn from other disciplines
Parker, T. H., & Nakagawa, S. (2014) · Frontiers in Ecology and Evolution · doi:10.3389/fevo.2014.00076 · Cited by 13
“Recently, it was reported that 47 of 53 “landmark” cancer studies from the past decade could not be reproduced (Begley and Ellis, 2012).”
BackgroundNew forms of checks and balances are needed to improve research integrity
Iorns, E. J., & Chong, C. (2014) · F1000Research · doi:10.12688/f1000research.3714.1 · Cited by 10
McBee, M. T., & Matthews, M. S. (2014) · Journal of Advanced Academics · doi:10.1177/1932202x14532177 · Cited by 14
“Begley and Ellis (2012) recently attempted to replicate findings published in a series of 53 "landmark" pre-clinical cancer studies and were only able to replicate 11% of the findings.”
ResultStatistical Rigor and the Perils of Chance
Button, K. S. (2016) · eNeuro · doi:10.1523/eneuro.0030-16.2016 · Cited by 12
Training to Improve Precision and Accuracy in the Measurement of Fiber Morphology
Hotaling, N. A., Jeon, J., Wade, M. B., Luong, D., Palmer, X., Bharti, K., & Simon, C. G. (2016) · PLoS ONE · doi:10.1371/journal.pone.0167664 · Cited by 11
PyMICE: APython library for analysis of IntelliCage data
Dzik, J. M., Puścian, A., Harda, Z., Radwańska, K., & Łęski, S. (2017) · Behavior Research Methods · doi:10.3758/s13428-017-0907-5 · Cited by 15
“Current efforts for reproducible research are mostly focused on improving the experimental procedures, statistical analysis, and the publishing policy (Begley & Ellis, 2012; Begley, 2013; Halsey, Curran-Everett, Vowler, & Drummond, 2015).”
Mikó, Z., Ujszegi, J., Gál, Z., & Hettyey, A. (2017) · Archives of Environmental Contamination and Toxicology · doi:10.1007/s00244-017-0427-4 · Cited by 14
“Replicability problems of scientific results have surfaced recently within various fields, such as cancer biology, drug discovery, or psychology (Prinz et al. 2011; Begley and Ellis 2012; Open Science Collaboration 2015).”
BackgroundSakaluk, J. K., & Graham, C. A. (2017) · The Journal of Sex Research · doi:10.1080/00224499.2017.1395387 · Cited by 13
“But while psychological (Nelson, Simmons, & Simonsohn, 2017; Spellman, 2015) and medical scientists (Baker, 2016; Begley & Ellis, 2012) have debated—often heatedly—the extent of the replicability problem (e.g., Open Science Collaboration, 2015; versus Gilbert, King, Pettigrew, & Wilson, 2016;…”
BackgroundWhen Scholarly Publishing Goes Awry: Educating Ourselves and Our Patrons about Retracted Articles
Thielen, J. (2018) · portal Libraries and the Academy · doi:10.1353/pla.2018.0009 · Cited by 13
Reporting in Experimental Philosophy: Current Standards and Recommendations for Future Practice
Polonioli, A., Vega‐Mendoza, M., Blankinship, B., & Carmel, D. (2018) · Review of Philosophy and Psychology · doi:10.1007/s13164-018-0414-3 · Cited by 12
Osimani, B. (2020) · Boston studies in the philosophy of science · doi:10.1007/978-3-030-29179-2_15 · Cited by 13
Small Animal Shanoir (SAS) A Cloud-Based Solution for Managing Preclinical MR Brain Imaging Studies
Kain, M., Bodin, M., Loury, S., Chi, Y., Louis, J., Simon, M., Lamy, J., Barillot, C., & Michel Dojat, D. (2020) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2020.00020 · Cited by 14
“The problem of replication and reproducibility in preclinical studies is now well documented (Sena et al., 2010; Begley and Ellis, 2012; Tsilidis et al., 2013) and can negatively impacted the translation to human studies, leading to waste of resources, unnecessary risks for volunteers and slowing…”
BackgroundQuality management for robust and reliable research
Giesen, E. (2015) · International Journal of Metrology and Quality Engineering · doi:10.1051/ijmqe/2015029 · Cited by 16
Reviews in Research on Adolescence: Genres, Trends, and Challenges
Levesque, R. J. R. (2015) · Adolescent Research Review · doi:10.1007/s40894-015-0022-8 · Cited by 12
“…in the highly regarded scientific journal Nature reported that Amgen, a major American biopharmaceutical company, sought to replicate the findings of 53 published cancer research studies that it deemed ‘‘high profile;’’ it was able to reproduce only 11 % (6 of the 53) (Begley and Ellis 2012).”
BackgroundReporting of Preclinical Research in Anesthesiology
Eisenach, J. C., Warner, D. S., & Houle, T. T. (2016) · Anesthesiology · doi:10.1097/aln.0000000000001044 · Cited by 12
“Among many reporting elements, COnSORT requires the following: (1) an adequate description of the experiments to allow other researchers to replicate them, (2) report of the measures used to reduce bias, including whether and how random allocation and blinding methods were used, (3) how the sample size was determined,…”
MethodState of the Art Review: The Data Revolution in Critical Care
Ghassemi, M., Celi, L. A., & Stone, D. J. (2015) · Annual update in intensive care and emergency medicine · doi:10.1007/978-3-319-13761-2_42 · Cited by 12
Why Is It so Hard to Do Good Science?
Dingledine, R. J. (2018) · eNeuro · doi:10.1523/eneuro.0188-18.2018 · Cited by 13
“…practices such as blinding and inclusion of both positive and negative controls have all been highlighted as likely sources of misleading preclinical research (Ioannidis, 2005, 2018; Begley and Ellis, 2012; Button et al., 2013; Marino, 2014; Halsey et al., 2015; Steward, 2016; Colquhoun, 2017).”
BackgroundThe Replicability Crisis and Public Trust in Psychological SciencePreprint
Anvari, F., & Lakens, D. (2019) · doi:10.31234/osf.io/vtmpc · Cited by 12
A simple model suggesting economically rational sample-size choice drives irreproducibility
Braganza, O. (2020) · PLoS ONE · doi:10.1371/journal.pone.0229615 · Cited by 12
“Two, now prominent, studies from the pharmaceutical industry suggested reproducibility rates of 11 and 22% [1, 2].”
BackgroundMcGenity, C., Bossuyt, P. M., & Treanor, D. E. (2022) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2022.100091 · Cited by 16
Chiriboga, L. A., Callis, G. M., Wang, Y., & Chlipala, E. A. (2022) · Journal of Histotechnology · doi:10.1080/01478885.2022.2134022 · Cited by 14
Jamaladdin, N., Sigaud, R., Kocher, D., Kolodziejczak, A. S., Nonnenbroich, L. F., Ecker, J., Usta, D., Benzel, J., Peterziel, H., Pajtler, K. W., et al. (2023) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.3002 · Cited by 13
Zhu, J., Zhang, Y., Zhao, Y., Zhang, J., Hao, K., & He, H. (2023) · Pharmaceutics · doi:10.3390/pharmaceutics15092274 · Cited by 12
“In addition, a successful translational approach is essential in applying these models to predict clinical efficacy for candidate screening and dosing schedule optimization [2,3].”
von Kortzfleisch, V. T., & Richter, S. H. (2023) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2023.109992 · Cited by 15
Grid/lattice therapy: consideration of small field dosimetry
Das, I. J., Khan, A. U., Dogan, S. K., et al. (2024) · British Journal of Radiology · doi:10.1093/bjr/tqae060 · Cited by 11
Buffenstein, R., & Amoroso, V. G. (2024) · The Journals of Gerontology Series A · doi:10.1093/gerona/glae110 · Cited by 13
A review on the development and application of microfluidic concentration gradient generators
Wu, Z., Yang, H., Xu, H., Dai, W., Xu, L., Du, H., & Zhang, D. (2024) · Physics of Fluids · doi:10.1063/5.0219368 · Cited by 12
Human Organotypic Lung Tumor Models: Suitable For Preclinical 18F-FDG PET-Imaging
Fecher, D., Hofmann, E., Buck, A. K., Bundschuh, R. A., Nietzer, S., Dandekar, G., Walles, T., Walles, H., Lückerath, K., & Steinke, M. R. (2016) · PLoS ONE · doi:10.1371/journal.pone.0160282 · Cited by 15
“One reason for this is the lack of appropriate predictive model systems to distinguish efficient from non-efficient drug compounds early and cost-effectively [11 – 13].”
Metadata
- Authors
- Begley, C. G., & Ellis, L. M.
- Affiliations
- Amgen (United States) (US)
- The University of Texas MD Anderson Cancer Center (US)
- Corresponding author
- Ellis, Lee M.
- Venue
- Nature
- Published
- 2012-03-28
- Volume
- 483
- Issue
- 7391
- Pages
- 531-533
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W1987777080
- Semantic Scholar
- 1a81d3d1e8a983e0551edf485f4ac1a8cdd7fea4
- Cited by
- 2,979
- References
- 5
- Citations by year
- 2026: 106 · 2025: 116 · 2024: 127 · 2023: 169 · 2022: 179 · 2021: 206 · 2020: 236 · 2019: 255 · 2018: 262 · 2017: 271 · 2016: 283 · 2015: 262 · 2014: 226 · 2013: 133 · 2012: 62
- FWCI
- 210.63
- Type
- journal article
- Language
- en
- Update notices
- correction (2012-05-02)
- Open access
- bronze
- doi.org
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health and Medical Research Impacts
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Health Sciences › Nursing › Issues, ethics and legal aspects › Biomedical Ethics and Regulation
- Keywords
- preclinical cancer research, research standards, publication practices, research incentives, translational research, clinical trials
- MeSH terms
- Animals, Clinical Trials as Topic, Drug Evaluation, Preclinical, Humans, Neoplasms, Research Design, Reproducibility of Results, Survival Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.