Raise standards for preclinical cancer research
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index; 3 of the 9 sources its claims cite were checked for support.
Overview
CoChat Summary
TL;DR
This Nature Comment argues that many landmark preclinical oncology findings are not reproducible and calls for higher standards in methods, reporting, and incentives. It reports that Amgen confirmed only 6 of 53 landmark papers (11%) and cites Bayer’s finding that about 25% of published preclinical studies could be validated.
Key findings
- In Amgen’s ten-year retrospective analysis, scientific findings were confirmed in only 6 of 53 “landmark” preclinical cancer papers (11%); non-reproduced studies often lacked blinding, relied on single experiments such as one Western blot, or presented selected data.
- Bayer HealthCare reported that only about 25% of published preclinical studies could be validated to the point at which projects could continue; published cancer research represented 70% of the studies analysed in that report, with possible overlap with Amgen’s 53 papers.
- Citation data showed non-reproduced articles in journals with impact factor >20 had a mean of 248 citations (range 3–800) versus 231 (range 82–519) for reproduced articles; for impact factor 5–19, non-reproduced articles had a mean of 169 citations (range 6–1,909) versus 13 (range 3–24) for reproduced articles.
- The authors recommend reporting negative data, blinding preclinical experiments, validating reagents, repeating critical experiments, avoiding single cell lines or models, requiring patient-selection biomarkers, and changing academic incentives.
Methods
- This is a Nature Comment/perspective, not a primary experimental study; it synthesizes Amgen’s ten-year retrospective analysis of experiments performed prospectively and cites a Bayer HealthCare report.
- Amgen selected 53 “landmark” papers describing completely new approaches or alternative clinical uses, attempted confirmation, used additional models, contacted original authors, exchanged reagents, and repeated experiments under the authors’ direction, occasionally in the original investigator’s laboratory.
- The analysis reports counts of confirmed findings and citation metrics from Google Scholar, May 2011; the recommendations are proposed changes rather than tested interventions.
Limitations
- Stated: validation attempts may have failed because of technical differences or difficulties despite efforts to ensure otherwise; additional models were used; when findings could not be reproduced, authors were contacted and experiments were repeated under their direction.
- Stated: papers were deliberately selected as “landmark” and described something completely new, so some data might not hold up; Bayer’s analysed studies may overlap with Amgen’s 53 papers.
- Evident: as a Comment, the text provides no detailed systematic protocol, inclusion criteria, statistical methods, or full dataset; the citation analysis is descriptive and limited to two impact-factor strata.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 1 Not valid · 6 Not checked · 12 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Sadly, clinical trials in oncology have the highest failure rate compared with other therapeutic areas.”
What the sources say
- Partly supportedcites [1] Hutchinson, L. & Kirk, R. Nature Rev. Clin. Oncol. 8, 189–190 (2011). sourceExperimental — accuracy not yet measured
The opening passage of our copy (labelled Abstract; page unavailable) says cancer-drug attrition is much higher than in other therapeutic areas, citing 5% of anticancer agents versus 20% of cardiovascular agents reaching licensure. That comparison covers development from preclinical activity through phase III, not failure rates specifically among clinical trials; it also does not explicitly establish a highest-rate ranking. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Although we in the cancer field hoped that this would lead to more effective drugs, historically, our ability to translate cancer research to clinical success has been remarkably low 1 .”
What the sources say
- Supportedcites [1] Hutchinson, L. & Kirk, R. Nature Rev. Clin. Oncol. 8, 189–190 (2011). source
The opening passage (labeled Abstract in our copy; no page given) reports that only 5% of agents with preclinical anticancer activity are licensed after demonstrating sufficient efficacy in phase III testing. It also says cancer drug attrition is much higher than in other therapeutic areas, supporting the claim of historically low translation to clinical success. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In studies for which findings could be reproduced, authors had paid close attention to controls, reagents, investigator bias and describing the complete data set.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For results that could not be reproduced, however, data were not routinely analysed by investigators blinded to the experimental versus control groups.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Investigators frequently presented the results of one experiment, such as a single Western-blot analysis.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
12 more claims
Create a free account to see all 22 claims and every source verdict.
- It may state two separate claims as one. 2
- Probably not a substantial claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“there are other more common sources of unreliability in the literature (41)(42)(43)(44)”
From Misconduct accounts for the majority of retracted scientific publications (2012) · cites this paper as [42]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“many established scientific results in the social (49, 50) and biomedical (51-53) sciences cannot readily be replicated.”
From Misinformation in and about science (2021) · cites this paper as [51]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A landmark study conducted by Glenn Begley and his team in 2012 concluded that most cancer research studies were not reproducible, by alarmingly reporting that 47 of 53 published findings could not be replicated [13].”
From Guidelines for the use of flow cytometry and cell sorting in immunological studies (third edition) (2021) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In cell biology, two industrial laboratories reported success replicating the original results of landmark findings in only 11% and 25% of the attempted cases ( 10, 11 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Concerns about reproducibility are widespread across disciplines ( 921 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [10]
Experimental — accuracy not yet measured
13 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888
Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010271 · Cited by 805
High drug attrition rates—where are we going wrong?
Hutchinson, L., & Kirk, R. (2011) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2011.34 · Cited by 517
Drug development and clinical trials—the path to an approved cancer drug
Rubin, E. H., & Gilliland, D. G. (2012) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2012.22 · Cited by 135
Raising the bar for cancer therapy models
Francia, G., & Kerbel, R. S. (2010) · Nature Biotechnology · doi:10.1038/nbt0610-561 · Cited by 54
Cited by
Items 1,801–1,900 of 2,889
Ejima, K., Brown, A. W., Smith, D. L., Beyaztaş, U., & Allison, D. B. (2020) · International Journal of Obesity · doi:10.1038/s41366-020-0554-2 · Cited by 5
Blockchain Based Peer-Review Interfaces for Digital Medicine
Dhillon, V. (2020) · Frontiers in Blockchain · doi:10.3389/fbloc.2020.00008 · Cited by 10
“…was also brought to light by Dr. Lee Ellis from MD Anderson who wrote a similar article in 2012 on cancer biology titled “Raise standards for preclinical cancer research” reporting that only 10% of published preclinical studies could be validated to enter clinical trials (Begley and Ellis, 2012).”
BackgroundCo-occurrence of Cell Lines, Basal Media and Supplementation in the Biomedical Research Literature
Cox, J., McBeath, D., Harper, C. A., & Daniel, R. (2020) · Journal of Data and Information Science · doi:10.2478/jdis-2020-0016 · Cited by 4
“In 2012, Amgen reported an 11% reproducibility rate of 53 landmark papers in cancer biology (Begley & Ellis, 2012), and in 2018 The Reproducibility Project: Cancer Biology cut the number of studies they worked to replicate from 50 to 18, due to a variety of factors, and of these could only…”
BackgroundBuckett, A., Becker, J. R., Melchers, K. G., & Roodt, G. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.00459 · Cited by 6
“Given recent concerns about the replicability of research findings in many different domains (Ioannidis, 2005; Begley and Ellis, 2012; Laws, 2013; American Psychological Society, 2015; Maxwell et al., 2015), this would help to reassure AC designers and practitioners who are targeting dimensions.”
BackgroundBiomonitoring and Digital Data Technology as an Opportunity for Enhancing Animal Study Translation
Defensor, E. B., Lim, M. A., & Schaevitz, L. R. (2021) · ILAR Journal · doi:10.1093/ilar/ilab018 · Cited by 4
Sonawane, A., Vadloori, B., Poosala, S., Kanďárová, H., Kulkarni, M., Olayanju, A., Dey, T., Saxena, U., Smirnova, L., Kanda, Y., et al. (2022) · Alternatives to Laboratory Animals · doi:10.1177/02611929221089216 · Cited by 5
Miksa, T., Strodl, S., & Rauber, A. (2014) · International Journal of Digital Curation · doi:10.2218/ijdc.v9i1.303 · Cited by 5
“This hinders the possibility of reliable verification and validation of results (National Science Foundation, 2011), as well as the reproducibility of research, which is one of the key assumptions of credible data-driven science (Lesk, 2012; Begley and Ellis, 2012).”
Boulton, G. S. (2014) · Insights the UKSG journal · doi:10.1629/2048-7754.148 · Cited by 6
Andrology is desperate for a new assay – Let us make sure we get it right this time…
Barratt, C. L. R., & Mansell, S. A. (2013) · Middle East Fertility Society Journal · doi:10.1016/j.mefs.2013.01.008 · Cited by 6
The Need to Improve Experimental Design
Fry, D. (2013) · Alternatives to Laboratory Animals · doi:10.1177/026119291304100512 · Cited by 5
Statistics Corner: Data Cleaning-I
Kishore, K., Singh, A., & Kapoor, R. (2019) · Journal of Postgraduate Medicine Education and Research · doi:10.5005/jp-journals-10028-1330 · Cited by 5
COEL: A Web-based Chemistry Simulation FrameworkPreprint
Banda, P., Blount, D., & Teuscher, C. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1407.4027 · Cited by 5
“Furthermore, because of private and local storing only 11% of the academic research in the literature was reproducible by the original research groups, as reported in Nature [2].”
BackgroundAn Assessment of Retractions as a Measure of Scientific Misconduct and Impact on Public Health Risks
Abritis, A. J. (2015) · Digital Commons - University of South Florida (University of South Florida) · Cited by 6
Towards structured sharing of raw and derived neuroimaging data across existing resourcesPreprint
Keator, D. B., Helmer, K. G., Steffener, J., et al. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1209.5922 · Cited by 5
“…neuroimaging data, meta-data, and complete provenance data along with research publications is also likely to accelerate scientific discovery through earlier detection of non-replicable research findings; a topic that has recently received a lot of attention (Begley and Ellis, 2012 ; Yong, 2012).”
BackgroundInfluentialFecher, B., Frrrdorf, M., & Wagner, G. G. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2768629 · Cited by 6
“A recent paper in Nature reported a failure to replicate significant experiments in the domain of cancer research in 47 out of 53 cases [16].”
BackgroundShould There Be Minimum Information Reporting Standards for Sensors?
Gooding, J. J., Bakker, E., Kelley, S. O., Long, Y., Merkx, M., Sailor, M. J., & Tao, N. J. (2017) · ACS Sensors · doi:10.1021/acssensors.7b00737 · Cited by 5
Open Science Badges in the Journal of Neurochemistry
Schweitzer, B., & Schulz, J. B. (2018) · Journal of Neurochemistry · doi:10.1111/jnc.14536 · Cited by 6
An open source web application for distributed geospatial data exploration
Curry, P. A., & Moosdorf, N. (2019) · Scientific Data · doi:10.1038/sdata.2019.14 · Cited by 5
Cheleuitte-Nieves, C., & Lipman, N. S. (2020) · ILAR Journal · doi:10.1093/ilar/ilaa022 · Cited by 6
GGM knockoff filter: False Discovery Rate Control for Gaussian Graphical ModelsPreprint
Li, J., & Maathuis, M. H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.11611 · Cited by 5
“The Benjamini-Hochberg (BH) procedure (see Benjamini and Hochberg (1995)) is also commonly used to control the FDR based on p-values while it requires the socalled Positive Regression Dependency on each one from a Subset (PRDS) assumption (see Benjamini and Yekutieli (2001)).”
BackgroundMethodInfluentialMaximizing the Reproducibility of Your ResearchPreprint
Bosco, F. A., Carp, J., Field, J. G., IJzerman, H. R., Lewis, M., Munafo, M. R., Nosek, B. A., Prenoveau, J. M., Spies, J. R., & Giner‐Sorolla, R. (2016) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/k9mn3 · Cited by 5
Beauchemin, C. A. A., Paradis, E. G., Pinilla, L. T., et al. (2021) · doi:10.32920/14638854.v1 · Cited by 5
Evaluating Research Transparency and Openness in Communication Sciences and Disorders JournalsPreprint
Schroeder, S. R., Gaeta, L. M., Amin, M. E., et al. (2022) · doi:10.31234/osf.io/dy5zs · Cited by 5
Optimizing murine sample sizes for RNA-seq studies revealed from large-scale comparative analysisPreprint
Halász, G., Schmahl, J., Negron, N., Ni, M., Atwal, G. S., & Glass, D. J. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.07.08.602525 · Cited by 5
“This is a result of several studies indicating that quite a few high profile papers were found to be non-reproducible upon attempts to repeat those studies 17, 18 .”
ResultScience as a global public good: A necessary responsibility for science
Boulton, G. S. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-23655-6.00001-0 · Cited by 4
Romero-Becerra, R., Zhao, Z. Q., Nebdal, D. J. H., Müller, E., Bergholtz, H., Norum, J. H., & Aittokallio, T. A. (2025) · Nature Communications · doi:10.1038/s41467-025-65218-9 · Cited by 8
“Reproducibility of results from preclinical cancer models remains a critical challenge, requiring improvements in consistency 30,31 .”
Filipova, D., Boomgaarden, W., Fox, L., Knight, A., Lickiss, B., Maaß, C., Neumann, G., Watkins, J. C., Herrmann, K., Ewart, L., et al. (2026) · NAM journal. · doi:10.1016/j.namjnl.2026.100119 · Cited by 6
“…for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) since the 1990s, researchers and pharmaceutical companies continue to report low reproducibility of preclinical data, partly due to inadequate cell lines and animal data (Prinz et al., 2011; Begley and Ellis, 2012).”
Pursuitworthiness in urgent research: Lessons on well-ordered science from sustainability science
Stojanovic, M. (2023) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2023.01.004 · Cited by 5
Hanley, B. P., & Gross, G. A. (2024) · Journal of Biological Methods · doi:10.14440/jbm.2024.0037 · Cited by 4
The Controversy behind the Controversies: Scientific Discourse in the Twenty-First Century
Fahnestock, J. (2025) · Rhetoric Society Quarterly · doi:10.1080/02773945.2025.2484162 · Cited by 6
ter Riet, G., Storosum, B. W. C., & Zwinderman, A. H. (2019) · F1000Research · doi:10.12688/f1000research.17615.1 · Cited by 4
Two-sample t α -test for testing hypotheses in small-sample experiments
Tan, Y.-D. (2022) · The International Journal of Biostatistics · doi:10.1515/ijb-2021-0047 · Cited by 5
Parish, A. J., Cristea, I. A., Schuit, E., & Ioannidis, J. P. A. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.06.013 · Cited by 4
Frequency Fitness Assignment on JSSP: A Critical Review
de Bruin, K. E., Thomson, S. L., & van den Berg, D. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-30229-9_23 · Cited by 4
Park, S. H., Lee, S. B., Park, S., Kim, E. Y., Pizzol, D., Trott, M., Barnett, Y., Koyanagi, A., Jacob, L., Soysal, P., et al. (2023) · British Journal of Urology · doi:10.1111/bju.16171 · Cited by 4
“Compliance with the guidelines ensures the improved reliability and reproducibility of the experiment and can subsequently affect success in clinical research and lead to positive results for patients with disease [6,7,23].”
BackgroundLessons in immune adaptations to hypoxia revealed by comparative and evolutionary physiology
Heinrich, E. C., & Tift, M. S. (2023) · BMC Biology · doi:10.1186/s12915-023-01788-8 · Cited by 4
“It has become clear that many discoveries made in rodent models are not replicated in humans, leading to low rates of successful translation to clinical trials [9].”
BackgroundA powerful approach to identify replicable variants in genome-wide association studies
Li, Y., Lei, H., Wen, X., & Cao, H. (2024) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2024.04.004 · Cited by 4
The Problem-Ladenness of Theory
Levenstein, D., Santo, A. D., Heijnen, S., Narayan, M., Maatman, F. O., Rawski, J., & Wright, C. (2024) · Computational Brain & Behavior · doi:10.1007/s42113-024-00219-3 · Cited by 7
Pelot, N. A., Wang, B., Marshall, D. P., Hussain, M. A., Musselman, E. D., Yu, G. J., Dale, J., Baumgart, I. W., Dardani, D., Zamani, P. T., et al. (2025) · Journal of Neural Engineering · doi:10.1088/1741-2552/adb997 · Cited by 6
Behavioral Pharmacology of Pain
Berge, O. (2014) · Current topics in behavioral neurosciences · doi:10.1007/7854_2014_281 · Cited by 5
Evaluating the Effectiveness of Cancer Drug Sensitization In Vitro and In Vivo
Rytelewski, M., Buensuceso, A. V., Leong, H. S., Deroo, B. J., Chambers, A. F., & Koropatnick, J. D. (2015) · Journal of Visualized Experiments · doi:10.3791/52388 · Cited by 4
Is there a robust future for research in reproduction?
Barratt, C. L. R. (2015) · Molecular Human Reproduction · doi:10.1093/molehr/gav062 · Cited by 4
“Remarkably, the findings could only be confirmed in six cases (Begley and Ellis, 2012).”
BackgroundHow Feedback Loops Can Improve Aid and Governance
Whittle, D. B. (2016) · Springer eBooks · doi:10.1007/978-3-319-28643-3_4 · Cited by 4
Pharmacology in 21st century biomedical research
Mullane, K., Williams, M. J., & Winquist, R. J. (2013) · Biochemical Pharmacology · doi:10.1016/j.bcp.2013.10.018 · Cited by 4
How Will Open Science Impact on University/Industry Collaborations?
Chataway, J., Parks, S., & Smith, E. (2018) · Springer eBooks · doi:10.1007/978-3-319-62649-9_12 · Cited by 4
A patient called Medical Research
Fürst, T., & Strojil, J. (2017) · Biomedical Papers · doi:10.5507/bp.2017.005 · Cited by 3
Bermúdez‐Tamayo, C., Muñoz, J. B., Briones‐Vozmediano, E., Carrasco‐Portiño, M., Zunzunegui, M. V., Negrín, M. Á., González, L., García, L. P., & Díaz, C. Á.-D. (2017) · Gaceta Sanitaria · doi:10.1016/j.gaceta.2017.03.002 · Cited by 5
Novel Early Phase Clinical Trial Design in Oncology
O’Brien, C., Carter, L., Cook, N., & Dean, E. J. (2017) · Pharmaceutical Medicine · doi:10.1007/s40290-017-0205-7 · Cited by 4
“In the past, there have been doubts about the reliability and reproducibility of preclinical data; going forward, preclinical data need to be held to the same rigorous standards as clinical data [76].”
BackgroundNational Cancer Institute awards—a work in progress
Fojo, T., & Giannakakou, P. (2014) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2014.173 · Cited by 4
Analytical Cascades of Enzymes for Sensitive Detection of Structural Variations in Protein Samples
Hollerweger, J. C., Hoppe, I. J., Regl, C., Stock, L. G., Huber, C. G., Lohrig, U., Stutz, H., & Brandstetter, H. (2018) · Analytical Chemistry · doi:10.1021/acs.analchem.7b04874 · Cited by 4
Cribiù, F. M., Burrello, C., Ercoli, G., Garavaglia, F., Villanacci, V., Caprioli, F., Bòsari, S., & Facciotti, F. (2018) · European Journal of Inflammation · doi:10.1177/2058739218776883 · Cited by 4
“5 Nevertheless, while human samples are processed and scored by histopathology core facilities and experienced human pathologists through precise histopathological criteria, murine tissues are usually fixed, included and analysed by researchers with limited experience in histopathology and…”
BackgroundFricker, S. P., Sprott, K. M., Spyra, M., Uhlig, P. C., Lange, N., David, K. A., & Wang, Y. (2018) · Journal of Histochemistry & Cytochemistry · doi:10.1369/0022155418818788 · Cited by 4
Planejamento e divulgação da pesquisa com animais como parâmetro de integridade
Fischer, M. L., & Rodrigues, G. S. (2018) · Revista Bioética · doi:10.1590/1983-80422018264273 · Cited by 4
Toward Replicability With Confidence Intervals for the Exceedance Probability
Segal, B. D. (2019) · The American Statistician · doi:10.1080/00031305.2019.1678521 · Cited by 6
“Several scientific fields including psychology are undergoing a replication crisis (Ioannidis, 2005; Prinz et al., 2011; Begley and Ellis, 2012; Pashler and Wagenmakers, 2012), which has led to increased attempts to understand the replicability and reliability of scientific studies (Open Science Collaboration, 2015; J…”
BackgroundReplication to assess statistical adequacy
Owen, P. D. (2018) · Economics · doi:10.5018/economics-ejournal.ja.2018-60 · Cited by 4
“…crisis” affecting multiple disciplines in the sciences, biomedicine and the social sciences (Ioannidis, 2005; Ioannidis and Panagiotou, 2011; Begley and Ellis, 2012; Doyen et al., 2012; Button et al., 2013; Open Science Collaboration, 2015; Dumas-Mallet et al., 2016; Baker, 2016; Hubbard,…”
BackgroundMisreferencing Practice of Scientists: Inside Researchers’ Sociological and Bibliometric Profiles
Sauvayre, R. (2022) · Social Epistemology · doi:10.1080/02691728.2021.2022807 · Cited by 6
“The literature (Fenton et al. 2000; Begley and Ellis 2012; Hicks et al. 2015) suggests that the professional survival of researchers – in terms of tenure, academic advancement, and funding – may depend on citation analysis.”
BackgroundFitt, B., Loy, G., Christopher, E., Brennan, P. M., & Poon, M. T. C. (2022) · PLoS ONE · doi:10.1371/journal.pone.0264740 · Cited by 5
Advances in haploid embryonic stem cell research
Liu, C., & Li, W. (2022) · Biology of Reproduction · doi:10.1093/biolre/ioac110 · Cited by 4
“This will affect the analysis of results and reduce the repeatability of the experiment [52].”
BackgroundBioTEA: Containerized Methods of Analysis for Microarray-Based Transcriptomics Data
Visentin, L., Scarpellino, G., Chinigò, G., Munaron, L., & Ruffinatti, F. A. (2022) · Biology · doi:10.3390/biology11091346 · Cited by 5
Effects of home‐cage elevation on behavioral tests in mice
Ueno, H., Takahashi, Y., Murakami, S., Wani, K., Matsumoto, Y., Okamoto, M., & Ishihara, T. (2023) · Brain and Behavior · doi:10.1002/brb3.3269 · Cited by 5
“In fact, the nonreproducible prevalence has been estimated to range from 50% to 90% (Begley & Ellis, 2012; Prinz et al., 2011).”
BackgroundBała, M. M., Peričić, T. P., Žuljević, M. F., Bralić, N., Zając, J., Motaze, N. V., Rohwer, A. C., Gajdzica, M., & Young, T. (2024) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2023-112647 · Cited by 6
“5 However, studies found low replication rates in biomedical research, 6 and the same pattern was observed in other fields such as psychology 7 8 or social science.”
Ronquillo, J. G., South, B. R., Naik, P., Singh, R., Jesus, M. D., WATT, S. J., & Habtezion, A. (2024) · JCO Clinical Cancer Informatics · doi:10.1200/cci.24.00087 · Cited by 4
Establishment of a syngeneic orthotopic model of prostate cancer in immunocompetent rats
Suzuki, S., Naiki‐Ito, A., Kuno, T., et al. (2015) · Journal of Toxicologic Pathology · doi:10.1293/tox.2014-0050 · Cited by 4
From academia to industry: a road more travelled
Pao, W. J. (2017) · Annals of Oncology · doi:10.1093/annonc/mdx319 · Cited by 4
Kato, T. (2018) · The Japanese Journal of Personality · doi:10.2132/personality.27.2.11 · Cited by 6
Munkácsy, G., Herman, P. G., & Győrffy, B. (2019) · Scientific Reports · doi:10.1038/s41598-018-36122-8 · Cited by 7
Statistical inference and reproducibility in geobiology
Sperling, E. A., Tecklenburg, S., & Duncan, L. E. (2019) · Geobiology · doi:10.1111/gbi.12333 · Cited by 5
“For example, independent efforts to reproduce “landmark papers” in cancer biomedical research by pharmaceutical companies Amgen (Begley & Ellis, 2012) and Bayer (Prinz, Schlange, & Asadullah, 2011) have only been able to replicate 11% and 20%–25% of published results, respectively.”
BackgroundDaskalopoulos, E. P., Hermans, K. C. M., Debets, J., et al. (2019) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2019.00118 · Cited by 4
Costs of Implementing Quality in Research Practice
Littrell, O. M., Stoeger, C., Maier, H., Fuchs, H., de Angelis, M. H., Cassis, L. A., Gerhardt, G. A., Grondin, R., & Gailus-Durner, V. (2019) · Handbook of experimental pharmacology · doi:10.1007/164_2019_294 · Cited by 5
Improving quality of preclinical academic research through auditing: A feasibility study
Kurreck, C., Castaños‐Vélez, E., Freyer, D., Blumenau, S., Przesdzing, I., Bernard, R., & Dirnagl, U. (2020) · PLoS ONE · doi:10.1371/journal.pone.0240719 · Cited by 6
COVID-19: The urgent call for academic research in research ethics
Bommier, C., Stoeklé, H., & Hervé, C. (2021) · Ethics Medicine and Public Health · doi:10.1016/j.jemep.2021.100679 · Cited by 4
Factor Investing in Sovereign Bond Markets: Deep Sample Evidence
Baltussen, G., Martens, M. P. E., & Penninga, O. (2021) · The Journal of Portfolio Management · doi:10.3905/jpm.2021.1.311 · Cited by 7
What does literature teach about digital pathology? A bibliometric study in Web of Science
López-Belmonte, J., Segura-Robles, A., Cho, W. C. S., et al. (2021) · IJERI International Journal of Educational Research and Innovation · doi:10.46661/ijeri.4918 · Cited by 4
The approach to reproducible research of the Image Processing On Line (IPOL) journal
Nicolaï, A., Bammey, Q., Gardella, M., Nikoukhah, T., Boulant, O., Bargiotas, I., Monzón, N., Truong, C., Kerautret, B., Monasse, P., et al. (2022) · Informatio · doi:10.35643/info.27.1.7 · Cited by 4
Keep 'reproducibility' in context
Picha, B., Thompson, M., & Vondriska, T. M. (2012) · Nature · doi:10.1038/485041d · Cited by 4
Treating Rationality as a Continuous Variable
Etzioni, A. (2014) · Society · doi:10.1007/s12115-014-9798-6 · Cited by 8
“One study of preclinical cancer research found that, of 53 studies that contributed “landmark” advances to the field, only six could be replicated—a mere 11 % (Begley and Ellis, 2012).”
BackgroundCOEL: A Cloud-Based Reaction Network Simulator
Banda, P., & Teuscher, C. (2016) · Frontiers in Robotics and AI · doi:10.3389/frobt.2016.00013 · Cited by 4
“Similarly, only 11% of the academic research was reproducible by the original research groups, as reported by Begley and Ellis (2012).”
Janero, D. R. (2016) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2016.1212014 · Cited by 6
Mixed outcomes for computational predictions
Dang, C. V. (2017) · eLife · doi:10.7554/elife.22661 · Cited by 4
“There have been growing concerns about irreproducibility in the biological literature in recent years (Begley and Ellis, 2012; Errington et al., 2014; Baker, 2016).”
BackgroundMansour, N. M., Balas, E. A., Yang, F. M., & Vernon, M. (2020) · Medical Science Monitor · doi:10.12659/msm.922016 · Cited by 4
“, drug target identification and validation), the majority of published preclinical results could not be validated, implying poor quality research and wasted efforts to replicate [2,3].”
BackgroundFishburn, C. S. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-800687-0.00032-3 · Cited by 4
Kern, S. E. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.0919 · Cited by 15
“” Begley and Ellis reported that nearly 90% of preclinical drug studies could not be replicated (21).”
BackgroundExperimental Screening Strategies to Reduce Attrition Risk
Peakman, M., Troutman, M. D., Gonzales, R., & Schmidt, A. W. (2015) · Wiley eBooks · doi:10.1002/9781118819586.ch7 · Cited by 7
Science with no fiction: measuring the veracity of scientific reports by citation analysisPreprint
Grabitz, P., Lazebnik, Y. A., Nicholson, J. M., & Rife, S. C. (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/172940 · Cited by 5
“The current crisis of veracity in biomedical research, and having less than a half of preclinical studies reproducible (Begley & Ellis, 2012; Prinz, Schlange, & Asadullah, 2011) is truly a crisis, has spilled from a discussion in scientific journals (Begley & Ellis, 2012; Casadevall & Fang, 2010;…”
BackgroundInfluentialGenetics and Family Systems: Articulation and Disarticulation
Reiss, D. (2016) · Wiley eBooks · doi:10.1002/9781119125556.devpsy209 · Cited by 4
Computational Tools for ADMET Profiling
Fourches, D., Williams, A. J., Patlewicz, G. Y., Shah, I., Grulke, C., Wambaugh, J. F., Richard, A., & Tropsha, A. (2018) · Wiley eBooks · doi:10.1002/9781119282594.ch8 · Cited by 7
The Fair Reward Problem: The Illusion of Success and How to Solve ItPreprint
Sornette, D., Wheatley, S., & Cauwels, P. P. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3377177 · Cited by 5
McCabe, M. J., & Mueller‐Langer, F. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3329272 · Cited by 5
“, 2008) or other fields such as medicine (Alsheikh-Ali et al., 2011; Begley and Ellis, 2012).”
Samota, E. K., & Davey, R. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/581033 · Cited by 4
Bikard, M. A., & Marx, M. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3006859 · Cited by 4
Stewart, A. G. (2017) · Frontiers in Pharmacology · doi:10.3389/fphar.2017.00008 · Cited by 5
“, Amgen) to reproduce preclinical findings, with very low success rates being reported (Begley and Ellis, 2012).”
Background(2012) · Nature · doi:10.1038/485041e · Cited by 4
A Call for Open Science in Giftedness ResearchPreprint
McBee, M. T., Makel, M. C., Peters, S. J., & Matthews, M. S. (2017) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/nhuv3 · Cited by 7
Nonspecific membrane bilayer perturbations by ivermectin underlie SARS-CoV-2 in vitro activityPreprint
Eastman, R. T., Rusinova, R., Herold, K. F., Huang, X., Dranchak, P. K., Voss, T. C., Rana, S., Shrimp, J. H., White, A. D., Hemmings, H. C., et al. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.10.23.563088 · Cited by 3
DeScAI: the convergence of decentralized science and artificial intelligence
Shilina, S. (2025) · Frontiers in Blockchain · doi:10.3389/fbloc.2025.1657050 · Cited by 5
“…incentives, opaque peer review, and inequitable access to resources reveal that legacy models of knowledge production are increasingly un fi t for present needs (Baker, 2016; Begley and Ellis, 2012; Fang and Casadevall, 2016; Ioannidis, 2014; Kitcher, 1993; Prinz et al., 2011; Powell, 2016).”
Interventions to promote medical research integrity: a scoping review
Liu, S., Yang, Y., & Li, M. (2025) · BMC Medical Ethics · doi:10.1186/s12910-025-01275-5 · Cited by 5
“Begley and Ellis [16] pointed out that in cancer research, only 11% of the research results can be repeated by other researchers.”
Jansen, J. A. W., van den Brand, M. A. M., de Vegt, F., & Swinkels, D. W. (2025) · Biopreservation and Biobanking · doi:10.1177/19475535251378202 · Cited by 6
“2,3 More recently, the Center for Open Science con fi rmed these fi ndings, with replication success rates of only 46% for results from high-impact articles.”
Hamilton, D. G., McKenzie, J. E., Nguyen, P., Rethlefsen, M. L., McDonald, S. J., Brennan, S. E., Fidler, F., Higgins, J. P. T., Kanukula, R., Karunananthan, S., et al. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2025.10064 · Cited by 8
“For the sensitivity analyses that removed potentially unreliable hazard ratios from one meta-analysis and included estimated summary statistics for another (Supplementary Figures 12 and 13), the number of meta-analyses based on the blinded data that were classified as not fully replicable did not change.”
InfluentialComparative Study of Reproducibility of Ranked Set Sampling Methods using Predictive Inference
Rehman, S. A., Coolen‐Maturi, T., Coolen, F. P. A., & Shabbir, J. (2026) · Journal of Statistical Theory and Practice · doi:10.1007/s42519-026-00555-6 · Cited by 4
A novel humanized immune stroma PDX cancer model for therapeutic studies
Yang, D., Beddows, I., Tang, H., Bai, S., Cascio, S., McGonigal, S. C., Johnson, B. K., Powers, J. J., Acharya, R., Bao, R., et al. (2026) · Cancer Immunology Immunotherapy · doi:10.1007/s00262-026-04349-4 · Cited by 8
Metadata
- Authors
- Begley, C. G., & Ellis, L. M.
- Affiliations
- Amgen (United States) (US)
- The University of Texas MD Anderson Cancer Center (US)
- Corresponding author
- Ellis, Lee M.
- Venue
- Nature
- Published
- 2012-03-28
- Volume
- 483
- Issue
- 7391
- Pages
- 531-533
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W1987777080
- Semantic Scholar
- 1a81d3d1e8a983e0551edf485f4ac1a8cdd7fea4
- Cited by
- 2,979
- References
- 5
- Citations by year
- 2026: 106 · 2025: 116 · 2024: 127 · 2023: 169 · 2022: 179 · 2021: 206 · 2020: 236 · 2019: 255 · 2018: 262 · 2017: 271 · 2016: 283 · 2015: 262 · 2014: 226 · 2013: 133 · 2012: 62
- FWCI
- 210.63
- Type
- journal article
- Language
- en
- Update notices
- correction (2012-05-02)
- Open access
- bronze
- doi.org
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health and Medical Research Impacts
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Health Sciences › Nursing › Issues, ethics and legal aspects › Biomedical Ethics and Regulation
- Keywords
- preclinical cancer research, research standards, publication practices, research incentives, translational research, clinical trials
- MeSH terms
- Animals, Clinical Trials as Topic, Drug Evaluation, Preclinical, Humans, Neoplasms, Research Design, Reproducibility of Results, Survival Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.