Raise standards for preclinical cancer research
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index; 3 of the 9 sources its claims cite were checked for support.
Overview
CoChat Summary
TL;DR
This Nature Comment argues that many landmark preclinical oncology findings are not reproducible and calls for higher standards in methods, reporting, and incentives. It reports that Amgen confirmed only 6 of 53 landmark papers (11%) and cites Bayer’s finding that about 25% of published preclinical studies could be validated.
Key findings
- In Amgen’s ten-year retrospective analysis, scientific findings were confirmed in only 6 of 53 “landmark” preclinical cancer papers (11%); non-reproduced studies often lacked blinding, relied on single experiments such as one Western blot, or presented selected data.
- Bayer HealthCare reported that only about 25% of published preclinical studies could be validated to the point at which projects could continue; published cancer research represented 70% of the studies analysed in that report, with possible overlap with Amgen’s 53 papers.
- Citation data showed non-reproduced articles in journals with impact factor >20 had a mean of 248 citations (range 3–800) versus 231 (range 82–519) for reproduced articles; for impact factor 5–19, non-reproduced articles had a mean of 169 citations (range 6–1,909) versus 13 (range 3–24) for reproduced articles.
- The authors recommend reporting negative data, blinding preclinical experiments, validating reagents, repeating critical experiments, avoiding single cell lines or models, requiring patient-selection biomarkers, and changing academic incentives.
Methods
- This is a Nature Comment/perspective, not a primary experimental study; it synthesizes Amgen’s ten-year retrospective analysis of experiments performed prospectively and cites a Bayer HealthCare report.
- Amgen selected 53 “landmark” papers describing completely new approaches or alternative clinical uses, attempted confirmation, used additional models, contacted original authors, exchanged reagents, and repeated experiments under the authors’ direction, occasionally in the original investigator’s laboratory.
- The analysis reports counts of confirmed findings and citation metrics from Google Scholar, May 2011; the recommendations are proposed changes rather than tested interventions.
Limitations
- Stated: validation attempts may have failed because of technical differences or difficulties despite efforts to ensure otherwise; additional models were used; when findings could not be reproduced, authors were contacted and experiments were repeated under their direction.
- Stated: papers were deliberately selected as “landmark” and described something completely new, so some data might not hold up; Bayer’s analysed studies may overlap with Amgen’s 53 papers.
- Evident: as a Comment, the text provides no detailed systematic protocol, inclusion criteria, statistical methods, or full dataset; the citation analysis is descriptive and limited to two impact-factor strata.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 1 Not valid · 6 Not checked · 12 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Sadly, clinical trials in oncology have the highest failure rate compared with other therapeutic areas.”
What the sources say
- Partly supportedcites [1] Hutchinson, L. & Kirk, R. Nature Rev. Clin. Oncol. 8, 189–190 (2011). sourceExperimental — accuracy not yet measured
The opening passage of our copy (labelled Abstract; page unavailable) says cancer-drug attrition is much higher than in other therapeutic areas, citing 5% of anticancer agents versus 20% of cardiovascular agents reaching licensure. That comparison covers development from preclinical activity through phase III, not failure rates specifically among clinical trials; it also does not explicitly establish a highest-rate ranking. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Although we in the cancer field hoped that this would lead to more effective drugs, historically, our ability to translate cancer research to clinical success has been remarkably low 1 .”
What the sources say
- Supportedcites [1] Hutchinson, L. & Kirk, R. Nature Rev. Clin. Oncol. 8, 189–190 (2011). source
The opening passage (labeled Abstract in our copy; no page given) reports that only 5% of agents with preclinical anticancer activity are licensed after demonstrating sufficient efficacy in phase III testing. It also says cancer drug attrition is much higher than in other therapeutic areas, supporting the claim of historically low translation to clinical success. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In studies for which findings could be reproduced, authors had paid close attention to controls, reagents, investigator bias and describing the complete data set.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For results that could not be reproduced, however, data were not routinely analysed by investigators blinded to the experimental versus control groups.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Investigators frequently presented the results of one experiment, such as a single Western-blot analysis.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
12 more claims
Create a free account to see all 22 claims and every source verdict.
- It may state two separate claims as one. 2
- Probably not a substantial claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“there are other more common sources of unreliability in the literature (41)(42)(43)(44)”
From Misconduct accounts for the majority of retracted scientific publications (2012) · cites this paper as [42]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“many established scientific results in the social (49, 50) and biomedical (51-53) sciences cannot readily be replicated.”
From Misinformation in and about science (2021) · cites this paper as [51]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A landmark study conducted by Glenn Begley and his team in 2012 concluded that most cancer research studies were not reproducible, by alarmingly reporting that 47 of 53 published findings could not be replicated [13].”
From Guidelines for the use of flow cytometry and cell sorting in immunological studies (third edition) (2021) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In cell biology, two industrial laboratories reported success replicating the original results of landmark findings in only 11% and 25% of the attempted cases ( 10, 11 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Concerns about reproducibility are widespread across disciplines ( 921 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [10]
Experimental — accuracy not yet measured
13 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888
Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010271 · Cited by 805
High drug attrition rates—where are we going wrong?
Hutchinson, L., & Kirk, R. (2011) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2011.34 · Cited by 517
Drug development and clinical trials—the path to an approved cancer drug
Rubin, E. H., & Gilliland, D. G. (2012) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2012.22 · Cited by 135
Raising the bar for cancer therapy models
Francia, G., & Kerbel, R. S. (2010) · Nature Biotechnology · doi:10.1038/nbt0610-561 · Cited by 54
Cited by
Items 1,901–2,000 of 2,889
Branco, A. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4hkgqvka3hi7 · Cited by 3
“…a number of factors, including the realization that for some published results, their replication is not being obtained (e.g. Prinz et al. (2011), Begley and Ellis (2012)); that there may be problems with the commonly accepted reviewing procedures, even besides their possible lack of quality,…”
BackgroundJanero, D. R. (2021) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2021.1893690 · Cited by 3
Decatastrophizing research irreproducibility
Jarvis, M. F. (2024) · Biochemical Pharmacology · doi:10.1016/j.bcp.2024.116090 · Cited by 3
Connecting the dots in neuroscience research: The future of evidence synthesis
Hair, K., Arroyo-Araujo, M., Vojvodić, S., Economou, M., Wong, C., Tinsdeall, F., Smith, S., Rackoll, T., Sena, E. S., & McCann, S. (2024) · Experimental Neurology · doi:10.1016/j.expneurol.2024.115047 · Cited by 5
When to adjust for multiplicity in cancer clinical trials
Unger, J. M., Mazza, G. L., Elsaid, M. I., Duan, F., Dressler, E. V. M., Snavely, A. C., Enserro, D. M., & Pugh, S. L. (2025) · JNCI Monographs · doi:10.1093/jncimonographs/lgae051 · Cited by 4
Parametric Predictive Bootstrap Method for the Reproducibility of Hypothesis Tests
Aldawsari, A. M. A., Coolen‐Maturi, T., & Coolen, F. P. A. (2025) · Journal of Statistical Theory and Practice · doi:10.1007/s42519-025-00438-2 · Cited by 4
Becker, J. O., Shijo, S., Huynh, H., Forrest, K., MacCoss, M. J., Emrick, M. A., Goonatilleke, E., & Hoofnagle, A. N. (2025) · Journal of Mass Spectrometry and Advances in the Clinical Lab · doi:10.1016/j.jmsacl.2025.04.002 · Cited by 3
“Biomedical science has long suffered from a lack of reproducibility between laboratories [41].”
Smith, N. J., McGuire, H. M., & Groth, B. F. D. S. (2025) · Cytometry Part A · doi:10.1002/cyto.a.24938 · Cited by 4
Duan, Y., Jin, Y., Jia, Y., Zhao, M., Bai, L., Zhu, S., Wei, Z., Zhang, S., Yang, B., & Li, Y. (2025) · Biomaterials · doi:10.1016/j.biomaterials.2025.123723 · Cited by 5
Are Impact Factors corrupting truth and utility in biomedical research?
Suhrbier, A., & Poland, G. A. (2013) · Vaccine · doi:10.1016/j.vaccine.2013.09.076 · Cited by 3
Systems Biology in Human Health and Disease
Orešič, M. (2013) · Springer eBooks · doi:10.1007/978-3-319-01008-3_2 · Cited by 3
“, the ‘other networks’ cannot be controlled for, it is thus not surprising that reproducibility of conclusions from molecular biology experiments is strikingly poor (Begley and Ellis 2012).”
BackgroundNeglecting the null: the pitfalls of underreporting negative results in preclinical research
Foster, W., & Putos, S. (2014) · University of Ottawa Journal of Medicine · doi:10.18192/uojm.v4i1.1036 · Cited by 5
“Currently, no similar initiatives exist for addressing positive reporting bias in preclinical research despite mounting evidence and calls to remedy the problem [13-16].”
BackgroundLa reproducibilidad de las investigaciones biomédicas: Quo vadis?
Dal‐Ré, R., Bernad, A., & Garesse, R. (2016) · Medicina Clínica · doi:10.1016/j.medcli.2015.11.029 · Cited by 4
Sample Size Calculation of RNA-sequencing Experiment-A Simulation-Based Approach of TCGA Data
Li, D. S. C.-I. (2014) · Journal of Biometrics & Biostatistics · doi:10.4172/2155-6180.1000198 · Cited by 2
“Among fiftythree cancer papers that were published in high-impacting journals and known as "landmark" studies, only six of them were reproducible [5,6].”
BackgroundPublicly questioned papers more likely to be retracted
Noorden, R. V. (2014) · Nature · doi:10.1038/nature.2014.14979 · Cited by 3
Biosynthonics: Charting the Future Role of Biocatalysis and Metabolic Engineering in Drug Discovery
Yadav, V. G. (2014) · Industrial & Engineering Chemistry Research · doi:10.1021/ie500329d · Cited by 3
Fulmer, T. (2012) · Science-Business eXchange · doi:10.1038/scibx.2012.888 · Cited by 3
Matched preclinical designs for improved translatability
Aittokallio, T. A., Scherer, A., POUTANEN, M. H., & Freedman, L. P. (2017) · Science Translational Medicine · doi:10.1126/scitranslmed.aal4101 · Cited by 3
The Use of Pediatric Patient-Derived Xenografts for Identifying Novel Agents and Combinations
Kurmasheva, R. T., & Houghton, P. J. (2017) · Molecular and translational medicine · doi:10.1007/978-3-319-57424-0_11 · Cited by 5
Validierung von molekularen Simulationen: eine Übersicht verschiedener Aspekte
van Gunsteren, W. F., Daura, X., Hansen, N., Mark, A. E., Oostenbrink, C., Riniker, S., & Smith, L. J. (2017) · Angewandte Chemie · doi:10.1002/ange.201702945 · Cited by 3
PM2.5 and ozone, indicators of air quality, and acute deaths in California, 2004–2007
You, C., Lin, D. K. J., & Young, S. S. (2018) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2018.05.012 · Cited by 7
“It is becoming clear that many scientific claims are failing to replicate (Begley and Ellis, 2012; Ioannidis, 2005; Young and Karr, 2011).”
BackgroundWang, M., Li, P., Wan, R., & Liu, X. (2019) · Biomedicine & Pharmacotherapy · doi:10.1016/j.biopha.2019.01.051 · Cited by 3
Critical Reagents: Evolution of WRIB Recommendations on Critical Reagents
Amaravadi, L. S. (2021) · Bioanalysis · doi:10.4155/bio-2021-0066 · Cited by 3
Scientific Misconduct and Fraud
Herbst, M. (2021) · The European Legacy · doi:10.1080/10848770.2021.1923247 · Cited by 3
The Tangled Knots of Neuroscientific Experimentation
Frisch, S. (2021) · Integrative Psychological and Behavioral Science · doi:10.1007/s12124-021-09617-9 · Cited by 3
Jurišica, I. (2022) · Springer eBooks · doi:10.1007/978-3-031-00119-2_13 · Cited by 3
Feger, S. S., Pertiwi, C., & Bonaiuti, E. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555213 · Cited by 5
“This is understandable as the traditional academic reputation economy is heavily focused on novel work, rather than replications [5, 20, 29].”
BackgroundPreclinical target validation for non-addictive therapeutics development for pain
Hargreaves, R. J., Akinsanya, K., Ajit, S. K., Dhruv, N. T., Driscoll, J., Farina, P., Gavva, N. R., Gill, M., Houghton, A. K., Iyengar, S., et al. (2022) · Expert Opinion on Therapeutic Targets · doi:10.1080/14728222.2022.2147063 · Cited by 3
“Amgen successfully replicated key data from only 6 out of 53 papers recognized as ‘landmark’ preclinical oncology studies – a reproducibility rate of only 11% [12].”
MethodRevisiting the left ear advantage for phonetic cues to talker identification
Drown, L., Philip, B., Francis, A. L., & Theodore, R. M. (2022) · The Journal of the Acoustical Society of America · doi:10.1121/10.0015093 · Cited by 3
Between therapy effect and false-positive result in animal experimentation
Sosnowski, P., Sass, P., Stanisławska‐Sachadyn, A., Krzemiński, M., & Sachadyn, P. (2023) · Biomedicine & Pharmacotherapy · doi:10.1016/j.biopha.2023.114317 · Cited by 4
BASIC RESEARCH IN SHOCK AND SEPSIS
Remick, D. G., Szabó, A., Juffermans, N. P., & Osuchowski, M. F. (2022) · Shock · doi:10.1097/shk.0000000000001953 · Cited by 4
“Begley (7) also queried his colleagues at Amgen about their ability to reproduce landmark studies from the literature, and the results were even more alarming: only 11% of the studies could be replicated in the laboratories.”
BackgroundFake Research: How Can We Recognise it and Respond to it?
CARRIER, M. J. (2023) · International Studies in the Philosophy of Science · doi:10.1080/02698595.2023.2187206 · Cited by 5
“Anecdotal evidence suggests that less than one half of the published biomedical e ff ects can be con fi rmed and reproduced (Begley and Ellis 2012; Harris 2017, 9 – 11; Errington et al. 2021, 5).”
BackgroundMartin, A. K. (2023) · Springer eBooks · doi:10.1007/978-3-031-25078-1_6 · Cited by 4
Buckley, J., Araujo, J. A., Aribilola, I., Arshad, I., Azeem, M., Buckley, C., Fagan, A., Fitzpatrick, D. P., Herrera, D. A. G., Hyland, T., et al. (2023) · International Journal of Technology and Design Education · doi:10.1007/s10798-023-09827-9 · Cited by 7
“…as a “reproducibility crisis” (e.g., Baker, 2016), has occurred in several fields such as psychology (Open Science Collaboration, 2015), cancer biology (Begley & Ellis, 2012; Errington et al., 2021), experimental economics (Camerer et al., 2016), and experimental philosophy (Cova et al., 2021),…”
A replication study on the intuitiveness of programming language syntax
Lappi, V., Tirronen, V., & Itkonen, J. (2023) · Software Quality Journal · doi:10.1007/s11219-023-09631-7 · Cited by 8
“Despite being such a fundamental part of empirical science, in many fields of research, repetitions are rarely attempted (Begley & Ellis, 2012; Makel & Plucker, 2014; Stefik & Hanenberg, 2014; Aarts et al., 2015; Brown et al., 2018; Hao et al., 2018).”
BackgroundWe must improve conditions and options for Australian ECRs
Christian, K., Larkins, J., & Doran, M. R. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01621-w · Cited by 3
Rewiring Drug Research and Development through Human Data-Driven Discovery (HD3)
Jackson, D., Racz, R., Kim, S., Brock, S., & Burkhart, K. (2023) · Pharmaceutics · doi:10.3390/pharmaceutics15061673 · Cited by 4
On a Scientific Approach for Deep Time Investigations
Harrison, T. M. (2023) · Perspectives of Earth and Space Scientists · doi:10.1029/2022cn000193 · Cited by 4
“…half, and often much less than half, of all psychology, cancer, and pharmaceutical research studies can be replicated (e.g., Aarts et al., 2015; Begley & Ellis, 2012; Ioannidis, 2005; Prinz et al., 2011) with human fallibility often a significant contributing cause (Collins & Tabak, 2014;…”
BackgroundBuilding a rheumatology biobank for reliable basic/translational research and precision medicine
Assirelli, E., Naldi, S., Brusi, V., et al. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1228874 · Cited by 3
“Even though the outcomes of preclinical research cannot be replicated, many secondary papers are nonetheless created (50).”
Lyons‐Abbott, S., Abramov, A., Chan, C., Deer, J. R., Fu, G., Hassouneh, W., Koch, T., Misquith, A., O’Neill, J. W., Simon, S. A., et al. (2023) · Biotechnology Journal · doi:10.1002/biot.202300162 · Cited by 3
Zhai, Z., Kong, F., Zhu, Z., Dai, J., Cai, J., Xie, D., Shen, Y., Xu, Y., & Sun, T. (2023) · American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2023.11.011 · Cited by 5
Specifics of creating clinical abstract of biospecimens
Kopylova, O. V., Ershova, A. I., Borisova, A. L., Metelskaya, V. A., & Drapkina, O. M. (2023) · CARDIOVASCULAR THERAPY AND PREVENTION · doi:10.15829/1728-8800-2023-3855 · Cited by 4
Hamilton, D. G., Page, M. J., Everitt, S., Fraser, H. S., & Fidler, F. M. (2024) · Accountability in Research · doi:10.1080/08989621.2024.2308606 · Cited by 5
“…funds into cancer research worldwide, significant concerns about the reliability of published findings have been raised following the repeated inability to replicate the key results of promising pre-clinical research (Begley and Lee 2012; Errington et al. 2021; Prinz, Schlange, and Asadullah 2011).”
InfluentialBode, G., Starck-Lantova, P., & Kramer, P.-J. (2023) · Springer eBooks · doi:10.1007/978-3-030-73317-9_142-1 · Cited by 3
Li, Y., Zhou, X., Chen, R., Zhang, X., & Cao, H. (2024) · PLoS Genetics · doi:10.1371/journal.pgen.1011423 · Cited by 6
Ameen‐Ali, K. E., & Allen, C. E. (2024) · CABI eBooks · doi:10.1079/9781800629431.0008 · Cited by 4
Franzen, D., Salholz‐Hillel, M., Müller-Ohlraun, S., & Strech, D. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02482-9 · Cited by 6
“The inability to reproduce scientific findings [1–4] and the non-or incomplete reporting of research results [5–9] have raised concerns about research waste, and highlighted the need to promote responsible research practices as a means to increase research value.”
Provide proactive reproducible analysis transparency with every publication
Meijer, P., Howard, N., Liang, J., Kelsey, A., Subramanian, S., Johnson, E., Mariz, P., Harvey, J. F., Ambrose, M., Tereshchenko, V., et al. (2025) · Royal Society Open Science · doi:10.1098/rsos.241936 · Cited by 11
“Although HISE is specifically designed for immunology and omics research, we contend that analogous platforms can be developed for other scientific disciplines by following these principles while addressing the unique requirements of these fields.”
Introducing a checklist for manuscript submission to Pharmacological Research
Cho, K., Clementi, E., Levi‐Schaffer, F., Smalley, K. S. M., Radice, S., & Watts, S. W. (2015) · Pharmacological Research · doi:10.1016/j.phrs.2015.10.017 · Cited by 3
Network Analysis of Cancer-focused Association Network Reveals Distinct Network Association Patterns
Zhang, Y., & Tao, C. (2014) · Cancer Informatics · doi:10.4137/cin.s14033 · Cited by 3
Bryon, E. (2017) · Connection Science · doi:10.1080/09540091.2016.1273880 · Cited by 5
“In a similar investigation, Begley and Ellis (2012) reported a meager 11% replication rate.”
BackgroundFesting, M. F. W. (2018) · Neuromethods · doi:10.1007/978-1-4939-8564-7_13 · Cited by 3
Patel, N. (2019) · Journal of Neurological Research And Therapy · doi:10.14302/issn.2470-5020.jnrt-19-2618 · Cited by 3
Dynamic integration of manual and automated biological process skills into MES
Brecher, C., Malik, A., Blanke, P., & Herfs, W. (2019) · IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) · doi:10.1109/etfa.2019.8869389 · Cited by 5
“According to a report from 2012, it was only possible to reproduce 11% of the results from a total of 53 studies [11].”
ResultDeinzer, G., & Herb, U. (2020) · Zeitschrift für Bibliothekswesen und Bibliographie · doi:10.3196/186429502067147 · Cited by 6
Some Personal Advice Concerning How to Write Precise, Concise and Eloquent Research Articles
Holmberg, H., & Sperlich, B. (2021) · Frontiers in Sports and Active Living · doi:10.3389/fspor.2021.648929 · Cited by 4
“Be aware that much published research in the life sciences has proven to be extremely difficult to reproduce (Begley and Ellis, 2012; Baker, 2016).”
BackgroundOn the roles of reproducibility, ethics, and statistical modeling in medical research
deKemp, R. A., Juneau, D., Wu, K. Y., Wells, G. A., Ruddy, T. D., & Beanlands, R. S. (2021) · Journal of Nuclear Cardiology · doi:10.1007/s12350-021-02592-z · Cited by 3
Validation of Isoform- and Mutation-Specific RAS Antibodies
Waters, A. M., & Der, C. J. (2021) · Methods in molecular biology · doi:10.1007/978-1-0716-1190-6_5 · Cited by 3
Summit: Automated Analysis of Arrayed Single-Cell Gel Electrophoresis
Vlassakis, J., Yamauchi, K. A., & Herr, A. E. (2021) · SLAS TECHNOLOGY · doi:10.1177/24726303211036869 · Cited by 3
Science in Transition How Science Goes Wrong and What to Do About It
Miedema, F. F. (2021) · Springer eBooks · doi:10.1007/978-94-024-2115-6_3 · Cited by 4
Rix, A., Girbig, R. M., Porte, C., Lederle, W., Leenaars, C. H. C., & Kiessling, F. M. (2021) · Molecular Imaging and Biology · doi:10.1007/s11307-021-01686-x · Cited by 3
Myers, P. O. (2015) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2015.08.041 · Cited by 3
Connecting the dots in translational research
Kahn, J. (2012) · Nature Reviews Drug Discovery · doi:10.1038/nrd3357-c2 · Cited by 3
Carrell, D. T., & Meyts, E. R. (2013) · Andrology · doi:10.1111/j.2047-2927.2013.00149.x · Cited by 3
“For example, Begley recently reported that in a serious effort to reproduce results from 53 ‘landmark papers’ in the field of cancer biology, only six studies could be validated (Begley & Ellis, 2012).”
BackgroundOpening up drug development to everyone
Adelman, B. (2013) · Hematology · doi:10.1182/asheducation-2013.1.311 · Cited by 3
Quantifying the probability of clinical trial success from scientific articles
Joshi, V., & Milletti, F. (2014) · Drug Discovery Today · doi:10.1016/j.drudis.2014.06.013 · Cited by 3
Randomization and Statistical Power: Paramount in Trial Reproducibility (Even for Rare Cancers)
Baker, L. H., Crowley, J. J., & Maki, R. G. (2012) · The Oncologist · doi:10.1634/theoncologist.2012-0209 · Cited by 3
“In a fascinating study of preclinical cancer research, Begley and Ellis reported in a commentary that only 6 of 53 publications cited as landmark publications could be replicated in an independent laboratory [3].”
BackgroundFair treatment comparisons in observational research
Lopiano, K. K., Obenchain, R. L., & Young, S. S. (2014) · Statistical Analysis and Data Mining The ASA Data Science Journal · doi:10.1002/sam.11235 · Cited by 4
“We note in passing that designed studies can also have severe reproducibility problems [28].”
BackgroundJohnston, I. G. (2015) · Mitochondrial DNA Part A · doi:10.3109/19401736.2015.1022732 · Cited by 3
Bad Science: Cause and Consequence
Elliott, S. (2016) · Journal of Pharmaceutical Sciences · doi:10.1016/j.xphs.2016.01.002 · Cited by 5
Are Medical Articles True on Health, Disease?
Young, S. S., & Miller, H. I. (2014) · Genetic Engineering & Biotechnology News · doi:10.1089/gen.34.09.03 · Cited by 3
Introduction to Nonclinical Statistics for Pharmaceutical and Biotechnology Industries
Zhang, L., & Su, C. (2016) · Statistics for biology and health · doi:10.1007/978-3-319-23558-5_1 · Cited by 3
How To Be a Good Nonclinical Statistician
Pikounis, B., & Bijnens, L. (2016) · Statistics for biology and health · doi:10.1007/978-3-319-23558-5_3 · Cited by 3
Randomness and the Games of Science
Goeman, J. J. (2016) · The frontiers collection · doi:10.1007/978-3-319-26300-7_5 · Cited by 5
“More dramatic was 1The Ig Nobel Prizes honor scientific achievements that make people laugh, and then think. the experience reported by Begley and Ellis (2012), scientists working at Amgen, a pharmaceutical company in California, who tried to replicate the results of many ‘landmark’ papers…”
BackgroundBiofocussed chemoprospecting: An efficient approach for drug discovery
Thakkar, B. S., Albrigtsen, M., Svendsen, J. S. M., Andersen, J. H., & Engh, R. A. (2017) · Chemical Biology & Drug Design · doi:10.1111/cbdd.12934 · Cited by 3
Structural Chemogenomics Databases to Navigate Protein–Ligand Interaction Space
Kanev, G. K., Kooistra, A. J., de Esch, I. J. P., & de Graaf, C. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-409547-2.12298-x · Cited by 3
A Modelling Ecosystem for Prognostics
Astfalck, L., Hodkiewicz, M. R., KEATING, A. J., Cripps, E., & PECHT, M. G. (2016) · Annual Conference of the PHM Society · doi:10.36001/phmconf.2016.v8i1.2568 · Cited by 3
“The biotechnology firm Amgen was only able to confirm the scientific findings of 6 out of 53 papers (Begley & Ellis, 2012), and pharmaceutical company Bayer HealthCare reported that only 20%-25% of 67 studies aligned with their in-house findings (Prinz, Schlange, & Asadullah, 2011).”
BackgroundKumar, G., Chaudhary, K. K., Misra, K., & Tripathi, A. (2017) · International Journal of Pharmacology · doi:10.3923/ijp.2017.709.723 · Cited by 3
Chapter 1: Errors in Statistical Decision Making
Garland-Campbell, K. A., Glaz, B., & Yeater, K. M. (2018) · ASSA, CSSA and SSSA · doi:10.2134/appliedstatistics.2016.0007 · Cited by 14
Benveniste’s Experiments Explained by a Non-Conventional Experimenter Effect
Beauvais, F. (2018) · Medicines · doi:10.3390/medicines5020028 · Cited by 3
“Another possible application of the present approach is the current debate about reproducibility in biology, medicine, oncology and psychology [41-44].”
BackgroundA Statistical Model to Investigate the Reproducibility Rate Based on Replication Experiments
Pauli, F. (2018) · International Statistical Review · doi:10.1111/insr.12273 · Cited by 3
“However, it is reasonable to see lack of reproducibility as an indicator of reliability [if not a proxy for falsity (Prinz et al., 2011; Begley & Ellis, 2012)] as is discussed more thoroughly in Section 3.”
BackgroundMűködési zavarok és megújulási törekvések a tudományos pszichológiában, a 21. század elején
Szokolszky, Á. (2018) · Magyar Pszichológiai Szemle · doi:10.1556/0016.2018.73.4.3 · Cited by 3
Aguado, T., Romero‐Revilla, J. A., Granados, R., Campuzano, S., Torrente-Rodríguez, R. M., Cuesta, Á. M., Albiñana, V., Botella, L. M., Santamaría, S. S., García‐Sanz, J. A., et al. (2019) · Scientific Reports · doi:10.1038/s41598-019-48359-y · Cited by 4
Forschen für die Tonne? – Die Research-Waste-Problematik
Hachmann, J. (2019) · Public Health Forum · doi:10.1515/pubhef-2019-0037 · Cited by 4
From One Environment to Many: The Problem of Replicability of Statistical Inferences
Higgins, J. J., Higgins, M., & Lin, J. (2020) · The American Statistician · doi:10.1080/00031305.2020.1829047 · Cited by 2
“…scientific studies are unable to be verified in follow-up studies—was dramatically brought to the forefront when researchers at the biotech company Amgen attempted to replicate 53 well-regarded preclinical studies only to find that just 6 supported the original conclusions (Begley and Ellis 2012).”
BackgroundLes pratiques de recherche ouvertes en psychologie
Lantian, A. (2020) · Psychologie Française · doi:10.1016/j.psfr.2020.09.001 · Cited by 6
Welty, L. J., Rasmussen, L. V., Baldridge, A. S., & Whitley, E. W. (2020) · JAMIA Open · doi:10.1093/jamiaopen/ooaa043 · Cited by 3
Bittlinger, M., Schwietering, J., & Strech, D. (2021) · Drug Discovery Today · doi:10.1016/j.drudis.2021.06.007 · Cited by 3
Data publication consensus and controversies
Kratz, J. E., & Strasser, C. A. (2014) · F1000Research · doi:10.12688/f1000research.4264 · Cited by 3
Interrogating the Cancer Genome to Deliver More Precise Cancer Care
Mateo, J., & Bono, J. S. D. (2016) · American Society of Clinical Oncology Educational Book · doi:10.1200/edbk_156908 · Cited by 3
Ridiculously good writing: How to write like a pro and publish like a boss
Modesitt, S. C., Havrilesky, L. J., Previs, R. A., Rauh‐Hain, J. A., Straughn, J. M., Bakkum-Gamez, J. N., Fuh, K. C., & Cohn, D. E. (2022) · Gynecologic Oncology Reports · doi:10.1016/j.gore.2022.101024 · Cited by 3
“Ultimately, the entire data set should be represented (and/or made available) in the final publication (Begley and Ellis, 2012).”
BackgroundEggert, S., Kahl, M., Kent, R., Gaats, L., Bock, N., Meinert, C., & Hutmacher, D. W. (2022) · Journal of Visualized Experiments · doi:10.3791/61261-v · Cited by 3
Biochemistry—Not Oncogenes—May Demystify and Defeat Cancer
Kulsh, J. (2023) · Oncology and Therapy · doi:10.1007/s40487-023-00221-y · Cited by 3
Cancer research and innovation: conceptualising a persistent anomaly
Xu, Y., & Gagliardi, D. (2023) · Innovation · doi:10.1080/14479338.2023.2204322 · Cited by 3
“…technology are converging 6 towards an increasingly symbiotic relationship (Cech, 2005; Coccia, 2014; Narin & Noma, 1985) that is marked by non-trivial investments in translational research (Begley & Ellis, 2012; Khoury et al., 2011; Lee et al., 2021; Prinz et al., 2011; Seyhan, 2019; Woolf, 2008).”
Epigenetics of Inflammatory Bowel Disease
Häsler, R., Schreiber, S., Beck, S., & Rosenstiel, P. C. (2013) · Springer eBooks · doi:10.1007/978-1-4614-8256-7_9 · Cited by 4
Adaptive Filtering Procedures for Replicability Analysis of High-throughput Experiments
Wang, J., Su, W., Sabatti, C., & Owen, A. B. (2016) · Cited by 3
Griffioen, G., Matheson, G. J., Červenka, S., Farde, L., & Borg, J. (2018) · PeerJ · doi:10.7717/peerj.5790 · Cited by 3
Breakthrough Articles: Putting science first
Corey, D. R., Wise, J. A., Fox, K. R., & Stoddard, B. L. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku853 · Cited by 3
Resource Reallocation and Innovation: Converting Enterprise Risks into Opportunities
Dutz, M. A. (2013) · World Bank, Washington, DC eBooks · doi:10.1596/1813-9450-6534 · Cited by 3
“As an illustration, researchers at a US-based human therapeutics company were able to confirm the results of only six of 53 ‘landmark studies’ in preclinical cancer research (Begley and Ellis, 2012).”
Challenges and Options for Drug Discovery
Kramer, W. (2016) · Wiley eBooks · doi:10.1002/9783527693405.ch6 · Cited by 4
Metadata
- Authors
- Begley, C. G., & Ellis, L. M.
- Affiliations
- Amgen (United States) (US)
- The University of Texas MD Anderson Cancer Center (US)
- Corresponding author
- Ellis, Lee M.
- Venue
- Nature
- Published
- 2012-03-28
- Volume
- 483
- Issue
- 7391
- Pages
- 531-533
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W1987777080
- Semantic Scholar
- 1a81d3d1e8a983e0551edf485f4ac1a8cdd7fea4
- Cited by
- 2,979
- References
- 5
- Citations by year
- 2026: 106 · 2025: 116 · 2024: 127 · 2023: 169 · 2022: 179 · 2021: 206 · 2020: 236 · 2019: 255 · 2018: 262 · 2017: 271 · 2016: 283 · 2015: 262 · 2014: 226 · 2013: 133 · 2012: 62
- FWCI
- 210.63
- Type
- journal article
- Language
- en
- Update notices
- correction (2012-05-02)
- Open access
- bronze
- doi.org
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health and Medical Research Impacts
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Health Sciences › Nursing › Issues, ethics and legal aspects › Biomedical Ethics and Regulation
- Keywords
- preclinical cancer research, research standards, publication practices, research incentives, translational research, clinical trials
- MeSH terms
- Animals, Clinical Trials as Topic, Drug Evaluation, Preclinical, Humans, Neoplasms, Research Design, Reproducibility of Results, Survival Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.