Raise standards for preclinical cancer research
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index; 3 of the 9 sources its claims cite were checked for support.
Overview
CoChat Summary
TL;DR
This Nature Comment argues that many landmark preclinical oncology findings are not reproducible and calls for higher standards in methods, reporting, and incentives. It reports that Amgen confirmed only 6 of 53 landmark papers (11%) and cites Bayer’s finding that about 25% of published preclinical studies could be validated.
Key findings
- In Amgen’s ten-year retrospective analysis, scientific findings were confirmed in only 6 of 53 “landmark” preclinical cancer papers (11%); non-reproduced studies often lacked blinding, relied on single experiments such as one Western blot, or presented selected data.
- Bayer HealthCare reported that only about 25% of published preclinical studies could be validated to the point at which projects could continue; published cancer research represented 70% of the studies analysed in that report, with possible overlap with Amgen’s 53 papers.
- Citation data showed non-reproduced articles in journals with impact factor >20 had a mean of 248 citations (range 3–800) versus 231 (range 82–519) for reproduced articles; for impact factor 5–19, non-reproduced articles had a mean of 169 citations (range 6–1,909) versus 13 (range 3–24) for reproduced articles.
- The authors recommend reporting negative data, blinding preclinical experiments, validating reagents, repeating critical experiments, avoiding single cell lines or models, requiring patient-selection biomarkers, and changing academic incentives.
Methods
- This is a Nature Comment/perspective, not a primary experimental study; it synthesizes Amgen’s ten-year retrospective analysis of experiments performed prospectively and cites a Bayer HealthCare report.
- Amgen selected 53 “landmark” papers describing completely new approaches or alternative clinical uses, attempted confirmation, used additional models, contacted original authors, exchanged reagents, and repeated experiments under the authors’ direction, occasionally in the original investigator’s laboratory.
- The analysis reports counts of confirmed findings and citation metrics from Google Scholar, May 2011; the recommendations are proposed changes rather than tested interventions.
Limitations
- Stated: validation attempts may have failed because of technical differences or difficulties despite efforts to ensure otherwise; additional models were used; when findings could not be reproduced, authors were contacted and experiments were repeated under their direction.
- Stated: papers were deliberately selected as “landmark” and described something completely new, so some data might not hold up; Bayer’s analysed studies may overlap with Amgen’s 53 papers.
- Evident: as a Comment, the text provides no detailed systematic protocol, inclusion criteria, statistical methods, or full dataset; the citation analysis is descriptive and limited to two impact-factor strata.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 1 Not valid · 6 Not checked · 12 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Sadly, clinical trials in oncology have the highest failure rate compared with other therapeutic areas.”
What the sources say
- Partly supportedcites [1] Hutchinson, L. & Kirk, R. Nature Rev. Clin. Oncol. 8, 189–190 (2011). sourceExperimental — accuracy not yet measured
The opening passage of our copy (labelled Abstract; page unavailable) says cancer-drug attrition is much higher than in other therapeutic areas, citing 5% of anticancer agents versus 20% of cardiovascular agents reaching licensure. That comparison covers development from preclinical activity through phase III, not failure rates specifically among clinical trials; it also does not explicitly establish a highest-rate ranking. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Although we in the cancer field hoped that this would lead to more effective drugs, historically, our ability to translate cancer research to clinical success has been remarkably low 1 .”
What the sources say
- Supportedcites [1] Hutchinson, L. & Kirk, R. Nature Rev. Clin. Oncol. 8, 189–190 (2011). source
The opening passage (labeled Abstract in our copy; no page given) reports that only 5% of agents with preclinical anticancer activity are licensed after demonstrating sufficient efficacy in phase III testing. It also says cancer drug attrition is much higher than in other therapeutic areas, supporting the claim of historically low translation to clinical success. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In studies for which findings could be reproduced, authors had paid close attention to controls, reagents, investigator bias and describing the complete data set.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For results that could not be reproduced, however, data were not routinely analysed by investigators blinded to the experimental versus control groups.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Investigators frequently presented the results of one experiment, such as a single Western-blot analysis.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
12 more claims
Create a free account to see all 22 claims and every source verdict.
- It may state two separate claims as one. 2
- Probably not a substantial claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“there are other more common sources of unreliability in the literature (41)(42)(43)(44)”
From Misconduct accounts for the majority of retracted scientific publications (2012) · cites this paper as [42]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“many established scientific results in the social (49, 50) and biomedical (51-53) sciences cannot readily be replicated.”
From Misinformation in and about science (2021) · cites this paper as [51]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A landmark study conducted by Glenn Begley and his team in 2012 concluded that most cancer research studies were not reproducible, by alarmingly reporting that 47 of 53 published findings could not be replicated [13].”
From Guidelines for the use of flow cytometry and cell sorting in immunological studies (third edition) (2021) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In cell biology, two industrial laboratories reported success replicating the original results of landmark findings in only 11% and 25% of the attempted cases ( 10, 11 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Concerns about reproducibility are widespread across disciplines ( 921 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [10]
Experimental — accuracy not yet measured
13 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888
Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010271 · Cited by 805
High drug attrition rates—where are we going wrong?
Hutchinson, L., & Kirk, R. (2011) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2011.34 · Cited by 517
Drug development and clinical trials—the path to an approved cancer drug
Rubin, E. H., & Gilliland, D. G. (2012) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2012.22 · Cited by 135
Raising the bar for cancer therapy models
Francia, G., & Kerbel, R. S. (2010) · Nature Biotechnology · doi:10.1038/nbt0610-561 · Cited by 54
Cited by
Items 1,501–1,600 of 2,889
Predicting Target and Chemical Druggability
Oprea, T. I., & Hasselgren, C. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-409547-2.12342-x · Cited by 12
Building capacity to encourage research reproducibility and #MakeResearchTrue
Rethlefsen, M. L., Lackey, M. J., & Zhao, S. (2018) · Journal of the Medical Library Association JMLA · doi:10.5195/jmla.2018.273 · Cited by 10
Aptamer-facilitated mass cytometry
Mironov, G. G., Bouzekri, A., Watson, J. L., Loboda, O., Ornatsky, O., & Berezovski, M. V. (2018) · Analytical and Bioanalytical Chemistry · doi:10.1007/s00216-018-1011-0 · Cited by 10
Minimum Information in In Vivo Research
Voehringer, P., & Nicholson, J. R. (2019) · Handbook of experimental pharmacology · doi:10.1007/164_2019_285 · Cited by 13
“However, it has been described many times over the last few years that in preclinical research – particularly preclinical animal research – many findings presented in high-profile journals are not reliable and cannot be replicated (Begley and Ellis 2012; Peers et al. 2012; Prinz et al. 2011).”
BackgroundEfficient Modulation of TP53 Expression in Human Induced Pluripotent Stem Cells
Uhlmann, C., Kuhn, L. M., Tigges, J., Fritsche, E., & Kahlert, U. D. (2019) · Current Protocols in Stem Cell Biology · doi:10.1002/cpsc.102 · Cited by 8
Comparing negative emissions and high renewable scenarios for the European power system
Gaffney, F., Deane, P., Drayton, G., Glynn, J. D. M., & Gallachóir, B. Ó. (2020) · BMC Energy · doi:10.1186/s42500-020-00013-4 · Cited by 11
MORPHEUS: An automated tool for unbiased and reproducible cell morphometry
Ruffinatti, F. A., Genova, T., Mussano, F. D., & Munaron, L. (2020) · Journal of Cellular Physiology · doi:10.1002/jcp.29768 · Cited by 8
“…or replication crisis has been largely debated on both generalist and more specialized journals in the last years (An, 2018; Baker, 2016; Begley & Ellis, 2012; Coiera, Ammenwerth, Georgiou, & Magrabi, 2018; Fanelli, 2018; França & Monserrat, 2018; Samsa & Samsa, 2019; Sandve et al.,…”
BackgroundWhy the preclinical imaging field needs nuclear medicine technologists and radiographers?
Dillenseger, J., Choquet, P., Snay, E. R., & Costa, P. F. (2020) · European Journal of Hybrid Imaging · doi:10.1186/s41824-020-00081-z · Cited by 9
“Post safety monitoring from regulatory agencies Preclinical research (step 2) addresses basic questions about safety and efficacy of novel drugs, therapeutic procedures and treatments using animal models (Begley and Ellis 2012).”
BackgroundMicro- and Macroenvironmental Conditions and Stability of Terrestrial Models
Lee, V. K., David, J. M., & Huerkamp, M. J. (2019) · ILAR Journal · doi:10.1093/ilar/ilaa013 · Cited by 12
Enhanced Anti-Amyloid Effect of Combined Leptin and Pioglitazone in APP/PS1 Transgenic Mice
Liu, Y., Hanson, K., McCormack, G. H., Atkinson, R. A. K., Dittmann, J., Vickers, J. C., Fernández-Martos, C. M., & King, A. E. (2021) · Current Alzheimer Research · doi:10.2174/1567205018666210218163857 · Cited by 10
Jones, M. M., & Chataway, J. (2021) · Technology Analysis and Strategic Management · doi:10.1080/09537325.2021.1882673 · Cited by 10
“The incentive structure in academia may also help to explain the disparity between the reproducibility of academic research and that produced by the SGC (Begley and Ellis 2012).”
ResultLigand engineering for theranostic applications
Altmann, A., Kratochwil, C., Giesel, F. L., & Haberkorn, U. (2021) · Current Opinion in Chemical Biology · doi:10.1016/j.cbpa.2021.04.006 · Cited by 9
Lost in translation? Beyond sex as a biological variable in animal research
Pape, M. (2021) · Health Sociology Review · doi:10.1080/14461242.2021.1969981 · Cited by 14
Mear, S. J., Lucas, T., Ahlqvist, G. P., Robey, J. M. S., Dietz, J., Khairnar, P. V., Maity, S., Williams, C. L., Snead, D. R., Nelson, R. C., et al. (2022) · Chemistry - A European Journal · doi:10.1002/chem.202201311 · Cited by 9
Protocols.io: Reducing the knowledge that perishes because we do not publish it
Teytelman, L., & Stoliartchouk, A. (2015) · Information Services & Use · doi:10.3233/isu-150769 · Cited by 8
“In pre-clinical research, Bayer and Amgen reported failure to reproduce published results in 70–90% of the cases [1].”
BackgroundPast, Present, and Future of Statistical Science
Maindonald, J. H. (2015) · International Statistical Review · doi:10.1111/insr.12111_7 · Cited by 9
Abnormal tissue proliferation and life span variability in chronically irradiated dogs
Shoutko, A. N., & Ekimova, L. P. (2013) · Radiation and Environmental Biophysics · doi:10.1007/s00411-013-0504-7 · Cited by 13
“This dearth of concrete evidence provokes increased negativity in the mass media toward the efficacy of conventional cancer therapy (Begley and Ellis 2012).”
BackgroundNeurobiological studies of chronic pain and analgesia: Rationale and refinements
Fairbanks, C. A., & Goracke‐Postle, C. J. (2015) · European Journal of Pharmacology · doi:10.1016/j.ejphar.2015.03.049 · Cited by 9
“Recently, a much discussed publication (Begley and Ellis, 2012) featured the discordance of hematological and oncology models of cancer pain and the limited associated therapeutic development; there has been some discourse suggesting that rodent models of sensory dysfunction suffer similar limitations.”
InfluentialEmerging Requirements for Technology Management: A Sector-based Scenario Planning Approach
Philbin, S. P. (2013) · Journal of technology management & innovation · doi:10.4067/s0718-27242013000400004 · Cited by 9
Translation failure and medical reversal: Two sides to the same coin
Prasad, V. K. (2015) · European Journal of Cancer · doi:10.1016/j.ejca.2015.08.024 · Cited by 10
A collaborative approach to computational reproducibility
Chirigati, F., Capone, R., Rampin, R., Freire, J., & Shasha, D. E. (2016) · Information Systems · doi:10.1016/j.is.2016.03.002 · Cited by 13
“Third, studies indicate that reproducibility increases impact, visibility, and research quality [2, 1, 9, 10, 11, 13, 15], and helps defeat self-deception [12].”
BackgroundRusso, F., Righelli, D., & Angelini, C. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-44332-4_19 · Cited by 12
“More and more papers [3–10] are reporting the impossibility to reproduce”
BackgroundMeier, B. P., Fetterman, A. K., & Robinson, M. D. (2015) · Cited by 9
How to understand and teach P values: a diagnostic test framework
Baduashvili, A., Evans, A. T., & Cutler, T. S. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.03.003 · Cited by 9
Poikilosis – pervasive biological variation
Vihinen, M. (2020) · F1000Research · doi:10.12688/f1000research.24173.1 · Cited by 9
BEING PROFILED: COGITAS ERGO SUM: 10 Years of Profiling the European Citizen
Bayamiloglu, E., Baraliuc, I., Janssens, L., & Hildebrandt, M. (2018) · Amsterdam University Press eBooks · doi:10.2307/j.ctvhrd092 · Cited by 24
Grahe, J. E., & Cuccolo, K. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009000796.014 · Cited by 9
Bridging the gap between basic and applied biology: towards preclinical translation
Cagan, R. L., Justice, M. J., & Tidmarsh, G. F. (2013) · Disease Models & Mechanisms · doi:10.1242/dmm.012450 · Cited by 9
“Recently, two major pharmaceutical companies, Bayer and Amgen, reported on the lack of reproducibility of published preclinical research (Begley and Ellis, 2012; Prinz et al., 2011).”
BackgroundMade in Academia: The Effect of Institutional Origin on Inventors’ Attention to SciencePreprint
Bikard, M. A. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2333413 · Cited by 9
The history, landscape, and outlook of human cell line authentication and security
Harbut, E., Makris, Y., Pertsemlidis, A., & Bleris, L. (2024) · SLAS Discovery · doi:10.1016/j.slasd.2024.100194 · Cited by 8
Types and progress of clinical trial design for breast cancer: a narrative review
Zhou, T., & Zhang, J. (2023) · Translational Breast Cancer Research · doi:10.21037/tbcr-23-22 · Cited by 9
“While traditional phase I–IV trials laid the groundwork, their limitations fuelled the emergence of more efficient designs (6).”
Replication of null results: Absence of evidence or evidence of absence?
Pawel, S., Heyard, R., Micheloud, C., & Held, L. (2023) · eLife · doi:10.7554/elife.92311 · Cited by 25
“…various large-scale replication projects have been conducted in diverse fields, from the biomedical to the social sciences ( Prinz et al., 2011 ; Begley and Ellis, 2012 ; Klein et al., 2014 ; Open Science Collaboration, 2015 ; Camerer et al., 2016 ; Camerer et al., 2018 ; Klein et al., 2018 ;…”
BackgroundTowards replicability and sustainability in cancer research
Grimes, D. R. (2024) · BJC Reports · doi:10.1038/s44276-024-00090-6 · Cited by 9
“A systematic replication trial as early as 2012 of what were deemed landmark cancer biology experiments exposed an alarming fi nding [9] – that only 6 of the 53 experiments, approximately 11% those analysed, had replicable results.”
BackgroundSkubera, M., Korbmacher, M., Evans, T. R., Azevedo, F., & Pennington, C. R. (2025) · Royal Society Open Science · doi:10.1098/rsos.241726 · Cited by 18
“1 Despite similar concerns being debated passionately in the 1960s (see [6]), and not being exclusive to psychology [7,8], a series of landmark events in the 2010s has led to fast-paced action aiming to reform this discipline (see [3]).”
Protocols and tools to enable reproducibility in 2D materials research
Bøggild, P., Booth, T. J., Jessen, B. S., Shivayogimath, A., Lassaline, N., Hofmann, S., Burton, O. J., Daasbjerg, K., Smith, A., Nørgaard, K., et al. (2025) · Nature Reviews Physics · doi:10.1038/s42254-025-00875-9 · Cited by 14
“A 2012 Nature study attempted to replicate 53 landmark cancer papers and succeeded in only 6 cases 1 , and a recent follow-up study showed similar results 2 .”
Halász, G., Schmahl, J., Negron, N., Ni, M., Lim, W. K., Atwal, G. S., Bai, Y., & Glass, D. J. (2025) · Nature Communications · doi:10.1038/s41467-025-65022-5 · Cited by 20
Arkas: Rapid reproducible RNAseq analysis
Colombo, A., Triche, T. J., Jr., & Ramsingh, G. (2017) · F1000Research · doi:10.12688/f1000research.11355.2 · Cited by 8
“Re-creating a clinical study has many challenges, for example the difficult nature of a disease, the complexity of cell-line models in mouse and human that attempt to capture human tumor environment, and limited power through small enrollments in clinical trials 19 .”
Casadevall, A., & Fang, F. C. (2014) · Microbe Magazine · doi:10.1128/microbe.9.390.1 · Cited by 8
State of FAIRness in ESFRI Projects
Wittenburg, P., de Jong, F. M. G., Uytvanck, D. V., Cocco, M., Jeffery, K., Lautenschlager, M., Thiemann, H., Hellström, M., Asmi, A., & Holub, P. (2019) · Data Intelligence · doi:10.1162/dint_a_00045 · Cited by 10
Tran, V. P., Kim, R., Maertens, M., Härtung, T., & Maertens, A. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.674370 · Cited by 16
“In 2012, Amgen researchers attempted to replicate 53 landmark cancer papers and found that 47 studies were not reproducible (Begley and Ellis, 2012); the result is in keeping with a broader estimate that most research studies are likely to be not reproducible (Ioannidis, 2005).”
George, R. P., Semendric, I., Bowley-Schubert, E. R., Chivonivoni, C. T., Warrender, A. P., & Whittaker, A. L. (2021) · Supportive Care in Cancer · doi:10.1007/s00520-021-06312-8 · Cited by 8
Johnston, I. G., Slater, M. W., & Cazier, J. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.826951 · Cited by 10
“…reproducibility crisis across scientific disciplines (Collaboration et al., 2015) including drug discovery for cancer, where only 11% of pre-clinical research results were found to be reproducible ( Begley and Ellis, 2012), and Phase II success rates had fallen to 18% in 2011 (Prinz et al., 2011).”
BackgroundMaschinelles Lernen zur Vorhersage chemischer Reaktivität: Die Bedeutung “gescheiterter” Experimente
Strieth‐Kalthoff, F., Sandfort, F., Kühnemund, M., Schäfer, F. R., Kuchen, H., & Glorius, F. (2022) · Angewandte Chemie · doi:10.1002/ange.202204647 · Cited by 9
Meta-analysis on reporting practices as a source of heterogeneity in in vitro cancer research
Sander, T., Ghanawi, J., Wilson, E., Muhammad, S., Macleod, M. R., & Kahlert, U. D. (2022) · BMJ Open Science · doi:10.1136/bmjos-2021-100272 · Cited by 6
Reis, D., & Friese, M. (2022) · Springer eBooks · doi:10.1007/978-3-031-04968-2_5 · Cited by 14
A Database of Accurate Electrophoretic Migration Patterns for Human Proteins
Mylonas, R., Potts, A., Waridel, P., Barblan, J., Rubio, M. D. C. C., Widmann, C., & Quadroni, M. (2022) · Journal of Molecular Biology · doi:10.1016/j.jmb.2022.167933 · Cited by 8
Automated cell segmentation for reproducibility in bioimage analysis
Robitaille, M. C., Byers, J. M., Christodoulides, J. A., & Raphael, M. P. (2023) · Synthetic Biology · doi:10.1093/synbio/ysad001 · Cited by 9
Neuroethics: Agency in the Age of Brain Science
May, J. (2023) · doi:10.1093/oso/9780197648087.001.0001 · Cited by 14
Side-by-Side Comparison of Compensation Beads Used in Polychromatic Flow Cytometry
Bhowmick, D., Lowe, S. K., & Ratliff, M. (2023) · ImmunoHorizons · doi:10.4049/immunohorizons.2300066 · Cited by 9
Data standards in drug discovery: A long way to go
Kelm, J. M., Ferrer, M., Bittner, M.-I., & Lal‐Nag, M. (2024) · Drug Discovery Today · doi:10.1016/j.drudis.2024.103879 · Cited by 9
Research Wranglers: Initiatives to Improve Reproducibility of Study Findings
Schmidt, C. W. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.122-a188 · Cited by 8
“In another widely publicized example, the biopharmaceutical company Amgen claimed it could reproduce just 6 of 53 studies that were considered landmarks in basic cancer research, despite close cooperation with the original scientists to make sure the same experimental protocols were used.(5) Experts blame the irreprod…”
ResultWearable digitization of life science experiments
Scholl, P., & Laerhoven, K. V. (2014) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2638728.2641719 · Cited by 7
Baggerly, K. (2013) · Clinical Chemistry · doi:10.1373/clinchem.2012.200501 · Cited by 9
Change starts with journal editors: In response to Makel (2014).
McBee, M. T., & Matthews, M. S. (2014) · Psychology of Aesthetics Creativity and the Arts · doi:10.1037/a0035801 · Cited by 9
Show and tell: disclosure and data sharing in experimental pathology
Schofield, P. N., Ward, J. M., & Sundberg, J. P. (2016) · Disease Models & Mechanisms · doi:10.1242/dmm.026054 · Cited by 12
“The inability to replicate drug-target discovery studies and to reliably replicate phenotype observations from the literature (Begley and Ellis, 2012) has caused profound concern amongst investigators and funding agencies alike, which has been mirrored in discussions and commentaries in the…”
Open Science als wissenschaftspolitische Problemlösungsformel?
Franzen, M. (2016) · Springer eBooks · doi:10.1007/978-3-658-05455-7_23 · Cited by 10
Extraordinary Science and Psychiatry: Responses to the Crisis in Mental Health Research
Poland, J. S., & Tekin, Ş. (2017) · Project Muse (Johns Hopkins University) · Cited by 12
Docility as a Primary Virtue in Scientific Research
Bezuidenhout, L. M., Ratti, E., Warne, N. A., & Beeler, D. (2018) · Minerva · doi:10.1007/s11024-018-9356-2 · Cited by 10
Towards Responsible Design with Internet of Things Data
Bourgeois, J., & Kortuem, G. (2019) · Proceedings of the ... International Conference on Engineering Design · doi:10.1017/dsi.2019.349 · Cited by 11
Rallapalli, H., Tan, I., Volkova, E., Wojcinski, A., Darwin, B. C., Lerch, J. P., Joyner, A. L., & Turnbull, D. H. (2019) · Magnetic Resonance in Medicine · doi:10.1002/mrm.27904 · Cited by 10
Sex differences in corneal neovascularization in response to superficial corneal cautery in the rat
Irani, Y., Pulford, E., Mortimer, L. A., Irani, S., Butler, L. M., Klebe, S., & Williams, K. A. (2019) · PLoS ONE · doi:10.1371/journal.pone.0221566 · Cited by 9
“There is a growing consensus that the standards of pre-clinical testing of novel therapies should mirror clinical trials [61], with an emphasis on well characterized [62], reproducible [23] models, and with the same transparency in reporting [63, 64].”
BackgroundLarson, M. J. (2020) · Biofeedback · doi:10.5298/1081-5937-48.1.2 · Cited by 10
A modern method of multiple working hypotheses to improve inference in ecology
Yanco, S. W., McDevitt, A., Trueman, C. N., Hartley, L., & Wunder, M. B. (2020) · Royal Society Open Science · doi:10.1098/rsos.200231 · Cited by 13
Woutersen-Windhouwer, S., Méndez, E., Sondervan, J., & Oort, F. J. (2020) · LIBER Quarterly The Journal of the Association of European Research Libraries · doi:10.18352/lq.10323 · Cited by 8
Open Science als Beitrag zur Qualität in der Bildungsforschung
Krammer, G., & Svecnik, E. (2020) · Zeitschrift für Bildungsforschung · doi:10.1007/s35834-020-00286-z · Cited by 14
Huang, L., Huang, J., Nie, H., Li, Y., Song, L., & Wu, F. (2021) · RSC Medicinal Chemistry · doi:10.1039/d0md00372g · Cited by 8
Ross, A. (2021) · Education science, evidence, and the public good · doi:10.1007/978-3-030-62572-6_1 · Cited by 8
Tailored Science Badges: Enabling New Forms of Research Interaction
Feger, S. S., Woźniak, P. W., Niess, J., & Schmidt, A. (2021) · Designing Interactive Systems Conference · doi:10.1145/3461778.3462067 · Cited by 10
“As the academic reputation economy focuses on rewarding novel contributions, scientists often perceive following Research Data Management (RDM) practices personally unrewarding [4, 11, 14].”
BackgroundInfluentialIssues and Challenges to Reproducibility of Cancer Research: A Commentary
da Silva, J. A. T. (2022) · Future Oncology · doi:10.2217/fon-2021-1378 · Cited by 9
Mah, K. M., Wu, W., Al‐Ali, H., Sun, Y., Han, Q., Ding, Y., Muñoz, M., Xu, X., Lemmon, V., & Bixby, J. L. (2022) · Experimental Neurology · doi:10.1016/j.expneurol.2022.114117 · Cited by 9
Semken, C., & Rossell, D. (2022) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/rssc.12578 · Cited by 10
“Choosing an appropriate statistical model is an old and important question that received renewed attention in light of the reproducibility crisis that plagues many research fields (Begley and Ellis, 2012; Open Science Collaboration, 2015; Baker, 2016; Camerer et al., 2018).”
BackgroundPatel, Y. T., Jacus, M. O., Boulos, N., Dapper, J. D., Davis, A. D., Vuppala, P. K., Freeman, B. B., Mohankumar, K. M., Throm, S. L., Gilbertson, R. J., et al. (2015) · Cancer Chemotherapy and Pharmacology · doi:10.1007/s00280-015-2713-z · Cited by 9
“Overall, the field of oncology has the highest clinical trial failure rate compared to other therapeutic areas [35].”
BackgroundBetter reporting for better research: a checklist for reproducibility
Kenall, A., Edmunds, S., Goodman, L., Bal, L., Flintoft, L., Shanahan, D. R., & Shipley, T. (2015) · Genome biology · doi:10.1186/s13059-015-0710-5 · Cited by 9
Effect of Initial Conditions on Reproducibility of Scientific Research
Djulbegović, B., & Hozo, I. (2014) · Acta Informatica Medica · doi:10.5455/aim.2014.22.156-159 · Cited by 9
Statistical evidence and the reliability of medical research
Andreoletti, M., & Teira, D. (2016) · UNED repository · doi:10.4324/9781315720739.ch20 · Cited by 10
Vyas, D., Balakrishnan, A., & Vyas, A. K. (2015) · American Journal of Robotic Surgery · doi:10.1166/ajrs.2015.1017 · Cited by 11
Oligoclonality and new agent evaluation in acute lymphoblastic leukaemia
Gaynon, P. S., & Sun, W. (2016) · British Journal of Haematology · doi:10.1111/bjh.14143 · Cited by 8
“…survival; Ph, Philadelphia chromosome; RR, relapse risk; TF, treatment failure. ª 2016 John Wiley & Sons Ltd 951 British Journal of Haematology, 2016, 173, 950–957 (Begley & Ellis, 2012) Laboratory investigators may modify inoculum size or time to treatment to optimize cytotoxicity.”
BackgroundPromoting reproducibility in addiction research
Munafo, M. R. (2017) · Addiction · doi:10.1111/add.13853 · Cited by 7
“Similar results have been reported by Amgen in 2012 [3], and more recently by the Reproducibility Project: Psychology [4].”
ResultPerformance qualification for reproducible Surface Plasmon Resonance analysis
Steinicke, F., Oltmann‐Norden, I., & Wätzig, H. (2017) · Analytical Biochemistry · doi:10.1016/j.ab.2017.12.027 · Cited by 8
Logical Experimental Design and Execution in the Biomedical Sciences
Holder, D., & Marino, M. J. (2017) · Current Protocols in Pharmacology · doi:10.1002/cpph.20 · Cited by 9
“In the field of oncology, attempts to reproduce 53 landmark studies by a group at Amgen and MD Anderson found that only 11% of the studies could be reproduced, despite working closely with the investigators responsible for the original peer-reviewed publications (Begley & Ellis, 2012).”
BackgroundOkabe, M. (2018) · FEBS Letters · doi:10.1002/1873-3468.13101 · Cited by 8
The reliability of an environmental epidemiology meta-analysis, a case study
Young, S. S., Acharjee, M. K., & Das, K. P. (2018) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2018.12.013 · Cited by 9
We are not Born Knowing how to Design and Analyse Scientific Experiments
Festing, M. F. W. (2013) · Alternatives to Laboratory Animals · doi:10.1177/026119291304100212 · Cited by 7
Does medicine need philosophy?
Andreoletti, M., & Maugeri, P. (2019) · Oral Diseases · doi:10.1111/odi.13143 · Cited by 10
Queau, C. L., Phillips, W. A., & Srinivasan, A. (2020) · médecine/sciences · doi:10.1051/medsci/2020044 · Cited by 8
Using the concept of “deserved trust” to strengthen the value and integrity of biomedical research
Yarborough, M. A. (2020) · Accountability in Research · doi:10.1080/08989621.2020.1855427 · Cited by 10
“This verdict echoes a similar sentiment expressed in the context of oncology studies, where commentators noted that “[u] nquestionably, a significant contributor to failure in oncology trials is the quality of published preclinical data” (Begley and Ellis 2012).”
BackgroundPorcine pancreatic ductal epithelial cells transformed with KRASG12D and SV40T are tumorigenic
Bailey, K. L., Cartwright, S., Patel, N. S., Remmers, N. A., Lazenby, A. J., Hollingsworth, M. A., & Carlson, M. A. (2021) · Scientific Reports · doi:10.1038/s41598-021-92852-2 · Cited by 9
Saraff, S., Singh, T., & Biswal, R. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.626638 · Cited by 11
“This led to a loss of confidence in empirical psychological researches (Ioannidis, 2005; Begley and Ellis, 2012; John et al., 2012; Nosek and Bar-Anan, 2012; Pashler and Wagenmakers, 2012; Button et al., 2013; Morey et al., 2016).”
BackgroundEveritt, J. I., Treuting, P. M., Scudamore, C. L., Sellers, R. S., Turner, P. V., Ward, J. M., & Zeiss, C. J. (2018) · ILAR Journal · doi:10.1093/ilar/ily020 · Cited by 8
Scientific Misconduct: Falsification, Fabrication, and Misappropriation of Credit
Vaux, D. L. (2016) · Springer eBooks · doi:10.1007/978-981-287-098-8_26 · Cited by 9
Mouse Medicine and Human Biology
Young, N. S. (2013) · Seminars in Hematology · doi:10.1053/j.seminhematol.2013.03.024 · Cited by 9
Liu, Y., Yang, M., Mueller, T., Kreibich, S. K., Williams, E. G., van Drogen, A., Borel, C., Germain, P., Frank, M., Bludau, I., et al. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/307421 · Cited by 8
How far have we come with contextual data integration in drug discovery?
Κάτσιλα, Θ., & Matsoukas, M. (2018) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2018.1504767 · Cited by 8
Of mice and men: lost in translation
Groisberg, R., Maitra, A., & Subbiah, V. (2019) · Annals of Oncology · doi:10.1093/annonc/mdz041 · Cited by 8
“Another complication is that even the pre-clinical studies themselves have a reproducibility problem [4].”
BackgroundFeger, S. S., Dallmeier-Tiessen, S., Schmidt, A., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.05875 · Cited by 8
“sources [13]. But those efforts are often not matched by the perceived gain. In fact, studies claim that the scientific culture does not support or even impairs compliance with reproducible practices [5, 21]. As research preservation tools are emerging, we set out to study design requirements for technology that suppo…”
BackgroundOpen Science Collaboration (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2195999 · Cited by 10
Fitzpatrick, B. G., Gorman, D. M., & Trombatore, C. (2024) · PLoS ONE · doi:10.1371/journal.pone.0303262 · Cited by 7
Mundinger, C., Schulz, N. K. E., Singh, P., Janz, S., Schurig, M., Seidemann, J., Kurtz, J., Müller, C., Schielzeth, H., von Kortzfleisch, V. T., et al. (2025) · PLoS Biology · doi:10.1371/journal.pbio.3003019 · Cited by 9
Solomon, A. D., Dabral, S., Brajesh, R. G., Day, B. W., Juric, M., Zielonka, J., Bosnjak, Z. J., & Pant, T. (2025) · International Journal of Molecular Sciences · doi:10.3390/ijms26093966 · Cited by 13
“Additionally, discrepancies in the cardiac parameters such as anatomy, kinetics, contractility, and action potential have led to substantial discordance between animal models and are a significant hurdle in clinical translation from bench to bedside [9].”
Tocchetti, C. G., González, A., Backs, J., Pollesello, P., Rainer, P. P., Schiattarella, G. G., Bellin, M., Begley, G., Bock-Marquette, I., Balligand, J., et al. (2025) · European Journal of Heart Failure · doi:10.1002/ejhf.3709 · Cited by 8
Metadata
- Authors
- Begley, C. G., & Ellis, L. M.
- Affiliations
- Amgen (United States) (US)
- The University of Texas MD Anderson Cancer Center (US)
- Corresponding author
- Ellis, Lee M.
- Venue
- Nature
- Published
- 2012-03-28
- Volume
- 483
- Issue
- 7391
- Pages
- 531-533
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W1987777080
- Semantic Scholar
- 1a81d3d1e8a983e0551edf485f4ac1a8cdd7fea4
- Cited by
- 2,979
- References
- 5
- Citations by year
- 2026: 106 · 2025: 116 · 2024: 127 · 2023: 169 · 2022: 179 · 2021: 206 · 2020: 236 · 2019: 255 · 2018: 262 · 2017: 271 · 2016: 283 · 2015: 262 · 2014: 226 · 2013: 133 · 2012: 62
- FWCI
- 210.63
- Type
- journal article
- Language
- en
- Update notices
- correction (2012-05-02)
- Open access
- bronze
- doi.org
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health and Medical Research Impacts
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Health Sciences › Nursing › Issues, ethics and legal aspects › Biomedical Ethics and Regulation
- Keywords
- preclinical cancer research, research standards, publication practices, research incentives, translational research, clinical trials
- MeSH terms
- Animals, Clinical Trials as Topic, Drug Evaluation, Preclinical, Humans, Neoplasms, Research Design, Reproducibility of Results, Survival Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.