Raise standards for preclinical cancer research
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index; 3 of the 9 sources its claims cite were checked for support.
Overview
CoChat Summary
TL;DR
This Nature Comment argues that many landmark preclinical oncology findings are not reproducible and calls for higher standards in methods, reporting, and incentives. It reports that Amgen confirmed only 6 of 53 landmark papers (11%) and cites Bayer’s finding that about 25% of published preclinical studies could be validated.
Key findings
- In Amgen’s ten-year retrospective analysis, scientific findings were confirmed in only 6 of 53 “landmark” preclinical cancer papers (11%); non-reproduced studies often lacked blinding, relied on single experiments such as one Western blot, or presented selected data.
- Bayer HealthCare reported that only about 25% of published preclinical studies could be validated to the point at which projects could continue; published cancer research represented 70% of the studies analysed in that report, with possible overlap with Amgen’s 53 papers.
- Citation data showed non-reproduced articles in journals with impact factor >20 had a mean of 248 citations (range 3–800) versus 231 (range 82–519) for reproduced articles; for impact factor 5–19, non-reproduced articles had a mean of 169 citations (range 6–1,909) versus 13 (range 3–24) for reproduced articles.
- The authors recommend reporting negative data, blinding preclinical experiments, validating reagents, repeating critical experiments, avoiding single cell lines or models, requiring patient-selection biomarkers, and changing academic incentives.
Methods
- This is a Nature Comment/perspective, not a primary experimental study; it synthesizes Amgen’s ten-year retrospective analysis of experiments performed prospectively and cites a Bayer HealthCare report.
- Amgen selected 53 “landmark” papers describing completely new approaches or alternative clinical uses, attempted confirmation, used additional models, contacted original authors, exchanged reagents, and repeated experiments under the authors’ direction, occasionally in the original investigator’s laboratory.
- The analysis reports counts of confirmed findings and citation metrics from Google Scholar, May 2011; the recommendations are proposed changes rather than tested interventions.
Limitations
- Stated: validation attempts may have failed because of technical differences or difficulties despite efforts to ensure otherwise; additional models were used; when findings could not be reproduced, authors were contacted and experiments were repeated under their direction.
- Stated: papers were deliberately selected as “landmark” and described something completely new, so some data might not hold up; Bayer’s analysed studies may overlap with Amgen’s 53 papers.
- Evident: as a Comment, the text provides no detailed systematic protocol, inclusion criteria, statistical methods, or full dataset; the citation analysis is descriptive and limited to two impact-factor strata.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 1 Not valid · 6 Not checked · 12 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Sadly, clinical trials in oncology have the highest failure rate compared with other therapeutic areas.”
What the sources say
- Partly supportedcites [1] Hutchinson, L. & Kirk, R. Nature Rev. Clin. Oncol. 8, 189–190 (2011). sourceExperimental — accuracy not yet measured
The opening passage of our copy (labelled Abstract; page unavailable) says cancer-drug attrition is much higher than in other therapeutic areas, citing 5% of anticancer agents versus 20% of cardiovascular agents reaching licensure. That comparison covers development from preclinical activity through phase III, not failure rates specifically among clinical trials; it also does not explicitly establish a highest-rate ranking. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Although we in the cancer field hoped that this would lead to more effective drugs, historically, our ability to translate cancer research to clinical success has been remarkably low 1 .”
What the sources say
- Supportedcites [1] Hutchinson, L. & Kirk, R. Nature Rev. Clin. Oncol. 8, 189–190 (2011). source
The opening passage (labeled Abstract in our copy; no page given) reports that only 5% of agents with preclinical anticancer activity are licensed after demonstrating sufficient efficacy in phase III testing. It also says cancer drug attrition is much higher than in other therapeutic areas, supporting the claim of historically low translation to clinical success. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In studies for which findings could be reproduced, authors had paid close attention to controls, reagents, investigator bias and describing the complete data set.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For results that could not be reproduced, however, data were not routinely analysed by investigators blinded to the experimental versus control groups.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Investigators frequently presented the results of one experiment, such as a single Western-blot analysis.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
12 more claims
Create a free account to see all 22 claims and every source verdict.
- It may state two separate claims as one. 2
- Probably not a substantial claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“there are other more common sources of unreliability in the literature (41)(42)(43)(44)”
From Misconduct accounts for the majority of retracted scientific publications (2012) · cites this paper as [42]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“many established scientific results in the social (49, 50) and biomedical (51-53) sciences cannot readily be replicated.”
From Misinformation in and about science (2021) · cites this paper as [51]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A landmark study conducted by Glenn Begley and his team in 2012 concluded that most cancer research studies were not reproducible, by alarmingly reporting that 47 of 53 published findings could not be replicated [13].”
From Guidelines for the use of flow cytometry and cell sorting in immunological studies (third edition) (2021) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In cell biology, two industrial laboratories reported success replicating the original results of landmark findings in only 11% and 25% of the attempted cases ( 10, 11 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Concerns about reproducibility are widespread across disciplines ( 921 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [10]
Experimental — accuracy not yet measured
13 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888
Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010271 · Cited by 805
High drug attrition rates—where are we going wrong?
Hutchinson, L., & Kirk, R. (2011) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2011.34 · Cited by 517
Drug development and clinical trials—the path to an approved cancer drug
Rubin, E. H., & Gilliland, D. G. (2012) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2012.22 · Cited by 135
Raising the bar for cancer therapy models
Francia, G., & Kerbel, R. S. (2010) · Nature Biotechnology · doi:10.1038/nbt0610-561 · Cited by 54
Cited by
Items 1,001–1,100 of 2,889
Wittau, J., & Seifert, R. (2024) · Naunyn-Schmiedeberg s Archives of Pharmacology · doi:10.1007/s00210-024-03272-8 · Cited by 36
Cousido-Siah, A., Ruiz, F. X., Fanfrlík, J., Giménez-Dejoz, J., Mitschler, A., Kamlar, M., Veselý, J., Ajani, H., Parés, X., Farrés, J., et al. (2016) · ACS Chemical Biology · doi:10.1021/acschembio.6b00382 · Cited by 26
Functional metabolic interactions of human neuron-astrocyte 3D in vitro networks
Simão, D. F., Terrasso, A. P., Teixeira, A. P., Brito, C., Sonnewald, U., & Alves, P. M. (2016) · Scientific Reports · doi:10.1038/srep33285 · Cited by 29
EMICORON: A multi-targeting G4 ligand with a promising preclinical profile
Porru, M., Zizza, P., Franceschin, M., Leonetti, C., & Biroccio, A. (2016) · Biochimica et Biophysica Acta (BBA) - General Subjects · doi:10.1016/j.bbagen.2016.11.010 · Cited by 24
Surgery-induced tumor growth in (metastatic) colorectal cancer
Govaert, K. M., Jongen, J. M. J., Kranenburg, O., & Rinkes, I. H. M. B. (2017) · Surgical Oncology · doi:10.1016/j.suronc.2017.10.004 · Cited by 28
Falls, K. C., Sharma, R. A., Lawrence, Y. R., Amos, R. A., Advani, S. J., Ahmed, M. M., Vikram, B., Coleman, C. N., & Prasanna, P. G. S. (2018) · Radiation Research · doi:10.1667/rr15121.1 · Cited by 27
“However, one impediment to drug development has been the irreproducibility of preclinical data (46); thus, there is an urgent need to improve reproducibility and translatability of preclinical data to fully exploit opportunities for molecularly-targeted therapeutics involving radiation and radiochemotherapy (10).”
Schofield, P. N., Kulka, U., Tapio, S., & Grosche, B. (2019) · International Journal of Radiation Biology · doi:10.1080/09553002.2019.1589026 · Cited by 24
“…increasing political and scientific concern about the issues of data sharing, accountability and reproducibility in the biological sciences (Begley and Ellis 2012; Collins and Tabak 2014; Baker 2016), and the preservation and sharing of individual datasets from current studies, especially…”
BackgroundWhy Does Brain Trauma Research Fail?
Alves, J. L., Rato, J. R., & da Silva, V. M. M. (2019) · World Neurosurgery · doi:10.1016/j.wneu.2019.06.212 · Cited by 27
Global Factor PremiumsPreprint
Baltussen, G., Swinkels, L. A. P., & Vliet, P. V. A. D. T. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3325720 · Cited by 24
Advances in microfluidic 3D cell culture for preclinical drug development
(2022) · Progress in Molecular Biology and Translational Science · doi:10.1016/bs.pmbts.2021.07.022 · Cited by 23
Using ontologies for verification and validation of workflow-based experiments
Miksa, T., & Rauber, A. (2017) · Journal of Web Semantics · doi:10.1016/j.websem.2017.01.002 · Cited by 23
Sluggish data sharing hampers reproducibility effort
Noorden, R. V. (2015) · Nature · doi:10.1038/nature.2015.17694 · Cited by 23
Linder, C., & Farahbakhsh, S. (2020) · Business Ethics Quarterly · doi:10.1017/beq.2019.52 · Cited by 26
“…that studies repeatedly find the reproducibility of published scientific findings in some academic disciplines are as low as 15 to 30 percent (Begley & Ellis, 2012; Camerer et al., 2018; Ioannidis, 2005; Prinz, Schlange, & Asadullah, 2011), leading to serious questions regarding the…”
BackgroundCo-production, multiplied: Enactments of sex as a biological variable in US biomedicine
Pape, M. (2021) · Social Studies of Science · doi:10.1177/0306312720985939 · Cited by 36
Korch, C. T., & Capes-Davis, A. (2021) · SLAS Discovery · doi:10.1177/24725552211051963 · Cited by 23
Use and application of organ-on-a-chip platforms in cancer research
Yu, Y., Zhou, T., & Cao, L.-L. (2023) · Journal of Cell Communication and Signaling · doi:10.1007/s12079-023-00790-7 · Cited by 31
“…while animal models, particularly mouse models, used in tumor research do not present ethical concerns and can account for some of the differences that exist between humans and animal species, they cannot accurately represent actual responses in humans (Begley and Ellis 2012; Imparato et al. 2022).”
BackgroundSynthetic Biology: evolution or revolution? A co-founder's perspective
Gardner, T. S., & Hawkins, K. M. (2013) · Current Opinion in Chemical Biology · doi:10.1016/j.cbpa.2013.09.013 · Cited by 27
Bravo, S. B., García-Rendueles, M. E. R., Garcia-Rendueles, Á. R., Rodrigues, J. S., Pérez-Romero, S., García-Lavandeira, M., Suarez-Fariña, M., Barreiro, F. J. J., Czarnocka, B., Senra, A., et al. (2013) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2012-3812 · Cited by 23
How many scientific papers are not original?
Lesk, M. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1422282112 · Cited by 23
Education and Training Needs in the Radiation Sciences: Problems and Potential Solutions
Dynlacht, J. R., Zeman, E. M., Held, K. D., et al. (2015) · Radiation Research · doi:10.1667/rr14199.1 · Cited by 23
A lost opportunity for science: journals promote data sharing in metabolomics but do not enforce it
Spicer, R., & Steinbeck, C. (2017) · Metabolomics · doi:10.1007/s11306-017-1309-5 · Cited by 25
“Areas as diverse as psychology (Open Science Collaboration 2015), medicine (Begley and Ellis 2012) and computer science (Collberg and Proebsting 2016) have been affected by the ‘replication crisis’ and two-thirds of scientists report being concerned about reproducibility (Reality check on reproducibility 2016).”
BackgroundEggel, M., & Grimm, H. (2018) · Animals · doi:10.3390/ani8030034 · Cited by 29
Khatami, M. (2018) · Clinical and Translational Medicine · doi:10.1186/s40169-018-0193-6 · Cited by 25
Webster, J. D., Santagostino, S. F., & Foreman, O. (2019) · Cell and Tissue Research · doi:10.1007/s00441-019-03101-y · Cited by 25
A Path to Greater Credibility: Large-Scale Collaborative Education Research
Makel, M. C., Smith, K. N., McBee, M. T., Peters, S. J., & Miller, E. M. (2019) · AERA Open · doi:10.1177/2332858419891963 · Cited by 31
“His analysis proved prescient; when disciplines began to experiment with systematic replication studies (e.g., Begley & Ellis, 2012; OSC, 2015), the results were grim.”
BackgroundBrain Organoids: A Promising Living Biobank Resource for Neuroscience Research
Li, S., Wang, M., & Zhou, J. (2020) · Biopreservation and Biobanking · doi:10.1089/bio.2019.0111 · Cited by 25
Frohner, I. E., Mudrak, I., Schüchner, S., Anrather, D., Hartl, M., Sontag, J., Sontag, E., Wadzinski, B. E., Preglej, T., Ellmeier, W., et al. (2020) · Cell Reports · doi:10.1016/j.celrep.2020.02.035 · Cited by 25
Holden, J. K., Francisco, E., Lensch, R., Tommerdahl, A., Kirsch, B., Zai, L. J., Dennis, R. G., & Tommerdahl, M. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/726364 · Cited by 25
“(3) Tactile stimulus with response on a computer mouse (orange) was significantly better than either visual modality although not as accurate as (4) tactile stimulus with dedicated tactile hardware response device (gray).”
Stablecoins 2.0: Economic Foundations and Risk-based ModelsPreprint
Klages‐Mundt, A., Harz, D., Gudgeon, L., et al. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3633542 · Cited by 23
Replicability in Brain Imaging
Kelly, R. E., & Hoptman, M. J. (2022) · Brain Sciences · doi:10.3390/brainsci12030397 · Cited by 24
“This situation applies to fields where tested hypotheses are seldom true, which could in part explain the low replication rates observed in cancer studies [11,12].”
BackgroundImproving the reliability of peer-reviewed publications: We are all in it together
Bustin, S. A., & Nolan, T. (2016) · Biomolecular Detection and Quantification · doi:10.1016/j.bdq.2015.11.002 · Cited by 22
Empirical evidence for low reproducibility indicates low pre-study odds
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature reviews. Neuroscience · doi:10.1038/nrn3475-c6 · Cited by 21
Multiplicity corrections in life sciences: challenges and consequences
Menyhárt, O., & Győrffy, B. (2025) · International Journal of Epidemiology · doi:10.1093/ije/dyaf098 · Cited by 31
Targeting Peroxisome Proliferator-Activated Receptor-β/δ (PPARβ/δ) for Cancer Chemoprevention
Peters, J. M., Yao, P., & Gonzalez, F. J. (2015) · Current Pharmacology Reports · doi:10.1007/s40495-015-0026-x · Cited by 24
Sun, Y. M., Ban, B., Bradbury, A. M., Ansari, G. A. S., & Blake, D. A. (2016) · Analytical Chemistry · doi:10.1021/acs.analchem.6b02334 · Cited by 21
Ingre, M., & Nilsonne, G. (2018) · Royal Society Open Science · doi:10.1098/rsos.181190 · Cited by 23
“…with 70-90% significant findings supporting the authors' hypothesis (Fanelli, 2010; T. D. Sterling et al., 1995), but even worse reproducibility rates in the range 11—24% in certain fields (Begley & Ellis, 2012; Prinz, Schlange, & Asadullah, 2011; Steward, Popovich, Dietrich, & Kleitman, 2012).”
BackgroundRoth, S., Yang, J., Cramer, J. V., Malik, R., & Liesz, A. (2020) · Brain Behavior and Immunity · doi:10.1016/j.bbi.2020.11.016 · Cited by 24
“While numerous suggestions have been made to overcome this translational roadblock (Begley and Ellis, 2012; Howells et al., 2014; Llovera and Liesz, 2016), an alternative approach could be to target therapies beyond tissue protection and focal deficits.”
BackgroundVelázquez‐Campoy, A., Claro, B., Abián, O., Höring, J., Bourlon, L., Claveria‐Gimeno, R., Ennifar, E., England, P., Chaires, J. B., Wu, D., et al. (2021) · European Biophysics Journal · doi:10.1007/s00249-021-01523-7 · Cited by 23
“The lack of reproducibility is a matter of severe concern among researchers from academia and industry (Baker 2016; Begley and Ellis 2012; Ioannidis 2016; Mobley et al.”
BackgroundKopylova, O. V., Ershova, A. I., Pokrovskaya, M. S., Meshkov, A. N., Efimova, I. A., Serebryanskaya, Z. Z., Блохина, А. В., Borisova, A. L., Kondratskaya, V. A., Limonova, A. S., et al. (2022) · CARDIOVASCULAR THERAPY AND PREVENTION · doi:10.15829/1728-8800-2021-3119 · Cited by 26
Reimagining peer review as an expert elicitation process
Marcoci, A., Vercammen, A., Bush, M., Hamilton, D. G., Hanea, A. M., Hemming, V., Wintle, B. C., Burgman, M. A., & Fidler, F. M. (2022) · BMC Research Notes · doi:10.1186/s13104-022-06016-0 · Cited by 26
Relevance relations for the concept of reproducibility
Atmanspacher, H., Lambert, L. B., Folkers, G., & Schubiger, P. A. (2014) · Journal of The Royal Society Interface · doi:10.1098/rsif.2013.1030 · Cited by 25
The National Biomarker Development Alliance Accelerating the Translation of Biomarkers to the Clinic
Barker, A. D., Compton, C. C., & Poste, G. H. (2014) · Biomarkers in Medicine · doi:10.2217/bmm.14.52 · Cited by 22
Hey, S. P., Heilig, C. M., & Weijer, C. (2013) · Trials · doi:10.1186/1745-6215-14-159 · Cited by 26
Cifuentes, F., VALENZUELA, R. H., Contreras, H. R., & Castellón, E. A. (2015) · Oncology Letters · doi:10.3892/ol.2015.3522 · Cited by 26
“These models are useful in a preclinical setting, as they facilitate the investigation of the biology of cancer and its response to therapeutic agents (2,3).”
BackgroundA New Replication Norm for Psychology
LeBel, E. P. (2015) · Collabra · doi:10.1525/collabra.23 · Cited by 33
“Though there are different ways to interpret successful versus unsuccessful replication results [95], taken together, these observations strongly suggest psychology currently has a general replicability problem (as do several other areas of science including cancer cell biology and cardiovascular health literatures [8…”
ResultReproducibility Issues: Avoiding Pitfalls in Animal Inflammation Models
Laman, J. D., Kooistra, S. M., & Clausen, B. E. (2017) · Methods in molecular biology · doi:10.1007/978-1-4939-6786-5_1 · Cited by 24
Strategies for Managing the Structural and Dynamic Consequences of Project Complexity
Floricel, S., Piperca, S., & Tee, R. (2018) · Complexity · doi:10.1155/2018/3190251 · Cited by 28
“One example is biotech projects, in which the large number of levels and interactions in living beings can hardly be considered at the same time in order to predict the properties of drug candidates [142] or even to replicate successful experiments [143].”
BackgroundSchnell, S. D. (2018) · Bulletin of Mathematical Biology · doi:10.1007/s11538-018-0500-9 · Cited by 28
“…peer-reviewed scientific journals cannot be replicated in the biomedical sciences (Hirschhorn et al. 2002, Ioannidis 2005, Prinz et al. 2011, Begley and Ellis 2012), psychology (Bakker and Wicherts 2011, John et al 2012, Pashler and Wagenmakers 2012, Van Bavel et al. 2016), sociology and…”
BackgroundAn open toolkit for tracking open science partnership implementation and impact
Gold, E. R., Ali‐Khan, S. E., Allen, L., Ballell, L., Barral‐Netto, M., Carr, D. L., Chalaud, D., Chaplin, S., Clancy, M., Clarke, P., et al. (2019) · Gates Open Research · doi:10.12688/gatesopenres.12958.2 · Cited by 24
In vitro disease models 4.0 via automation and high-throughput processing
Eggert, S., & Hutmacher, D. W. (2019) · Biofabrication · doi:10.1088/1758-5090/ab296f · Cited by 25
Lightweight Distributed Provenance Model for Complex Real–world Environments
Wittner, R., Mascia, C., Gallo, M., Frexia, F., Müller, H., Plass, M., Geiger, J., & Holub, P. (2022) · Scientific Data · doi:10.1038/s41597-022-01537-6 · Cited by 29
Camillo, F. D., Grimaldi, D. A., Cattarinussi, G., Giorgio, A. D., Locatelli, C., Khuntia, A. T., Enrico, P., Brambilla, P., Koutsouleris, N., & Sambataro, F. (2024) · Psychiatry and Clinical Neurosciences · doi:10.1111/pcn.13736 · Cited by 32
Degen, P. M., & Medo, M. (2025) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011630 · Cited by 28
“Although the broader replication crisis in science [20–22, 44 , 45 ] includes numerous human • Significant DEGs from one small cohort are unlikely to be significant in another small cohort (low replicability), unless it is known that the population is very homogeneous (e.g. cell cultures).”
Cirello, V., Vaira, V., Grassi, E., Vezzoli, V., Ricca, D., Colombo, C. M., Bòsari, S., Vicentini, L., Persani, L., Ferrero, S., et al. (2016) · Oncotarget · doi:10.18632/oncotarget.14187 · Cited by 21
Xie, J. Y., Qu, C., Munro, G., Petersen, K. A., & Porreca, F. (2019) · Pain · doi:10.1097/j.pain.0000000000001569 · Cited by 19
Biobank Quality Management in the BBMRI.be Network
Linsen, L., T’Joen, V., Straeten, C. V. D., Landuyt, K. V., Marbaix, É., Bekaert, S., & Ectors, N. (2019) · Frontiers in Medicine · doi:10.3389/fmed.2019.00141 · Cited by 22
“Irreproducibility of results has been identified as a major undermining factor for translating research results into clinical applications (1).”
BackgroundMethodA Novel SimpleDrop Chip for 3D Spheroid Formation and Anti-Cancer Drug Assay
Liu, X., Lin, H., Song, J., Zhang, T., Wang, X., Huang, X., & Zheng, C. (2021) · Micromachines · doi:10.3390/mi12060681 · Cited by 27
Karagianni, F., Piperi, C., Casar, B., de la Fuente‐Vivas, D., García‐Gómez, R., Lampadaki, K., Pappa, V., & Papadavid, E. (2022) · Cancers · doi:10.3390/cancers14041070 · Cited by 20
“The benefits of the CAM assay include easy monitoring of the tumor extravasation and intravasation into the microvasculature and consequent formation of metastases in organs, evaluated by qPCR [49,50].”
Backgroundde Ridder, J. (2022) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2022.02.003 · Cited by 26
King, D. R., Hardin, K. M., Hoeker, G. S., & Poelzing, S. (2022) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.00164.2022 · Cited by 22
Open Science Practices in Communication Sciences and Disorders: A Survey
Amin, M. E., Borders, J. C., Long, H. L., Keller, M. A., & Kearney, E. (2022) · Journal of Speech Language and Hearing Research · doi:10.1044/2022_jslhr-22-00062 · Cited by 28
Three Paths to Drug Discovery in Psychiatry
Berk, M., & Nierenberg, A. A. (2015) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2014.14070858 · Cited by 21
NIH Peer Review: Scored Review Criteria and Overall Impact
Lindner, M. D., Vancea, A., Chen, M., & Chacko, G. W. (2015) · American Journal of Evaluation · doi:10.1177/1098214015582049 · Cited by 23
“…Young, Ioannidis, & Al-Ubaydli, 2008) and there is increasing evidence that many of the findings published in the literature are not reproducible (Begley & Ellis, 2012; Lohmueller, Pearce, Pike, Lander, & Hirschhorn, 2003; Prinz, Schlange, & Asadullah, 2011; Steward, Popovich, Dietrich, &…”
BackgroundA Survey of the Neuroscience Resource Landscape
Cachat, J., Bandrowski, A. E., Grethe, J. S., Gupta, A., Astakhov, V., Imam, F. T., Larson, S., & Martone, M. E. (2012) · International review of neurobiology · doi:10.1016/b978-0-12-388408-4.00003-4 · Cited by 22
Apps, J. R., & Martı́nez-Barberá, J. P. (2017) · Brain Pathology · doi:10.1111/bpa.12501 · Cited by 21
“the use of the smoothened inhibitor GDC-0449 in a preclinical model of medulloblastoma, later validated in the clinic), there have been many disappointments, where promising results in the preclinical setting have failed to translate to benefit in patients with the disease (7, 14, 38).”
BackgroundResearch Misconduct in the Age of Open Science: The Case of STAP Stem Cells
Meskus, M., Marelli, L., & D’Agostino, G. A. (2017) · Science as Culture · doi:10.1080/09505431.2017.1316975 · Cited by 21
“Ioannidis (2005) and Begley and Ellis (2012) claim that the traditional peer review is unable to ensure reproducibility.”
Backgroundden Broeder, A. A., van Herwaarden, N., & van den Bemt, B. J. F. (2018) · Current Opinion in Rheumatology · doi:10.1097/bor.0000000000000487 · Cited by 22
Replicable Services for Reproducible Research: A Model for Academic Libraries
Sayre, F., & Riegelman, A. (2019) · College & Research Libraries · doi:10.5860/crl.80.2.260 · Cited by 25
“В последние годы в ряде дисциплин, включая психологию [1], биологию [2], биомедицину [3], неврологию [4], разработку лекарств [5], химию [6], климатологию [7], экономику [8, 9] и образование [10], появились свидетельства того, что наука стоит перед лицом кризиса воспроизводимости.”
Pu, Q., Li, Y., Zhang, H., Yao, H., Zhang, B., Hou, B., Li, L., Zhao, Y., & Zhao, L. (2019) · Science China Chemistry · doi:10.1007/s11426-018-9412-6 · Cited by 21
Ramirez, F. D., Jung, R. G., Motazedian, P., Perry‐Nguyen, D., Santo, P. D., MacDonald, Z., Clancy, A. A., Labinaz, A., Promislow, S., Simard, T., et al. (2019) · Stroke · doi:10.1161/strokeaha.119.026564 · Cited by 24
Nelissen, I., Haase, A., Anguissola, S., Rocks, L., Jacobs, A., Willems, H., Riebeling, C., Luch, A., Piret, J., Toussaint, O., et al. (2020) · Nanomaterials · doi:10.3390/nano10081430 · Cited by 23
Is the Political Slant of Psychology Research Related to Scientific Replicability?
Reinero, D. A., Wills, J. A., Brady, W. J., Mende‐Siedlecki, P., Crawford, J. T., & Bavel, J. J. V. (2020) · Perspectives on Psychological Science · doi:10.1177/1745691620924463 · Cited by 21
“…replicating findings in genetics (Hirschhorn, Lohmueller, Byrne, & Hirschhorn, 2002), pharmacology (Prinz, Schlange, & Asadullah, 2011), oncology (Begley & Ellis, 2012), biology (Reaves, Sinha, Rabinowitz, Kruglyak, & Redfield, 2012), psychology (OSC, 2015), and economics (Chang & Li, 2015),…”
Feger, S. S., Dallmeier-Tiessen, S., Schmidt, A., & Woźniak, P. W. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300685 · Cited by 34
“sources [13]. But those efforts are often not matched by the perceived gain. In fact, studies claim that the scientific culture does not support or even impairs compliance with reproducible practices [5, 21]. As research preservation tools are emerging, we set out to study design requirements for technology that suppo…”
BackgroundImpact of analytic provenance in genome analysis
Morrison, S. S., Pyzh, R., Jeon, M. S., Amaro, C., Roig, F. J., Baker‐Austin, C., Oliver, J. D., & Gibas, C. (2014) · BMC Genomics · doi:10.1186/1471-2164-15-s8-s1 · Cited by 22
“In recent years, the biomedical research community has increasingly recognized the failure of many studies to achieve reproducibility [16] in data analysis protocols.”
BackgroundDunkley, T. P. J., Costa, V., Friedlein, A., Lugert, S., Aigner, S., Ebeling, M., Miller, M. T., Patsch, C., Piraino, P., Cutler, P., et al. (2015) · PROTEOMICS - CLINICAL APPLICATIONS · doi:10.1002/prca.201400150 · Cited by 19
“Low translational success from cancer research to clinical success, for example, has been attributed in part to the use of poorly characterized preclinical cellular models [2].”
BackgroundSome Methodological Deficiencies in Empirical Research Articles in AccountingPreprint
Dyckman, T. R., & Zeff, S. A. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2324266 · Cited by 21
Kocak, B., Keles, A., & Akinci D’Antonoli, T. (2023) · European Radiology · doi:10.1007/s00330-023-10243-9 · Cited by 21
“Although it is critical to recognize that inadequate reporting does not always imply poor methodology, numerous studies showed that it is frequently impossible to replicate and evaluate the strengths and weaknesses of studies due, in part, to poor reporting [3–5].”
Incidence of Data Duplications in a Randomly Selected Pool of Life Science Publications
Oksvold, M. P. (2016) · Science and Engineering Ethics · doi:10.1007/s11948-015-9668-7 · Cited by 20
“The disappointing conclusions from previous studies (Prinz et al. 2011; Begley and Ellis 2012) and this study, should encourage scientific journals and editors to implement measures in their review and erratum/retraction processes, to improve the quality and reliability of the scientific literature: (1) Apply an autom…”
BackgroundResultBreast cancer organoids and their applications for precision cancer immunotherapy
Guan, D., Liu, X., Shi, Q., He, B., Zheng, C., & Meng, X. (2023) · World Journal of Surgical Oncology · doi:10.1186/s12957-023-03231-2 · Cited by 23
“But they cannot assess all types of immunotherapies, owing to the huge inherent disparities in immune systems between animals and humans [25].”
BackgroundAssessing risk of bias in toxicological studies in the era of artificial intelligence
Härtung, T., Hoffmann, S., & Whaley, P. (2025) · Archives of Toxicology · doi:10.1007/s00204-025-03978-5 · Cited by 33
“Begley and Ellis (2012) discuss the implications of bias in preclinical research for drug development, highlighting the importance of addressing bias to improve translational success.”
A Human-Based Integrated Framework forAlzheimer’s Disease Research
Pistollato, F., Cavanaugh, S. E., & Chandrasekera, P. C. (2015) · Journal of Alzheimer s Disease · doi:10.3233/jad-150281 · Cited by 23
Failure to Replicate: Sound the Alarm.
Ioannidis, J. P. A. (2015) · PubMed Central · Cited by 22
Multidimensional complexity of cancer. Simple solutions are needed
Sverdlov, E. D. (2016) · Biochemistry (Moscow) · doi:10.1134/s0006297916070099 · Cited by 41
Peters, T. J., French, H. J., Bradford, S. T., Pidsley, R., Stirzaker, C., Varinli, H., Nair, S. S., Qu, W., Song, J. Z., Giles, K. A., et al. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty675 · Cited by 21
“In many cases, the degree of this variation exceeds that of what is expected, leading to a “crisis of reproducibility” (Baker 2016; Begley and Ellis, 2012; Nosek and Errington 2017) in science.”
BackgroundSignaling Targets Related to Antiobesity Effects of Capsaicin: A Scoping Review
Ávila, D. L., Nunes, N. A. M., Almeida, P. H. R. F., Gomes, J. D. A. S., Rosa, C. D. O. B., & Alvarez-Leite, J. I. (2021) · Advances in Nutrition · doi:10.1093/advances/nmab064 · Cited by 20
Collaborative methods to enhance reproducibility and accelerate discovery
Leins, D. A., Haase, S. B., Eslami, M., Schrier, J., & Freeman, J. (2022) · Digital Discovery · doi:10.1039/d2dd00061j · Cited by 25
Zhou, L., Liu, L., Chang, M. A., Ma, C., Chen, W., & Chen, P. (2023) · Biosensors and Bioelectronics · doi:10.1016/j.bios.2023.115064 · Cited by 22
Toward a common standard for data and specimen provenance in life sciences
Wittner, R., Holub, P., Mascia, C., Frexia, F., Müller, H., Plass, M., Allocca, C. M., Betsou, F., Burdett, T., Cancio, I., et al. (2023) · Learning Health Systems · doi:10.1002/lrh2.10365 · Cited by 30
“Experts from multiple life sciences domains have called for the improvement and standardization of the documentation of research and scientific service processes.(20-26) This has led in turn”
BackgroundSteckler, T., Brose, K., Haas, M., Kas, M. J., Koustova, E., & Bespalov, A. (2015) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2015.05.011 · Cited by 22
“substantial difficulties to reproduce internally what has been reported in the public domain (Begley and Ellis, 2012; Prinz et al., 2011) – experiences shared by many scientists working in other pharmaceutical companies and academia.”
Tian, D., & Kreeger, P. K. (2014) · BMC Systems Biology · doi:10.1186/s12918-014-0098-y · Cited by 22
“Given recent concerns about the reproducibility of key findings in cancer research [65], metrics such as cellular dose that may better enable experimental consistency should be utilized.”
BackgroundPost-normal institutional identities: Quality assurance, reflexivity and ethos of care
Pereira, Â. G., & Saltelli, A. (2017) · Futures · doi:10.1016/j.futures.2016.11.009 · Cited by 27
The importance of behavioural bioassays in neuroscience
Brown, R. E., & Bolivar, S. (2017) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2017.05.022 · Cited by 25
“Poorly characterized antibodies thus provide one of the most significant problems in the use of bioassays (Bradbury et al., 2015; Begley and Ellis, 2012).”
BackgroundOn the Merits and Limits of Replication and Negation for IS Research
Olbrich, S., Frank, U., Gregor, S. D., Niederman, F., & Rowe, F. (2017) · AIS Transactions on Replication Research · doi:10.17705/1atrr.00016 · Cited by 19
“In 2012, researchers at Amgen, a biotech firm, found they could reproduce just 6 out of 53 so called “landmark” studies in cancer research (Begley & Ellis, 2012).”
BackgroundInnovation Policy in a Networked World
Sorenson, O. (2017) · Innovation Policy and the Economy · doi:10.1086/694407 · Cited by 21
Indicators for the use of robotic labs in basic biomedical research: a literature analysis
Groth, P., & Cox, J. (2017) · PeerJ · doi:10.7717/peerj.3997 · Cited by 25
Bolon, B. N., Baze, W. B., Shilling, C. J., Keatley, K. L., Patrick, D. J., & Schafer, K. A. (2018) · ILAR Journal · doi:10.1093/ilar/ily008 · Cited by 23
Resolving the Tension Between Exploration and Confirmation in Preclinical Biomedical Research
Dirnagl, U. (2019) · Handbook of experimental pharmacology · doi:10.1007/164_2019_278 · Cited by 26
A systematic review and meta-analysis of pregabalin preclinical studies
Federico, C. A., Mogil, J. S., Ramsay, T., Fergusson, D. A., & Kimmelman, J. (2020) · Pain · doi:10.1097/j.pain.0000000000001749 · Cited by 21
“These arguments echo those raised by commentators in other disease areas, such as cancer and stroke.(4,43,50) Sound preclinical study design and reporting takes on special importance in pain research, since studies require at least a transient induction of unrelieved suffering in animals.”
BackgroundCohen, K. B., Xia, J., Roeder, C., & Hunter, L. (2016) · PubMed · Cited by 20
Metadata
- Authors
- Begley, C. G., & Ellis, L. M.
- Affiliations
- Amgen (United States) (US)
- The University of Texas MD Anderson Cancer Center (US)
- Corresponding author
- Ellis, Lee M.
- Venue
- Nature
- Published
- 2012-03-28
- Volume
- 483
- Issue
- 7391
- Pages
- 531-533
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W1987777080
- Semantic Scholar
- 1a81d3d1e8a983e0551edf485f4ac1a8cdd7fea4
- Cited by
- 2,979
- References
- 5
- Citations by year
- 2026: 106 · 2025: 116 · 2024: 127 · 2023: 169 · 2022: 179 · 2021: 206 · 2020: 236 · 2019: 255 · 2018: 262 · 2017: 271 · 2016: 283 · 2015: 262 · 2014: 226 · 2013: 133 · 2012: 62
- FWCI
- 210.63
- Type
- journal article
- Language
- en
- Update notices
- correction (2012-05-02)
- Open access
- bronze
- doi.org
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health and Medical Research Impacts
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Health Sciences › Nursing › Issues, ethics and legal aspects › Biomedical Ethics and Regulation
- Keywords
- preclinical cancer research, research standards, publication practices, research incentives, translational research, clinical trials
- MeSH terms
- Animals, Clinical Trials as Topic, Drug Evaluation, Preclinical, Humans, Neoplasms, Research Design, Reproducibility of Results, Survival Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.