Raise standards for preclinical cancer research
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index; 3 of the 9 sources its claims cite were checked for support.
Overview
CoChat Summary
TL;DR
This Nature Comment argues that many landmark preclinical oncology findings are not reproducible and calls for higher standards in methods, reporting, and incentives. It reports that Amgen confirmed only 6 of 53 landmark papers (11%) and cites Bayer’s finding that about 25% of published preclinical studies could be validated.
Key findings
- In Amgen’s ten-year retrospective analysis, scientific findings were confirmed in only 6 of 53 “landmark” preclinical cancer papers (11%); non-reproduced studies often lacked blinding, relied on single experiments such as one Western blot, or presented selected data.
- Bayer HealthCare reported that only about 25% of published preclinical studies could be validated to the point at which projects could continue; published cancer research represented 70% of the studies analysed in that report, with possible overlap with Amgen’s 53 papers.
- Citation data showed non-reproduced articles in journals with impact factor >20 had a mean of 248 citations (range 3–800) versus 231 (range 82–519) for reproduced articles; for impact factor 5–19, non-reproduced articles had a mean of 169 citations (range 6–1,909) versus 13 (range 3–24) for reproduced articles.
- The authors recommend reporting negative data, blinding preclinical experiments, validating reagents, repeating critical experiments, avoiding single cell lines or models, requiring patient-selection biomarkers, and changing academic incentives.
Methods
- This is a Nature Comment/perspective, not a primary experimental study; it synthesizes Amgen’s ten-year retrospective analysis of experiments performed prospectively and cites a Bayer HealthCare report.
- Amgen selected 53 “landmark” papers describing completely new approaches or alternative clinical uses, attempted confirmation, used additional models, contacted original authors, exchanged reagents, and repeated experiments under the authors’ direction, occasionally in the original investigator’s laboratory.
- The analysis reports counts of confirmed findings and citation metrics from Google Scholar, May 2011; the recommendations are proposed changes rather than tested interventions.
Limitations
- Stated: validation attempts may have failed because of technical differences or difficulties despite efforts to ensure otherwise; additional models were used; when findings could not be reproduced, authors were contacted and experiments were repeated under their direction.
- Stated: papers were deliberately selected as “landmark” and described something completely new, so some data might not hold up; Bayer’s analysed studies may overlap with Amgen’s 53 papers.
- Evident: as a Comment, the text provides no detailed systematic protocol, inclusion criteria, statistical methods, or full dataset; the citation analysis is descriptive and limited to two impact-factor strata.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 1 Not valid · 6 Not checked · 12 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Sadly, clinical trials in oncology have the highest failure rate compared with other therapeutic areas.”
What the sources say
- Partly supportedcites [1] Hutchinson, L. & Kirk, R. Nature Rev. Clin. Oncol. 8, 189–190 (2011). sourceExperimental — accuracy not yet measured
The opening passage of our copy (labelled Abstract; page unavailable) says cancer-drug attrition is much higher than in other therapeutic areas, citing 5% of anticancer agents versus 20% of cardiovascular agents reaching licensure. That comparison covers development from preclinical activity through phase III, not failure rates specifically among clinical trials; it also does not explicitly establish a highest-rate ranking. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Although we in the cancer field hoped that this would lead to more effective drugs, historically, our ability to translate cancer research to clinical success has been remarkably low 1 .”
What the sources say
- Supportedcites [1] Hutchinson, L. & Kirk, R. Nature Rev. Clin. Oncol. 8, 189–190 (2011). source
The opening passage (labeled Abstract in our copy; no page given) reports that only 5% of agents with preclinical anticancer activity are licensed after demonstrating sufficient efficacy in phase III testing. It also says cancer drug attrition is much higher than in other therapeutic areas, supporting the claim of historically low translation to clinical success. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In studies for which findings could be reproduced, authors had paid close attention to controls, reagents, investigator bias and describing the complete data set.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For results that could not be reproduced, however, data were not routinely analysed by investigators blinded to the experimental versus control groups.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Investigators frequently presented the results of one experiment, such as a single Western-blot analysis.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
12 more claims
Create a free account to see all 22 claims and every source verdict.
- It may state two separate claims as one. 2
- Probably not a substantial claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“there are other more common sources of unreliability in the literature (41)(42)(43)(44)”
From Misconduct accounts for the majority of retracted scientific publications (2012) · cites this paper as [42]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“many established scientific results in the social (49, 50) and biomedical (51-53) sciences cannot readily be replicated.”
From Misinformation in and about science (2021) · cites this paper as [51]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A landmark study conducted by Glenn Begley and his team in 2012 concluded that most cancer research studies were not reproducible, by alarmingly reporting that 47 of 53 published findings could not be replicated [13].”
From Guidelines for the use of flow cytometry and cell sorting in immunological studies (third edition) (2021) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In cell biology, two industrial laboratories reported success replicating the original results of landmark findings in only 11% and 25% of the attempted cases ( 10, 11 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Concerns about reproducibility are widespread across disciplines ( 921 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [10]
Experimental — accuracy not yet measured
13 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888
Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010271 · Cited by 805
High drug attrition rates—where are we going wrong?
Hutchinson, L., & Kirk, R. (2011) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2011.34 · Cited by 517
Drug development and clinical trials—the path to an approved cancer drug
Rubin, E. H., & Gilliland, D. G. (2012) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2012.22 · Cited by 135
Raising the bar for cancer therapy models
Francia, G., & Kerbel, R. S. (2010) · Nature Biotechnology · doi:10.1038/nbt0610-561 · Cited by 54
Cited by
Items 1,701–1,800 of 2,889
Robustness of arterial blood gas analysis for assessment of respiratory safety pharmacology in rats
Whiteside, G. T., Hummel, M., Boulet, J. M., Beyenhof, J., Strenkowski, B. P., John, J. D., Knappenberger, T., Maselli, H., & Koetzner, L. (2015) · Journal of Pharmacological and Toxicological Methods · doi:10.1016/j.vascn.2015.11.001 · Cited by 9
No preclinical rationale for IGF1R directed therapy in chondrosarcoma of bone
Peterse, E. F. P., Cleven, A. H. G., de Jong, Y., Bruijn, I. H. B., Fletcher, J. A., Danen, E. H. J., Cleton-Jansen, A., & Bovée, J. V. M. G. (2016) · BMC Cancer · doi:10.1186/s12885-016-2522-8 · Cited by 6
“Lack of reproducibility is a well described phenomenon in preclinical studies with antibodies [42, 43].”
BackgroundFundamentally low reproducibility in molecular genetic cancer research
Алексеенко, И. В., Pleshkan, V. V., MONASTYRSKAYA, G. S., Kuzmich, A. I., Snezhkov, E. V., Didych, D. A., & Sverdlov, E. D. (2016) · Russian Journal of Genetics · doi:10.1134/s1022795416070036 · Cited by 6
“Even taking into account the specificity of preclinical studies, that was a shocking result [22, 23].”
BackgroundColeman, B. G., Johnson, T. M., Erley, K. J., Topolski, R. L., Rethman, M. P., & Lancaster, D. D. (2016) · Journal of Dental Education · doi:10.1002/j.0022-0337.2016.80.10.tb06198.x · Cited by 8
Identifying impact of software dependencies on replicability of biomedical workflows
Miksa, T., Rauber, A., & Mina, E. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.10.011 · Cited by 6
Critical Aspects of Integrated Nonclinical Drug Development
Koch, E., & Plassmann, S. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-803620-4.00002-5 · Cited by 6
Operationalizing Precision Cardiovascular Medicine: Three Innovations
Wang, J., Aboulhosn, J., Hofer, I. S., Mahajan, A., Wang, Y., & Vondriska, T. M. (2016) · Circulation Research · doi:10.1161/circresaha.116.309776 · Cited by 6
“We propose 3 key areas of focus: (1) healthcare delivery redesign; (2) physician and patient engagement; and (3) changes to the discovery science pipeline.”
BackgroundMethodDesigning Successful Proteomics Experiments
Ruderman, D. (2016) · Methods in molecular biology · doi:10.1007/978-1-4939-6747-6_19 · Cited by 7
Digital health: A science at crossroads
Lovis, C. (2017) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2017.12.006 · Cited by 7
Polymeric micelles as a versatile tool in oral chemotherapy
Ribeiro, A. M., Amaral, C., Veiga, F., & Figueiras, A. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-813627-0.00008-9 · Cited by 11
Simmons, J. E., Eide, I., Rice, G. E., & Feder, P. I. (2018) · Springer eBooks · doi:10.1007/978-3-319-56234-6_13 · Cited by 8
Understanding and Mitigating the Replication Crisis, for Environmental Epidemiologists
Bartell, S. M. (2019) · Current Environmental Health Reports · doi:10.1007/s40572-019-0225-4 · Cited by 8
Smith, C. J., & Perfetti, T. A. (2019) · Toxicology Research and Application · doi:10.1177/2397847318822137 · Cited by 6
Research and Testing Without Animals: Where Are We Now and Where Are We Heading?
Härtung, T. (2019) · BRILL eBooks · doi:10.1163/9789004391192_029 · Cited by 9
Park, J. S., Kang, H. J., Lee, B., Choi, J. S., & Yim, J. (2020) · ACS Applied Polymer Materials · doi:10.1021/acsapm.9b01166 · Cited by 6
CHiSEL: a user-oriented framework for simplifing database evolution
Schuler, R. E., & Kesselman, C. (2020) · Distributed and Parallel Databases · doi:10.1007/s10619-020-07314-x · Cited by 7
Resource Reallocation and Innovation: Converting Enterprise Risks into Opportunities
Dutz, M. A. (2014) · Springer eBooks · doi:10.1007/978-3-319-07533-4_10 · Cited by 8
The reproducibility crisis and statistical review of clinical and translational studies
Block, J. A. (2021) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2021.04.008 · Cited by 6
Ease of articulation: A replication
Shuster, L., & Cottrill, C. (2015) · Journal of Communication Disorders · doi:10.1016/j.jcomdis.2015.05.004 · Cited by 6
Habermann, D., Kharimzadeh, H., Walker, A., Li, Y., Yang, R., Kaiser, R., Brumme, Z. L., Timm, J., Roggendorf, M., & Hoffmann, D. E. (2022) · Bioinformatics · doi:10.1093/bioinformatics/btac134 · Cited by 8
Harremoës, P. (2019) · Publications · doi:10.3390/publications7030053 · Cited by 7
“In a meta-analysis from 2012 [21], only 11% of preclinical results in cancer research could be reproduced, and these results were published in high impact journals, while the papers all had a high number of citations.”
ResultFishburn, C. S. (2014) · Science-Business eXchange · doi:10.1038/scibx.2014.275 · Cited by 7
Haslberger, M., Schorr, S. G., Strech, D., & Haven, T. L. (2022) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15503 · Cited by 7
Sumpter, J. P., Scott, A. P., & Katsiadaki, I. (2016) · Chemosphere · doi:10.1016/j.chemosphere.2016.08.049 · Cited by 6
“In the biomedical field, rigorous attempts at repeating high-profile studies have shown that the results of many of those studies are not repeatable (Ioannidis, 2005; Begley and Ellis, 2012).”
BackgroundExamining the Reproducibility of 6 Published Studies in Public Health Services and Systems Research
Harris, J. K., Wondmeneh, S. B., Zhao, Y., & Leider, J. P. (2018) · Journal of Public Health Management and Practice · doi:10.1097/phh.0000000000000694 · Cited by 8
“To begin to shift the culture toward reproducibility, there are many strategies researchers, journals, employers of researchers, funders, and others can use.(23,24) Specifically, reproducibility increases when researchers (1) describe the methods used in enough detail in publications so that they can be reproduced;(25…”
BackgroundEpitope mapping of commercial antibodies that detect myocilin
Patterson-Orazem, A. C., Hill, S. E., Fautsch, M. P., & Lieberman, R. L. (2018) · Experimental Eye Research · doi:10.1016/j.exer.2018.05.002 · Cited by 6
“The rigorous validation of research reagents, such as antibodies, has become a high priority for NIH and other funding bodies due to the multitude of research studies that cannot be independently replicated (Begley and Ellis, 2012).”
BackgroundMarrone, M. T., Joshu, C. E., Peskoe, S. B., Marzo, A. M. D., Heaphy, C. M., Lupold, S. E., Meeker, A. K., & Platz, E. A. (2018) · Clinical Chemistry · doi:10.1373/clinchem.2018.293365 · Cited by 7
“That up to 90% of biomedical research findings cannot be replicated points to fundamental flaws in translational research (12, 13).”
BackgroundThe Division of Replication Labor
Romero, F. (2020) · Philosophy of Science · doi:10.1086/710625 · Cited by 13
Robust latent-variable interpretation of in vivo regression models by nested resampling
Caulk, A. W., & Janes, K. A. (2019) · Scientific Reports · doi:10.1038/s41598-019-55796-2 · Cited by 5
Wagenmakers, E., & Pashler, H. (2012) · Cited by 6
Acta Diabetologica is 50 and well: long live Acta!
Porta, M. (2014) · Acta Diabetologica · doi:10.1007/s00592-014-0554-6 · Cited by 6
“Staff at a leading biotech firm were able to replicate the results of only 6 out of 53 landmark papers in cancer [3] M.”
BackgroundHoughton, P. J. (2013) · Expert Review of Anticancer Therapy · doi:10.1586/14737140.2013.845094 · Cited by 7
Publishing confirming and non-confirming data
Alberts, B., & Kamb, A. (2016) · F1000Research · doi:10.12688/f1000research.7847.1 · Cited by 6
US societies push back against NIH reproducibility guidelines
Baker, M. (2015) · Nature · doi:10.1038/nature.2015.17354 · Cited by 6
Genotypic variability enhances the reproducibility of an ecological studyPreprint
Milcu, A., Freitas, R. P., Ellison, A. M., Blouin, M., Scheu, S., Girin, T., Freschet, G. T., Rose, L., Scherer‐Lorenzen, M., Barot, S., et al. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/080119 · Cited by 6
How to End Selective Reporting in Animal Research
ter Riet, G., & Bouter, L. M. (2016) · Methods and principles in medicinal chemistry · doi:10.1002/9783527695881.ch5 · Cited by 6
Smaldino, P. E., Turner, M. A., & Kallens, P. C. (2019) · Royal Society Open Science · doi:10.1098/rsos.191249 · Cited by 6
Using Ontologies for Verification and Validation of Workflow-Based ExperimentsPreprint
Miksa, T., & Rauber, A. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3199133 · Cited by 6
Chen, L., Benjamin, R., Guo, Y., Lai, A., & Heine, S. J. (2023) · Research Square · doi:10.21203/rs.3.rs-1254756/v1 · Cited by 5
Reproducibility of mean estimators under ranked set sampling
Rehman, S. A., Coolen‐Maturi, T., Coolen, F. P. A., & Shabbir, J. (2024) · Franklin Open · doi:10.1016/j.fraope.2024.100139 · Cited by 6
Digitalization of the natural sciences: Design science research and computational science
Storey, V. C., & Baskerville, R. (2024) · Decision Support Systems · doi:10.1016/j.dss.2024.114368 · Cited by 10
Wang, B., Kim, H.-J., Bradley, K. M., Chen, C., McLendon, C., Yang, Z., & Benner, S. A. (2024) · Journal of the American Chemical Society · doi:10.1021/jacs.4c11043 · Cited by 5
A Philosophy of the Humanities
Grimm, S. R., Peels, R., & van Woudenberg, R. (2025) · doi:10.1093/9780197749647.001.0001 · Cited by 6
Eastman, R. T., Rusinova, R., Herold, K. F., Huang, X.-P., Voss, T. C., White, A. D., Hemmings, H. C., Andersen, O. S., & Dahlin, J. L. (2025) · Journal of Medicinal Chemistry · doi:10.1021/acs.jmedchem.5c01610 · Cited by 6
“170 Though seemingly obvious, with the benefit of hindsight, such studies were not done in the case of IVM, which led to clinical studies that should not have been done because they were based on incomplete mechanistic understanding, which may compromise clinical equipoise 171 and subject patients…”
InfluentialOn “Confirmatory” Methodological Research in Statistics and Related Fields
Lange, F. J. D., Wilcke, J. C., Hoffmann, S., Herrmann, M., & Boulesteix, A. (2025) · Statistics in Medicine · doi:10.1002/sim.70303 · Cited by 10
“This proliferation of methods and the lack of evidence to guide”
Lamponi, S. (2026) · Life · doi:10.3390/life16040538 · Cited by 8
Maneuvrier, A. (2024) · Virtual Reality · doi:10.1007/s10055-024-00993-2 · Cited by 7
“This replicability crisis extends far beyond psychology and can be observed in many other fields, such as pharmacology (Prinz et al. 2011), medicine (Begley & Ellis 2012), or even economics (Bergh et al. 2017) and the social sciences (Camerer et al. 2018).”
Ethics of gene and cell therapy development for neurologic disorders
Kimmelman, J. (2024) · Handbook of clinical neurology · doi:10.1016/b978-0-323-90120-8.00002-2 · Cited by 6
A systematic review of pre-registration in autism research journals
Poole, D., Linden, A., Sedgewick, F., Allchin, O., & Hobson, H. (2024) · Autism · doi:10.1177/13623613241308312 · Cited by 5
“A number of registries for hosting pre-registration emerged, following the publication of high-profile papers highlighting that questionable research practices and false positives were likely widespread across research fields (Begley & Ellis, 2012; Ioannidis, 2005; Simmons et al., 2011).”
InfluentialKerpel‐Fronius, S., Becker, A. L., & Members of the IFAPP Ethics Working Group (2025) · Pharmaceutical Medicine · doi:10.1007/s40290-024-00547-6 · Cited by 7
Ncube, K. N., van den Bout, I., Willers, C., Gouws, C., & Cordier, W. (2025) · International Journal of Molecular Sciences · doi:10.3390/ijms26157503 · Cited by 8
“Various factors contribute to these high attrition rates, and the historical inability to efficiently translate preclinical research into clinical success is one of the main driving factors [13].”
DOIs for DICOM Raw Images: Enabling Science Reproducibility
Bourne, P. E. (2015) · Radiology · doi:10.1148/radiol.15150144 · Cited by 5
The STAT3-Ser/Hes3 signaling axis in cancer
Poser, S., Park, D. M., & Androutsellis-Theotokis, A. (2014) · Frontiers in bioscience · doi:10.2741/4238 · Cited by 4
Reanalysis of an oft-cited paper on honeybee magnetoreception reveals random behavior
Baltzley, M. J., & Nabity, M. (2018) · Journal of Experimental Biology · doi:10.1242/jeb.185454 · Cited by 6
“Concerns about the reproducibility of results have also gained significant attention in recent years, particularly in medical research and in psychology research (Begley and Ellis, 2012; Open Science Collaboration, 2015; Baker, 2016; Johnson et al., 2017).”
BackgroundPlan S: In service or disservice to society?
Guzik, T. J., & Ahluwalia, A. (2019) · British Journal of Pharmacology · doi:10.1111/bph.14590 · Cited by 5
Future of processing and facilitating change and learning
Löffler‐Stastka, H., & Steinmair, D. (2021) · World Journal of Psychiatry · doi:10.5498/wjp.v11.i9.507 · Cited by 6
Gannon, F. (2020) · EMBO Reports · doi:10.15252/embr.202051371 · Cited by 5
“The public is sensitized already by the claim that a high percentage of published papers cannot be replicated for a variety of reasons (Begley & Ellis, 2012; Baker & Dolgin, 2017).”
BackgroundData publication consensus and controversies
Kratz, J. E., & Strasser, C. A. (2014) · F1000Research · doi:10.12688/f1000research.3979.2 · Cited by 7
Ott, A. W., Sol‐Church, K., Deshpande, G. M., Knudtson, K. L., Meyn, S. M., Mische, S. M., Taatjes, D. J., Sturges, M., & Gregory, C. W. (2022) · Journal of Biomolecular Techniques JBT · doi:10.7171/3fc1f5fe.fa789303 · Cited by 8
Produção Científica: um Guia Prático
Sampaio, M. I. C., Sabadini, A. A. Z. P., & Koller, S. H. (2022) · Universidade de São Paulo. Instituto de Psicologia eBooks · doi:10.11606/9786587596280 · Cited by 21
The Moral Implications of Human and Animal Vulnerability
Martin, A. K. (2023) · Springer eBooks · doi:10.1007/978-3-031-25078-1 · Cited by 12
Mosconi, G., de Carvalho, A. F. P., Syed, H. A., Randall, D. W., Karasti, H., & Pipek, V. (2023) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-023-09467-6 · Cited by 12
“Consequently, as already highlighted by previous literature (Begley and Ellis, 2012; Collaboration, 2012; Fecher et al., 2017), data curation and sharing practices, needed to meet the OS goals, are perceived by many as an unrewarding chore, especially when targeted at preserving and sharing data…”
Shajib, M. S., Futrega, K., Davies, A. M., Franco, R. A., McKenna, E., Guillesser, B., Klein, T. J., Crawford, R., & Doran, M. R. (2023) · Journal of The Royal Society Interface · doi:10.1098/rsif.2023.0468 · Cited by 7
“The reproducibility crisis refers to the observation that many scientific studies are difficult or impossible to replicate or reproduce, including many studies reported in high-impact journals [7,8].”
BackgroundScheidecker, B., Poulain, S., Sugimoto, M., Kido, T., Kawanishi, T., Miyajima, A., Kim, S. H., Arakawa, H., Kato, Y., Nishikawa, M., et al. (2024) · Biofabrication · doi:10.1088/1758-5090/ad30c5 · Cited by 6
Karabağ, S. F., Berggren, C., Pielaszkiewicz, J., & Gerdin, B. (2024) · Journal of Academic Ethics · doi:10.1007/s10805-024-09520-z · Cited by 12
“…of grey zone practices, Extended author information available on the last page of the article low replicability of findings even in top journals (Begley & Ellis, 2012; Camerer Colin et al., 2016; Chalkia et al., 2020; Open Science Collaboration, 2015; Prinz et al., 2011) have brought attention…”
Starbuck, W. H., Schwab, A., & Hodgkinson, G. P. (2024) · Scandinavian Journal of Management · doi:10.1016/j.scaman.2024.101355 · Cited by 5
Predicting the replicability of social and behavioural science claims in COVID-19 preprints
Marcoci, A., Wilkinson, D. P., Vercammen, A., Wintle, B. C., Abatayo, A. L., Baskin, E., Berkman, H., Buchanan, E. M., Capitán, S., Capitán, T., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01961-1 · Cited by 21
“The models are grouped into two categories: behavioural models that were fit to the assessments of all 100 papers in the dataset, and accuracy models that were fit only to the subset of papers with a replication outcome and a replication power of ≥0.8 at a two-sided α -level of 0.05 (29 papers) and a single-sided α -l…”
InfluentialMartins, A. C. R. (2013) · Intelligent systems reference library · doi:10.1007/978-3-642-39149-1_3 · Cited by 6
“Furthermore, there are fields in which published results have been consistently shown to be wrong[38, 4, 78], exactly due to the way the Science and publications are structured.”
BackgroundExpectations for methodology and translation of animal research: a survey of health care workers
Joffe, A. R., Bara, M., Anton, N. R., & Nobis, N. (2015) · BMC Medical Ethics · doi:10.1186/s12910-015-0024-x · Cited by 6
“Extensive AR in the fields of sepsis [19-21], stroke [22,23], spinal cord injury [24,25], traumatic brain injury [26], cancer [27,28], degenerative neurological diseases [29,30], acquired immunodeficiency syndrome [31], asthma [32], and other fields [15-18] has translated to humans in 0-5% of cases.”
BackgroundSome Salt with Your Statin, Professor?
Macleod, M. R. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001768 · Cited by 5
“Scientists in the haematology and oncology departments at Amgen were able to replicate findings in only six out of 53 publications identified as ‘‘landmark’’ studies [21].”
BackgroundOnly Human: Scientists, Systems, and Suspect Statistics
Hardwicke, T. E., Jameel, L., Jones, M., Walczak, E., & Magis‐Weinberg, L. (2014) · Opticon1826 · doi:10.5334/opt.ch · Cited by 8
“…research findings are false’ (Ioannidis, 2005) and subsequent efforts to replicate existing findings have suggested that the suspected ‘reproducibility crisis’ is not just theoretically plausible; it is an empirical reality (Begley & Ellis, 2012; Prinz, Schlange, & Asadullah, 2011; Gilbert, 2014).”
Data Infrastructures for Estuarine and Coastal Ecological Syntheses
Thessen, A., Fertig, B., Jarvis, J. C., & Rhodes, A. C. E. (2015) · Estuaries and Coasts · doi:10.1007/s12237-015-0045-1 · Cited by 6
Scientific Misconduct: Falsification, Fabrication, and Misappropriation of Credit
Vaux, D. L. (2015) · Springer eBooks · doi:10.1007/978-981-287-079-7_26-1 · Cited by 6
Engineering in vitro complex pathophysiologies for drug discovery purposes
Gaspar, D., & Zeugolis, D. I. (2016) · Drug Discovery Today · doi:10.1016/j.drudis.2016.08.007 · Cited by 5
Antibodies That Work Again and Again and Again
Simpson, K., & Browning, M. D. (2017) · Methods in molecular biology · doi:10.1007/978-1-4939-6759-9_2 · Cited by 8
Keane, M. J., & Berg, C. (2017) · British Journal of Anaesthesia · doi:10.1093/bja/aex334 · Cited by 5
Glass, D. J., & Flier, J. S. (2017) · Cell Metabolism · doi:10.1016/j.cmet.2017.10.010 · Cited by 5
“Increasingly there have been findings that a significant component of published biological science research cannot be reproduced (Begley and Ellis, 2012; Ioannidis, 2005; Prinz et al., 2011).”
BackgroundReproducibility in Biomedical Research
Mullane, K., Curtis, M. J., & Williams, M. J. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-804725-5.00001-x · Cited by 11
It has not been proven why or that most research findings are false
Ashton, J. C. (2018) · Medical Hypotheses · doi:10.1016/j.mehy.2018.02.004 · Cited by 4
The “false-positive” conundrum in the NTP 2-year rodent cancer study database
Smith, C. J., & Perfetti, T. A. (2018) · Toxicology Research and Application · doi:10.1177/2397847318772839 · Cited by 5
The 3R approach to experimental dermatology
Schneider, M. R., & Pincelli, C. (2018) · Experimental Dermatology · doi:10.1111/exd.13673 · Cited by 5
Desquilbet, L. (2020) · Journal of the American Veterinary Medical Association · doi:10.2460/javma.256.2.187 · Cited by 6
Fu, S., Carlson, L., Peterson, K. J., Wang, N., Zhou, X., Peng, S., Jiang, J., Wang, Y., Sauver, J. L. S., & Liu, H. (2020) · PubMed · Cited by 5
Tebbe, J., Humble, E., Stoffel, M. A., Tewes, L. J., Müller, C., Forcada, J., Caspers, B. A., & Hoffman, J. I. (2020) · PeerJ · doi:10.7717/peerj.10131 · Cited by 6
Mean comparisons and power calculations to ensure reproducibility in preclinical drug discovery
Novick, S. J., & Zhang, T. (2020) · Statistics in Medicine · doi:10.1002/sim.8848 · Cited by 5
“In this work, the concept behind model (3) is applied to the testing and calculation of statistical power for a single study so that reproducibility of results may be claimed from a single preclinical study.”
BackgroundMethodInfluentialRamarao, P. (2021) · Springer eBooks · doi:10.1007/978-981-15-5534-3_17 · Cited by 6
Designing Product Development Contracts in the Presence of Managerial Lobbying
Bao, Y., Shi, M., & Kalra, A. (2022) · Management Science · doi:10.1287/mnsc.2021.4217 · Cited by 5
(2012) · Nature Medicine · doi:10.1038/nm.2978 · Cited by 5
Multiple deadlocks in the development of nonprofit drugs
Cvek, B. (2022) · Drug Discovery Today · doi:10.1016/j.drudis.2022.06.001 · Cited by 5
“According to the 10-year efforts of scientists at Amgen, 89% of the studies they examined in the field of hematology and oncology could not be reproduced.(7) The causes of this are related to serious questions of the sociology of science and the ability of biomedicine to solve its long-known systemic problems.”
BackgroundCausio, F. A., Hoxhaj, I., Beccia, F., Marcantonio, M. D., Strohäker, T., Cadeddu, C., Ricciardi, W., & Boccia, S. (2023) · Personalized Medicine · doi:10.2217/pme-2023-0003 · Cited by 4
“Several actors are active in so-called ‘basic research’, thanks to the progress of technology and the vast amount of data and research papers available in recent years, proving that this sector is flourishing despite challenges raised by the increased attention toward preclinical research [11] .”
BackgroundA meta-analysis of the effect of protein synthesis inhibitors on rodent fear conditioning
Carneiro, C. F. D., Amorim, F. E., & Amaral, O. B. (2023) · Peer Community Journal · doi:10.24072/pcjournal.339 · Cited by 7
Multistage Carcinogenesis: Cell and Animal Models
Kulesz-Martin, M. F., Ouyang, X., Barling, A. N., Gallegos, J. R., Liu, Y., & Medler, T. R. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-801238-3.02218-2 · Cited by 5
A Virtual Crowdsourcing Community for Open Collaboration in Science Processes
Gil, Y., Hauder, M., Michel, F., & Ratnakar, V. (2015) · Americas Conference on Information Systems · Cited by 8
Experimental toxicology: Issues of statistics, experimental design, and replication
Briner, W., & Kirwan, J. R. (2016) · NeuroToxicology · doi:10.1016/j.neuro.2016.11.002 · Cited by 7
227th ENMC International Workshop:
Willmann, R., Buccella, F., Luca, A. D., Grounds, M. D., Versnel, J., Vroom, E., Ribeiro, D., Ambrosini, A., Pavlath, G. K., Porter, J. D., et al. (2017) · Neuromuscular Disorders · doi:10.1016/j.nmd.2017.11.002 · Cited by 5
Lesk, M., Mattern, J. B., & Moulaison, H. L. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-15742-5_14 · Cited by 6
Biology is the root of variability: cautionary tales in Caenorhabditis elegans biology
Pho, K. B., & MacNeil, L. T. (2019) · Biochemical Society Transactions · doi:10.1042/bst20190001 · Cited by 7
Hedayat, K. M., & Lapraz, J.-C. (2019) · Elsevier eBooks · doi:10.1016/b978-0-12-816903-2.00001-x · Cited by 5
Simulation enabled search for explanatory mechanisms of the fracture healing process
Kennedy, R., Marmor, M. T., Marcucio, R., & Hunt, C. A. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005980 · Cited by 5
Guidelines and Initiatives for Good Research Practice
Kabitzke, P., Cheng, K., & Altevogt, B. M. (2019) · Handbook of experimental pharmacology · doi:10.1007/164_2019_275 · Cited by 4
“For example, in 2011, Bayer HealthCare was unable to validate the results of 43 out of 67 studies (Prinz et al. 2011), while Amgen reported its inability to validate 47 out of 53 seminal publications that claimed a new drug discovery in oncology (Begley and Ellis 2012).”
Result
Metadata
- Authors
- Begley, C. G., & Ellis, L. M.
- Affiliations
- Amgen (United States) (US)
- The University of Texas MD Anderson Cancer Center (US)
- Corresponding author
- Ellis, Lee M.
- Venue
- Nature
- Published
- 2012-03-28
- Volume
- 483
- Issue
- 7391
- Pages
- 531-533
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W1987777080
- Semantic Scholar
- 1a81d3d1e8a983e0551edf485f4ac1a8cdd7fea4
- Cited by
- 2,979
- References
- 5
- Citations by year
- 2026: 106 · 2025: 116 · 2024: 127 · 2023: 169 · 2022: 179 · 2021: 206 · 2020: 236 · 2019: 255 · 2018: 262 · 2017: 271 · 2016: 283 · 2015: 262 · 2014: 226 · 2013: 133 · 2012: 62
- FWCI
- 210.63
- Type
- journal article
- Language
- en
- Update notices
- correction (2012-05-02)
- Open access
- bronze
- doi.org
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health and Medical Research Impacts
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Health Sciences › Nursing › Issues, ethics and legal aspects › Biomedical Ethics and Regulation
- Keywords
- preclinical cancer research, research standards, publication practices, research incentives, translational research, clinical trials
- MeSH terms
- Animals, Clinical Trials as Topic, Drug Evaluation, Preclinical, Humans, Neoplasms, Research Design, Reproducibility of Results, Survival Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.