Raise standards for preclinical cancer research
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index; 3 of the 9 sources its claims cite were checked for support.
Overview
CoChat Summary
TL;DR
This Nature Comment argues that many landmark preclinical oncology findings are not reproducible and calls for higher standards in methods, reporting, and incentives. It reports that Amgen confirmed only 6 of 53 landmark papers (11%) and cites Bayer’s finding that about 25% of published preclinical studies could be validated.
Key findings
- In Amgen’s ten-year retrospective analysis, scientific findings were confirmed in only 6 of 53 “landmark” preclinical cancer papers (11%); non-reproduced studies often lacked blinding, relied on single experiments such as one Western blot, or presented selected data.
- Bayer HealthCare reported that only about 25% of published preclinical studies could be validated to the point at which projects could continue; published cancer research represented 70% of the studies analysed in that report, with possible overlap with Amgen’s 53 papers.
- Citation data showed non-reproduced articles in journals with impact factor >20 had a mean of 248 citations (range 3–800) versus 231 (range 82–519) for reproduced articles; for impact factor 5–19, non-reproduced articles had a mean of 169 citations (range 6–1,909) versus 13 (range 3–24) for reproduced articles.
- The authors recommend reporting negative data, blinding preclinical experiments, validating reagents, repeating critical experiments, avoiding single cell lines or models, requiring patient-selection biomarkers, and changing academic incentives.
Methods
- This is a Nature Comment/perspective, not a primary experimental study; it synthesizes Amgen’s ten-year retrospective analysis of experiments performed prospectively and cites a Bayer HealthCare report.
- Amgen selected 53 “landmark” papers describing completely new approaches or alternative clinical uses, attempted confirmation, used additional models, contacted original authors, exchanged reagents, and repeated experiments under the authors’ direction, occasionally in the original investigator’s laboratory.
- The analysis reports counts of confirmed findings and citation metrics from Google Scholar, May 2011; the recommendations are proposed changes rather than tested interventions.
Limitations
- Stated: validation attempts may have failed because of technical differences or difficulties despite efforts to ensure otherwise; additional models were used; when findings could not be reproduced, authors were contacted and experiments were repeated under their direction.
- Stated: papers were deliberately selected as “landmark” and described something completely new, so some data might not hold up; Bayer’s analysed studies may overlap with Amgen’s 53 papers.
- Evident: as a Comment, the text provides no detailed systematic protocol, inclusion criteria, statistical methods, or full dataset; the citation analysis is descriptive and limited to two impact-factor strata.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 1 Not valid · 6 Not checked · 12 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Sadly, clinical trials in oncology have the highest failure rate compared with other therapeutic areas.”
What the sources say
- Partly supportedcites [1] Hutchinson, L. & Kirk, R. Nature Rev. Clin. Oncol. 8, 189–190 (2011). sourceExperimental — accuracy not yet measured
The opening passage of our copy (labelled Abstract; page unavailable) says cancer-drug attrition is much higher than in other therapeutic areas, citing 5% of anticancer agents versus 20% of cardiovascular agents reaching licensure. That comparison covers development from preclinical activity through phase III, not failure rates specifically among clinical trials; it also does not explicitly establish a highest-rate ranking. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Although we in the cancer field hoped that this would lead to more effective drugs, historically, our ability to translate cancer research to clinical success has been remarkably low 1 .”
What the sources say
- Supportedcites [1] Hutchinson, L. & Kirk, R. Nature Rev. Clin. Oncol. 8, 189–190 (2011). source
The opening passage (labeled Abstract in our copy; no page given) reports that only 5% of agents with preclinical anticancer activity are licensed after demonstrating sufficient efficacy in phase III testing. It also says cancer drug attrition is much higher than in other therapeutic areas, supporting the claim of historically low translation to clinical success. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In studies for which findings could be reproduced, authors had paid close attention to controls, reagents, investigator bias and describing the complete data set.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For results that could not be reproduced, however, data were not routinely analysed by investigators blinded to the experimental versus control groups.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Investigators frequently presented the results of one experiment, such as a single Western-blot analysis.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
12 more claims
Create a free account to see all 22 claims and every source verdict.
- It may state two separate claims as one. 2
- Probably not a substantial claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“there are other more common sources of unreliability in the literature (41)(42)(43)(44)”
From Misconduct accounts for the majority of retracted scientific publications (2012) · cites this paper as [42]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“many established scientific results in the social (49, 50) and biomedical (51-53) sciences cannot readily be replicated.”
From Misinformation in and about science (2021) · cites this paper as [51]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A landmark study conducted by Glenn Begley and his team in 2012 concluded that most cancer research studies were not reproducible, by alarmingly reporting that 47 of 53 published findings could not be replicated [13].”
From Guidelines for the use of flow cytometry and cell sorting in immunological studies (third edition) (2021) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In cell biology, two industrial laboratories reported success replicating the original results of landmark findings in only 11% and 25% of the attempted cases ( 10, 11 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Concerns about reproducibility are widespread across disciplines ( 921 ).”
From Estimating the reproducibility of psychological science (2015) · cites this paper as [10]
Experimental — accuracy not yet measured
13 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888
Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010271 · Cited by 805
High drug attrition rates—where are we going wrong?
Hutchinson, L., & Kirk, R. (2011) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2011.34 · Cited by 517
Drug development and clinical trials—the path to an approved cancer drug
Rubin, E. H., & Gilliland, D. G. (2012) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2012.22 · Cited by 135
Raising the bar for cancer therapy models
Francia, G., & Kerbel, R. S. (2010) · Nature Biotechnology · doi:10.1038/nbt0610-561 · Cited by 54
Cited by
Items 1,601–1,700 of 2,889
Is There a Reproducibility Crisis? On the Need for Evidence-based Approaches
Pethick, S., Wass, M. N., & Michaelis, M. (2025) · International Studies in the Philosophy of Science · doi:10.1080/02698595.2025.2538937 · Cited by 11
“…in survey responses or published as ‘Comments or Correspondence’ without providing detailed information (Prinz, Schlange, and Asadullah 2011; Begley and Ellis 2012; Mobley et al. 2013; Baker 2016; Nature Editorial 2018; Hopp and Hoover 2019; Wass, Ray, and Michaelis 2019; Farrar, Ostojić,…”
Gangwal, A., & Lavecchia, A. (2026) · Wiley Interdisciplinary Reviews Computational Molecular Science · doi:10.1002/wcms.70072 · Cited by 13
Promoting research resource identification at JCN
Bandrowski, A. E., Tan, S. C., & Hof, P. R. (2014) · The Journal of Comparative Neurology · doi:10.1002/cne.23585 · Cited by 7
“…scientists, editors, and policymakers alike have all turned recently to the issue of reproducibility in scientific research, focusing on research spanning from the pharmaceutical industry (Begley and Ellis, 2012) to the highest levels of government (Collins and Tabak, 2014; see also McNutt, 2014).”
BackgroundRegistered Reports: A new submission format at Drug and Alcohol Dependence
Munafo, M. R., & Strain, E. C. (2014) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2014.02.699 · Cited by 7
“The problem of the poor reproducibility of many findings in the biomedical sciences has been highlighted recently by failures to replicate a number of key findings (Begley and Ellis, 2012; Prinz et al., 2011).”
BackgroundIdentification of Potential MicroRNA Biomarkers by Meta-analysis
Zhu, H., & Leung, S.-W. (2018) · Methods in molecular biology · doi:10.1007/978-1-4939-7756-7_24 · Cited by 8
Data Heterogeneity: The Enzyme to Catalyze Translational Bioinformatics?
Cahan, E. M., & Khatri, P. (2020) · Journal of Medical Internet Research · doi:10.2196/18044 · Cited by 7
“Up to 89% of studies demonstrating significant preclinical effects of novel molecules are nonreplicable [4], and the translation failure rate of novel interventions demonstrating significant effects preclinically that are never approved for clinical use reaches up to 95% [5].”
BackgroundSkoworonska, M., Blank, A., Centeno, I., et al. (2022) · The Journal of Pathology Clinical Research · doi:10.1002/cjp2.305 · Cited by 7
“Documentation of preanalytical variables for biological material has become increasingly important in recent years due to growing concerns about the reproducibility of research results in the domain of personalized medicine [1,2].”
BackgroundSharma, S., Kumaraswamy, E., & Kumar, S. (2023) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-7346-8_5 · Cited by 9
Advances in iPSC Technology in Neural Disease Modeling, Drug Screening, and Therapy
Dai, S., Qiu, L., Veeraraghavan, V. P., Sheu, C., & Mony, U. (2023) · Current Stem Cell Research & Therapy · doi:10.2174/1574888x18666230608105703 · Cited by 9
Scandal in scientific reform: the breaking and remaking of science
Penders, B. (2024) · Journal of Responsible Innovation · doi:10.1080/23299460.2024.2371172 · Cited by 9
Fast skeletal muscle troponin activator in the dy2J muscular dystrophy model
Miciak, J. J., Warsing, L. C., Tibbs, M. E., Jasper, J. R., Jampel, S. B., Malik, F. I., Tankersley, C. G., & Wagner, K. R. (2013) · Muscle & Nerve · doi:10.1002/mus.23848 · Cited by 7
“In a study by Bayer, less than 40% of published therapy studies could be validated,(32) and more recently, a study by Amgen could only replicate 6 of 53 “landmark” papers in oncology (11%).(33) These studies have led to new NIH guidelines for preclinical research, which encourage the use of clinical trial-type rigor w…”
BackgroundMethodMun, E., Atkins, D. C., & Walters, S. T. (2015) · Psychology of Addictive Behaviors · doi:10.1037/adb0000100 · Cited by 8
Iorns, E. J., Gunn, W., Erath, J., Rodríguez, A., Zhou, J., Benzinou, M., & Initiative, T. R. (2014) · PLoS ONE · doi:10.1371/journal.pone.0114614 · Cited by 7
“However, recent work by the pharmaceutical companies Bayer Healthcare [1] and Amgen [2] suggest that ,75% of academic preclinical biomedical studies may not be reproducible.”
BackgroundYuan, K. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1213 · Cited by 7
“Associated with heterogeneous populations is a substantive concern that many results in medicine, social, and behavioral sciences are not reproducible (Begley, 2013; Begley and Ellis, 2012; Nosek and Lakens, 2014; Open Science Collaboration, 2015; Prinz et al., 2011; Wagenmakers et al., 2012).”
BackgroundReproducibility and transparency in biomedical sciences
Gannot, G., Cutting, M., Fischer, D., & Hsu, L. (2016) · Oral Diseases · doi:10.1111/odi.12588 · Cited by 10
“…company tried to confirm oncologyrelated findings from publications that they identified as ‘landmark’ papers, they found that only 11% of these studies (6 of the 53 assessed) had scientifically reproducible data, although all studies had been cited numerous times (Begley and Ellis, 2012).”
BackgroundRogers, K. V., Bhattacharya, I., Martin, S. W., & Nayak, S. (2017) · Clinical and Translational Science · doi:10.1111/cts.12503 · Cited by 7
Errores conceptuales de estadística más comunes en publicaciones científicas
Chile, U. S. T. , Pinel, R. A. M., Márquez, A. C., & de Caldas, U. (2017) · CES Medicina Veterinaria y Zootecnia · doi:10.21615/cesmvz.12.3.4 · Cited by 10
Brønstad, A., & Sandøe, P. (2019) · Laboratory Animals · doi:10.1177/0023677219841080 · Cited by 12
Cohen, N., Lavie, R. G., Manor, Y., Mimouni, M., Fürst, D. E., & Amarilyo, G. (2019) · Seminars in Arthritis and Rheumatism · doi:10.1016/j.semarthrit.2019.06.006 · Cited by 8
Stanford, N., Scharm, M., Dobson, P. D., Golebiewski, M., Hucka, M., Kothamachu, V. B., Nickerson, D. P., Owen, S., Pahle, J., Wittig, U., et al. (2019) · Methods in molecular biology · doi:10.1007/978-1-4939-9736-7_17 · Cited by 9
A path to next-generation reproducibility in cheminformatics
Clark, R. D. (2019) · Journal of Cheminformatics · doi:10.1186/s13321-019-0385-0 · Cited by 7
A new ethos for science? Exploring emerging DIY science “qualities”
Ferretti, F., & Pereira, Â. G. (2020) · Futures · doi:10.1016/j.futures.2020.102653 · Cited by 9
Upregulation of circular and linear METTL3 and USP3 in colorectal cancer
Alkhizzi, B., Khan, M. I., Al-Ghafari, A. B., & Choudhry, H. (2021) · Oncology Letters · doi:10.3892/ol.2021.12936 · Cited by 9
“Investigations by Prinz et al (23) and by Begley and Ellis (24) revealed that 92 and 89% of the surveyed reverse‑transcription‑quantitative PCR (RT‑qPCR) studies could not be reproduced, respectively, even when the experiments were repeated by the same laboratories in which Correspondence to: Dr Hani Choudhry, Biochem…”
ResultLong, J. A. (2021) · Annals of the International Communication Association · doi:10.1080/23808985.2021.1979421 · Cited by 12
Pyfer, B. J., Hernandez, J. A., Glener, A. D., Cason, R. W., Levinson, H., & Phillips, B. T. (2021) · Annals of Plastic Surgery · doi:10.1097/sap.0000000000003029 · Cited by 7
“Furthermore, the bar for publication of scientific research is ever raising.(5,6) Nuanced understanding of finance, accounting, and quantitative decision sciences has long been known to be a prerequisite for managing health systems,(7) but the need for these and other “soft” skills has become paramount in daily medica…”
BackgroundReproducibility: A Researcher-Centered Definition
Feger, S. S., & Woźniak, P. W. (2022) · Multimodal Technologies and Interaction · doi:10.3390/mti6020017 · Cited by 11
“Common barriers relate to the mismatch between the extensive effort needed to conduct good RDM [25] and the perceived low personal gain for following these practices [26,27], the fear of judgement for erroneous or less-than-perfect quality shared data [9], and the fierce competition in academia [28].”
BackgroundHolst, M., Faust, A., & Strech, D. (2022) · Health Research Policy and Systems · doi:10.1186/s12961-022-00841-2 · Cited by 9
Quantifying Antibody Binding Kinetics on Fixed Cells and Tissues via Fluorescence Lifetime Imaging
Mathur, P., Khartchenko, A. F., Stavrakis, S., Kaigala, G. V., & deMello, A. J. (2022) · Analytical Chemistry · doi:10.1021/acs.analchem.2c01076 · Cited by 7
Brough, D., Murkin, J., Amos, H. E., et al. (2022) · Comparative Medicine · doi:10.30802/aalas-cm-22-000033 · Cited by 7
Analytic transparency is key for reproducibility of agricultural research
Sileshi, G. W. (2023) · CABI Agriculture and Bioscience · doi:10.1186/s43170-023-00144-8 · Cited by 8
“This problem, now called the “reproducibility crisis” or “replication crisis”, has been documented in the agricultural sciences, biomedical sciences, computer science, economics and many other disciplines (Baker 2016; Begley and Ellis 2012; Breznau et al.”
BackgroundResultPäll, T., Luidalepp, H., Tenson, T., & Maiväli, Ü. (2023) · PLoS Biology · doi:10.1371/journal.pbio.3002007 · Cited by 9
Journeys to Failure that Litter the Path to Developing New Cancer Therapeutics
Fojo, T. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.24949 · Cited by 7
Improving access to novel agents for childhood leukemia
Sun, W., Gaynon, P. S., Sposto, R., & Wayne, A. S. (2015) · Cancer · doi:10.1002/cncr.29267 · Cited by 7
“Misinterpretation and low reproducibility of preclinical data are common and can result in the termination of oncology drug development.(28,29) It is important that the predictive power of in vitro and animal model testing for drug screening should never be assumed.”
Triggle, C. R., & Williams, M. J. (2015) · Biochemical Pharmacology · doi:10.1016/j.bcp.2015.08.089 · Cited by 7
Variations in Scientific Data Production: What Can We Learn from #Overlyhonestmethods?
Bezuidenhout, L. M. (2014) · Science and Engineering Ethics · doi:10.1007/s11948-014-9618-9 · Cited by 7
“For example, a 2012 survey done for Nature found that 47 out of 53 medical research papers on the subject of cancer were irreproducible (Begley and Ellis 2012), and similar studies in different fields abound (Mobley et al. 2013).”
BackgroundIs target validation all we need?
Frigault, M. M., & Barrett, J. C. (2014) · Current Opinion in Pharmacology · doi:10.1016/j.coph.2014.09.004 · Cited by 6
Lavania, C., Thulasidasan, S., LaMarca, A., Scofield, J., & Bilmes, J. (2016) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2971648.2971716 · Cited by 8
“An alarming study by Begley [4] showed that 47 out of 53 landmark results in cancer biology could not be replicated, sometimes even with the assistance of the original researchers.”
BackgroundChallenges, Opportunities, and Methods for Large-Scale Evaluations
Derzon, J. H. (2018) · Evaluation & the Health Professions · doi:10.1177/0163278718762292 · Cited by 7
Beyond the coin toss: Examining Wiseman’s criticisms of parapsychology.
Baptista, J., & Derakhshani, M. (2014) · Journal of Parapsychology · Cited by 7
How well are Phase 2 cancer trial publications supported by preclinical efficacy evidence?
Pratte, M., Ganeshamoorthy, S., Carlisle, B. G., & Kimmelman, J. (2019) · International Journal of Cancer · doi:10.1002/ijc.32405 · Cited by 7
Research Data Management Status of Science and Technology Research Institutes in Korea
Choi, M.-S., & Lee, S. (2020) · Data Science Journal · doi:10.5334/dsj-2020-029 · Cited by 8
Predicting Bond Returns: 70 Years of International Evidence
Baltussen, G., Martens, M. P. E., & Penninga, O. (2021) · Financial Analysts Journal · doi:10.1080/0015198x.2021.1908775 · Cited by 8
“Begley and Ellis (2012) showed that out of 53 studies on preclinical cancer, only 11% could be Table B3.”
Bruno, A. M., Olmsted, M. W., Martin, V., Einerson, B. D., Metz, T. D., Allshouse, A. A., Scott, J. R., & Blue, N. R. (2021) · American Journal of Obstetrics & Gynecology MFM · doi:10.1016/j.ajogmf.2021.100450 · Cited by 10
Eggert, S., Kahl, M., Kent, R., Gaats, L., Bock, N., Meinert, C., & Hutmacher, D. W. (2022) · Journal of Visualized Experiments · doi:10.3791/61261 · Cited by 8
Stem Cell Theory of Cancer: Implications for Translational Research from Bedside to Bench
Tu, S., Singh, S. R. K., Arnaoutakis, K., Malapati, S. J., Bhatti, S. A., Joon, A. Y., Atiq, O. T., & Pisters, L. L. (2022) · Cancers · doi:10.3390/cancers14143345 · Cited by 7
“In addition, reviews of preclinical biological studies suggest that only 20–25% of findings were reproducible [13,14], and of drugs and other treatments targeting cancer only 11% of findings from “landmark” studies were replicable [15].”
BackgroundSystematic deciphering of cancer genome networks.
Fendler, B. J., & Atwal, G. S. (2012) · Europe PMC (PubMed Central) · Cited by 9
Kostenexplosion in der medikamentösen Therapie onkologischer Erkrankungen
Ludwig, W.-D., & Schildmann, J. (2015) · Der Onkologe · doi:10.1007/s00761-015-2958-5 · Cited by 7
Authentication of collagen VI antibodies
Endicott, J., Holden, P. K., & Fitzgerald, J. (2017) · BMC Research Notes · doi:10.1186/s13104-017-2674-x · Cited by 8
“One recent study attempted to replicate the findings from 53 clinical experiments but found that results from just six of these studies could be independently confirmed [1].”
The Importance of Reproducibility in Plastic Surgery Research
Ascha, M., Ascha, M. S., & Gatherwright, J. R. (2019) · Plastic & Reconstructive Surgery · doi:10.1097/prs.0000000000005784 · Cited by 7
Bleakley, C., Reijgers, J., & Smoliga, J. M. (2020) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2020.9264 · Cited by 9
Brain and Behavior: we want you to share your data
Martone, M. E. (2013) · Brain and Behavior · doi:10.1002/brb3.192 · Cited by 8
“We have seen the abuses and the biases in our current system (Begley and Ellis 2012; Mueck 2013); perhaps we ought to be open to at least a trial period where we make an effort to determine whether routine publishing data is an exercise in futility or whether it opens a gateway to faster and more…”
BackgroundNonparametric Predictive Methods for Bootstrap and Test Reproducibility
Binhimd, S. M. S. (2014) · Durham e-Theses (Durham University) · Cited by 7
Biomedical Research: A House of Cards?
Lushington, G. H., & Chaguturu, R. (2015) · Future Medicinal Chemistry · doi:10.4155/fmc.15.171 · Cited by 7
Pauthner, M. G., Yeung, J., Ullman, C., Bakker, J., Wurch, T., Reichert, J. M., Lund‐Johansen, F., Bradbury, A. M., Carter, P. J., & Melis, J. P. M. (2016) · mAbs · doi:10.1080/19420862.2016.1153211 · Cited by 9
Data from a pre-publication independent replication initiative examining ten moral judgement effects
Tierney, W., Schweinsberg, M., Jordan, J., Kennedy, D. M., Qureshi, I., Sommer, S. A., Thornley, N., Madan, N., Vianello, M., Awtrey, E. C., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.82 · Cited by 7
“Results of replication initiatives attempting to reproduce previously published fi ndings reveal that the majority of independent studies do not produce the same signi fi cant effects as the original investigation 1 – 5 .”
Peer review: (r)evolution needed
Sipido, K. R., Gal, D., Luttun, A., Janssens, S. P., Sampaolesi, M., & Holvoet, P. (2017) · Cardiovascular Research · doi:10.1093/cvr/cvx191 · Cited by 7
Guilt by Association: How Scientific Misconduct Harms Prior CollaboratorsPreprint
Hussinger, K., & Pellens, M. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3072290 · Cited by 9
Increasing the reproducibility of research will reduce the problem of apophenia (and more)
Jones, P. M., & Martin, J. E. (2021) · Canadian Journal of Anesthesia/Journal canadien d anesthésie · doi:10.1007/s12630-021-02006-1 · Cited by 8
“In preclinical cancer research, the biotechnology company Amgen attempted to reproduce the results of 53 ‘‘landmark’’ studies — only six (11%) were subsequently confirmed.(8) Similarly, in a study of 67 projects, the drug company Bayer could only validate 25% of their preclinical studies (mostly in oncology).”
ResultA comprehensive analysis of code and data availability in biomedical researchPreprint
Deshpande, D., Sarkar, A. P., Guo, R., Moore, A., Darci-Maher, N., & Mangul, S. (2021) · doi:10.31219/osf.io/uz7m5 · Cited by 6
Is mindfulness research methodology improving over time? A systematic reviewPreprint
Goldberg, S. B., Tucker, R. P., Greene, P. A., Simpson, T. L., Kearney, D., & Davidson, R. J. (2018) · MindRxiv (OSF Preprints) · doi:10.31231/osf.io/nm7a8 · Cited by 7
The Registration Continuum in Clinical Science: A Guide toward Transparent PracticesPreprint
Benning, S. D., Bachrach, R. L., Smith, E., et al. (2018) · doi:10.31234/osf.io/an8h3 · Cited by 7
Zhang, B. C., Schneider‐Hohendorf, T., Elyanow, R., Pignolet, B., Falk, S., Wünsch, C., Deffner, M., Yusko, E. C., May, D., Mattox, D. E., et al. (2025) · EBioMedicine · doi:10.1016/j.ebiom.2025.105873 · Cited by 8
“34”
Xie, J., Safdar, M., Mircea, A., Zhao, B. C., Lu, Y., Ko, H., Yang, Z., & Zhao, Y. F. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.112223 · Cited by 10
Goltermann, J., Winter, N. R., Meinert, S., Grotegerd, D., Kraus, A., Flinkenflügel, K., Altegoer, L., Krieger, J., Leehr, E. J., Böhnlein, J., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-62374-w · Cited by 8
Qiao, L., Chang, S., Zou, L., Zhang, F., Cui, C., & Huang, N. (2025) · Bio-Design and Manufacturing · doi:10.1631/bdm.2500076 · Cited by 7
“Biomedical research has traditionally relied on two-dimensional (2D) cell cultures and animal models for dis‐ ease modeling and drug screening [1, 2], which have yielded valuable insights into disease mechanisms and played a vital role in therapeutic development.”
Murkin, J., Amos, H. E., Brough, D., & Turley, K. (2022) · Cancer Informatics · doi:10.1177/11769351221139257 · Cited by 19
Peuter, S. D., Reck, J., Bellekens, S., & Storms, G. (2024) · Research Ethics · doi:10.1177/17470161241262244 · Cited by 11
Good Practice Guideline for Preclinical Alcohol Research: The STRINGENCY Framework
Meinhardt, M. W., Gerlach, B., & Spanagel, R. (2024) · Current topics in behavioral neurosciences · doi:10.1007/7854_2024_484 · Cited by 7
Carlin, S. P., & Standing, L. (2013) · Psychological Reports · doi:10.2466/04.03.pr0.112.2.533-544 · Cited by 8
Louandre, C., Donnadieu, J., Lachaier, E., Page, C., Chauffert, B., & Galmiche, A. (2016) · International Journal of Oncology · doi:10.3892/ijo.2016.3345 · Cited by 6
Hephaestus: Data Reuse for Accelerating Scientific Discovery
Duggan, J., & Brodie, M. L. (2015) · doi:10.13140/2.1.2162.9764 · Cited by 7
Ott, T., Barth, A., Kriege, M., Jahn‐Eimermacher, A., Piepho, T., & Noppens, R. (2017) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.12872 · Cited by 7
Güttinger, S. (2018) · BioEssays · doi:10.1002/bies.201800055 · Cited by 6
Golbraikh, A., & Tropsha, A. (2018) · Wiley-VCH eBooks · doi:10.1002/9783527816880.ch12 · Cited by 7
Is the Pace of Biomedical Innovation Slowing?
Casadevall, A. (2018) · Perspectives in biology and medicine · doi:10.1353/pbm.2018.0067 · Cited by 6
A new initiative for AJP-Cell Physiology: “Making Cell Culture More Physiological”
Adams, J. C. (2019) · American Journal of Physiology-Cell Physiology · doi:10.1152/ajpcell.00157.2019 · Cited by 7
“Although cell culture is the methodology of choice for detailed investigative experiments to pin down molecular pathways, instances of unexpected phenotypes in gene-knockout mice, (as one example, cell culture-based experiments led to a view that D- and E-type cyclins are essential for cell cycle progression yet genek…”
BackgroundSorokowski, P., Groyecka-Bernard, A., Błaszczyński, K., Frąckowiak, T., & Kobylarek, A. (2019) · Journal of Education Culture and Society · doi:10.15503/jecs20192.67.75 · Cited by 8
“The tendency to publish mostly signi cant results was shown in medical (Begley & Ellis, 2012), social science (Camerer et al., 2018) or psychological (Open Science Collaboration, 2015) studies.”
BackgroundGuidelines for systematic reporting of sequence alignments
Vihinen, M. (2020) · Biology Methods and Protocols · doi:10.1093/biomethods/bpaa001 · Cited by 6
“Results even in many prominent journals been irrepro-ducible [1, 2].”
Abdou, A. G., Holah, N. S., Elazab, D., Elgendy, W., Badr, M. T., & Al‐Sharaky, D. R. (2020) · Applied immunohistochemistry & molecular morphology · doi:10.1097/pai.0000000000000840 · Cited by 7
“(6) Lympho-vascular invasion: present or absent.”
The ABCs of Reproducibility: Effecting Attitudes, Behaviors and Change to Realize Reproducibility
Free, T. (2020) · BioTechniques · doi:10.2144/btn-2020-0082 · Cited by 7
Griffiths, S. L., & Birchwood, M. J. (2020) · Medicina · doi:10.3390/medicina56120638 · Cited by 6
“For example, a replication rate of around 11% has been estimated for preclinical studies [62], and for high dimensional data, there is a greater probability of having false-positive findings; neuroimaging analyses have a 70% false-positive rate even after correction for multi-comparisons [61,63].”
BackgroundA Peer-To-Peer Publication Model on Blockchain
Khan, I., & Shahaab, A. (2021) · Frontiers in Blockchain · doi:10.3389/fbloc.2021.615726 · Cited by 8
““ landmark ” research papers could not be reproduced (Begley and Ellis, 2012).”
BackgroundMeasuring and interpreting pervasive heterogeneity, poikilosis
Vihinen, M. (2021) · FASEB BioAdvances · doi:10.1096/fba.2021-00015 · Cited by 6
Small volume retinol binding protein measurement by liquid chromatography-tandem mass spectrometry
Phipps, W. S., Greene, D. N., Pflaum, H., Laha, T. J., Dickerson, J. A., Irvine, J., Merrill, A. E., Ranjitkar, P., Henderson, C. M., & Hoofnagle, A. N. (2021) · Clinical Biochemistry · doi:10.1016/j.clinbiochem.2021.10.005 · Cited by 6
Transforming Academic Drug Discovery
Meinke, P. T. (2022) · ChemBioChem · doi:10.1002/cbic.202100671 · Cited by 6
Machine learning approaches for predicting the onset time of the adverse drug events in oncology
Timilsina, M., Tandan, M., & Nováček, V. (2022) · Machine Learning with Applications · doi:10.1016/j.mlwa.2022.100367 · Cited by 14
Sansbury, B. E., Nystoriak, M. A., Uchida, S., Wysoczynski, M., & Moore, J. B. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.913612 · Cited by 9
Bayesian estimation reveals that reproducible models in Systems Biology get more citations
Höpfl, S., Pleiss, J., & Radde, N. E. (2023) · Scientific Reports · doi:10.1038/s41598-023-29340-2 · Cited by 10
Cooper, T. (2023) · FEMS Yeast Research · doi:10.1093/femsyr/foac065 · Cited by 5
The Relationship of Musical Expertise, Working Memory, and Intelligence
Arndt, C., Schlemmer, K., & van der Meer, E. E. (2023) · Music Perception An Interdisciplinary Journal · doi:10.1525/mp.2023.40.4.334 · Cited by 12
The 3Rs in Experimental Liver Disease
Martínez‐López, S., Ángel‐Gomis, E., Sánchez, E., Pastor-Campos, A., Picó, J., & Gómez‐Hurtado, I. (2023) · Animals · doi:10.3390/ani13142357 · Cited by 10
Ursić, L., Bralić, N., Žuljević, M., Puljak, L., & Buljan, I. (2024) · Accountability in Research · doi:10.1080/08989621.2024.2345723 · Cited by 6
“…also identified in social sciences in general (Camerer et al. 2018), neuroscience (Button et al. 2013), preclinical research (Begley and John 2015; Begley and Lee 2012), cancer biology (Errington et al. 2014), and other disciplines, with researchers like-wise reporting inability to reproduce…”
Studying Cellular Signal Transduction with OMIC Technologies
Landry, B. D., Clarke, D. C., & Lee, M. J. (2015) · Journal of Molecular Biology · doi:10.1016/j.jmb.2015.07.021 · Cited by 12
“Accordingly, OMIC-based measurements based on weak or non-existent hypotheses should be expected to be rife with false results, which in some cases may exacerbate the problem of irreproducibility of biomedical science [101,102].”
Next Generation Scientific Publishing and the Web of Data
Clark, T. (2014) · Semantic Web · doi:10.3233/sw-140139 · Cited by 6
The importance of being transparent
Jackson, S. M. (2015) · Journal of Clinical Investigation · doi:10.1172/jci80370 · Cited by 6
“We are all aware that there have been numerous high-profile retractions in recent years as well as a number of prominent studies documenting irreproducibility (6, 7).”
Selecting the Best Drugs for Phase I Clinical Development and Beyond
Loong, H. H.-F., & Siu, L. L. (2013) · American Society of Clinical Oncology Educational Book · doi:10.14694/edbook_am.2013.33.469 · Cited by 8
Ludwig, W.-D., & Schott, G. (2013) · Oncology Research and Treatment · doi:10.1159/000348253 · Cited by 6
The changing world of cancer drug development: the regulatory bodies' perspective.
Gravanis, I., Vleminckx, C., Jönsson, B., & Pignatti, F. (2014) · PubMed · doi:10.3978/j.issn.2304-3865.2014.05.08 · Cited by 6
“Finally, animal models of cancer are similarly unable to predict the clinical situation (10).”
BackgroundAccounting and Public Policy: The Importance of Credible Research
Denison, C. A., Ravenscroft, S., & Williams, P. F. (2014) · Accounting and the Public Interest · doi:10.2308/apin-51158 · Cited by 7
A Medium-Throughput Single Cell CRISPR-Cas9 Assay to Assess Gene Essentiality
Grassian, A. R., Scales, T. M. E., Knutson, S. K., Kuntz, K. W., McCarthy, N., Lowe, C. E., Moore, J. D., Copeland, R. A., Keilhack, H., Smith, J. J., et al. (2015) · Biological Procedures Online · doi:10.1186/s12575-015-0028-4 · Cited by 6
“Furthermore, many targets reported in the literature to be drivers of cancer have not been confirmed on further analysis [3, 4].”
Background
Metadata
- Authors
- Begley, C. G., & Ellis, L. M.
- Affiliations
- Amgen (United States) (US)
- The University of Texas MD Anderson Cancer Center (US)
- Corresponding author
- Ellis, Lee M.
- Venue
- Nature
- Published
- 2012-03-28
- Volume
- 483
- Issue
- 7391
- Pages
- 531-533
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W1987777080
- Semantic Scholar
- 1a81d3d1e8a983e0551edf485f4ac1a8cdd7fea4
- Cited by
- 2,979
- References
- 5
- Citations by year
- 2026: 106 · 2025: 116 · 2024: 127 · 2023: 169 · 2022: 179 · 2021: 206 · 2020: 236 · 2019: 255 · 2018: 262 · 2017: 271 · 2016: 283 · 2015: 262 · 2014: 226 · 2013: 133 · 2012: 62
- FWCI
- 210.63
- Type
- journal article
- Language
- en
- Update notices
- correction (2012-05-02)
- Open access
- bronze
- doi.org
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Health and Medical Research Impacts
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
- Health Sciences › Nursing › Issues, ethics and legal aspects › Biomedical Ethics and Regulation
- Keywords
- preclinical cancer research, research standards, publication practices, research incentives, translational research, clinical trials
- MeSH terms
- Animals, Clinical Trials as Topic, Drug Evaluation, Preclinical, Humans, Neoplasms, Research Design, Reproducibility of Results, Survival Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.