Believe it or not: how much can we rely on published data on potential drug targets?
The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“reports from the pharmaceutical industry (Begley & Ellis, 2012;Prinz et al., 2011) and systematic assessments of replicability in specific fields, such as cancer biology (Errington, Mathur, et al., 2021), spinal cord injury (Steward et al., 2012) and amyotrophic lateral sclerosis (Scott et al., 2008), have reported success rates ranging from 0 to 40%.”
From Estimating the replicability of Brazilian biomedical science (2025) · cites this paper as Prinz et al., 2011
In the citing paper
“These numbers are in a similar range as previous estimates from the biomedical literature (Begley & Ellis, 2012;Errington, Mathur, et al., 2021;Prinz et al., 2011;Scott et al., 2008;Steward et al., 2012).”
From Estimating the replicability of Brazilian biomedical science (2025) · cites this paper as Prinz et al., 2011
In the citing paper
“This may contribute to publication bias, selective reporting, and questionable research practices that reduce the credibility of the published literature 18-25 .”
From Initial evidence of research quality of registered reports compared with the standard publishing model (2021) · cites this paper as [19]
In the citing paper
“Scientists at Bayer have reported that only ~25% of published preclinical studies could be validated to the point at which projects could progress.2”
From Reproducibility of Quantitative Systems Pharmacology Models: Current Challenges and Future Opportunities (2019) · cites this paper as [2]
In the citing paper
“A team at Bayer HealthCare in Germany last year reported 4 that only about 25% of published preclinical studies could be validated to the point at which projects could continue.”
From Raise standards for preclinical cancer research (2012) · cites this paper as [4]
10 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
Arrowsmith, J. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439 · Cited by 622
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2019) · CHANCE · doi:10.1080/09332480.2019.1579573 · Cited by 463
What errors do peer reviewers detect, and does training improve their ability to detect them?
Schroter, S., Black, N., Evans, S., Godlee, F., Osorio, L., & Smith, R. (2008) · Journal of the Royal Society of Medicine · doi:10.1258/jrsm.2008.080062 · Cited by 226
Osherovich, L. Z. (2011) · Science-Business eXchange · doi:10.1038/scibx.2011.416 · Cited by 43
Factors Affecting Reproducibility between Genome-Scale siRNA-Based Screens
Barrows, N. J., Sommer, C. L., Garcia-Blanco, M. A., & Pearson, J. L. (2010) · SLAS Discovery · doi:10.1177/1087057110374994 · Cited by 39
McDonald, R. J., Cloft, H. J., & Kallmes, D. F. (2007) · American Journal of Neuroradiology · doi:10.3174/ajnr.a0766 · Cited by 31
Nemery, B. (2001) · Occupational and Environmental Medicine · doi:10.1136/oem.58.9.604 · Cited by 23 · cited 1×
“Adding to this problem, many initially rejected papers will subsequently be published in other journals without substantial changes or improvement”
Background
Research works printed but not linked: 2
- Lehrer, J. The truth wears off: is there something wrong with the scientific method? The New Yorker [online], (2010).
- Freeman, D. H. Lies, damned lies, and medical science. The Atlantic [online], (2010).
Cited by
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Radiomics: Images Are More than Pictures, They Are Data
Gillies, R. J., Kinahan, P. E., & Hricak, H. (2016) · Radiology · doi:10.1148/radiol.2015151169 · Cited by 8,421
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979
Increasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597
Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Nave, G., Nosek, B. A., Pfeiffer, T., et al. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0399-z · Cited by 1,521
The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta‐analyses
IOANNIDIS, J. P. A. (2016) · The Milbank Quarterly · doi:10.1111/1468-0009.12210 · Cited by 1,483
How to increase value and reduce waste when research priorities are set
Chalmers, I., Bracken, M. B., Djulbegovic, B., Garattini, S., Grant, J., Gülmezoglu, A. M., Howells, D. W., Ioannidis, J. P. A., & Oliver, S. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62229-1 · Cited by 1,436
Evaluating replicability of laboratory experiments in economics
Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Almenberg, J., Altmejd, A., Chan, T., et al. (2016) · Science · doi:10.1126/science.aaf0918 · Cited by 1,314
Pashler, H., & Wagenmakers, E. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612465253 · Cited by 1,321
High-throughput discovery of novel developmental phenotypes
The International Mouse Phenotyping Consortium, Dickinson, M. E., Flenniken, A. M., Ji, X., Teboul, L., Wong, M. D., White, J. K., Meehan, T. F., Weninger, W. J., Westerberg, H., et al. (2016) · Nature · doi:10.1038/nature19356 · Cited by 1,196
“Moreover, recent challenges in reproducibility of animal model experimentation 5,6 emphasize the need for careful standardization of allele design, genetic background, and phenotyping protocols.”
Nosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301
“False effects can remain for decades, slowly fading or continuing to inspire and influence new research (Prinz et al., 2011).”
Policy: NIH plans to enhance reproducibility
Collins, F. S., & Tabak, L. A. (2014) · Nature · doi:10.1038/505612a · Cited by 1,247
Network pharmacology: curing causal mechanisms instead of treating symptoms
Nogales, C., Mamdouh, Z. M., List, M., Kiel, C., Casas, A. I., & Schmidt, H. H. H. W. (2022) · Trends in Pharmacological Sciences · doi:10.1016/j.tips.2021.11.004 · Cited by 1,201
How to get statistically significant effects in any ERP experiment (and why you shouldn't)
Luck, S. J., & Gaspelin, N. (2017) · Psychophysiology · doi:10.1111/psyp.12639 · Cited by 1,195
A call for transparent reporting to optimize the predictive value of preclinical research
Landis, S. C., Amara, S. G., Asadullah, K., Austin, C. P., Blumenstein, R., Bradley, E. W., Crystal, R. G., Darnell, R. B., Ferrante, R. J., Fillit, H., et al. (2012) · Nature · doi:10.1038/nature11556 · Cited by 1,196
Self-citationCombinatorial drug therapy for cancer in the post-genomic era
Al-Lazikani, B., Banerji, U., & Workman, P. (2012) · Nature Biotechnology · doi:10.1038/nbt.2284 · Cited by 1,070
Hou, K., Xue, C., & Zhang, L. (2020) · The Review of Financial Studies · doi:10.1093/rfs/hhy131 · Cited by 989
“Table 1 shows the list, which includes 57, 69, 38, 79, 103, and 106 anomalies from the momentum, value-versus-growth, investment, profitability, 5For example, in oncology, Prinz, Schlange, and Asadullah (2011) report that scientists at Bayer fail to reproduce two thirds of 67 published studies.”
The economics of reproducibility in preclinical research
Freedman, L. P., Cockburn, I. M., & Simcoe, T. S. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002165 · Cited by 1,055
“[7], Vasilevsky [8], Hartshorne and Schachner [5], and Glasziou et al.”
BackgroundAn Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042
Reproducibility in Science: Improving the Standard for Basic and Preclinical Research
Begley, C. G., & Ioannidis, J. P. A. (2015) · Circulation Research · doi:10.1161/circresaha.114.303819 · Cited by 1,035
Principles for Valid Histopathologic Scoring in Research
Gibson-Corley, K. N., Olivier, A. K., & Meyerholz, D. K. (2013) · Veterinary Pathology · doi:10.1177/0300985813485099 · Cited by 936
“Recent reports have highlighted the importance of repeatability in research.(1,52) For instance, Begley and Ellis(1) attempted to repeat the work of 53 major ‘‘landmark’’ papers but were successful in only 11% (6 of 53) of the cases.”
BackgroundOpportunities and challenges in phenotypic drug discovery: an industry perspective
Moffat, J. G., Vincent, F., Lee, J. A., Eder, J., & Prunotto, M. (2017) · Nature Reviews Drug Discovery · doi:10.1038/nrd.2017.111 · Cited by 939
Genetic and transcriptional evolution alters cancer cell line drug response
Ben-David, U., Siranosian, B., Ha, G., Tang, H., Oren, Y., Hinohara, K., Strathdee, C. A., Dempster, J., Lyons, N. J., Burns, R., et al. (2018) · Nature · doi:10.1038/s41586-018-0409-3 · Cited by 931
Structure and dynamics of molecular networks: A novel paradigm of drug discovery
Csermely, P., Korcsmáros, T., Kiss, H. J. M., London, G., & Nussinov, R. (2013) · Pharmacology & Therapeutics · doi:10.1016/j.pharmthera.2013.01.016 · Cited by 892
“Currently we often lack these high-quality datasets especially on system dynamics (Henney & Superti-Furga, 2008; Prinz et al., 2011; Begley & Ellis, 2012; Landis et al., 2012).”
BackgroundResultIncreasing value and reducing waste: addressing inaccessible research
Chan, A.-W., Song, F., Vickers, A., Jefferson, T., Dickersin, K., Gøtzsche, P. C., Krumholz, H. M., Ghersi, D., & van der Worp, H. B. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62296-5 · Cited by 887
Ten Simple Rules for Reproducible Computational Research
Sandve, G. K., Nekrutenko, A., Taylor, J., & Hovig, E. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003285 · Cited by 880
“…through studies showing that scientific papers commonly leave out experimental details essential for reproduction [5], studies showing difficulties with replicating published experimental results [6], an increase in retracted papers [7], and through a high number of failing clinical trials [8,9].”
ResultRescuing US biomedical research from its systemic flaws
Alberts, B., Kirschner, M. W., Tilghman, S., & Varmus, H. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1404402111 · Cited by 817
The rules of the game called psychological science
Bakker, M., van Dijk, A., & Wicherts, J. M. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612459060 · Cited by 800
How to Make More Published Research True
Ioannidis, J. P. A. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001747 · Cited by 760
“Pharmaceutical industry teams have championed reproducibility in preclinical research [56,57] because they depend on pre-clinical academic investigations accurately pinpointing useful drug targets.”
BackgroundDonoho, D. (2017) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2017.1384734 · Cited by 695
“There seem to be significant flaws in the validity of the scientific literature (Ioannidis 2007; Sullivan 2007; Prinz, Schlange, and Asadullah 2011; Open Science Collaboration et al. 2015).”
BackgroundCurrent hurdles to the translation of nanomedicines from bench to the clinic
Đorđević, S., Gonzalez, M. M., Conejos-Sánchez, I., Carreira, B., Pozzi, S., Acúrcio, R. C., Satchi-Fainaro, R., Florindo, H. F., & Vicent, M. J. (2022) · Drug Delivery and Translational Research · doi:10.1007/s13346-021-01024-2 · Cited by 657
“In contrast, inconsistent results observed among published and industry-obtained data [227, 228] and the inability to find appropriate commercial partners due to the challenging gaps in translation have prompted the termination of the remaining studies.”
ResultDrug discovery and development: Role of basic biological research
Mohs, R. C., & Greig, N. H. (2017) · Alzheimer's & Dementia: Translational Research & Clinical Interventions · doi:10.1016/j.trci.2017.10.005 · Cited by 673
“The reasons for this high failure rate are a matter of some discussion; for present purposes, it is important to note that investigators proposing that a new finding is relevant to drug discovery should expect a high level of skepticism and, even if industry investigators show interest, it is almost certain that there…”
BackgroundFacts Are More Important Than Novelty: Replication in the Education Sciences
Makel, M. C., & Plucker, J. A. (2014) · Educational Researcher · doi:10.3102/0013189x14545513 · Cited by 609
An Open, Large-Scale, Collaborative Effort to Estimate the Reproducibility of Psychological Science
Open Science Collaboration (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612462588 · Cited by 614
Animal models of human disease: challenges in enabling translation
McGonigle, P., & Ruggeri, B. (2014) · Biochemical Pharmacology · doi:10.1016/j.bcp.2013.08.006 · Cited by 610
PREPARE: guidelines for planning animal research and testing
Smith, A. J., Clutton, R. E., Lilley, E., Hansen, K. E. A., & Brattelid, T. (2018) · Laboratory Animals · doi:10.1177/0023677217724823 · Cited by 587
“This can, for example, contribute towards the failure of drugs when they enter human trials.(8) These issues come in addition to other concerns, not unique to animal research, about publication bias, which tends to favour the reporting of positive results and can lead to the acceptance of claims as fact.”
BackgroundDiscovery of small molecule cancer drugs: Successes, challenges and opportunities
Hoelder, S., Clarke, P. A., & Workman, P. (2012) · Molecular Oncology · doi:10.1016/j.molonc.2012.02.004 · Cited by 587
Collins, B. C., Hunter, C. L., Liu, Y., Schilling, B., Rosenberger, G., Bader, S. L., Chan, D. W., Gibson, B. W., Gingras, A.-C., Held, J. M., et al. (2017) · Nature Communications · doi:10.1038/s41467-017-00249-5 · Cited by 576
Why Science Is Not Necessarily Self-Correcting
Ioannidis, J. P. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612464056 · Cited by 563
Yu, Z., Guindani, M., Grieco, S. F., Chen, L., Holmes, T. C., & Xu, X. (2022) · Neuron · doi:10.1016/j.neuron.2021.10.030 · Cited by 563
“The growing concern over reproducibility 64 in biomedical research is often referred to as a “problem of inadequate rigor” (Kilkenny et al., 65 2009; Prinz et al., 2011).”
BackgroundEmerging therapeutic opportunities for integrin inhibitors
Slack, R. J., Macdonald, S. J. F., Roper, J. A., Jenkins, R. G., & Hatley, R. J. D. (2022) · Nature Reviews Drug Discovery · doi:10.1038/s41573-021-00284-4 · Cited by 540
Next-generation probes, particles, and proteins for neural interfacing
Rivnay, J., Wang, H., Fenno, L., Deisseroth, K., & Malliaras, G. G. (2017) · Science Advances · doi:10.1126/sciadv.1601649 · Cited by 514
Top 10 Replicated Findings From Behavioral Genetics
Plomin, R., DeFries, J. C., Knopik, V. S., & Neiderhiser, J. M. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691615617439 · Cited by 517
Phenotypic screening in cancer drug discovery — past, present and future
Moffat, J. G., Rudolph, J., & Bailey, D. (2014) · Nature Reviews Drug Discovery · doi:10.1038/nrd4366 · Cited by 507
Deep learning in clinical natural language processing: a methodical review
Wu, S., Roberts, K., Datta, S., Du, J., Ji, Z., Si, Y., Soni, S., Wang, Q., Wei, Q., Xiang, Y., et al. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz200 · Cited by 491
“A renewed emphasis on scientific rigor has permeated the academic establishment, 108,109 with replicability and reproducibility studies (eg, from ACM’s definitions 110,111 ) showing how much of the literature has fallen short of its scientific goals.”
BackgroundDetecting and avoiding likely false‐positive findings – a practical guide
Forstmeier, W., Wagenmakers, E., & Parker, T. H. (2017) · Biological Reviews · doi:10.1111/brv.12315 · Cited by 436
Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention
Ioannidis, J. P. A., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.02.010 · Cited by 471
Deep impact: unintended consequences of journal rank
Brembs, B., Button, K., & Munafò, M. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00291 · Cited by 441
“Three recent studies reported attempts to replicate published findings in preclinical medicine (Scott et al., 2008; Prinz et al., 2011; Begley and Ellis, 2012).”
Pre-registration in social psychology—A discussion and suggested template
van 't Veer, A. E., & Giner-Sorolla, R. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.03.004 · Cited by 439
AI for life: Trends in artificial intelligence for biotechnology
Holzinger, A., Keiblinger, K., Holub, P., Zatloukal, K., & Müller, H. (2023) · New Biotechnology · doi:10.1016/j.nbt.2023.02.001 · Cited by 438
Poynton, F. E., Bright, S. A., Blasco, S., Williams, D. C., Kelly, J. M., & Gunnlaugsson, T. (2017) · Chemical Society Reviews · doi:10.1039/c7cs00680b · Cited by 421
Quintana, D. S., Alvares, G. A., & Heathers, J. A. J. (2016) · Translational Psychiatry · doi:10.1038/tp.2016.73 · Cited by 420
“51 Ostensibly, the goal of a methods section is the description of methodology such that fi ndings may be replicated from written papers without clari fi cation or reference to other sources.”
Contextual sensitivity in scientific reproducibility
Van Bavel, J. J., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1521897113 · Cited by 398
“Unsuccessful attempts to replicate findings in genetics (1), pharmacology (2), oncology (3), biology (4), and economics (5) have given credence to previous speculation that most published research findings are false (6).”
BackgroundResultInfluentialReproducibility of animal research in light of biological variation
Voelkl, B., Altman, N. S., Forsman, A., Forstmeier, W., Gurevitch, J., Jaric, I., Karp, N. A., Kas, M. J., Schielzeth, H., Van de Casteele, T., et al. (2020) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-0313-3 · Cited by 390
Investigating the replicability of preclinical cancer biology
Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.71601 · Cited by 462
“And, in cancer biology and related fields, two drug companies (Bayer and Amgen) reported failures to replicate findings from promising studies that could have led to new therapies (Prinz et al., 2011; Begley and Ellis, 2012).”
BackgroundWeingarten, E., Chen, Q., McAdams, M., Yi, J., Hepler, J., & Albarracín, D. (2016) · Psychological Bulletin · doi:10.1037/bul0000030 · Cited by 380
ITK: enabling reproducible research and open science
McCormick, M., Liu, X., Jomier, J., Marion, C., & Ibanez, L. (2014) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2014.00013 · Cited by 376
“…replacement for the verification of reproducibility, a mistake by which experimental work has been replaced by thought experiments and opinion-based evaluations that do little to further the scientific enterprise (Couzin, 2006; Baker, 2006; Prinz et al., 2011; Russel, 2013; Collins and Tabak, 2014).”
Mirowski, P. (2018) · Social Studies of Science · doi:10.1177/0306312718772086 · Cited by 372
“Looking at 67 recent drug development projects based on preclinical cancer biology research, they found that in more than 75% of cases the published data did not conform to their in-house attempts to replicate (Prinz et al., 2011).”
Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations
McKiernan, E. C., Schimanski, L. A., Muñoz Nieves, C., Matthias, L., Niles, M. T., & Alperin, J. P. (2019) · eLife · doi:10.7554/elife.47338 · Cited by 347
“Prinz et al. (2011) found, “that the reproducibility of327 published data did not significantly correlate with journal impact factors” (pg. 2).328 Thus, at least as viewed through the aspects above, there is little to no evidence to justify a329 relationship between the JIF and research quality.”
Chen, X., Lu, B., & Yan, C. (2018) · Human Brain Mapping · doi:10.1002/hbm.23843 · Cited by 340
“However, concerns regarding the reproducibility of biomedical and psychological research are increasingly being expressed [Ioannidis, 2005; Open Science Collaboration, 2015; Prinz et al., 2011].”
BackgroundZushin, P.-J. H., Mukherjee, S., & Wu, J. C. (2023) · Journal of Clinical Investigation · doi:10.1172/jci175824 · Cited by 347
Statistical and Methodological Considerations for the Interpretation of Intranasal Oxytocin Studies
Walum, H., Waldman, I. D., & Young, L. J. (2016) · Biological Psychiatry · doi:10.1016/j.biopsych.2015.06.016 · Cited by 347
“Bias of this kind could have serious consequences such as failure to replicate findings (24), and there are also reasons to believe that publication bias is more likely to affect low powered studies (25).”
BackgroundUsing prediction markets to estimate the reproducibility of scientific research
Dreber, A., Pfeiffer, T., Almenberg, J., Isaksson, S., Wilson, B., Chen, Y., Nosek, B. A., & Johannesson, M. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1516179112 · Cited by 331
“J Exp Psychol Learn Mem Cogn 34(1):204–218.”
BackgroundInfluentialRestoring function after spinal cord injury: towards clinical translation of experimental strategies
Ramer, L. M., Ramer, M. S., & Bradbury, E. J. (2014) · The Lancet Neurology · doi:10.1016/s1474-4422(14)70144-9 · Cited by 324
Opportunities and challenges in the collection and analysis of digital phenotyping data
Onnela, J.-P. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-020-0771-3 · Cited by 319
“One study found that only 6% of the sampled medical studies were completely reproducible [33], and a survey published in the journal Nature in 2012 found that 47 out of 53 medical research papers on cancer were irreproducible [34].”
ResultImplementing Reproducible Research
Stodden, V. (2014) · doi:10.1201/b16868 · Cited by 314
Animal to human translation: a systematic scoping review of reported concordance rates
Leenaars, C. H. C., Kouwenaar, C., Stafleu, F. R., Bleich, A., Ritskes-Hoitinga, M., De Vries, R. B. M., & Meijboom, F. L. B. (2019) · Journal of Translational Medicine · doi:10.1186/s12967-019-1976-2 · Cited by 314
Reproducible Research Practices and Transparency across the Biomedical Literature
Iqbal, S. A., Wallach, J. D., Khoury, M. J., Schully, S. D., & Ioannidis, J. P. A. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002333 · Cited by 310
Challenges for assessing replicability in preclinical cancer biology
Errington, T. M., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.67995 · Cited by 267
O'Bryant, S. E., Mielke, M. M., Rissman, R. A., Lista, S., Vanderstichele, H., Zetterberg, H., Lewczuk, P., Posner, H., Hall, J., Johnson, L., et al. (2017) · Alzheimer's & Dementia · doi:10.1016/j.jalz.2016.09.014 · Cited by 303
“Current model of bio industry personnel) of published studies will not replicate in industrial laboratories [52].”
BackgroundHenderson, V. C., Kimmelman, J., Fergusson, D., Grimshaw, J. M., & Hackam, D. G. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001489 · Cited by 302
“Various commentators have reported problems such as difficulty in replicating preclinical studies [5,6], publication bias [7], and the prevalence of methodological practices that result in threats to validity [8].”
BackgroundA machine learning Automated Recommendation Tool for synthetic biology
Radivojević, T., Costello, Z., Workman, K., & Garcia Martin, H. (2020) · Nature Communications · doi:10.1038/s41467-020-18008-4 · Cited by 277
No raw data, no science: another possible source of the reproducibility crisis
Miyakawa, T. (2020) · Molecular Brain · doi:10.1186/s13041-020-0552-2 · Cited by 268
Trust, but Verify II: A Practical Guide to Chemogenomics Data Curation
Fourches, D., Muratov, E., & Tropsha, A. (2016) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6b00129 · Cited by 294
“Conversely, looking at the biological data accuracy, Prinz et al. 12 observed that only 20–25% of published assertions concerning purported biological functions for novel deorphanized proteins was consistent with the Bayer’s in-house findings.”
ResultState of the Art: Reproducibility in Artificial Intelligence
Gundersen, O. E., & Kjensmo, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11503 · Cited by 293
“This is even the case for results published in the most prestigious journals; even the original researchers cannot reproduce their own results (Aarts et al. 2016; Begley and Ellis 2012; Begley and Ioannidis 2014; Prinz et al. 2011).”
ResultO'Bryant, S. E., Gupta, V., Henriksen, K., Edwards, M., Jeromin, A., Lista, S., Bazenet, C., Soares, H., Lovestone, S., Hampel, H., et al. (2015) · Alzheimer's & Dementia · doi:10.1016/j.jalz.2014.08.099 · Cited by 290
“In fact, an “unspoken rule” among venture capital firms is that 50% of published studies will not replicate in industrial labs [4] and the National Institutes of Health (NIH) recently outlined a plan to address this problem [2].”
The 3Rs in research: a contemporary approach to replacement, reduction and refinement
MacArthur Clark, J. (2018) · British Journal of Nutrition · doi:10.1017/s0007114517002227 · Cited by 290
“This confirmed earlier findings by Ioannidis((22)) and Prinz et al.((23)).”
ResultToward Benchmarking in Catalysis Science: Best Practices, Challenges, and Opportunities
Bligaard, T., Bullock, R. M., Campbell, C. T., Chen, J. G., Gates, B. C., Gorte, R. J., Jones, C. W., Jones, W. D., Kitchin, J. R., & Scott, S. L. (2016) · ACS Catalysis · doi:10.1021/acscatal.6b00183 · Cited by 285
Soderberg, C. K., Errington, T. M., Schiavone, S. R., Bottesini, J., Thorn, F. S., Vazire, S., Esterling, K. M., & Nosek, B. A. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01142-4 · Cited by 295
Boyd, A., Zhang, H., & Williams, A. (2013) · Acta Neuropathologica · doi:10.1007/s00401-013-1112-y · Cited by 287
“We previously showed that lentiviral delivery of Sema3F to a demyelinated lesion in a different mouse model improved remyelination [37], giving confidence in these results; an important factor considering the number of drug targets that look promising in initial publications, but which later fail to be repeated by pha…”
BackgroundChanging R&D models in research-based pharmaceutical companies
Schuhmacher, A., Gassmann, O., & Hinder, M. (2016) · Journal of Translational Medicine · doi:10.1186/s12967-016-0838-4 · Cited by 286
“• lack of reliability of published data [14], • biopharmaceutical issues including suboptimal PK [4], • poorly predictive preclinical models in discovery research and preclinical testing [15], • the concept of target-based drug discovery with the related advanced complexity of target selection, a competition for propr…”
BackgroundDrug repurposing: far beyond new targets for old drugs
Oprea, T. I., & Mestres, J. (2012) · The AAPS Journal · doi:10.1208/s12248-012-9390-1 · Cited by 285
“ibility analysis of 67 target validation studies, target–disease associations are not always fully reliable [40].”
BackgroundThe Prevalence of Inappropriate Image Duplication in Biomedical Research Publications
Bik, E. M., Casadevall, A., & Fang, F. C. (2016) · mBio · doi:10.1128/mbio.00809-16 · Cited by 287
When Quality Beats Quantity: Decision Theory, Drug Discovery, and the Reproducibility Crisis
Scannell, J. W., & Bosley, J. (2016) · PLOS ONE · doi:10.1371/journal.pone.0147215 · Cited by 279
Giner-Sorolla, R. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612457576 · Cited by 274
Zhu, H., & Leung, S. W. (2015) · Diabetologia · doi:10.1007/s00125-015-3510-2 · Cited by 271
“Recent studies have indicated that the seminal findings from academic laboratories could only be reproduced 11–50% of the time [13, 14].”
BackgroundMolecular Docking: Principles, Advances, and Its Applications in Drug Discovery
Muhammed, M. T., & Aki-Yalcin, E. (2024) · Letters in Drug Design & Discovery · doi:10.2174/1570180819666220922103109 · Cited by 267
“Therefore, it is recommended not to overinterpret docking results regarding the nature of the ligand unless other validations like lab experiments, are performed [150].”
BackgroundMeta-analysis using effect size distributions of only statistically significant studies.
van Assen, M. A. L. M., van Aert, R. C. M., & Wicherts, J. M. (2015) · Psychological Methods · doi:10.1037/met0000025 · Cited by 265
Statistical Rituals: The Replication Delusion and How We Got There
Gigerenzer, G. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918771329 · Cited by 263
Reproducibility of preclinical animal research improves with heterogeneity of study samples
Voelkl, B., Vogt, L., Sena, E. S., & Würbel, H. (2018) · PLOS Biology · doi:10.1371/journal.pbio.2003693 · Cited by 262
Identification of new epilepsy treatments: Issues in preclinical methodology
Galanopoulou, A. S., Buckmaster, P. S., Staley, K. J., Moshé, S. L., Perucca, E., Engel, J., Löscher, W., Noebels, J. L., Pitkänen, A., Stables, J., et al. (2012) · Epilepsia · doi:10.1111/j.1528-1167.2011.03391.x · Cited by 261
“…of preclinical data for compounds progressing from academic laboratories to industrial development programs and, ultimately, to clinical trials (Stroke Therapy Academic Industry Roundtable [STAIR], 1999, Ioannidis, 2005; Benatar, 2007; Fisher et al., 2009; Mullard, 2011; Prinz et al., 2011).”
BackgroundThe readability of scientific texts is decreasing over time
Plavén-Sigray, P., Matheson, G. J., Schiffler, B. C., & Thompson, W. H. (2017) · eLife · doi:10.7554/elife.27725 · Cited by 257
Inherited Retinal Degenerations: Current Landscape and Knowledge Gaps
Duncan, J. L., Pierce, E. A., Laster, A. M., Daiger, S. P., Birch, D. G., Ash, J. D., Iannaccone, A., Flannery, J. G., Sahel, J. A., Zack, D. J., et al. (2018) · Translational Vision Science & Technology · doi:10.1167/tvst.7.4.6 · Cited by 257
Oncogene addiction: pathways of therapeutic response, resistance, and road maps toward a cure
Pagliarini, R., Shao, W., & Sellers, W. R. (2015) · EMBO reports · doi:10.15252/embr.201439949 · Cited by 253
“As such, the industry validation rate of many novel targets identified through ‘one-off’ functional approaches has been poor [152].”
Llovera, G., Hofmann, K., Roth, S., Salas-Pérdomo, A., Ferrer-Ferrer, M., Perego, C., Zanier, E. R., Mamrak, U., Rex, A., Party, H., et al. (2015) · Science Translational Medicine · doi:10.1126/scitranslmed.aaa9853 · Cited by 253
Preserving Statistical Validity in Adaptive Data Analysis
Dwork, C., Feldman, V., Hardt, M., Pitassi, T., Reingold, O., & Roth, A. L. (2015) · Proceedings of the forty-seventh annual ACM symposium on Theory of Computing · doi:10.1145/2746539.2746580 · Cited by 248
“Throughout the scientific community there is a growing recognition that claims of statistical significance in published research are frequently invalid [29, 28, 35, 8].”
BackgroundRodent models of post-traumatic stress disorder: behavioral assessment
Verbitsky, A., Dopfel, D., & Zhang, N. (2020) · Translational Psychiatry · doi:10.1038/s41398-020-0806-x · Cited by 247
A systematic review of smartphone-based human activity recognition methods for health research
Straczkiewicz, M., James, P., & Onnela, J.-P. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00514-4 · Cited by 247
“The reason for data collection is to produce generalizable knowledge, but according to an often-cited study, 65% of medical studies were inconsistent when retested and only 6% were completely reproducible 12 .”
BackgroundImproving the odds of drug development success through human genomics: modelling study
Hingorani, A. D., Kuan, V., Finan, C., Kruger, F. A., Gaulton, A., Chopade, S., Sofat, R., MacAllister, R. J., Overington, J. P., Hemingway, H., et al. (2019) · Scientific Reports · doi:10.1038/s41598-019-54849-w · Cited by 247
Metadata
- Authors
- Prinz, F., Schlange, T., & Asadullah, K.
- Affiliations
- Bayer (Germany) (DE)
- Corresponding author
- Asadullah, Khusru
- ORCID
- Schlange, Thomas
- Venue
- Nature Reviews Drug Discovery
- Published
- 2011-08-31
- Volume
- 10
- Issue
- 9
- Pages
- 712-712
- ISSN
- 1474-1776 (ISSN-L)
- 1474-1784
- OpenAlex
- W2067833766
- Cited by
- 1,888
- References
- 8
- Citations by year
- 2026: 56 · 2025: 60 · 2024: 86 · 2023: 108 · 2022: 138 · 2021: 126 · 2020: 131 · 2019: 171 · 2018: 147 · 2017: 176 · 2016: 202 · 2015: 183 · 2014: 141 · 2013: 98 · 2012: 57
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Nature Reviews Drug Discovery (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Keywords
- therapeutic target, reproducibility, publication bias, false positives, peer review, manuscript rejection, siRNA screen, drug target validation
- MeSH terms
- Drug Industry, Humans, Drug Design, Clinical Trials, Phase II as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.