On the reproducibility of science: unique identification of research resources in the biomedical literature
Open accessAbstract
Scientific reproducibility has been at the forefront of many news stories and there exist numerous initiatives to help address this problem. We posit that a contributor is simply a lack of specificity that is required to enable adequate research reproducibility. In particular, the inability to uniquely identify research resources, such as antibodies and model organisms, makes it difficult or impossible to reproduce experiments even where the science is otherwise sound. In order to better understand the magnitude of this problem, we designed an experiment to ascertain the "identifiability" of research resources in the biomedical literature. We evaluated recent journal articles in the fields of Neuroscience, Developmental Biology, Immunology, Cell and Molecular Biology and General Biology, selected randomly based on a diversity of impact factors for the journals, publishers, and experimental method reporting guidelines. We attempted to uniquely identify model organisms (mouse, rat, zebrafish, worm, fly and yeast), antibodies, knockdown reagents (morpholinos or RNAi), constructs, and cell lines. Specific criteria were developed to determine if a resource was uniquely identifiable, and included examining relevant repositories (such as model organism databases, and the Antibody Registry), as well as vendor sites. The results of this experiment show that 54% of resources are not uniquely identifiable in publications, regardless of domain, journal impact factor, or reporting requirements. For example, in many cases the organism strain in which the experiment was performed or antibody that was used could not be identified. Our results show that identifiability is a serious problem for reproducibility. Based on these results, we provide recommendations to authors, reviewers, journal editors, vendors, and publishers. Scientific efficiency and reproducibility depend upon a research-wide improvement of this substantial problem in science today.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In 2013, Vasilevsky et al. evaluated 200 papers and reported that 57% of cell lines were not uniquely identifiable.
“In 2013, Vasilevsky, et al. evaluated 200 papers and reported that 57% of cell lines were not uniquely identifiable; the true incidence was probably much higher because the survey did not include”
From The history, landscape, and outlook of human cell line authentication and security (2024) · cites this paper as [88]
The Results and Discussion (Cell lines; Table 1) report that 43% of cell lines were identifiable, so approximately 57% were not. But the Methods (Article selection) say the study analyzed 238 manuscripts, not 200; Table 1 shows that the cell-line findings came from 104 cell lines reported in 48 papers. (LLM, conf=0.94)
Experimental — accuracy not yet measuredResearch Resource Identifiers have been shown to decrease CCCM.
“Research Resource Identifiers (”
From The history, landscape, and outlook of human cell line authentication and security (2024) · cites this paper as [88]
The Results and Discussion report how often biomedical research resources were uniquely identifiable, and the Conclusions recommend stable identifiers. Neither section reports that Research Resource Identifiers decreased CCCM; searches for “CCCM” and for RRIDs reducing it found no such result in our copy. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
2 retracted
Retracted Science and the Retraction Index
Fang, F. C., & Casadevall, A. (2011) · Infection and Immunity · doi:10.1128/iai.05661-11 · Cited by 405 · see citing works
RetractedRetraction rates are on the rise
Cokol, M., Ozbay, F., & Rodriguez‐Esteban, R. (2008) · The EMBO Reports · doi:10.1038/sj.embor.7401143 · Cited by 149 · see citing works
RetractedRaise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 8× · see citing works
“Unique antibody identification required at least one of the following: (1) an identifier resolving to a universal registry/database identifier such as the Antibody Registry (www. antibodyregistry.org) or eagle-i repository (http://www.eagle-i.net), or a vendor name and catalog number for resolving to a single offering…”
BackgroundInfluentialEffective targeted gene ‘knockdown’ in zebrafish
Nasevicius, A., & Ekker, S. C. (2000) · Nature Genetics · doi:10.1038/79951 · Cited by 2,485 · see citing works
Kilkenny, C., Parsons, N., Kadyszewski, E., Festing, M. F. W., Cuthill, I. C., Fry, D., Hutton, J., & Altman, D. G. (2009) · PLoS ONE · doi:10.1371/journal.pone.0007824 · Cited by 860 · see citing works
Identification of essential genes in cultured mammalian cells using small interfering RNAs
Harborth, J., Elbashir, S. M., Bechert, K., Tuschl, T., & Weber, K. K. (2001) · Journal of Cell Science · doi:10.1242/jcs.114.24.4557 · Cited by 828 · see citing works
Check your cultures! A list of cross‐contaminated or misidentified cell lines
Capes‐Davis, A., Theodosopoulos, G., Atkin, I., Drexler, H. G., Kohara, A., MacLeod, R. A. F., Masters, J. R., Nakamura, Y., Reid, Y. A., Reddel, R. R., et al. (2010) · International Journal of Cancer · doi:10.1002/ijc.25242 · Cited by 520 · see citing works
Toward interoperable bioscience data
Sansone, S.-A., Rocca-Serra, P., Field, D., Maguire, E., Taylor, C., Hofmann, O., Fang, H., Neumann, S., Tong, W., Amaral-Zettler, L., et al. (2012) · Nature Genetics · doi:10.1038/ng.1054 · Cited by 475 · see citing works
Regional and strain-specific gene expression mapping in the adult mouse brain
Sandberg, R., Yasuda, R., Pankratz, D. G., Carter, T. A., Rio, J. A. D., Wodicka, L. M., Mayford, M. R., Lockhart, D. J., & Barlow, C. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.97.20.11038 · Cited by 446 · see citing works
The Costs of using Unauthenticated, Over-Passaged Cell Lines: How Much more Data do we Need?
Hughes, P., Marshall, D., Reid, Y., Parkes, H., & Gelber, C. (2007) · BioTechniques · doi:10.2144/000112598 · Cited by 434 · see citing works
ZFIN: enhancements and updates to the zebrafish model organism database
Bradford, Y. M., Conlin, T., Dunn, N., Fashena, D. S., Frazer, K., Howe, D. G., Knight, J., Mani, P., Martin, R., Moxon, S. A. T., et al. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkq1077 · Cited by 230 · see citing works
Self-citationFlyBase: improvements to the bibliography
Marygold, S. J., Leyland, P. C., Seal, R. L., Goodman, J. L., Thurmond, J., Strelets, V. B., & Wilson, R. J. (2012) · Nucleic Acids Research · doi:10.1093/nar/gks1024 · Cited by 227 · see citing works
Apparent HeLa Cell Contamination of Human Heteroploid Cell Lines
GARTLER, S. M. (1968) · Nature · doi:10.1038/217750a0 · Cited by 226 · see citing works
WormBase 2012: more genomes, more data, new website
Yook, K., Harris, T. W., Bieri, T. A., Cabunoc, A., Chan, J., Chen, W. J., Davis, P. A., Cruz, N., Duong, A., Fang, R., et al. (2011) · Nucleic Acids Research · doi:10.1093/nar/gkr954 · Cited by 184 · see citing works
Xenbase: gene expression and improved integration
Bowes, J. B., Snyder, K. A., Segerdell, E., Jarabek, C. J., Azam, K., Zorn, A. M., & Vize, P. D. (2009) · Nucleic Acids Research · doi:10.1093/nar/gkp953 · Cited by 138 · see citing works
Peering into peer-review at GigaScience
Edmunds, S. (2013) · GigaScience · doi:10.1186/2047-217x-2-1 · Cited by 109 · see citing works
Phuchareon, J., Ohta, Y., Woo, J. M., Eisele, D. W., & Tetsu, O. (2009) · PLoS ONE · doi:10.1371/journal.pone.0006040 · Cited by 105 · cited 1× · see citing works
“There are increasingly documented occurrences of cell line misidentification and contamination, as highlighted by the infamous HeLa contamination statistics (Gartler, 1968) and other cell line contaminations (Phuchareon et al., 2009).”
BackgroundISA-TAB-Nano: A Specification for Sharing Nanomaterial Research Data in Spreadsheet-based Format
Thomas, D., Gaheen, S., Harper, S. L., Fritts, M. J., Klaessig, F., Hahn-Dantona, E. A., Paik, D. S., Pan, S., Stafford, G. A., Freund, E. T., et al. (2013) · BMC Biotechnology · doi:10.1186/1472-6750-13-2 · Cited by 99 · see citing works
The Mouse Genome Database: Genotypes, Phenotypes, and Models of Human Disease
Bult, C. J., Eppig, J. T., Blake, J. A., Kadin, J. A., Richardson, J. E., & the Mouse Genome Database Group (2012) · Nucleic Acids Research · doi:10.1093/nar/gks1115 · Cited by 78 · see citing works
Dealing with Data: A Case Study on Information and Data Management Literacy
Haendel, M. A., Vasilevsky, N., & Wirz, J. A. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001339 · Cited by 68 · see citing works
Self-citationShen, S. P., Aleksic, J., & Russell, S. R. H. (2013) · BMC Developmental Biology · doi:10.1186/1471-213x-13-1 · Cited by 59 · see citing works
Minimum information about a protein affinity reagent (MIAPAR)
Bourbeillon, J., Orchard, S. E., Benhar, I., Borrebaeck, C. A. K., de Daruvar, A., Dübel, S., Frank, R., Gibson, F., Gloriam, D. E. I., Haslam, N. J., et al. (2010) · Nature Biotechnology · doi:10.1038/nbt0710-650 · Cited by 54 · see citing works
Open semantic annotation of scientific publications using DOMEO
Ciccarese, P., Ocana, M., & Clark, T. (2012) · Journal of Biomedical Semantics · doi:10.1186/2041-1480-3-s1-s1 · Cited by 52 · see citing works
Defects of the fetal forebrain in mice with hereditary agenesis of the corpus callosum
Wahłsten, D. (1987) · The Journal of Comparative Neurology · doi:10.1002/cne.902620205 · Cited by 52 · see citing works
Quantifying the use of bioresources for promoting their sharing in scientific research
BRIF working group, Mabile, L., Dalgleish, R. W. M., Þórisson, G. Á., Deschênes, M., Hewitt, R. E., Carpenter, J. E., Bravo, E., Filocamo, M., Gourraud, P., et al. (2013) · GigaScience · doi:10.1186/2047-217x-2-7 · Cited by 50 · see citing works
Electronic Laboratory Notebooks Progress and Challenges in Implementation
Machina, H. K., & Wild, D. J. (2013) · SLAS TECHNOLOGY · doi:10.1177/2211068213484471 · Cited by 49 · see citing works
Portelli, J., Aourz, N., Bundel, D. D., Meurs, A., Smolders, I. J., Michotte, Y. M., & Clinckers, R. (2009) · Epilepsy Research · doi:10.1016/j.eplepsyres.2009.09.009 · Cited by 39 · see citing works
Research resources: curating the new eagle-i discovery system
Vasilevsky, N., Johnson, T., Corday, K., Torniai, C., Brush, M., Segerdell, E., Wilson, M., Shaffer, C. M., Robinson, D., & Haendel, M. A. (2012) · Database · doi:10.1093/database/bar067 · Cited by 36 · see citing works
Self-citationAccuracy (trueness and precision) of measurement methods and results
(1995) · doi:10.3403/02526933 · Cited by 36 · see citing works
PhenoMiner: quantitative phenotype curation at the rat genome database
Laulederkind, S. J. F., Liu, W., Smith, J. R., Hayman, G. T., Wang, S., Nigam, R., Petri, V., Lowry, T. F., Pons, J. D., Dwinell, M. R., et al. (2013) · Database · doi:10.1093/database/bat015 · Cited by 30 · see citing works
A hybrid human and machine resource curation pipeline for the Neuroscience Information Framework
Bandrowski, A. E., Cachat, J., Li, Y., Muller, H. M., Sternberg, P. W., Ciccarese, P., Clark, T., Marenco, L. N., Wang, R., Astakhov, V., et al. (2012) · Database · doi:10.1093/database/bas005 · Cited by 27 · see citing works
Ezerman, E. B., & Kromer, L. F. (1985) · Experimental Brain Research · doi:10.1007/bf00261354 · Cited by 22 · see citing works
Involvement of early growth response-2 (Egr-2) in lipopolysaccharide-induced neuroinflammation
Yan, Y., Tan, X., Wu, X., Shao, B., Wu, X., Cao, J., Xu, J., Jin, W., Li, L., Xu, W., et al. (2013) · Journal of Molecular Histology · doi:10.1007/s10735-013-9482-y · Cited by 19 · see citing works
The Future of the Journal? Integrating research data with scientific discoursePreprint
de Waard, A. (2010) · Nature Precedings · doi:10.1038/npre.2010.4742.1 · Cited by 8 · see citing works
Buehring (2004) · doi:10.1290/1543-706x(2004)40
Flagged first, then by citations
Research works printed but not linked: 4
- Taylor M 2013 Peer review does not mean we can trust a published paper. Sauropod Vertebra Picture of the Week. Available at: http://svpow.com/2013/05/03/peer-review-does-not-mean-we-can-trust-a-published-paper/
- Thomson Reuters 2011 Journal Citation Report. http://wokinfo.com/products_tools/analytical/jcr/
- Wiley Online Publishing Immunology Author Guidelines. Available at: http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1365-2567/homepage/ForAuthors.html · DOI
- Wiley Online Publishing 2013 JCN Antibody Database. Available at: http://onlinelibrary.wiley.com/journal/10.1002/(ISSN)1096-9861/homepage/jcn_antibody_database.htm · DOI
Cited by
Sorted by citations
The ethics of algorithms: Mapping the debate
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016) · Big Data & Society · doi:10.1177/2053951716679679 · Cited by 3,432 · see citing works
Policy: NIH plans to enhance reproducibility
Collins, F. S., & Tabak, L. A. (2014) · Nature · doi:10.1038/505612a · Cited by 1,247 · see citing works
The economics of reproducibility in preclinical research
Freedman, L. P., Cockburn, I. M., & Simcoe, T. S. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002165 · Cited by 1,055 · see citing works
Reproducibility: Standardize antibodies used in research
Bradbury, A., & Plückthun, A. (2015) · Nature · doi:10.1038/518027a · Cited by 647 · see citing works
ITK: enabling reproducible research and open science
McCormick, M. M., Liu, X., Jomier, J., Marion, C., & Ibáñez, L. (2014) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2014.00013 · Cited by 371 · see citing works
Public Data Archiving in Ecology and Evolution: How Well Are We Doing?
Roche, D. G., Kruuk, L. E. B., Lanfear, R., & Binning, S. A. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002295 · Cited by 303 · see citing works
An open investigation of the reproducibility of cancer biology research
Errington, T. M., Iorns, E., Gunn, W., Tan, F. E., Lomax, J., & Nosek, B. A. (2014) · eLife · doi:10.7554/elife.04333 · Cited by 312 · see citing works
OMICtools: an informative directory for multi-omic data analysis
Henry, V., Bandrowski, A. E., Pepin, A.-S., Gonzalez, B. J., & Desfeux, A. (2014) · Database · doi:10.1093/database/bau069 · Cited by 236 · see citing works
Zebrafish information network, the knowledgebase for <i>Danio rerio</i> research
Bradford, Y. M., Van Slyke, C. E., Ruzicka, L., Singer, A., Eagle, A., Fashena, D., Howe, D. G., Frazer, K., Martin, R., Paddock, H., et al. (2022) · Genetics · doi:10.1093/genetics/iyac016 · Cited by 229 · see citing works
Self-citationToward best practice in cancer mutation detection with whole-genome and whole-exome sequencing
Xiao, W., Ren, L., Chen, Z., Fang, L. T., Zhao, Y., Lack, J., Guan, M., Zhu, B., Jaeger, E. B., Kerrigan, L., et al. (2021) · Nature Biotechnology · doi:10.1038/s41587-021-00994-5 · Cited by 187 · see citing works
Gil, Y., David, C. H., Demir, İ., Essawy, B. T., Fulweiler, R. W., Goodall, J. L., Karlstrom, L., Lee, H., Mills, H. J., Oh, J., et al. (2016) · Earth and Space Science · doi:10.1002/2015ea000136 · Cited by 182 · see citing works
Fixing problems with cell lines
Lorsch, J. R., Collins, F. S., & Lippincott-Schwartz, J. (2014) · Science · doi:10.1126/science.1259110 · Cited by 179 · see citing works
Reproducible and reusable research: are journal data sharing policies meeting the mark?
Vasilevsky, N. A., Minnier, J., Haendel, M. A., & Champieux, R. E. (2017) · PeerJ · doi:10.7717/peerj.3208 · Cited by 165 · see citing works
Self-citationA Review on Blockchain Technology and Blockchain Projects Fostering Open Science
Leible, S., Steffen Schlager, C., Schubotz, M., & Gipp, B. (2019) · Frontiers in Blockchain · doi:10.3389/fbloc.2019.00016 · Cited by 176 · see citing works
Oxygenation in Cell Culture: Critical Parameters for Reproducibility Are Routinely Not Reported
Al-Ani, A., Toms, D., Kondro, D., Thundathil, J., Yu, Y., & Ungrin, M. (2018) · PLOS ONE · doi:10.1371/journal.pone.0204269 · Cited by 152 · see citing works
Editorial: preclinical data reproducibility for R&D - the challenge for neuroscience
Steckler, T. (2015) · SpringerPlus · doi:10.1186/2193-1801-4-1 · Cited by 141 · see citing works
Antibody anarchy: A call to order
Baker, M. (2015) · Nature · doi:10.1038/527545a · Cited by 140 · see citing works
Reproducibility: changing the policies and culture of cell line authentication
Freedman, L. P., Gibson, M. C., Ethier, S. P., Soule, H. R., Neve, R. M., & Reid, Y. A. (2015) · Nature Methods · doi:10.1038/nmeth.3403 · Cited by 136 · see citing works
Achieving human and machine accessibility of cited data in scholarly publications
Starr, J., Castro, E., Crosas, M., Dumontier, M., Downs, R. R., Duerr, R. E., Laurel, H., Haendel, M. A., Herman, I., Hodson, S., et al. (2015) · PeerJ Computer Science · doi:10.7717/peerj-cs.1 · Cited by 129 · see citing works
Self-citationIrreproducibility in Preclinical Biomedical Research: Perceptions, Uncertainties, and Knowledge Gaps
Jarvis, M. F., & Williams, M. (2016) · Trends in Pharmacological Sciences · doi:10.1016/j.tips.2015.12.001 · Cited by 129 · see citing works
Never waste a good crisis: confronting reproducibility in translational research
Drucker, D. J. (2016) · Cell Metabolism · doi:10.1016/j.cmet.2016.08.006 · Cited by 117 · see citing works
Biocuration: Distilling data into knowledge
International Society for Biocuration (2018) · PLoS Biology · doi:10.1371/journal.pbio.2002846 · Cited by 104 · see citing works
Quality Issues of Research Antibodies
Weller, M. G. (2016) · Analytical Chemistry Insights · doi:10.4137/aci.s31614 · Cited by 117 · see citing works
The center for expanded data annotation and retrieval
Musen, M. A., Bean, C. A., Cheung, K., Dumontier, M., Durante, K., Gevaert, O., Gonzalez-Beltran, A. N., Khatri, P., Kleinstein, S. H., O’Connor, M. J., et al. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv048 · Cited by 111 · see citing works
Toward standard practices for sharing computer code and programs in neuroscience
Eglen, S. J., Marwick, B., Halchenko, Y. O., Hanke, M., Sufi, S., Gleeson, P., Silver, R. A., Davison, A. P., Lanyon, L., Abrams, M. B., et al. (2017) · Nature Neuroscience · doi:10.1038/nn.4550 · Cited by 108 · see citing works
Calibrating the Scientific Ecosystem Through Meta-Research
Hardwicke, T. E., Serghiou, S., Janiaud, P., Danchev, V., Crüwell, S., Goodman, S. N., & Ioannidis, J. P. A. (2020) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-031219-041104 · Cited by 98 · see citing works
The Hemiptera (Insecta) of Canada: Constructing a Reference Library of DNA Barcodes
Gwiazdowski, R., Foottit, R. G., Maw, H. E. L., & Hebert, P. D. N. (2015) · PLoS ONE · doi:10.1371/journal.pone.0125635 · Cited by 102 · see citing works
Bridging Translation by Improving Preclinical Study Design in AKI
deCaestecker, M. P., Humphreys, B. D., Liu, K. D., Fissell, W. H., Cerdá, J., Nolin, T. D., Askenazi, D. J., Mour, G. K., Harrell, F. E., Pullen, N., et al. (2015) · Journal of the American Society of Nephrology · doi:10.1681/asn.2015070832 · Cited by 98 · see citing works
The Antibody Registry: ten years of registering antibodies
Bandrowski, A. E., Pairish, M., Eckmann, P., Grethe, J. S., & Martone, M. E. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkac927 · Cited by 89 · see citing works
The Increasing Urgency for Standards in Basic Biologic Research
Freedman, L. P., & Inglese, J. (2014) · Cancer Research · doi:10.1158/0008-5472.can-14-0925 · Cited by 87 · see citing works
The Rigor and Transparency Index Quality Metric for Assessing Biological and Medical Science Methods
Menke, J., Roelandse, M., Ozyurt, B., Martone, M., & Bandrowski, A. (2020) · iScience · doi:10.1016/j.isci.2020.101698 · Cited by 90 · see citing works
Sources of error in the retracted scientific literature
Casadevall, A., Steen, R. G., & Fang, F. C. (2014) · The FASEB Journal · doi:10.1096/fj.14-256735 · Cited by 89 · see citing works
Improving transparency and scientific rigor in academic publishing
Prager, E. M., Chambers, K. E., Plotkin, J. L., McArthur, D. L., Bandrowski, A. E., Bansal, N., Martone, M. E., Bergstrom, H. C., Bespalov, A., & Graf, C. (2019) · Journal of Neuroscience Research · doi:10.1002/jnr.24340 · Cited by 83 · see citing works
Andrade, E., Bento, A. F., Cavalli, J., de Oliveira, S. K. F., Freitas, C. S., Marcon, R., Schwanke, R. C., Siqueira, J. M., & Calixto, J. Β. (2016) · Brazilian Journal of Medical and Biological Research · doi:10.1590/1414-431x20165644 · Cited by 81 · see citing works
The Resource Identification Initiative: A cultural shift in publishing
Bandrowski, A., Brush, M., Grethe, J. S., Haendel, M. A., Kennedy, D. N., Hill, S., Hof, P. R., Martone, M. E., Pols, M., Tan, S., et al. (2015) · F1000Research · doi:10.12688/f1000research.6555.2 · Cited by 80 · see citing works
Self-citationBogue, M. A., Philip, V. M., Walton, D. O., Grubb, S. C., Dunn, M. H., Kolishovski, G., Emerson, J., Mukherjee, G., Stearns, T. M., He, H., et al. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz1032 · Cited by 80 · see citing works
Tripathy, S. J., Burton, S. D., Geramita, M. A., Gerkin, R. C., & Urban, N. N. (2015) · Journal of Neurophysiology · doi:10.1152/jn.00237.2015 · Cited by 72 · see citing works
Self-citationBandrowski, A. E., & Martone, M. E. (2016) · Neuron · doi:10.1016/j.neuron.2016.04.030 · Cited by 75 · see citing works
Navigating the Phenotype Frontier: The Monarch Initiative
McMurry, J. A., Köhler, S., Balhoff, J. P., Borromeo, C. D., Brush, M., Carbon, S., Conlin, T., Dunn, N., Engelstad, M., Foster, E. D., et al. (2016) · Genetics · doi:10.1534/genetics.116.188870 · Cited by 74 · see citing works
Self-citationBogue, M. A., Grubb, S. C., Walton, D. O., Philip, V. M., Kolishovski, G., Stearns, T. M., Dunn, M. H., Skelly, D. A., Kadakkuzha, B. M., TeHennepe, G., et al. (2017) · Nucleic Acids Research · doi:10.1093/nar/gkx1082 · Cited by 72 · see citing works
Sharing biological data: why, when, and how
Wilson, S. L., Way, G. P., Bittremieux, W., Armache, J., Haendel, M. A., & Hoffman, M. M. (2021) · FEBS Letters · doi:10.1002/1873-3468.14067 · Cited by 68 · see citing works
Self-citationSTRENDA DB: enabling the validation and sharing of enzyme kinetics data
Swainston, N., Baici, A., Bakker, B. M., Cornish-Bowden, A. J., Fitzpatrick, P. F., Halling, P. J., Leyh, T. S., O’Donovan, C., Raushel, F. M., Reschel, U., et al. (2018) · FEBS Journal · doi:10.1111/febs.14427 · Cited by 63 · see citing works
Biomedical text mining for research rigor and integrity: tasks, challenges, directions
Kilicoglu, H. (2017) · Briefings in Bioinformatics · doi:10.1093/bib/bbx057 · Cited by 61 · see citing works
Improving transparency and scientific rigor in academic publishing
Prager, E. M., Chambers, K. E., Plotkin, J. L., McArthur, D. L., Bandrowski, A. E., Bansal, N., Martone, M. E., Bergstrom, H. C., Bespalov, A., & Graf, C. (2018) · Brain and Behavior · doi:10.1002/brb3.1141 · Cited by 61 · see citing works
User-friendly workflows for catchment modelling: Towards reproducible SWAT+ model studies
Chawanda, C. J., George, C., Thiery, W., van Griensven, A., Tech, J., Arnold, J. G., & Srinivasan, R. (2020) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2020.104812 · Cited by 60 · see citing works
Quantity and Reporting Quality of Kidney Research
Chatzimanouil, M. K. T., Wilkens, L., & Anders, H. (2018) · Journal of the American Society of Nephrology · doi:10.1681/asn.2018050515 · Cited by 60 · see citing works
Gonçalves, R. S., O’Connor, M. J., Martínez-Romero, M., Egyedi, A. L., Willrett, D., Graybeal, J. B., & Musen, M. A. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-68204-4_10 · Cited by 56 · see citing works
The reproducibility of biomedical research: Sleepers awake!
Bustin, S. A. (2014) · Biomolecular Detection and Quantification · doi:10.1016/j.bdq.2015.01.002 · Cited by 56 · see citing works
Ten Basic Rules of Antibody Validation
Weller, M. G. (2018) · Analytical Chemistry Insights · doi:10.1177/1177390118757462 · Cited by 55 · see citing works
The Antibody Society’s antibody validation webinar series
Voskuil, J. L. A., Bandrowski, A. E., Begley, C. G., Bradbury, A. M., Chalmers, A. D., Gomes, A. V., Hardcastle, T., Lund‐Johansen, F., Plückthun, A., Roncador, G., et al. (2020) · mAbs · doi:10.1080/19420862.2020.1794421 · Cited by 54 · see citing works
Antibodies and methods for immunohistochemistry of extracellular matrix proteins
Rickelt, S., & Hynes, R. O. (2018) · Matrix Biology · doi:10.1016/j.matbio.2018.04.011 · Cited by 51 · see citing works
Harnessing QbD, Programming Languages, and Automation for Reproducible Biology
Sadowski, M. I., Grant, C., & Fell, T. S. (2015) · Trends in biotechnology · doi:10.1016/j.tibtech.2015.11.006 · Cited by 51 · see citing works
Accelerating Biomedical Discoveries through Rigor and Transparency
Hewitt, J., Brown, L. L., Murphy, S. J., Grieder, F. B., & Silberberg, S. D. (2017) · ILAR Journal · doi:10.1093/ilar/ilx011 · Cited by 53 · see citing works
The Resource Identification Initiative: a cultural shift in publishing
Bandrowski, A. E., Brush, M., Grethe, J. S., Haendel, M. A., Kennedy, D. N., Hill, S., Hof, P. R., Martone, M. E., Pols, M. S., Tan, S. C., et al. (2015) · Brain and Behavior · doi:10.1002/brb3.417 · Cited by 52 · see citing works
Self-citationScanlan, L. D., Lund, S. P., Coşkun, S. H., Hanna, S. K., Johnson, M. E., Sims, C. M., Brignoni, K., Lapasset, P. E., Petersen, E. J., Elliott, J. T., et al. (2018) · Scientific Reports · doi:10.1038/s41598-018-19187-3 · Cited by 52 · see citing works
Managing risks in drug discovery: reproducibility of published findings
Kannt, A., & Wieland, T. (2016) · Naunyn-Schmiedeberg s Archives of Pharmacology · doi:10.1007/s00210-016-1216-8 · Cited by 51 · see citing works
Launch of an Infrastructure for Health Research: BBMRI-ERIC
Litton, J. (2018) · Biopreservation and Biobanking · doi:10.1089/bio.2018.0027 · Cited by 52 · see citing works
The resource identification initiative: a cultural shift in publishing
Bandrowski, A., Brush, M., Grethe, J. S., Haendel, M. A., Kennedy, D. N., Hill, S., Hof, P. R., Martone, M. E., Pols, M., Tan, S., et al. (2015) · F1000Research · doi:10.12688/f1000research.6555.1 · Cited by 50 · see citing works
Self-citationReiser, L., Berardini, T., Li, D., Muller, R., Strait, E., Li, Q., Mezheritsky, Y., Vetushko, A., & Huala, E. (2016) · Database · doi:10.1093/database/baw018 · Cited by 47 · see citing works
Incidences of problematic cell lines are lower in papers that use RRIDs to identify cell lines
Babic, Z., Capes-Davis, A., Martone, M. E., Bairoch, A., Ozyurt, I. B., Gillespie, T. H., & Bandrowski, A. E. (2019) · eLife · doi:10.7554/elife.41676 · Cited by 44 · see citing works
The Impact of Preclinical Irreproducibility on Drug Development
Freedman, L. P., & Gibson, M. (2014) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.9 · Cited by 43 · see citing works
I'll take that to go: Big data bags and minimal identifiers for exchange of large, complex datasets
Chard, K., D’Arcy, M., Heavner, B., Foster, I. T., Kesselman, C., Madduri, R., Rodriguez, A., Soiland‐Reyes, S., Goble, C., Clark, K. A., et al. (2016) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2016.7840618 · Cited by 43 · see citing works
OpenWorkstation: A modular open-source technology for automated in vitro workflows
Eggert, S., Mieszczanek, P., Meinert, C., & Hutmacher, D. W. (2020) · HardwareX · doi:10.1016/j.ohx.2020.e00152 · Cited by 41 · see citing works
The Resource Identification Initiative: a cultural shift in publishing
Bandrowski, A., Brush, M., Grethe, J. S., Haendel, M. A., Kennedy, D. N., Hill, S., Hof, P. R., Martone, M. E., Pols, M., Tan, S. C., et al. (2016) · Journal of Comparative Neurology · doi:10.1002/cne.23913 · Cited by 41 · see citing works
Self-citationGonzalez-Beltran, A. N., Li, P., Zhao, J., Ávila-García, M. S., Roos, M., Thompson, M., van der Horst, E., Kaliyaperumal, R., Luo, R., Lee, T., et al. (2015) · PLoS ONE · doi:10.1371/journal.pone.0127612 · Cited by 41 · see citing works
Reporting transparency: making the ethical mandate explicit
Nicholls, S. G., Langan, S., Benchimol, E. I., & Moher, D. (2016) · BMC Medicine · doi:10.1186/s12916-016-0587-5 · Cited by 41 · see citing works
Baseline mRNA expression differs widely between common laboratory strains of zebrafish
Holden, L. A., & Brown, K. H. (2018) · Scientific Reports · doi:10.1038/s41598-018-23129-4 · Cited by 40 · see citing works
An Open Science Peer Review Oath
Aleksic, J., Alexa, A., Attwood, T. K., Hong, N. C., Dahlö, M., Davey, R., Dinkel, H., Förstner, K. U., Grigorov, I., Hèriché, J.-K., et al. (2015) · F1000Research · doi:10.12688/f1000research.5686.2 · Cited by 35 · see citing works
Combining hypothesis- and data-driven neuroscience modeling in FAIR workflows
Eriksson, O., Bhalla, U. S., Blackwell, K. T., Crook, S. M., Keller, D. X., Kramer, A., Linne, M., Saudargienė, A., Wade, R. C., & Kotaleski, J. H. (2022) · eLife · doi:10.7554/elife.69013 · Cited by 35 · see citing works
Freedman, L. P., Gibson, M. C., Bradbury, A. M., Buchberg, A. M., Davis, D., Dolled-Filhart, M. P., Lund‐Johansen, F., & Rimm, D. L. (2016) · BioTechniques · doi:10.2144/000114431 · Cited by 35 · see citing works
The Resource Identification Initiative: A Cultural Shift in Publishing
Initiative, R. R. I., Bandrowski, A. E., Brush, M., Grethe, J. S., Haendel, M. A., Kennedy, D. N., Hill, S., Hof, P. R., Martone, M. E., Pols, M. S., et al. (2015) · Neuroinformatics · doi:10.1007/s12021-015-9284-3 · Cited by 34 · see citing works
Self-citationReproducibility in Machine Learning for HealthPreprint
McDermott, M. B. A., Wang, S., Marinsek, N., Ranganath, R., Ghassemi, M., & Foschini, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.01463 · Cited by 34 · see citing works
Authentication: A Standard Problem or a Problem of Standards?
Capes-Davis, A., & Neve, R. M. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002477 · Cited by 33 · see citing works
How Transparency Affects Survey Responses
Connors, E. C., Krupnikov, Y., & Ryan, J. B. (2019) · Public Opinion Quarterly · doi:10.1093/poq/nfz013 · Cited by 32 · see citing works
Kroon, C., Breuer, L., Jones, L., An, J., Akan, A., Mohamed Ali, E. A., Busch, F., Fislage, M., Ghosh, B., Hellrigel-Holderbaum, M., et al. (2022) · PLOS Biology · doi:10.1371/journal.pbio.3001783 · Cited by 31 · see citing works
Antibody characterization is critical to enhance reproducibility in biomedical research
Kahn, R. A., Virk, H., Laflamme, C., Houston, D. W., Polinski, N. K., Meijers, R., Levey, A. I., Saper, C. B., Errington, T. M., Turn, R. E., et al. (2024) · eLife · doi:10.7554/elife.100211 · Cited by 40 · see citing works
Do Corresponding Authors Take Responsibility for Their Work? A Covert Survey
Teunis, T., Nota, S. P. F. T., & Schwab, J. H. (2014) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-014-3868-3 · Cited by 33 · see citing works
Gong, B., Murray, K. D., & Trimmer, J. S. (2015) · New Biotechnology · doi:10.1016/j.nbt.2015.11.007 · Cited by 29 · see citing works
Complexity of Delivering Precision Medicine: Opportunities and Challenges
Davis, A. A., McKee, A. E., Kibbe, W. A., & Villaflor, V. M. (2018) · American Society of Clinical Oncology Educational Book · doi:10.1200/edbk_200279 · Cited by 31 · see citing works
Enhancing reproducibility: Failures from Reproducibility Initiatives underline core challenges
Mullane, K., & Williams, M. (2017) · Biochemical Pharmacology · doi:10.1016/j.bcp.2017.04.008 · Cited by 28 · see citing works
Chemical Tools for Studying Lipid-Binding Class A G Protein–Coupled Receptors
Cooper, A. G., Singh, S., Hook, S. M., Tyndall, J. D. A., & Vernall, A. J. (2017) · Pharmacological Reviews · doi:10.1124/pr.116.013243 · Cited by 26 · see citing works
Preclinical data reproducibility for R&D—the challenge for neuroscience
Steckler, T. (2015) · Psychopharmacology · doi:10.1007/s00213-014-3836-3 · Cited by 25 · see citing works
How to avoid pitfalls in antibody use
Pauly, D., & Hanack, K. (2015) · F1000Research · doi:10.12688/f1000research.6894.1 · Cited by 24 · see citing works
Menke, J., Eckmann, P., Ozyurt, I. B., Roelandse, M., Anderson, N., Grethe, J., Gamst, A., & Bandrowski, A. (2022) · Journal of Medical Internet Research · doi:10.2196/37324 · Cited by 25 · see citing works
Disease insights through cross-species phenotype comparisons
Haendel, M. A., Vasilevsky, N., Brush, M., Hochheiser, H., Jacobsen, J. O. B., Oellrich, A., Mungall, C., Washington, N. L., Köhler, S., Lewis, S., et al. (2015) · Mammalian Genome · doi:10.1007/s00335-015-9577-8 · Cited by 23 · see citing works
Self-citationA computational modelling framework to quantify the effects of passaging cell lines
Jin, W., Penington, C. J., McCue, S. W., & Simpson, M. J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0181941 · Cited by 23 · see citing works
The health care and life sciences community profile for dataset descriptions
Dumontier, M., Gray, A. J. G., Marshall, M. S., Alexiev, V., Ansell, P., Bader, G. D., Baran, J., Bolleman, J. T., Callahan, A., Cruz-Toledo, J., et al. (2016) · PeerJ · doi:10.7717/peerj.2331 · Cited by 22 · see citing works
Self-citationLindner, M. D., Vancea, A., Chen, M., & Chacko, G. W. (2015) · American Journal of Evaluation · doi:10.1177/1098214015582049 · Cited by 21 · see citing works
Propelling the paradigm shift from reductionism to systems nutrition
Kaput, J., Perozzi, G., Radonjić, M., & Virgili, F. (2017) · Genes & Nutrition · doi:10.1186/s12263-016-0549-8 · Cited by 20 · see citing works
Steckler, T., Brose, K., Haas, M., Kas, M. J., Koustova, E., & Bespalov, A. (2015) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2015.05.011 · Cited by 20 · see citing works
Indicators for the use of robotic labs in basic biomedical research: a literature analysis
Groth, P., & Cox, J. (2017) · PeerJ · doi:10.7717/peerj.3997 · Cited by 21 · see citing works
Checklist for Reproducibility of Deep Learning in Medical Imaging
Moassefi, M., Singh, Y., Conte, G. M., Khosravi, B., Rouzrokh, P., Vahdati, S., Safdar, N. M., Moy, L. P., Kitamura, F., Gentili, A., et al. (2024) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-024-01065-2 · Cited by 19 · see citing works
Towards FAIR protocols and workflows: the OpenPREDICT use case
Çelebi, R., Moreira, J., Hassan, A., Ayyar, S., Ridder, L., Kuhn, T., & Dumontier, M. (2020) · PeerJ Computer Science · doi:10.7717/peerj-cs.281 · Cited by 19 · see citing works
Wang, D., Ostenson, J., & Smith, D. S. (2019) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2019.11.015 · Cited by 19 · see citing works
Misspellings or “miscellings”—Non‐verifiable and unknown cell lines in cancer research publications
Oste, D. J., Pathmendra, P., Richardson, R. A. K., Johnson, G., Ao, Y., Arya, M. D., Enochs, N. R., Hussein, M., Kang, J., Lee, A., et al. (2024) · International Journal of Cancer · doi:10.1002/ijc.34995 · Cited by 18 · see citing works
Addgene: Making Materials Sharing “Science As Usual”
Kamens, J. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001991 · Cited by 18 · see citing works
Antibody Validation in Bioimaging Applications Based on Endogenous Expression of Tagged Proteins
Skogs, M., Stadler, C., Schutten, R., Hjelmare, M., Gnann, C., Björk, L., Poser, I., Hyman, A. A., Uhlén, M., & Lundberg, E. K. (2016) · Journal of Proteome Research · doi:10.1021/acs.jproteome.6b00821 · Cited by 18 · see citing works
Ernst, L., Kopaczka, M., Schulz, M., Talbot, S. R., Struve, B., Häger, C., Bleich, A., Durst, M. S., Jirkof, P., Arras, M., et al. (2019) · Laboratory Animals · doi:10.1177/0023677219881664 · Cited by 18 · see citing works
Establishment, authenticity, and characterization of cervical cancer cell lines
Martinez, M. D. L. Z., Mendoza, C. M. L., Salazar, J. T., García-Carrancá, A. M., Cervantes, M. A. C., & Alcántara-Quintana, L. E. (2022) · Molecular & Cellular Oncology · doi:10.1080/23723556.2022.2078628 · Cited by 17 · see citing works
Enhancing Scientific Foundations to Ensure Reproducibility
Hsieh, T., Vaickus, M. H., & Remick, D. G. (2017) · American Journal Of Pathology · doi:10.1016/j.ajpath.2017.08.028 · Cited by 17 · see citing works
- Authors
- Vasilevsky, N. A., Brush, M. H., Paddock, H., Ponting, L., Tripathy, S. J., LaRocca, G. M., & Haendel, M. A.
- Affiliations
- Oregon Health & Science University (US)
- University of Oregon (US)
- University of Cambridge (GB)
- Center for the Neural Basis of Cognition (US)
- Corresponding author
- Vasilevsky, Nicole A.
- Venue
- PeerJ
- Publisher
- PeerJ
- Published
- 2013-09-05
- Volume
- 1
- Pages
- e148
- ISSN
- 2167-8359 (ISSN-L)
- PMC
- PMC3771067
- OpenAlex
- W2114224807
- Cited by
- 271
- References
- 35
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 8 · 2023: 7 · 2022: 17 · 2021: 10 · 2020: 42 · 2019: 17 · 2018: 28 · 2017: 34 · 2016: 30 · 2015: 45 · 2014: 21 · 2013: 2
- FWCI
- 96.45
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- PeerJ (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Purdue e-Pubs (Purdue University System) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- National Institutes of Health (R24OD011883, 5r01dc005798-10)
- National Human Genome Research Institute (P41HG000739, P41HG002659)
- National Institute on Deafness and Other Communication Disorders (R01-DC-005798, R01DC011184)
- Topics
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Keywords
- scientific reproducibility, model organisms, immunoglobulins, RNA interference, morpholino, cell lines, research constructs, model organism databases, biomedical literature, reporting guidelines
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
