Toil enables reproducible, open source, big biomedical data analyses
Open accessPublished version of a preprint
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The new MGnify pipeline defined the CWL workflow first and utilizes toil-cwl-runner to orchestrate execution on their compute infrastructure.
“The new pipeline is a major departure from this practice, where we have defined the CWL workflow first, and now utilize toil-cwl-runner (a CWL compatibility layer that is a part of the Toil workflow execution engine (26)) to orchestrate the execution on our compute infrastructure.”
From MGnify: the microbiome analysis resource in 2020 (2019) · cites this paper as [26]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
STAR: ultrafast universal RNA-seq aligner
Dobin, A., Davis, C. A., Schlesinger, F., Drenkow, J., Zaleski, C., Jha, S., Batut, P., Chaisson, M., & Gingeras, T. R. (2013) · Bioinformatics · doi:10.1093/bioinformatics/bts635 · Cited by 51,709 · see citing works
Lecture Notes in Computer Science 1205
(1999) · Industrial Robot the international journal of robotics research and application · doi:10.1108/ir.1999.04926fae.001 · Cited by 39,228 · see citing works
The Genome Analysis Toolkit: A MapReduce framework for analyzing next-generation DNA sequencing data
McKenna, A., Hanna, M., Banks, E., Sivachenko, A., Cibulskis, K., Kernytsky, A., Garimella, K., Altshuler, D., Gabriel, S., Daly, M., et al. (2010) · Genome Research · doi:10.1101/gr.107524.110 · Cited by 27,895 · see citing works
Self-citationRSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome
Li, B., & Dewey, C. N. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-323 · Cited by 22,179 · see citing works
The Human Genome Browser at UCSC
Kent, W. J., Sugnet, C. W., Furey, T. S., Roskin, K. M., Pringle, T. H., Zahler, A. M., & Haussler, A. D. (2002) · Genome Research · doi:10.1101/gr.229102 · Cited by 10,400 · see citing works
Self-citationNear-optimal probabilistic RNA-seq quantification
Bray, N. L., Pimentel, H., Melsted, P., & Pachter, L. (2016) · Nature Biotechnology · doi:10.1038/nbt.3519 · Cited by 9,723 · see citing works
The cancer genome atlas pan-cancer analysis project
The Cancer Genome Atlas Research Network, Weinstein, J. N., Collisson, E. A., Mills, G. B., Shaw, K. R. M., Ozenberger, B. A., Ellrott, K., Shmulevich, I., Sander, C., & Stuart, J. M. (2013) · Nature Genetics · doi:10.1038/ng.2764 · Cited by 9,092 · see citing works
Integrated genomic analyses of ovarian carcinoma
Perou, C. M. (2019) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/hvp3-wg08 · Cited by 7,927 · see citing works
The Genotype-Tissue Expression (GTEx) pilot analysis: Multitissue gene regulation in humans
The GTEx Consortium, Ardlie, K. G., Deluca, D. S., Segrè, A. V., Sullivan, T. J., Young, T. R., Gelfand, E. T., Trowbridge, C. A., Maller, J. B., Tukiainen, T., et al. (2015) · Science · doi:10.1126/science.1262110 · Cited by 5,062 · see citing works
Self-citationGoecks, J., Nekrutenko, A., Taylor, J., & The Galaxy Team (2010) · Genome biology · doi:10.1186/gb-2010-11-8-r86 · Cited by 3,437 · see citing works
SLURM: Simple Linux Utility for Resource Management
Yoo, A., Jette, M. A., & Grondona, M. (2003) · Lecture notes in computer science · doi:10.1007/10968987_3 · Cited by 1,578 · see citing works
The human transcriptome across tissues and individuals
Melé, M., Ferreira, P. G., Reverter, F., DeLuca, D. S., Monlong, J., Sammeth, M., Young, T. R., Goldmann, J. M., Pervouchine, D. D., Sullivan, T. J., et al. (2015) · Science · doi:10.1126/science.aaa0355 · Cited by 1,304 · see citing works
Self-citationI-SPY 2: An Adaptive Breast Cancer Trial Design in the Setting of Neoadjuvant Chemotherapy
Barker, A., Sigman, C. C., Gj, K., Hylton, N., Berry, D., & Esserman, L. J. (2009) · Clinical Pharmacology & Therapeutics · doi:10.1038/clpt.2009.68 · Cited by 767 · see citing works
International Cancer Genome Consortium Data Portal--a one-stop shop for cancer genomics data
Zhang, J., Baran, J., Cros, A., Guberman, J. M., Haider, S. A., Hsu, J. C., Liang, Y., Rivkin, E., Wang, J., Whitty, B. R., et al. (2011) · Database · doi:10.1093/database/bar026 · Cited by 637 · see citing works
Proceedings of the 2008 ACM SIGMOD international conference on Management of data
Lakshmanan, L. V. S., Ng, R. T., & Shasha, D. E. (2008) · Cited by 421 · see citing works
Sun Grid Engine: towards creating a compute power grid
Gentzsch, W. (2002) · IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid) · doi:10.1109/ccgrid.2001.923173 · Cited by 378 · see citing works
The Genotype-Tissue Expression (GTEx) project
Lonsdale, J., Thomas, J., Salvatore, M., Phillips, R., Lo, E., Shad, S., Hasz, R., Walters, G., Garcia, F., Young, N., et al. (2013) · Nature Genetics · doi:10.1038/ng.2653 · Cited by 263 · see citing works
Self-citationAlbrecht, M., Donnelly, P., Bui, P. J., & Thain, D. (2012) · doi:10.1145/2443416.2443417 · Cited by 155 · see citing works
ADAM: Genomics Formats and Processing Patterns for Cloud Scale Computing
Massie, M., Nothaft, F. A., Hartl, C., et al. (2013) · UC Berkeley · Cited by 94 · see citing works
Self-citationLindahl, G., Saarinen, N., Abrahamsson, A. E., & Dabrosin, C. (2010) · Cancer Research · doi:10.1158/0008-5472.can-10-2289 · Cited by 83 · see citing works
Common Workflow LanguagePreprint
Crusoe, M. R. (2015) · OSF Preprints (OSF Preprints) · Cited by 65 · see citing works
UK gears up to decode 100 000 genomes from NHS patients
Siva, N. (2014) · The Lancet · doi:10.1016/s0140-6736(14)62453-3 · Cited by 53 · see citing works
Thieu, T., Joshi, S., Warren, S., & Korkin, D. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts042 · Cited by 36 · see citing works
Proceedings of the 1st ACM SIGMOD Workshop on Scalable Workflow Execution Engines and Technologies
Hidders, J., Missier, P., & Sroka, J. (2012) · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Visualizing and interpreting cancer genomics data via the Xena platform
Goldman, M. J., Craft, B., Hastie, M., Repečka, K., McDade, F., Kamath, A., Banerjee, A., Luo, Y., Rogers, D., Brooks, A. N., et al. (2020) · Nature Biotechnology · doi:10.1038/s41587-020-0546-8 · Cited by 4,481 · see citing works
Self-citationxCell: digitally portraying the tissue cellular heterogeneity landscape
Aran, D., Hu, Z., & Butte, A. J. (2017) · Genome Biology · doi:10.1186/s13059-017-1349-1 · Cited by 4,397 · see citing works
A pan-cancer single-cell transcriptional atlas of tumor infiltrating myeloid cells
Cheng, S., Li, Z., Gao, R., Xing, B., Gao, Y., Yang, Y., Qin, S., Zhang, L., Ouyang, H., Du, P., et al. (2021) · Cell · doi:10.1016/j.cell.2021.01.010 · Cited by 1,601 · see citing works
Conserved pan-cancer microenvironment subtypes predict response to immunotherapy
Bagaev, A., Kotlov, N., Nomie, K., Svekolkin, V., Gafurov, A., Isaeva, O., Osokin, N., Kozlov, I., Frenkel, F., Gancharova, O., et al. (2021) · Cancer Cell · doi:10.1016/j.ccell.2021.04.014 · Cited by 1,302 · see citing works
Massive mining of publicly available RNA-seq data from human and mouse
Lachmann, A., Torre, D., Keenan, A. B., Jagodnik, K. M., Lee, H. J., Wang, L., Silverstein, M. C., & Ma’ayan, A. (2018) · Nature Communications · doi:10.1038/s41467-018-03751-6 · Cited by 701 · see citing works
The human body at cellular resolution: the NIH Human Biomolecular Atlas Program
HuBMAP Consortium, Writing Group, Snyder, M. P., Lin, S., Posgai, A., Atkinson, M., Regev, A., Rood, J., Rozenblatt-Rosen, O., Gaffney, L., et al. (2019) · Nature · doi:10.1038/s41586-019-1629-x · Cited by 595 · see citing works
Self-citationMicroenvironment drives cell state, plasticity, and drug response in pancreatic cancer
Raghavan, S., Winter, P. S., Navia, A. W., Williams, H. L., DenAdel, A., Lowder, K. E., Galvez-Reyes, J., Kalekar, R. L., Mulugeta, N., Kapner, K. S., et al. (2021) · Cell · doi:10.1016/j.cell.2021.11.017 · Cited by 592 · see citing works
Progressive Cactus is a multiple-genome aligner for the thousand-genome era
Armstrong, J., Hickey, G., Diekhans, M., Fiddes, I. T., Novak, A. M., Deran, A., Fang, Q., Xie, D., Feng, S., Stiller, J., et al. (2020) · Nature · doi:10.1038/s41586-020-2871-y · Cited by 586 · see citing works
Self-citationMGnify: the microbiome analysis resource in 2020
Mitchell, A. L., Almeida, A., Beracochea, M., Boland, M., Burgin, J., Cochrane, G., Crusoe, M. R., Kale, V., Potter, S. C., Richardson, L. J., et al. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz1035 · Cited by 562 · see citing works
Cloud computing for genomic data analysis and collaboration
Langmead, B., & Nellore, A. (2018) · Nature Reviews Genetics · doi:10.1038/nrg.2017.113 · Cited by 310 · see citing works
“The Toil effort 63 used a standardized computational pipeline and AWS to analyse nearly 20,000 samples spanning four major studies, including TCGA and GTEx, in under 4 days.”
MethodZaitsev, A., Chelushkin, M., Dyikanov, D., Cheremushkin, I., Shpak, B., Nomie, K., Zyrin, V., Nuzhdina, E., Lozinsky, Y., Zotova, A., et al. (2022) · Cancer Cell · doi:10.1016/j.ccell.2022.07.006 · Cited by 139 · see citing works
Zeng, B., Glicksberg, B. S., Newbury, P., Chekalin, E., Xing, J., Liu, K., Wen, A., Chow, C., & Chen, B. (2021) · Nature Protocols · doi:10.1038/s41596-020-00430-z · Cited by 48 · see citing works
Pangenome Graph Construction from Genome Alignment with Minigraph-Cactus
Hickey, G., Monlong, J., Ebler, J., Novak, A., Eizenga, J. M., Gao, Y., Human Pangenome Reference Consortium, Marschall, T., Li, H., & Paten, B. (2022) · doi:10.1101/2022.10.06.511217 · Cited by 345 · see citing works
Self-citationWu, S., & Wagner, G. (2021) · Cell Systems · doi:10.1016/j.cels.2021.07.002 · Cited by 20 · see citing works
Levy, J. J., Davis, M. J., Chacko, R. S., Davis, M. J., Fu, L. J., Goel, T., Pamal, A., Nafi, I., Angirekula, A., Suvarna, A., et al. (2024) · npj Precision Oncology · doi:10.1038/s41698-023-00477-7 · Cited by 20 · see citing works
Yang, M., Li, S., Li, R., Dong, Y., Zhou, M., Zhao, J., Liu, M., & Tan, R. (2026) · New Biotechnology · doi:10.1016/j.nbt.2026.05.006
- Authors
- Vivian, J., Rao, A. A., Nothaft, F. A., Ketchum, C., Armstrong, J., Novak, A., Pfeil, J., Narkizian, J., Deran, A. D., Musselman-Brown, A., Schmidt, H., Amstutz, P., Craft, B., Goldman, M., Rosenbloom, K., Cline, M., O'Connor, B., Hanna, M., Birger, C., Kent, W. J., Patterson, D. A., Joseph, A. D., Zhu, J., Zaranek, S., Getz, G., Haussler, D., & Paten, B.
- Affiliations
- University of California, Santa Cruz (US)
- University of California, Berkeley (US)
- Curoverse (United States) (US)
- Broad Institute (US)
- Corresponding author
- Paten, Benedict
- ORCID
- Venue
- Nature Biotechnology
- Published
- 2017-04
- Volume
- 35
- Issue
- 4
- Pages
- 314-316
- ISSN
- 0733-222X (ISSN-L)
- 1087-0156
- 1546-1696
- PMC
- PMC5546205
- OpenAlex
- W2607211817
- Cited by
- 1,365
- References
- 24
- Citations by year
- 2026: 97 · 2025: 182 · 2024: 210 · 2023: 244 · 2022: 250 · 2021: 170 · 2020: 82 · 2019: 70 · 2018: 36 · 2017: 22 · 2016: 2
- FWCI
- 92.90
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- cc-by-nc
- www.springer.com/tdm (tdm)
- Locations
- Nature Biotechnology (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- Broad Institute
- National Institutes of Health
- National Human Genome Research Institute (U41HG002371, U54HG007990)
- National Cancer Institute (U24CA143858, U24CA180951, U24CA210974)
- National Institute of General Medical Sciences (R44GM119858)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Physical Sciences › Computer Science › Computer Networks and Communications › Distributed and Parallel Computing Systems
- Keywords
- toil, scientific workflow management, reproducibility, open source software, cloud computing, biomedical data analysis
- MeSH terms
- Electronic Data Processing, Humans, Software, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
