CoChat
Sign Up
Cited by 786Scientific Computing and Data ManagementResearch Data Management PracticesGenomics and Phylogenetic Studies

Sustainable data analysis with Snakemake

Open access
Mölder, Felix, Jablonski, Kim Philipp, Letcher, Brice, Hall, Michael B., Tomkins-Tinch, Christopher H., Sochat, Vanessa, Forster, Jan, Lee, Soohyun +7 more·2021·F1000Research
DOI 10.12688/f1000research.29032.1PMC PMC8114187
Open this paper in CoChat — free accountOpen PDF

Abstract

Data analysis often entails a multitude of heterogeneous steps, from the application of various command line tools to the usage of scripting languages like R or Python for the generation of plots and tables. It is widely recognized that data analyses should ideally be conducted in a reproducible way. Reproducibility enables technical validation and regeneration of results on the original or even new data. However, reproducibility alone is by no means sufficient to deliver an analysis that is of lasting impact (i.e., sustainable) for the field, or even just one research group. We postulate that it is equally important to ensure adaptability and transparency. The former describes the ability to modify the analysis to answer extended or slightly different research questions. The latter describes the ability to understand the analysis in order to judge whether it is not only technically, but methodologically valid. Here, we analyze the properties needed for a data analysis to become reproducible, adaptable, and transparent. We show how the popular workflow management system Snakemake can be used to guarantee this, and how it enables an ergonomic, combined, unified representation of all steps involved in data analysis, ranging from raw data processing, to quality control and fine-grained, interactive exploration and plotting of final results.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.12688/f1000research.29032.1

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • CPU benchmarking workflows exactly matched the workflows used by Parabricks.

    “CPU benchmarking was conducted by writing custom workflows using Snakemake v.6.6.1 [26], following best practices for each tool and exactly matching the workflows used by Parabricks (Data and Materials).”

    From Accelerating genomic workflows using NVIDIA Parabricks (2023) · cites this paper as [26]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 35

  1. [2]Mesirov JP: Computer science. Accessible reproducible research. Science. 2010; 327(5964): 415– 6. 10.1126/science.1179653 20093459 PMC3878063 · DOI
  2. [3]Munafò MR Nosek BA Bishop DVM: A manifesto for reproducible science. Nat Hum Behav. 2017; 1: 0021. 10.1038/s41562-016-0021 33954258 PMC7610724 · DOI · Paper page
  3. [4]Afgan E Baker D Batut B: The Galaxy platform for accessible, reproducible and collaborative biomedical analyses: 2018 update. Nucleic Acids Res. 2018; 46(W1): W537– W544. 10.1093/nar/gky379 29790989 PMC6030816 · DOI · Paper page
  4. [5]Berthold MR Cebron N Dill F: KNIME: the Konstanz Information Miner.In: Studies in Classification, Data Analysis, and Knowledge Organization (GfKL 2007).Springer, 2007. Reference Source
  5. [6]Kluge M Friedl MS Menzel AL: Watchdog 2.0: new developments for reusability, reproducibility, and workflow execution. GigaScience. 2020; 9(6): giaa068. 10.1093/gigascience/giaa068 32556167 PMC7298769 · DOI
  6. [7]Cervera A Rantanen V Ovaska K: Anduril 2: upgraded large–scale data integration framework. Bioinformatics. 2019; 35(19): 3815– 3817. 10.1093/bioinformatics/btz133 30793160 · DOI
  7. [8]Salim M Uram T Childers JT: Balsam: automated scheduling and execution of dynamic, data-intensive HPC workflows. In: Proceedings of the 8th Workshop on Python for High-Performance and Scientific Computing. ACM Press, 2018. Reference Source
  8. [9]Cima V Böhm S Martinovič J: HyperLoom: a platform for defining and executing scientific pipelines in distributed environments.In: Proceedings of the 9th Workshop and 7th Workshop on Parallel Programming and RunTime Management Techniques for Manycore Architectures and Design Tools and Architectures for Multicore Embedded Computing Platforms. ACM, 2018; 1– 6. 10.1145/3183767.3183768 · DOI
  9. [10]Coelho LP: Jug: software for parallel reproducible computation in Python. J Open Res Softw. 2017; 5(1): 30. 10.5334/jors.161 · DOI
  10. [11]Tanaka M Tatebe O: Pwrake: a parallel and distributed flexible workflow management tool for wide-area data intensive computing.In: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing - HPDC 2010. ACM Press, 2010; 356– 359. 10.1145/1851476.1851529 · DOI
  11. [12]Goodstadt L: Ruffus: a lightweight Python library for computational pipelines. Bioinformatics. 2010; 26(21): 2778– 9. 10.1093/bioinformatics/btq524 20847218 · DOI
  12. [13]Lampa S Dahlö M Alvarsson J: SciPipe: a workflow library for agile development of complex and dynamic bioinformatics pipelines. Gigascience. 2019; 8(5): giz044. 10.1093/gigascience/giz044 31029061 PMC6486472 · DOI
  13. [14]Hold-Geoffroy Y Gagnon O Parizeau M: Once you SCOOP, no need to fork. In: Proceedings of the 2014 Annual Conference on Extreme Science and Engineering Discovery Environment. ACM, 2014; 1– 8. 10.1145/2616498.2616565 · DOI
  14. [15]Lordan F Tejedor E Ejarque J: ServiceSs: an Interoperable Programming Framework for the Cloud. J Grid Comput. 2013; 12(1): 67– 91. 10.1007/s10723-013-9272-5 · DOI
  15. [16]Pal S Przytycka TM: Bioinformatics pipeline using JUDI: Just Do It! Bioinformatics. 2020; 36(8): 2572– 2574. 10.1093/bioinformatics/btz956 31882996 PMC7868055 · DOI
  16. [17]Di Tommaso P Chatzou M Floden EW: Nextflow enables reproducible computational workflows. Nat Biotechnol. 2017; 35(4): 316– 319. 10.1038/nbt.3820 28398311 · DOI · Paper page
  17. [18]Köster J Rahmann S: Snakemake–a scalable bioinformatics workflow engine. Bioinformatics. 2012; 28(19): 2520– 2. 10.1093/bioinformatics/bts480 22908215 · DOI · Paper page
  18. [19]Yao L Wang H Song Y: BioQueue: a novel pipeline framework to accelerate bioinformatics analysis. Bioinformatics. 2017; 33(20): 3286– 3288. 10.1093/bioinformatics/btx403 28633441 · DOI
  19. [20]Sadedin SP Pope B Oshlack A: Bpipe: a tool for running and managing bioinformatics pipelines. Bioinformatics. 2012; 28(11): 1525– 6. 10.1093/bioinformatics/bts167 22500002 · DOI
  20. [21]Ewels P Krueger F Käller M: Cluster Flow: a user-friendly bioinformatics workflow tool [version 2; peer review: 3 approved]. F1000Res. 2016; 5: 2824. 10.12688/f1000research.10335.2 28299179 PMC5310375 · DOI
  21. [22]Oliver HJ Shin M Sanders O: Cylc: a workflow engine for cycling systems. J Open Source Softw. 2018; 3(27): 737. 10.21105/joss.00737 · DOI
  22. [23]Cingolani P Sladek R Blanchette M: BigDataScript: a scripting language for data pipelines. Bioinformatics. 2015; 31(1): 10– 16. 10.1093/bioinformatics/btu595 25189778 PMC4271142 · DOI
  23. [24]Jimenez I Sevilla M Watkins N: The popper convention: making reproducible systems evaluation practical. In: 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). IEEE, 2017. 10.1109/IPDPSW.2017.157 · DOI
  24. [27]Voss K Gentry J Van der Auwera G: Full-stack genomics pipelining with GATK4 + WDL + Cromwell. F1000Res. 2017; 6. 10.7490/f1000research.1114634.1 · DOI
  25. [28]Vivian J Rao AA Nothaft FA: Toil enables reproducible, open source, big biomedical data analyses. Nat Biotechnol. 2017; 35(4): 314– 316. 10.1038/nbt.3772 28398314 PMC5546205 · DOI · Paper page
  26. [29]Lee S Johnson J Vitzthum C: Tibanna: software for scalable execution of portable pipelines on the cloud. Bioinformatics. 2019; 35(21): 4424– 4426. 10.1093/bioinformatics/btz379 31077294 PMC6931271 · DOI
  27. [30]Kurtzer GM Sochat V Bauer MW: Singularity: scientific containers for mobility of compute. PLoS One. 2017; 12(5): e0177459. 10.1371/journal.pone.0177459 28494014 PMC5426675 · DOI · Paper page
  28. [31]Huizinga D kolawa A: Automated defect prevention: best practices in software management. Google-Books-ID: PhnoE90CmdIC. John Wiley & Sons, Aug. 2007. Reference Source
  29. [32]Chall JS Dale E: Readability revisited: the new Dale-Chall readability formula.English. OCLC: 32347586. Cambridge, Mass.: Brookline Books, 1995. Reference Source
  30. [34]Grüning B Chilton J Köster J: Practical computational reproducibility in the life sciences. Cell Syst. 2018; 6(6): 631– 635. 10.1016/j.cels.2018.03.014 29953862 PMC6263957 · DOI
  31. [37]Tannenbaum T Wright D Miller K: Condor – A distributed job scheduler.In: Beowulf Cluster Computing with Linux.Ed. by Thomas Sterling. MIT Press, 2001. 10.7551/mitpress/1556.003.0019 · DOI
  32. [38]Handschuh H: SHA Family (Secure Hash Algorithm). In: Encyclopedia of Cryptography and Security.Springer US. 2005; 565– 567. 10.1007/0-387-23483-7_388 · DOI
  33. [39]Narayanan A Bonneau J Felten E: Bitcoin and Cryptocurrency technologies: a comprehensive introduction.Google-Books-ID: LchFDAAAQBAJ. Princeton University Press, 2016. Reference Source
  34. [40]McKinney W: Data structures for statistical computing in Python. In: Proceedings of the 9th Python in Science Conference. Ed. by Stéfan van der Walt and Jarrod Millman. 2010; 56– 61. 10.25080/Majora-92bf1922-00a · DOI
  35. [42]Grüning B Dale R Sjödin A: Bioconda: sustainable and comprehensive software distribution for the life sciences. Nat Methods. 2018; 15(7): 475– 476. 10.1038/s41592-018-0046-7 29967506 PMC11070151 · DOI

Cited by

Sorted by citations

Authors
Mölder, F., Jablonski, K. P., Letcher, B., Hall, M. B., Tomkins-Tinch, C. H., Sochat, V., Forster, J., Lee, S., Twardziok, S. O., Kanitz, A., Wilm, A., Holtgrewe, M., Rahmann, S., Nahnsen, S., & Köster, J.
Affiliations
  • Essen University Hospital (DE)
  • SIB Swiss Institute of Bioinformatics (CH)
  • European Bioinformatics Institute (GB)
  • Broad Institute (US)
  • Stanford University (US)
  • Harvard University (US)
  • Humboldt-Universität zu Berlin (DE)
  • Microsoft (Singapore) (SG)
  • Quantitative Biology Center (DE)
Corresponding author
Köster, Johannes
ORCID
Venue
F1000Research
Publisher
F1000 Research Ltd
Published
2021-01-18
Volume
10
Pages
33
ISSN
2046-1402 (ISSN-L)
DOI
10.12688/f1000research.29032.1
PMC
PMC8114187
OpenAlex
W4210644165
Cited by
786
References
1
Citations by year
2026: 47 · 2025: 126 · 2024: 230 · 2023: 219 · 2022: 119 · 2021: 44 · 2020: 1
FWCI
131.14
Type
journal article
Language
en
Update notices
Open access
Licence
Locations
  • F1000Research (publishedVersion, cc-by)
  • Minerva Access (University of Melbourne) (submittedVersion, cc-by)
Funders
  • Deutsche Stiftung für Herzforschung (SFB876)
  • Netherlands Organisation for Scientific Research (VENI grant 016.Veni.173.076)
  • Google LLC
  • United States National Science Foundation Graduate Research Fellowship Program (1745303)
Topics
  • Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
  • Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
Keywords
Snakemake, workflow management systems, reproducible research, data analysis pipeline, adaptability, transparency, data quality control, interactive data exploration

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review