CoChat

NG-meta-profiler: fast processing of metagenomes using NGLess, a domain-specific language

Open access
Coelho, Luis Pedro, Alves, Renato, Monteiro, Paulo, Huerta-Cepas, Jaime, Freitas, Ana Teresa, Bork, Peer · 2019 · Microbiome

Published version of a preprint

Cited by 87

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Abstract

BACKGROUND: Shotgun metagenomes contain a sample of all the genomic material in an environment, allowing for the characterization of a microbial community. In order to understand these communities, bioinformatics methods are crucial. A common first step in processing metagenomes is to compute abundance estimates of different taxonomic or functional groups from the raw sequencing data. Given the breadth of the field, computational solutions need to be flexible and extensible, enabling the combination of different tools into a larger pipeline. RESULTS: We present NGLess and NG-meta-profiler. NGLess is a domain specific language for describing next-generation sequence processing pipelines. It was developed with the goal of enabling user-friendly computational reproducibility. It provides built-in support for many common operations on sequencing data and is extensible with external tools with configuration files. Using this framework, we developed NG-meta-profiler, a fast profiler for metagenomes which performs sequence preprocessing, mapping to bundled databases, filtering of the mapping results, and profiling (taxonomic and functional). It is significantly faster than either MOCAT2 or htseq-count and (as it builds on NGLess) its results are perfectly reproducible. CONCLUSIONS: NG-meta-profiler is a high-performance solution for metagenomics processing built on NGLess. It can be used as-is to execute standard analyses or serve as the starting point for customization in a perfectly reproducible fashion. NGLess and NG-meta-profiler are open source software (under the liberal MIT license) and can be downloaded from https://ngless.embl.de or installed through bioconda.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s40168-019-0684-8

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 12 references

Research works printed but not linked: 42

  1. Xiao L, et al. A catalog of the mouse gut metagenome. Nature Biotechnology. 2015;33:1103–8. · DOI · Paper page
  2. Xiao L, et al. A reference gene catalogue of the pig gut microbiome. Nature Microbiology. 2016;1:16161. · DOI · Paper page
  3. Coelho LP, et al. Similarity of the dog and human gut microbiomes in gene content and response to diet. Microbiome. 2018;6:72. · DOI · Paper page
  4. Sunagawa S, et al. Structure and function of the global ocean microbiome. Science. 2015;348:1261359. · DOI · Paper page
  5. Kultima JR, et al. MOCAT: A metagenomics assembly and gene prediction toolkit. PLoS ONE. 2012;7:e47656. · DOI · Paper page
  6. Kultima JR, et al. MOCAT2: A metagenomic assembly, annotation and profiling framework. Bioinformatics (Oxford, England). 2016;32:2520–3. · DOI · Paper page
  7. Treangen TJ, et al. MetAMOS: A modular and open source metagenomic assembly and analysis pipeline. Genome biology. 2013;14:R2. · DOI · Paper page
  8. Narayanasamy S, et al. IMP: A pipeline for reproducible reference-independent integrated metagenomic and metatranscriptomic analyses. Genome Biol. 2016;17:260. · DOI · Paper page
  9. McMurdie PJ, Holmes S. Phyloseq: an R package for reproducible interactive analysis and graphics of microbiome census data. PLoS One. 2013;8:e61217. · DOI · Paper page
  10. Oksanen J, et al. The vegan package. Community ecology package. 2007;10:631–7.
  11. Segata N, et al. Metagenomic biomarker discovery and explanation. Genome Biol. 2011;12:R60. · DOI · Paper page
  12. Prabhu, P. et al. A survey of the practice of computational science. in SC ’11: Proceedings of 2011 international conference for high performance computing, networking, storage and analysis 1–12 (2011). https://doi.org/10.1145/2063348.2063374 · DOI · Paper page
  13. Johanson AN, Hasselbring W. Effectiveness and efficiency of a domain-specific language for high-performance marine ecosystem simulation: a controlled experiment. Empirical Software Engineering. 2017;22:2206–36. · DOI
  14. Donoho DL, Maleki A, Rahman IU, Shahram M, Stodden V. Reproducible research in computational harmonic analysis. Computing in Science & Engineering. 2009;11:8–18. · DOI · Paper page
  15. Vandewalle P, Kovacevic J, Vetterli M. Reproducible research in signal processing. Signal Process Mag IEEE. 2009;26:37–47. · DOI · Paper page
  16. Fomel S. Reproducible research as a community effort: Lessons from the madagascar project. Computing in Science & Engineering. 2015;17:20–6. · DOI
  17. Carey MA, Papin JA. Ten simple rules for biologists learning to program. PLoS Comput. Biol. 2018;14:e1005871. · DOI
  18. Del Fabbro C, Scalabrin S, Morgante M, Giorgi FM. An extensive evaluation of read trimming effects on illumina NGS data analysis. PLoS One. 2013;8:e85024. · DOI · Paper page
  19. Huerta-Cepas J, et al. Fast genome-wide functional annotation through orthology assignment by eggNOG-mapper. Molecular Biology and Evolution. 2017;34:2115–22. · DOI · Paper page
  20. Ugarte A, Vicedomini R, Bernardes J, Carbone A. A multi-source domain annotation pipeline for quantitative metagenomic and metatranscriptomic functional profiling. Microbiome. 2018;6:149. · DOI
  21. Li J, et al. An integrated catalog of reference genes in the human gut microbiome. Nature Biotechnology. 2014;32:834–41. · DOI · Paper page
  22. Huerta-Cepas J, et al. eggNOG 4.5: A hierarchical orthology framework with improved functional annotations for eukaryotic, prokaryotic and viral sequences. Nucleic Acids Res. 2016;44:D286–93. · DOI · Paper page
  23. Kanehisa M, et al. Data, information, knowledge and principle: Back to metabolism in KEGG. Nucleic Acids Res. 2014;42:D199–205. · DOI · Paper page
  24. Overbeek R, et al. The subsystems approach to genome annotation and its use in the project to annotate 1000 genomes. Nucleic Acids Res. 2005;33:5691–702. · DOI · Paper page
  25. Li D, Liu C-M, Luo R, Sadakane K, Lam T-W. MEGAHIT: an ultra-fast single-node solution for large and complex metagenomics assembly via succinct de bruijn graph. Bioinformatics. 2015;31:1674–6. · DOI · Paper page
  26. Darling AE, et al. Critical assessment of metagenome interpretation—a benchmark of metagenomics software. Nature Methods. 2017;14:1063. · DOI · Paper page
  27. Awad S, Irber L, Titus Brown C. Evaluating metagenome assembly on a simple defined community with many strain variants. bioRxiv. 2017;155358. · DOI
  28. Hyatt D, et al. Prodigal: Prokaryotic gene recognition and translation initiation site identification. BMC Bioinformatics. 2010;11:1–11. · DOI · Paper page
  29. Li H. Aligning sequence reads, clone sequences and assembly contigs with BWA-MEM. arXiv preprint arXiv. 2013;1303.3997.
  30. Li R, et al. SOAP2: An improved ultrafast tool for short read alignment. Bioinformatics. 2009;25:1966–7. · DOI · Paper page
  31. Anders S, Pyl PT, Huber W. HTSeq–a python framework to work with high-throughput sequencing data. Bioinformatics. 2015;31:166–9. · DOI · Paper page
  32. Schwab M, Karrenbach M, Claerbout J. Making scientific computations reproducible. Comput Sci Eng. 2000;2:61–7. · DOI · Paper page
  33. Köster J, Rahmann S. Snakemake–a scalable bioinformatics workflow engine. Bioinformatics (Oxford, England). 2012;28:2520–2. · DOI · Paper page
  34. Li H, et al. The sequence alignment/map format and samtools. Bioinformatics. 2009;25:2078–9. · DOI · Paper page
  35. Huerta-Cepas J, et al. eggNOG 5.0: A hierarchical, functionally and phylogenetically annotated orthology resource based on 5090 organisms and 2502 viruses. Nucleic Acids Res. 2019;47:D309–14. · DOI · Paper page
  36. Smith AM, Katz DS, Niemeyer KE. Software citation principles. Peer J Comput Sci. 2016;2:e86. · DOI · Paper page
  37. Johanson A, Hasselbring W. Software engineering for computational science: Past, present, future. Computing in Science Engineering. 2018:1–1. · DOI
  38. Dale R, Grüning B, Sjödin A, Rowe J, Chapman BA, Tomkins-Tinch CH, Valieris R, Köster J. Bioconda: a sustainable and comprehensive software distribution for the life sciences. bioRxiv. 2017;207092. https://doi.org/10.1038/s41592-018-0046-7 . · DOI · Paper page
  39. da Veiga Leprevost F, et al. BioContainers: an open-source and community-driven framework for software standardization. Bioinformatics. 2017;33:2580–2. · DOI · Paper page
  40. Coelho LP. Jug: Software for parallel reproducible computation in python. Journal of Open Research Software. 2017;5. · DOI · Paper page
  41. Mende DR, Sunagawa S, Zeller G, Bork P. Accurate and universal delineation of prokaryotic species. Nature Methods. 2013;10:881–4. · DOI · Paper page
  42. Sunagawa S, et al. Metagenomic species profiling using universal phylogenetic marker genes. Nature Methods. 2013;10:1196–9. · DOI · Paper page

Cited by

See all 87

Metadata

Authors
Coelho, L. P., Alves, R., Monteiro, P., Huerta-Cepas, J., Freitas, A. T., & Bork, P.
Venue
Microbiome
Published
2019-06-03
Volume
7
Issue
1
Pages
84-84
ISSN
2049-2618 (ISSN-L)
OpenAlex
W2950747344
Cited by
87
References
12
FWCI
2.15
Type
journal article
Language
en
Open access
gold
Licence
Funders
  • Horizon 2020 Framework Programme (686070)
  • European Research Council (ERC-AdG-669830)
  • Fundação para a Ciência e a Tecnologia (EXCL/EEI-ESS/0257/2012)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: NG-meta-profiler: fast processing of metagenomes using NGLess, a domain-specific language. https://app.cochat.ai/papers/10.1186/s40168-019-0684-8
The paper
Coelho, L. P., Alves, R., Monteiro, P., Huerta-Cepas, J., Freitas, A. T., & Bork, P. (2019). NG-meta-profiler: fast processing of metagenomes using NGLess, a domain-specific language. Microbiome. https://doi.org/10.1186/s40168-019-0684-8

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.