NG-meta-profiler: fast processing of metagenomes using NGLess, a domain-specific language
Open accessPublished version of a preprint
The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Shotgun metagenomes contain a sample of all the genomic material in an environment, allowing for the characterization of a microbial community. In order to understand these communities, bioinformatics methods are crucial. A common first step in processing metagenomes is to compute abundance estimates of different taxonomic or functional groups from the raw sequencing data. Given the breadth of the field, computational solutions need to be flexible and extensible, enabling the combination of different tools into a larger pipeline. RESULTS: We present NGLess and NG-meta-profiler. NGLess is a domain specific language for describing next-generation sequence processing pipelines. It was developed with the goal of enabling user-friendly computational reproducibility. It provides built-in support for many common operations on sequencing data and is extensible with external tools with configuration files. Using this framework, we developed NG-meta-profiler, a fast profiler for metagenomes which performs sequence preprocessing, mapping to bundled databases, filtering of the mapping results, and profiling (taxonomic and functional). It is significantly faster than either MOCAT2 or htseq-count and (as it builds on NGLess) its results are perfectly reproducible. CONCLUSIONS: NG-meta-profiler is a high-performance solution for metagenomics processing built on NGLess. It can be used as-is to execute standard analyses or serve as the starting point for customization in a perfectly reproducible fashion. NGLess and NG-meta-profiler are open source software (under the liberal MIT license) and can be downloaded from https://ngless.embl.de or installed through bioconda.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Minimap2: Pairwise alignment for nucleotide sequences
Li, H. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty191 · Cited by 14,807
Salmon provides fast and bias-aware quantification of transcript expression
Patro, R., Duggal, G., Love, M. I., Irizarry, R. A., & Kingsford, C. (2017) · Nature Methods · doi:10.1038/nmeth.4197 · Cited by 12,700
Nextflow enables reproducible computational workflows
Di Tommaso, P., Chatzou, M., Floden, E. W., Barja, P. P., Palumbo, E., & Notredame, C. (2017) · Nature Biotechnology · doi:10.1038/nbt.3820 · Cited by 4,001
Topographical and Temporal Diversity of the Human Skin Microbiome
Grice, E. A., Kong, H. H., Conlan, S., Deming, C. B., Davis, J., Young, A. C., Program, N. C. S., Bouffard, G. G., Blakesley, R. W., Murray, P. R., et al. (2009) · Science · doi:10.1126/science.1171700 · Cited by 2,953
Current understanding of the human microbiome
Gilbert, J. A., Blaser, M. J., Caporaso, J. G., Jansson, J. K., Lynch, S. V., & Knight, R. (2018) · Nature Medicine · doi:10.1038/nm.4517 · Cited by 2,609
MetaPhlAn2 for enhanced metagenomic taxonomic profiling
Truong, D. T., Franzosa, E. A., Tickle, T. L., Scholz, M., Weingart, G., Pasolli, E., Tett, A., Huttenhower, C., & Segata, N. (2015) · Nature Methods · doi:10.1038/nmeth.3589 · Cited by 2,185
Potential of fecal microbiota for early-stage detection of colorectal cancer
Zeller, G., Tap, J., Voigt, A. Y., Sunagawa, S., Kultima, J. R., Costea, P. I., Amiot, A., Böhm, J., Brunetti, F., Habermann, N., et al. (2014) · Molecular Systems Biology · doi:10.15252/msb.20145645 · Cited by 1,375
SolexaQA: at-a-glance quality assessment of Illumina second-generation sequencing data
Cox, M. P., Peterson, D. A., & Biggs, P. J. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-485 · Cited by 1,357
BiGG Models: A platform for integrating, standardizing and sharing genome-scale models
King, Z. A., Lu, J., Dräger, A., Miller, P., Federowicz, S., Lerman, J. A., Ebrahim, A., Palsson, B. O., & Lewis, N. E. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkv1049 · Cited by 938 · cited 2×
“In the current version, NG-meta-profiler can produce eggNOG orthologous group 23 , KEGG orthologous groups (KO) 24 , SEED 25 , and BiGG 26 abundance profiles.”
The Human Gut Microbiome: From Association to Modulation
Schmidt, T. S. B., Raes, J., & Bork, P. (2018) · Cell · doi:10.1016/j.cell.2018.02.044 · Cited by 845
Research works printed but not linked: 42
- Xiao L, et al. A catalog of the mouse gut metagenome. Nature Biotechnology. 2015;33:1103–8. · DOI · Paper page
- Xiao L, et al. A reference gene catalogue of the pig gut microbiome. Nature Microbiology. 2016;1:16161. · DOI · Paper page
- Coelho LP, et al. Similarity of the dog and human gut microbiomes in gene content and response to diet. Microbiome. 2018;6:72. · DOI · Paper page
- Sunagawa S, et al. Structure and function of the global ocean microbiome. Science. 2015;348:1261359. · DOI · Paper page
- Kultima JR, et al. MOCAT: A metagenomics assembly and gene prediction toolkit. PLoS ONE. 2012;7:e47656. · DOI · Paper page
- Kultima JR, et al. MOCAT2: A metagenomic assembly, annotation and profiling framework. Bioinformatics (Oxford, England). 2016;32:2520–3. · DOI · Paper page
- Treangen TJ, et al. MetAMOS: A modular and open source metagenomic assembly and analysis pipeline. Genome biology. 2013;14:R2. · DOI · Paper page
- Narayanasamy S, et al. IMP: A pipeline for reproducible reference-independent integrated metagenomic and metatranscriptomic analyses. Genome Biol. 2016;17:260. · DOI · Paper page
- McMurdie PJ, Holmes S. Phyloseq: an R package for reproducible interactive analysis and graphics of microbiome census data. PLoS One. 2013;8:e61217. · DOI · Paper page
- Oksanen J, et al. The vegan package. Community ecology package. 2007;10:631–7.
- Segata N, et al. Metagenomic biomarker discovery and explanation. Genome Biol. 2011;12:R60. · DOI · Paper page
- Prabhu, P. et al. A survey of the practice of computational science. in SC ’11: Proceedings of 2011 international conference for high performance computing, networking, storage and analysis 1–12 (2011). https://doi.org/10.1145/2063348.2063374 · DOI · Paper page
- Johanson AN, Hasselbring W. Effectiveness and efficiency of a domain-specific language for high-performance marine ecosystem simulation: a controlled experiment. Empirical Software Engineering. 2017;22:2206–36. · DOI
- Donoho DL, Maleki A, Rahman IU, Shahram M, Stodden V. Reproducible research in computational harmonic analysis. Computing in Science & Engineering. 2009;11:8–18. · DOI · Paper page
- Vandewalle P, Kovacevic J, Vetterli M. Reproducible research in signal processing. Signal Process Mag IEEE. 2009;26:37–47. · DOI · Paper page
- Fomel S. Reproducible research as a community effort: Lessons from the madagascar project. Computing in Science & Engineering. 2015;17:20–6. · DOI
- Carey MA, Papin JA. Ten simple rules for biologists learning to program. PLoS Comput. Biol. 2018;14:e1005871. · DOI
- Del Fabbro C, Scalabrin S, Morgante M, Giorgi FM. An extensive evaluation of read trimming effects on illumina NGS data analysis. PLoS One. 2013;8:e85024. · DOI · Paper page
- Huerta-Cepas J, et al. Fast genome-wide functional annotation through orthology assignment by eggNOG-mapper. Molecular Biology and Evolution. 2017;34:2115–22. · DOI · Paper page
- Ugarte A, Vicedomini R, Bernardes J, Carbone A. A multi-source domain annotation pipeline for quantitative metagenomic and metatranscriptomic functional profiling. Microbiome. 2018;6:149. · DOI
- Li J, et al. An integrated catalog of reference genes in the human gut microbiome. Nature Biotechnology. 2014;32:834–41. · DOI · Paper page
- Huerta-Cepas J, et al. eggNOG 4.5: A hierarchical orthology framework with improved functional annotations for eukaryotic, prokaryotic and viral sequences. Nucleic Acids Res. 2016;44:D286–93. · DOI · Paper page
- Kanehisa M, et al. Data, information, knowledge and principle: Back to metabolism in KEGG. Nucleic Acids Res. 2014;42:D199–205. · DOI · Paper page
- Overbeek R, et al. The subsystems approach to genome annotation and its use in the project to annotate 1000 genomes. Nucleic Acids Res. 2005;33:5691–702. · DOI · Paper page
- Li D, Liu C-M, Luo R, Sadakane K, Lam T-W. MEGAHIT: an ultra-fast single-node solution for large and complex metagenomics assembly via succinct de bruijn graph. Bioinformatics. 2015;31:1674–6. · DOI · Paper page
- Darling AE, et al. Critical assessment of metagenome interpretation—a benchmark of metagenomics software. Nature Methods. 2017;14:1063. · DOI · Paper page
- Awad S, Irber L, Titus Brown C. Evaluating metagenome assembly on a simple defined community with many strain variants. bioRxiv. 2017;155358. · DOI
- Hyatt D, et al. Prodigal: Prokaryotic gene recognition and translation initiation site identification. BMC Bioinformatics. 2010;11:1–11. · DOI · Paper page
- Li H. Aligning sequence reads, clone sequences and assembly contigs with BWA-MEM. arXiv preprint arXiv. 2013;1303.3997.
- Li R, et al. SOAP2: An improved ultrafast tool for short read alignment. Bioinformatics. 2009;25:1966–7. · DOI · Paper page
- Anders S, Pyl PT, Huber W. HTSeq–a python framework to work with high-throughput sequencing data. Bioinformatics. 2015;31:166–9. · DOI · Paper page
- Schwab M, Karrenbach M, Claerbout J. Making scientific computations reproducible. Comput Sci Eng. 2000;2:61–7. · DOI · Paper page
- Köster J, Rahmann S. Snakemake–a scalable bioinformatics workflow engine. Bioinformatics (Oxford, England). 2012;28:2520–2. · DOI · Paper page
- Li H, et al. The sequence alignment/map format and samtools. Bioinformatics. 2009;25:2078–9. · DOI · Paper page
- Huerta-Cepas J, et al. eggNOG 5.0: A hierarchical, functionally and phylogenetically annotated orthology resource based on 5090 organisms and 2502 viruses. Nucleic Acids Res. 2019;47:D309–14. · DOI · Paper page
- Smith AM, Katz DS, Niemeyer KE. Software citation principles. Peer J Comput Sci. 2016;2:e86. · DOI · Paper page
- Johanson A, Hasselbring W. Software engineering for computational science: Past, present, future. Computing in Science Engineering. 2018:1–1. · DOI
- Dale R, Grüning B, Sjödin A, Rowe J, Chapman BA, Tomkins-Tinch CH, Valieris R, Köster J. Bioconda: a sustainable and comprehensive software distribution for the life sciences. bioRxiv. 2017;207092. https://doi.org/10.1038/s41592-018-0046-7 . · DOI · Paper page
- da Veiga Leprevost F, et al. BioContainers: an open-source and community-driven framework for software standardization. Bioinformatics. 2017;33:2580–2. · DOI · Paper page
- Coelho LP. Jug: Software for parallel reproducible computation in python. Journal of Open Research Software. 2017;5. · DOI · Paper page
- Mende DR, Sunagawa S, Zeller G, Bork P. Accurate and universal delineation of prokaryotic species. Nature Methods. 2013;10:881–4. · DOI · Paper page
- Sunagawa S, et al. Metagenomic species profiling using universal phylogenetic marker genes. Nature Methods. 2013;10:1196–9. · DOI · Paper page
Cited by
Extensive transmission of microbes along the gastrointestinal tract
Schmidt, T. S., Hayward, M. R., Coelho, L. P., Li, S. S., Costea, P. I., Voigt, A. Y., Wirbel, J., Maistrenko, O. M., Alves, R. J., Bergsten, E., et al. (2019) · eLife · doi:10.7554/elife.42693 · Cited by 596
Self-citationDiscovery of antimicrobial peptides in the global microbiome with machine learning
Santos-Júnior, C. D., Torres, M. D. T., Duan, Y., Rodríguez del Río, Á., Schmidt, T. S. B., Chong, H., Fullam, A., Kuhn, M., Zhu, C., Houseman, A., et al. (2024) · Cell · doi:10.1016/j.cell.2024.05.013 · Cited by 419
Self-citationA faecal microbiota signature with high specificity for pancreatic cancer
Kartal, E., Schmidt, T. S. B., Molina-Montes, E., Rodríguez-Perales, S., Wirbel, J., Maistrenko, O. M., Akanni, W. A., Alashkar Alhamwe, B., Alves, R. J., Carrato, A., et al. (2022) · Gut · doi:10.1136/gutjnl-2021-324755 · Cited by 290
Self-citationWirbel, J., Zych, K., Essex, M., Karcher, N., Kartal, E., Salazar, G., Bork, P., Sunagawa, S., & Zeller, G. (2021) · Genome Biology · doi:10.1186/s13059-021-02306-1 · Cited by 269
Self-citationDrivers and determinants of strain dynamics following fecal microbiota transplantation
Schmidt, T. S. B., Li, S. S., Maistrenko, O. M., Akanni, W., Coelho, L. P., Dolai, S., Fullam, A., Glazek, A. M., Hercog, R., Herrema, H., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01913-0 · Cited by 233
Self-citation
Metadata
- Authors
- Coelho, L. P., Alves, R., Monteiro, P., Huerta-Cepas, J., Freitas, A. T., & Bork, P.
- Venue
- Microbiome
- Published
- 2019-06-03
- Volume
- 7
- Issue
- 1
- Pages
- 84-84
- ISSN
- 2049-2618 (ISSN-L)
- OpenAlex
- W2950747344
- Cited by
- 87
- References
- 12
- FWCI
- 2.15
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0/ (tdm)
- creativecommons.org/licenses/by/4.0/ (vor)
- Funders
- Horizon 2020 Framework Programme (686070)
- European Research Council (ERC-AdG-669830)
- Fundação para a Ciência e a Tecnologia (EXCL/EEI-ESS/0257/2012)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: NG-meta-profiler: fast processing of metagenomes using NGLess, a domain-specific language. https://app.cochat.ai/papers/10.1186/s40168-019-0684-8
- The paper
- Coelho, L. P., Alves, R., Monteiro, P., Huerta-Cepas, J., Freitas, A. T., & Bork, P. (2019). NG-meta-profiler: fast processing of metagenomes using NGLess, a domain-specific language. Microbiome. https://doi.org/10.1186/s40168-019-0684-8
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.