CoChat
Sign Up
Cited by 114Genomics and Phylogenetic StudiesEnvironmental DNA in Biodiversity StudiesMicrobial Community Ecology and Physiology

The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.

Testing the advantages and disadvantages of short- and long- read eukaryotic metagenomics using simulated reads

Open access
Pearman, William S., Freed, Nikki E., Silander, Olin K.·2020·BMC Bioinformatics
DOI 10.1186/s12859-020-3528-4PMC PMC7257156

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Background: The first step in understanding ecological community diversity and dynamics is quantifying community membership. An increasingly common method for doing so is through metagenomics. Because of the rapidly increasing popularity of this approach, a large number of computational tools and pipelines are available for analysing metagenomic data. However, the majority of these tools have been designed and benchmarked using highly accurate short read data (i.e. Illumina), with few studies benchmarking classification accuracy for long error-prone reads (PacBio or Oxford Nanopore). In addition, few tools have been benchmarked for non-microbial communities. Results: Here we compare simulated long reads from Oxford Nanopore and Pacific Biosciences (PacBio) with high accuracy Illumina read sets to systematically investigate the effects of sequence length and taxon type on classification accuracy for metagenomic data from both microbial and non-microbial communities. We show that very generally, classification accuracy is far lower for non-microbial communities, even at low taxonomic resolution (e.g. family rather than genus). We then show that for two popular taxonomic classifiers, long reads can significantly increase classification accuracy, and this is most pronounced for non-microbial communities. Conclusions: This work provides insight on the expected accuracy for metagenomic analyses for different taxonomic groups, and establishes the point at which read length becomes more important than error rate for assigning the correct taxon.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12859-020-3528-4

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Recall is consistently higher in bacteria and fungi than plants or animals for both short Illumina and Nanopore reads. Each panel shows recall for the different kingdoms. Median recall at the genus level across each panel is shown by blue (Nanopore) or red (PacBio), and black (Illumina). Dashed lines indicate recall rates for reads classified using Kraken2; solid for reads classified using BLAST. Illumina reads exhibit consistently higher recall, followed by PacBio; bacteria and fungi exhibit higher recall than plants or animals
Fig. 1 Recall is consistently higher in bacteria and fungi than plants or animals for both short Illumina and Nanopore reads. Each panel shows recall for the different kingdoms. Median recall at the genus level across each panel is shown by blue (Nanopore) or red (PacBio), and black (Illumina). Dashed lines indicate recall rates for reads classified using Kraken2; solid for reads classified using BLAST. Illumina reads exhibit consistently higher recall, followed by PacBio; bacteria and fungi exhibit higher recall than plants or animalsCC BY · PubMed Central
Fig. 2 Classification success for short reads is weakly related to read length and strongly dependent on classification method. Each panel shows recall for the different kingdoms. Median classification success at the genus level across each panel is shown by blue (Nanopore) or red (PacBio), and black (Illumina). Dashed lines indicate recall rates for reads classified using Kraken2; solid for reads classified using BLAST. For bacteria and fungi, median classification rates of of all pairs except for Nanopore-Kraken2 are almost exactly 100% for all read lengths in Bacteria and Fungi, resulting in obscured lines at these higher levels
Fig. 2 Classification success for short reads is weakly related to read length and strongly dependent on classification method. Each panel shows recall for the different kingdoms. Median classification success at the genus level across each panel is shown by blue (Nanopore) or red (PacBio), and black (Illumina). Dashed lines indicate recall rates for reads classified using Kraken2; solid for reads classified using BLAST. For bacteria and fungi, median classification rates of of all pairs except for Nanopore-Kraken2 are almost exactly 100% for all read lengths in Bacteria and Fungi, resulting in obscured lines at these higher levelsCC BY · PubMed Central
Fig. 3 Long PacBio and Nanopore reads equal or surpass the recall of the longest Illumina reads for both BLAST and Kraken2. Each panel shows recall for the different kingdoms. Median recall at the genus level is indicated by blue lines for Nanopore or red lines for PacBio, either dashed (Kraken2) or solid (BLAST). The recall rates for 300 bp Illumina reads are shown as thin solid black lines, again either dashed (Kraken2) or solid (BLAST). Black points show the recall for the short Illumina reads at lengths of 100 bp, 150 bp, and 300 bp. Insets show the upper end of the y-axis in greater detail for both bacteria and fungi
Fig. 3 Long PacBio and Nanopore reads equal or surpass the recall of the longest Illumina reads for both BLAST and Kraken2. Each panel shows recall for the different kingdoms. Median recall at the genus level is indicated by blue lines for Nanopore or red lines for PacBio, either dashed (Kraken2) or solid (BLAST). The recall rates for 300 bp Illumina reads are shown as thin solid black lines, again either dashed (Kraken2) or solid (BLAST). Black points show the recall for the short Illumina reads at lengths of 100 bp, 150 bp, and 300 bp. Insets show the upper end of the y-axis in greater detail for both bacteria and fungiCC BY · PubMed Central
Fig. 4 Classification success for long reads is dependent on read length only for Kraken2 classification. Each panel shows classification success for the different kingdoms. Median classification success at the genus level is indicated by blue lines for Nanopore or red lines for PacBio, either dashed (Kraken2) or solid (BLAST). The recall rates for 300 bp Illumina reads are shown as thin solid black lines, again either dashed (Kraken2) or solid (BLAST). Black points show the classification success for all Illumina reads of all lengths (100 bp, 150 bp, and 300 bp). For animals and plants, the classification success of Kraken2 depends strongly on read length, and only surpasses BLAST for PacBio at approximately 3500 bp for both animals and plants. Insets show the upper end of the y-axis in greater detail for both bacteria and fungi
Fig. 4 Classification success for long reads is dependent on read length only for Kraken2 classification. Each panel shows classification success for the different kingdoms. Median classification success at the genus level is indicated by blue lines for Nanopore or red lines for PacBio, either dashed (Kraken2) or solid (BLAST). The recall rates for 300 bp Illumina reads are shown as thin solid black lines, again either dashed (Kraken2) or solid (BLAST). Black points show the classification success for all Illumina reads of all lengths (100 bp, 150 bp, and 300 bp). For animals and plants, the classification success of Kraken2 depends strongly on read length, and only surpasses BLAST for PacBio at approximately 3500 bp for both animals and plants. Insets show the upper end of the y-axis in greater detail for both bacteria and fungiCC BY · PubMed Central
Fig. 5 Cladogram of species included in the simulated mock community
Fig. 5 Cladogram of species included in the simulated mock communityCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 50 references

Research works printed but not linked: 1

  1. Pearman W, Smith ANH, Breckell G, Dale J, Freed NE, Silander OK. New tools for diet analyses: nanopore sequencing of metagenomic DNA from stomach contents to quantify diet in an invasive population of rats. bioRxiv. 2018:363622 [Cited 2018 Aug 8]. Available from: https://www.biorxiv.org/content/early/2018/07/06/363622.

Cited by

Sorted by citations

Related

See all

Metadata

Authors
Pearman, W. S., Freed, N. E., & Silander, O. K.
Affiliations
Massey University (NZ)
Corresponding author
  • Pearman, William S.
  • Silander, Olin K.
ORCID
Venue
BMC Bioinformatics
Publisher
Springer Science and Business Media LLC
Published
2020-05-29
Volume
21
Issue
1
Pages
220-220
ISSN
1471-2105 (ISSN-L)
DOI
10.1186/s12859-020-3528-4
PMC
PMC7257156
OpenAlex
W3031621849
Cited by
114
References
50
Citations by year
2026: 7 · 2025: 20 · 2024: 18 · 2023: 20 · 2022: 29 · 2021: 13 · 2020: 5
FWCI
5.43
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Bioinformatics (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • ResearchSpace (University of Auckland) (submittedVersion, cc-by)
Funders
Massey University (RM21565)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
  • Physical Sciences › Environmental Science › Ecology › Environmental DNA in Biodiversity Studies
  • Physical Sciences › Environmental Science › Ecology › Microbial Community Ecology and Physiology
Keywords
metagenomics, long-read sequencing, short-read sequencing, taxonomic classification, classification accuracy, simulated reads, Oxford Nanopore, PacBio, eukaryotic communities, read length, error rate, taxonomic resolution, benchmarking
MeSH terms
Nanopore Sequencing, Computer Simulation, Sequence Analysis, DNA, Metagenomics, Eukaryota, High-Throughput Nucleotide Sequencing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review