CoChat

Laboratory Validation of a Clinical Metagenomic Sequencing Assay for Pathogen Detection in Cerebrospinal Fluid

Open access
Miller, S, Naccache, SN, Samayoa, E, Messacar, K, Arevalo, S, Federman, S, Stryke, D, Pham, E, Fung, B, Bolosky, WJ, Ingebrigtsen, D, Lorizio, W, Paff, SM, Leake, JA, Pesano, R, DeBiasi, RL, Dominguez, SR, Chiu, CY · 2018 · Genome Research

Preprint — published as 10.1101/gr.238170.118 · Paper page

The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.

Abstract

Metagenomic next-generation sequencing (mNGS) for pan-pathogen detection has been successfully tested in proof-of-concept case studies in patients with acute illness of unknown etiology, but to date has been largely confined to research settings. Here we developed and validated an mNGS assay for diagnosis of infectious causes of meningitis and encephalitis from cerebrospinal fluid (CSF) in a licensed clinical laboratory. A clinical bioinformatics pipeline, SURPI+, was developed to rapidly analyze mNGS data, automatically report detected pathogens, and provide a graphical user interface for evaluating and interpreting results. We established quality metrics, threshold values, and limits of detection of between 0.16 – 313 genomic copies or colony forming units per milliliter for each representative organism type. Gross hemolysis and excess host nucleic acid reduced assay sensitivity; however, a spiked phage used as an internal control was a reliable indicator of sensitivity loss. Diagnostic test accuracy was evaluated by blinded mNGS testing of 95 patient samples, revealing 73% sensitivity and 99% specificity compared to original clinical test results, with 81% positive percent agreement and 99% negative percent agreement after discrepancy analysis. Subsequent mNGS challenge testing of 20 positive CSF samples prospectively collected from a cohort of pediatric patients hospitalized with meningitis, myelitis, and/or encephalitis showed 92% sensitivity and 96% specificity relative to conventional microbiological testing of CSF in identifying the causative pathogen. These results demonstrate the analytic performance of a laboratory-validated mNGS assay for pan-pathogen detection, to be used clinically for diagnosis of neurological infections from CSF.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1101/330381

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1. Schematic of the mNGS assay workflow. (A) CSF is extracted after lysis by bead-beating and internal control addition to allow viral, bacterial, fungal, and parasite nucleic acid retrieval. Total nucleic acid extracts are enriched for pathogen DNA by removal of methylated DNA (DNA libraries) and treatment with DNase (RNA libraries). (B) Libraries are generated using the Nextera XT protocol and amplified using two rounds of PCR. Libraries are quantified, pooled, and loaded onto the sequencer. (C) Sequences are processed using SURPI+ software for alignment and classification. Reads are preprocessed by trimming of adapters and removal of low-quality/low-complexity sequences, followed by computational subtraction of human reads and taxonomic classification of remaining microbial reads to family, genus, or species. For viruses, reads are mapped to the closest matched genome to identify nonoverlapping regions; for bacteria, fungi, and parasites, a read per million (RPM) ratio (RPM-r) metric is calculated, defined as RPM-r = RPMsample/NTC. To aid in analysis, automated result summaries, heat maps of raw/normalized read counts, and coverage/pairwise identity plots are generated for use in review and clinical interpretation.
Figure 1. Schematic of the mNGS assay workflow. (A) CSF is extracted after lysis by bead-beating and internal control addition to allow viral, bacterial, fungal, and parasite nucleic acid retrieval. Total nucleic acid extracts are enriched for pathogen DNA by removal of methylated DNA (DNA libraries) and treatment with DNase (RNA libraries). (B) Libraries are generated using the Nextera XT protocol and amplified using two rounds of PCR. Libraries are quantified, pooled, and loaded onto the sequencer. (C) Sequences are processed using SURPI+ software for alignment and classification. Reads are preprocessed by trimming of adapters and removal of low-quality/low-complexity sequences, followed by computational subtraction of human reads and taxonomic classification of remaining microbial reads to family, genus, or species. For viruses, reads are mapped to the closest matched genome to identify nonoverlapping regions; for bacteria, fungi, and parasites, a read per million (RPM) ratio (RPM-r) metric is calculated, defined as RPM-r = RPMsample/NTC. To aid in analysis, automated result summaries, heat maps of raw/normalized read counts, and coverage/pairwise identity plots are generated for use in review and clinical interpretation.CC BY-NC · PubMed Central
Figure 2. Identification and reporting of enterovirus infection in a patient with meningoencephalitis using a clinical CSF mNGS assay. SURPI+ provides tools to aid clinical interpretation and graphical visualization (the SURPIviz package), including (A) an automated mNGS results summary, (B) a heat map of aligned reads corresponding to detected pathogens, and (C) a coverage plot (green line) of reads corresponding to a detected pathogen that are mapped to the most closely matched genome or gene in the reference database, along with a corresponding pairwise identity plot (purple line, sliding window = 10 nt). Viral hits corresponding to one CSF patient sample (MNC_087_097, column highlighted in red in B) are taxonomically identified as enterovirus (enterovirus B and echovirus AMS721 species) and murine leukemia virus, a known reagent contaminant (Zheng et al. 2011). After an interpretive review, a laboratory physician prepares a clinical results report (D) that is submitted to the patient electronic medical record (EMR).
Figure 2. Identification and reporting of enterovirus infection in a patient with meningoencephalitis using a clinical CSF mNGS assay. SURPI+ provides tools to aid clinical interpretation and graphical visualization (the SURPIviz package), including (A) an automated mNGS results summary, (B) a heat map of aligned reads corresponding to detected pathogens, and (C) a coverage plot (green line) of reads corresponding to a detected pathogen that are mapped to the most closely matched genome or gene in the reference database, along with a corresponding pairwise identity plot (purple line, sliding window = 10 nt). Viral hits corresponding to one CSF patient sample (MNC_087_097, column highlighted in red in B) are taxonomically identified as enterovirus (enterovirus B and echovirus AMS721 species) and murine leukemia virus, a known reagent contaminant (Zheng et al. 2011). After an interpretive review, a laboratory physician prepares a clinical results report (D) that is submitted to the patient electronic medical record (EMR).CC BY-NC · PubMed Central
Figure 3. Accuracy of mNGS relative to clinical testing of CSF. (A) Flow chart of results from samples evaluated in the accuracy study. Results are separated by organism category (RNA virus, DNA virus, bacterium, fungus, and parasite). Shown are the number of samples positive or negative by clinical testing (first row), agreement between mNGS results and positive clinical results and additional positive detections by mNGS (second row), and, in samples with sufficient remaining volume (third row), the results of orthogonal confirmatory testing (fourth row). (B) 2 × 2 contingency tables comparing the performance of mNGS relative to clinical testing of CSF. The composite reference standards used are original clinical testing (left), combined original clinical and discrepancy testing (middle), and combined original clinical and discrepancy testing after excluding high host background samples (right). (PPA) Positive predictive agreement, (NPA) negative predictive agreement. (C) 2 × 2 contingency table showing the results of challenge study. Twenty CSF samples were analyzed by mNGS and compared with the results of conventional clinical testing.
Figure 3. Accuracy of mNGS relative to clinical testing of CSF. (A) Flow chart of results from samples evaluated in the accuracy study. Results are separated by organism category (RNA virus, DNA virus, bacterium, fungus, and parasite). Shown are the number of samples positive or negative by clinical testing (first row), agreement between mNGS results and positive clinical results and additional positive detections by mNGS (second row), and, in samples with sufficient remaining volume (third row), the results of orthogonal confirmatory testing (fourth row). (B) 2 × 2 contingency tables comparing the performance of mNGS relative to clinical testing of CSF. The composite reference standards used are original clinical testing (left), combined original clinical and discrepancy testing (middle), and combined original clinical and discrepancy testing after excluding high host background samples (right). (PPA) Positive predictive agreement, (NPA) negative predictive agreement. (C) 2 × 2 contingency table showing the results of challenge study. Twenty CSF samples were analyzed by mNGS and compared with the results of conventional clinical testing.CC BY-NC · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 45 references

Research works printed but not linked: 1

  1. Meiser KL. 2002. Reporting results from studies evaluating diagnostic tests. In Clinical Microbiology Newsletter, Vol 24 (ed. FaD Administration), pp. 60-63. Elsevier, Inc.

Cited by

Items 101–200 of 662

Metadata

Authors
Miller, S., Naccache, S., Samayoa, E., Messacar, K., Arevalo, S., Federman, S., Stryke, D., Pham, E., Fung, B., Bolosky, W., Ingebrigtsen, D., Lorizio, W., Paff, S., Leake, J., Pesano, R., DeBiasi, R., Dominguez, Sr., & Chiu, C.
Affiliations
  • University of California, San Francisco (US)
  • Children's Hospital Colorado (US)
  • Quest Diagnostics (United States) (US)
Venue
Genome Research
Publisher
openRxiv
Published
2018-05-25
Volume
29
Issue
5
Pages
831-842
ISSN
1088-9051 (ISSN-L)
OpenAlex
W2951983907
Cited by
583
References
45
Citations by year
2026: 56 · 2025: 109 · 2024: 96 · 2023: 87 · 2022: 94 · 2021: 61 · 2020: 49 · 2019: 29 · 2018: 2
Type
posted content
Language
en
Open access
Locations
  • Genome Research (publishedVersion)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • eScholarship (California Digital Library) (submittedVersion, cc-by-nc)
  • HighWire Press Open Archive (submittedVersion)
Funders
  • Abbott Laboratories
  • National Institutes of Health (4r33ai120977-03, R21/R33, UL1TR000004, UL1TR002535, TR000004, R21/R33 AI120977)
  • University of California, San Francisco (TR000004, UL1-TR000004)
  • Abbott Diagnostics
  • National Heart, Lung, and Blood Institute (R01HL105704, U54HL119893)
  • National Institute of Allergy and Infectious Diseases (R33-AI120977, R21AI120977)
  • National Center for Advancing Translational Sciences (TR000004, UCSF-CTSI grant UL1 TR000004)
Topics
  • Health Sciences › Medicine › Microbiology (medical) › Bacterial Identification and Susceptibility Testing
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
  • Life Sciences › Immunology and Microbiology › Microbiology › Bacterial Infections and Vaccines
Keywords
metagenomic next-generation sequencing, cerebrospinal fluid, meningitis, encephalitis, myelitis, diagnostic accuracy, detection limit, hemolysis, clinical microbiology
MeSH terms
Child, Encephalitis, Humans, Meningitis, Aseptic, Myelitis, Sensitivity and Specificity, Viruses, Computational Biology, Metagenomics, High-Throughput Nucleotide Sequencing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.