Laboratory Validation of a Clinical Metagenomic Sequencing Assay for Pathogen Detection in Cerebrospinal Fluid
Open accessPreprint — published as 10.1101/gr.238170.118 · Paper page
The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Abstract
Metagenomic next-generation sequencing (mNGS) for pan-pathogen detection has been successfully tested in proof-of-concept case studies in patients with acute illness of unknown etiology, but to date has been largely confined to research settings. Here we developed and validated an mNGS assay for diagnosis of infectious causes of meningitis and encephalitis from cerebrospinal fluid (CSF) in a licensed clinical laboratory. A clinical bioinformatics pipeline, SURPI+, was developed to rapidly analyze mNGS data, automatically report detected pathogens, and provide a graphical user interface for evaluating and interpreting results. We established quality metrics, threshold values, and limits of detection of between 0.16 – 313 genomic copies or colony forming units per milliliter for each representative organism type. Gross hemolysis and excess host nucleic acid reduced assay sensitivity; however, a spiked phage used as an internal control was a reliable indicator of sensitivity loss. Diagnostic test accuracy was evaluated by blinded mNGS testing of 95 patient samples, revealing 73% sensitivity and 99% specificity compared to original clinical test results, with 81% positive percent agreement and 99% negative percent agreement after discrepancy analysis. Subsequent mNGS challenge testing of 20 positive CSF samples prospectively collected from a cohort of pediatric patients hospitalized with meningitis, myelitis, and/or encephalitis showed 92% sensitivity and 96% specificity relative to conventional microbiological testing of CSF in identifying the causative pathogen. These results demonstrate the analytic performance of a laboratory-validated mNGS assay for pan-pathogen detection, to be used clinically for diagnosis of neurological infections from CSF.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Nevertheless, several groups have successfully validated mNGS in Clinical Laboratory Improvement Amendments (CLIA)-certified clinical laboratories for the diagnosis of infections, including meningitis or encephalitis36,37, sepsis33,57 and pneumonia58, and these assays are now available for clinical reference testing of patients.”
From Clinical metagenomics (2019) · cites this paper as [36]
In the citing paper
“A limitation of mNGS is that the sensitivity of the approach is critically dependent on the level of background. Tissues, for example, have increased human host background relative to cell-free body fluids, resulting in a reduced number and proportion of microbial reads and hence a decrease in mNGS sensitivity33,36,37.”
From Clinical metagenomics (2019) · cites this paper as [36]
In the citing paper
“Receiver–operator curve (ROC) analysis is a useful tool to determine optimal threshold values for a training set of clinical samples with known results, with verification of pre-established thresholds using an independent validation set36.”
From Clinical metagenomics (2019) · cites this paper as [36]
In the citing paper
“Ongoing monitoring is particularly important for mNGS assays to verify acceptable performance over time and to investigate atypical findings36.”
From Clinical metagenomics (2019) · cites this paper as [36]
In the citing paper
“Performance validation and verification for bioinformatics analysis constitute a time-consuming endeavour and include analysis of control and patient data sets and comparisons, with orthogonal clinical testing to determine the accuracy of the final result36.”
From Clinical metagenomics (2019) · cites this paper as [36]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Basic local alignment search tool
Altschul, S. F., Gish, W., Miller, W., Myers, E. W., & Lipman, D. J. (1990) · Journal of Molecular Biology · doi:10.1006/jmbi.1990.9999 · Cited by 98,669
Fast gapped-read alignment with Bowtie 2
Langmead, B., & Salzberg, S. L. (2012) · Nature Methods · doi:10.1038/nmeth.1923 · Cited by 56,400
Cutadapt removes adapter sequences from high-throughput sequencing reads
Martin, M. (2011) · EMBnet.journal · doi:10.14806/ej.17.1.200 · Cited by 33,889
Quality control and preprocessing of metagenomic datasets
Schmieder, R., & Edwards, R. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btr026 · Cited by 5,072
Reagent and laboratory contamination can critically impact sequence-based microbiome analyses
Salter, S. J., Cox, M. J., Turek, E. M., Calus, S. T., Cookson, W. O., Moffatt, M. F., Turner, P., Parkhill, J., Loman, N. J., & Walker, A. W. (2014) · BMC Biology · doi:10.1186/s12915-014-0087-z · Cited by 3,463
Clinical Metagenomic Next-Generation Sequencing for Pathogen Detection
Gu, W., Miller, S., & Chiu, C. Y. (2019) · Annual Review of Pathology: Mechanisms of Disease · doi:10.1146/annurev-pathmechdis-012418-012751 · Cited by 1,458
Self-citationGranerod, J., Ambrose, H. E., Davies, N. W., Clewley, J. P., Walsh, A. L., Morgan, D., Cunningham, R., Zuckerman, M., Mutton, K. J., Solomon, T., et al. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70222-x · Cited by 1,358
Clinical Metagenomic Sequencing for Diagnosis of Meningitis and Encephalitis
Wilson, M. R., Sample, H. A., Zorn, K. C., Arevalo, S., Yu, G., Neuhaus, J., Federman, S., Stryke, D., Briggs, B., Langelier, C., et al. (2019) · New England Journal of Medicine · doi:10.1056/nejmoa1803396 · Cited by 1,067
Self-citationActionable Diagnosis of Neuroleptospirosis by Next-Generation Sequencing
Wilson, M. R., Naccache, S. N., Samayoa, E., Biagtan, M., Bashir, H., Yu, G., Salamat, S. M., Somasekar, S., Federman, S., Miller, S., et al. (2014) · New England Journal of Medicine · doi:10.1056/nejmoa1401268 · Cited by 977
Self-citationSimner, P. J., Miller, S., & Carroll, K. C. (2018) · Clinical Infectious Diseases · doi:10.1093/cid/cix881 · Cited by 821
Self-citation
Research works printed but not linked: 1
- Meiser KL. 2002. Reporting results from studies evaluating diagnostic tests. In Clinical Microbiology Newsletter, Vol 24 (ed. FaD Administration), pp. 60-63. Elsevier, Inc.
Cited by
Chiu, C. Y., & Miller, S. A. (2019) · Nature Reviews Genetics · doi:10.1038/s41576-019-0113-7 · Cited by 1,620
Self-citationClinical Metagenomic Sequencing for Diagnosis of Meningitis and Encephalitis
Wilson, M. R., Sample, H. A., Zorn, K. C., Arevalo, S., Yu, G., Neuhaus, J., Federman, S., Stryke, D., Briggs, B., Langelier, C., et al. (2019) · New England Journal of Medicine · doi:10.1056/nejmoa1803396 · Cited by 1,067
Self-citationRapid pathogen detection by metagenomic next-generation sequencing of infected body fluids
Gu, W., Deng, X., Lee, M., Sucu, Y. D., Arevalo, S., Stryke, D., Federman, S., Gopez, A., Reyes, K., Zorn, K., et al. (2021) · Nature Medicine · doi:10.1038/s41591-020-1105-z · Cited by 689
Self-citationBenchmarking Metagenomics Tools for Taxonomic Classification
Ye, S. H., Siddle, K. J., Park, D. J., & Sabeti, P. C. (2019) · Cell · doi:10.1016/j.cell.2019.07.010 · Cited by 593
mNGS in clinical microbiology laboratories: on the road to maturity
Han, D., Li, Z., Li, R., Tan, P., Zhang, R., & Li, J. (2019) · Critical Reviews in Microbiology · doi:10.1080/1040841x.2019.1681933 · Cited by 361
Metadata
- Authors
- Miller, S., Naccache, S., Samayoa, E., Messacar, K., Arevalo, S., Federman, S., Stryke, D., Pham, E., Fung, B., Bolosky, W., Ingebrigtsen, D., Lorizio, W., Paff, S., Leake, J., Pesano, R., DeBiasi, R., Dominguez, Sr., & Chiu, C.
- Affiliations
- University of California, San Francisco (US)
- Children's Hospital Colorado (US)
- Quest Diagnostics (United States) (US)
- Venue
- Genome Research
- Publisher
- openRxiv
- Published
- 2018-05-25
- Volume
- 29
- Issue
- 5
- Pages
- 831-842
- ISSN
- 1088-9051 (ISSN-L)
- PMC
- PMC6499319
- OpenAlex
- W2951983907
- Cited by
- 583
- References
- 45
- Citations by year
- 2026: 56 · 2025: 109 · 2024: 96 · 2023: 87 · 2022: 94 · 2021: 61 · 2020: 49 · 2019: 29 · 2018: 2
- Type
- posted content
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Genome Research (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by-nc)
- HighWire Press Open Archive (submittedVersion)
- Funders
- Abbott Laboratories
- National Institutes of Health (4r33ai120977-03, R21/R33, UL1TR000004, UL1TR002535, TR000004, R21/R33 AI120977)
- University of California, San Francisco (TR000004, UL1-TR000004)
- Abbott Diagnostics
- National Heart, Lung, and Blood Institute (R01HL105704, U54HL119893)
- National Institute of Allergy and Infectious Diseases (R33-AI120977, R21AI120977)
- National Center for Advancing Translational Sciences (TR000004, UCSF-CTSI grant UL1 TR000004)
- Topics
- Health Sciences › Medicine › Microbiology (medical) › Bacterial Identification and Susceptibility Testing
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Life Sciences › Immunology and Microbiology › Microbiology › Bacterial Infections and Vaccines
- Keywords
- metagenomic next-generation sequencing, cerebrospinal fluid, meningitis, encephalitis, myelitis, diagnostic accuracy, detection limit, hemolysis, clinical microbiology
- MeSH terms
- Child, Encephalitis, Humans, Meningitis, Aseptic, Myelitis, Sensitivity and Specificity, Viruses, Computational Biology, Metagenomics, High-Throughput Nucleotide Sequencing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.