Laboratory Validation of a Clinical Metagenomic Sequencing Assay for Pathogen Detection in Cerebrospinal Fluid
Open accessPreprint — published as 10.1101/gr.238170.118 · Paper page
The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Abstract
Metagenomic next-generation sequencing (mNGS) for pan-pathogen detection has been successfully tested in proof-of-concept case studies in patients with acute illness of unknown etiology, but to date has been largely confined to research settings. Here we developed and validated an mNGS assay for diagnosis of infectious causes of meningitis and encephalitis from cerebrospinal fluid (CSF) in a licensed clinical laboratory. A clinical bioinformatics pipeline, SURPI+, was developed to rapidly analyze mNGS data, automatically report detected pathogens, and provide a graphical user interface for evaluating and interpreting results. We established quality metrics, threshold values, and limits of detection of between 0.16 – 313 genomic copies or colony forming units per milliliter for each representative organism type. Gross hemolysis and excess host nucleic acid reduced assay sensitivity; however, a spiked phage used as an internal control was a reliable indicator of sensitivity loss. Diagnostic test accuracy was evaluated by blinded mNGS testing of 95 patient samples, revealing 73% sensitivity and 99% specificity compared to original clinical test results, with 81% positive percent agreement and 99% negative percent agreement after discrepancy analysis. Subsequent mNGS challenge testing of 20 positive CSF samples prospectively collected from a cohort of pediatric patients hospitalized with meningitis, myelitis, and/or encephalitis showed 92% sensitivity and 96% specificity relative to conventional microbiological testing of CSF in identifying the causative pathogen. These results demonstrate the analytic performance of a laboratory-validated mNGS assay for pan-pathogen detection, to be used clinically for diagnosis of neurological infections from CSF.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Nevertheless, several groups have successfully validated mNGS in Clinical Laboratory Improvement Amendments (CLIA)-certified clinical laboratories for the diagnosis of infections, including meningitis or encephalitis36,37, sepsis33,57 and pneumonia58, and these assays are now available for clinical reference testing of patients.”
From Clinical metagenomics (2019) · cites this paper as [36]
In the citing paper
“A limitation of mNGS is that the sensitivity of the approach is critically dependent on the level of background. Tissues, for example, have increased human host background relative to cell-free body fluids, resulting in a reduced number and proportion of microbial reads and hence a decrease in mNGS sensitivity33,36,37.”
From Clinical metagenomics (2019) · cites this paper as [36]
In the citing paper
“Receiver–operator curve (ROC) analysis is a useful tool to determine optimal threshold values for a training set of clinical samples with known results, with verification of pre-established thresholds using an independent validation set36.”
From Clinical metagenomics (2019) · cites this paper as [36]
In the citing paper
“Ongoing monitoring is particularly important for mNGS assays to verify acceptable performance over time and to investigate atypical findings36.”
From Clinical metagenomics (2019) · cites this paper as [36]
In the citing paper
“Performance validation and verification for bioinformatics analysis constitute a time-consuming endeavour and include analysis of control and patient data sets and comparisons, with orthogonal clinical testing to determine the accuracy of the final result36.”
From Clinical metagenomics (2019) · cites this paper as [36]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Basic local alignment search tool
Altschul, S. F., Gish, W., Miller, W., Myers, E. W., & Lipman, D. J. (1990) · Journal of Molecular Biology · doi:10.1006/jmbi.1990.9999 · Cited by 98,669
Fast gapped-read alignment with Bowtie 2
Langmead, B., & Salzberg, S. L. (2012) · Nature Methods · doi:10.1038/nmeth.1923 · Cited by 56,400
Cutadapt removes adapter sequences from high-throughput sequencing reads
Martin, M. (2011) · EMBnet.journal · doi:10.14806/ej.17.1.200 · Cited by 33,889
Quality control and preprocessing of metagenomic datasets
Schmieder, R., & Edwards, R. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btr026 · Cited by 5,072
Reagent and laboratory contamination can critically impact sequence-based microbiome analyses
Salter, S. J., Cox, M. J., Turek, E. M., Calus, S. T., Cookson, W. O., Moffatt, M. F., Turner, P., Parkhill, J., Loman, N. J., & Walker, A. W. (2014) · BMC Biology · doi:10.1186/s12915-014-0087-z · Cited by 3,463
Clinical Metagenomic Next-Generation Sequencing for Pathogen Detection
Gu, W., Miller, S., & Chiu, C. Y. (2019) · Annual Review of Pathology: Mechanisms of Disease · doi:10.1146/annurev-pathmechdis-012418-012751 · Cited by 1,458
Self-citationGranerod, J., Ambrose, H. E., Davies, N. W., Clewley, J. P., Walsh, A. L., Morgan, D., Cunningham, R., Zuckerman, M., Mutton, K. J., Solomon, T., et al. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70222-x · Cited by 1,358
Clinical Metagenomic Sequencing for Diagnosis of Meningitis and Encephalitis
Wilson, M. R., Sample, H. A., Zorn, K. C., Arevalo, S., Yu, G., Neuhaus, J., Federman, S., Stryke, D., Briggs, B., Langelier, C., et al. (2019) · New England Journal of Medicine · doi:10.1056/nejmoa1803396 · Cited by 1,067
Self-citationActionable Diagnosis of Neuroleptospirosis by Next-Generation Sequencing
Wilson, M. R., Naccache, S. N., Samayoa, E., Biagtan, M., Bashir, H., Yu, G., Salamat, S. M., Somasekar, S., Federman, S., Miller, S., et al. (2014) · New England Journal of Medicine · doi:10.1056/nejmoa1401268 · Cited by 977
Self-citationSimner, P. J., Miller, S., & Carroll, K. C. (2018) · Clinical Infectious Diseases · doi:10.1093/cid/cix881 · Cited by 821
Self-citationBeyond Viruses: Clinical Profiles and Etiologies Associated with Encephalitis
Glaser, C. A., Honarmand, S., Anderson, L. J., Schnurr, D. P., Forghani, B., Cossen, C. K., Schuster, F. L., Christie, L. J., & Tureen, J. H. (2006) · Clinical Infectious Diseases · doi:10.1086/509330 · Cited by 650
Validation of Metagenomic Next-Generation Sequencing Tests for Universal Pathogen Detection
Schlaberg, R., Chiu, C. Y., Miller, S., Procop, G. W., Weinstock, G., the Professional Practice Committee and Committee on Laboratory Practices of the American Society for Microbiology, & the Microbiology Resource Committee of the College of American Pathologists (2017) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2016-0539-ra · Cited by 550
Self-citationNaccache, S. N., Federman, S., Veeraraghavan, N., Zaharia, M., Lee, D., Samayoa, E., Bouquet, J., Greninger, A. L., Luk, K.-C., Enge, B., et al. (2014) · Genome Research · doi:10.1101/gr.171934.113 · Cited by 460
Self-citationGlaser, C. A., Gilliam, S., Schnurr, D., Forghani, B., Honarmand, S., Khetsuriani, N., Fischer, M., Cossen, C. K., & Anderson, L. J. (2003) · Clinical Infectious Diseases · doi:10.1086/367841 · Cited by 434
Foulongne, V., Sauvage, V., Hebert, C., Dereure, O., Cheval, J., Gouilh, M. A., Pariente, K., Segondy, M., Burguière, A., Manuguerra, J.-C., et al. (2012) · PLoS ONE · doi:10.1371/journal.pone.0038499 · Cited by 401
Goldberg, B., Sichtig, H., Geyer, C., Ledeboer, N., & Weinstock, G. M. (2015) · mBio · doi:10.1128/mbio.01888-15 · Cited by 360
The blood DNA virome in 8,000 humans
Moustafa, A., Xie, C., Kirkness, E., Biggs, W., Wong, E., Turpaz, Y., Bloom, K., Delwart, E., Nelson, K. E., Venter, J. C., et al. (2017) · PLOS Pathogens · doi:10.1371/journal.ppat.1006292 · Cited by 307
Naccache, S. N., Peggs, K. S., Mattes, F. M., Phadke, R., Garson, J. A., Grant, P., Samayoa, E., Federman, S., Miller, S., Lunn, M. P., et al. (2015) · Clinical Infectious Diseases · doi:10.1093/cid/ciu912 · Cited by 294
Self-citationMolecular Methods for Diagnosis of Viral Encephalitis
DeBiasi, R. L., & Tyler, K. L. (2004) · Clinical Microbiology Reviews · doi:10.1128/cmr.17.4.903-925.2004 · Cited by 274
Self-citationChiu, C. Y. (2013) · Current Opinion in Microbiology · doi:10.1016/j.mib.2013.05.001 · Cited by 234
Self-citationSalipante, S. J., Sengupta, D. J., Rosenthal, C., Costa, G., Spangler, J., Sims, E. H., Jacobs, M. A., Miller, S. I., Hoogestraat, D. R., Cookson, B. T., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0065226 · Cited by 220
Self-citationRapid Molecular Microbiologic Diagnosis of Prosthetic Joint Infection
Cazanave, C., Greenwood‐Quaintance, K. E., Hanssen, A. D., Karau, M. J., Schmidt, S. M., Urena, E. O. G., Mandrekar, J. N., Osmon, D. R., Lough, L. E., Pritt, B. S., et al. (2013) · Journal of Clinical Microbiology · doi:10.1128/jcm.00335-13 · Cited by 202
Burden of encephalitis-associated hospitalizations in the United States, 1988-1997
Khetsuriani, N., Holman, R. C., & Anderson, L. J. (2002) · Clinical Infectious Diseases · doi:10.1086/341301 · Cited by 200
Parize, P., Muth, E., Richaud, C., Gratigny, M., Pilmis, B., Lamamy, A., Mainardi, J.-L., Cheval, J., de Visser, L., Jagorel, F., et al. (2017) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2017.02.006 · Cited by 198
A metagenomic analysis of pandemic influenza A (2009 H1N1) infection in patients from North America
Greninger, A. L., Chen, E. C., Sittler, T., Scheinerman, A., Roubinian, N., Yu, G., Kim, E., Pillai, D. R., Guyard, C., Mazzulli, T., et al. (2010) · PLoS ONE · doi:10.1371/journal.pone.0013381 · Cited by 193
Self-citationStrong, M. J., Xu, G., Morici, L., Splinter Bon-Durant, S., Baddoo, M., Lin, Z., Fewell, C., Taylor, C. M., & Flemington, E. K. (2014) · PLoS Pathogens · doi:10.1371/journal.ppat.1004437 · Cited by 186
Next-generation sequencing in neuropathologic diagnosis of infections of the nervous system
(2016) · doi:10.1212/nxi.0000000000000251 · Cited by 171
Frémond, M., Pérot, P., Muth, E., Cros, G., Dumarest, M., Mahlaoui, N., Seilhean, D., Desguerre, I., Hébert, C., Corre-Catelin, N., et al. (2015) · Journal of the Pediatric Infectious Diseases Society · doi:10.1093/jpids/piv040 · Cited by 136
(2015) · doi:10.1186/s13073-015-0235-2 · Cited by 123
Self-citationNeurobrucellosis: Unexpected Answer From Metagenomic Next-Generation Sequencing
Mongkolrattanothai, K., Naccache, S. N., Bender, J. M., Samayoa, E., Pham, E., Yu, G., Dien Bard, J., Miller, S., Aldrovandi, G., & Chiu, C. Y. (2017) · Journal of the Pediatric Infectious Diseases Society · doi:10.1093/jpids/piw066 · Cited by 113
Self-citationSchlaberg, R., Queen, K. J., Simmon, K. E., Tardif, K. D., Stockmann, C. R., Flygare, S. D., Kennedy, B., Voelkerding, K. V., Bramley, A. M., Zhang, J., et al. (2017) · The Journal of Infectious Diseases · doi:10.1093/infdis/jix148 · Cited by 107
Chiu, C. Y., Coffey, L. L., Murkey, J., Symmes, K., Sample, H. A., Wilson, M. R., Naccache, S. N., Arevalo, S., Somasekar, S., Federman, S., et al. (2017) · Emerging Infectious Diseases · doi:10.3201/eid2310.161986 · Cited by 90
Self-citationHuman anelloviruses and the central nervous system
Maggi, F., & Bendinelli, M. (2010) · Reviews in Medical Virology · doi:10.1002/rmv.668 · Cited by 74
Impact of three Illumina library construction methods on GC bias and HLA genotype calling
Lan, J. H., Yin, Y., Reed, E. F., Moua, K., Thomas, K. A., & Zhang, Q. (2014) · Human Immunology · doi:10.1016/j.humimm.2014.12.016 · Cited by 74
Pryce, T. M., Palladino, S., Price, D. M., Gardam, D. J., Campbell, P. B., Christiansen, K., & Murray, R. J. (2006) · Diagnostic Microbiology and Infectious Disease · doi:10.1016/j.diagmicrobio.2005.11.002 · Cited by 69
From Sample to Multi-Omics Conclusions in under 48 Hours
Quinn, R. A., Navas-Molina, J. A., Hyde, E. R., Song, S. J., Vázquez-Baeza, Y., Humphrey, G., Gaffney, J., Minich, J. J., Melnik, A. V., Herschend, J., et al. (2016) · mSystems · doi:10.1128/msystems.00038-16 · Cited by 61
Diagnostic Approach to Health Care- and Device-Associated Central Nervous System Infections
Martin, R. M., Zimmermann, L. L., Huynh, M., & Polage, C. R. (2018) · Journal of Clinical Microbiology · doi:10.1128/jcm.00861-18 · Cited by 55
Improved Multiple Displacement Amplification (iMDA) and Ultraclean Reagents
Motley, S. T., Picuri, J. M., Crowder, C. D., Minich, J. J., Hofstadler, S. A., & Eshoo, M. W. (2014) · BMC Genomics · doi:10.1186/1471-2164-15-443 · Cited by 50
Luk, K., Berg, M. G., Naccache, S. N., Kabre, B., Federman, S. M., Mbanya, D., Kaptué, L., Chiu, C. Y., Brennan, C. A., & Hackett, J. (2015) · PLoS ONE · doi:10.1371/journal.pone.0141723 · Cited by 46
Self-citationGenome Sequence of the Pathogenic Fungus Sporothrix schenckii (ATCC 58251)
Cuomo, C. A., Valle, N. R.-D., Pérez-Sánchez, L., Abouelleil, A., Goldberg, J. M., Young, S., Zeng, Q., & Birren, B. W. (2014) · Genome Announcements · doi:10.1128/genomea.00446-14 · Cited by 36
Washington, J. A. (1996) · Europe PMC (PubMed Central) · Cited by 36
Detection of Murine Leukemia Virus or Mouse DNA in Commercial RT-PCR Reagents and Human DNAs
Zheng, H., Jia, H., Shankar, A., Heneine, W., & Switzer, W. M. (2011) · PLoS ONE · doi:10.1371/journal.pone.0029050 · Cited by 30
Modeling of the GC content of the substituted bases in bacterial core genomes
Bohlin, J., Eldholm, V., Brynildsrud, O. B., Pettersson, J. H. ‐., & Alfsnes, K. (2018) · BMC Genomics · doi:10.1186/s12864-018-4984-3 · Cited by 24
Wilson, M. R., Fedewa, G., Stenglein, M. D., Olejnik, J., Rennick, L. J., Nambulli, S., Feldmann, F., Duprex, W. P., Connor, J. H., Mühlberger, E., et al. (2016) · mSystems · doi:10.1128/msystems.00058-16 · Cited by 17
(2016) · Molecular Microbiology · doi:10.1128/9781555819071.ch6 · Cited by 10
Research works printed but not linked: 1
- Meiser KL. 2002. Reporting results from studies evaluating diagnostic tests. In Clinical Microbiology Newsletter, Vol 24 (ed. FaD Administration), pp. 60-63. Elsevier, Inc.
Cited by
Chiu, C. Y., & Miller, S. A. (2019) · Nature Reviews Genetics · doi:10.1038/s41576-019-0113-7 · Cited by 1,620
Self-citationClinical Metagenomic Sequencing for Diagnosis of Meningitis and Encephalitis
Wilson, M. R., Sample, H. A., Zorn, K. C., Arevalo, S., Yu, G., Neuhaus, J., Federman, S., Stryke, D., Briggs, B., Langelier, C., et al. (2019) · New England Journal of Medicine · doi:10.1056/nejmoa1803396 · Cited by 1,067
Self-citationRapid pathogen detection by metagenomic next-generation sequencing of infected body fluids
Gu, W., Deng, X., Lee, M., Sucu, Y. D., Arevalo, S., Stryke, D., Federman, S., Gopez, A., Reyes, K., Zorn, K., et al. (2021) · Nature Medicine · doi:10.1038/s41591-020-1105-z · Cited by 689
Self-citationBenchmarking Metagenomics Tools for Taxonomic Classification
Ye, S. H., Siddle, K. J., Park, D. J., & Sabeti, P. C. (2019) · Cell · doi:10.1016/j.cell.2019.07.010 · Cited by 593
mNGS in clinical microbiology laboratories: on the road to maturity
Han, D., Li, Z., Li, R., Tan, P., Zhang, R., & Li, J. (2019) · Critical Reviews in Microbiology · doi:10.1080/1040841x.2019.1681933 · Cited by 361
Metadata
- Authors
- Miller, S., Naccache, S., Samayoa, E., Messacar, K., Arevalo, S., Federman, S., Stryke, D., Pham, E., Fung, B., Bolosky, W., Ingebrigtsen, D., Lorizio, W., Paff, S., Leake, J., Pesano, R., DeBiasi, R., Dominguez, Sr., & Chiu, C.
- Affiliations
- University of California, San Francisco (US)
- Children's Hospital Colorado (US)
- Quest Diagnostics (United States) (US)
- Venue
- Genome Research
- Publisher
- openRxiv
- Published
- 2018-05-25
- Volume
- 29
- Issue
- 5
- Pages
- 831-842
- ISSN
- 1088-9051 (ISSN-L)
- PMC
- PMC6499319
- OpenAlex
- W2951983907
- Cited by
- 583
- References
- 45
- Citations by year
- 2026: 56 · 2025: 109 · 2024: 96 · 2023: 87 · 2022: 94 · 2021: 61 · 2020: 49 · 2019: 29 · 2018: 2
- Type
- posted content
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Genome Research (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by-nc)
- HighWire Press Open Archive (submittedVersion)
- Funders
- Abbott Laboratories
- National Institutes of Health (4r33ai120977-03, R21/R33, UL1TR000004, UL1TR002535, TR000004, R21/R33 AI120977)
- University of California, San Francisco (TR000004, UL1-TR000004)
- Abbott Diagnostics
- National Heart, Lung, and Blood Institute (R01HL105704, U54HL119893)
- National Institute of Allergy and Infectious Diseases (R33-AI120977, R21AI120977)
- National Center for Advancing Translational Sciences (TR000004, UCSF-CTSI grant UL1 TR000004)
- Topics
- Health Sciences › Medicine › Microbiology (medical) › Bacterial Identification and Susceptibility Testing
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Life Sciences › Immunology and Microbiology › Microbiology › Bacterial Infections and Vaccines
- Keywords
- metagenomic next-generation sequencing, cerebrospinal fluid, meningitis, encephalitis, myelitis, diagnostic accuracy, detection limit, hemolysis, clinical microbiology
- MeSH terms
- Child, Encephalitis, Humans, Meningitis, Aseptic, Myelitis, Sensitivity and Specificity, Viruses, Computational Biology, Metagenomics, High-Throughput Nucleotide Sequencing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.