Validation of Metagenomic Next-Generation Sequencing Tests for Universal Pathogen Detection
Open accessAbstract
Context.—: Metagenomic sequencing can be used for detection of any pathogens using unbiased, shotgun next-generation sequencing (NGS), without the need for sequence-specific amplification. Proof-of-concept has been demonstrated in infectious disease outbreaks of unknown causes and in patients with suspected infections but negative results for conventional tests. Metagenomic NGS tests hold great promise to improve infectious disease diagnostics, especially in immunocompromised and critically ill patients. Objective.—: To discuss challenges and provide example solutions for validating metagenomic pathogen detection tests in clinical laboratories. A summary of current regulatory requirements, largely based on prior guidance for NGS testing in constitutional genetics and oncology, is provided. Data Sources.—: Examples from 2 separate validation studies are provided for steps from assay design, and validation of wet bench and bioinformatics protocols, to quality control and assurance. Conclusions.—: Although laboratory and data analysis workflows are still complex, metagenomic NGS tests for infectious diseases are increasingly being validated in clinical laboratories. Many parallels exist to NGS tests in other fields. Nevertheless, specimen preparation, rapidly evolving data analysis algorithms, and incomplete reference sequence databases are idiosyncratic to the field of microbiology and often overlooked.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Customized data sets can be prepared to mimic input sequence data and expand range of microorganisms detected through in silico analysis.
“Customized data sets can be prepared to mimic input sequence data and expand the range of microorganisms detected through in silico analysis37.”
From Clinical metagenomics (2019) · cites this paper as [37]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Fast and sensitive protein alignment using DIAMOND
Buchfink, B., Xie, C., & Huson, D. H. (2015) · Nature Methods · doi:10.1038/nmeth.3176 · Cited by 14,201 · see citing works
Kraken: ultrafast metagenomic sequence classification using exact alignments
Wood, D. E., & Salzberg, S. L. (2014) · Genome Biology · doi:10.1186/gb-2014-15-3-r46 · Cited by 4,521 · see citing works
The SILVA and “All-species Living Tree Project (LTP)” taxonomic frameworks
Yilmaz, P., Parfrey, L., Yarza, P., Gerken, J., Pruesse, E., Quast, C., Schweer, T., Peplies, J., Ludwig, W., & Glöckner, F. (2013) · Nucleic Acids Res. · doi:10.1093/nar/gkt1209 · Cited by 3,547 · see citing works
Insights into the phylogeny and coding potential of microbial dark matter
Rinke, C., Schwientek, P., Sczyrba, A., Ivanova, N. N., Anderson, I. J., Cheng, J.-F., Darling, A., Malfatti, S., Swan, B. K., Gies, E. A., et al. (2013) · Nature · doi:10.1038/nature12352 · Cited by 2,394 · see citing works
Huson, D. H., Beier, S., Flade, I., Górska, A., El-Hadidi, M., Mitra, S., Ruscheweyh, H.-J., & Tappu, R. (2016) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004957 · Cited by 1,761 · see citing works
Actionable Diagnosis of Neuroleptospirosis by Next-Generation Sequencing
Wilson, M. R., Naccache, S. N., Samayoa, E., Biagtan, M., Bashir, H., Yu, G., Salamat, S. M., Somasekar, S., Federman, S., Miller, S., et al. (2014) · New England Journal of Medicine · doi:10.1056/nejmoa1401268 · Cited by 977 · see citing works
Self-citationBeyond Viruses: Clinical Profiles and Etiologies Associated with Encephalitis
Glaser, C. A., Honarmand, S., Anderson, L. J., Schnurr, D. P., Forghani, B., Cossen, C. K., Schuster, F. L., Christie, L. J., & Tureen, J. H. (2006) · Clinical Infectious Diseases · doi:10.1086/509330 · Cited by 649 · see citing works
Ounit, R., Wanamaker, S. I., Close, T. J., & Lonardi, S. (2015) · BMC Genomics · doi:10.1186/s12864-015-1419-2 · Cited by 624 · see citing works
Naccache, S. N., Federman, S., Veeraraghavan, N., Zaharia, M., Lee, D., Samayoa, E., Bouquet, J., Greninger, A. L., Luk, K.-C., Enge, B., et al. (2014) · Genome Research · doi:10.1101/gr.171934.113 · Cited by 460 · see citing works
Self-citationA Strategy To Estimate Unknown Viral Diversity in Mammals
Anthony, S. J., Epstein, J. H., Murray, K. A., Navarrete‐Macias, I., Zambrana‐Torrelio, C., Solovyov, A., Flores, R. O., Arrigo, N. C., Islam, A., Khan, S. A., et al. (2013) · mBio · doi:10.1128/mbio.00598-13 · Cited by 416 · see citing works
National Inpatient Hospital Costs: The Most Expensive Conditions by Payer, 2011
Torio, C. M., & Andrews, R. M. (2013) · Cited by 413 · see citing works
College of American Pathologists' Laboratory Standards for Next-Generation Sequencing Clinical Tests
Aziz, N., Zhao, Q., Bry, L., Driscoll, D. K., Funke, B., Gibson, J. S., Grody, W. W., Hegde, M., Hoeltge, G. A., Leonard, D. G. B., et al. (2014) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2014-0250-cp · Cited by 386 · see citing works
Virome Capture Sequencing Enables Sensitive Viral Diagnosis and Comprehensive Virome Analysis
Briese, T., Kapoor, A., Mishra, N., Jain, K., Kumar, A., Jabado, O., & Lipkin, W. I. (2015) · mBio · doi:10.1128/mbio.01491-15 · Cited by 378 · see citing works
Goldberg, B., Sichtig, H., Geyer, C., Ledeboer, N., & Weinstock, G. M. (2015) · mBio · doi:10.1128/mbio.01888-15 · Cited by 360 · see citing works
Self-citationGu, W., Crawford, E. D., O’Donovan, B. D., Wilson, M. R., Chow, E. D., Retallack, H., & DeRisi, J. L. (2016) · Genome biology · doi:10.1186/s13059-016-0904-5 · Cited by 346 · see citing works
Petti, C. A., Bosshard, P. P., Brandt, M., et al. (2008) · Zurich Open Repository and Archive (University of Zurich) · Cited by 293 · see citing works
Enhanced virome sequencing using targeted sequence capturePreprint
Wylie, T., Wylie, K. M., Herter, B. N., & Storch, G. A. (2015) · Genome Research · doi:10.1101/gr.191049.115 · Cited by 267 · see citing works
Burden of encephalitis-associated hospitalizations in the United States, 1998-2010
Vora, N. M., Holman, R. C., Mehal, J. M., Steiner, C. A., Blanton, J., & Sejvar, J. (2014) · Neurology · doi:10.1212/wnl.0000000000000086 · Cited by 264 · see citing works
Naccache, S. N., Greninger, A. L., Lee, D., Coffey, L. L., Phan, T. G., Rein‐Weston, A., Aronsohn, A. I., Hackett, J., Delwart, E., & Chiu, C. Y. (2013) · Journal of Virology · doi:10.1128/jvi.02323-13 · Cited by 251 · see citing works
Self-citationHasan, M. R., Rawat, A., Tang, P., Jithesh, P. V., Thomas, E., Tan, R., & Tilley, P. (2016) · Journal of Clinical Microbiology · doi:10.1128/jcm.03050-15 · Cited by 243 · see citing works
Genomic Encyclopedia of Bacteria and Archaea: Sequencing a Myriad of Type Strains
Kyrpides, N. C., Hugenholtz, P., Eisen, J. A., Woyke, T., Göker, M., Parker, C. T., Amann, R., Beck, B. J., Chain, P., Chun, J., et al. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001920 · Cited by 223 · see citing works
Self-citationvan Gageldonk‐Lafeber, A. B., Heijnen, M.-L. A., Bartelds, A. I. M., Peters, M., van der Plas, S. M., & Wilbrink, B. (2005) · Clinical Infectious Diseases · doi:10.1086/431982 · Cited by 210 · see citing works
Graf, E. H., Simmon, K. E., Tardif, K. D., Hymas, W. C., Flygare, S. D., Eilbeck, K. L., Yandell, M. D., & Schlaberg, R. (2016) · Journal of Clinical Microbiology · doi:10.1128/jcm.03060-15 · Cited by 204 · see citing works
Self-citationLusk, R. W. (2014) · PLoS ONE · doi:10.1371/journal.pone.0110808 · Cited by 201 · see citing works
A metagenomic analysis of pandemic influenza A (2009 H1N1) infection in patients from North America
Greninger, A. L., Chen, E. C., Sittler, T., Scheinerman, A., Roubinian, N., Yu, G., Kim, E., Pillai, D. R., Guyard, C., Mazzulli, T., et al. (2010) · PLoS ONE · doi:10.1371/journal.pone.0013381 · Cited by 193 · see citing works
Self-citationStrong, M. J., Xu, G., Morici, L., Splinter Bon-Durant, S., Baddoo, M., Lin, Z., Fewell, C., Taylor, C. M., & Flemington, E. K. (2014) · PLoS Pathogens · doi:10.1371/journal.ppat.1004437 · Cited by 186 · see citing works
Fatal Zika Virus Infection with Secondary Nonsexual Transmission
Swaminathan, S., Schlaberg, R., Lewis, J., et al. (2016) · New England Journal of Medicine · doi:10.1056/nejmc1610613 · Cited by 169 · see citing works
Self-citationFlygare, S., Simmon, K., Miller, C., Qiao, Y., Kennedy, B., Di Sera, T., Graf, E. H., Tardif, K. D., Kapusta, A., Rynearson, S., et al. (2016) · Genome Biology · doi:10.1186/s13059-016-0969-1 · Cited by 164 · see citing works
Self-citationNeurobrucellosis: Unexpected Answer From Metagenomic Next-Generation Sequencing
Mongkolrattanothai, K., Naccache, S. N., Bender, J. M., Samayoa, E., Pham, E., Yu, G., Bard, J. D., Miller, S. A., Aldrovandi, G. M., & Chiu, C. Y. (2016) · Journal of the Pediatric Infectious Diseases Society · doi:10.1093/jpids/piw066 · Cited by 113 · see citing works
Self-citationLouie, J. K., Hacker, J. K., Gonzales, R., Mark, J., Maselli, J. H., Yagi, S., & Drew, W. (2005) · Clinical Infectious Diseases · doi:10.1086/432800 · Cited by 105 · see citing works
Fischer, N., Indenbirken, D., Meyer, T., Lütgehetmann, M., Lellek, H., Spohn, M., Aepfelbacher, M., Alawi, M., & Grundhoff, A. T. (2015) · Journal of Clinical Microbiology · doi:10.1128/jcm.02495-14 · Cited by 94 · see citing works
Factors associated with unknown aetiology in patients with community-acquired pneumonia
Ewig, S., Torres, A., Marcos, M. Á., et al. (2002) · European Respiratory Journal · doi:10.1183/09031936.02.01942001 · Cited by 91 · see citing works
Methods-Based Proficiency Testing in Molecular Genetic Pathology
Schrijver, I., Aziz, N., Jennings, L. J., Richards, C. S., Voelkerding, K. V., & Weck, K. E. (2014) · Journal of Molecular Diagnostics · doi:10.1016/j.jmoldx.2014.02.002 · Cited by 55 · see citing works
Cyclovirus in nasopharyngeal aspirates of Chilean children with respiratory infections
Phan, T. G., Luchsinger, V., Avendaño, L. F., Deng, X., & Delwart, E. (2014) · Journal of General Virology · doi:10.1099/vir.0.061143-0 · Cited by 50 · see citing works
Lack of Evidence for Plague or Anthrax on the New York City Subway
Ackelsberg, J., Rakeman, J. L., Hughes, S., et al. (2015) · Cell Systems · doi:10.1016/j.cels.2015.07.008 · Cited by 45 · see citing works
False-positive results in metagenomic virus discovery: a strong case for follow-up diagnosis.
Rosseel, T., Pardon, B., Clercq, K. D., Ozhelvaci, O., & van Borm, S. (2014) · Transboundary and Emerging Diseases · doi:10.1111/tbed.12251 · Cited by 44 · see citing works
Role of Clinicogenomics in Infectious Disease Diagnostics and Public Health Microbiology
Westblade, L. F., van Belkum, A., Grundhoff, A. T., Weinstock, G. M., Pamer, E. G., Pallen, M. J., & Dunne, W. M. (2016) · Journal of Clinical Microbiology · doi:10.1128/jcm.02664-15 · Cited by 39 · see citing works
Self-citationA Model Study of In Silico Proficiency Testing for Clinical Next-Generation Sequencing
Duncavage, E. J., Abel, H., Merker, J. D., Bodner, J. B., Zhao, Q., Voelkerding, K. V., & Pfeifer, J. D. (2016) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2016-0194-cp · Cited by 38 · see citing works
Alquezar‐Planas, D. E., Mourier, T., Bruhn, C., Hansen, A. J., Vitcetz, S. N., Mørk, S., Gorodkin, J., Nielsen, H., Guo, Y., Sethuraman, A., et al. (2013) · Scientific Reports · doi:10.1038/srep02468 · Cited by 34 · see citing works
Encephalitis: Why We Need to Keep Pushing the Envelope
Glaser, C., & Bloch, K. C. (2009) · Clinical Infectious Diseases · doi:10.1086/648420 · Cited by 21 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [3]Matulonis UA, Lorusso D, Oaknin A, et al. J Clin Oncol. 2023;41(13):2436-2445. doi:10.1200/JCO.22.01900 · DOI
Cited by
Sorted by citations
Chiu, C. Y., & Miller, S. A. (2019) · Nature Reviews Genetics · doi:10.1038/s41576-019-0113-7 · Cited by 1,620 · see citing works
Self-citationClinical Metagenomic Next-Generation Sequencing for Pathogen Detection
Gu, W., Miller, S., & Chiu, C. Y. (2019) · Annual Review of Pathology: Mechanisms of Disease · doi:10.1146/annurev-pathmechdis-012418-012751 · Cited by 1,458 · see citing works
Self-citationClinical Metagenomic Sequencing for Diagnosis of Meningitis and Encephalitis
Wilson, M. R., Sample, H. A., Zorn, K. C., Arevalo, S., Yu, G., Neuhaus, J., Federman, S., Stryke, D., Briggs, B., Langelier, C., et al. (2019) · New England Journal of Medicine · doi:10.1056/nejmoa1803396 · Cited by 1,067 · see citing works
Self-citationBlauwkamp, T. A., Thair, S., Rosen, M. J., Blair, L., Lindner, M. S., Vilfan, I. D., Kawli, T., Christians, F. C., Venkatasubrahmanyam, S., Wall, G. D., et al. (2019) · Nature Microbiology · doi:10.1038/s41564-018-0349-6 · Cited by 962 · see citing works
Simner, P. J., Miller, S., & Carroll, K. C. (2018) · Clinical Infectious Diseases · doi:10.1093/cid/cix881 · Cited by 821 · see citing works
Self-citationThe challenge of diagnostic metagenomics
Greninger, A. L. (2018) · Expert Review of Molecular Diagnostics · doi:10.1080/14737159.2018.1487292 · Cited by 156 · see citing works
Miller, S., Naccache, S., Samayoa, E., Messacar, K., Arevalo, S., Federman, S., Stryke, D., Pham, E., Fung, B., Bolosky, W., et al. (2018) · doi:10.1101/330381 · Cited by 583 · see citing works
Self-citationHasan, M. R., Sundararaju, S., Tang, P., Tsui, K.-M., Lopez, A. P., Janahi, M., Tan, R., & Tilley, P. (2020) · Scientific Reports · doi:10.1038/s41598-020-68159-z · Cited by 37 · see citing works
Molecular Diagnosis of the Central Nervous System (CNS) Infections
Vila, J., Bosch, J., & Muñoz-Almagro, C. (2021) · Enfermedades Infecciosas y Microbiología Clínica · doi:10.1016/j.eimc.2020.03.001 · Cited by 24 · see citing works
- Authors
- Schlaberg, R., Chiu, C. Y., Miller, S., Procop, G. W., Weinstock, G., the Professional Practice Committee and Committee on Laboratory Practices of the American Society for Microbiology, & the Microbiology Resource Committee of the College of American Pathologists
- Affiliations
- Cleveland Clinic (US)
- Venue
- Archives of Pathology & Laboratory Medicine
- Published
- 2017-02-07
- Volume
- 141
- Issue
- 6
- Pages
- 776-786
- ISSN
- 0003-9985 (ISSN-L)
- 1543-2165
- OpenAlex
- W2587565773
- Cited by
- 550
- References
- 40
- Citations by year
- 2026: 43 · 2025: 63 · 2024: 71 · 2023: 85 · 2022: 79 · 2021: 61 · 2020: 52 · 2019: 47 · 2018: 34 · 2017: 14 · 2015: 1
- FWCI
- 33.42
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Archives of Pathology & Laboratory Medicine (publishedVersion)
- PubMed (publishedVersion)
- The Mouseion at the JAXlibrary (Jackson Laboratory) (submittedVersion)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Clinical Biochemistry › Bacterial Identification and Susceptibility Testing
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Health Sciences › Medicine › Physiology › Clinical Laboratory Practices and Quality Control
- Keywords
- metagenomic sequencing, pathogen detection, test validation, infectious disease diagnostics, clinical laboratory testing, regulatory requirements, shotgun sequencing, specimen preparation, bioinformatics analysis, quality assurance and quality control, reference sequence database, immunocompromised patients, critically ill patients
- MeSH terms
- Algorithms, Communicable Diseases, Humans, Quality Control, Sensitivity and Specificity, Specimen Handling, Reproducibility of Results, Sequence Analysis, DNA, Computational Biology, Databases, Nucleic Acid, Metagenomics, Workflow, High-Throughput Nucleotide Sequencing, Clinical Laboratory Services
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
