The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Low concordance of multiple variant-calling pipelines: practical implications for exome and genome sequencing
Open accessAbstract
Background: To facilitate the clinical implementation of genomic medicine by next-generation sequencing, it will be critically important to obtain accurate and consistent variant calls on personal genomes. Multiple software tools for variant calling are available, but it is unclear how comparable these tools are or what their relative merits in real-world scenarios might be. Methods: We sequenced 15 exomes from four families using commercial kits (Illumina HiSeq 2000 platform and Agilent SureSelect version 2 capture kit), with approximately 120X mean coverage. We analyzed the raw data using near-default parameters with five different alignment and variant-calling pipelines (SOAP, BWA-GATK, BWA-SNVer, GNUMAP, and BWA-SAMtools). We additionally sequenced a single whole genome using the sequencing and analysis pipeline from Complete Genomics (CG), with 95% of the exome region being covered by 20 or more reads per base. Finally, we validated 919 single-nucleotide variations (SNVs) and 841 insertions and deletions (indels), including similar fractions of GATK-only, SOAP-only, and shared calls, on the MiSeq platform by amplicon sequencing with approximately 5000X mean coverage. Results: SNV concordance between five Illumina pipelines across all 15 exomes was 57.4%, while 0.5 to 5.1% of variants were called as unique to each pipeline. Indel concordance was only 26.8% between three indel-calling pipelines, even after left-normalizing and intervalizing genomic coordinates by 20 base pairs. There were 11% of CG variants falling within targeted regions in exome sequencing that were not called by any of the Illumina-based exome analysis pipelines. Based on targeted amplicon sequencing on the MiSeq platform, 97.1%, 60.2%, and 99.1% of the GATK-only, SOAP-only and shared SNVs could be validated, but only 54.0%, 44.6%, and 78.1% of the GATK-only, SOAP-only and shared indels could be validated. Additionally, our analysis of two families (one with four individuals and the other with seven), demonstrated additional accuracy gained in variant discovery by having access to genetic data from a multi-generational family. Conclusions: Our results suggest that more caution should be exercised in genomic medicine settings when analyzing individual genomes, including interpreting positive and negative findings with scrutiny, especially for indels. We advocate for renewed collection and sequencing of multi-generational families to increase the overall accuracy of whole genomes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Sequence Alignment/Map format and SAMtools
Li, H., Handsaker, B., Wysoker, A., Fennell, T., Ruan, J., Homer, N., Marth, G., Abecasis, G., Durbin, R., & 1000 Genome Project Data Processing Subgroup (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp352 · Cited by 62,041
Fast and accurate short read alignment with Burrows-Wheeler transform
Li, H., & Durbin, R. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp324 · Cited by 58,607
The Genome Analysis Toolkit: A MapReduce framework for analyzing next-generation DNA sequencing data
McKenna, A., Hanna, M., Banks, E., Sivachenko, A., Cibulskis, K., Kernytsky, A., Garimella, K., Altshuler, D., Gabriel, S., Daly, M., et al. (2010) · Genome Research · doi:10.1101/gr.107524.110 · Cited by 27,895
ANNOVAR: functional annotation of genetic variants from high-throughput sequencing data
Wang, K., Li, M., & Hakonarson, H. (2010) · Nucleic Acids Research · doi:10.1093/nar/gkq603 · Cited by 14,621
Self-citationA framework for variation discovery and genotyping using next-generation DNA sequencing data
DePristo, M. A., Banks, E., Poplin, R., Garimella, K. V., Maguire, J. R., Hartl, C., Philippakis, A. A., del Angel, G., Rivas, M. A., Hanna, M., et al. (2011) · Nature Genetics · doi:10.1038/ng.806 · Cited by 11,456
An integrated map of genetic variation from 1,092 human genomes
The 1000 Genomes Project Consortium (2012) · Nature · doi:10.1038/nature11632 · Cited by 7,711
SOAP2: an improved ultrafast tool for short read alignment
Li, R., Yu, C., Li, Y., Lam, T.-W., Yiu, S.-M., Kristiansen, K., & Wang, J. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp336 · Cited by 3,760
SOAP: Short oligonucleotide alignment program
Li, R., Li, Y., Kristiansen, K., & Wang, J. (2008) · Bioinformatics · doi:10.1093/bioinformatics/btn025 · Cited by 3,597
Browning, S. R., & Browning, B. L. (2007) · The American Journal of Human Genetics · doi:10.1086/521987 · Cited by 3,210
De novo assembly of human genomes with massively parallel short read sequencing
Li, R., Zhu, H., Ruan, J., Qian, W., Fang, X., Shi, Z., Li, Y., Li, S., Shan, G., Kristiansen, K., et al. (2010) · Genome Research · doi:10.1101/gr.097261.109 · Cited by 2,926 · cited 2×
“01 SOAP uses a method based on Bayes’ theorem to call consensus genotype by carefully considering the data quality, alignment, and recurring experimental errors [22].”
MethodSporadic autism exomes reveal a highly interconnected protein network of de novo mutations
O’Roak, B. J., Vives, L., Girirajan, S., Karakoc, E., Krumm, N., Coe, B. P., Levy, R., Ko, A., Lee, C., Smith, J. D., et al. (2012) · Nature · doi:10.1038/nature10989 · Cited by 2,165 · cited 2×
“Although most studies utilizing NGS data to date have focused on ‘quads’ or ‘trios’ [47-51], studying large and/or consanguineous families can maximize the utility of filtering strategies and statistical approaches for identifying disease-contributory variants in genetic disease [52].”
BackgroundDe novo mutations revealed by whole-exome sequencing are strongly associated with autism
Sanders, S. J., Murtha, M. T., Gupta, A. R., Murdoch, J. D., Raubeson, M. J., Willsey, A. J., Ercan-Sencicek, A. G., DiLullo, N. M., Parikshak, N. N., Stein, J. L., et al. (2012) · Nature · doi:10.1038/nature10945 · Cited by 2,084
Targeted capture and massively parallel sequencing of 12 human exomes
Ng, S. B., Turner, E. H., Robertson, P. D., Flygare, S. D., Bigham, A. W., Lee, C., Shaffer, T., Wong, M., Bhattacharjee, A., Eichler, E. E., et al. (2009) · Nature · doi:10.1038/nature08250 · Cited by 2,030
Exome sequencing identifies the cause of a mendelian disorder
Ng, S. B., Buckingham, K. J., Lee, C., Bigham, A. W., Tabor, H. K., Dent, K. M., Huff, C. D., Shannon, P. T., Jabs, E. W., Nickerson, D. A., et al. (2010) · Nature Genetics · doi:10.1038/ng.499 · Cited by 1,957
Fast and accurate genotype imputation in genome-wide association studies through pre-phasing
Howie, B., Fuchsberger, C., Stephens, M., Marchini, J., & Abecasis, G. R. (2012) · Nature Genetics · doi:10.1038/ng.2354 · Cited by 1,737 · cited 1×
“Hence, we did not evaluate pipelines that specifically employ imputation or multi-unrelated-sample variant-calling algorithms, such as Thunder [56], IMPUTE2 [57-59], or BEAGLE [60,61], or specific procedures within the software tools used that allow for calling of multiple unrelated sample (such as those available by…”
MethodPatterns and rates of exonic de novo mutations in autism spectrum disorders
Neale, B. M., Kou, Y., Liu, L., Ma’ayan, A., Samocha, K. E., Sabo, A., Lin, C.-F., Stevens, C., Wang, L.-S., Makarov, V., et al. (2012) · Nature · doi:10.1038/nature11011 · Cited by 1,734 · cited 2×
“Although most studies utilizing NGS data to date have focused on ‘quads’ or ‘trios’ [47-51], studying large and/or consanguineous families can maximize the utility of filtering strategies and statistical approaches for identifying disease-contributory variants in genetic disease [52].”
BackgroundSelf-citationEvolution and functional impact of rare coding variation from deep sequencing of human exomes
Tennessen, J. A., Bigham, A. W., O’Connor, T. D., Fu, W., Kenny, E. E., Gravel, S., McGee, S., Do, R., Liu, X., Jun, G., et al. (2012) · Science · doi:10.1126/science.1219240 · Cited by 1,613
De Novo Gene Disruptions in Children on the Autistic Spectrum
Iossifov, I., Ronemus, M., Levy, D., Wang, Z., Hakker, I., Rosenbaum, J., Yamrom, B., Lee, Y.-H., Narzisi, G., Leotta, A., et al. (2012) · Neuron · doi:10.1016/j.neuron.2012.04.009 · Cited by 1,446 · cited 2×
“Although most studies utilizing NGS data to date have focused on ‘quads’ or ‘trios’ [47-51], studying large and/or consanguineous families can maximize the utility of filtering strategies and statistical approaches for identifying disease-contributory variants in genetic disease [52].”
BackgroundGenotype and SNP calling from next-generation sequencing data
Nielsen, R., Paul, J. S., Albrechtsen, A., & Song, Y. S. (2011) · Nature Reviews Genetics · doi:10.1038/nrg2986 · Cited by 1,396
Exome sequencing identifies MLL2 mutations as a cause of Kabuki syndrome
Ng, S. B., Bigham, A. W., Buckingham, K. J., Hannibal, M. C., McMillin, M. J., Gildersleeve, H. I., Beck, A. E., Tabor, H. K., Cooper, G. M., Mefford, H. C., et al. (2010) · Nature Genetics · doi:10.1038/ng.646 · Cited by 1,347
Human Genome Sequencing Using Unchained Base Reads on Self-Assembling DNA Nanoarrays
Drmanac, R., Sparks, A. B., Callow, M. J., Halpern, A. L., Burns, N. L., Kermani, B. G., Carnevali, P., Nazarenko, I., Nilsen, G. B., Yeung, G., et al. (2010) · Science · doi:10.1126/science.1181498 · Cited by 1,212 · cited 1×
“The wholegenome DNA was sequenced using nanoarray-based shortread sequencing by ligation technology [14], including an adaptation of the pairwise end-sequencing strategy [15].”
MethodMapping copy number variation by population-scale genome sequencing
1000 Genomes Project, Mills, R. E., Walter, K., Stewart, C., Handsaker, R. E., Chen, K., Alkan, C., Abyzov, A., Yoon, S. C., Ye, K., et al. (2011) · Nature · doi:10.1038/nature09708 · Cited by 1,140 · cited 1×
“Although the focus of most variant-calling software has been on detecting SNVs, it is the case that large-scale structural copy number variants and small indels are known to also be a biologically relevant and prevalent form of genetic variation [40,41].”
BackgroundAnalysis of Genetic Inheritance in a Family Quartet by Whole-Genome Sequencing
Roach, J. C., Glusman, G., Smit, A. F. A., Huff, C. D., Hubley, R., Shannon, P. T., Rowen, L., Pant, K. P., Goodman, N., Bamshad, M., et al. (2010) · Science · doi:10.1126/science.1186802 · Cited by 1,084
Genotype Imputation with Thousands of Genomes
Howie, B., Marchini, J., & Stephens, M. (2011) · G3 Genes|Genomes|Genetics · doi:10.1534/g3.111.001198 · Cited by 966
SNP detection for massively parallel whole-genome resequencing
Li, R., Li, Y., Fang, X., Yang, H., Wang, J., Kristiansen, K., & Wang, J. (2009) · Genome Research · doi:10.1101/gr.088013.108 · Cited by 944
An initial map of insertion and deletion (INDEL) variation in the human genome
Mills, R. E., Luttig, C. T., Larkins, C. E., Beauchamp, A., Tsui, C., Pittard, W. S., & Devine, S. E. (2006) · Genome Research · doi:10.1101/gr.4565806 · Cited by 673
An abundance of rare functional variants in 202 drug target genes sequenced in 14,002 people
Nelson, M. R., Wegmann, D., Ehm, M. G., Kessner, D., St. Jean, P., Verzilli, C., Shen, J., Tang, Z., Bacanu, S.-A., Fraser, D., et al. (2012) · Science · doi:10.1126/science.1217876 · Cited by 647
Variation in genome-wide mutation rates within and between human families
the 1000 Genomes Project (2011) · Nature Genetics · doi:10.1038/ng.862 · Cited by 581
Performance comparison of exome DNA sequencing technologies
Clark, M. J., Chen, R., Lam, H. Y. K., Karczewski, K. J., Chen, R., Euskirchen, G., Butte, A. J., & Snyder, M. (2011) · Nature Biotechnology · doi:10.1038/nbt.1975 · Cited by 511
De novo gene mutations highlight patterns of genetic and neural complexity in schizophrenia
Xu, B., Ionita-Laza, I., Roos, J. L., Boone, B., Woodrick, S., Sun, Y., Levy, S., Gogos, J. A., & Karayiorgou, M. (2012) · Nature Genetics · doi:10.1038/ng.2446 · Cited by 442 · cited 2×
“Although most studies utilizing NGS data to date have focused on ‘quads’ or ‘trios’ [47-51], studying large and/or consanguineous families can maximize the utility of filtering strategies and statistical approaches for identifying disease-contributory variants in genetic disease [52].”
BackgroundCalculated risks: how to know when numbers deceive you
(2002) · Choice Reviews Online · doi:10.5860/choice.40-2217 · Cited by 437
A Fast, Powerful Method for Detecting Identity by Descent
Browning, B. L., & Browning, S. R. (2011) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2011.01.010 · Cited by 374
Small insertions and deletions (INDELs) in human genomes
Mullaney, J. M., Mills, R. E., Pittard, W. S., & Devine, S. E. (2010) · Human Molecular Genetics · doi:10.1093/hmg/ddq400 · Cited by 358
Performance comparison of whole-genome sequencing platforms
Lam, H. Y. K., Clark, M. J., Chen, R., Chen, R., Natsoulis, G., O'Huallachain, M., Dewey, F. E., Habegger, L., Ashley, E. A., Gerstein, M. B., et al. (2012) · Nature Biotechnology · doi:10.1038/nbt.2065 · Cited by 308
Low-coverage sequencing: Implications for design of complex trait association studies
Li, Y., Sidore, C., Kang, H. M., Boehnke, M., & Abecasis, G. R. (2011) · Genome Research · doi:10.1101/gr.117259.110 · Cited by 301
Natural genetic variation caused by small insertions and deletions in the human genome
Mills, R. E., Pittard, W. S., Mullaney, J. M., Farooq, U., Creasy, T. H., Mahurkar, A. A., Kemeza, D. M., Strassler, D. S., Ponting, C. P., Webber, C., et al. (2011) · Genome Research · doi:10.1101/gr.115907.110 · Cited by 268
Wei, Z., Wang, W., Hu, P., Lyon, G. J., & Hakonarson, H. (2011) · Nucleic Acids Research · doi:10.1093/nar/gkr599 · Cited by 252
Self-citationAccurate whole-genome sequencing and haplotyping from 10 to 20 human cells
Peters, B. A., Kermani, B. G., Sparks, A. B., Alferov, O., Hong, P., Alexeev, A., Jiang, Y., Dahl, F., Tang, Y. T., Haas, J., et al. (2012) · Nature · doi:10.1038/nature11236 · Cited by 251
Rope, A. F., Wang, K., Evjenth, R., Xing, J., Johnston, J. J., Swensen, J. J., Johnson, W. E., Moore, B., Huff, C. D., Bird, L. M., et al. (2011) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2011.05.017 · Cited by 250
Self-citationOptimized filtering reduces the error rate in detecting genomic variants by short-read sequencing
Reumers, J., De Rijk, P., Zhao, H., Liekens, A., Smeets, D., Cleary, J., Van Loo, P., Van Den Bossche, M., Catthoor, K., Sabbe, B., et al. (2012) · Nature Biotechnology · doi:10.1038/nbt.2053 · Cited by 240
A public resource facilitating clinical use of genomes
Ball, M. P., Thakuria, J. V., Zaranek, A. W., Clegg, T., Rosenbaum, A. M., Wu, X., Angrist, M., Bhak, J., Bobe, J., Callow, M. J., et al. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1201904109 · Cited by 221
SNVMix: predicting single nucleotide variants from next-generation sequencing of tumors
Goya, R., Sun, M. G. F., Morin, R. D., Leung, G., Ha, G., Wiegand, K. C., Senz, J., Crisan, A., Marra, M. A., Hirst, M., et al. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq040 · Cited by 211
Yu, T. W., Lim, E. T., Stevens, C. R., Gabriel, S., Chahrour, M. H., Ataman, B., Coulter, M. E., Hill, R. S., Schubert, C. R., Greenberg, M. E., et al. (2012) · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · Cited by 179
Phased Whole-Genome Genetic Risk in a Family Quartet Using a Major Allele Reference Sequence
Dewey, F. E., Chen, R., Cordero, S. P., Ormond, K. E., Caleshu, C. A., Karczewski, K. J., Whirl‐Carrillo, M., Wheeler, M. T., Dudley, J. T., Byrnes, J., et al. (2011) · PLoS Genetics · doi:10.1371/journal.pgen.1002280 · Cited by 151
Lee, H., & Schatz, M. C. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts330 · Cited by 131
Pairwise end sequencing: a unified approach to genomic mapping and sequencing
Roach, J. C., Boysen, C., Wang, K., & Hood, L. (1995) · Genomics · doi:10.1016/0888-7543(95)80219-c · Cited by 128
Self-citationDetecting and annotating genetic variations using the HugeSeq pipeline
Lam, H. Y. K., Pan, C., Clark, M. J., Lacroute, P., Chen, R., Haraksingh, R., O’Huallachain, M., Gerstein, M., Kidd, J. M., Bustamante, C. D., et al. (2012) · Nature Biotechnology · doi:10.1038/nbt.2134 · Cited by 128
Chahrour, M. H., Yu, T. W., Lim, E. T., Ataman, B., Coulter, M. E., Hill, R. S., Stevens, C. R., Schubert, C. R., ARRA Autism Sequencing Collaboration, Greenberg, M. E., et al. (2012) · PLoS Genetics · doi:10.1371/journal.pgen.1002635 · Cited by 125
Microindel detection in short-read sequence data
Krawitz, P. M., Rödelsperger, C., Jäger, M., Jostins, L., Bauer, S., & Robinson, P. N. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq027 · Cited by 119
Phasing of Many Thousands of Genotyped Samples
Williams, A. L., Patterson, N. J., Glessner, J., Hakonarson, H. H., & Reich, D. (2012) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2012.06.013 · Cited by 113
Self-citationState of the art de novo assembly of human genomes from massively parallel sequencing data
Li, Y., Hu, Y., Bolund, L., & Wang, J. (2010) · Human Genomics · doi:10.1186/1479-7364-4-4-271 · Cited by 110
Analysis of insertion-deletion from deep-sequencing data: software evaluation for optimal detection
Neuman, J. A., Isakov, O., & Shomron, N. (2012) · Briefings in Bioinformatics · doi:10.1093/bib/bbs013 · Cited by 107
Lyon, G. J., & Wang, K. (2012) · Genome Medicine · doi:10.1186/gm359 · Cited by 91
Self-citationLimitations of the Human Reference Genome for Personalized Genomics
Rosenfeld, J., Mason, C. E., & Smith, T. M. F. (2012) · PLoS ONE · doi:10.1371/journal.pone.0040294 · Cited by 89
Computational Techniques for Human Genome Resequencing Using Mated Gapped Reads
Carnevali, P., Baccash, J., Halpern, A. L., Nazarenko, I., Nilsen, G. B., Pant, K. P., Ebert, J. C., Brownley, A., Morenzoni, M., Karpinchyk, V., et al. (2011) · Journal of Computational Biology · doi:10.1089/cmb.2011.0201 · Cited by 87
Clement, N. L., Snell, Q. O., Clement, M., Hollenhorst, P. C., Purwar, J., Graves, B. J., Cairns, B. R., & Johnson, W. E. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp614 · Cited by 83
Self-citationThe advent of personal genome sequencing
Drmanac, R. (2011) · Genetics in Medicine · doi:10.1097/gim.0b013e31820f16e6 · Cited by 70
Lyon, G. J., Jiang, T., van Wijk, R., Wang, W., Bodily, P., Xing, J., Tian, L., Robison, R. J., Clement, M., Yang, L., et al. (2011) · PubMed · Cited by 63
Self-citationOlson, M. V. (2012) · Annual Review of Genomics and Human Genetics · doi:10.1146/annurev-genom-090711-163825 · Cited by 63
Heinrich, V., Stange, J., Dickhaus, T., Imkeller, P., Krüger, U., Bauer, S., Mundlos, S., Robinson, P. N., Hecht, J., & Krawitz, P. M. (2012) · Nucleic Acids Research · doi:10.1093/nar/gkr1073 · Cited by 43
Cited by
WGS500 Consortium, Rimmer, A., Phan, H., Mathieson, I., Iqbal, Z., Twigg, S. R. F., Wilkie, A. O. M., McVean, G., & Lunter, G. (2014) · Nature Genetics · doi:10.1038/ng.3036 · Cited by 1,057
Toward better understanding of artifacts in variant calling from high-coverage samples
Li, H. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu356 · Cited by 892
Integrating human sequence data sets provides a resource of benchmark SNP and indel genotype calls
Zook, J. M., Chapman, B., Wang, J., Mittelman, D., Hofmann, O., Hide, W., & Salit, M. (2014) · Nature Biotechnology · doi:10.1038/nbt.2835 · Cited by 814
A spectral approach integrating functional genomic annotations for coding and noncoding variants
Ionita-Laza, I., McCallum, K., Xu, B., & Buxbaum, J. D. (2016) · Nature Genetics · doi:10.1038/ng.3477 · Cited by 607
Jiang, Y.-H., Yuen, R. K. C., Jin, X., Wang, M., Chen, N., Wu, X., Ju, J., Mei, J., Shi, Y., He, M., et al. (2013) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2013.06.012 · Cited by 474
Metadata
- Authors
- O'Rawe, J., Jiang, T., Sun, G., Wu, Y., Wang, W., Hu, J., Bodily, P., Tian, L., Hakonarson, H., Johnson, W. E., Wei, Z., Wang, K., & Lyon, G. J.
- Venue
- Genome Medicine
- Published
- 2013-03-27
- Volume
- 5
- Issue
- 3
- Pages
- 28-28
- ISSN
- 1756-994X (ISSN-L)
- OpenAlex
- W2124465358
- Cited by
- 442
- References
- 60
- FWCI
- 33.49
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.