The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
An open resource for accurately benchmarking small variant and reference calls
Open accessAbstract
Benchmark small variant calls are required for developing, optimizing and assessing the performance of sequencing and bioinformatics methods. Here, as part of the Genome in a Bottle (GIAB) Consortium, we apply a reproducible, cloud-based pipeline to integrate multiple short- and linked-read sequencing datasets and provide benchmark calls for human genomes. We generate benchmark calls for one previously analyzed GIAB sample, as well as six genomes from the Personal Genome Project. These new genomes have broad, open consent, making this a ‘first of its kind’ resource that is available to the community for multiple downstream applications. We produce 17% more benchmark single nucleotide variations, 176% more indels and 12% larger benchmark regions than previously published GIAB benchmarks. We demonstrate that this benchmark reliably identifies errors in existing callsets and highlight challenges in interpreting performance metrics when using benchmarks that are not perfect or comprehensive. Finally, we identify strengths and weaknesses of callsets by stratifying performance according to variant type and genome context. Genome in a Bottle Consortium presents a high-confidence dataset for benchmarking small variant calls in human genomes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Genome Analysis Toolkit: A MapReduce framework for analyzing next-generation DNA sequencing data
McKenna, A., Hanna, M., Banks, E., Sivachenko, A., Cibulskis, K., Kernytsky, A., Garimella, K., Altshuler, D., Gabriel, S., Daly, M., et al. (2010) · Genome Research · doi:10.1101/gr.107524.110 · Cited by 27,895
A global reference for human genetic variation
The 1000 Genomes Project Consortium, authors, C., Auton, A., Abecasis, G. R., Steering committee, Altshuler, D. M., Durbin, R. M., Abecasis, G. R., Bentley, D. R., Chakravarti, A., et al. (2015) · Nature · doi:10.1038/nature15393 · Cited by 18,247
Self-citationA framework for variation discovery and genotyping using next-generation DNA sequencing data
DePristo, M. A., Banks, E., Poplin, R., Garimella, K. V., Maguire, J. R., Hartl, C., Philippakis, A. A., del Angel, G., Rivas, M. A., Hanna, M., et al. (2011) · Nature Genetics · doi:10.1038/ng.806 · Cited by 11,456
Wenger, A. M., Peluso, P., Rowell, W. J., Chang, P.-C., Hall, R. J., Concepcion, G. T., Ebler, J., Fungtammasan, A., Kolesnikov, A., Olson, N. D., et al. (2019) · Nature Biotechnology · doi:10.1038/s41587-019-0217-9 · Cited by 1,841
Self-citationA universal SNP and small-indel variant caller using deep neural networks
Poplin, R., Chang, P.-C., Alexander, D., Schwartz, S., Colthurst, T., Ku, A., Newburger, D., Dijamco, J., Nguyen, N., Afshar, P. T., et al. (2018) · Nature Biotechnology · doi:10.1038/nbt.4235 · Cited by 1,840
Self-citationHuman Genome Sequencing Using Unchained Base Reads on Self-Assembling DNA Nanoarrays
Drmanac, R., Sparks, A. B., Callow, M. J., Halpern, A. L., Burns, N. L., Kermani, B. G., Carnevali, P., Nazarenko, I., Nilsen, G. B., Yeung, G., et al. (2010) · Science · doi:10.1126/science.1181498 · Cited by 1,212
The Association between Job Strain and Atrial Fibrillation: Results from the Swedish WOLF Study
Fransson, E. I., Stadin, M., Nordin, M., Malm, D., Knutsson, A., Alfredsson, L., & Westerholm, P. J. M. (2015) · BioMed Research International · doi:10.1155/2015/371905 · Cited by 914
Integrating human sequence data sets provides a resource of benchmark SNP and indel genotype calls
Zook, J. M., Chapman, B., Wang, J., Mittelman, D., Hofmann, O., Hide, W., & Salit, M. (2014) · Nature Biotechnology · doi:10.1038/nbt.2835 · Cited by 814
Self-citationExtensive sequencing of seven human genomes to characterize benchmark reference materials
Zook, J. M., Catoe, D., McDaniel, J., Vang, L., Spies, N., Sidow, A., Weng, Z., Liu, Y., Mason, C. E., Alexander, N., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.25 · Cited by 811
Self-citationVariation in genome-wide mutation rates within and between human families
the 1000 Genomes Project (2011) · Nature Genetics · doi:10.1038/ng.862 · Cited by 581
Cited by
Haplotype-resolved de novo assembly using phased assembly graphs with hifiasm
Cheng, H., Concepcion, G. T., Feng, X., Zhang, H., & Li, H. (2021) · Nature Methods · doi:10.1038/s41592-020-01056-5 · Cited by 5,859
Wenger, A. M., Peluso, P., Rowell, W. J., Chang, P.-C., Hall, R. J., Concepcion, G. T., Ebler, J., Fungtammasan, A., Kolesnikov, A., Olson, N. D., et al. (2019) · Nature Biotechnology · doi:10.1038/s41587-019-0217-9 · Cited by 1,841
Self-citationByrska-Bishop, M., Evani, U. S., Zhao, X., Basile, A. O., Abel, H. J., Regier, A. A., Corvelo, A., Clarke, W. E., Musunuri, R., Nagulapalli, K., et al. (2022) · Cell · doi:10.1016/j.cell.2022.08.004 · Cited by 1,094
Self-citationGenomic data in the all of us research program
The All of Us Research Program Genomics Investigators, Manuscript Writing Group, Bick, A. G., Metcalf, G. A., Mayo, K. R., Lichtenstein, L., Rura, S., Carroll, R. J., Musick, A., Linder, J. E., et al. (2024) · Nature · doi:10.1038/s41586-023-06957-x · Cited by 829
Nurk, S., Walenz, B. P., Rhie, A., Vollger, M. R., Logsdon, G. A., Grothe, R., Miga, K. H., Eichler, E. E., Phillippy, A. M., & Koren, S. (2020) · Genome Research · doi:10.1101/gr.263566.120 · Cited by 741
Metadata
- Authors
- Zook, J. M., McDaniel, J., Olson, N. D., Wagner, J., Parikh, H., Heaton, H., Irvine, S. A., Trigg, L., Truty, R., McLean, C. Y., De La Vega, F. M., Xiao, C., Sherry, S., & Salit, M.
- Venue
- Nature Biotechnology
- Published
- 2019-04-01
- Volume
- 37
- Issue
- 5
- Pages
- 561-566
- ISSN
- 0733-222X (ISSN-L)
- 1087-0156
- 1546-1696
- OpenAlex
- W2929354011
- Cited by
- 399
- References
- 32
- FWCI
- 16.61
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: An open resource for accurately benchmarking small variant and reference calls. https://app.cochat.ai/papers/10.1038/s41587-019-0074-6
- The paper
- Zook, J. M., McDaniel, J., Olson, N. D., Wagner, J., Parikh, H., Heaton, H., Irvine, S. A., Trigg, L., Truty, R., McLean, C. Y., De La Vega, F. M., Xiao, C., Sherry, S., & Salit, M. (2019). An open resource for accurately benchmarking small variant and reference calls. Nature Biotechnology. https://doi.org/10.1038/s41587-019-0074-6
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.