The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Efficient storage of high throughput DNA sequencing data using reference-based compression
Open accessAbstract
Data storage costs have become an appreciable proportion of total cost in the creation and analysis of DNA sequence data. Of particular concern is that the rate of increase in DNA sequencing is significantly outstripping the rate of increase in disk storage capacity. In this paper we present a new reference-based compression method that efficiently compresses DNA sequences for storage. Our approach works for resequencing experiments that target well-studied genomes. We align new sequences to a reference genome and then encode the differences between the new sequence and the reference genome for storage. Our compression method is most efficient when we allow controlled loss of data in the saving of quality information and unaligned sequences. With this new compression method we observe exponential efficiency gains as read lengths increase, and the magnitude of this efficiency gain can be controlled by changing the amount of quality information stored. Our compression method is tunable: The storage of quality scores and unaligned sequences may be adjusted for different experiments to conserve information or to minimize storage costs, and provides one opportunity to address the threat that increasing DNA sequence volumes will overcome our ability to store the sequences.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
DNA sequencing with chain-terminating inhibitors
Sanger, F., Nicklen, S., & Coulson, A. R. (1977) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.74.12.5463 · Cited by 68,572
The Sequence Alignment/Map format and SAMtools
Li, H., Handsaker, B., Wysoker, A., Fennell, T., Ruan, J., Homer, N., Marth, G., Abecasis, G., Durbin, R., & 1000 Genome Project Data Processing Subgroup (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp352 · Cited by 62,041
Fast and accurate short read alignment with Burrows-Wheeler transform
Li, H., & Durbin, R. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp324 · Cited by 58,607
A map of human genome variation from population-scale sequencing.
The 1000 Genomes Project Consortium (2010) · Nature · doi:10.1038/nature09534 · Cited by 7,877
Genome sequencing in microfabricated high-density picolitre reactors
Margulies, M., Egholm, M., Altman, W. E., Attiya, S., Bader, J. S., Bemben, L. A., Berka, J., Braverman, M. S., Chen, Y.-J., Chen, Z., et al. (2005) · Nature · doi:10.1038/nature03959 · Cited by 7,693
Base-Calling of Automated Sequencer Traces Using Phred. I. Accuracy Assessment
Ewing, B., Hillier, L., Wendl, M. C., & Green, P. (1998) · Genome Research · doi:10.1101/gr.8.3.175 · Cited by 6,967
A Method for the Construction of Minimum-Redundancy Codes
Huffman, D. (1952) · Proceedings of the IRE · doi:10.1109/jrproc.1952.273898 · Cited by 6,334
Base-Calling of Automated Sequencer Traces Using Phred. II. Error Probabilities
Ewing, B., & Green, P. (1998) · Genome Research · doi:10.1101/gr.8.3.186 · Cited by 5,439
The ENCODE (ENCyclopedia Of DNA Elements) Project
(2004) · Science · doi:10.1126/science.1105136 · Cited by 2,390
International network of cancer genome projects
The International Cancer Genome Consortium (2010) · Nature · doi:10.1038/nature08987 · Cited by 2,299
Universal codeword sets and representations of the integers
Elias, P. (1975) · IEEE Transactions on Information Theory · doi:10.1109/tit.1975.1055349 · Cited by 1,165
Run-length encodings (Corresp.)
Golomb, S. (1966) · IEEE Transactions on Information Theory · doi:10.1109/tit.1966.1053907 · Cited by 988
The case for cloud computing in genome informatics
Stein, L. D. (2010) · Genome Biology · doi:10.1186/gb-2010-11-5-207 · Cited by 520
A method for the construction of minimum-redundancy codes
Huffman, D. A. (2006) · Resonance · doi:10.1007/bf02837279 · Cited by 426
DNACompress: fast and effective DNA sequence compression
Chen, X., Li, M., Ma, B., & Tromp, J. (2002) · Bioinformatics · doi:10.1093/bioinformatics/18.12.1696 · Cited by 209
Human genomes as email attachments
Christley, S., Lu, Y., Li, C., & Xie, X. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btn582 · Cited by 166
Attention-driven discrete sampling of motion perception
VanRullen, R., Reddy, L., & Koch, C. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0409172102 · Cited by 152
Biological sequence compression algorithms.
Matsumoto, T., Sadakane, K., & Imai, H. (2000) · PubMed · Cited by 132
Archiving next generation sequencing data
Shumway, M. F., Cochrane, G., & Sugawara, H. (2009) · Nucleic Acids Research · doi:10.1093/nar/gkp1078 · Cited by 122
Self-citationDDBJ launches a new archive database with analytical tools for next-generation sequence data
Kaminuma, E., Mashima, J., Kodama, Y., Gojobori, T., Ogasawara, O., Okubo, K., Takagi, T., & Nakamura, Y. (2009) · Nucleic Acids Research · doi:10.1093/nar/gkp847 · Cited by 109
Improvements to services at the European Nucleotide Archive
Leinonen, R., Akhtar, R. A., Birney, E., Bonfield, J., Bower, L., Corbett, M., Cheng, Y., Demiralp, F., Faruque, N., Goodgame, N., et al. (2009) · Nucleic Acids Research · doi:10.1093/nar/gkp998 · Cited by 70
Self-citationData structures and compression algorithms for high-throughput sequencing technologies
Daily, K., Rigor, P., Christley, S., Xie, X. S., & Baldi, P. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-514 · Cited by 53
Stephenson, J. (2008) · JAMA · doi:10.1001/jama.299.19.2267-d · Cited by 5
Cited by
Twelve years of SAMtools and BCFtools
Danecek, P., Bonfield, J. K., Liddle, J., Marshall, J., Ohan, V., Pollard, M. O., Whitwham, A., Keane, T., McCarthy, S. A., Davies, R. M., et al. (2021) · GigaScience · doi:10.1093/gigascience/giab008 · Cited by 15,343
Treangen, T. J., Ondov, B., Koren, S., & Phillippy, A. M. (2014) · Genome biology · doi:10.1186/preaccept-2573980311437212 · Cited by 1,654
Big Data: Astronomical or Genomical?
Stephens, Z. D., Lee, S. Y., Faghri, F., Campbell, R. H., Zhai, C., Efron, M. J., Iyer, R., Schatz, M. C., Sinha, S., & Robinson, G. E. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002195 · Cited by 1,340
Schneider, V. A., Graves-Lindsay, T., Howe, K., Bouk, N., Chen, H.-C., Kitts, P. A., Murphy, T. D., Pruitt, K. D., Thibaud-Nissen, F., Albracht, D., et al. (2017) · Genome Research · doi:10.1101/gr.213611.116 · Cited by 1,193
Ashley, E. A. (2016) · Nature Reviews Genetics · doi:10.1038/nrg.2016.86 · Cited by 1,116
Metadata
- Authors
- Hsi-Yang Fritz, M., Leinonen, R., Cochrane, G., & Birney, E.
- Venue
- Genome Research
- Publisher
- Cold Spring Harbor Laboratory
- Published
- 2011-01-18
- Volume
- 21
- Issue
- 5
- Pages
- 734-740
- ISSN
- 1088-9051 (ISSN-L)
- OpenAlex
- W2159084616
- Cited by
- 395
- References
- 23
- Type
- posted content
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.