Cooler: scalable storage for Hi-C data and other genomically labeled arrays
Open accessPublished version of a preprint
Abstract
Motivation: Most existing coverage-based (epi)genomic datasets are one-dimensional, but newer technologies probing interactions (physical, genetic, etc.) produce quantitative maps with two-dimensional genomic coordinate systems. Storage and computational costs mount sharply with data resolution when such maps are stored in dense form. Hence, there is a pressing need to develop data storage strategies that handle the full range of useful resolutions in multidimensional genomic datasets by taking advantage of their sparse nature, while supporting efficient compression and providing fast random access to facilitate development of scalable algorithms for data analysis. Results: We developed a file format called cooler, based on a sparse data model, that can support genomically labeled matrices at any resolution. It has the flexibility to accommodate various descriptions of the data axes (genomic coordinates, tracks and bin annotations), resolutions, data density patterns and metadata. Cooler is based on HDF5 and is supported by a Python library and command line suite to create, read, inspect and manipulate cooler data collections. The format has been adopted as a standard by the NIH 4D Nucleome Consortium. Availability and implementation: Cooler is cross-platform, BSD-licensed and can be installed from the Python package index or the bioconda repository. The source code is maintained on Github at https://github.com/mirnylab/cooler. Supplementary information: Supplementary data are available at Bioinformatics online.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Hi-C data was mapped to mm10 and converted to .cool format considering only read pairs with both sides mapping with high confidence (mapq ≥30).
“Data was mapped to mm10 using the distiller-nf pipeline (https://github.com/open2c/distiller-nf), iteratively corrected and converted into .cool format with cooler (85), only considering read pairs with both sides mapping with high confidence (mapq ≥30).”
From Synergy between regulatory elements can render cohesin dispensable for distal enhancer function (2026) · cites this paper as [85]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A global reference for human genetic variation
The 1000 Genomes Project Consortium, authors, C., Auton, A., Abecasis, G. R., Steering committee, Altshuler, D. M., Durbin, R. M., Abecasis, G. R., Bentley, D. R., Chakravarti, A., et al. (2015) · Nature · doi:10.1038/nature15393 · Cited by 18,247 · see citing works
Integrative analysis of 111 reference human epigenomes
Roadmap Epigenomics Consortium, Kundaje, A., Meuleman, W., Ernst, J., Bilenky, M., Yen, A., Heravi-Moussavi, A., Kheradpour, P., Zhang, Z., Wang, J., et al. (2015) · Nature · doi:10.1038/nature14248 · Cited by 6,368 · see citing works
Iterative correction of Hi-C data reveals hallmarks of chromosome organization
Imakaev, M., Fudenberg, G., McCord, R. P., Naumova, N., Goloborodko, A., Lajoie, B. R., Dekker, J., & Mirny, L. A. (2012) · Nature Methods · doi:10.1038/nmeth.2148 · Cited by 1,423 · see citing works
Flagged first, then by citations
Research works printed but not linked: 24
- Job Dekker, Karsten Rippe, Martijn Dekker, and Nancy Kleckner. Capturing chromosome conformation. Science, 295(5558):1306-1311, 2002.
- Annette Denker and Wouter De Laat. The second decade of 3C technologies: detailed insights into nuclear organization. Genes & development, 30(12):1357-1382, 2016.
- Erez Lieberman-Aiden, Nynke L Van Berkum, Louise Williams, Maxim Imakaev, Tobias Ragoczy, Agnes Telling, Ido Amit, Bryan R Lajoie, Peter J Sabo, Michael O Dorschner, et al. Comprehensive mapping of long-range interactions reveals folding principles of the human genome. Science, 326(5950):289-293, 2009.
- James OJ Davies, A Marieke Oudelaar, Douglas R Higgs, and Jim R Hughes. How best to identify chromosomal interactions: a comparison of approaches. Nature methods, 14(2): 125, 2017.
- Heng Li. Tabix: fast retrieval of sequence features from generic tab-delimited files. Bioinfor- matics, 27(5):718-719, 2011.
- Yanli Wang, Bo Zhang, Lijun Zhang, Lin An, Jie Xu, Daofeng Li, Mayank NK Choudhary, Yun Li, Ming Hu, Ross Hardison, et al. The 3D Genome Browser: a web-based browser for visualizing 3D genome organization and long-range chromatin interactions. BioRxiv, page 112268, 2017.
- Neva C Durand, James T Robinson, Muhammad S Shamim, Ido Machol, Jill P Mesirov, Eric S Lander, and Erez Lieberman Aiden. Juicebox provides a visualization system for hi-c contact maps with unlimited zoom. Cell systems, 3(1):99-101, 2016.
- Michael EG Sauria, Jennifer E Phillips-Cremins, Victor G Corces, and James Taylor. HiFive: a tool suite for easy and efficient HiC and 5C data analysis. Genome Biology, 16(1):237, 2015.
- Rajendra Kumar, Haitham Sobhy, Per Stenberg, and Ludvig Lizana. Genome contact map explorer: a platform for the comparison, interactive visualization and analysis of genome contact maps. Nucleic Acids Research, 45(17):e152-e152, 2017.
- Daniel J. Abadi, Samuel R. Madden, and Nabil Hachem. Column-stores vs. row-stores: How different are they really? In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, SIGMOD '08, pages 967-980, New York, NY, USA, 2008. ACM. ISBN 978-1-60558-102-6. doi: 10.1145/1376616.1376712. · DOI
- Peter Kerpedjiev, Nezar Abdennur, Fritz Lekschas, Chuck McCallum, Kasper Dinkla, Hen- drik Strobelt, Jacob M. Luber, Scott B. Ouellette, Alaleh Azhir, Nikhil Kumar, Jeewon Hwang, Soohyun Lee, Burak H. Alver, Hanspeter Pfister, Leonid A. Mirny, Peter J. Park, and Nils Gehlenborg. HiGlass: web-based visual exploration and analysis of genome interaction maps. Genome Biology, 19(1):125, Aug 2018. ISSN 1474-760X. doi: 10.1186/s13059-018-1486-1. · DOI
- Suhas S P Rao, Miriam H Huntley, Neva C Durand, Elena K Stamenova, Ivan D Bochkov, James T Robinson, Adrian L Sanborn, Ido Machol, Arina D Omer, Eric S Lander, and Erez Lieberman Aiden. A 3D map of the human genome at kilobase resolution reveals principles of chromatin looping. Cell, 159(7):1665-1680, December 2014.
- ENCODE Project Consortium et al. An integrated encyclopedia of dna elements in the human genome. Nature, 489(7414):57, 2012.
- Andrew Collette. Python and HDF5. O'Reilly, 2013.
- Daofeng Li, Silas Hsu, Deepak Purushotham, and Ting Wang. Chromatin interaction data visualization in the WashU Epigenome Browser. bioRxiv, page 239368, 2017.
- Kadir Caner Akdemir and Lynda Chin. HiCPlotter integrates genomic data with interaction matrices. Genome Biology, 16(1):198, 2015.
- Thomas Kluyver, Benjamin Ragan-Kelley, Fernando Pérez, Brian Granger, Matthias Bus- sonnier, Jonathan Frederic, Kyle Kelley, Jessica Hamrick, Jason Grout, Sylvain Corlay, Paul Ivanov, Damián Avila, Safia Abdalla, and Carol Willing. Jupyter notebooks -a publishing format for reproducible computational workflows. In F. Loizides and B. Schmidt, editors, Positioning and Power in Academic Publishing: Players, Agents and Agendas, pages 87 - 90. IOS Press, 2016.
- Björn Grüning, Ryan Dale, Andreas Sjödin, Brad A Chapman, Jillian Rowe, Christopher H Tomkins-Tinch, Renan Valieris, Johannes Köster, and Team Bioconda. Bioconda: sustain- able and comprehensive software distribution for the life sciences. Nature methods, 15(7): 475, 2018.
- Felipe da Veiga Leprevost, Björn A Grüning, Saulo Alves Aflitos, Hannes L Röst, Julian Uszkoreit, Harald Barsnes, Marc Vaudel, Pablo Moreno, Laurent Gatto, Jonas Weber, et al. BioContainers: an open-source and community-driven framework for software standardiza- tion. Bioinformatics, 33(16):2580-2582, 2017.
- Matthew T Dougherty, Michael J Folk, Erez Zadok, Herbert J Bernstein, Frances C Bern- stein, Kevin W Eliceiri, Werner Benger, and Christoph Best. Unifying biological image for- mats with HDF5. Queue, 7(9):20, 2009.
- Mike Folk, Gerd Heber, Quincey Koziol, Elena Pourmal, and Dana Robinson. An overview of the HDF5 technology suite and its applications. In Proceedings of the EDBT/ICDT 2011 Workshop on Array Databases, pages 36-47. ACM, 2011.
- Nicolas Servant, Nelle Varoquaux, Bryan R Lajoie, Eric Viara, Chong-Jian Chen, Jean- Philippe Vert, Edith Heard, Job Dekker, and Emmanuel Barillot. HiC-Pro: an optimized and flexible pipeline for Hi-C data processing. Genome Biology, 16(1):259, 2015.
- Valerio Pascucci and Randall J Frank. Hierarchical indexing for out-of-core access to multi- resolution data. In Hierarchical and Geometrical Methods in Scientific Visualization, pages 225-241. Springer, 2003.
- Jeffrey Scott Vitter. Algorithms and data structures for external memory. Foundations and Trends® in Theoretical Computer Science, 2(4):305-474, 2008. ISSN 1551-305X. doi: 10.1561/0400000014. · DOI
Cited by
Sorted by citations
Heterochromatin drives compartmentalization of inverted and conventional nuclei
Falk, M., Feodorova, Y., Naumova, N., Imakaev, M., Lajoie, B. R., Leonhardt, H., Joffe, B., Dekker, J., Fudenberg, G., Solovei, I., et al. (2019) · Nature · doi:10.1038/s41586-019-1275-3 · Cited by 646 · see citing works
Loss of epigenetic information as a cause of mammalian aging
Yang, J.-H., Hayano, M., Griffin, P. T., Amorim, J. A., Bonkowski, M. S., Apostolides, J. K., Salfati, E. L., Blanchette, M., Munding, E. M., Bhakta, M., et al. (2023) · Cell · doi:10.1016/j.cell.2022.12.027 · Cited by 643 · see citing works
Single-cell CUT&Tag profiles histone modifications and transcription factors in complex tissues
Bartosovic, M., Kabbe, M., & Castelo-Branco, G. (2021) · Nature Biotechnology · doi:10.1038/s41587-021-00869-9 · Cited by 483 · see citing works
Ancient gene linkages support ctenophores as sister to other animals
Schultz, D. T., Haddock, S. H. D., Bredeson, J. V., Green, R. E., Simakov, O., & Rokhsar, D. S. (2023) · Nature · doi:10.1038/s41586-023-05936-6 · Cited by 362 · see citing works
Hsieh, T.-H. S., Cattoglio, C., Slobodyanyuk, E., Hansen, A. S., Darzacq, X., & Tjian, R. (2022) · Nature Genetics · doi:10.1038/s41588-022-01223-8 · Cited by 406 · see citing works
Simultaneous profiling of 3D genome structure and DNA methylation in single human cells
Lee, D.-S., Luo, C., Zhou, J., Chandran, S., Rivkin, A., Bartlett, A., Nery, J. R., Fitzpatrick, C., O’Connor, C., Dixon, J. R., et al. (2019) · Nature Methods · doi:10.1038/s41592-019-0547-z · Cited by 340 · see citing works
Distinct Classes of Chromatin Loops Revealed by Deletion of an RNA-Binding Region in CTCF
Hansen, A. S., Hsieh, T.-H. S., Cattoglio, C., Pustova, I., Saldaña-Meyer, R., Reinberg, D., Darzacq, X., & Tjian, R. (2019) · Molecular Cell · doi:10.1016/j.molcel.2019.07.039 · Cited by 279 · see citing works
Subtype-specific 3D genome alteration in acute myeloid leukaemia
Xu, J., Song, F., Lyu, H., Kobayashi, M., Zhang, B., Zhao, Z., Hou, Y., Wang, X., Luan, Y., Jia, B., et al. (2022) · Nature · doi:10.1038/s41586-022-05365-x · Cited by 208 · see citing works
Cohesin Disrupts Polycomb-Dependent Chromosome Interactions in Embryonic Stem Cells
Rhodes, J. D. P., Feldmann, A., Hernández-Rodríguez, B., Díaz, N., Brown, J. M., Fursova, N. A., Blackledge, N. P., Prathapan, P., Dobrinic, P., Huseyin, M. K., et al. (2020) · Cell Reports · doi:10.1016/j.celrep.2019.12.057 · Cited by 187 · see citing works
Sequence-based modeling of three-dimensional genome architecture from kilobase to chromosome scale
Zhou, J. (2022) · Nature Genetics · doi:10.1038/s41588-022-01065-4 · Cited by 177 · see citing works
Chromatin alternates between A and B compartments at kilobase scale for subgenic organization
Harris, H. L., Gu, H., Olshansky, M., Wang, A., Farabella, I., Eliaz, Y., Kalluchi, A., Krishna, A., Jacobs, M., Cauer, G., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-38429-1 · Cited by 184 · see citing works
Hi-C analyses with GENOVA: a case study with cohesin variants
van der Weide, R. H., van den Brand, T., Haarhuis, J. H. I., Teunissen, H., Rowland, B. D., & de Wit, E. (2021) · NAR Genomics and Bioinformatics · doi:10.1093/nargab/lqab040 · Cited by 161 · see citing works
Wang, X., Luan, Y., & Yue, F. (2022) · Science Advances · doi:10.1126/sciadv.abn9215 · Cited by 148 · see citing works
Large-scale chromatin reorganization reactivates placenta-specific genes that drive cellular aging
Liu, Z., Ji, Q., Ren, J., Yan, P., Wu, Z., Wang, S., Sun, L., Wang, Z., Li, J., Sun, G., et al. (2022) · Developmental Cell · doi:10.1016/j.devcel.2022.05.004 · Cited by 109 · see citing works
Down-syndrome-induced senescence disrupts the nuclear architecture of neural progenitors
Meharena, H. S., Marco, A., Dileep, V., Lockshin, E. R., Akatsu, G. Y., Mullahoo, J., Watson, L. A., Ko, T., Guerin, L. N., Abdurrob, F., et al. (2022) · Cell Stem Cell · doi:10.1016/j.stem.2021.12.002 · Cited by 92 · see citing works
A chromosome-scale genome assembly and karyotype of the ctenophore <i>Hormiphora californensis</i>
Schultz, D. T., Francis, W. R., McBroome, J. D., Christianson, L. M., Haddock, S. H. D., & Green, R. E. (2021) · G3 Genes|Genomes|Genetics · doi:10.1093/g3journal/jkab302 · Cited by 35 · see citing works
Synergy between regulatory elements can render cohesin dispensable for distal enhancer function
Hansen, K. L., Adachi, A. S., Braccioli, L., Kadvani, S., Boileau, R. M., Martinovic, M., Pokorny, B., Shah, R., Anderson, E. C., Zhang, K., et al. (2026) · Science · doi:10.1126/science.adt4221 · Cited by 29 · see citing works
Genome-wide absolute quantification of chromatin looping
Jusuf, J. M., Yang, J. H., Toppen, J., Grosse-Holz, S., Gabriele, M., Mach, P., Flyamer, I. M., Zechner, C., Giorgetti, L., Mirny, L. A., et al. (2026) · Nature Structural & Molecular Biology · doi:10.1038/s41594-026-01819-2 · Cited by 22 · see citing works
Self-citationChakraborty, S., Wenzlitschke, N., Anderson, M. J., Eraso, A., Baudic, M., Thompson, J. J., Evans, A. A., Shatford-Adams, L. M., Chari, R., Awasthi, P., et al. (2025) · Developmental Cell · doi:10.1016/j.devcel.2025.02.002 · Cited by 22 · see citing works
Paldi, F., Szalay, M.-F., Dufau, S., Di Stefano, M., Reboul, H., Jost, D., Bantignies, F., & Cavalli, G. (2026) · Nature Genetics · doi:10.1038/s41588-025-02489-4 · Cited by 18 · see citing works
Choppavarapu, L., Nosirov, B., Kahn, M., & Jin, V. X. (2026) · Cancers · doi:10.3390/cancers18213416
- Authors
- Abdennur, N., & Mirny, L. A.
- Affiliations
- Institute for Medical Engineering and Science, Cambridge, MA 02139, USA
- Massachusetts Institute of Technology (US)
- Corresponding author
- Abdennur, Nezar
- Mirny, Leonid A
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2019-07-10
- Volume
- 36
- Issue
- 1
- Pages
- 311-316
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC8205516
- OpenAlex
- W2960979937
- Cited by
- 1,759
- References
- 3
- Citations by year
- 2026: 158 · 2025: 382 · 2024: 638 · 2023: 375 · 2022: 100 · 2021: 58 · 2020: 35 · 2019: 13
- FWCI
- 17.78
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- Bioinformatics (publishedVersion)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health Common Fund 4D Nucleome Program
- Center for Structure and Physics of the Genome (DK107980)
- 4D Nucleome Network Data Coordination and Integration Center (U01 CA200059)
- NIH
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Chromatin Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Keywords
- Hi-C data, HDF5, data compression, scalable data storage, Python library
- MeSH terms
- Metadata, Algorithms, Software, Information Storage and Retrieval, Genomics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
