Learning the histone codes with large genomic windows and three-dimensional chromatin interactions using transformer
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
The quantitative characterization of the transcriptional control by histone modifications has been challenged by many computational studies, but most of them only focus on narrow and linear genomic regions around promoters, leaving a room for improvement. We present Chromoformer, a transformer-based, three-dimensional chromatin conformation-aware deep learning architecture that achieves the state-of-the-art performance in the quantitative deciphering of the histone codes in gene regulation. The core essence of Chromoformer architecture lies in the three variants of attention operation, each specialized to model individual hierarchy of transcriptional regulation involving from core promoters to distal elements in contact with promoters through three-dimensional chromatin interactions. In-depth interpretation of Chromoformer reveals that it adaptively utilizes the long-range dependencies between histone modifications associated with transcription initiation and elongation. We also show that the quantitative kinetics of transcription factories and Polycomb group bodies can be captured by Chromoformer. Together, our study highlights the great advantage of attention-based deep modeling of complex interactions in epigenomes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280 · cited 2×
“Meanwhile, a deep learning model architecture named transformer, which was originallydeveloped for natural language processing 13 , has been exhibiting great potential for understanding the latent grammar of DNA sequences 14 , amino acid sequences 15 , and even their alignments 16 .”
BackgroundPredicting the sequence specificities of DNA- and RNA-binding proteins by deep learning
Alipanahi, B., Delong, A., Weirauch, M. T., & Frey, B. J. (2015) · Nature Biotechnology · doi:10.1038/nbt.3300 · Cited by 2,873
A Structured Self-attentive Sentence Embedding
Lin, Z., Feng, M., Santos, C. N. D., Yu, M., Xiang, B., Zhou, B., & Bengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.03130 · Cited by 1,743 · cited 1×
“Meanwhile, combining regulatory embeddings at different resolutions using self-attention operation 13,18 – 20 did not seem to have signi fi cant advantages over concatenation, so we decided to concatenate regulatory embeddings for the sake of the simplicity of the model (Supplementary Fig.”
MethodTranscriptional enhancers: from properties to genome-wide predictions
Shlyueva, D., Stampfel, G., & Stark, A. (2014) · Nature Reviews Genetics · doi:10.1038/nrg3682 · Cited by 1,356
Effective gene expression prediction from sequence by integrating long-range interactions
Avsec, Ž., Agarwal, V., Visentin, D., Ledsam, J. R., Grabska-Barwinska, A., Taylor, K. R., Assael, Y., Jumper, J., Kohli, P., & Kelley, D. R. (2021) · Nature Methods · doi:10.1038/s41592-021-01252-x · Cited by 1,326 · cited 1×
“Regarding the former case (rescuing false-negative interactions), we agree that using TAD dramatically reduces the search space of plausible promoter-pCRE interactions and helps imputing the missing interactions in the pcHi-C data, but we thought that exploiting it directly to Chromoformer training will be hard since…”
Ji, Y., Zhou, Z., Liu, H., & Davuluri, R. V. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab083 · Cited by 1,303
Genome-Wide Quantitative Enhancer Activity Maps Identified by STARR-seq
Arnold, C. D., Gerlach, D., Stelzer, C., Boryń, Ł. M., Rath, M., & Stark, A. (2013) · Science · doi:10.1126/science.1232542 · Cited by 1,125 · cited 1×
“To date, the collective effect of distal cis -regulatory elements on gene expression remains incompletely understood, but nevertheless, the pioneering works exploiting modern technologies such as STARR-seq 37 or CRISPRi-FlowFISH 38 certainly provide us with deep insights about their dynamics.”
Background
Research works printed but not linked: 31
- Jenuwein, T. & Allis, C. D. Translating the histone code. Science 293, 1074–1080 (2001). · DOI · Paper page
- Singh, R., Lanchantin, J., Robins, G. & Qi, Y. DeepChrome: deep-learning for predicting gene expression from histone modifications. Bioinformatics 32, i639–i648 (2016). · DOI · Paper page
- Singh, R., Lanchantin, J., Sekhon, A. & Qi, Y. Attend and predict: understanding gene regulation by selective attention on chromatin. Adv. Neural Inf. Process Syst. 30, 6785–6795 (2017).
- Cho, K. et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 1724–1734 (2014). · DOI · Paper page
- Harmston, N. & Lenhard, B. Chromatin and epigenetic features of long-range gene regulation. Nucleic Acids Res. 41, 7185–7199 (2013). · DOI · Paper page
- Lieberman-Aiden, E. et al. Comprehensive mapping of long-range interactions reveals folding principles of the human genome. Science 326, 289–293 (2009). · DOI · Paper page
- Bigness, J., Loinaz, X., Patel, S., Larschan, E. & Singh, R. Integrating long-range regulatory interactions to predict gene expression using graph convolutional networks. J. Computational Biol. 29, 409–424 (2022). · DOI
- Vaswani, A. et al. Attention is all you need. In Advances in neural information processing systems (2017).
- Rao, R. et al. MSA transformer. In International Conference on Machine Learning. 8844–8856 (2021).
- Bahdanau, D., Cho, K. Bengio, Y. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations (2015).
- Luong, M.-T., Pham, H. Manning C. D. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 1412–1421 (2015). · DOI · Paper page
- Sekhon, A., Singh, R. & Qi, Y. DeepDiff: DEEP-learning for predicting DIFFerential gene expression from histone modifications. Bioinformatics 34, i891–i900 (2018). · DOI · Paper page
- Kang, M., Lee, S., Lee, D. & Kim, S. Learning cell-type-specific gene regulation mechanisms by multi-attention based deep learning with regulatory latent space. Front Genet 11, 869 (2020). · DOI
- Roadmap Epigenomics Consortium. et al. Integrative analysis of 111 reference human epigenomes. Nature 518, 317–330 (2015). · DOI · Paper page
- Encode Project Consortium. An integrated encyclopedia of DNA elements in the human genome. Nature 489, 57–74 (2012). · DOI · Paper page
- Yoh, S. M., Lucas, J. S. & Jones, K. A. The Iws1:Spt6:CTD complex controls cotranscriptional mRNA biosynthesis and HYPB/Setd2-mediated histone H3K36 methylation. Genes Dev. 22, 3422–3434 (2008). · DOI · Paper page
- Dong, X. et al. Modeling gene expression using chromatin features in various cellular contexts. Genome Biol. 13, 1–17 (2012). · DOI · Paper page
- Sutherland, H. & Bickmore, W. A. Transcription factories: gene expression in unions? Nat. Rev. Genet 10, 457–466 (2009). · DOI · Paper page
- Yang, Y. M. et al. Gα12 overexpressed in hepatocellular carcinoma reduces microRNA-122 expression via HNF4alpha inactivation, which causes c-Met induction. Oncotarget 6, 19055–19069 (2015). · DOI
- Xu, C. et al. RNA-binding protein 39: a promising therapeutic target for cancer. Cell Death Disco. 7, 214 (2021). · DOI
- Jung, D. J., Na, S. Y., Na, D. S. & Lee, J. W. Molecular cloning and characterization of CAPER, a novel coactivator of activating protein-1 and estrogen receptors. J. Biol. Chem. 277, 1229–1234 (2002). · DOI · Paper page
- Wang, X. J., Li, F. F., Zhang, Y. J., Jiang, M. & Ren, W. H. TRIB3 promotes hepatocellular carcinoma growth and predicts poor prognosis. Cancer Biomark. 29, 307–315 (2020). · doi:10.3233/cbm-201577
- Jia, Q., Dong, Q. & Qin, L. CCN: core regulatory proteins in the microenvironment that affect the metastasis of hepatocellular carcinoma? Oncotarget 7, 1203–1214 (2016). · DOI
- Kahn, T. G. et al. Interdependence of PRC1 and PRC2 for recruitment to Polycomb Response Elements. Nucleic Acids Res 44, 10132–10149 (2016).
- Dixon, J. R. et al. Topological domains in mammalian genomes identified by analysis of chromatin interactions. Nature 485, 376–380 (2012). · DOI · Paper page
- Tarasov, A., Vilella, A. J., Cuppen, E., Nijman, I. J. & Prins, P. Sambamba: fast processing of NGS alignment formats. Bioinformatics 31, 2032–2034 (2015). · DOI · Paper page
- Quinlan, A. R. & Hall, I. M. BEDTools: a flexible suite of utilities for comparing genomic features. Bioinformatics 26, 841–842 (2010). · DOI · Paper page
- Paszke, A. et al. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems (2019).
- Loshchilov, I. Hutter, F. Decoupled weight decay regularization. In International Conference on Learning Representations (2019).
- Kuleshov, M. V. et al. Enrichr: a comprehensive gene set enrichment analysis web server 2016 update. Nucleic Acids Res 44, W90–W97 (2016). · DOI · Paper page
- Li, H. & Durbin, R. Fast and accurate short read alignment with Burrows-Wheeler transform. Bioinformatics 25, 1754–1760 (2009). · DOI · Paper page
Cited by
Generative Artificial Intelligence in Anatomic Pathology
Brodsky, V., Ullah, E., Bychkov, A., Song, A. H., Walk, E. E., Louis, P., Rasool, G., Singh, R. S., Mahmood, F., Bui, M. M., et al. (2025) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2024-0215-ra · Cited by 48
The Dark Side of Dataset Scaling: Evaluating Racial Classification in Multimodal Models
Birhane, A., Dehdashtian, S., Prabhu, V., & Boddeti, V. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658968 · Cited by 28
CREaTor: zero-shot cis-regulatory pattern modeling with attention mechanisms
Li, Y., Ju, F., Chen, Z., Qu, Y., Xia, H., He, L., Wu, L., Zhu, J., Shao, B., & Deng, P. (2023) · Genome Biology · doi:10.1186/s13059-023-03103-8 · Cited by 12
“Three-dimensional (3D) chromatin folding allows physical interactions between distal cCRE and genes and the information can also guide gene regulation modeling [13, 28, 42].”
BackgroundMurphy, A. E., Askarova, A., Lenhard, B., Skene, N. G., & Marzi, S. J. (2025) · Nucleic Acids Research · doi:10.1093/nar/gkae1212 · Cited by 10
Metadata
- Authors
- Lee, D., Yang, J., & Kim, S.
- Venue
- Nature Communications
- Published
- 2022-11-05
- Volume
- 13
- Issue
- 1
- Pages
- 6678-6678
- ISSN
- 2041-1723 (ISSN-L)
- OpenAlex
- W4308239184
- Cited by
- 38
- References
- 20
- FWCI
- 2.28
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Learning the histone codes with large genomic windows and three-dimensional chromatin interactions using transformer. https://app.cochat.ai/papers/10.1038/s41467-022-34152-5
- The paper
- Lee, D., Yang, J., & Kim, S. (2022). Learning the histone codes with large genomic windows and three-dimensional chromatin interactions using transformer. Nature Communications. https://doi.org/10.1038/s41467-022-34152-5
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.