The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
CpG Transformer for imputation of single-cell methylomes
Open accessPreprint — published as 10.1093/bioinformatics/btab746 · Paper page
Abstract
A bstract: Motivation: The adoption of current single-cell DNA methylation sequencing protocols is hindered by incomplete coverage, outlining the need for effective imputation techniques. The task of imputing single-cell (methylation) data requires models to build an understanding of underlying biological processes. Results: We adapt the transformer neural network architecture to operate on methylation matrices through combining axial attention with sliding window self-attention. The obtained CpG Transformer displays state-of-the-art performances on a wide range of scBS-seq and scRRBS-seq datasets. Further-more, we demonstrate the interpretability of CpG Transformer and illustrate its rapid transfer learning properties, allowing practitioners to train models on new datasets with a limited computational and time budget. Availability and Implementation: CpG Transformer is freely available at https://github.com/gdewael/cpg-transformer .
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Flagged first, then by citations
Research works printed but not linked: 40
- 10.1016/0092-8674(88)90479-5 · DOI
- 10.1101/gad.947102 · DOI · Paper page
- 2012, Dna methylome analysis using short bisulfite sequencing data, Nature methods, 9, 145, 10.1038/nmeth.1828 · DOI
- 2014, Single-cell genome-wide bisulfite sequencing for assessing epigenetic heterogeneity, Nature methods, 11, 817, 10.1038/nmeth.3035 · DOI · Paper page
- 10.1101/gr.161679.113 · DOI · Paper page
- 2016, Parallel single-cell sequencing links transcriptional and epigenetic heterogeneity, Nature methods, 13, 229, 10.1038/nmeth.3728 · DOI · Paper page
- 2016, Dna methylation dynamics of human hematopoietic stem cell differentiation, Cell stem cell, 19, 808, 10.1016/j.stem.2016.10.019 · DOI
- 10.1186/s12859-020-3475-0 · DOI
- 2019, Missing value estimation methods for dna methylation data, Bioinformatics, 35, 3786, 10.1093/bioinformatics/btz134 · DOI
- 10.1186/s13059-014-0572-2 · DOI
- Yeping Lina Qiu , Hong Zheng , and Olivier Gevaert . A deep learning framework for imputing missing values in genomic data. bioRxiv, page 406066, 2018.
- 10.1186/s12864-017-4368-0 · DOI
- 10.1186/s12859-020-3475-0 · DOI
- 10.1186/s13059-019-1727-y · DOI · Paper page
- 10.1186/s13059-016-1145-3 · DOI
- 10.1186/s12864-019-5556-x · DOI
- 2018, Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint
- 10.1038/nrg1272 · DOI · Paper page
- 2017, Attention is all you need, arXiv preprint
- Jim Clauwaert and Willem Waegeman . Novel transformer networks for improved sequence labeling in genomics. bioRxiv, page 836163, 2020. · DOI
- Alexander Rives , Joshua Meier , Tom Sercu , Siddharth Goyal , Zeming Lin , Jason Liu , Demi Guo , Myle Ott , C Lawrence Zitnick , Jerry Ma , et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences, 118(15), 2021. · DOI · Paper page
- 2020, Prottrans: Towards cracking the language of life’s code through self-supervised deep learning and high performance computing, arXiv preprint
- Roshan Rao , Jason Liu , Robert Verkuil , Joshua Meier , John F Canny , Pieter Abbeel , Tom Sercu , and Alexander Rives. Msa transformer. bioRxiv, 2021.
- 2019, Axial attention in multidimensional transformers, arXiv preprint
- 2020, Longformer: The long-document transformer, arXiv preprint
- Xiangnan He , Lizi Liao , Hanwang Zhang , Liqiang Nie , Xia Hu , and Tat-Seng Chua . Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web, pages 173–182, 2017.
- 10.1038/nature06745 · DOI
- 2020, Big bird: Transformers for longer sequences, arXiv preprint
- 2019, Transformer-xl: Attentive language models beyond a fixed-length context, arXiv preprint
- Kaiming He , Xiangyu Zhang , Shaoqing Ren , and Jian Sun . Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
- 2016, Layer normalization, arXiv preprint
- 10.1038/cr.2016.23 · DOI · Paper page
- 10.1158/2643-3230.BCD-19-0058 · DOI
- 2014, Adam: A method for stochastic optimization, arXiv preprint
- 10.1038/nrg2341 · DOI · Paper page
- 2020, The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?, arXiv preprint
- Mukund Sundararajan , Ankur Taly , and Qiqi Yan . Axiomatic attribution for deep networks. In International Conference on Machine Learning, pages 3319–3328. PMLR, 2017.
- 10.2307/2286407 · DOI
- 2017, Collaborations between cpg sites in dna methylation, International Journal of Modern Physics B, 31, 1750243, 10.1142/S0217979217502435 · DOI
- 2021, Efficient content-based sparse attention with routing transformers, Transactions of the Association for Computational Linguistics, 9, 53, 10.1162/tacl_a_00353 · DOI · Paper page
Metadata
- Authors
- De Waele, G., Clauwaert, J., Menschaert, G., & Waegeman, W.
- Affiliations
- Ghent University (BE)
- Corresponding author
- Waegeman, Willem
- Venue
- Bioinformatics
- Publisher
- openRxiv
- Published
- 2021-06-09
- Volume
- 38
- Issue
- 3
- Pages
- 597-603
- ISSN
- 1367-4803 (ISSN-L)
- PMC
- PMC8756163
- OpenAlex
- W3210982033
- Cited by
- 37
- References
- 1
- Citations by year
- 2026: 8 · 2025: 8 · 2024: 5 · 2023: 10 · 2022: 6
- FWCI
- 1.65
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Bioinformatics (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Ghent University Academic Bibliography (Ghent University) (submittedVersion, cc-by)
- Funders
- Universiteit Gent (BOFGOA2020000703)
- Vlaamse regering
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Epigenetics and DNA Methylation
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Chromatin Dynamics
- Keywords
- single-cell DNA methylation, transformer models, axial attention, interpretability, transfer learning
- MeSH terms
- Epigenome, Base Sequence, Neural Networks, Computer, Sequence Analysis, DNA, DNA Methylation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
