The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Transformers in single-cell omics: a review and new perspectives
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Accurate structure prediction of biomolecular interactions with AlphaFold 3
Abramson, J., Adler, J., Dunger, J., Evans, R., Green, T., Pritzel, A., Ronneberger, O., Willmore, L., Ballard, A. J., Bambrick, J., et al. (2024) · Nature · doi:10.1038/s41586-024-07487-w · Cited by 15,861
Evolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,990
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280 · cited 1×
“Transformers have been very impactful in the processing of sequential data, for which they were originally designed 5 , such as natural language 18 , DNA 10 or protein sequences 19 .”
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Research works printed but not linked: 50
- Zhu, C., Preissl, S. & Ren, B. Single-cell multimodal omics: the power of many. Nat. Methods 17, 11–14 (2020). · DOI · Paper page
- Method of the Year 2013. Nat. Methods 11, 1 (2013). · DOI
- Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (eds Burstein, J., Doran, C. & Solorio, T.) 4171–4186 (2019).
- Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. 30, 5998–6008 (2017). This work introduced the transformer architecture, originally designed and evaluated on NLP tasks.
- Radford, A. et al. Robust speech recognition via large-scale weak supervision. PMLR (ed. Lawrence, N.) 202, 28492–28518 (2023).
- Amodio, M. et al. Exploring single-cell data with deep multitasking neural networks. Nat. Methods 16, 1139–1145 (2019). · DOI · Paper page
- Shen, H. et al. A universal approach for integrating super large-scale single-cell transcriptomes by exploring gene rankings. Brief. Bioinform. 23, bbab573 (2022). This work introduced the first gene-ranking-based single-cell transformer. It was also the first single-cell transformer pretrained on a large dataset of over 10 million cells. · DOI
- Shen, H. et al. Generative pretraining from large-scale transcriptomes for single-cell deciphering. iScience 26, 106536 (2023). · DOI
- Levine, D. et al. Cell2Sentence: teaching large language models the language of biology. In Proc. 41st International Conference on Machine Learning (eds Salakhutdinov, R. et al.) (2024). · DOI
- Wen, H. et al. CellPLM: pre-training of cell language model beyond single cells. In International Conference on Learning Representations (eds Kim, B. et al.) (2024). · DOI
- Wen, H. et al. Single cells are spatial tokens: transformers for spatial transcriptomic data imputation. Preprint at https://doi.org/10.48550/arXiv.2302.03038 (2023). · DOI
- Dao, T., Fu, D. Y., Ermon, S., Rudra, A. & Ré, C. FlashAttention: fast and memory-efficient exact attention with IO-awareness. Adv. Neural Inf. Process. Syst. 35, 16344–16359 (2024).
- Bian, H. et al. scMulan: a multitask generative pre-trained language model for single-cell analysis. In Research in Computational Molecular Biology. RECOMB 2024. Lecture Notes in Computer Science (ed. Ma, J.) Vol. 14758, 479–482 (Springer, Cham, 2024). · DOI
- Domcke, S. & Shendure, J. A reference cell tree will serve science better than a reference cell atlas. Cell 186, 1103–1114 (2023). · DOI
- Oh, G., Choi, B., Jung, I. & Ye, J. C. scHyena: foundation model for full-length single-cell RNA-seq analysis in brain. Preprint at https://doi.org/10.48550/arXiv.2310.02713 (2023). · DOI
- Tang, W. et al. Single-cell multimodal prediction via transformers. In Proc. 32nd ACM International Conference on Information and Knowledge Management 2422–2431 (2023). · DOI
- Gong, J. et al. xTrimoGene: an efficient and scalable representation learner for single-cell RNA-seq data. Adv. Neural Inform. Process. Syst. 36, 69391–69403 (2024).
- Tarashansky, A. J. et al. Mapping single-cell atlases throughout Metazoa unravels cell type evolution. eLife 10, e66747 (2021). · DOI · Paper page
- Badia-I-Mompel, P. et al. Gene regulatory network inference in the era of single-cell multi-omics. Nat. Rev. Genet. 24, 739–754 (2023). · DOI · Paper page
- Claringbould, A. & Zaugg, J. B. Enhancers in disease: molecular basis and emerging treatment strategies. Trends Mol. Med. 27, 1060–1073 (2021). · DOI · Paper page
- Liu, X. et al. Pathformer: a biological pathway informed transformer for disease diagnosis and prognosis using multi-omics data. Bioinformatics 40, btae316 (2024). · DOI
- Liu, L., Li, W., Wong, K.-C., Yang, F. & Yao, J. A pre-trained large generative model for translating single-cell transcriptome to proteome. Preprint at bioRxiv https://doi.org/10.1101/2023.07.04.547619 (2023). · DOI
- Tyler, S. R., Guccione, E. & Schadt, E. E. Erasure of biologically meaningful signal by unsupervised scRNAseq batch-correction methods. Preprint at bioRxiv https://doi.org/10.1101/2021.11.15.468733 (2023). · DOI
- Rosen, Y. et al. Universal Cell Embeddings: a foundation model for cell biology. Preprint at bioRxiv https://doi.org/10.1101/2023.11.28.568918 (2023). · DOI · Paper page
- Kingma, D. P. & Welling, M. Auto-encoding variational Bayes. In International Conference on Learning Representations (eds Bengio, Y. et al.) (2014).
- Lotfollahi, M. et al. Predicting cellular responses to complex perturbations in high-throughput screens. Mol. Syst. Biol. 19, e11517 (2023). · DOI · Paper page
- Köhler, N. D., Büttner, M., Andriamanga, N. & Theis, F. J. Deep learning does not outperform classical machine learning for cell-type annotation. Preprint at bioRxiv https://doi.org/10.1101/653907 (2021). · DOI
- Domínguez Conde, C. et al. Cross-tissue immune cell analysis reveals tissue-specific features in humans. Science 376, eabl5197 (2022). · DOI · Paper page
- Fischer, F., Fischer, D. S., Biederstedt, E., Villani, A.-C. & Theis, F. J. Scaling cross-tissue single-cell annotation models. Preprint at bioRxiv https://doi.org/10.1101/2023.10.07.561331 (2023). · DOI
- OpenAI. GPT-4 technical report. Preprint at https://doi.org/10.48550/arXiv.2303.08774 (2023). · DOI · Paper page
- Schaefer, M. et al. GPT-4 as a biomedical simulator. Comput. Biol. Med. 178, 108796 (2024). · DOI
- Chen, Y. T. & Zou, J. GenePT: a simple but effective foundation model for genes and cells built from ChatGPT. Preprint at bioRxiv https://doi.org/10.1101/2023.10.16.562533 (2024). · DOI
- Mao, H. et al. Position: graph foundation models are already here. In International Conference on Machine Learning (eds Salakhutdinov, R. et al.) (2024).
- Liu, J. et al. Towards graph foundation models: a survey and beyond. Preprint at https://doi.org/10.48550/arXiv.2310.11829 (2023). · DOI
- Fischer, D. S., Schaar, A. C. & Theis, F. J. Modeling intercellular communication in tissues using spatial graphs of cells. Nat. Biotechnol. 41, 332–336 (2022). · DOI · Paper page
- Khan, S. A. et al. Reusability report: learning the transcriptional grammar in single-cell RNA-sequencing data using transformers. Nat. Mach. Intel. 5, 1437–1446 (2023). · DOI
- Alsabbagh, A. R. et al. Foundation models meet imbalanced single-cell data when learning cell type annotations. Preprint at bioRxiv https://doi.org/10.1101/2023.10.24.563625 (2023). · DOI
- Pearson, K. LIII. On lines and planes of closest fit to systems of points in space. Lond. Edinb. Dublin Philos. Mag. J. Sci. 2, 559–572 (1901). · DOI · Paper page
- Wang, A. et al. GLUE: a multi-task benchmark and analysis platform for natural language understanding. In International Conference on Learning Representations (eds Sainath, T. et al.) (2019). · DOI · Paper page
- Luecken, M. D. et al. Defining and benchmarking open problems in single-cell analysis. Preprint at Research Square https://doi.org/10.21203/rs.3.rs-4181617/v1 (2024). · DOI
- Börner, K. et al. Anatomical structures, cell types and biomarkers of the Human Reference Atlas. Nat. Cell Biol. 23, 1117–1128 (2021). · DOI
- Sikkema, L. et al. An integrated cell atlas of the lung in health and disease. Nat. Med. 29, 1563–1577 (2023). · DOI · Paper page
- Vandereyken, K., Sifrim, A., Thienpont, B. & Voet, T. Methods and applications for single-cell and spatial multi-omics. Nat. Rev. Genet. 24, 494–515 (2023). · DOI · Paper page
- Guo, L. L. et al. EHR foundation models improve robustness in the presence of temporal distribution shift. Sci. Rep. 13, 3767 (2023). · DOI
- Fleck, J. S., Camp, J. G. & Treutlein, B. What is a cell type? Science 381, 733–734 (2023). · DOI
- Polychronidou, M. et al. Single-cell biology: what does the future hold? Mol. Syst. Biol. 19, e11799 (2023). · DOI
- Zhao, S., Zhang, J. & Nie, Z. Large-scale cell representation learning via divide-and-conquer contrastive learning. Preprint at https://doi.org/10.48550/arXiv.2306.04371 (2023). · DOI
- Xiong, L., Chen, T. & Kellis, M. scCLIP: multi-modal single-cell contrastive learning integration pre-training. In NeurIPS 2023 AI for Science Workshop (eds Welling, M. et al.) (2023).
- Bahdanau, D., Cho, K. & Bengio, Y. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations (eds Bengio, Y. et al.) (2015).
- Ba, J. L., Kiros, J. R. & Hinton, G. E. Layer normalization. In NIPS 2016 Deep Learning Symposium (eds Fitzgibbon, A. et al.) (2016).
Cited by
Ahlmann-Eltze, C., Huber, W., & Anders, S. (2025) · Nature Methods · doi:10.1038/s41592-025-02772-6 · Cited by 238
Predicting cellular responses to perturbation across diverse contexts with State
Adduri, A. K., Gautam, D., Bevilacqua, B., Naghipourfar, M., Imran, A., Shah, R., Teyssier, N., Verma, R., Carpenter, C., Eraslan, B., et al. (2026) · Cell · doi:10.1016/j.cell.2026.07.052 · Cited by 100
Ahlmann-Eltze, C., Huber, W., & Anders, S. (2024) · doi:10.1101/2024.09.16.613342 · Cited by 268
Wong, D. R., Hill, A. S., & Moccia, R. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf317 · Cited by 27
Acri, D. J., Mustaklem, R., Horan-Portelance, L., Dabin, L. C., Park, J. H., Hartigan, K. A., Kersey, H. N., Mesecar, M. E., Gibbs, J. R., Cookson, M. R., et al. (2026) · bioRxiv · doi:10.64898/2026.07.30.741795
“Technologies focused on assaying transcriptomes, proteomes, and epigenomes with single-cell precision or spatial resolution have increased the granularity and dimensionality with which cell types can be classified 2–4 .”
Metadata
- Authors
- Szałata, A., Hrovatin, K., Becker, S., Tejada-Lapuerta, A., Cui, H., Wang, B., & Theis, F. J.
- Venue
- Nature Methods
- Published
- 2024-08
- Volume
- 21
- Issue
- 8
- Pages
- 1430-1443
- ISSN
- 1548-7091 (ISSN-L)
- 1548-7105
- OpenAlex
- W4401455022
- Cited by
- 171
- References
- 53
- FWCI
- 19.87
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Transformers in single-cell omics: a review and new perspectives. https://app.cochat.ai/papers/10.1038/s41592-024-02353-z
- The paper
- Szałata, A., Hrovatin, K., Becker, S., Tejada-Lapuerta, A., Cui, H., Wang, B., & Theis, F. J. (2024). Transformers in single-cell omics: a review and new perspectives. Nature Methods. https://doi.org/10.1038/s41592-024-02353-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.