Student or educator? Sign up with your school email and get 6 months of Pro free.
Cited by 171

The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.

Transformers in single-cell omics: a review and new perspectives

Szałata, Artur, Hrovatin, Karin, Becker, Sören, Tejada-Lapuerta, Alejandro, Cui, Haotian, Wang, Bo, Theis, Fabian J.·2024·Nature Methods

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41592-024-02353-z

Cites

53 of 53 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 53 references

Research works printed but not linked: 50

  1. Zhu, C., Preissl, S. & Ren, B. Single-cell multimodal omics: the power of many. Nat. Methods 17, 11–14 (2020). · DOI · Paper page
  2. Method of the Year 2013. Nat. Methods 11, 1 (2013). · DOI
  3. Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (eds Burstein, J., Doran, C. & Solorio, T.) 4171–4186 (2019).
  4. Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. 30, 5998–6008 (2017). This work introduced the transformer architecture, originally designed and evaluated on NLP tasks.
  5. Radford, A. et al. Robust speech recognition via large-scale weak supervision. PMLR (ed. Lawrence, N.) 202, 28492–28518 (2023).
  6. Amodio, M. et al. Exploring single-cell data with deep multitasking neural networks. Nat. Methods 16, 1139–1145 (2019). · DOI · Paper page
  7. Shen, H. et al. A universal approach for integrating super large-scale single-cell transcriptomes by exploring gene rankings. Brief. Bioinform. 23, bbab573 (2022). This work introduced the first gene-ranking-based single-cell transformer. It was also the first single-cell transformer pretrained on a large dataset of over 10 million cells. · DOI
  8. Shen, H. et al. Generative pretraining from large-scale transcriptomes for single-cell deciphering. iScience 26, 106536 (2023). · DOI
  9. Levine, D. et al. Cell2Sentence: teaching large language models the language of biology. In Proc. 41st International Conference on Machine Learning (eds Salakhutdinov, R. et al.) (2024). · DOI
  10. Wen, H. et al. CellPLM: pre-training of cell language model beyond single cells. In International Conference on Learning Representations (eds Kim, B. et al.) (2024). · DOI
  11. Wen, H. et al. Single cells are spatial tokens: transformers for spatial transcriptomic data imputation. Preprint at https://doi.org/10.48550/arXiv.2302.03038 (2023). · DOI
  12. Dao, T., Fu, D. Y., Ermon, S., Rudra, A. & Ré, C. FlashAttention: fast and memory-efficient exact attention with IO-awareness. Adv. Neural Inf. Process. Syst. 35, 16344–16359 (2024).
  13. Bian, H. et al. scMulan: a multitask generative pre-trained language model for single-cell analysis. In Research in Computational Molecular Biology. RECOMB 2024. Lecture Notes in Computer Science (ed. Ma, J.) Vol. 14758, 479–482 (Springer, Cham, 2024). · DOI
  14. Domcke, S. & Shendure, J. A reference cell tree will serve science better than a reference cell atlas. Cell 186, 1103–1114 (2023). · DOI
  15. Oh, G., Choi, B., Jung, I. & Ye, J. C. scHyena: foundation model for full-length single-cell RNA-seq analysis in brain. Preprint at https://doi.org/10.48550/arXiv.2310.02713 (2023). · DOI
  16. Tang, W. et al. Single-cell multimodal prediction via transformers. In Proc. 32nd ACM International Conference on Information and Knowledge Management 2422–2431 (2023). · DOI
  17. Gong, J. et al. xTrimoGene: an efficient and scalable representation learner for single-cell RNA-seq data. Adv. Neural Inform. Process. Syst. 36, 69391–69403 (2024).
  18. Tarashansky, A. J. et al. Mapping single-cell atlases throughout Metazoa unravels cell type evolution. eLife 10, e66747 (2021). · DOI · Paper page
  19. Badia-I-Mompel, P. et al. Gene regulatory network inference in the era of single-cell multi-omics. Nat. Rev. Genet. 24, 739–754 (2023). · DOI · Paper page
  20. Claringbould, A. & Zaugg, J. B. Enhancers in disease: molecular basis and emerging treatment strategies. Trends Mol. Med. 27, 1060–1073 (2021). · DOI · Paper page
  21. Liu, X. et al. Pathformer: a biological pathway informed transformer for disease diagnosis and prognosis using multi-omics data. Bioinformatics 40, btae316 (2024). · DOI
  22. Liu, L., Li, W., Wong, K.-C., Yang, F. & Yao, J. A pre-trained large generative model for translating single-cell transcriptome to proteome. Preprint at bioRxiv https://doi.org/10.1101/2023.07.04.547619 (2023). · DOI
  23. Tyler, S. R., Guccione, E. & Schadt, E. E. Erasure of biologically meaningful signal by unsupervised scRNAseq batch-correction methods. Preprint at bioRxiv https://doi.org/10.1101/2021.11.15.468733 (2023). · DOI
  24. Rosen, Y. et al. Universal Cell Embeddings: a foundation model for cell biology. Preprint at bioRxiv https://doi.org/10.1101/2023.11.28.568918 (2023). · DOI · Paper page
  25. Kingma, D. P. & Welling, M. Auto-encoding variational Bayes. In International Conference on Learning Representations (eds Bengio, Y. et al.) (2014).
  26. Lotfollahi, M. et al. Predicting cellular responses to complex perturbations in high-throughput screens. Mol. Syst. Biol. 19, e11517 (2023). · DOI · Paper page
  27. Köhler, N. D., Büttner, M., Andriamanga, N. & Theis, F. J. Deep learning does not outperform classical machine learning for cell-type annotation. Preprint at bioRxiv https://doi.org/10.1101/653907 (2021). · DOI
  28. Domínguez Conde, C. et al. Cross-tissue immune cell analysis reveals tissue-specific features in humans. Science 376, eabl5197 (2022). · DOI · Paper page
  29. Fischer, F., Fischer, D. S., Biederstedt, E., Villani, A.-C. & Theis, F. J. Scaling cross-tissue single-cell annotation models. Preprint at bioRxiv https://doi.org/10.1101/2023.10.07.561331 (2023). · DOI
  30. OpenAI. GPT-4 technical report. Preprint at https://doi.org/10.48550/arXiv.2303.08774 (2023). · DOI · Paper page
  31. Schaefer, M. et al. GPT-4 as a biomedical simulator. Comput. Biol. Med. 178, 108796 (2024). · DOI
  32. Chen, Y. T. & Zou, J. GenePT: a simple but effective foundation model for genes and cells built from ChatGPT. Preprint at bioRxiv https://doi.org/10.1101/2023.10.16.562533 (2024). · DOI
  33. Mao, H. et al. Position: graph foundation models are already here. In International Conference on Machine Learning (eds Salakhutdinov, R. et al.) (2024).
  34. Liu, J. et al. Towards graph foundation models: a survey and beyond. Preprint at https://doi.org/10.48550/arXiv.2310.11829 (2023). · DOI
  35. Fischer, D. S., Schaar, A. C. & Theis, F. J. Modeling intercellular communication in tissues using spatial graphs of cells. Nat. Biotechnol. 41, 332–336 (2022). · DOI · Paper page
  36. Khan, S. A. et al. Reusability report: learning the transcriptional grammar in single-cell RNA-sequencing data using transformers. Nat. Mach. Intel. 5, 1437–1446 (2023). · DOI
  37. Alsabbagh, A. R. et al. Foundation models meet imbalanced single-cell data when learning cell type annotations. Preprint at bioRxiv https://doi.org/10.1101/2023.10.24.563625 (2023). · DOI
  38. Pearson, K. LIII. On lines and planes of closest fit to systems of points in space. Lond. Edinb. Dublin Philos. Mag. J. Sci. 2, 559–572 (1901). · DOI · Paper page
  39. Wang, A. et al. GLUE: a multi-task benchmark and analysis platform for natural language understanding. In International Conference on Learning Representations (eds Sainath, T. et al.) (2019). · DOI · Paper page
  40. Luecken, M. D. et al. Defining and benchmarking open problems in single-cell analysis. Preprint at Research Square https://doi.org/10.21203/rs.3.rs-4181617/v1 (2024). · DOI
  41. Börner, K. et al. Anatomical structures, cell types and biomarkers of the Human Reference Atlas. Nat. Cell Biol. 23, 1117–1128 (2021). · DOI
  42. Sikkema, L. et al. An integrated cell atlas of the lung in health and disease. Nat. Med. 29, 1563–1577 (2023). · DOI · Paper page
  43. Vandereyken, K., Sifrim, A., Thienpont, B. & Voet, T. Methods and applications for single-cell and spatial multi-omics. Nat. Rev. Genet. 24, 494–515 (2023). · DOI · Paper page
  44. Guo, L. L. et al. EHR foundation models improve robustness in the presence of temporal distribution shift. Sci. Rep. 13, 3767 (2023). · DOI
  45. Fleck, J. S., Camp, J. G. & Treutlein, B. What is a cell type? Science 381, 733–734 (2023). · DOI
  46. Polychronidou, M. et al. Single-cell biology: what does the future hold? Mol. Syst. Biol. 19, e11799 (2023). · DOI
  47. Zhao, S., Zhang, J. & Nie, Z. Large-scale cell representation learning via divide-and-conquer contrastive learning. Preprint at https://doi.org/10.48550/arXiv.2306.04371 (2023). · DOI
  48. Xiong, L., Chen, T. & Kellis, M. scCLIP: multi-modal single-cell contrastive learning integration pre-training. In NeurIPS 2023 AI for Science Workshop (eds Welling, M. et al.) (2023).
  49. Bahdanau, D., Cho, K. & Bengio, Y. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations (eds Bengio, Y. et al.) (2015).
  50. Ba, J. L., Kiros, J. R. & Hinton, G. E. Layer normalization. In NIPS 2016 Deep Learning Symposium (eds Fitzgibbon, A. et al.) (2016).

Cited by

See all 171

Metadata

Authors
Szałata, A., Hrovatin, K., Becker, S., Tejada-Lapuerta, A., Cui, H., Wang, B., & Theis, F. J.
Venue
Nature Methods
Published
2024-08
Volume
21
Issue
8
Pages
1430-1443
ISSN
  • 1548-7091 (ISSN-L)
  • 1548-7105
OpenAlex
W4401455022
Cited by
171
References
53
FWCI
19.87
Type
journal article
Language
en
Open access
closed
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Transformers in single-cell omics: a review and new perspectives. https://app.cochat.ai/papers/10.1038/s41592-024-02353-z
The paper
Szałata, A., Hrovatin, K., Becker, S., Tejada-Lapuerta, A., Cui, H., Wang, B., & Theis, F. J. (2024). Transformers in single-cell omics: a review and new perspectives. Nature Methods. https://doi.org/10.1038/s41592-024-02353-z

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.