CoChat

Evaluating the Utilities of Foundation Models in Single‐Cell Data Analysis

Open access
Tianyu Liu, Kexing Li, Yuge Wang, Hongyu Li, Hongyu Zhao · 2026 · Advanced Science

Published version of a preprint

Cited by 61

The DOI matches its OpenAlex record; 71 of the 71 references it cites were found in a registry or an index.

Abstract

Foundation Models (FMs) have made significant strides in both industrial and scientific domains. In this paper, we evaluate the performance of FMs for single‐cell sequencing data analysis through comprehensive experiments across eight downstream tasks pertinent to single‐cell data. Overall, the top FMs include scGPT, Geneformer, and CellFM by considering model performances and user accessibility among ten single‐cell FMs. However, by comparing these FMs with task‐specific methods, we found that single‐cell FMs may not consistently excel than task‐specific methods in all tasks, which challenges the necessity of developing foundation models for single‐cell analysis. In addition, we evaluated the effects of hyperparameters, initial settings, and stability for training single‐cell FMs based on a proposed scEval framework, and provide guidelines for pre‐training and fine‐tuning to enhance the performances of single‐cell FMs. Our work summarizes the current state of single‐cell FMs, points to their constraints and avenues for future development, and offers a freely available evaluation pipeline to benchmark new models and improve method development.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

OpenAlex · 10.1002/advs.202514490

Cites

71 of 71 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 96

  1. 10.1126/science.aan6828 · DOI · Paper page
  2. 10.1146/annurev-genom-111320-090436 · DOI · Paper page
  3. 10.1038/s12276-018-0071-8 · DOI · Paper page
  4. 10.1038/ncomms14049 · DOI · Paper page
  5. Teichmann S., 2020, Method of the Year 2019: Single‐Cell Multimodal Omics, Nature Methods, 17, 2020
  6. 10.1146/annurev-biodatasci-020422-050645 · DOI
  7. J.Achiam S.Adler S.Agarwal et al. “GPT‐4 technical report ”arXiv preprint arXiv:2303.08774 (2023). · arXiv
  8. 10.18653/v1/2023.acl-industry.4 · DOI · Paper page
  9. 10.1016/j.cell.2019.05.031 · DOI · Paper page
  10. Driess D., Proceedings of the 40th International Conference on Machine Learning (ICML'23), 84698488
  11. T.Kuzman N.Ljubešić andI.Mozetič “Chatgpt: Beginning of an End of Manual Annotation? Use Case of Automatic Genre Identification ”arXiv preprint arXiv:2303.03953(2023). · arXiv
  12. 10.1093/bib/bbr012 · DOI · Paper page
  13. Selva Birunda S., 2021, A Review on Word Embedding Techniques for Text Classification, Innovative Data Communication Technologies and Application: Proceedings of ICIDCA 2020, 267, 10.1007/978-981-15-9651-3_23 · DOI · Paper page
  14. 10.1038/s41467-023-36476-2 · DOI
  15. 10.1038/s42256-022-00534-z · doi:10.1038/s42256-022-00534-z · Paper page
  16. S.Zhao J.Zhang andZ.Nie “Large‐Scale Cell Representation Learning via Divide‐and‐Conquer Contrastive Learning ”arXiv preprint arXiv:2306.04371(2023). · arXiv
  17. 10.1038/s41592-024-02201-0 · doi:10.1038/s41592-024-02201-0 · Paper page
  18. 10.1016/j.isci.2023.106536 · DOI
  19. 10.1038/s41422-024-01034-y · DOI · Paper page
  20. 10.1038/s41586-024-08411-y · DOI · Paper page
  21. Y.Rosen Y.Roohani A.Agrawal et al. “Universal cell embeddings: A foundation model for cell biology ”bioRxiv(2023): 2023–11. · DOI · Paper page
  22. 10.1038/s41467-025-59926-5 · DOI
  23. H.Wen W.Tang X.Dai et al. “CellPLM: Pre‐Training of Cell Language Model Beyond Single Cells ” inThe Twelfth International Conference on Learning Representations(2023). · DOI · Paper page
  24. Q.Ye H.Xu G.Xu et al. “mplug‐owl: Modularization Empowers Large Language Models With Multimodality ”arXiv preprint arXiv:2304.14178(2023). · arXiv
  25. 10.1038/s41586-019-1876-x · DOI · Paper page
  26. 10.1093/bioinformatics/btac427 · DOI
  27. D.KingmaandJ.Ba “Adam: A Method for Stochastic Optimization ” inInternational Conference on Learning Representations San Diega CA USA(ICLR 2015).
  28. S.Ruder “An Overview of Gradient Descent Optimization Algorithms ”arXiv preprint arXiv:1609.04747(2016). · arXiv
  29. H.Liu Z.Li D. L. W.Hall P.Liang andT.Ma “Sophia: A Scalable Stochastic Second‐Order Optimizer for Language Model Pre‐Training ” inThe Twelfth International Conference on Learning Representations(2024) https://openreview.net/forum?id=3xHDeA8Noi.
  30. Chen X., 2024, Symbolic Discovery of Optimization Algorithms, Advances in Neural Information Processing Systems, 36, 49205
  31. 10.1038/s41587-021-01161-6 · DOI · Paper page
  32. Pedregosa F., 2011, Scikit‐Learn: Machine Learning in Python, Journal of Machine Learning Research, 12, 2825
  33. 10.1186/s12864-018-5370-x · doi:10.1186/s12864-018-5370-x · Paper page
  34. 10.1093/nar/gkx1132 · DOI · Paper page
  35. 10.1002/dvdy.384 · DOI
  36. 10.1038/s41576-023-00618-5 · DOI · Paper page
  37. 10.1093/bib/bbac482 · DOI
  38. 10.1007/s00251-017-1017-3 · DOI · Paper page
  39. 10.1534/g3.114.015784 · DOI · Paper page
  40. 10.1038/s41596-023-00853-4 · DOI
  41. 10.1016/S1672-6529(08)60101-8 · DOI
  42. 10.1038/ni.3768 · DOI · Paper page
  43. 10.1038/s41467-023-37168-7 · DOI · Paper page
  44. Lun A. T., 2016, A Step‐by‐Step Workflow for Low‐Level Analysis of Single‐Cell RNA‐seq Data With Bioconductor, F1000Research, 5, 2122
  45. 10.1038/nbt.3192 · DOI · Paper page
  46. 10.1186/s13059-017-1382-0 · DOI · Paper page
  47. 10.1038/s41587-023-01772-1 · DOI · Paper page
  48. J.Kaplan S.McCandlish T.Henighan et al. “Scaling Laws for Neural Language Models ”arXiv preprint arXiv:2001.08361(2020). · arXiv
  49. B.Zhang Z.Liu C.Cherry andO.Firat “When Scaling Meets LLM Finetuning: The Effect of data Model and Finetuning Method ” inThe Twelfth International Conference on Learning Representations(2024) https://openreview.net/forum?id=5HCnKDeTws.
  50. 10.1038/s41592-024-02191-z · DOI · Paper page
  51. 10.1038/s41592-025-02899-6 · DOI
  52. 10.1093/nar/gkae1071 · DOI · Paper page
  53. 10.1038/s41467-023-41855-w · DOI
  54. 10.1016/j.cell.2021.04.048 · doi:10.1016/j.cell.2021.04.048 · Paper page
  55. E. J.Hu yelong shen P.Wallis et al. “LoRA: Low‐Rank Adaptation of Large Language Models ” inInternational Conference on Learning Representations(2022) https://openreview.net/forum?id=nZeVKeeFYf9.
  56. Z.Hu L.Wang Y.Lan et al. “LLM‐Adapters: An Adapter Family for Parameter‐Efficient Fine‐Tuning of Large Language Models ” inThe 2023 Conference on Empirical Methods in Natural Language Processing(2023). · DOI · Paper page
  57. 10.1038/s42256-023-00626-4 · DOI · Paper page
  58. Y.Wang S.Si D.Li et al. “Preserving In‐Context Learning Ability in Large Language Model Fine‐Tuning ”arXiv preprint arXiv:2211.00635(2022). · arXiv
  59. R.Xu F.Luo Z.Zhang et al. “Raise a Child in Large Language Model: Towards Effective and Generalizable Fine‐Tuning ” inProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing M. F.Moens X.Huang L.Specia andS. W. T.Yih Eds. (Online and Punta Cana Dominican Republic: Association for Computational Linguistics 2021) 9514–9528 https://aclanthology.org/2021.emnlp‐main.749. · DOI
  60. Y.Ji Y.Deng Y.Gong et al. “Exploring the impact of instruction data scaling on large language models: An empirical study on real‐world use cases ”arXiv preprint arXiv:2303.14742(2023). · arXiv
  61. T.Carta C.Romac T.Wolf S.Lamprier O.Sigaud andP. Y.Oudeyer “Grounding large language models in interactive environments with online reinforcement learning ” inInternational Conference on Machine Learning(PMLR 2023) 3676–3713.
  62. Chen Y. T., 2023, Simple and effective embedding model for single‐cell biology built from ChatGPT, Nature biomedical engineering, 483
  63. 10.1038/s42256-022-00516-1 · DOI · Paper page
  64. Lundberg S. M., 2017, Proceedings of the 31st Advances in Neural Information Processing Systems, 4768
  65. M.Sundararajan A.Taly andQ.Yan “Axiomatic Attribution for Deep Networks ” inInternational Conference on Machine Learning(PMLR 2017) 3319–3328.
  66. S.JainandB. C.Wallace “Attention is not Explanation ” inProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Volume 1 (Long and Short Papers) (Minneapolis Minnesota: Association for Computational Linguistics 2019) 3543–3556 https://aclanthology.org/N19‐1357.
  67. A.Bibal R.Cardon D.Alfter et al. “Is Attention Explanation? An Introduction to the Debate ” inProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (Dublin Ireland: Association for Computational Linguistics 2022) 3889–3900 https://aclanthology.org/2022.acl‐long.269. · DOI
  68. T.Huang T.Liu M.Babadi R.Ying andW.Jin “Stpath: A Generative Foundation Model for Integrating Spatial Transcriptomics and Whole Slide Images ”bioRxiv(2025): 2025–04. · doi:10.1101/2025.04.19.649665
  69. 10.15252/msb.202211517 · DOI · Paper page
  70. 10.1016/j.neucom.2013.09.055 · DOI · Paper page
  71. Y.Chang X.Wang J.Wang et al. “A Survey on Evaluation of Large Language Models ”arXiv preprint arXiv:2307.03109(2023). · arXiv
  72. 10.1186/s13059-024-03183-0 · doi:10.1186/s13059-024-03183-0 · Paper page
  73. 10.1038/s41592-019-0535-3 · doi:10.1038/s41592-019-0535-3 · Paper page
  74. N.Ravindra A.Sehanobish J. L.Pappalardo D. A.Hafler andD.vanDijk “Disease State Prediction From Single‐Cell Data Using Graph Attention Networks ” inProceedings of the ACM Conference on Health Inference and Learning(ACM 2020):121–130. · DOI · Paper page
  75. 10.1186/s12915-018-0564-x · DOI · Paper page
  76. 10.1038/s41467-023-40503-7 · DOI
  77. Pearl J., 2018, The Book of Why: The New Science of Cause and Effect
  78. 10.1038/s41467-018-03405-7 · DOI · Paper page
  79. 10.1186/s13059-020-02132-x · DOI · Paper page
  80. 10.1038/s41592-025-02617-2 · DOI · Paper page
  81. Christiano P. F., 2017, Proceedings of the 31st Advances in Neural Information Processing Systems, 4302
  82. Radford A., 2019, Language Models are Unsupervised Multitask Learners, OpenAI blog, 1, 9
  83. K. M.Choromanski V.Likhosherstov D.Dohan et al. “Rethinking Attention With Performers ” inInternational Conference on Learning Representations(2021) https://openreview.net/forum?id=Ua6zuk0WRH.
  84. Wu C., 2014, Mygene. info: gene annotation query as a service, bioRxiv
  85. Goodfellow I., 2014, Advances in Neural Information Processing Systems
  86. Gulrajani I., 2017, the Proceedings of the 31st Advances in Neural Information Processing Systems, 5769
  87. C.Doersch “Tutorial on Variational Autoencoders ”arXiv preprint arXiv:1606.05908(2016). · arXiv
  88. D.Misra “Mish: A Self Regularized Non‐Monotonic Activation Function ”arXiv preprint arXiv:1908.08681(2019). · DOI · Paper page
  89. 10.1017/S1351324916000334 · DOI · Paper page
  90. Schmidt T., 2007, Coping with Copulas, Copulas‐From Theory to Application in Finance, 3, 1
  91. 10.1038/s41591-023-02327-2 · DOI · Paper page
  92. 10.1038/s41592-021-01358-2 · DOI · Paper page
  93. 10.15252/msb.20199389 · DOI · Paper page
  94. 10.1038/s41467-021-22368-w · DOI · Paper page
  95. Zhu Q., 2020, Developmental Trajectory of Prehematopoietic stem Cell Formation From Endothelium, Blood, The Journal of the American Society of Hematology, 136, 845
  96. 10.1186/s13059-022-02734-7 · DOI · Paper page

Cited by

See all 61

Metadata

Authors
Liu, T., Li, K., Wang, Y., Li, H., & Zhao, H.
Venue
Advanced Science
Publisher
Wiley
Published
2026-03-23
Volume
13
Issue
27
Pages
e14490-e14490
ISSN
2198-3844 (ISSN-L)
OpenAlex
W7140132101
Cited by
61
References
71
FWCI
31.83
Type
journal article
Language
en
Open access
gold
Licence
Funders
National Institutes of Health (R01 GM134005, P50CA196530, U01HG013840)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.