Evaluating the Utilities of Foundation Models in Single‐Cell Data Analysis
Open accessPublished version of a preprint
The DOI matches its OpenAlex record; 68 of the 68 references it cites were found in a registry or an index.
Abstract
Foundation Models (FMs) have made significant strides in both industrial and scientific domains. In this paper, we evaluate the performance of FMs for single‐cell sequencing data analysis through comprehensive experiments across eight downstream tasks pertinent to single‐cell data. Overall, the top FMs include scGPT, Geneformer, and CellFM by considering model performances and user accessibility among ten single‐cell FMs. However, by comparing these FMs with task‐specific methods, we found that single‐cell FMs may not consistently excel than task‐specific methods in all tasks, which challenges the necessity of developing foundation models for single‐cell analysis. In addition, we evaluated the effects of hyperparameters, initial settings, and stability for training single‐cell FMs based on a proposed scEval framework, and provide guidelines for pre‐training and fine‐tuning to enhance the performances of single‐cell FMs. Our work summarizes the current state of single‐cell FMs, points to their constraints and avenues for future development, and offers a freely available evaluation pipeline to benchmark new models and improve method development.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Fast, sensitive and accurate integration of single-cell data with Harmony
Korsunsky, I., Millard, N., Fan, J., Slowikowski, K., Zhang, F., Wei, K., Baglaenko, Y., Brenner, M., Loh, P.-R., & Raychaudhuri, S. (2019) · Nature Methods · doi:10.1038/s41592-019-0619-0 · Cited by 10,488
UMAP: Uniform Manifold Approximation and Projection
McInnes, L., Healy, J., Saul, N., & Großberger, L. (2018) · Journal of Open Source Software · doi:10.21105/joss.00861 · Cited by 9,826
Improved training of Wasserstein GANs
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (2017) · arXiv · doi:10.48550/arxiv.1704.00028 · Cited by 6,800
From Louvain to Leiden: guaranteeing well-connected communities
Traag, V. A., Waltman, L., & van Eck, N. J. (2019) · Scientific Reports · doi:10.1038/s41598-019-41695-z · Cited by 5,635
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
Performance of neural network basecalling tools for Oxford Nanopore sequencing
Wick, R. R., Judd, L. M., & Holt, K. E. (2019) · Genome Biology · doi:10.1186/s13059-019-1727-y · Cited by 3,221
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Research works printed but not linked: 99
- 10.1126/science.aan6828 · DOI · Paper page
- 10.1146/annurev-genom-111320-090436 · DOI · Paper page
- 10.1038/s12276-018-0071-8 · DOI · Paper page
- 10.1038/ncomms14049 · DOI · Paper page
- 10.1038/nmeth.4380 · DOI · Paper page
- Teichmann S., 2020, Method of the Year 2019: Single‐Cell Multimodal Omics, Nature Methods, 17, 2020
- 10.1146/annurev-biodatasci-020422-050645 · DOI
- J.Achiam S.Adler S.Agarwal et al. “GPT‐4 technical report ”arXiv preprint arXiv:2303.08774 (2023). · arXiv
- 10.1038/s41591-023-02448-8 · DOI · Paper page
- 10.18653/v1/2023.acl-industry.4 · DOI · Paper page
- 10.1016/j.cell.2019.05.031 · DOI · Paper page
- Driess D., Proceedings of the 40th International Conference on Machine Learning (ICML'23), 84698488
- T.Kuzman N.Ljubešić andI.Mozetič “Chatgpt: Beginning of an End of Manual Annotation? Use Case of Automatic Genre Identification ”arXiv preprint arXiv:2303.03953(2023). · arXiv
- 10.1093/bib/bbr012 · DOI · Paper page
- Selva Birunda S., 2021, A Review on Word Embedding Techniques for Text Classification, Innovative Data Communication Technologies and Application: Proceedings of ICIDCA 2020, 267, 10.1007/978-981-15-9651-3_23 · DOI · Paper page
- 10.1038/s41467-023-36476-2 · DOI
- 10.1038/s42256-022-00534-z · doi:10.1038/s42256-022-00534-z · Paper page
- S.Zhao J.Zhang andZ.Nie “Large‐Scale Cell Representation Learning via Divide‐and‐Conquer Contrastive Learning ”arXiv preprint arXiv:2306.04371(2023). · arXiv
- 10.1038/s41592-024-02201-0 · doi:10.1038/s41592-024-02201-0 · Paper page
- 10.1016/j.isci.2023.106536 · DOI
- 10.1038/s41422-024-01034-y · DOI · Paper page
- 10.1038/s41586-024-08411-y · DOI · Paper page
- Y.Rosen Y.Roohani A.Agrawal et al. “Universal cell embeddings: A foundation model for cell biology ”bioRxiv(2023): 2023–11. · DOI · Paper page
- 10.1038/s41467-025-59926-5 · DOI
- H.Wen W.Tang X.Dai et al. “CellPLM: Pre‐Training of Cell Language Model Beyond Single Cells ” inThe Twelfth International Conference on Learning Representations(2023). · DOI · Paper page
- Q.Ye H.Xu G.Xu et al. “mplug‐owl: Modularization Empowers Large Language Models With Multimodality ”arXiv preprint arXiv:2304.14178(2023). · arXiv
- 10.1038/s41586-019-1876-x · DOI · Paper page
- 10.1093/bioinformatics/btac427 · DOI
- D.KingmaandJ.Ba “Adam: A Method for Stochastic Optimization ” inInternational Conference on Learning Representations San Diega CA USA(ICLR 2015).
- S.Ruder “An Overview of Gradient Descent Optimization Algorithms ”arXiv preprint arXiv:1609.04747(2016). · arXiv
- H.Liu Z.Li D. L. W.Hall P.Liang andT.Ma “Sophia: A Scalable Stochastic Second‐Order Optimizer for Language Model Pre‐Training ” inThe Twelfth International Conference on Learning Representations(2024) https://openreview.net/forum?id=3xHDeA8Noi.
- Chen X., 2024, Symbolic Discovery of Optimization Algorithms, Advances in Neural Information Processing Systems, 36, 49205
- 10.1038/s41587-021-01161-6 · DOI · Paper page
- Pedregosa F., 2011, Scikit‐Learn: Machine Learning in Python, Journal of Machine Learning Research, 12, 2825
- 10.1186/s12864-018-5370-x · doi:10.1186/s12864-018-5370-x · Paper page
- 10.1093/nar/gkx1132 · DOI · Paper page
- 10.1002/dvdy.384 · DOI
- 10.1038/s41576-023-00618-5 · DOI · Paper page
- 10.1093/bib/bbac482 · DOI
- 10.1007/s00251-017-1017-3 · DOI · Paper page
- 10.1534/g3.114.015784 · DOI · Paper page
- 10.1038/s41596-023-00853-4 · DOI
- 10.1016/S1672-6529(08)60101-8 · DOI
- 10.1038/ni.3768 · DOI · Paper page
- 10.1038/s41467-023-37168-7 · DOI · Paper page
- Lun A. T., 2016, A Step‐by‐Step Workflow for Low‐Level Analysis of Single‐Cell RNA‐seq Data With Bioconductor, F1000Research, 5, 2122
- 10.1038/nbt.3192 · DOI · Paper page
- 10.1186/s13059-017-1382-0 · DOI · Paper page
- 10.1038/s41587-023-01772-1 · DOI · Paper page
- J.Kaplan S.McCandlish T.Henighan et al. “Scaling Laws for Neural Language Models ”arXiv preprint arXiv:2001.08361(2020). · arXiv
- B.Zhang Z.Liu C.Cherry andO.Firat “When Scaling Meets LLM Finetuning: The Effect of data Model and Finetuning Method ” inThe Twelfth International Conference on Learning Representations(2024) https://openreview.net/forum?id=5HCnKDeTws.
- 10.1038/s41592-024-02191-z · DOI · Paper page
- 10.1038/s41592-025-02899-6 · DOI
- 10.1093/nar/gkae1071 · DOI · Paper page
- 10.1038/s41467-023-41855-w · DOI
- 10.1016/j.cell.2021.04.048 · doi:10.1016/j.cell.2021.04.048 · Paper page
- E. J.Hu yelong shen P.Wallis et al. “LoRA: Low‐Rank Adaptation of Large Language Models ” inInternational Conference on Learning Representations(2022) https://openreview.net/forum?id=nZeVKeeFYf9.
- Z.Hu L.Wang Y.Lan et al. “LLM‐Adapters: An Adapter Family for Parameter‐Efficient Fine‐Tuning of Large Language Models ” inThe 2023 Conference on Empirical Methods in Natural Language Processing(2023). · DOI · Paper page
- 10.1038/s42256-023-00626-4 · DOI · Paper page
- Y.Wang S.Si D.Li et al. “Preserving In‐Context Learning Ability in Large Language Model Fine‐Tuning ”arXiv preprint arXiv:2211.00635(2022). · arXiv
- R.Xu F.Luo Z.Zhang et al. “Raise a Child in Large Language Model: Towards Effective and Generalizable Fine‐Tuning ” inProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing M. F.Moens X.Huang L.Specia andS. W. T.Yih Eds. (Online and Punta Cana Dominican Republic: Association for Computational Linguistics 2021) 9514–9528 https://aclanthology.org/2021.emnlp‐main.749. · DOI
- Y.Ji Y.Deng Y.Gong et al. “Exploring the impact of instruction data scaling on large language models: An empirical study on real‐world use cases ”arXiv preprint arXiv:2303.14742(2023). · arXiv
- T.Carta C.Romac T.Wolf S.Lamprier O.Sigaud andP. Y.Oudeyer “Grounding large language models in interactive environments with online reinforcement learning ” inInternational Conference on Machine Learning(PMLR 2023) 3676–3713.
- Chen Y. T., 2023, Simple and effective embedding model for single‐cell biology built from ChatGPT, Nature biomedical engineering, 483
- 10.1038/s42256-022-00516-1 · DOI · Paper page
- Lundberg S. M., 2017, Proceedings of the 31st Advances in Neural Information Processing Systems, 4768
- M.Sundararajan A.Taly andQ.Yan “Axiomatic Attribution for Deep Networks ” inInternational Conference on Machine Learning(PMLR 2017) 3319–3328.
- S.JainandB. C.Wallace “Attention is not Explanation ” inProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Volume 1 (Long and Short Papers) (Minneapolis Minnesota: Association for Computational Linguistics 2019) 3543–3556 https://aclanthology.org/N19‐1357.
- A.Bibal R.Cardon D.Alfter et al. “Is Attention Explanation? An Introduction to the Debate ” inProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (Dublin Ireland: Association for Computational Linguistics 2022) 3889–3900 https://aclanthology.org/2022.acl‐long.269. · DOI
- T.Huang T.Liu M.Babadi R.Ying andW.Jin “Stpath: A Generative Foundation Model for Integrating Spatial Transcriptomics and Whole Slide Images ”bioRxiv(2025): 2025–04. · doi:10.1101/2025.04.19.649665
- 10.15252/msb.202211517 · DOI · Paper page
- 10.1016/j.neucom.2013.09.055 · DOI · Paper page
- Y.Chang X.Wang J.Wang et al. “A Survey on Evaluation of Large Language Models ”arXiv preprint arXiv:2307.03109(2023). · arXiv
- 10.1186/s13059-024-03183-0 · doi:10.1186/s13059-024-03183-0 · Paper page
- 10.1038/s41592-019-0535-3 · doi:10.1038/s41592-019-0535-3 · Paper page
- N.Ravindra A.Sehanobish J. L.Pappalardo D. A.Hafler andD.vanDijk “Disease State Prediction From Single‐Cell Data Using Graph Attention Networks ” inProceedings of the ACM Conference on Health Inference and Learning(ACM 2020):121–130. · DOI · Paper page
- 10.1186/s12915-018-0564-x · DOI · Paper page
- 10.1016/j.celrep.2022.110333 · DOI · Paper page
- 10.1038/s41467-023-40503-7 · DOI
- Pearl J., 2018, The Book of Why: The New Science of Cause and Effect
- 10.1038/s41467-018-03405-7 · DOI · Paper page
- 10.1186/s13059-020-02132-x · DOI · Paper page
- 10.1038/s41592-025-02617-2 · DOI · Paper page
- Christiano P. F., 2017, Proceedings of the 31st Advances in Neural Information Processing Systems, 4302
- Radford A., 2019, Language Models are Unsupervised Multitask Learners, OpenAI blog, 1, 9
- K. M.Choromanski V.Likhosherstov D.Dohan et al. “Rethinking Attention With Performers ” inInternational Conference on Learning Representations(2021) https://openreview.net/forum?id=Ua6zuk0WRH.
- Wu C., 2014, Mygene. info: gene annotation query as a service, bioRxiv
- Goodfellow I., 2014, Advances in Neural Information Processing Systems
- Gulrajani I., 2017, the Proceedings of the 31st Advances in Neural Information Processing Systems, 5769
- C.Doersch “Tutorial on Variational Autoencoders ”arXiv preprint arXiv:1606.05908(2016). · arXiv
- D.Misra “Mish: A Self Regularized Non‐Monotonic Activation Function ”arXiv preprint arXiv:1908.08681(2019). · DOI · Paper page
- 10.1017/S1351324916000334 · DOI · Paper page
- Schmidt T., 2007, Coping with Copulas, Copulas‐From Theory to Application in Finance, 3, 1
- 10.1038/s41591-023-02327-2 · DOI · Paper page
- 10.1038/s41592-021-01358-2 · DOI · Paper page
- 10.15252/msb.20199389 · DOI · Paper page
- 10.1038/s41467-021-22368-w · DOI · Paper page
- Zhu Q., 2020, Developmental Trajectory of Prehematopoietic stem Cell Formation From Endothelium, Blood, The Journal of the American Society of Hematology, 136, 845
- 10.1186/s13059-022-02734-7 · DOI · Paper page
Cited by
Ahlmann-Eltze, C., Huber, W., & Anders, S. (2025) · Nature Methods · doi:10.1038/s41592-025-02772-6 · Cited by 238
Benchmarking foundation cell models for post-perturbation RNA-seq prediction
Csendes, G., Sanz, G., Szalay, K. Z., & Szalai, B. (2025) · BMC Genomics · doi:10.1186/s12864-025-11600-2 · Cited by 47
Single-cell foundation models: bringing artificial intelligence into cell biology
Baek, S., Song, K., & Lee, I. (2025) · Experimental & Molecular Medicine · doi:10.1038/s12276-025-01547-5 · Cited by 46
Harnessing the deep learning power of foundation models in single-cell omics
Ma, Q., Jiang, Y., Cheng, H., & Xu, D. (2024) · Nature Reviews Molecular Cell Biology · doi:10.1038/s41580-024-00756-6 · Cited by 34
Wong, D. R., Hill, A. S., & Moccia, R. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf317 · Cited by 27
Metadata
- Authors
- Liu, T., Li, K., Wang, Y., Li, H., & Zhao, H.
- Venue
- Advanced Science
- Publisher
- Wiley
- Published
- 2026-03-23
- Volume
- 13
- Issue
- 27
- Pages
- e14490-e14490
- ISSN
- 2198-3844 (ISSN-L)
- OpenAlex
- W7140132101
- Cited by
- 61
- References
- 68
- FWCI
- 31.83
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0/ (vor)
- doi.wiley.com/10.1002/tdm_license_1.1 (tdm)
- Funders
- National Institutes of Health (R01 GM134005, P50CA196530, U01HG013840)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evaluating the Utilities of Foundation Models in Single‐Cell Data Analysis. https://app.cochat.ai/papers/10.1002/advs.202514490
- The paper
- Liu, T., Li, K., Wang, Y., Li, H., & Zhao, H. (2026). Evaluating the Utilities of Foundation Models in Single‐Cell Data Analysis. Advanced Science. https://doi.org/10.1002/advs.202514490
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.