Evaluating the Utilities of Foundation Models in Single‐Cell Data Analysis
Open accessPublished version of a preprint
The DOI matches its OpenAlex record; 71 of the 71 references it cites were found in a registry or an index.
Abstract
Foundation Models (FMs) have made significant strides in both industrial and scientific domains. In this paper, we evaluate the performance of FMs for single‐cell sequencing data analysis through comprehensive experiments across eight downstream tasks pertinent to single‐cell data. Overall, the top FMs include scGPT, Geneformer, and CellFM by considering model performances and user accessibility among ten single‐cell FMs. However, by comparing these FMs with task‐specific methods, we found that single‐cell FMs may not consistently excel than task‐specific methods in all tasks, which challenges the necessity of developing foundation models for single‐cell analysis. In addition, we evaluated the effects of hyperparameters, initial settings, and stability for training single‐cell FMs based on a proposed scEval framework, and provide guidelines for pre‐training and fine‐tuning to enhance the performances of single‐cell FMs. Our work summarizes the current state of single‐cell FMs, points to their constraints and avenues for future development, and offers a freely available evaluation pipeline to benchmark new models and improve method development.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Fast, sensitive and accurate integration of single-cell data with Harmony
Korsunsky, I., Millard, N., Fan, J., Slowikowski, K., Zhang, F., Wei, K., Baglaenko, Y., Brenner, M., Loh, P.-R., & Raychaudhuri, S. (2019) · Nature Methods · doi:10.1038/s41592-019-0619-0 · Cited by 10,488
UMAP: Uniform Manifold Approximation and Projection
McInnes, L., Healy, J., Saul, N., & Großberger, L. (2018) · Journal of Open Source Software · doi:10.21105/joss.00861 · Cited by 9,826
Improved training of Wasserstein GANs
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (2017) · arXiv · doi:10.48550/arxiv.1704.00028 · Cited by 6,800
From Louvain to Leiden: guaranteeing well-connected communities
Traag, V. A., Waltman, L., & van Eck, N. J. (2019) · Scientific Reports · doi:10.1038/s41598-019-41695-z · Cited by 5,635
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Large Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944
Performance of neural network basecalling tools for Oxford Nanopore sequencing
Wick, R. R., Judd, L. M., & Holt, K. E. (2019) · Genome Biology · doi:10.1186/s13059-019-1727-y · Cited by 3,221
Simultaneous epitope and transcriptome measurement in single cells
Stoeckius, M., Hafemeister, C., Stephenson, W., Houck-Loomis, B., Chattopadhyay, P. K., Swerdlow, H., Satija, R., & Smibert, P. (2017) · Nature Methods · doi:10.1038/nmeth.4380 · Cited by 3,098
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Single-cell transcriptomic analysis of Alzheimer’s disease
Mathys, H., Davila-Velderrain, J., Peng, Z., Gao, F., Mohammadi, S., Young, J. Z., Menon, M., He, L., Abdurrob, F., Jiang, X., et al. (2019) · Nature · doi:10.1038/s41586-019-1195-2 · Cited by 2,568
Deep generative modeling for single-cell transcriptomics
Lopez, R., Regier, J., Cole, M. B., Jordan, M. I., & Yosef, N. (2018) · Nature Methods · doi:10.1038/s41592-018-0229-2 · Cited by 2,530
Opportunities and challenges in long-read sequencing data analysis
Amarasinghe, S. L., Su, S., Dong, X., Zappia, L., Ritchie, M. E., & Gouil, Q. (2020) · Genome Biology · doi:10.1186/s13059-020-1935-5 · Cited by 2,530
Tackling the widespread and critical impact of batch effects in high-throughput data
Leek, J. T., Scharpf, R. B., Bravo, H. C., Simcha, D., Langmead, B., Johnson, W. E., Geman, D., Baggerly, K., & Irizarry, R. A. (2010) · Nature Reviews Genetics · doi:10.1038/nrg2825 · Cited by 2,196
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
PANTHER : Making genome‐scale phylogenetics accessible to all
Thomas, P. D., Ebert, D., Muruganujan, A., Mushayahama, T., Albou, L., & Mi, H. (2022) · Protein Science · doi:10.1002/pro.4218 · Cited by 2,010
Dixit, A., Parnas, O., Li, B., Chen, J., Fulco, C. P., Jerby-Arnon, L., Marjanovic, N. D., Dionne, D., Burks, T., Raychowdhury, R., et al. (2016) · Cell · doi:10.1016/j.cell.2016.11.038 · Cited by 1,863
Single-cell chromatin state analysis with Signac
Stuart, T., Srivastava, A., Madad, S., Lareau, C. A., & Satija, R. (2021) · Nature Methods · doi:10.1038/s41592-021-01282-5 · Cited by 1,769
Cells of the adult human heart
Litviňuková, M., Talavera-López, C., Maatz, H., Reichart, D., Worth, C. L., Lindberg, E. L., Kanda, M., Polanski, K., Heinig, M., Lee, M., et al. (2020) · Nature · doi:10.1038/s41586-020-2797-4 · Cited by 1,729
scGPT: Towards Building a Foundation Model for Single-Cell Multi-omics Using Generative AI
Cui, H., Wang, C., Maan, H., Pang, K., Luo, F., & Wang, B. (2023) · doi:10.1101/2023.04.30.538439 · Cited by 1,694
A comparison of single-cell trajectory inference methods
Saelens, W., Cannoodt, R., Todorov, H., & Saeys, Y. (2019) · Nature Biotechnology · doi:10.1038/s41587-019-0071-9 · Cited by 1,670
PanglaoDB: a web server for exploration of mouse and human single-cell RNA sequencing data
Franzén, O., Gan, L.-M., & Björkegren, J. L. M. (2019) · Database · doi:10.1093/database/baz046 · Cited by 1,551
Benchmarking atlas-level data integration in single-cell genomics
Luecken, M. D., Büttner, M., Chaichoompu, K., Danese, A., Interlandi, M., Mueller, M. F., Strobl, D. C., Zappia, L., Dugas, M., Colomé-Tatché, M., et al. (2022) · Nature Methods · doi:10.1038/s41592-021-01336-8 · Cited by 1,438
Effective gene expression prediction from sequence by integrating long-range interactions
Avsec, Ž., Agarwal, V., Visentin, D., Ledsam, J. R., Grabska-Barwinska, A., Taylor, K. R., Assael, Y., Jumper, J., Kohli, P., & Kelley, D. R. (2021) · Nature Methods · doi:10.1038/s41592-021-01252-x · Cited by 1,326
Deciphering cell–cell interactions and communication from gene expression
Armingol, E., Officer, A., Harismendy, O., & Lewis, N. E. (2021) · Nature Reviews Genetics · doi:10.1038/s41576-020-00292-x · Cited by 1,267
Multiplex single-cell profiling of chromatin accessibility by combinatorial cellular indexing
Cusanovich, D. A., Daza, R., Adey, A., Pliner, H. A., Christiansen, L., Gunderson, K. L., Steemers, F. J., Trapnell, C., & Shendure, J. (2015) · Science · doi:10.1126/science.aab1601 · Cited by 1,248
Transfer learning enables predictions in network biology
Theodoris, C. V., Xiao, L., Chopra, A., Chaffin, M. D., Al Sayed, Z. R., Hill, M. C., Mantineo, H., Brydon, E. M., Zeng, Z., Liu, X. S., et al. (2023) · Nature · doi:10.1038/s41586-023-06139-9 · Cited by 1,193
A rapid and robust method for single cell chromatin accessibility profiling
Chen, X., Miragaia, R. J., Natarajan, K. N., & Teichmann, S. A. (2018) · Nature Communications · doi:10.1038/s41467-018-07771-0 · Cited by 1,147
Method of the Year: spatially resolved transcriptomics
Marx, V. (2021) · Nature Methods · doi:10.1038/s41592-020-01033-y · Cited by 1,132
A benchmark of batch-effect correction methods for single-cell RNA sequencing data
Tran, H. T. N., Ang, K. S., Chevrier, M., Zhang, X., Lee, N. Y. S., Goh, M., & Chen, J. (2020) · Genome Biology · doi:10.1186/s13059-019-1850-9 · Cited by 1,113
Finetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101
Construction of a human cell landscape at single-cell level
Han, X., Zhou, Z., Fei, L., Sun, H., Wang, R., Chen, Y., Chen, H., Wang, J., Tang, H., Ge, W., et al. (2020) · Nature · doi:10.1038/s41586-020-2157-4 · Cited by 1,087
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058
Best practices for single-cell analysis across modalities
Heumos, L., Schaar, A. C., Lance, C., Litinetskaya, A., Drost, F., Zappia, L., Lücken, M. D., Strobl, D. C., Henao, J., Curion, F., et al. (2023) · Nature Reviews Genetics · doi:10.1038/s41576-023-00586-w · Cited by 1,051
Attention is not not Explanation
Wiegreffe, S., & Pinter, Y. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1002 · Cited by 966
Deep learning and alignment of spatially resolved single-cell transcriptomes with Tangram
Biancalani, T., Scalia, G., Buffoni, L., Avasthi, R., Lu, Z., Sanger, A., Tokcan, N., Vanderburg, C. R., Segerstolpe, Å., Zhang, M., et al. (2021) · Nature Methods · doi:10.1038/s41592-021-01264-7 · Cited by 931
Splatter: simulation of single-cell RNA sequencing data
Zappia, L., Phipson, B., & Oshlack, A. (2017) · Genome Biology · doi:10.1186/s13059-017-1305-0 · Cited by 905
Single-cell RNA-seq: advances and future challenges
Saliba, A.-E., Westermann, A. J., Gorski, S. A., & Vogel, J. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku555 · Cited by 847
Gene regulatory network inference: Data integration in dynamic models—A review
Hecker, M., Lambeck, S., Toepfer, S., van Someren, E., & Guthke, R. (2009) · Biosystems · doi:10.1016/j.biosystems.2008.12.004 · Cited by 755
Single-cell multi-omics analysis of the immune response in COVID-19
Cambridge Institute of Therapeutic Immunology and Infectious Disease-National Institute of Health Research (CITIID-NIHR) COVID-19 BioResource Collaboration, Stephenson, E., Reynolds, G., Botting, R. A., Calero-Nieto, F. J., Morgan, M. D., Tuong, Z. K., Bach, K., Sungnak, W., Worlock, K. B., et al. (2021) · Nature Medicine · doi:10.1038/s41591-021-01329-2 · Cited by 742
Xu, C., Lopez, R., Mehlman, E., Regier, J., Jordan, M. I., & Yosef, N. (2021) · Molecular Systems Biology · doi:10.15252/msb.20209620 · Cited by 691
Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., et al. (2023) · Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.ijcnlp-main.45 · Cited by 682
Large Scale Foundation Model on Single-cell Transcriptomics
Hao, M., Gong, J., Zeng, X., Liu, C., Guo, Y., Cheng, X., Wang, T., Ma, J., Song, L., & Zhang, X. (2023) · doi:10.1101/2023.05.29.542705 · Cited by 639
Large-scale foundation model on single-cell transcriptomics
Hao, M., Gong, J., Zeng, X., Liu, C., Guo, Y., Cheng, X., Wang, T., Ma, J., Zhang, X., & Song, L. (2024) · Nature Methods · doi:10.1038/s41592-024-02305-7 · Cited by 628
A comparison of automatic cell identification methods for single-cell RNA sequencing data
Abdelaal, T., Michielsen, L., Cats, D., Hoogduin, D., Mei, H., Reinders, M. J. T., & Mahfouz, A. (2019) · Genome Biology · doi:10.1186/s13059-019-1795-z · Cited by 628
Yang, F., Wang, W., Wang, F., Fang, Y., Tang, D., Huang, J., Lu, H., & Yao, J. (2021) · doi:10.1101/2021.12.05.471261 · Cited by 617
Multi-omics single-cell data integration and regulatory inference with graph-linked embedding
Cao, Z.-J., & Gao, G. (2022) · Nature Biotechnology · doi:10.1038/s41587-022-01284-4 · Cited by 616
PaLM-E: An Embodied Multimodal Language Model
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03378 · Cited by 591
Single-cell methylomes identify neuronal subtypes and regulatory elements in mammalian cortex
Luo, C., Keown, C. L., Kurihara, L., Zhou, J., He, Y., Li, J., Castanon, R., Lucero, J., Nery, J. R., Sandoval, J. P., et al. (2017) · Science · doi:10.1126/science.aan3351 · Cited by 545
A test metric for assessing single-cell RNA-seq batch correction
Büttner, M., Miao, Z., Wolf, F. A., Teichmann, S. A., & Theis, F. J. (2019) · Nature Methods · doi:10.1038/s41592-018-0254-1 · Cited by 511
How Is ChatGPT’s Behavior Changing Over Time?
Chen, L., Zaharia, M., & Zou, J. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.5317da47 · Cited by 507
How is ChatGPT's behavior changing over time?
Chen, L., Zaharia, M., & Zou, J. (2023) · arXiv · doi:10.48550/arxiv.2307.09009 · Cited by 507
Supervised classification enables rapid annotation of cell atlases
Pliner, H. A., Shendure, J., & Trapnell, C. (2019) · doi:10.1101/538652 · Cited by 498
Li, B., Zhang, W., Guo, C., Xu, H., Li, L., Fang, M., Hu, Y., Zhang, X., Yao, X., Tang, M., et al. (2022) · Nature Methods · doi:10.1038/s41592-022-01480-9 · Cited by 459
A comprehensive single cell transcriptional landscape of human hematopoietic progenitors
Pellin, D., Loperfido, M., Baricordi, C., Wolock, S. L., Montepeloso, A., Weinberg, O. K., Biffi, A., Klein, A. M., & Biasco, L. (2019) · Nature Communications · doi:10.1038/s41467-019-10291-0 · Cited by 360
Predicting transcriptional outcomes of novel multigene perturbations with GEARS
Roohani, Y., Huang, K., & Leskovec, J. (2024) · Nature Biotechnology · doi:10.1038/s41587-023-01905-6 · Cited by 355
Highly scalable generation of DNA methylation profiles in single cells
Mulqueen, R. M., Pokholok, D., Norberg, S. J., Torkenczy, K. A., Fields, A. J., Sun, D., Sinnamon, J. R., Shendure, J., Trapnell, C., O'Roak, B. J., et al. (2018) · Nature Biotechnology · doi:10.1038/nbt.4112 · Cited by 289
Gene2Vec: Distributed Representation of Genes Based on Co-Expression
Du, J., Jia, P., Dai, Y., Tao, C., Zhao, Z., & Zhi, D. (2018) · doi:10.1101/286096 · Cited by 204
Transformer for one stop interpretable cell type annotation
Chen, J., Xu, H., Tao, W., Chen, Z., Zhao, Y., & Han, J.-D. J. (2023) · Nature Communications · doi:10.1038/s41467-023-35923-4 · Cited by 189
Evaluation of Cell Type Annotation R Packages on Single-Cell RNA-Seq Data
Huang, Q., Liu, Y., Du, Y., & Garmire, L. X. (2021) · Genomics, Proteomics & Bioinformatics · doi:10.1016/j.gpb.2020.07.004 · Cited by 149
Yang, X., Liu, G., Feng, G., Bu, D., Wang, P., Jiang, J., Chen, S., Yang, Q., Zhang, Y., Man, Z., et al. (2023) · doi:10.1101/2023.09.26.559542 · Cited by 148
Self-citationNetwork inference with Granger causality ensembles on single-cell transcriptomics
Deshpande, A., Chu, L.-F., Stewart, R., & Gitter, A. (2022) · Cell Reports · doi:10.1016/j.celrep.2022.110333 · Cited by 130
A comparison of marker gene selection methods for single-cell RNA sequencing data
Pullin, J. M., & McCarthy, D. J. (2022) · doi:10.1101/2022.05.09.490241 · Cited by 122
Contrastive Learning Reduces Hallucination in Conversations
Sun, W., Shi, Z., Gao, S., Ren, P., De Rijke, M., & Ren, Z. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26596 · Cited by 69
Hernandez, D., Kaplan, J., Henighan, T., & McCandlish, S. (2021) · arXiv · doi:10.48550/arxiv.2102.01293 · Cited by 51
Mapping the dynamic genetic regulatory architecture of HLA genes at single-cell resolution
Kang, J. B., Shen, A. Z., Gurajala, S., Nathan, A., Rumker, L., Aguiar, V. R. C., Valencia, C., Lagattuta, K. A., Zhang, F., Jonsson, A. H., et al. (2023) · Nature Genetics · doi:10.1038/s41588-023-01586-6 · Cited by 49
Heimberg, G., Kuo, T., DePianto, D., Heigl, T., Diamant, N., Salem, O., Scalia, G., Biancalani, T., Turley, S., Rock, J., et al. (2023) · doi:10.1101/2023.07.18.549537 · Cited by 40
To Repeat or Not To Repeat: Insights from Scaling LLM under Token-Crisis
Xue, F., Fu, Y., Zhou, W., Zheng, Z., & You, Y. (2023) · arXiv · doi:10.48550/arxiv.2305.13230 · Cited by 22
Zhang, L., Liu, T., He, C. H., Russo, A., Coffey, S., Yin, D., Hsu, I.-U., Zhang, C., Ye, Y., Su, C., et al. (2026) · Journal of Alzheimer’s Disease · doi:10.1177/13872877261480719 · Cited by 22
Self-citation
Research works printed but not linked: 96
- 10.1126/science.aan6828 · DOI · Paper page
- 10.1146/annurev-genom-111320-090436 · DOI · Paper page
- 10.1038/s12276-018-0071-8 · DOI · Paper page
- 10.1038/ncomms14049 · DOI · Paper page
- Teichmann S., 2020, Method of the Year 2019: Single‐Cell Multimodal Omics, Nature Methods, 17, 2020
- 10.1146/annurev-biodatasci-020422-050645 · DOI
- J.Achiam S.Adler S.Agarwal et al. “GPT‐4 technical report ”arXiv preprint arXiv:2303.08774 (2023). · arXiv
- 10.18653/v1/2023.acl-industry.4 · DOI · Paper page
- 10.1016/j.cell.2019.05.031 · DOI · Paper page
- Driess D., Proceedings of the 40th International Conference on Machine Learning (ICML'23), 84698488
- T.Kuzman N.Ljubešić andI.Mozetič “Chatgpt: Beginning of an End of Manual Annotation? Use Case of Automatic Genre Identification ”arXiv preprint arXiv:2303.03953(2023). · arXiv
- 10.1093/bib/bbr012 · DOI · Paper page
- Selva Birunda S., 2021, A Review on Word Embedding Techniques for Text Classification, Innovative Data Communication Technologies and Application: Proceedings of ICIDCA 2020, 267, 10.1007/978-981-15-9651-3_23 · DOI · Paper page
- 10.1038/s41467-023-36476-2 · DOI
- 10.1038/s42256-022-00534-z · doi:10.1038/s42256-022-00534-z · Paper page
- S.Zhao J.Zhang andZ.Nie “Large‐Scale Cell Representation Learning via Divide‐and‐Conquer Contrastive Learning ”arXiv preprint arXiv:2306.04371(2023). · arXiv
- 10.1038/s41592-024-02201-0 · doi:10.1038/s41592-024-02201-0 · Paper page
- 10.1016/j.isci.2023.106536 · DOI
- 10.1038/s41422-024-01034-y · DOI · Paper page
- 10.1038/s41586-024-08411-y · DOI · Paper page
- Y.Rosen Y.Roohani A.Agrawal et al. “Universal cell embeddings: A foundation model for cell biology ”bioRxiv(2023): 2023–11. · DOI · Paper page
- 10.1038/s41467-025-59926-5 · DOI
- H.Wen W.Tang X.Dai et al. “CellPLM: Pre‐Training of Cell Language Model Beyond Single Cells ” inThe Twelfth International Conference on Learning Representations(2023). · DOI · Paper page
- Q.Ye H.Xu G.Xu et al. “mplug‐owl: Modularization Empowers Large Language Models With Multimodality ”arXiv preprint arXiv:2304.14178(2023). · arXiv
- 10.1038/s41586-019-1876-x · DOI · Paper page
- 10.1093/bioinformatics/btac427 · DOI
- D.KingmaandJ.Ba “Adam: A Method for Stochastic Optimization ” inInternational Conference on Learning Representations San Diega CA USA(ICLR 2015).
- S.Ruder “An Overview of Gradient Descent Optimization Algorithms ”arXiv preprint arXiv:1609.04747(2016). · arXiv
- H.Liu Z.Li D. L. W.Hall P.Liang andT.Ma “Sophia: A Scalable Stochastic Second‐Order Optimizer for Language Model Pre‐Training ” inThe Twelfth International Conference on Learning Representations(2024) https://openreview.net/forum?id=3xHDeA8Noi.
- Chen X., 2024, Symbolic Discovery of Optimization Algorithms, Advances in Neural Information Processing Systems, 36, 49205
- 10.1038/s41587-021-01161-6 · DOI · Paper page
- Pedregosa F., 2011, Scikit‐Learn: Machine Learning in Python, Journal of Machine Learning Research, 12, 2825
- 10.1186/s12864-018-5370-x · doi:10.1186/s12864-018-5370-x · Paper page
- 10.1093/nar/gkx1132 · DOI · Paper page
- 10.1002/dvdy.384 · DOI
- 10.1038/s41576-023-00618-5 · DOI · Paper page
- 10.1093/bib/bbac482 · DOI
- 10.1007/s00251-017-1017-3 · DOI · Paper page
- 10.1534/g3.114.015784 · DOI · Paper page
- 10.1038/s41596-023-00853-4 · DOI
- 10.1016/S1672-6529(08)60101-8 · DOI
- 10.1038/ni.3768 · DOI · Paper page
- 10.1038/s41467-023-37168-7 · DOI · Paper page
- Lun A. T., 2016, A Step‐by‐Step Workflow for Low‐Level Analysis of Single‐Cell RNA‐seq Data With Bioconductor, F1000Research, 5, 2122
- 10.1038/nbt.3192 · DOI · Paper page
- 10.1186/s13059-017-1382-0 · DOI · Paper page
- 10.1038/s41587-023-01772-1 · DOI · Paper page
- J.Kaplan S.McCandlish T.Henighan et al. “Scaling Laws for Neural Language Models ”arXiv preprint arXiv:2001.08361(2020). · arXiv
- B.Zhang Z.Liu C.Cherry andO.Firat “When Scaling Meets LLM Finetuning: The Effect of data Model and Finetuning Method ” inThe Twelfth International Conference on Learning Representations(2024) https://openreview.net/forum?id=5HCnKDeTws.
- 10.1038/s41592-024-02191-z · DOI · Paper page
- 10.1038/s41592-025-02899-6 · DOI
- 10.1093/nar/gkae1071 · DOI · Paper page
- 10.1038/s41467-023-41855-w · DOI
- 10.1016/j.cell.2021.04.048 · doi:10.1016/j.cell.2021.04.048 · Paper page
- E. J.Hu yelong shen P.Wallis et al. “LoRA: Low‐Rank Adaptation of Large Language Models ” inInternational Conference on Learning Representations(2022) https://openreview.net/forum?id=nZeVKeeFYf9.
- Z.Hu L.Wang Y.Lan et al. “LLM‐Adapters: An Adapter Family for Parameter‐Efficient Fine‐Tuning of Large Language Models ” inThe 2023 Conference on Empirical Methods in Natural Language Processing(2023). · DOI · Paper page
- 10.1038/s42256-023-00626-4 · DOI · Paper page
- Y.Wang S.Si D.Li et al. “Preserving In‐Context Learning Ability in Large Language Model Fine‐Tuning ”arXiv preprint arXiv:2211.00635(2022). · arXiv
- R.Xu F.Luo Z.Zhang et al. “Raise a Child in Large Language Model: Towards Effective and Generalizable Fine‐Tuning ” inProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing M. F.Moens X.Huang L.Specia andS. W. T.Yih Eds. (Online and Punta Cana Dominican Republic: Association for Computational Linguistics 2021) 9514–9528 https://aclanthology.org/2021.emnlp‐main.749. · DOI
- Y.Ji Y.Deng Y.Gong et al. “Exploring the impact of instruction data scaling on large language models: An empirical study on real‐world use cases ”arXiv preprint arXiv:2303.14742(2023). · arXiv
- T.Carta C.Romac T.Wolf S.Lamprier O.Sigaud andP. Y.Oudeyer “Grounding large language models in interactive environments with online reinforcement learning ” inInternational Conference on Machine Learning(PMLR 2023) 3676–3713.
- Chen Y. T., 2023, Simple and effective embedding model for single‐cell biology built from ChatGPT, Nature biomedical engineering, 483
- 10.1038/s42256-022-00516-1 · DOI · Paper page
- Lundberg S. M., 2017, Proceedings of the 31st Advances in Neural Information Processing Systems, 4768
- M.Sundararajan A.Taly andQ.Yan “Axiomatic Attribution for Deep Networks ” inInternational Conference on Machine Learning(PMLR 2017) 3319–3328.
- S.JainandB. C.Wallace “Attention is not Explanation ” inProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Volume 1 (Long and Short Papers) (Minneapolis Minnesota: Association for Computational Linguistics 2019) 3543–3556 https://aclanthology.org/N19‐1357.
- A.Bibal R.Cardon D.Alfter et al. “Is Attention Explanation? An Introduction to the Debate ” inProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (Dublin Ireland: Association for Computational Linguistics 2022) 3889–3900 https://aclanthology.org/2022.acl‐long.269. · DOI
- T.Huang T.Liu M.Babadi R.Ying andW.Jin “Stpath: A Generative Foundation Model for Integrating Spatial Transcriptomics and Whole Slide Images ”bioRxiv(2025): 2025–04. · doi:10.1101/2025.04.19.649665
- 10.15252/msb.202211517 · DOI · Paper page
- 10.1016/j.neucom.2013.09.055 · DOI · Paper page
- Y.Chang X.Wang J.Wang et al. “A Survey on Evaluation of Large Language Models ”arXiv preprint arXiv:2307.03109(2023). · arXiv
- 10.1186/s13059-024-03183-0 · doi:10.1186/s13059-024-03183-0 · Paper page
- 10.1038/s41592-019-0535-3 · doi:10.1038/s41592-019-0535-3 · Paper page
- N.Ravindra A.Sehanobish J. L.Pappalardo D. A.Hafler andD.vanDijk “Disease State Prediction From Single‐Cell Data Using Graph Attention Networks ” inProceedings of the ACM Conference on Health Inference and Learning(ACM 2020):121–130. · DOI · Paper page
- 10.1186/s12915-018-0564-x · DOI · Paper page
- 10.1038/s41467-023-40503-7 · DOI
- Pearl J., 2018, The Book of Why: The New Science of Cause and Effect
- 10.1038/s41467-018-03405-7 · DOI · Paper page
- 10.1186/s13059-020-02132-x · DOI · Paper page
- 10.1038/s41592-025-02617-2 · DOI · Paper page
- Christiano P. F., 2017, Proceedings of the 31st Advances in Neural Information Processing Systems, 4302
- Radford A., 2019, Language Models are Unsupervised Multitask Learners, OpenAI blog, 1, 9
- K. M.Choromanski V.Likhosherstov D.Dohan et al. “Rethinking Attention With Performers ” inInternational Conference on Learning Representations(2021) https://openreview.net/forum?id=Ua6zuk0WRH.
- Wu C., 2014, Mygene. info: gene annotation query as a service, bioRxiv
- Goodfellow I., 2014, Advances in Neural Information Processing Systems
- Gulrajani I., 2017, the Proceedings of the 31st Advances in Neural Information Processing Systems, 5769
- C.Doersch “Tutorial on Variational Autoencoders ”arXiv preprint arXiv:1606.05908(2016). · arXiv
- D.Misra “Mish: A Self Regularized Non‐Monotonic Activation Function ”arXiv preprint arXiv:1908.08681(2019). · DOI · Paper page
- 10.1017/S1351324916000334 · DOI · Paper page
- Schmidt T., 2007, Coping with Copulas, Copulas‐From Theory to Application in Finance, 3, 1
- 10.1038/s41591-023-02327-2 · DOI · Paper page
- 10.1038/s41592-021-01358-2 · DOI · Paper page
- 10.15252/msb.20199389 · DOI · Paper page
- 10.1038/s41467-021-22368-w · DOI · Paper page
- Zhu Q., 2020, Developmental Trajectory of Prehematopoietic stem Cell Formation From Endothelium, Blood, The Journal of the American Society of Hematology, 136, 845
- 10.1186/s13059-022-02734-7 · DOI · Paper page
Cited by
Ahlmann-Eltze, C., Huber, W., & Anders, S. (2025) · Nature Methods · doi:10.1038/s41592-025-02772-6 · Cited by 238
Benchmarking foundation cell models for post-perturbation RNA-seq prediction
Csendes, G., Sanz, G., Szalay, K. Z., & Szalai, B. (2025) · BMC Genomics · doi:10.1186/s12864-025-11600-2 · Cited by 47
Single-cell foundation models: bringing artificial intelligence into cell biology
Baek, S., Song, K., & Lee, I. (2025) · Experimental & Molecular Medicine · doi:10.1038/s12276-025-01547-5 · Cited by 46
Harnessing the deep learning power of foundation models in single-cell omics
Ma, Q., Jiang, Y., Cheng, H., & Xu, D. (2024) · Nature Reviews Molecular Cell Biology · doi:10.1038/s41580-024-00756-6 · Cited by 34
Wong, D. R., Hill, A. S., & Moccia, R. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf317 · Cited by 27
Metadata
- Authors
- Liu, T., Li, K., Wang, Y., Li, H., & Zhao, H.
- Venue
- Advanced Science
- Publisher
- Wiley
- Published
- 2026-03-23
- Volume
- 13
- Issue
- 27
- Pages
- e14490-e14490
- ISSN
- 2198-3844 (ISSN-L)
- OpenAlex
- W7140132101
- Cited by
- 61
- References
- 71
- FWCI
- 31.83
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0/ (vor)
- doi.wiley.com/10.1002/tdm_license_1.1 (tdm)
- Funders
- National Institutes of Health (R01 GM134005, P50CA196530, U01HG013840)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.