Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 114 of the 114 references it cites were found in a registry or an index; 11 of the 16 sources its claims cite were checked for support.

Large-scale foundation model on single-cell transcriptomics

Open access

Abstract

Large pretrained models have become foundation models leading to breakthroughs in natural language processing and related fields. Developing foundation models for deciphering the 'languages' of cells and facilitating biomedical research is promising yet challenging. Here we developed a large pretrained model scFoundation, also named 'xTrimoscFoundation α ', with 100 million parameters covering about 20,000 genes, pretrained on over 50 million human single-cell transcriptomic profiles. scFoundation is a large-scale model in terms of the size of trainable parameters, dimensionality of genes and volume of training data. Its asymmetric transformer-like architecture and pretraining task design empower effectively capturing complex context relations among genes in a variety of cell types and states. Experiments showed its merit as a foundation model that achieved state-of-the-art performances in a diverse array of single-cell analysis tasks such as gene expression enhancement, tissue drug response prediction, single-cell drug response classification, single-cell perturbation prediction, cell type annotation and gene module inference.

Overview

CoChat Summary

TL;DR

scFoundation is a 100M-parameter foundation model pretrained on over 50 million human single-cell transcriptomics profiles covering 19,264 genes. It achieves improved or state-of-the-art performance across gene expression enhancement, cancer drug response prediction, single-cell drug response classification, and perturbation prediction without fine-tuning the pretrained model.

Key findings

  • Pretrained on >50 million human scRNA-seq profiles and 19,264 genes with 100M parameters, scFoundation is described as the largest model in parameter size, gene coverage, and data scale in the single-cell field.
  • For read-depth enhancement without fine-tuning, scFoundation reduced MAE and MRE by half versus the non-enhanced baseline; when downsampling was below 10%, baseline MRE was 97.5% while scFoundation maintained an MRE of 60%. On the Zheng68K PBMC dataset (~600,000 cells), scFoundation embeddings had comparable NMI and ARI to scVI and a higher SIL score.
  • In cancer drug response prediction with DeepCDR, scFoundation embeddings improved PCC for most drugs and all cancer types; in leave-one-drug-out blind tests, PHA-793887 improved from PCC 0.07 to 0.73 and Zobotentan from 0.49 to 0.64, with the best case achieving PCC above 0.93.
  • In single-cell drug response classification, scFoundation embeddings achieved higher AUROC for all four tested drugs; for NVP-TAE684 and Sorafenib, AUROC improved by above 0.2, and Spearman correlations with EpiSen scores were 0.56 and −0.55 versus baseline 0.24 and −0.06. In perturbation prediction, scFoundation-combined GEARS achieved lower MSE on all 1-gene perturbation datasets and the lowest MSE in the 0/2 unseen 2-gene case.

Methods

  • Model architecture: xTrimoGene, an asymmetric encoder-decoder transformer with an embedding module that converts continuous gene expression scalars into learnable embeddings; the encoder processes only non-zero and non-masked expressed genes, while the decoder integrates all genes.
  • Pretraining data and task: over 50 million human scRNA-seq profiles collected from GEO, Single Cell Portal, HCA, EMBL-EBI, and other resources, aligned to 19,264 genes and filtered for cells with >200 expressed genes; read-depth-aware modeling used hierarchical Bayesian downsampling, 30% masking, and MSE regression loss with total count indicators T and S.
  • Downstream evaluation: pretrained cell or gene embeddings were fed into DeepCDR, SCAD, and GEARS without fine-tuning the pretrained model; metrics included MAE, MRE, PCC, NMI, ARI, SIL, AUROC, MSE, and Spearman correlation.

Limitations

  • The paper does not include an explicit limitations section.
  • Evident from the text: pretraining used only human scRNA-seq data, so generalization to other species or multi-omics settings is not demonstrated.
  • Evident from the text: downstream performance depends on existing task-specific models (DeepCDR, SCAD, GEARS) and public datasets, and the paper does not quantify residual batch effects or technical noise beyond describing read-depth-aware pretraining as a mitigation.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41592-024-02305-7

Cites

114 of 114 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 The schematic overview of the pre-training framework. A) 50 million single-cell gene expression profiles were collected, covering tumor and non-tumor cells from various tissues. These data were used for the read-depth-aware (RDA) task to pre-train the model. In the RDA task, the input consists of the masked gene expression vector and two total count indicators (T and S). The output is the predicted expression value for all genes, and the loss is computed at the masked positions. B) Outline of the pre-training process. A raw gene expression vector serves as a training sample. A hierarchical Bayesian downsampling strategy generates the input sample. The gene expression total count values (T and S) of the raw and input samples are computed. Values in the input sample are randomly masked. The scalar values are converted into embeddings. Only embeddings corresponding to non-zero and non-masked values (including T and S) are fed into the model encoder. The output embeddings of the encoder are then combined with mask and zero embeddings and fed into the decoder. Also, the encoder output can be pooled to generate a cell embedding for downstream usage. The decoder output embeddings are projected to the gene expression value via a shared multilayer perceptron (MLP) layer. The regression loss between the predicted and raw sample's gene expression values is computed. C) The pre-training embeddings can be leveraged as substitutes for the gene expression profiles, facilitating read depth enhanced cell clustering, drug response prediction, and perturbation prediction.
Figure 1 The schematic overview of the pre-training framework. A) 50 million single-cell gene expression profiles were collected, covering tumor and non-tumor cells from various tissues. These data were used for the read-depth-aware (RDA) task to pre-train the model. In the RDA task, the input consists of the masked gene expression vector and two total count indicators (T and S). The output is the predicted expression value for all genes, and the loss is computed at the masked positions. B) Outline of the pre-training process. A raw gene expression vector serves as a training sample. A hierarchical Bayesian downsampling strategy generates the input sample. The gene expression total count values (T and S) of the raw and input samples are computed. Values in the input sample are randomly masked. The scalar values are converted into embeddings. Only embeddings corresponding to non-zero and non-masked values (including T and S) are fed into the model encoder. The output embeddings of the encoder are then combined with mask and zero embeddings and fed into the decoder. Also, the encoder output can be pooled to generate a cell embedding for downstream usage. The decoder output embeddings are projected to the gene expression value via a shared multilayer perceptron (MLP) layer. The regression loss between the predicted and raw sample's gene expression values is computed. C) The pre-training embeddings can be leveraged as substitutes for the gene expression profiles, facilitating read depth enhanced cell clustering, drug response prediction, and perturbation prediction.CC-BY · From the PDF
Figure 2 A) Training loss under different parameter sizes and FLOPs. The dots noted as previous models are xTrimoGene architecture models with parameter size equivalent to their scale. B) Evaluation of read depth enhancement performance on an unseen dataset. Mean relative error among non-zero expressed genes and Pearson correlation among all genes were used as metrics to evaluate the recovered gene expression performance. Lower error values indicate better performance, while higher Pearson correlation values indicate better performance. C) Comparison of the scFoundation model with other imputation methods based on cell clustering metrics. The x-axis represents the fold change between the desired total count value and the input total count value, while the y-axis represents the score. D) UMAP plots of cell embeddings generated
Figure 2 A) Training loss under different parameter sizes and FLOPs. The dots noted as previous models are xTrimoGene architecture models with parameter size equivalent to their scale. B) Evaluation of read depth enhancement performance on an unseen dataset. Mean relative error among non-zero expressed genes and Pearson correlation among all genes were used as metrics to evaluate the recovered gene expression performance. Lower error values indicate better performance, while higher Pearson correlation values indicate better performance. C) Comparison of the scFoundation model with other imputation methods based on cell clustering metrics. The x-axis represents the fold change between the desired total count value and the input total count value, while the y-axis represents the score. D) UMAP plots of cell embeddings generatedCC-BY · From the PDF
Figure 3 Performance of drug response prediction using scFoundation embeddings. A) Illustration of the embeddingbased CDR prediction model. B) Pearson correlation coefficient (PCC) between all drugs and cancer types in the test set. Each dot represents a drug or cancer type, with the x-axis and y-axis showing the PCC obtained by DeepCDR using gene expression and scFoundation embeddings, respectively. C) Comparison of predicted and observed IC50 values for the
Figure 3 Performance of drug response prediction using scFoundation embeddings. A) Illustration of the embeddingbased CDR prediction model. B) Pearson correlation coefficient (PCC) between all drugs and cancer types in the test set. Each dot represents a drug or cancer type, with the x-axis and y-axis showing the PCC obtained by DeepCDR using gene expression and scFoundation embeddings, respectively. C) Comparison of predicted and observed IC50 values for theCC-BY · From the PDF
Figure 4 Single-cell drug response classification tasks based on scFoundation cell embeddings. A) Illustration of the embedding-based single-cell response classification model. B) Receiver operating characteristic (ROC) curves for the
Figure 4 Single-cell drug response classification tasks based on scFoundation cell embeddings. A) Illustration of the embedding-based single-cell response classification model. B) Receiver operating characteristic (ROC) curves for theCC-BY · From the PDF
Figure 5 Perturbation prediction tasks using scFoundation gene context embeddings. A) Illustration of the perturbation prediction model based on cell-specific gene embeddings. B) Mean square error between predicted and ground truth postgene expressions. Results given by the GEARS model using scFoundation cell-specific gene embeddings and gene expression are shown in red and blue, respectively. C) Proportion of predicted values within (+-5%) of the true mean expression value of the top 20 differentially expressed (DE) genes. The black dashed line represents the expected percentage (10%). Colors represent the same results as in panel A. D) Predicted gene expression over control for the top
Figure 5 Perturbation prediction tasks using scFoundation gene context embeddings. A) Illustration of the perturbation prediction model based on cell-specific gene embeddings. B) Mean square error between predicted and ground truth postgene expressions. Results given by the GEARS model using scFoundation cell-specific gene embeddings and gene expression are shown in red and blue, respectively. C) Proportion of predicted values within (+-5%) of the true mean expression value of the top 20 differentially expressed (DE) genes. The black dashed line represents the expected percentage (10%). Colors represent the same results as in panel A. D) Predicted gene expression over control for the topCC-BY · From the PDF

Verification

Claims the paper makes

2 Valid · 7 Not valid · 2 Not checked · 74 Own results and interpretations

37 more claims

Create a free account to see all 85 claims and to verify the 3 references still open.

See all claims — free account
  • Probably not a substantial claim. 13
  • It may state two separate claims as one. 36
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 93 references

Research works printed but not linked: 3

  1. He, K. et al. in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 16000–16009 (IEEE, 2022).
  2. Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. in Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics 4171–4186 (ACL, 2019).
  3. Saddoughi, S. A., Song, P. & Ogretmen, B. in Lipids in Health and Disease (eds Quinn, P. J. & Wang, X.) 413–440 (Springer, 2008).

Cited by

See all 639

Metadata

Authors
Hao, M., Gong, J., Zeng, X., Liu, C., Guo, Y., Cheng, X., Wang, T., Ma, J., Zhang, X., & Song, L.
Affiliations
  • Tsinghua University (CN)
  • Beijing National Research Center for Information Science and Technology (CN)
  • Mohamed bin Zayed University of Artificial Intelligence (AE)
Corresponding author
  • Ma, Jianzhu
  • Zhang, Xuegong
  • Song, Le
Venue
Nature Methods
Published
2024-06-06
Volume
21
Issue
8
Pages
1481-1491
ISSN
  • 1548-7091 (ISSN-L)
  • 1548-7105
OpenAlex
W4399387478
Cited by
628
References
93
Citations by year
2026: 286 · 2025: 247 · 2024: 89
FWCI
71.63
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Methods (publishedVersion)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
  • National Natural Science Foundation of China (61721003, 62250005)
  • Tsinghua University
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
Keywords
single-cell transcriptomics, foundation models, transformer models, gene expression enhancement, drug response prediction, cell type annotation
MeSH terms
Algorithms, Humans, Computational Biology, Gene Expression Profiling, Single-Cell Analysis, Transcriptome

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-11-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Large-scale foundation model on single-cell transcriptomics. https://app.cochat.ai/papers/10.1038/s41592-024-02305-7
The paper
Hao, M., Gong, J., Zeng, X., Liu, C., Guo, Y., Cheng, X., Wang, T., Ma, J., Zhang, X., & Song, L. (2024). Large-scale foundation model on single-cell transcriptomics. Nature Methods. https://doi.org/10.1038/s41592-024-02305-7

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.