CoChat

scDrugMap: benchmarking large foundation models for drug response prediction

Open access
Wang, Qing, Pan, Yining, Zhou, Minghao, Tang, Zijia, Wang, Yanfei, Wang, Guangyu, Song, Qianqian · 2025 · Nature Communications

The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.

Abstract

Drug resistance remains a significant barrier to improving the effectiveness of cancer therapies. To better understand the biological mechanisms driving resistance, single-cell profiling has emerged as a powerful tool for characterizing cellular heterogeneity. Recent advancements in large-scale foundation models have demonstrated potential in enhancing single-cell analysis, yet their performance in drug response prediction remains underexplored. In this study, we developed scDrugMap, an integrated framework for drug response prediction that features both a Python command-line tool and an interactive web server. scDrugMap supports the evaluation of a wide range of foundation models, including eight single-cell foundation models and two large language models (LLMs), using large-scale single-cell datasets across diverse tissue types, cancer types, and treatment regimens. The framework incorporates a curated data resource consisting of a primary collection of 326,751 cells from 36 datasets across 23 studies, and a validation collection of 18,856 cells from 17 datasets across 6 studies. Using scDrugMap, we conducted comprehensive benchmarking under two evaluation scenarios: pooled-data evaluation and cross-data evaluation. In both settings, we implemented two model training strategies-layer freezing and fine-tuning using Low-Rank Adaptation (LoRA) of foundation models. In the pooled-data evaluation, scFoundation outperformed all others, while most models achieved competitive performance. Specifically, scFoundation achieved the highest mean F1 scores of 0.971 and 0.947 using layer-freezing and fine-tuning, outperforming the lowest-performing model by 54% and 57%, respectively. In the cross-data evaluation, UCE achieved the highest performance (mean F1 score: 0.774) after fine-tuning on tumor tissue, while scGPT demonstrated superior performance (mean F1 score: 0.858) in a zero-shot learning setting. Together, this study presents the first comprehensive benchmarking of large-scale foundation models for drug response prediction in single-cell data and introduces a user-friendly, flexible platform to support drug discovery and translational research.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41467-025-67481-2

Cites

0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Overview of the scDrugMap framework and curated datasets. a Schematic of the scDrugMap framework, which integrates a benchmarking platform, computational pipeline, interactive web server, and curated drug response datasets. Users can select from a range of foundation models (FMs), including single-cell-specific foundation models (scFMs) and general-purpose language models, and apply different training strategies (layer-freezing or fine-tuning) to predict drug response outcomes (sensitive or resistant). b Categories used for benchmarking model performance, including tissue types, drug types, and cancer types. c Summary of the primary dataset collection, showing the number of datasets (left) and number of cells (right) across tissue types, drug types, and cancer types. Dark shades represent the number of sensitive cells and light shades represent the number of resistant cells. d Summary of the validation dataset collection, with the number of datasets (left) and number of cells (right) across tissue types, drug types, and cancer types. Dark shades represent the number of sensitive cells and light shades represent the number of resistant cells. Source data are provided as a Source Data file. FT fine-tuning, PBMC Peripheral blood mononuclear cells, MM Multiple myeloma, ALL Acute lymphoblastic leukemia, CLL Chronic lymphocytic leukemia, AML Acute myeloid leukemia, NSCLC Non-small cell lung cancer, HCC Hepatocellular cancer. Created in BioRender. Song, Q. (2025) https://BioRender.com/vbfzsbb.
Fig. 1 Overview of the scDrugMap framework and curated datasets. a Schematic of the scDrugMap framework, which integrates a benchmarking platform, computational pipeline, interactive web server, and curated drug response datasets. Users can select from a range of foundation models (FMs), including single-cell-specific foundation models (scFMs) and general-purpose language models, and apply different training strategies (layer-freezing or fine-tuning) to predict drug response outcomes (sensitive or resistant). b Categories used for benchmarking model performance, including tissue types, drug types, and cancer types. c Summary of the primary dataset collection, showing the number of datasets (left) and number of cells (right) across tissue types, drug types, and cancer types. Dark shades represent the number of sensitive cells and light shades represent the number of resistant cells. d Summary of the validation dataset collection, with the number of datasets (left) and number of cells (right) across tissue types, drug types, and cancer types. Dark shades represent the number of sensitive cells and light shades represent the number of resistant cells. Source data are provided as a Source Data file. FT fine-tuning, PBMC Peripheral blood mononuclear cells, MM Multiple myeloma, ALL Acute lymphoblastic leukemia, CLL Chronic lymphocytic leukemia, AML Acute myeloid leukemia, NSCLC Non-small cell lung cancer, HCC Hepatocellular cancer. Created in BioRender. Song, Q. (2025) https://BioRender.com/vbfzsbb.CC BY-NC-ND · PubMed Central
Fig. 2 Model performance in predicting drug response in pooled-data evaluation using primary single-cell data. F1 scores across tissue, drug, and cancer types using a layer-freezing and b fine-tuning training method. Error bars on the bar plots/dots on the vertical line charts represent standard deviation of the mean F1 score of each method in each category. Bars represent the mean ± s.e.m. of 10 independent folds from cross-validation. Each dot overlaid on the bars indicates the F1 score from an individual fold (n = 10). The folds represent independent validation splits from the same dataset and are considered technical replicates for model evaluation, not independent biological replicates. Source data are provided as a Source Data file. PBMC Peripheral blood mononuclear cells, MM Multiple myeloma, ALL Acute lymphoblastic leukemia, CLL Chronic lymphocytic leukemia, AML Acute myeloid leukemia.
Fig. 2 Model performance in predicting drug response in pooled-data evaluation using primary single-cell data. F1 scores across tissue, drug, and cancer types using a layer-freezing and b fine-tuning training method. Error bars on the bar plots/dots on the vertical line charts represent standard deviation of the mean F1 score of each method in each category. Bars represent the mean ± s.e.m. of 10 independent folds from cross-validation. Each dot overlaid on the bars indicates the F1 score from an individual fold (n = 10). The folds represent independent validation splits from the same dataset and are considered technical replicates for model evaluation, not independent biological replicates. Source data are provided as a Source Data file. PBMC Peripheral blood mononuclear cells, MM Multiple myeloma, ALL Acute lymphoblastic leukemia, CLL Chronic lymphocytic leukemia, AML Acute myeloid leukemia.CC BY-NC-ND · PubMed Central
Fig. 3 Model performance in predicting drug response in cross-data evaluation using primary single-cell data. F1 scores across tissue, drug, and cancer types using A layer-freezing and B fine-tuning training method. Radar plots represent mean F1 scores for different tissue, drug types, and regimens (the radial axis is scaled from 0–1) based on 10-fold cross-validation (n = 10). Violin plots represent the kernel density distribution and the box plots inside represent the median (center line), upper and lower quartiles and 1.5× the interquartile range (whiskers) for all the cancer types. In the circular bar charts, each color segment showed the mean F1 score of the corresponding category across tissue, drug type, and regimen category. In the boxplots, the middle line is the median, the lower and upper hinges correspond to the first and third quartiles, the upper whisker extends from the hinge to the largest value no further than 1.5× the inter-quartile range (IQR) from the hinge, and the lower whisker extends from the hinge to the smallest value at most 1.5× IQR of the hinge. Replicates are technical replicates (10-fold cross-validation) used to estimate model stability. Source data are provided as a Source Data file. MM Multiple myeloma, ALL Acute lymphoblastic leukemia, CLL Chronic lymphocytic leukemia.
Fig. 3 Model performance in predicting drug response in cross-data evaluation using primary single-cell data. F1 scores across tissue, drug, and cancer types using A layer-freezing and B fine-tuning training method. Radar plots represent mean F1 scores for different tissue, drug types, and regimens (the radial axis is scaled from 0–1) based on 10-fold cross-validation (n = 10). Violin plots represent the kernel density distribution and the box plots inside represent the median (center line), upper and lower quartiles and 1.5× the interquartile range (whiskers) for all the cancer types. In the circular bar charts, each color segment showed the mean F1 score of the corresponding category across tissue, drug type, and regimen category. In the boxplots, the middle line is the median, the lower and upper hinges correspond to the first and third quartiles, the upper whisker extends from the hinge to the largest value no further than 1.5× the inter-quartile range (IQR) from the hinge, and the lower whisker extends from the hinge to the smallest value at most 1.5× IQR of the hinge. Replicates are technical replicates (10-fold cross-validation) used to estimate model stability. Source data are provided as a Source Data file. MM Multiple myeloma, ALL Acute lymphoblastic leukemia, CLL Chronic lymphocytic leukemia.CC BY-NC-ND · PubMed Central
Fig. 4 UMAP projection of primary single-cell data by different methods. UMAP embeddings using layer freezing training method are shown for a scFoundation b scGPT and c UCE, with cells colored cancer type and cell response. Source data are provided as a Source Data file. MM Multiple myeloma, ALL Acute lymphoblastic leukemia, CLL Chronic lymphocytic leukemia, AML Acute myeloid leukemia.
Fig. 4 UMAP projection of primary single-cell data by different methods. UMAP embeddings using layer freezing training method are shown for a scFoundation b scGPT and c UCE, with cells colored cancer type and cell response. Source data are provided as a Source Data file. MM Multiple myeloma, ALL Acute lymphoblastic leukemia, CLL Chronic lymphocytic leukemia, AML Acute myeloid leukemia.CC BY-NC-ND · PubMed Central
Fig. 5 Performance of GPT4o-mini with few-shots learning in pooled-data evaluation using primary single-cell data. A Prompt display used for GPT4o-mini. For the prompt word template, we first use a technique like the thought chain to prompt the model how it should think about the output, then tell the model the data source and sequence information, and finally we repeat telling the model and give an output template to ensure the consistent of the output format. A complete input example with prompt is also showed. b Radar plots showing the mean F1 scores of GPT4o-mini in predicting drug response across different tissue, drug types, and cancer types (the radial axis is scaled from 0–1). Source data are provided as a Source Data file.
Fig. 5 Performance of GPT4o-mini with few-shots learning in pooled-data evaluation using primary single-cell data. A Prompt display used for GPT4o-mini. For the prompt word template, we first use a technique like the thought chain to prompt the model how it should think about the output, then tell the model the data source and sequence information, and finally we repeat telling the model and give an output template to ensure the consistent of the output format. A complete input example with prompt is also showed. b Radar plots showing the mean F1 scores of GPT4o-mini in predicting drug response across different tissue, drug types, and cancer types (the radial axis is scaled from 0–1). Source data are provided as a Source Data file.CC BY-NC-ND · PubMed Central
Fig. 6 Model performance in predicting drug response using the validation data collection. Radar plots illustrate mean a F1 scores and b AUROC of each model in predicting single-cell drug response using layer-freezing training method across different tissue, drug types, and cancer types. Each radial axis represents the mean of the corresponding scores (0–1, where 1= best performance). The outermost vertices indicate the maximum value for the metric in the given data type. The lines are colored by categories in each data type.
Fig. 6 Model performance in predicting drug response using the validation data collection. Radar plots illustrate mean a F1 scores and b AUROC of each model in predicting single-cell drug response using layer-freezing training method across different tissue, drug types, and cancer types. Each radial axis represents the mean of the corresponding scores (0–1, where 1= best performance). The outermost vertices indicate the maximum value for the metric in the given data type. The lines are colored by categories in each data type.CC BY-NC-ND · PubMed Central
Fig. 7 Summary of properties, computational efficiency, and scalability of each evaluated model. Rows correspond to algorithms ordered chronologically by year and months of publication. The first three columns display model characteristics: whether it uses an encoder-decoder architecture, the type of input embeddings, and whether it is a single-cell foundation model. The next two columns present parameters and output dimensions for each model. The next set of columns show the training and inference time and speed. For each model, the color in each cell is proportional to the corresponding value (scaled between corresponding minimum and maximum values, ignoring values of the two natural language models, shown as dashes). Source data are provided as a Source Data file.
Fig. 7 Summary of properties, computational efficiency, and scalability of each evaluated model. Rows correspond to algorithms ordered chronologically by year and months of publication. The first three columns display model characteristics: whether it uses an encoder-decoder architecture, the type of input embeddings, and whether it is a single-cell foundation model. The next two columns present parameters and output dimensions for each model. The next set of columns show the training and inference time and speed. For each model, the color in each cell is proportional to the corresponding value (scaled between corresponding minimum and maximum values, ignoring values of the two natural language models, shown as dashes). Source data are provided as a Source Data file.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 7

  1. Cui, H. et al. scGPT: toward building a foundation model for single-cell multi-omics using generative AI. Nat. Methods 21, 1470–1480 (2024). · doi:10.1038/s41592-024-02201-0 · Paper page
  2. Zhao, S., Zhang, J. & Nie, Z. Large-scale cell representation learning via divide-and-conquer contrastive learning. arXiv preprint arXiv:2306.04371 (2023). · arXiv
  3. Wen, H. et al. CellPLM: pre-training of cell language model beyond single cells. bioRxiv, 2023.2010. 2003.560734 (2023). · DOI
  4. Rosen, Y. et al. Universal cell embeddings: A foundation model for cell biology. bioRxiv, 2023.2011. 2028.568918 (2023). · DOI · Paper page
  5. Shen, H. et al. Generative pretraining from large-scale transcriptomes: Implications for single-cell deciphering and clinical translation. bioRxiv, 2022.2001. 2031.478596 (2022). · DOI
  6. Hu, E. J. et al. LoRA: Low-Rank Adaptation of Large Language Models. In Proceedings of International Conference on Learning Representations (2022).
  7. Boerner, T. J., Deems, S., Furlani, T. R., Knuth, S. L. & Towns, J. in Practice and Experience in Advanced Research Computing 2023: Computing for the Common Good 173–176 (Association for Computing Machinery, Portland, OR, USA, 2023). · DOI

Cited by

See all 16

Metadata

Authors
Wang, Q., Pan, Y., Zhou, M., Tang, Z., Wang, Y., Wang, G., & Song, Q.
Affiliations
  • University of Florida Health (US)
  • Duke University (US)
  • Houston Methodist (US)
Corresponding author
Song, Qianqian
Venue
Nature Communications
Published
2025-12-11
Volume
17
Issue
1
Pages
730-730
ISSN
2041-1723 (ISSN-L)
OpenAlex
W4410510777
Cited by
12
References
0
Citations by year
2026: 12
FWCI
3.61
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Communications (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion)
  • PubMed Central (submittedVersion, cc-by-nc-nd)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
National Institute of General Medical Sciences (R35GM151089, R35GM150460)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
  • Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Drug Discovery › Computational Drug Discovery Methods
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
drug response prediction, drug resistance, cancer therapy, single-cell data, foundation models, large language models, benchmarking, scGPT, ultraconserved elements, low-rank adaptation, layer freezing, drug discovery

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: scDrugMap: benchmarking large foundation models for drug response prediction. https://app.cochat.ai/papers/10.1038/s41467-025-67481-2
The paper
Wang, Q., Pan, Y., Zhou, M., Tang, Z., Wang, Y., Wang, G., & Song, Q. (2025). scDrugMap: benchmarking large foundation models for drug response prediction. Nature Communications. https://doi.org/10.1038/s41467-025-67481-2

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.