Simple controls exceed best deep learning algorithms and reveal foundation model effectiveness for predicting genetic perturbations
Open accessPublished version of a preprint
The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Abstract
Motivation: Modeling genetic perturbations and their effect on the transcriptome is a key area of pharmaceutical research. Due to the complexity of the transcriptome, there has been much excitement and development in deep learning (DL) because of its ability to model complex relationships. In particular, the transformer-based foundation model paradigm emerged as the gold-standard of predicting post-perturbation responses. However, understanding these increasingly complex models and evaluating their practical utility is lacking, along with simple but appropriate benchmarks to compare predictive methods. Results: Here, we present a simple baseline method that outperforms both state of the art (SOTA) in DL and other proposed simpler neural architectures, setting a necessary benchmark to evaluate in the field of post-perturbation prediction. We also elucidate the utility of foundation models for the task of post-perturbation prediction via generalizable fine-tuning experiments that can be translated to different applications of transformer-based foundation models to tasks of interest. Furthermore, we provide a corrected version of a popular dataset used for benchmarking perturbation prediction models. Our hope is that this work will properly contextualize further development of DL models in the perturbation space with necessary control procedures. Availability and implementation: All source code is available at: https://github.com/pfizer-opensource/perturb_seq. The DOI is 10.5281/zenodo.15352937.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Since the release of our paper as a preprint, several other benchmarks15,16,22–27 were released that also show that deep learning models struggle to outperform simple baselines.”
From Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines (2025) · cites this paper as [27]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Evolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,990
Genome-Scale CRISPR-Mediated Control of Gene Repression and Activation
Gilbert, L. A., Horlbeck, M. A., Adamson, B., Villalta, J. E., Chen, Y., Whitehead, E. H., Guimaraes, C., Panning, B., Ploegh, H. L., Bassik, M. C., et al. (2014) · Cell · doi:10.1016/j.cell.2014.09.029 · Cited by 2,728 · cited 1×
“Since CRISPRi in practice hardly eliminates expression in full 48 and CRISPRa has a wide range of enhancement based on the biological context 49 , this model is an overly simplistic approximation.”
Dixit, A., Parnas, O., Li, B., Chen, J., Fulco, C. P., Jerby-Arnon, L., Marjanovic, N. D., Dionne, D., Burks, T., Raychowdhury, R., et al. (2016) · Cell · doi:10.1016/j.cell.2016.11.038 · Cited by 1,863 · cited 2×
“Perturb-seq 4 emerged as a tool to aid this discovery effort by measuring complex post-perturbation transcription profiles.”
scGPT: Towards Building a Foundation Model for Single-Cell Multi-omics Using Generative AI
Cui, H., Wang, C., Maan, H., Pang, K., Luo, F., & Wang, B. (2023) · doi:10.1101/2023.04.30.538439 · Cited by 1,694
Effective gene expression prediction from sequence by integrating long-range interactions
Avsec, Ž., Agarwal, V., Visentin, D., Ledsam, J. R., Grabska-Barwinska, A., Taylor, K. R., Assael, Y., Jumper, J., Kohli, P., & Kelley, D. R. (2021) · Nature Methods · doi:10.1038/s41592-021-01252-x · Cited by 1,326
CRISPR interference (CRISPRi) for sequence-specific control of gene expression
Larson, M. H., Gilbert, L. A., Wang, X., Lim, W. A., Weissman, J. S., & Qi, L. S. (2013) · Nature Protocols · doi:10.1038/nprot.2013.132 · Cited by 1,212
Adamson, B., Norman, T. M., Jost, M., Cho, M. Y., Nuñez, J. K., Chen, Y., Villalta, J. E., Gilbert, L. A., Horlbeck, M. A., Hein, M. Y., et al. (2016) · Cell · doi:10.1016/j.cell.2016.11.048 · Cited by 1,193 · cited 1×
“Many of these mismatches were the result of cells mislabeled as “ctrl” (i.e. control) in GEARS, but with gene perturbations documented in the cell_identities.csv files provided in the original Adamson et al. paper 45 .”
Transfer learning enables predictions in network biology
Theodoris, C. V., Xiao, L., Chopra, A., Chaffin, M. D., Al Sayed, Z. R., Hill, M. C., Mantineo, H., Brydon, E. M., Zeng, Z., Liu, X. S., et al. (2023) · Nature · doi:10.1038/s41586-023-06139-9 · Cited by 1,193 · cited 1×
“As a result of the excitement around foundation models in biology, there has been a large uptake of studies and methods that model single-cell biology via transformers 23,26–31 .”
Mapping information-rich genotype-phenotype landscapes with genome-scale Perturb-seq
Replogle, J. M., Saunders, R. A., Pogson, A. N., Hussmann, J. A., Lenail, A., Guna, A., Mascibroda, L., Wagner, E. J., Adelman, K., Lithwick-Yanai, G., et al. (2022) · Cell · doi:10.1016/j.cell.2022.05.013 · Cited by 795
Harnessing the power of llms in practice: a survey on chatGPT and beyond
Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Zhong, S., Yin, B., & Hu, X. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3649506 · Cited by 644 · cited 1×
“As excitement around large language model (LLM) success in Natural Language Processing grows 15 , so too do efforts of applying core concepts of LLMs to the “language” of biology 16–20 .”
Research works printed but not linked: 12
- Anil R, Dai A, Firat O et al PaLM 2 Technical Report. arXiv [cs.CL], 2023.
- Bendidi I, Whitfield S, Kenyon-Dean K et al Benchmarking Transcriptomics Foundation Models for Perturbation Analysis: one PCA still rules them all. arXiv [cs.LG], 2024.
- Biderman D, Portes J, Ortiz JJG et al LoRA learns less and forgets less. arXiv [cs.LG], 2024.
- Bommasani R, Hudson DA, Adeli E et al On the opportunities and risks of foundation models. arXiv [cs.LG], 2021.
- Bunne C, Roohani Y, Rosen Y et al How to build the virtual cell with artificial intelligence: Priorities and opportunities. arXiv [q-bio.QM], 2024. PMC12148494 39672099
- Dosovitskiy A, Beyer L, Kolesnikov A et al An image is worth 16 × 16 words: Transformers for image recognition at scale. arXiv [cs.CV], 2020.
- Hu EJ, Shen Y, Wallis P et al LoRA: Low-Rank Adaptation of large language models. arXiv [cs.CL], 2021.
- Huang C-ZA, Vaswani A, Uszkoreit J et al Music Transformer. arXiv [cs.LG], 2018.
- Kaplan J, McCandlish S, Henighan T et al Scaling laws for neural language models. arXiv [cs.LG], 2020.
- Li C, Gao H, She Y et al Benchmarking AI models forin silicogene perturbation of cells. bioRxiv 2024.12.20.629581, 2024a.
- Wu Y, Wershof E, Schmon SM et al PerturBench: Benchmarking machine learning models for cellular perturbation analysis. arXiv [cs.LG], 2024.
- Xia W, Qin C, Hazan E. Chain of LoRA: Efficient fine-tuning of language models via residual learning. arXiv [cs.LG], 2024.
Cited by
Ahlmann-Eltze, C., Huber, W., & Anders, S. (2025) · Nature Methods · doi:10.1038/s41592-025-02772-6 · Cited by 238
TxPert: using multiple knowledge graphs for prediction of transcriptomic perturbation effects
Wenkel, F., Tu, W., Masschelein, C., Shirzad, H., Hodgson, L. M., Bendidi, I., Eastwood, C., Whitfield, S. T., Russell, C., Mesbahi, Y. E., et al. (2026) · Nature Biotechnology · doi:10.1038/s41587-026-03113-4 · Cited by 57
A comparison of computational methods for expression forecasting
Kernfeld, E. M., Yang, Y., Weinstock, J. S., Battle, A. J., & Cahan, P. (2025) · Genome biology · doi:10.1186/s13059-025-03840-y · Cited by 51
Yiu, T., Chen, B., Wang, H., Feng, G., Fu, Q.-Q., & Hu, H. (2025) · Journal of Translational Medicine · doi:10.1186/s12967-025-07091-0 · Cited by 42
A Systematic Comparison of Single-Cell Perturbation Response Prediction Models
Li, L., You, Y., Fu, Y., Liao, W., Fan, X., Lu, S., Cao, Y., Li, B., Ren, W., Kong, J., et al. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.12.23.630036 · Cited by 25
“In single-cell perturbation prediction tasks, baseline models provide reference performance levels that facilitate the evaluation of more expressive predictive models ( 28, 52–54 ).”
Virtual Cells Need Context, Not Just ScalePreprint
Dibaeinia, P., Babu, S., Knudson, M. K., ElSheikh, A., Wen, Y., Liu, H., Perera, J. F., & Khan, A. A. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.02.04.703804 · Cited by 28
Shang, E., Wei, Y., & Roeder, K. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2525268122 · Cited by 17
Data-driven discovery of digital twins in biomedical research
Métayer, C., Ballesta, A., & Martinelli, J. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf722 · Cited by 15
Signal, Bounds, and Baselines: Principles for Evaluating Virtual Cell Perturbation ModelsPreprint
Vollenweider, M., & Bühlmann, P. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.04.20.719650 · Cited by 11
Mapping transcriptional responses to cellular perturbations with RNA fingerprinting
Grabski, I. N., Lee, J., Blair, J. D., Dalgarno, C., Mascio, I., Bradu, A., Knowles, D. A., & Satija, R. (2026) · Cell · doi:10.1016/j.cell.2026.09.017 · Cited by 8
Benchmarking biomedical foundation models
Sáez-Rodríguez, J., Schäfer, P. S. L., Kalavros, N., & Stolovitzky, G. A. (2026) · Nature Methods · doi:10.1038/s41592-026-03182-y · Cited by 7
SpatialProp: tissue perturbation modeling with spatially resolved single-cell transcriptomicsPreprint
Sun, E., Buendia, A., Brunet, A., & Zou, J. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2025.11.30.691355 · Cited by 9
Towards Principled Evaluation of Single-Cell Perturbation Prediction ModelsPreprint
Schäfer, P. S. L., Reid, K. A., Boldyga, Z., Aksu, E. D., Hakem, H., & Sáez-Rodríguez, J. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.07.23.740433 · Cited by 6
Noori, A., Fishman, N., Fang, A., Fesser, L., & Žitnik, M. (2026) · Cell · doi:10.1016/j.cell.2026.08.032 · Cited by 4
Sonder, E., Aymergen, İ. G., Sun, J., Feuvrier, A., Schratt, G. M., Gapp, K., Bohacek, J., Robinson, M. D., & Germain, P. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.06.16.732527 · Cited by 3
Wong, D., Piper, M., Qiao, J., Russo, M., Jean, P., Clevert, D.-A., Arroyo, J. D., & Pashos, E. E. (2026) · Digital Discovery · doi:10.1039/d6dd00287k · Cited by 2
Self-citationGenerative genomics accurately predicts future experimental results
Koytiger, G., Walsh, A. M., Marar, V., Johnson, K. A., Highsmith, M. R., Abbas, A., Stirn, A., Brumbaugh, A. R., David, A., Hui, D., et al. (2025) · doi:10.7490/f1000research.1120377.1 · Cited by 3
Pang, J., Qiu, P. S., He, Y., Deng, Y., Tang, W., Huang, Z., Yan, J., Li, B., Lin, A., Cao, L., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2025.12.29.696804 · Cited by 3
A systematic comparison of single-cell perturbation response prediction models
Li, L., You, Y., Fu, Y., Liao, W., Fan, X., Lu, S., Cao, Y., Li, B., Ren, W., Kong, J., et al. (2026) · Science Advances · doi:10.1126/sciadv.aed3414 · Cited by 1
Zhao, M., Ma, X., Shi, W., & Sun, Y. (2026) · Life Sciences · doi:10.1016/j.lfs.2026.124600 · Cited by 1
Zahavi, L., Levine, Z., Godneva, A., Dubinkina, V. B., Dhir, R., Pollard, K. S., Weinberger, A., & Segal, E. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.10.19.683269 · Cited by 2
Jiang, X., & Xie, J. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.04.12.718016 · Cited by 2
High-resolution dissection of concept acquisition in different families of protein language modelsPreprint
Whitfield, S., Marty, T., Vernon, R. M., LANGMEAD, C. J., Sridhar, D., & Fournier, Q. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.07.20.739599 · Cited by 1
Artificial Intelligence in Pharmaceutical Modelling
Alshehade, S. A., Qi, L. W., Qi, L. Q., Jun, R. S. R., Kee, C. P., Dayoob, M., Alshehade, H., Jabal, G. A. A., Aziz, S., & Alshehade, R. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-96631-6_22
DINOcell: learning generalizable perturbation effects through self-distillationPreprint
Li, J., Tasdemir, N., Kyriazopoulou-Panagiotopoulou, S., Xu, S., Merrell, A. J., Susanto, J., Cass, A. A., DeTomaso, D., Steach, H., Chow, J. C., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2025.12.16.694747
Wang, X., Tan, R., & Cristea, S. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.02.12.705621
Magnitov, M. D., van der Weide, R. H., Witvliet, A. F., Hernández-Quiles, M., Martinovic, M., Teunissen, H., Braccioli, L., Vermeulen, M., & de Wit, E. (2026) · Nature Communications · doi:10.1038/s41467-026-72392-x
Epigenomic regulatory programs reveal the latent heterogeneity of complex diseasesPreprint
Shim, W. J., Bao, S. C., Chow, C. S. Y., Shen, S., Riedlshah, Z., Zhao, Q., Zeng, J., Shah, S. H., Mizikovsky, D., Bodén, M., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.05.01.722123
Computational modelling of cell identity
Shim, W. J., Chow, C. S. Y., Bao, S. C., Zhao, Q., & Palpant, N. J. (2026) · Biochemical Journal · doi:10.1042/bcj20250132
A benchmarking framework for perturbational signature recontextualization across biological contextsPreprint
Chen, A., Rappaport, N., Schork, N. J., Girke, T., & Monti, S. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.08.14.744937
Li, L., Duan, S., Zha, X., Ye, F., Zhang, Y., Zhang, X., Cao, Y., Liu, C., & Fang, B. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.08.19.745729
The Unreasonable Effectiveness of Cell Types in Describing Neuronal Physiological FeaturesPreprint
Sharma, H., Liu, X.-P., Chartrand, T., Kalmbach, B. E., Lein, E. S., Mihalaş, Ş., & Lu, Z. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.08.27.744753
A transcription factor regulatory atlas for activity inference and perturbation prediction
Sugimoto, H., Tsuyuzaki, K., Zou, Z., Oki, S., Ohta, T., & Kawakami, E. (2026) · Nucleic Acids Research · doi:10.1093/nar/gkag897
Zhu, Y., Han, D., & Wang, Z. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag513
MRFlow: Mechanism-Structured Residual Flow Matching for Virtual Cell Perturbation PredictionPreprint
Yang, H., & Su, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.13966
On the emergence of in silico biology for intervention in living systemsPreprint
Colter, J., Rosin, N. L., Biernaskie, J. A., & Forkert, N. D. (2026) · Research Square · doi:10.21203/rs.3.rs-11321044/v1
How Much Can AI Virtual Cells Gain from Unified Perturbational Modeling?Preprint
魏宗婷, Ye, Y., Cui, S. R., Ma, W., & You, Y. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.10.25.761625
A low-rank decoder bottleneck bounds reliability in foundation-model perturbation predictionPreprint
Zhang, Z., Huang, Y., Feng, F., & Liu, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2611.05849
Lei, Y., Liang, H., Peng, K., & Wen, J. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.11.05.764505
Single-cell foundation models in perturbation prediction: progress, pitfalls and moving forward
Curean, V., Brown, S. D., Mahale, M., Berindan‐Neagoe, I., Seal, S., Szalay, K. Z., Veres, D. V., & Bender, A. (2026) · Digital Discovery · doi:10.1039/d6dd00209a
Gao, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2611.10658
Metadata
- Authors
- Wong, D. R., Hill, A. S., & Moccia, R.
- Affiliations
- Pfizer (United States) (US)
- Corresponding author
- Wong, Daniel R
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2025-05-23
- Volume
- 41
- Issue
- 6
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC12202205
- OpenAlex
- W4410689365
- Cited by
- 27
- References
- 40
- Citations by year
- 2026: 16 · 2025: 11
- FWCI
- 13.95
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Bioinformatics (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Pfizer Inc
- Topics
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Drug Discovery › Computational Drug Discovery Methods
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene Regulatory Network Analysis
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Keywords
- transcriptomic response, control sequences, baseline model, benchmarking, fine-tuning, Perturb-seq, deep learning evaluation
- MeSH terms
- Deep Learning, Algorithms, Humans, Models, Genetic, Computational Biology, Transcriptome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.