Conditional out-of-distribution generation for unpaired data using transfer VAE
Open accessThe DOI matches its Crossref record; 11 of the 14 references it cites were found in a registry or an index.
Abstract
Motivation: While generative models have shown great success in sampling high-dimensional samples conditional on low-dimensional descriptors (stroke thickness in MNIST, hair color in CelebA, speaker identity in WaveNet), their generation out-of-distribution poses fundamental problems due to the difficulty of learning compact joint distribution across conditions. The canonical example of the conditional variational autoencoder (CVAE), for instance, does not explicitly relate conditions during training and, hence, has no explicit incentive of learning such a compact representation. Results: We overcome the limitation of the CVAE by matching distributions across conditions using maximum mean discrepancy in the decoder layer that follows the bottleneck. This introduces a strong regularization both for reconstructing samples within the same condition and for transforming samples across conditions, resulting in much improved generalization. As this amount to solving a style-transfer problem, we refer to the model as transfer VAE (trVAE). Benchmarking trVAE on high-dimensional image and single-cell RNA-seq, we demonstrate higher robustness and higher accuracy than existing approaches. We also show qualitatively improved predictions by tackling previously problematic minority classes and multiple conditions in the context of cellular perturbation response to treatment and disease based on high-dimensional single-cell gene expression data. For generic tasks, we improve Pearson correlations of high-dimensional estimated means and variances with their ground truths from 0.89 to 0.97 and 0.75 to 0.87, respectively. We further demonstrate that trVAE learns cell-type-specific responses after perturbation and improves the prediction of most cell-type-specific genes by 65%. Availability and implementation: The trVAE implementation is available via github.com/theislab/trvae. The results of this article can be reproduced via github.com/theislab/trvae_reproducibility.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Multiple distance measures for scRNA-seq have been explored by the single-cell community in recent years, including Wasserstein distance computed in an optimal transport framework22,23, maximum mean discrepancy24, neighborhood-based measures17,18, and energy distance (E-distance)20.”
From scPerturb: harmonized single-cell perturbation data (2024) · cites this paper as [24]
In the citing paper
“Contrary to scGEN, trVAE can address the issue of performing OOD predictions for multiple perturbations within a given dataset.”
From A mini-review on perturbation modelling across single-cell omic modalities (2024) · cites this paper as [53]
In the citing paper
“Aside from CPA, existing methods (Lotfollahi et al, 2020; Russkikh et al, 2020) such as scGen (Lotfollahi et al, 2019) have also been shown capable of predicting single‐cell perturbation responses when the dataset contains no combinatorial treatment or dose‐dependent perturbations.”
From Predicting cellular responses to complex perturbations in high‐throughput screens (2023) · cites this paper as Lotfollahi et al, 2020
Citations
Cites
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
(2012) · MPG.PuRe (Max Planck Society) · doi:10.5555/2188385.2188410 · Cited by 3,603
Deep generative modeling for single-cell transcriptomics
Lopez, R., Regier, J., Cole, M. B., Jordan, M. I., & Yosef, N. (2018) · Nature Methods · doi:10.1038/s41592-018-0229-2 · Cited by 2,530
A single-cell survey of the small intestinal epithelium
Haber, A. L., Biton, M., Rogel, N., Herbst, R. H., Shekhar, K., Smillie, C., Burgin, G., Delorey, T. M., Howitt, M. R., Katz, Y., et al. (2017) · Nature · doi:10.1038/nature24489 · Cited by 1,750
Multiplexed droplet single-cell RNA-sequencing using natural genetic variation
Kang, H. M., Subramaniam, M., Targ, S., Nguyen, M., Maliskova, L., McCarthy, E., Wan, E., Wong, S., Byrnes, L., Lanata, C. M., et al. (2018) · Nature Biotechnology · doi:10.1038/nbt.4042 · Cited by 1,101
Single-cell RNA-seq denoising using a deep count autoencoder
Eraslan, G., Simon, L. M., Mircea, M., Mueller, N. S., & Theis, F. J. (2019) · Nature Communications · doi:10.1038/s41467-018-07931-2 · Cited by 1,084
Self-citationscGen predicts single-cell perturbation responses
Lotfollahi, M., Wolf, F. A., & Theis, F. J. (2019) · Nature Methods · doi:10.1038/s41592-019-0494-8 · Cited by 661
Self-citationMassively multiplex chemical transcriptomics at single-cell resolution
Srivatsan, S. R., McFaline-Figueroa, J. L., Ramani, V., Saunders, L., Cao, J., Packer, J., Pliner, H. A., Jackson, D. L., Daza, R. M., Christiansen, L., et al. (2020) · Science · doi:10.1126/science.aax6234 · Cited by 401
Exploring single-cell data with deep multitasking neural networks
Amodio, M., van Dijk, D., Srinivasan, K., Chen, W. S., Mohsen, H., Moon, K. R., Campbell, A., Zhao, Y., Wang, X., Venkataswamy, M., et al. (2019) · Nature Methods · doi:10.1038/s41592-019-0576-7 · Cited by 297
Research works printed but not linked: 6
- Arjovsky, 2017, Proceedings of the 34th International Conference on Machine Learning, Volume 70 of Proceedings of Machine Learning Research, 214
- Dziugaite, 2015, Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence, UAI’15, 258
- Johansson, 2016, International Conference on Machine Learning, 3020
- Lopez, 2018, Advances in Neural Information Processing Systems, 6114
- Louizos, 2017, Advances in Neural Information Processing Systems, 6446
- Sohn, 2015, Learning structured output representation using deep conditional generative models, Advances in Neural Information Processing Systems, Vol. 28, 3483
Cited by
Benchmarking atlas-level data integration in single-cell genomics
Luecken, M. D., Büttner, M., Chaichoompu, K., Danese, A., Interlandi, M., Mueller, M. F., Strobl, D. C., Zappia, L., Dugas, M., Colomé-Tatché, M., et al. (2022) · Nature Methods · doi:10.1038/s41592-021-01336-8 · Cited by 1,438
Self-citationMapping single-cell data to reference atlases by transfer learning
Lotfollahi, M., Naghipourfar, M., Luecken, M. D., Khajavi, M., Büttner, M., Wagenstetter, M., Avsec, Ž., Gayoso, A., Yosef, N., Interlandi, M., et al. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01001-7 · Cited by 602
Self-citationJoint probabilistic modeling of single-cell multi-omic data with totalVI
Gayoso, A., Steier, Z., Lopez, R., Regier, J., Nazor, K. L., Streets, A., & Yosef, N. (2021) · Nature Methods · doi:10.1038/s41592-020-01050-x · Cited by 592
Predicting cellular responses to complex perturbations in high‐throughput screens
Lotfollahi, M., Klimovskaia Susmelj, A., De Donno, C., Hetzel, L., Ji, Y., Ibarra, I. L., Srivatsan, S. R., Naghipourfar, M., Daza, R. M., Martin, B., et al. (2023) · Molecular Systems Biology · doi:10.15252/msb.202211517 · Cited by 307
“Aside from CPA, existing methods (Lotfollahi et al , 2020; Russ-kikh et al , 2020) such as scGen (Lotfollahi et al , 2019) have also been shown capable of predicting single-cell perturbation responses when the dataset contains no combinatorial treatment or dose-dependent perturbations.”
BackgroundMethodSelf-citationEfficient and precise single-cell reference atlas mapping with Symphony
Kang, J. B., Nathan, A., Weinand, K., Zhang, F., Millard, N., Rumker, L., Moody, D. B., Korsunsky, I., & Raychaudhuri, S. (2021) · Nature Communications · doi:10.1038/s41467-021-25957-x · Cited by 231
Metadata
- Authors
- Lotfollahi, M., Naghipourfar, M., Theis, F. J., & Wolf, F. A.
- Affiliations
- Helmholtz Munich (DE)
- Corresponding author
- Theis, Fabian J
- Wolf, F Alexander
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2020-12
- Volume
- 36
- Issue
- Supplement_2
- Pages
- i610-i617
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- OpenAlex
- W3117464799
- Cited by
- 130
- References
- 14
- Citations by year
- 2026: 25 · 2025: 31 · 2024: 24 · 2023: 25 · 2022: 12 · 2021: 12 · 2020: 1
- FWCI
- 4.05
- Type
- journal article
- Language
- en
- Open access
- Licence
- Locations
- Bioinformatics (publishedVersion)
- PubMed (publishedVersion)
- Site cant be reached (submittedVersion, other-oa)
- PuSH - Publication Server of Helmholtz Zentrum München (submittedVersion)
- Funders
- BMBF (01IS18036A, 01IS18053A)
- German Research Foundation (ZT-I-0007)
- Chan Zuckerberg Initiative DAF (182835)
- Joachim Herz Stiftung
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Keywords
- conditional variational autoencoder, maximum mean discrepancy, style transfer, unpaired data, single-cell RNA sequencing, cell type-specific gene expression, perturbation response prediction, minority class, generative models
- MeSH terms
- Exome Sequencing, RNA-Seq, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.