CoChat

Conditional out-of-distribution generation for unpaired data using transfer VAE

Open access

The DOI matches its Crossref record; 11 of the 14 references it cites were found in a registry or an index.

Abstract

Motivation: While generative models have shown great success in sampling high-dimensional samples conditional on low-dimensional descriptors (stroke thickness in MNIST, hair color in CelebA, speaker identity in WaveNet), their generation out-of-distribution poses fundamental problems due to the difficulty of learning compact joint distribution across conditions. The canonical example of the conditional variational autoencoder (CVAE), for instance, does not explicitly relate conditions during training and, hence, has no explicit incentive of learning such a compact representation. Results: We overcome the limitation of the CVAE by matching distributions across conditions using maximum mean discrepancy in the decoder layer that follows the bottleneck. This introduces a strong regularization both for reconstructing samples within the same condition and for transforming samples across conditions, resulting in much improved generalization. As this amount to solving a style-transfer problem, we refer to the model as transfer VAE (trVAE). Benchmarking trVAE on high-dimensional image and single-cell RNA-seq, we demonstrate higher robustness and higher accuracy than existing approaches. We also show qualitatively improved predictions by tackling previously problematic minority classes and multiple conditions in the context of cellular perturbation response to treatment and disease based on high-dimensional single-cell gene expression data. For generic tasks, we improve Pearson correlations of high-dimensional estimated means and variances with their ground truths from 0.89 to 0.97 and 0.75 to 0.87, respectively. We further demonstrate that trVAE learns cell-type-specific responses after perturbation and improves the prediction of most cell-type-specific genes by 65%. Availability and implementation: The trVAE implementation is available via github.com/theislab/trvae. The results of this article can be reproduced via github.com/theislab/trvae_reproducibility.

Overview

Integrity signals

Registry

Crossref · 10.1093/bioinformatics/btaa800

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Research works printed but not linked: 6

  1. Arjovsky, 2017, Proceedings of the 34th International Conference on Machine Learning, Volume 70 of Proceedings of Machine Learning Research, 214
  2. Dziugaite, 2015, Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence, UAI’15, 258
  3. Johansson, 2016, International Conference on Machine Learning, 3020
  4. Lopez, 2018, Advances in Neural Information Processing Systems, 6114
  5. Louizos, 2017, Advances in Neural Information Processing Systems, 6446
  6. Sohn, 2015, Learning structured output representation using deep conditional generative models, Advances in Neural Information Processing Systems, Vol. 28, 3483

Cited by

See all 131

Metadata

Authors
Lotfollahi, M., Naghipourfar, M., Theis, F. J., & Wolf, F. A.
Affiliations
Helmholtz Munich (DE)
Corresponding author
  • Theis, Fabian J
  • Wolf, F Alexander
Venue
Bioinformatics
Published
2020-12
Volume
36
Issue
Supplement_2
Pages
i610-i617
ISSN
  • 1367-4803 (ISSN-L)
  • 1367-4811
OpenAlex
W3117464799
Cited by
130
References
14
Citations by year
2026: 25 · 2025: 31 · 2024: 24 · 2023: 25 · 2022: 12 · 2021: 12 · 2020: 1
FWCI
4.05
Type
journal article
Language
en
Locations
  • Bioinformatics (publishedVersion)
  • PubMed (publishedVersion)
  • Site cant be reached (submittedVersion, other-oa)
  • PuSH - Publication Server of Helmholtz Zentrum München (submittedVersion)
Funders
  • BMBF (01IS18036A, 01IS18053A)
  • German Research Foundation (ZT-I-0007)
  • Chan Zuckerberg Initiative DAF (182835)
  • Joachim Herz Stiftung
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
Keywords
conditional variational autoencoder, maximum mean discrepancy, style transfer, unpaired data, single-cell RNA sequencing, cell type-specific gene expression, perturbation response prediction, minority class, generative models
MeSH terms
Exome Sequencing, RNA-Seq, Reproducibility of Results

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.