LT-GAN: Self-Supervised GAN with Latent Transformation Detection
Open accessAbstract
Generative Adversarial Networks (GANs) coupled with self-supervised tasks have shown promising results in unconditional and semi-supervised image generation. We propose a self-supervised approach (LT-GAN) to improve the generation quality and diversity of images by estimating the GAN-induced transformation (i.e. transformation induced in the generated images by perturbing the latent space of generator). Specifically, given two pairs of images where each pair comprises of a generated image and its transformed version, the self-supervision task aims to identify whether the latent transformation applied in the given pair is same as that of the other pair. Hence, this auxiliary loss encourages the generator to produce images that are distinguishable by the auxiliary network, which in turn promotes the synthesis of semantically consistent images with respect to latent transformations. We show the efficacy of this pretext task by improving the image generation quality in terms of FID on state-of-the-art models in conditional and unconditional settings on CIFAR-10, CelebA-HQ and ImageNet datasets. Moreover, we empirically show that LT-GAN helps in improving controlled image editing for CelebA-HQ, and ImageNet over baseline models. We experimentally demonstrate that our proposed LT self-supervision task can be effectively combined with other state-of-the-art training techniques for added benefits. Consequently, we show that our approach achieves the new state-of-the-art FID score of 9.8 on conditional CIFAR-10 image generation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Core aspects of Interactive and Cooperative AI involve advancing AI in explainability, understanding human intentions and behavior (theory of mind), and scalable coordination between AI systems and humans.
“Core aspects of this subdomain involve advancing AI in areas such as explainability [278], understanding human intentions and behavior (theory of mind) [279], [280], and scalable coordination between AI systems and humans, a collaborative approach crucial in creating more intuitive and interactive AI systems, capable of assisting and augmenting human capabilities in diverse contexts [281], [35].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [278]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,268 · see citing works
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714 · see citing works
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365 · see citing works
Extracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436 · see citing works
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691 · see citing works
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196 · see citing works
Large Scale GAN Training for High Fidelity Natural Image Synthesis
Brock, A., Donahue, J., & Simonyan, K. (2018) · arXiv · doi:10.48550/arxiv.1809.11096 · Cited by 3,717 · see citing works
Spectral Normalization for Generative Adversarial Networks
Miyato, T., Kataoka, T., Koyama, M., & Yoshida, Y. (2018) · arXiv · doi:10.48550/arxiv.1802.05957 · Cited by 3,438 · see citing works
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial NetsPreprint
Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.03657 · Cited by 3,379 · see citing works
Zhang, R., Isola, P., & Efros, A. A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46487-9_40 · Cited by 3,361 · see citing works
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
Noroozi, M., & Favaro, P. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_5 · Cited by 2,775 · see citing works
An analysis of single-layer networks in unsupervised feature learning
Coates, A., Ng, A. Y., & Lee, H. (2011) · Cited by 2,760 · see citing works
Deep Clustering for Unsupervised Learning of Visual Features
Caron, M., Bojanowski, P., Joulin, A., & Douze, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01264-9_9 · Cited by 2,674 · see citing works
Unsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629 · see citing works
Generative Visual Manipulation on the Natural Image Manifold
Zhu, J.-Y., Krähenbühl, P., Shechtman, E., & Efros, A. A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46454-1_36 · Cited by 1,215 · see citing works
(1993) · Cited by 1,173 · see citing works
Interpreting the Latent Space of GANs for Semantic Face Editing
Shen, Y., Gu, J., Tang, X., & Zhou, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00926 · Cited by 1,063 · see citing works
Which Training Methods for GANs do actually Converge?Preprint
Mescheder, L., Geiger, A., & Nowozin, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.04406 · Cited by 1,029 · see citing works
Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction
Zhang, R., Isola, P., & Efros, A. A. (2016) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2017.76 · Cited by 724 · see citing works
Audio-Visual Scene Analysis with Self-Supervised Multisensory Features
Owens, A., & Efros, A. A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_39 · Cited by 704 · see citing works
Charting the Right Manifold: Manifold Mixup for Few-shot Learning
Mangla, P., Singh, M., Sinha, A., Kumari, N., Balasubramanian, V. N., & Krishnamurthy, B. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093338 · Cited by 402 · see citing works
Self-citationRepresentation Learning by Learning to Count
Noroozi, M., Pirsiavash, H., & Favaro, P. (2017) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2017.628 · Cited by 379 · see citing works
Scaling and Benchmarking Self-Supervised Visual Representation Learning
Goyal, P., Mahajan, D. K., Gupta, A., & Misra, I. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00649 · Cited by 344 · see citing works
Self-Supervised GANs via Auxiliary Rotation Loss
Chen, T., Zhai, X., Ritter, M., Lucic, M., & Houlsby, N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01243 · Cited by 328 · see citing works
Learning and Using the Arrow of Time
Wei, D., Lim, J. J., Zisserman, A., & Freeman, W. T. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00840 · Cited by 318 · see citing works
Deep Feature Interpolation for Image Content Changes
Upchurch, P., Gardner, J. R., Pleiss, G., Pless, R. B., Snavely, N., Bala, K., & Weinberger, K. Q. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.645 · Cited by 303 · see citing works
Boosting Self-Supervised Learning via Knowledge Transfer
Noroozi, M., Vinjimoor, A., Favaro, P., & Pirsiavash, H. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00975 · Cited by 298 · see citing works
GANalyze: Toward Visual Definitions of Cognitive Image Properties
Goetschalckx, L., Andonian, A., Oliva, A., & Isola, P. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00584 · Cited by 294 · see citing works
GANSynth: Adversarial Neural Audio SynthesisPreprint
Engel, J., Agrawal, K. K., Chen, S., Gulrajani, I., Donahue, C., & Roberts, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.08710 · Cited by 291 · see citing works
AET vs. AED: Unsupervised Representation Learning by Auto-Encoding Transformations Rather Than Data
Zhang, L., Qi, G.-J., Wang, L., & Luo, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00265 · Cited by 204 · see citing works
Classification Accuracy Score for Conditional Generative ModelsPreprint
Ravuri, S. V., & Vinyals, O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10887 · Cited by 152 · see citing works
High-Fidelity Image Generation With Fewer LabelsPreprint
Lučić, M., Tschannen, M., Ritter, M., Zhai, X., Bachem, O., & Gelly, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.02271 · Cited by 126 · see citing works
Improvements to Context Based Self-Supervised Learning
Mundhenk, T. N., Ho, D., & Chen, B. Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00973 · Cited by 122 · see citing works
Adversarial Audio SynthesisPreprint
Donahue, C., McAuley, J., & Puckette, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04208 · Cited by 89 · see citing works
Controlling generative models with continuous factors of variationsPreprint
Plumerault, A., Borgne, H. L., & Hudelot, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.10238 · Cited by 87 · see citing works
Grasp2Vec: Learning Object Representations from Self-Supervised GraspingPreprint
Jang, E., Devin, C., Vanhoucke, V., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.06964 · Cited by 82 · see citing works
Cross and learn: Cross-modal self-supervision
Sayed, N., Brattoli, B., & Ommer, B. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-12939-2_17 · Cited by 76 · see citing works
Detecting Bias with Generative Counterfactual Face Attribute AugmentationPreprint
Denton, E., Hutchinson, B., Mitchell, M., & Gebru, T. (2019) · arXiv (Cornell University) · Cited by 61 · see citing works
Deep and Hierarchical Implicit Models.Preprint
Tran, D., Ranganath, R., & Blei, D. M. (2017) · arXiv (Cornell University) · Cited by 56 · see citing works
Self-supervised GAN: Analysis and Improvement with Multi-class Minimax GamePreprint
Tran, N.-T., Tran, V.-H., Nguyen, N.-B., Yang, L., & Cheung, N. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.06997 · Cited by 47 · see citing works
FX-GAN: Self-Supervised GAN Learning via Feature Exchange
Huang, R., Xu, W., Lee, T., Cherian, A., Wang, Y., & Marks, T. K. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093525 · Cited by 13 · see citing works
An Improved Self-supervised GAN via Adversarial TrainingPreprint
Tran, N.-T., Tran, V.-H., Nguyen, N.-B., & Cheung, N. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.05469 · Cited by 12 · see citing works
Flagged first, then by citations
Research works printed but not linked: 10
- David Bau, Jun-Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua B. Tenenbaum, William T. Freeman, and Antonio Torralba. Gan dissection: Visualizing and understanding generative adversarial networks. In ICLR, 2019.
- Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Un- supervised representation learning by predicting image rota- tions. In ICLR, 2018.
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672-2680, 2014.
- Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training of wasserstein gans. In Advances in neural information pro- cessing systems, pages 5767-5777, 2017.
- Ali Jahanian, Lucy Chai, and Phillip Isola. On the "steerabil- ity" of generative adversarial networks. In ICLR, 2020.
- Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In ICLR, 2018.
- Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxiliary classifier gans. In ICML, 207.
- Alec Radford, Luke Metz, and Soumith Chintala. Unsuper- vised representation learning with deep convolutional gener- ative adversarial networks. In ICLR, 2016.
- Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In NeurIPS, 2016.
- Han Zhang, Zizhao Zhang, Augustus Odena, and Honglak Lee. Consistency regularization for generative adversarial networks. In ICLR, 2020.
Cited by
Sorted by citations
McIntosh, T. R., Susnjak, T., Liu, T., Watters, P., Xu, D., Liu, D., & Halgamuge, M. N. (2025) · Technologies · doi:10.3390/technologies13020051 · Cited by 102 · see citing works
“It incorporates generative models like autoencoders and GANs for data distribution learning and original input reconstruction [244], [245], [246], and also includes contrastive methods such as SimCLR [247] and MoCo [248], designed to differentiate between positive and negative sample pairs.”
Method
- Authors
- Patel, P., Kumari, N., Singh, M., & Krishnamurthy, B.
- Affiliations
- Birla Institute of Technology and Science, Pilani (IN)
- Adobe Systems (United States) (US)
- Venue
- 2021 IEEE Winter Conference on Applications of Computer Vision (WACV)
- Publisher
- IEEE
- Published
- 2021-01
- Pages
- 3188-3197
- ISSN
- 2472-6737 (ISSN-L)
- OpenAlex
- W3094091227
- Cited by
- 18
- References
- 45
- Citations by year
- 2025: 2 · 2024: 3 · 2023: 4 · 2022: 6 · 2021: 2 · 2020: 1
- FWCI
- 1.23
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Winter Conference on Applications of Computer Vision (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- image generation quality, image diversity, conditional image generation, unconditional image generation, auxiliary loss, CelebA-HQ, ImageNet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
