The DOI matches its Crossref record; 44 of the 49 references it cites were found in a registry or an index.
Semantic Image Synthesis via Adversarial Learning
Open accessAbstract
In this paper, we propose a way of synthesizing realistic images directly with natural language description, which has many useful applications, e.g. intelligent image manipulation. We attempt to accomplish such synthesis: given a source image and a target text description, our model synthesizes images to meet two requirements: 1) being realistic while matching the target text description; 2) maintaining other image features that are irrelevant to the text description. The model should be able to disentangle the semantic information from the two modalities (image and text), and generate new images from the combined semantics. To achieve this, we proposed an end-to-end neural architecture that leverages adversarial learning to automatically learn implicit loss functions, which are optimized to fulfill the aforementioned two requirements. We have evaluated our model by conducting experiments on Caltech-200 bird dataset and Oxford-102 flower dataset, and have demonstrated that our model is capable of synthesizing realistic images that match the given descriptions, while still maintain other features of original images.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These can also be used for biological image synthesis [208, 209] and text-to-image synthesis [210, 211, 212].”
From An overview of deep learning in medical imaging focusing on MRI (2019) · cites this paper as [210]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
Labaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,119
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 20,507
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
Flagged first, then by citations
5 of the 49 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- gregor, 2015, Stochastic Backpropagation and Approximate Inference in Deep Generative Models, ICML, 2
Cited by
Sorted by citations
Items 201–273 of 273
Weakly Supervised Attention Inference Generative Adversarial Network for Text-to-Image
Mei, L., Ran, X., & Hu, J. (2019) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci44817.2019.9002666 · Cited by 2
Bo, Y. (2021) · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C) · doi:10.1109/qrs-c55045.2021.00070 · Cited by 2
SketchCLIP: Text-based Attribute Manipulation for Face Sketch Synthesis
Dong, M., Peng, C., Liu, D., Zheng, Y., Wang, N., & Gao, X. (2022) · IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb54206.2022.10007972 · Cited by 2
Oh, Y., Oh, S., Noh, S.-W., Kim, H., & Seo, H. (2023) · PLoS ONE · doi:10.1371/journal.pone.0293885 · Cited by 2
HGAN: Editable Visual Generation from Hindi Descriptions
Singh, V., Gupta, S., & Tiwary, U. S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-53827-8_1 · Cited by 2
Robust sound-guided image manipulation
Lee, S. H., Chi, H., Oh, G., Byeon, W., Yoon, S. H., Park, H., Cho, W., Kim, J., & Kim, S. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106271 · Cited by 2
EmoStyle: Emotion-Aware Semantic Image Manipulation with Audio Guidance
Shen, Q., Xu, J., Mei, J., Wu, X., & Dong, D. (2024) · Applied Sciences · doi:10.3390/app14083193 · Cited by 3
A review of multimodal learning for text to images
Chen, W., Yang, Y., Tian, Z., Chen, Q., & Liu, J. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19117-8 · Cited by 2
FastFaceCLIP: A lightweight text‐driven high‐quality face image manipulation
Ren, J., Qin, J., Ma, Q., & Cao, Y. (2024) · IET Computer Vision · doi:10.1049/cvi2.12295 · Cited by 3
Named Entity Driven Zero-Shot Image Manipulation
Feng, Z., Chen, L., Tian, J., Liu, J., & Feng, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00870 · Cited by 2
Prompt-Based Modality Bridging for Unified Text-to-Face Generation and Manipulation
Ma, Y., Kuang, H., Yang, H., Fu, J., & Liu, J. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3694974 · Cited by 2
Self-Supervised Audio-Visual Soundscape Stylization
Li, T., Wang, R., Huang, P.-Y., Owens, A., & Anumanchipalli, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72989-8_2 · Cited by 3
Modelling of Material Ageing with Generative Adversarial Networks
Papadopoulos, S., Drosou, A., & Tzovaras, D. K. (2018) · Multidimensional Signal Processing Workshop · doi:10.1109/ivmspw.2018.8448972 · Cited by 2
Review on Generative Adversarial Network in Computer Vision: Methods and Metrics
Bao, S. (2021) · International Conference on Big Data & Artificial Intelligence & Software Engineering (ICBASE) · doi:10.1109/icbase53849.2021.00105 · Cited by 2
Obj-GloVe: Scene-Based Contextual Object EmbeddingPreprint
Xu, C., Chen, Z., & Li, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.01478 · Cited by 2
Grounded and Controllable Image Completion by Incorporating Lexical SemanticsPreprint
Zhang, S., Jiang, T., Huang, Q., Tan, Z., Zhao, Z., Tang, S., Yu, J., Yang, H., Yang, Y., & Wu, F. Q. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.00303 · Cited by 2
Rendering the image of glare effect based on paired and unpaired dual generative adversarial network
Ao, J., Zhang, X., & Lin, Y. (2023) · Displays · doi:10.1016/j.displa.2023.102368 · Cited by 1
Artistic image synthesis from unsupervised segmentation maps
Liu, D., Yao, H., & Lu, X. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15318-9 · Cited by 1
High-fidelity instructional fashion image editing
Zheng, Y., Zhang, T., Bao, J., Chen, D., & Zeng, M. (2024) · Graphical Models · doi:10.1016/j.gmod.2024.101223 · Cited by 2
ImageCraft - Text To Image Synthesis
Rao, D., Sriram, P. Y., Dakshayani, M., & Gupta, A. (2024) · DMIHER International Conference on Artificial Intelligence in Healthcare, Education and Industry (IDICAIHEI) · doi:10.1109/idicaiei61867.2024.10842937 · Cited by 1
Liu, J., Zhao, D., Yuan, B., Li, W., & Li, T. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3552802 · Cited by 1
A novel hybrid augmented loss discriminator for text‐to‐image synthesis
Gan, Y., Ye, M., Liu, D. C., Yang, S., & Xiang, T. (2020) · International Journal of Intelligent Systems · doi:10.1002/int.22333 · Cited by 1
Rank in Style: A Ranking-based Approach to Find Interpretable Directions
Kocasarı, U., Zaman, K., Tiftikci, M., Simsar, E., & Yanardag, P. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00255 · Cited by 1
Text-Guided Image Manipulation Based on Sentence-Aware and Word-Aware Network
Zhang, Z., Fu, C. M., Ho, M. M., Zhou, J., Jiang, N., & Yu, W. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9859585 · Cited by 1
StyleGAN-based CLIP-guided Image Shape Manipulation
Qian, Y., Yamamoto, K., & Yanai, . (2022) · International Workshop on Content-Based Multimedia Indexing (CBMI) · doi:10.1145/3549555.3549556 · Cited by 1
24-GAN: Portrait Generation with Composite Attributes
Kim, J., Yang, H., & Min, K. (2022) · Mathematics · doi:10.3390/math10203887 · Cited by 1
Controlling Facial Attribute Synthesis by Disentangling Attribute Feature Axes in Latent Space
Wei, Q., Zheng, W., Li, Y., Cheng, Z., Zeng, Z., & Yang, X. (2023) · Proceedings - International Conference on Image Processing · doi:10.1109/icip49359.2023.10223056 · Cited by 1
Text-driven Face Image Generation and Manipulation via Multi-level Residual Mapper
Li, Z., Zhang, S., Liu, Y., Zhang, Z., Zhang, W., & Huang, Q. (2023) · International Journal of Software and Informatics · doi:10.21655/ijsi.1673-7288.00313 · Cited by 1
A novel generalized adversarial image method using descriptive features
Hồ, T. P., Trung, P. D., & Bui, L. T. (2023) · Journal of Science and Technology on Information security · doi:10.54654/isj.v3i20.1002 · Cited by 1
Hong, H., Zheng, Z., Huang, W., & Lin, X. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130194 · Cited by 1
Transforming Single Photon Camera Images to Color High Dynamic Range Images
Sharma, S., Girish, R. S. V., & Mitra, K. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-93709-5_10 · Cited by 1
A background-induced generative network with multi-level discriminator for text-to-image generation
Wang, P., Liu, L., Zhang, H., & Wang, T. (2021) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3444685.3446291 · Cited by 1
Grounded, Controllable and Debiased Image Completion with Lexical Semantics
Zhang, S., Jiang, T., Huang, Q., Tan, Z., Kuang, K., Zhao, Z., Tang, S., Yu, J., Yang, H., Yang, Y., et al. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00192 · Cited by 1
Generative Adversarial Networks and its Applications
Matsubara, T. (2018) · The Journal of The Institute of Image Information and Television Engineers · doi:10.3169/itej.72.709 · Cited by 1
DLGAN: Disentangling Label-Specific Fine-Grained Features for Image ManipulationPreprint
Zhan, G., Zhao, Y., Zhao, B., Yuan, H., Chen, B., & Dong, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.09943 · Cited by 1
Self-citationEvolving Loss Functions with Multivariate Taylor Polynomial ParameterizationsPreprint
González, S., & Miikkulainen, R. P. (2020) · arXiv (Cornell University) · Cited by 2
EMIXER: End-to-end Multimodal X-ray Generation via Self-supervisionPreprint
Biswal, S., Zhuang, P., Pyrros, A. T., Siddiqui, N., Koyejo, S., & Sun, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.05597 · Cited by 1
Retraction: A systematic review for image enhancement using deep learning techniques
Manaa, M. J., Abbas, A. R., & Shakur, W. A. (2023) · AIP conference proceedings · doi:10.1063/5.0182081 · Cited by 1
RetractedDownhole Track Detection via Multiscale Conditional Generative Adversarial NetsPreprint
Li, J., Wei, X., Yang, G., Sun, X. N., & Li, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.08177
Semantics Images Synthesis and Resolution Refinement Using Generative Adversarial Networks
Han, J., Zhang, Z., Mao, A., & Zhou, Y. (2019) · Lecture notes in electrical engineering · doi:10.1007/978-981-13-6504-1_74
Systematic Analysis of Image Generation using GANsPreprint
Akut, R., Marathe, S., Apte, R., Joshi, I., & Kulkarni, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.11863
Towards Generating Remote Sensing Images of the Far Past
Bejiga, M. B., & Melgani, F. (2019) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss.2019.8899834
Image Modification using Text with GANs
Doshi, F., Doshi, P., Gandhi, J., Dwivedi, K., & Mangrulkar, R. S. (2020) · International Journal of Computer Applications Technology and Research · doi:10.7753/ijcatr0911.1001
Multiple Facial Expressions Synthesis Driven by Editable Line Maps
Liu, D., Yang, Y., & Jing, X. (2020) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc42975.2020.9283195
Interactive Style Space of Deep Features and Style Innovation
Guo, B., & Hao, P. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9413328
Data-Driven Fault Symptoms Generation and Augmentation for Satellite Attitude Control System
Ma, Y., Zhang, W., Liu, X., Wang, Z., & Shen, Y. (2021) · IEEE Data Driven Control and Learning Systems Conference (DDCLS) · doi:10.1109/ddcls52934.2021.9455626
Dilated Residual Aggregation Network for Text-Guided Image Manipulation
Lu, S., Luo, D., Yang, Z., Hao, T., Li, Q., & Liu, W. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86365-4_3
SpaceEdit: Learning a Unified Editing Space for Open-Domain Image EditingPreprint
Shi, J., Xu, N., Zheng, H., Smith, A., Luo, J., & Xu, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.00180
Reconstruction and manipulation of portraits
Song, G. (2022) · doi:10.32657/10356/155400
Text-Guided Customizable Image Synthesis and Manipulation
Zhang, Z., Fu, C. M., Weng, W., & Zhou, J. (2022) · Applied Sciences · doi:10.3390/app122010645
Image Attribute Modification Based on Text Guidance
Zhao, L., Li, X., Fu, C., & Chen, Z. (2022) · Communications in computer and information science · doi:10.1007/978-981-19-8350-4_16
Multimodal Adversarial Learning
Osahor, U. (2022) · doi:10.33915/etd.11546
International Conference on Recent and Future Trends in Smart Electronics System and Manufacturing
Kotecha, K. V., Shah, P., Kshirsagar, U., Nandan, D., & Kantipudi, M. V. V. P. (2023) · doi:10.13052/rp-9788770229852
Diffusion-Adapter: Text Guided Image Manipulation with Frozen Diffusion Models
Wei, R., Fan, C., & Wu, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44210-0_18
Semantic image synthesis with vision graph neural network
Xu, Z., Qian, X., & Wang, P. (2023) · International Conference on Computer Graphics, Image and Virtualization (ICCGIV) · doi:10.1117/12.3008021
Watanabe, Y., Togo, R., Maeda, K., Ogawa, T., & Haseyama, M. (2023) · Sensors · doi:10.3390/s23229287 · Cited by 1
Attention-driven Text-guided Image Manipulation
Su, J., Zhang, Z., Han, Q., Cheng, X., Zhao, W., Zhang, W., & Yu, W. (2023) · doi:10.1145/3628454.3631665
Singh, H., Garg, P., Gupta, M., Shah, K., Goswami, A., Modi, S. K., Mondal, A. K., Khandelwal, D., Garg, D. K., & Singla, P. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.181
Hybrid Prior-Based Diminished Reality for Indoor Panoramic Images
Liu, J., Zhang, Q., Xuelin, S., Wu, W., & Wang, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-50075-6_30
An Efficient Deep Neural Network Based Image Enhancement with Improved Filtering Technics
Begum, J. (2024) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2024.59396
3d Cartoon Face Generation with Controllable Expressions from a Single Gan ImagePreprint
Wang, H., Lin, G., Hoi, S. C. H., & Miao, C. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4794779
ETBHD‐HMF: A Hierarchical Multimodal Fusion Architecture for Enhanced Text‐Based Hair Design
He, R., Jiao, G., & Li, C. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15194
UATST: Towards unpaired arbitrary text-guided style transfer with cross-space modulation
Chen, H., & Zhao, L. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.104246
PaintDiffusion: Towards text-driven painting variation via collaborative diffusion guidance
Chen, H., Chen, Z., Zhao, L., Li, J., & Yang, J. J. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129284
Deep Learning-Based Text-To-Image Synthesis for Criminal Face Generation
Sriramulu, M. (2025) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2025.67502
Accelerating Attentional Generative Adversarial Networks with Sampling Blocks
Zhang, C., Jin, M., Yu, Q., Xue, H., Yang, X., & Jin, X. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-7036-9_9
3D Cartoon Face Generation with Controllable Expressions from a Single GAN Image
Wang, H., Shen, W., Lin, G., Hoi, S. C. H., & Miao, C. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228790
Robust wavefront sensing based on graph attention adversarial learning
Chen, Y. X., Luo, X., Jia, Q., & Li, X. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.104079
T2IW: Joint Text to Image & Watermark Generation
Zhang, G., Su, Y., Wang, L., Liu, G., Song, D., & Liu, A.-A. (2026) · Machine Intelligence Research · doi:10.1007/s11633-025-1606-9
H-AlignEdit: Hierarchical Semantic Alignment for Latent Refinement in Text-Driven Face Editing
Zhang, Y. J. (2026) · Conference on Fully Actuated System Theory and Applications (FASTA) · doi:10.1109/fasta70174.2026.11548800
BrainGAN Framework for Generating Synthetic Contrast-Enhanced Multisequence MRI in Brain Metastases
Solak, M., Tören, M., Asan, B., Kaba, E., Beyazal, M., & Çeliker, F. B. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-02059-y
Metadata
- Authors
- Dong, H., Yu, S., Wu, C., & Guo, Y.
- Affiliations
- Imperial College London (GB)
- Venue
- 2017 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 5707-5715
- OpenAlex
- W2964313012
- Cited by
- 276
- References
- 49
- Citations by year
- 2026: 3 · 2025: 18 · 2024: 28 · 2023: 45 · 2022: 45 · 2021: 46 · 2020: 34 · 2019: 32 · 2018: 22 · 2017: 3
- FWCI
- 7.16
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Virginia Agricultural Experiment Station, Virginia Polytechnic Institute and State University
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- semantic image synthesis, text-to-image generation, adversarial learning, image editing, realistic image synthesis, multimodal learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
