Semantic Image Synthesis via Adversarial Learning
Open accessAbstract
In this paper, we propose a way of synthesizing realistic images directly with natural language description, which has many useful applications, e.g. intelligent image manipulation. We attempt to accomplish such synthesis: given a source image and a target text description, our model synthesizes images to meet two requirements: 1) being realistic while matching the target text description; 2) maintaining other image features that are irrelevant to the text description. The model should be able to disentangle the semantic information from the two modalities (image and text), and generate new images from the combined semantics. To achieve this, we proposed an end-to-end neural architecture that leverages adversarial learning to automatically learn implicit loss functions, which are optimized to fulfill the aforementioned two requirements. We have evaluated our model by conducting experiments on Caltech-200 bird dataset and Oxford-102 flower dataset, and have demonstrated that our model is capable of synthesizing realistic images that match the given descriptions, while still maintain other features of original images.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
GANs can be used for biological image synthesis and text-to-image synthesis.
“These can also be used for biological image synthesis [208, 209] and text-to-image synthesis [210, 211, 212].”
From An overview of deep learning in medical imaging focusing on MRI (2019) · cites this paper as [210]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · see citing works
Labaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,100 · see citing works
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · see citing works
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,268 · see citing works
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696 · see citing works
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235 · see citing works
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283 · see citing works
Conditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863 · see citing works
Improved Techniques for Training GANs
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016) · arXiv · doi:10.48550/arxiv.1606.03498 · Cited by 6,965 · see citing works
Caltech-UCSD Birds-200-2011 Dataset
Wah, C. (2024) · CaltechAUTHORS (California Institute of Technology) · doi:10.57702/i9oeewf6 · Cited by 3,875 · see citing works
Stochastic Backpropagation and Approximate Inference in Deep Generative ModelsPreprint
Rezende, D. J., Mohamed, S., & Wierstra, D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1401.4082 · Cited by 3,679 · see citing works
Automated Flower Classification over a Large Number of Classes
Nilsback, M.-E., & Zisserman, A. (2008) · 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing · doi:10.1109/icvgip.2008.47 · Cited by 3,442 · see citing works
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial NetsPreprint
Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.03657 · Cited by 3,379 · see citing works
Pixel Recurrent Neural NetworksPreprint
van den Oord, A., Kalchbrenner, N., & Kavukcuoglu, K. (2016) · International Conference on Machine Learning · doi:10.48550/arxiv.1601.06759 · Cited by 3,111 · see citing works
Conditional Image Generation with PixelCNN DecodersPreprint
van den Oord, A., Kalchbrenner, N., Vinyals, O., Espeholt, L., Graves, A., & Kavukcuoglu, K. (2016) · Neural Information Processing Systems · doi:10.48550/arxiv.1606.05328 · Cited by 3,066 · see citing works
StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks
Zhang, H., Xu, T., Li, H., Zhang, S., Wang, X., Huang, X., & Metaxas, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.629 · Cited by 3,021 · see citing works
Generative Adversarial Text to Image Synthesis
Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., & Lee, H. (2016) · arXiv · doi:10.48550/arxiv.1605.05396 · Cited by 2,494 · see citing works
Deep Generative Image Models using a Laplacian Pyramid of Adversarial NetworksPreprint
Denton, E., Chintala, S., Szlam, A., & Fergus, R. (2015) · Neural Information Processing Systems · doi:10.48550/arxiv.1506.05751 · Cited by 2,360 · see citing works
Unsupervised Image-to-Image Translation NetworksPreprint
Liu, M.-Y., Breuel, T. M., & Kautz, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00848 · Cited by 2,121 · see citing works
Adversarial Feature LearningPreprint
Donahue, J., Krähenbühl, P., & Darrell, T. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1605.09782 · Cited by 1,982 · see citing works
Conditional Image Generation with PixelCNN DecodersPreprint
van den Oord, A., Kalchbrenner, N., Vinyals, O., et al. (2016) · arXiv (Cornell University) · Cited by 1,586 · see citing works
Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., & Frey, B. (2015) · arXiv · doi:10.48550/arxiv.1511.05644 · Cited by 1,579 · see citing works
Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks
Li, C., & Wand, M. (2016) · European Conference on Computer Vision · doi:10.1007/978-3-319-46487-9_43 · Cited by 1,535 · see citing works
Unifying Visual-Semantic Embeddings with Multimodal Neural Language ModelsPreprint
Kiros, R., Salakhutdinov, R., & Zemel, R. S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.2539 · Cited by 1,386 · see citing works
Autoencoding beyond pixels using a learned similarity metric
Larsen, A., Sønderby, S. K., Larochelle, H., & Winther, O. (2016) · Research at the University of Copenhagen (University of Copenhagen) · Cited by 1,317 · see citing works
Generative Visual Manipulation on the Natural Image Manifold
Zhu, J.-Y., Krähenbühl, P., Shechtman, E., & Efros, A. A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46454-1_36 · Cited by 1,215 · see citing works
Adversarially Learned InferencePreprint
Dumoulin, V., Belghazi, M. I., Poole, B., Mastropietro, O., Lamb, A., Arjovsky, M., & Courville, A. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00704 · Cited by 1,015 · see citing works
Unsupervised Cross-Domain Image GenerationPreprint
Taigman, Y., Polyak, A., & Wolf, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02200 · Cited by 901 · see citing works
Learning Deep Representations of Fine-Grained Visual Descriptions
Reed, S., Akata, Z., Lee, H., & Schiele, B. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.13 · Cited by 802 · see citing works
Attribute2Image: Conditional Image Generation from Visual Attributes
Yan, X., Yang, J., Sohn, K., & Lee, H. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_47 · Cited by 688 · see citing works
Learning to generate chairs with convolutional neural networks
Dosovitskiy, A., Springenberg, J. T., & Brox, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298761 · Cited by 640 · see citing works
Generative Image Modeling Using Style and Structure Adversarial Networks
Wang, X., & Gupta, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_20 · Cited by 607 · see citing works
Invertible Conditional GANs for image editing
Perarnau, G., van de Weijer, J., Raducanu, B., & Álvarez, J. M. (2016) · arXiv · doi:10.48550/arxiv.1611.06355 · Cited by 563 · see citing works
Learning What and Where to DrawPreprint
Reed, S., Akata, Z., Mohan, S., Tenka, S., Schiele, B., & Lee, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.02454 · Cited by 411 · see citing works
Yoo, D., Kim, N. I., Park, S., Paek, A. S., & Kweon, I. S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_31 · Cited by 306 · see citing works
Neural Photo Editing with Introspective Adversarial Networks
Brock, A., Lim, T., Ritchie, J. M., & Weston, N. (2016) · arXiv · doi:10.48550/arxiv.1609.07093 · Cited by 275 · see citing works
Weakly-supervised Disentangling with Recurrent Transformations for 3D View SynthesisPreprint
Yang, J., Reed, S., Yang, M., & Lee, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.00706 · Cited by 247 · see citing works
Reed, S., Zhang, Y., Zhang, Y., & Lee, H. (2015) · neural information processing systems · Cited by 197 · see citing works
Neural Photo Editing With Introspective Adversarial Networks
Brock, A., Lim, T., Ritchie, J. M., & Weston, N. (2017) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.807637 · Cited by 189 · see citing works
Flagged first, then by citations
5 of the 49 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Tao, A., Kautz, J., & Catanzaro, B. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00917 · Cited by 4,514 · see citing works
An overview of deep learning in medical imaging focusing on MRI
Lundervold, A. S., & Lundervold, A. (2019) · Zeitschrift für Medizinische Physik · doi:10.1016/j.zemedi.2018.11.002 · Cited by 2,058 · see citing works
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
Gui, J., Sun, Z., Wen, Y., Tao, D., & Ye, J. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3130191 · Cited by 1,231 · see citing works
Yang, G., Yu, S., Dong, H., Slabaugh, G., Dragotti, P. L., Ye, X., Liu, F., Arridge, S., Keegan, J., Guo, Y., et al. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2785879 · Cited by 1,075 · see citing works
Self-citationStyleCLIP: Text-Driven Manipulation of StyleGAN Imagery
Patashnik, O., Wu, Z., Shechtman, E., Cohen‐Or, D., & Lischinski, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00209 · Cited by 1,079 · see citing works
An overview of deep learning in medical imaging focusing on MRI
Lundervold, A. S., & Lundervold, A. (2018) · NORA - Norwegian Open Research Archives · Cited by 1,012 · see citing works
Bandi, A., Adapa, P. V. S. R., & Kuchi, Y. E. V. P. K. (2023) · Future Internet · doi:10.3390/fi15080260 · Cited by 638 · see citing works
DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-To-Image Synthesis
Zhu, M., Pan, P., Chen, W., & Yang, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00595 · Cited by 605 · see citing works
A Survey on Generative Adversarial Networks: Variants, Applications, and Training
Jabbar, A., Li, X., & Omar, B. (2020) · ACM Computing Surveys · doi:10.1145/3463475 · Cited by 387 · see citing works
TediGAN: Text-Guided Diverse Face Image Generation and Manipulation
Xia, W., Yang, Y., Xue, J.-H., & Wu, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00229 · Cited by 376 · see citing works
Photographic Text-to-Image Synthesis with a Hierarchically-Nested Adversarial Network
Zhang, Z., Xie, Y., & Yang, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00649 · Cited by 331 · see citing works
VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance
Crowson, K., Biderman, S., Kornis, D., Stander, D., Hallahan, E., Castricato, L., & Raff, E. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_6 · Cited by 316 · see citing works
ManiGAN: Text-Guided Image Manipulation
Li, B., Qi, X., Lukasiewicz, T., & Torr, P. H. S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00790 · Cited by 287 · see citing works
Towards Open-Set Identity Preserving Face Synthesis
Bao, J., Chen, D., Wen, F., Li, H., & Hua, G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00702 · Cited by 270 · see citing works
Generative Adversarial Networks in Computer Vision: A Survey and Taxonomy
Wang, Z., She, Q., & Ward, T. E. (2022) · ACM Computing Surveys · doi:10.1145/3439723 · Cited by 267 · see citing works
Generative Adversarial Networks in Computer Vision: A Survey and TaxonomyPreprint
Wang, Z., She, Q., & Ward, T. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.01529 · Cited by 254 · see citing works
A Review on Generative Adversarial Networks: Algorithms, Theory, and ApplicationsPreprint
Gui, J., Sun, Z., Wen, Y., Tao, D., & Ye, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.06937 · Cited by 252 · see citing works
More Control for Free! Image Synthesis with Semantic Diffusion Guidance
Liu, X., Park, D. H., Azadi, S., Zhang, G., Chopikyan, A., Hu, Y., Shi, H., Rohrbach, A., & Darrell, T. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00037 · Cited by 239 · see citing works
Text2LIVE: Text-Driven Layered Image and Video Editing
Bar-Tal, O., Ofri-Amar, D., Fridman, R., Kasten, Y., & Dekel, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19784-0_41 · Cited by 222 · see citing works
Adversarial text-to-image synthesis: A review
Frolov, S., Hinz, T., Raue, F., Hees, J., & Dengel, A. R. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.07.019 · Cited by 214 · see citing works
Neural Rerendering in the Wild
Meshry, M., Goldman, D. B., Khamis, S., Hoppe, H., Pandey, R., Snavely, N., & Martin-Brualla, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00704 · Cited by 182 · see citing works
LayoutTransformer: Layout Generation and Completion with Self-attention
Gupta, K., Lazarow, J., Achille, A., Davis, L. S., Mahadevan, V., & Shrivastava, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00104 · Cited by 149 · see citing works
Tag2Pix: Line Art Colorization Using Text Tag With SECat and Changing Loss
Kim, H., Jhoo, H. Y., Park, E., & Yoo, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00915 · Cited by 120 · see citing works
A survey and taxonomy of adversarial neural networks for text‐to‐image synthesis
Agnese, J., Herrera, J., Tao, H., & Xingquan Zhu, A. (2020) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1345 · Cited by 112 · see citing works
Semantics-Enhanced Adversarial Nets for Text-to-Image Synthesis
Tan, H., Liu, X., Li, X. S., Zhang, Y., & Yin, B. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01060 · Cited by 93 · see citing works
Understanding GANs: fundamentals, variants, training challenges, applications, and open problems
Ahmad, Z., Jaffri, Z. U. A., Chen, M., & Bao, S. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19361-y · Cited by 92 · see citing works
Envedit: Environment Editing for Vision-and-Language Navigation
Li, J., Tan, H., & Bansal, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01497 · Cited by 87 · see citing works
Learning Canonical Representations for Scene Graph to Image Generation
Herzig, R., Bar, A., Xu, H., Chechik, G., Darrell, T., & Globerson, A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58574-7_13 · Cited by 92 · see citing works
Khan, M. U. G., Jabeen, S., Khan, M. U. G., Saba, T., Rehmat, A., Rehman, A. U., & Tariq, U. (2020) · IEEE Access · doi:10.1109/access.2020.3015656 · Cited by 77 · see citing works
Comprehensive Linguistic-Visual Composition Network for Image Retrieval
Wen, H., Song, X., Yang, X., Zhan, Y., & Nie, L. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462967 · Cited by 73 · see citing works
Liu, L., Zhang, H., Xu, X., Zhang, Z., & Yan, S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2944979 · Cited by 71 · see citing works
CPGAN: Content-Parsing Generative Adversarial Networks for Text-to-Image Synthesis
Liang, J., Pei, W., & Lu, F. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58548-8_29 · Cited by 77 · see citing works
Generative Adversarial Networks: A Survey and Taxonomy.Preprint
Wang, Z., She, Q., & Ward, T. E. (2019) · arXiv (Cornell University) · Cited by 65 · see citing works
Dong, H., Supratak, A., Mai, L., Liu, F., Oehmichen, A., Yu, S., & Guo, Y. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3129391 · Cited by 61 · see citing works
Self-citationStyleMC: Multi-Channel Based Fast Text-Guided Image Generation and Manipulation
Kocasarı, U., Dirik, A., Tiftikci, M., & Yanardag, P. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00350 · Cited by 57 · see citing works
A Diffusion Model Translator for Efficient Image-to-Image Translation
Xia, M., Zhou, Y., Yi, R., Liu, Y., & Wang, W. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3435448 · Cited by 57 · see citing works
A survey on generative adversarial network-based text-to-image synthesis
Zhou, R. H., Jiang, C. F., & Xu, Q. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.04.069 · Cited by 52 · see citing works
Cycle-Consistent Inverse GAN for Text-to-Image Synthesis
Wang, H., Lin, G., Hoi, S. C. H., & Miao, C. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475226 · Cited by 52 · see citing works
AnyFace: Free-style Text-to-Face Synthesis and Manipulation
Sun, J., Deng, Q., Li, Q., Sun, M., Ren, M., & Sun, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01813 · Cited by 52 · see citing works
Visual privacy attacks and defenses in deep learning: a survey
Zhang, G., Liu, B., Zhu, T., Zhou, A., & Zhou, W. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10123-y · Cited by 53 · see citing works
MC-GAN: Multi-conditional Generative Adversarial Network for Image SynthesisPreprint
Park, H., Yoo, Y., & Kwak, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.01123 · Cited by 51 · see citing works
Language-based Photo Color Adjustment for Graphic Designs
Wang, Z., Zhao, N., Hancke, G. P., & Lau, R. W. H. (2023) · ACM Transactions on Graphics · doi:10.1145/3592111 · Cited by 49 · see citing works
Text Guided Person Image Synthesis
Zhou, X., Huang, S., Li, B., Li, Y., Li, J., & Zhang, Z. M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00378 · Cited by 47 · see citing works
Zhao, J., Zhou, X., Shi, G., Xiao, N., Song, K., Zhao, J., Hao, R., & Li, K. (2022) · Applied Intelligence · doi:10.1007/s10489-021-03025-7 · Cited by 46 · see citing works
Sound-Guided Semantic Image Manipulation
Lee, S. H., Roh, W., Byeon, W., Yoon, S. H., Kim, C. Y., Kim, J., & Kim, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00337 · Cited by 45 · see citing works
FICE: Text-conditioned fashion-image editing with guided GAN inversion
Pernuš, M., Fookes, C., Štruc, V., & Dobrišek, S. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.111022 · Cited by 44 · see citing works
Xu, Z., Lin, T., Tang, H., Li, F., He, D., Sebe, N., Timofte, R., Gool, L. V., & Ding, E. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01769 · Cited by 45 · see citing works
Lightweight Generative Adversarial Networks for Text-Guided Image ManipulationPreprint
Li, B., Qi, X., Torr, P. H. S., & Lukasiewicz, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.12136 · Cited by 45 · see citing works
Knowledge-Driven Generative Adversarial Network for Text-to-Image Synthesis
Peng, J., Zhou, Y., Sun, X., Cao, L., Wu, Y., Huang, F., & Ji, R. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3116416 · Cited by 41 · see citing works
TensorLayer: A Versatile Library for Efficient Deep Learning DevelopmentPreprint
Dong, H., Supratak, A., Mai, L., Liu, F., Oehmichen, A., Yu, S., & Guo, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.08551 · Cited by 39 · see citing works
Self-citationA game-theoretic approach for federated learning: A trade-off among privacy, accuracy and energy
Yin, L., Lin, S., Sun, Z., Li, R., He, Y., & Hao, Z. (2023) · Digital Communications and Networks · doi:10.1016/j.dcan.2022.12.024 · Cited by 37 · see citing works
Retro-Remote Sensing: Generating Images From Ancient Texts
Bejiga, M. B., Melgani, F., & Vascotto, A. (2019) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2019.2895693 · Cited by 35 · see citing works
Optimizing loss functions through multi-variate taylor polynomial parameterization
Gonzalez, S., & Miikkulainen, R. P. (2021) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3449639.3459277 · Cited by 34 · see citing works
CF-GAN: cross-domain feature fusion generative adversarial network for text-to-image synthesis
Zhang, Y., Han, S., Zhang, Z., Wang, J., & Bi, H. (2022) · The Visual Computer · doi:10.1007/s00371-022-02404-6 · Cited by 34 · see citing works
Deceptive jamming template synthesis for SAR based on generative adversarial nets
Fan, W., Zhou, F., Zhang, Z., Bai, X., & Tian, T. (2020) · Signal Processing · doi:10.1016/j.sigpro.2020.107528 · Cited by 33 · see citing works
Learning by Planning: Language-Guided Global Image Editing
Shi, J., Xu, N., Xu, Y., Bui, T., Dernoncourt, F., & Xu, C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01338 · Cited by 33 · see citing works
PanoDR: Spherical Panorama Diminished Reality for Indoor Scenes
Gkitsas, V., Sterzentsenko, V., Zioulis, N., Albanis, G., & Zarpalas, D. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00412 · Cited by 33 · see citing works
OpenGAN: Open Set Generative Adversarial Networks
Ditria, L., Meyer, B. J., & Drummond, T. W. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69538-5_29 · Cited by 32 · see citing works
Generative Attribute Manipulation Scheme for Flexible Fashion Search
Yang, X., Song, X., Han, X., Wen, H., Nie, J., & Nie, L. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401150 · Cited by 32 · see citing works
Covariance Attention for Semantic Segmentation
Liu, Y., Chen, Y., Lasang, P., & Sun, Q. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3026069 · Cited by 30 · see citing works
Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions
Liu, X., Lin, Z., Zhang, J., Zhao, H., Tran, Q., Wang, X., & Li, H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58621-8_6 · Cited by 32 · see citing works
High-fidelity 3D Face Generation from Natural Language Descriptions
Wu, M., Zhu, H., Huang, L., Zhuang, Y., Lu, Y., & Cao, X. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00439 · Cited by 31 · see citing works
TediGAN: Text-Guided Diverse Face Image Generation and ManipulationPreprint
Xia, W., Yang, Y., Xue, J., & Wu, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.03308 · Cited by 31 · see citing works
Language Guided Fashion Image Manipulation with Feature-wise TransformationsPreprint
Günel, M., Erdem, E., & Erdem, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.04000 · Cited by 33 · see citing works
Show Me a Story: Towards Coherent Neural Story Illustration
Ravi, H., Wang, L., Muñiz, C. M., Sigal, L., Metaxas, D., & Kapadia, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00794 · Cited by 28 · see citing works
TextFace: Text-to-Style Mapping Based Face Generation and Manipulation
Hou, X., Zhang, X., Li, Y., & Shen, L. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3160360 · Cited by 28 · see citing works
Li, P., Wang, Y., Si, T., Ullah, K., Han, W., & Wang, L. (2023) · International Journal of Digital Earth · doi:10.1080/17538947.2023.2229794 · Cited by 30 · see citing works
Mao, X., Chen, Y., Li, Y., Xiong, T., He, Y., & Xue, H. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683008 · Cited by 27 · see citing works
FTGAN: A Fully-trained Generative Adversarial Networks for Text to Face GenerationPreprint
Chen, X., Qing, L., He, X., Luo, X., & Xu, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.05729 · Cited by 27 · see citing works
Berrahal, M., & Azizi, M. (2022) · Indonesian Journal of Electrical Engineering and Computer Science · doi:10.11591/ijeecs.v25.i2.pp972-979 · Cited by 26 · see citing works
UFC-BERT: Unifying Multi-Modal Controls for Conditional Image SynthesisPreprint
Zhang, Z., Ma, J., Zhou, C., Men, R., Li, Z., Ding, M., Tang, J., Zhou, J., & Yang, H. (2021) · arXiv (Cornell University) · Cited by 26 · see citing works
Máttyus, G., & Urtasun, R. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00837 · Cited by 26 · see citing works
Deng, X., Zhu, Y., & Newsam, S. D. (2018) · ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (SIGSPATIAL GIS) · doi:10.1145/3274895.3274969 · Cited by 24 · see citing works
Neurocognition-inspired design with machine learning
Wang, P., Wang, S., Peng, D., Chen, L., Wu, C., Wei, Z. L., Childs, P., Guo, Y., & Li, L. (2020) · Design Science · doi:10.1017/dsj.2020.23 · Cited by 24 · see citing works
Self-citationSub-GAN: An Unsupervised Generative Model via Subspaces
Liang, J., Yang, J., Lee, H.-Y., Wang, K., & Yang, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_43 · Cited by 24 · see citing works
Towards Open-World Text-Guided Face Image Generation and ManipulationPreprint
Xia, W., Yang, Y., Xue, J., & Wu, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08910 · Cited by 27 · see citing works
Recent Advances in Text-to-Image Synthesis: Approaches, Datasets and Future Research Prospects
Tan, Y. X., Lee, C. P., Neo, M., Lim, K. M., Lim, J. Y., & Alqahtani, A. (2023) · IEEE Access · doi:10.1109/access.2023.3306422 · Cited by 23 · see citing works
TexFit: Text-Driven Fashion Image Editing with Diffusion Models
Wang, T., & Ye, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i9.28885 · Cited by 23 · see citing works
Zhuang, W., Chu, Q., Yuan, H.-J., Miao, C.-T., Liu, B., & Yu, N.-H. (2022) · IEEE International Conference on Multimedia and Expo · doi:10.1109/icme52920.2022.9859586 · Cited by 17 · see citing works
Visual-Relation Conscious Image Generation from Structured-Text
Vo, D. M., & Sugimoto, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58604-1_18 · Cited by 22 · see citing works
Qi, Y., Liu, Y., & Sun, Q. (2019) · IEEE Access · doi:10.1109/access.2019.2953698 · Cited by 21 · see citing works
Toward Multi-Modal Conditioned Fashion Image Translation
Gu, X., Yu, J., Wong, Y., & Kankanhalli, M. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3009500 · Cited by 20 · see citing works
Notice of Removal: DeltaEdit: Exploring Text-free Training for Text-Driven Image Manipulation
Lyu, Y., Lin, T., Li, F., He, D., Dong, J., & Tan, T. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00666 · Cited by 20 · see citing works
Image Generation with Gans-based Techniques: A Survey
Esfahani, S. N., & Latifi, S. (2019) · International Journal of Computer Science and Information Technology · doi:10.5121/ijcsit.2019.11503 · Cited by 20 · see citing works
IR-GAN: Image Manipulation with Linguistic Instruction by Increment Reasoning
Liu, Z., Deng, J., Li, L., Cai, S., Xu, Q., Wang, S., & Huang, Q. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413777 · Cited by 19 · see citing works
Language-Guided Global Image Editing via Cross-Modal Cyclic Mechanism
Jiang, W., Xu, N., Wang, J., Gao, C., Shi, J., Lin, Z., & Liu, S. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00212 · Cited by 19 · see citing works
Chen, Y., Chen, P., Zhou, X., Lei, Y., Zhou, Z., & Li, M. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650029 · Cited by 19 · see citing works
Wang, J., Lu, G., Xu, H., Li, Z., Xu, C., & Fu, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01044 · Cited by 19 · see citing works
Sampat, S. K., Kumar, A., Yang, Y., & Baral, C. R. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.289 · Cited by 18 · see citing works
A Review of Multi-Modal Learning from the Text-Guided Visual Processing Viewpoint
Ullah, U., Lee, J.-S., An, C.-H., Lee, H., Park, S.-Y., Baek, R., & Choi, H. (2022) · Sensors · doi:10.3390/s22186816 · Cited by 18 · see citing works
Word-Level Fine-Grained Story Visualization
Li, B. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_20 · Cited by 18 · see citing works
SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Color Editing
Shi, J., Xu, N., Zheng, H., Smith, A., Luo, J., & Xu, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01911 · Cited by 18 · see citing works
Multimodality-guided Image Style Transfer using Cross-modal GAN Inversion
Wang, H., Wu, P., Rosa, K. D., Wang, C., & Shrivastava, A. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00490 · Cited by 20 · see citing works
Zhang, Z., Zhao, Z., Zhang, Z., Huai, B., & Yuan, J. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413939 · Cited by 18 · see citing works
Pande, S., Chouhan, S., Sonavane, R., Walambe, R. A., Ghinea, G., & Kotecha, K. V. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.08.055 · Cited by 18 · see citing works
Learning by Imagination: A Joint Framework for Text-Based Image Manipulation and Change Captioning
Ak, K. E., Sun, Y., & Lim, J. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3154154 · Cited by 17 · see citing works
Text-Guided Human Image Manipulation via Image-Text Shared Space
Xu, X., Chen, Y.-C., Tao, X., & Jia, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3085339 · Cited by 16 · see citing works
Text-Guided Sketch-to-Photo Image Synthesis
Osahor, U., & Nasrabadi, N. M. (2022) · IEEE Access · doi:10.1109/access.2022.3206771 · Cited by 16 · see citing works
Human-in-the-Loop Design with Machine Learning
Wang, P., Peng, D., Li, L., Chen, L., Wu, C., Wang, X., Childs, P., & Guo, Y. (2019) · Proceedings of the ... International Conference on Engineering Design · doi:10.1017/dsi.2019.264 · Cited by 16 · see citing works
Self-citationAttrLostGAN: Attribute Controlled Image Synthesis from Reconfigurable Layout and Style
Frolov, S., Sharma, A., Hees, J., Karayil, T., Raue, F., & Dengel, A. R. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-92659-5_23 · Cited by 16 · see citing works
- Authors
- Dong, H., Yu, S., Wu, C., & Guo, Y.
- Affiliations
- Imperial College London (GB)
- Venue
- 2017 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 5707-5715
- OpenAlex
- W2964313012
- Cited by
- 276
- References
- 49
- Citations by year
- 2026: 3 · 2025: 18 · 2024: 28 · 2023: 45 · 2022: 45 · 2021: 46 · 2020: 34 · 2019: 32 · 2018: 22 · 2017: 3
- FWCI
- 7.16
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Virginia Agricultural Experiment Station, Virginia Polytechnic Institute and State University
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- semantic image synthesis, text-to-image generation, adversarial learning, image editing, realistic image synthesis, multimodal learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
