Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization
Abstract
Gatys et al. recently introduced a neural algorithm that renders a content image in the style of another image, achieving so-called style transfer. However, their framework requires a slow iterative optimization process, which limits its practical application. Fast approximations with feed-forward neural networks have been proposed to speed up neural style transfer. Unfortunately, the speed improvement comes at a cost: the network is usually tied to a fixed set of styles and cannot adapt to arbitrary new styles. In this paper, we present a simple yet effective approach that for the first time enables arbitrary style transfer in real-time. At the heart of our method is a novel adaptive instance normalization (AdaIN) layer that aligns the mean and variance of the content features with those of the style features. Our method achieves speed comparable to the fastest existing approach, without the restriction to a pre-defined set of styles. In addition, our approach allows flexible user controls such as content-style trade-off, style interpolation, color & spatial controls, all using a single feed-forward neural network.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
Return of Frustratingly Easy Domain Adaptation
Sun, B., Feng, J., & Saenko, K. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10306 · Cited by 2,004 · see citing works
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
Bousmalis, K., Silberman, N., Dohan, D., Erhan, D., & Krishnan, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.18 · Cited by 1,603 · see citing works
Wang, X., Oxholm, G., Zhang, D., & Wang, Y.-F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.759 · Cited by 182 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714 · see citing works
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332 · see citing works
Semantic Image Synthesis With Spatially-Adaptive Normalization
Park, T., Liu, M.-Y., Wang, T.-C., & Zhu, J.-Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00244 · Cited by 2,974 · see citing works
Film: Visual reasoning with a general conditioning layer
Perez, E., Strub, F., De Vries, H., Dumoulin, V., & Courville, A. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11671 · Cited by 2,490 · see citing works
Deep visual domain adaptation: A survey
Wang, M., & Deng, W. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.05.083 · Cited by 2,209 · see citing works
StarGAN v2: Diverse Image Synthesis for Multiple Domains
Choi, Y., Uh, Y., Yoo, J., & Ha, J.-W. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00821 · Cited by 1,663 · see citing works
MaskGAN: Towards Diverse and Interactive Facial Image Manipulation
Lee, C.-H., Liu, Z., Wu, L., & Luo, P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00559 · Cited by 1,130 · see citing works
Deep learning modelling techniques: Current progress, applications, advantages, and challenges
Ahmed, S. F., Alam, M. S. B., Hassan, M., Rozbu, M. R., Ishtiak, T., Rafa, N., Mofijur, M., Shawkat Ali, A. B. M., & Gandomi, A. H. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10466-8 · Cited by 1,079 · see citing works
Generalizing to unseen domains: A survey on domain generalization
Wang, J., Lan, C., Liu, C., Ouyang, Y., Qin, T., Lu, W., Chen, Y., Zeng, W., & Yu, P. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3178128 · Cited by 1,046 · see citing works
Few-Shot Adversarial Learning of Realistic Neural Talking Head Models
Zakharov, E., Shysheya, A., Burkov, E., & Lempitsky, V. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00955 · Cited by 687 · see citing works
Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., & Brendel, W. (2018) · arXiv · doi:10.48550/arxiv.1811.12231 · Cited by 1,490 · see citing works
“…a new data set (termed Stylized-ImageNet or SIN) by strip-ping every single image of its original texture and replacing it with the style of a randomly selected painting through AdaIN style transfer (Huang & Belongie, 2017) (see examples in Figure 3) with a stylization coefficient of α = 1 .”
MethodDeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection
Jiang, L., Li, R., Wu, W., Qian, C., & Loy, C. C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00296 · Cited by 599 · see citing works
Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation: The M&Ms Challenge
Campello, V. M., Gkontra, P., Izquierdo, C., Martin-Isla, C., Sojoudi, A., Full, P. M., Maier-Hein, K., Zhang, Y., He, Z., Ma, J., et al. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3090082 · Cited by 481 · see citing works
Self-citationHoloGAN: Unsupervised Learning of 3D Representations From Natural Images
Nguyen-Phuoc, T., Li, C., Theis, L., Richardt, C., & Yang, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00768 · Cited by 340 · see citing works
Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning
Deng, Y., Yang, J., Chen, D., Wen, F., & Tong, X. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00520 · Cited by 329 · see citing works
Unsupervised Domain Adaptation of Object Detectors: A Survey
Oza, P., Sindagi, V. A., VS, V., & Patel, V. M. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3217046 · Cited by 313 · see citing works
Advancing High Fidelity Identity Swapping for Forgery Detection
Li, L., Bao, J., Yang, H., Chen, D., & Wen, F. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00512 · Cited by 288 · see citing works
Combining Noise-to-Image and Image-to-Image GANs: Brain MR Image Augmentation for Tumor Detection
Han, C., Rundo, L., Araki, R., Nagano, Y., Furukawa, Y., Mauri, G., Nakayama, H., & Hayashi, H. (2019) · IEEE Access · doi:10.1109/access.2019.2947606 · Cited by 253 · see citing works
Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
Liu, M.-Y., Huang, X., Yu, J., Wang, T.-C., & Mallya, A. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3049196 · Cited by 236 · see citing works
Self-citationImage-To-Image Translation via Group-Wise Deep Whitening-And-Coloring Transformation
Cho, W., Choi, S., Park, D. K., Shin, I., & Choo, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01089 · Cited by 154 · see citing works
“To alleviate the limitation, several approaches enable an unseen, arbitrary neural style transfer [13, 23, 24].”
BackgroundMethodInfluentialMarioNETte: Few-Shot Face Reenactment Preserving Identity of Unseen Targets
Ha, S., Kersner, M., Kim, B., Seo, S., & Kim, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6721 · Cited by 157 · see citing works
Zhou, Y., Wang, B., He, X., Cui, S., & Shao, L. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3045475 · Cited by 155 · see citing works
Deep learning-based spacecraft relative navigation methods: A survey
Song, J., Rondao, D., & Aouf, N. (2022) · Acta Astronautica · doi:10.1016/j.actaastro.2021.10.025 · Cited by 106 · see citing works
“[50], they generate a new dataset by applying neural style transfer techniques [51] to a custom synthetic dataset with the same pose distribution as SPEED.”
MethodMultimodal Style Transfer via Graph Cuts
Zhang, Y., Fang, C., Wang, Y., Wang, Z., Lin, Z., Fu, Y., & Yang, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00604 · Cited by 98 · see citing works
X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers
Cho, J., Lu, J., Schwenk, D., Hajishirzi, H., & Kembhavi, A. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.707 · Cited by 99 · see citing works
VQVC+: One-Shot Voice Conversion by Vector Quantization and U-Net Architecture
Wu, D.-Y., Chen, Y.-H., & Lee, H.-Y. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-1443 · Cited by 88 · see citing works
Conceptualizing generative AI as style engines: Application archetypes and implications
Riemer, K., & Peter, S. (2024) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2024.102824 · Cited by 63 · see citing works
Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
Li, K., Patel, O., Viégas, F., Pfister, H., & Wattenberg, M. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1797 · Cited by 117 · see citing works
“The latter connects to the whitening and coloring transformation commonly used in deep learning (Ioffe and Szegedy, 2015; Huang and Belongie, 2017).”
BackgroundSykas, D., Sdraka, M., Zografakis, D., & Papoutsis, I. (2022) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2022.3164771 · Cited by 59 · see citing works
HQG-Net: Unpaired Medical Image Enhancement With High-Quality Guidance
He, C., Li, K., Xu, G., Yan, J., Tang, L., Zhang, Y., Wang, Y., & Li, X. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3315307 · Cited by 54 · see citing works
Visual Deepfake Detection: Review of Techniques, Tools, Limitations, and Future Prospects
Ur Rehman Ahmed, N., Badshah, A., Adeel, H., Tajammul, A., Daud, A., & Alsahfi, T. (2025) · IEEE Access · doi:10.1109/access.2024.3523288 · Cited by 45 · see citing works
Retrieval-based Spatially Adaptive Normalization for Semantic Image Synthesis
Shi, Y., Liu, X., Wei, Y., Wu, Z., & Zuo, W. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01094 · Cited by 36 · see citing works
Audio–visual deepfake detection using articulatory representation learning
Wang, Y., & Huang, H. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.104133 · Cited by 28 · see citing works
Empowering a Creative City: Engage Citizens in Creating Street Art through Human-AI Collaboration
Li, Z., Wang, Y., Wang, W., Greuter, S., & Mueller, F. '. (2020) · Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3334480.3382976 · Cited by 27 · see citing works
AOT: Appearance Optimal Transport Based Identity Swapping for Forgery Detection
Zhu, H., Fu, C., Wu, Q., Wu, W., Qian, C., & He, R. (2020) · arXiv · doi:10.48550/arxiv.2011.02674 · Cited by 20 · see citing works
GAN-based Defect Synthesis for Anomaly Detection in Fabrics
Rippel, O., Muller, M., & Merhof, D. (2020) · 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) · doi:10.1109/etfa46521.2020.9212099 · Cited by 22 · see citing works
What Do Self-Supervised Vision Transformers Learn?
Park, N., Kim, W., Heo, B., Kim, T., & Yun, S. (2023) · arXiv · doi:10.48550/arxiv.2305.00729 · Cited by 29 · see citing works
“To demonstrate this claim, we use Stylized ImageNet (Geirhos et al., 2019), a texture-altered dataset, by using AdaIN (Huang & Belongie, 2017).”
MethodTunable image quality control of 3-D ultrasound using switchable CycleGAN
Huh, J., Khan, S., Choi, S., Shin, D., Lee, J. E., Lee, E. S., & Ye, J. C. (2023) · Medical Image Analysis · doi:10.1016/j.media.2022.102651 · Cited by 14 · see citing works
Real-time style transfer with efficient vision transformers
Benmeziane, H., Ouarnoughi, H., Maghraoui, K. E., & Niar, S. (2022) · Proceedings of the 5th International Workshop on Edge Systems, Analytics and Networking · doi:10.1145/3517206.3526271 · Cited by 5 · see citing works
- Authors
- Huang, X., & Belongie, S.
- Affiliations
- Cornell University (US)
- ORCID
- Belongie, Serge
- Venue
- 2017 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 1510-1519
- OpenAlex
- W2603777577
- Cited by
- 5,076
- References
- 5
- Citations by year
- 2026: 391 · 2025: 761 · 2024: 836 · 2023: 867 · 2022: 723 · 2021: 726 · 2020: 461 · 2019: 226 · 2018: 76 · 2017: 8 · 2016: 1
- FWCI
- 61.66
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
- Keywords
- neural style transfer, adaptive instance normalization, feedforward neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
