The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
StyTr2: Image Style Transfer with Transformers
Abstract
The goal of image style transfer is to render an image with artistic features guided by a style reference while maintaining the original content. Owing to the locality in convolutional neural networks (CNNs), extracting and maintaining the global information of input images is difficult. Therefore, traditional neural style transfer methods face biased content representation. To address this critical issue, we take long-range dependencies of input images into account for image style transfer by proposing a transformer-based approach called StyTr2. In contrast with visual transformers for other vision tasks, StyTr2 contains two different transformer encoders to generate domain-specific sequences for content and style, respectively. Following the encoders, a multi-layer transformer decoder is adopted to stylize the content sequence according to the style sequence. We also analyze the deficiency of existing positional encoding methods and propose the content-aware positional encoding (CAPE), which is scale-invariant and more suitable for image style transfer tasks. Qualitative and quantitative experiments demonstrate the effectiveness of the proposed StyTr2 compared with state-of-the-art CNN-based and flow-based approaches. Code and models are available at https://github.com/diyiiyiii/StyTR-2.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687
Image Style Transfer Using Convolutional Neural Networks
Gatys, L. A., Ecker, A. S., & Bethge, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.265 · Cited by 6,025
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization
Huang, X., & Belongie, S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.167 · Cited by 5,076
Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., Torr, P. H. S., et al. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00681 · Cited by 4,008 · cited 2×
“Inspired by the breakthrough of transformer in NLP, many researchers have developed vision transformers for various image/video related tasks [60], including object detection [6, 12, 65], semantic segmentation [53, 64], image classification [10, 16, 37, 55, 61], image processing and generation [8, 10, 24].”
MethodSelf-Attention with Relative Position Representations
Shaw, P., Uszkoreit, J., & Vaswani, A. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2074 · Cited by 2,505
Multimodal Unsupervised Image-to-Image Translation
Huang, X., Liu, M.-Y., Belongie, S., & Kautz, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_11 · Cited by 2,451
Image quilting for texture synthesis and transfer
Efros, A. A., & Freeman, W. T. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383296 · Cited by 2,347
Pre-Trained Image Processing Transformer
Chen, H., Wang, Y., Guo, T., Xu, C., Deng, Y., Liu, Z., Ma, S., Xu, C., Xu, C., & Gao, W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01212 · Cited by 2,308
Self-citationContrastive Learning for Unpaired Image-to-Image Translation
Park, T., Efros, A. A., Zhang, R., & Zhu, J.-Y. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58545-7_19 · Cited by 1,442
Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks
Li, C., & Wand, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_43 · Cited by 1,421
DEBERTA: DECODING-ENHANCED BERT WITH DISENTANGLED ATTENTION
He, P., Liu, X., Gao, J., & Chen, W. (2021) · International Conference on Learning Representations · Cited by 1,116
(2020) · International Conference on Machine Learning · Cited by 1,012
A Learned Representation For Artistic Style
Dumoulin, V., Shlens, J., & Kudlur, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.07629 · Cited by 763
End-to-End Video Instance Segmentation with Transformers
Wang, Y., Xu, Z., Wang, X., Shen, C., Cheng, B., Shen, H., & Xia, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00863 · Cited by 679
UP-DETR: Unsupervised Pre-training for Object Detection with Transformers
Dai, Z., Cai, B., Lin, Y., & Chen, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00165 · Cited by 520
Demystifying Neural Style Transfer
Li, Y., Wang, N., Liu, J., & Hou, X. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/310 · Cited by 514
StyleBank: An Explicit Representation for Neural Image Style Transfer
Chen, D., Yuan, L., Liao, J., Yu, N., & Hua, G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.296 · Cited by 502
Arbitrary Style Transfer With Style-Attentional Networks
Park, D. Y., & Lee, K. H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00603 · Cited by 466
AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer
Liu, S., Lin, T., He, D., Li, F., Wang, M., Li, X., Sun, Z., Li, Q., & Ding, E. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00658 · Cited by 420
A Closed-Form Solution to Photorealistic Image Stylization
Li, Y., Liu, M.-Y., Li, X., Yang, M.-H., & Kautz, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_28 · Cited by 400
Visual Transformers: Token-based Image Representation and Processing for Computer Vision
Wu, B., Xu, C., Dai, X., Wan, A., Zhang, P., Yan, Z., Tomizuka, M., Gonzalez, J., Keutzer, K., & Vajda, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03677 · Cited by 357
Self-citationUniversal Style Transfer via Feature Transforms
Li, Y., Chen, F., Yang, J., Wang, Z., Lü, X., & Yang, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08086 · Cited by 346
Avatar-Net: Multi-scale Zero-Shot Style Transfer by Feature Decoration
Sheng, L., Lin, Z., Shao, J., & Wang, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00860 · Cited by 313
ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows
An, J., Huang, S., Song, Y., Dou, D., Liu, W., & Luo, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00092 · Cited by 262
TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up
Jiang, Y., Chang, S., & Wang, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07074 · Cited by 254
How Much Position Information Do Convolutional Neural Networks Encode?
Islam, M. A., Jia, S., & Bruce, N. D. B. (2020) · arXiv · doi:10.48550/arxiv.2001.08248 · Cited by 253
Vision Transformers Are Robust Learners
Paul, S., & Chen, P.-Y. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20103 · Cited by 244 · cited 1×
“Second, the transformer architecture models relationships in input shapes [41], and different layers extract similar structural information [46] (see Fig.”
BackgroundArbitrary Style Transfer via Multi-Adaptation Network
Deng, Y., Tang, F., Dong, W., Sun, W., Huang, F., & Xu, C. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3414015 · Cited by 197
Self-citationEvo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformer
Xu, Y., Zhang, Z., Zhang, M., Sheng, K., Li, K., Dong, W., Zhang, L., Xu, C., & Sun, X. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20202 · Cited by 192
Self-citationStyle Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation
Dai, N., Liang, J., Qiu, X., & Huang, X. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1601 · Cited by 190
Attention-Aware Multi-Stroke Style Transfer
Yao, Y., Ren, J., Xie, X., Liu, W., Liu, Y.-J., & Wang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00156 · Cited by 190
Wiki Art Gallery, Inc.: A Case for Critical Thinking
Phillips, F., & Mackintosh, B. (2011) · Issues in Accounting Education · doi:10.2308/iace-50038 · Cited by 184
Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses
Risser, E., Wilmot, P., & Barnes, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.08893 · Cited by 177
Arbitrary Video Style Transfer via Multi-Channel Correlation
Deng, Y., Tang, F., Dong, W., Huang, H., Ma, C., & Xu, C. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16208 · Cited by 159
Self-citationTransformers in computational visual media: A survey
Xu, Y., Wei, H., Lin, M., Deng, Y., Sheng, K., Zhang, M., Tang, F., Dong, W., Huang, F., & Xu, C. (2021) · Computational Visual Media · doi:10.1007/s41095-021-0247-3 · Cited by 159
Self-citationStyleFormer: Real-time Arbitrary Style Transfer via Parametric Style Composition
Xiao-lei, W., Hu, Z., Sheng, L., & Xu, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01435 · Cited by 134
Collaborative Distillation for Ultra-Resolution Universal Style Transfer
Wang, H., Li, Y., Wang, Y., Hu, H. R., & Yang, M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00193 · Cited by 128
Diversified Arbitrary Style Transfer via Deep Feature Perturbation
Wang, Z., Zhao, L., Chen, H., Qiu, L., Mo, Q., Lin, S., Xing, W., & Lu, D. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00781 · Cited by 114
Positional Encoding as Spatial Inductive Bias in GANs
Xu, R., Wang, X., Chen, K., Zhou, B., & Loy, C. C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01336 · Cited by 108
Drafting and Revision: Laplacian Pyramid Network for Fast High-Quality Artistic Style Transfer
Lin, T., Ma, Z., Li, F., He, D., Li, X. S., Ding, E., Wang, N., Li, J., & Gao, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00510 · Cited by 105
Two-Stage Peer-Regularized Feature Recombination for Arbitrary Image Style Transfer
Svoboda, J., Anoosheh, A., Osendorfer, C., & Masci, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01383 · Cited by 86
Ultrafast Photorealistic Style Transfer via Neural Architecture Search
An, J., Xiong, H., Huan, J., & Luo, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6614 · Cited by 75
EFANet: Exchangeable Feature Alignment Network for Arbitrary Style Transfer
Wu, Z., Song, C., Zhou, Y., Gong, M., & Huang, H. F. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6914 · Cited by 32
Distribution Aligned Multimodal and Multi-domain Image Stylization
Lin, M., Tang, F., Dong, W., Li, X., Xu, C., & Ma, C. (2021) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3450525 · Cited by 25
Self-citationReal-time Universal Style Transfer on High-resolution Images via Zero-channel PruningPreprint
An, J., Li, T., Huang, H., Li, S., Wang, X., Tang, Y., Ma, J., Liu, W., & Luo, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.09029 · Cited by 17
Cited by
SRFormer: Permuted Self-Attention for Single Image Super-Resolution
Zhou, Y., Li, Z., Guo, C.-L., Bai, S., Cheng, M.-M., & Hou, Q. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01174 · Cited by 374
Inversion-based Style Transfer with Diffusion Models
Zhang, Y., Huang, N., Tang, F., Huang, H., Ma, C., Dong, W., & Xu, C. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00978 · Cited by 353
Self-citationMitigating Viewer Impact From Disturbing Imagery Using AI Filters: A User-Study
Sarridis, I., Spangenberg, J., Papadopoulou, O., & Papadopoulos, S. (2025) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2313890 · Cited by 8
Real-time style transfer with efficient vision transformers
Benmeziane, H., Ouarnoughi, H., Maghraoui, K. E., & Niar, S. (2022) · Proceedings of the 5th International Workshop on Edge Systems, Analytics and Networking · doi:10.1145/3517206.3526271 · Cited by 5
Metadata
- Authors
- Deng, Y., Tang, F., Dong, W., Ma, C., Pan, X., Wang, L., & Xu, C.
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 11316-11326
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4313029666
- Cited by
- 402
- References
- 60
- FWCI
- 32.16
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.