Multimodal Style Transfer via Graph Cuts
Open accessAbstract
An assumption widely used in recent neural style transfer methods is that image styles can be described by global statics of deep features like Gram or covariance matrices. Alternative approaches have represented styles by decomposing them into local pixel or neural patches. Despite the recent progress, most existing methods treat the semantic patterns of style image uniformly, resulting unpleasing results on complex styles. In this paper, we introduce a more flexible and general universal style transfer technique: multimodal style transfer (MST). MST explicitly considers the matching of semantic patterns in content and style images. Specifically, the style image features are clustered into sub-style components, which are matched with local content features under a graph cut formulation. A reconstruction network is trained to transfer each sub-style and render the final stylized result. We also generalize MST to improve some existing methods. Extensive experiments demonstrate the superior effectiveness, robustness, and flexibility of MST.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A universal graph-based style transfer was proposed that transforms multimodal features by matching style patterns with semantic content and appearance, avoiding inflexibility in real-world scenarios.
“The authors of [159] proposed a universal graph-based style transfer to transform multimodal features by matching style patterns and semantic content and appearance in a way that avoids the lack of flexibility in real-world scenarios.”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [159]
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283 · see citing works
Fast approximate energy minimization via graph cuts
Boykov, Y., Veksler, O., & Zabih, R. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.969114 · Cited by 6,988 · see citing works
Image Style Transfer Using Convolutional Neural Networks
Gatys, L. A., Ecker, A. S., & Bethge, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.265 · Cited by 6,025 · see citing works
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization
Huang, X., & Belongie, S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.167 · Cited by 5,076 · see citing works
Texture synthesis by non-parametric sampling
Efros, A. A., & Leung, T. K. (1999) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.1999.790383 · Cited by 3,058 · see citing works
Kwatra, V., Schödl, A., Essa, I. A., Turk, G., & Bobick, A. (2003) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1201775.882264 · Cited by 1,216 · see citing works
Exact Maximum A Posteriori Estimation for Binary Images
Greig, D. M., Porteous, B. T., & Seheult, A. H. (1989) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1989.tb01764.x · Cited by 1,066 · see citing works
Neural Style Transfer: A Review
Jing, Y., Yang, Y., Feng, Z., Ye, J., Yu, Y., & Song, M. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2921336 · Cited by 894 · see citing works
Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis
Li, C., & Wand, M. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.272 · Cited by 758 · see citing works
Multi-camera Scene Reconstruction via Graph Cuts
Kolmogorov, V., & Ramin Zabih, A. (2002) · Lecture notes in computer science · doi:10.1007/3-540-47977-5_6 · Cited by 745 · see citing works
(2012) · SpringerReference · doi:10.1007/springerreference_70943 · Cited by 731 · see citing works
Luan, F., Paris, S., Shechtman, E., & Bala, K. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.740 · Cited by 729 · see citing works
Visual Semantic Reasoning for Image-Text Matching
Li, K., Zhang, Y., Li, K., Li, Y., & Fu, Y. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00475 · Cited by 628 · see citing works
Self-citationDemystifying Neural Style Transfer
Li, Y., Wang, N., Liu, J., & Hou, X. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/310 · Cited by 539 · see citing works
Universal Style Transfer via Feature TransformsPreprint
Li, Y., Chen, F., Yang, J., Wang, Z., Lü, X., & Yang, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08086 · Cited by 538 · see citing works
Self-citationA maximum-flow formulation of the N-camera stereo correspondence problem
Roy, S., & Cox, I. J. (2002) · International Conference on Computer Vision · doi:10.1109/iccv.1998.710763 · Cited by 489 · see citing works
The Contextual Loss for Image Transformation with Non-aligned Data
Mechrez, R., Talmi, I., & Zelnik‐Manor, L. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_47 · Cited by 424 · see citing works
Avatar-Net: Multi-scale Zero-Shot Style Transfer by Feature Decoration
Sheng, L., Lin, Z., Shao, J., & Wang, X. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00860 · Cited by 323 · see citing works
Self-citationUniversal Style Transfer via Feature Transforms
Li, Y., Chen, F., Yan, S., Wang, Z., Lü, X., & Yang, M. (2017) · neural information processing systems · Cited by 300 · see citing works
Self-citationArtistic Style Transfer for Videos
Ruder, M., Dosovitskiy, A., & Brox, T. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-45886-1_3 · Cited by 295 · see citing works
State of the "Art”: A Taxonomy of Artistic Stylization Techniques for Images and Video
Kyprianidis, J. E., Collomosse, J. P., Wang, T., et al. (2012) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.160 · Cited by 280 · see citing works
A Style-Aware Content Loss for Real-Time HD Style Transfer
Sanakoyeu, A., Kotovenko, D., Lang, S., & Ommer, B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_43 · Cited by 238 · see citing works
Fast Patch-based Style Transfer of Arbitrary StylePreprint
Chen, T. Q., & Schmidt, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.04337 · Cited by 221 · see citing works
Separating Style and Content for Generalized Style Transfer
Zhang, Y., Zhang, Y., & Cai, W. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00881 · Cited by 204 · see citing works
Self-citationArbitrary Style Transfer with Deep Feature Reshuffle
Gu, S., Chen, C., Liao, J., et al. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00858 · Cited by 195 · see citing works
Wang, X., Oxholm, G., Zhang, D., & Wang, Y.-F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.759 · Cited by 182 · see citing works
Self-citationStyle Transfer Via Texture Synthesis
Elad, M., & Milanfar, P. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2678168 · Cited by 160 · see citing works
Rethinking Zero-Shot Learning: A Conditional Visual Classification Perspective
Li, K., Min, M. R., & Fu, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00368 · Cited by 144 · see citing works
Self-citationNeural Style Transfer via Meta Networks
Shen, F., Yan, S., & Zeng, G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00841 · Cited by 143 · see citing works
Stroke Controllable Fast Style Transfer with Adaptive Receptive Fields
Jing, Y., Liu, Y., Yang, Y., Feng, Z., Yu, Y., Tao, D., & Song, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01261-8_15 · Cited by 125 · see citing works
Stereoscopic Neural Style Transfer
Chen, D., Yuan, L., Liao, J., Yu, N., & Hua, G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00696 · Cited by 110 · see citing works
Neural Style Transfer: A ReviewPreprint
Jing, Y., Yang, Y., Feng, Z., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.04058 · Cited by 102 · see citing works
Learning Linear Transformations for Fast Arbitrary Style TransferPreprint
Li, X., Liu, S., Kautz, J., & Yang, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.04537 · Cited by 90 · see citing works
Image segmentation by nested cuts
Veksler, O. (2002) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2000.855838 · Cited by 82 · see citing works
Guided Attention Inference Network
Li, K., Wu, Z., Peng, K., Ernst, J., & Fu, Y. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2921543 · Cited by 59 · see citing works
Self-citationA new Bayesian framework for object recognition
Boykov, Y., & Huttenlocher, D. P. (2003) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1999.784730 · Cited by 52 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Bayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A. (2022) · The Visual Computer · doi:10.1007/s00371-021-02166-7 · Cited by 456 · see citing works
Wang, Y., Yu, K., Chen, C., Hu, X., & Peng, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00703 · Cited by 161 · see citing works
Self-citationRethinking Zero-Shot Learning: A Conditional Visual Classification Perspective
Li, K., Min, M. R., & Fu, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00368 · Cited by 144 · see citing works
Self-citation
- Authors
- Zhang, Y., Fang, C., Wang, Y., Wang, Z., Lin, Z., Fu, Y., & Yang, J.
- Affiliations
- Northeastern University (US)
- Adobe Systems (United States) (US)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Volume
- 2019
- Pages
- 5942-5950
- OpenAlex
- W2990798426
- Cited by
- 98
- References
- 40
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 17 · 2023: 19 · 2022: 22 · 2021: 27 · 2020: 5 · 2019: 2
- FWCI
- 4.51
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- IEEE Conference Proceedings
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- neural style transfer, graph cuts
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
