The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Recovering Realistic Texture in Image Super-Resolution by Deep Spatial Feature Transform
Open accessAbstract
Despite that convolutional neural networks (CNN) have recently demonstrated high-quality reconstruction for single-image super-resolution (SR), recovering natural and realistic texture remains a challenging problem. In this paper, we show that it is possible to recover textures faithful to semantic classes. In particular, we only need to modulate features of a few intermediate layers in a single network conditioned on semantic segmentation probability maps. This is made possible through a novel Spatial Feature Transform (SFT) layer that generates affine transformation parameters for spatial-wise feature modulation. SFT layers can be trained end-to-end together with the SR network using the same loss function. During testing, it accepts an input image of arbitrary size and generates a high-resolution image with just a single forward pass conditioned on the categorical priors. Our final results show that an SR network equipped with SFT can generate more realistic and visually pleasing textures in comparison to state-of-the-art SRGAN [27] and EnhanceNet [38].
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
Image Super-Resolution Using Deep Convolutional Networks
Dong, C., Loy, C. C., He, K., & Tang, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2439281 · Cited by 10,105
Self-citation
Cited by
ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Qiao, Y., & Loy, C. C. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-11021-5_5 · Cited by 4,731
Self-citationSemantic Image Synthesis With Spatially-Adaptive Normalization
Park, T., Liu, M.-Y., Wang, T.-C., & Zhu, J.-Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00244 · Cited by 2,974
Deep Learning for Image Super-Resolution: A Survey
Wang, Z., Chen, J., & Hoi, S. C. H. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2982166 · Cited by 1,969
Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data
Wang, X., Xie, L., Dong, C., & Shan, Y. (2021) · 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw54120.2021.00217 · Cited by 1,684
Self-citationEdvr: video restoration with enhanced deformable convolutional networks
Wang, X., Chan, K. C. K., Yu, K., Dong, C., & Loy, C. C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00247 · Cited by 1,272
Self-citationA Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
Gui, J., Sun, Z., Wen, Y., Tao, D., & Ye, J. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3130191 · Cited by 1,231
Deep Learning for Single Image Super-Resolution: A Brief Review
Yang, W., Zhang, X., Tian, Y., Wang, W., Xue, J.-H., & Liao, Q. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2919431 · Cited by 1,154
MaskGAN: Towards Diverse and Interactive Facial Image Manipulation
Lee, C.-H., Liu, Z., Wu, L., & Luo, P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00559 · Cited by 1,130
Low-Light Image and Video Enhancement Using Deep Learning: A Survey
Li, C., Guo, C., Han, L., Jiang, J., Cheng, M.-M., Gu, J., & Loy, C. C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3126387 · Cited by 657
The 2018 PIRM Challenge on Perceptual Image Super-Resolution
Blau, Y., Mechrez, R., Timofte, R., Michaeli, T., & Zelnik-Manor, L. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-11021-5_21 · Cited by 593
Blind Super-Resolution With Iterative Kernel Correction
Gu, J., Lu, H., Zuo, W., & Dong, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00170 · Cited by 580
Self-citationImage Dehazing Transformer with Transmission-Aware 3D Position Embedding
Guo, C., Yan, Q., Anwar, S., Cong, R., Ren, W., & Li, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00572 · Cited by 552
Towards Real-World Blind Face Restoration with Generative Facial Prior
Wang, X., Li, Y., Zhang, H., & Shan, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00905 · Cited by 538
Self-citationMeta-SR: A Magnification-Arbitrary Network for Super-Resolution
Hu, X., Mu, H., Zhang, X., Wang, Z., Tan, T., & Sun, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00167 · Cited by 526
Domain Adaptation for Image Dehazing
Shao, Y., Li, L., Ren, W., Gao, C., & Sang, N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00288 · Cited by 474
Exploiting Diffusion Prior for Real-World Image Super-Resolution
Wang, J., Yue, Z., Zhou, S., Chan, K. C. K., & Loy, C. C. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02168-7 · Cited by 439
All-In-One Image Restoration for Unknown Corruption
Li, B., Liu, X., Hu, P., Wu, Z., Lv, J., & Peng, X. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01693 · Cited by 432
State of the Art on Neural Rendering
Tewari, A., Fried, O., Thies, J., Sitzmann, V., Lombardi, S., Sunkavalli, K., Martin‐Brualla, R., Simon, T., Saragih, J., Nießner, M., et al. (2020) · Computer Graphics Forum · doi:10.1111/cgf.14022 · Cited by 430
Structure-Preserving Super Resolution With Gradient Guidance
Ma, C., Rao, Y., Cheng, Y., Chen, C., Lu, J., & Zhou, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00779 · Cited by 415
RankSRGAN: Generative Adversarial Networks With Ranker for Image Super-Resolution
Zhang, W., Liu, Y., Dong, C., & Qiao, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00319 · Cited by 405
Self-citationImage Super-Resolution by Neural Texture Transfer
Zhang, Z., Wang, Z., Lin, Z., & Qi, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00817 · Cited by 379
SRFormer: Permuted Self-Attention for Single Image Super-Resolution
Zhou, Y., Li, Z., Guo, C.-L., Bai, S., Cheng, M.-M., & Hou, Q. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01174 · Cited by 374
DeepHuman: 3D Human Reconstruction From a Single Image
Zheng, Z., Yu, T., Wei, Y., Dai, Q., & Liu, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00783 · Cited by 368
DiffIR: Efficient Diffusion Model for Image Restoration
Xia, B., Zhang, Y., Wang, S., Wang, Y., Wu, X., Tian, Y., Yang, W., & Van Gool, L. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01204 · Cited by 412
Real-World Super-Resolution via Kernel Estimation and Noise Injection
Ji, X., Cao, Y., Tai, Y., Wang, C., Li, J., & Huang, F. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00241 · Cited by 332
Channel-Wise and Spatial Feature Modulation Network for Single Image Super-Resolution
Hu, Y., Li, J., Huang, Y., & Gao, X. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2915238 · Cited by 316
Image super-resolution: A comprehensive review, recent trends, challenges and applications
Lepcha, D. C., Goyal, B., Dogra, A., & Goyal, V. (2023) · Information Fusion · doi:10.1016/j.inffus.2022.10.007 · Cited by 305
Efficient and Explicit Modelling of Image Hierarchies for Image Restoration
Li, Y., Fan, Y., Xiang, X., Demandolx, D., Ranjan, R., Timofte, R., & Van Gool, L. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01753 · Cited by 302
Deep Generalized Unfolding Networks for Image Restoration
Mou, C., Wang, Q., & Zhang, J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01688 · Cited by 290
Densely Residual Laplacian Super-Resolution
Anwar, S., & Barnes, N. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3021088 · Cited by 281
GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution
Chan, K. C. K., Wang, X., Xu, X., Gu, J., & Loy, C. C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01402 · Cited by 268
Self-citationVideo Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics
Duan, L., Liu, J., Yang, W., Huang, T., & Gao, W. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3016485 · Cited by 260
Zhang, X., Chen, Q., Ng, R., & Koltun, V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00388 · Cited by 260
Component Divide-and-Conquer for Real-World Image Super-Resolution
Wei, P., Xie, Z., Lu, H., Zhan, Z., Ye, Q., Zuo, W., & Lin, L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58598-3_7 · Cited by 259
Effects of Image Degradation and Degradation Removal to CNN-Based Image Classification
Pei, Y., Huang, Y., Zou, Q., Zhang, X., & Wang, S. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2950923 · Cited by 253
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications
Gui, J., Sun, Z., Wen, Y., Tao, D., & Ye, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.06937 · Cited by 253
EDTER: Edge Detection with Transformer
Pu, M., Huang, Y., Liu, Y., Guan, Q., & Ling, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00146 · Cited by 253
Understanding deformable alignment in video super-resolution
Chan, K. C. K., Wang, X., Yu, K., Dong, C., & Loy, C. C. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16181 · Cited by 162
Self-citation
Metadata
- Authors
- Wang, X., Yu, K., Dong, C., & Change Loy, C.
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 606-615
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2963814095
- Cited by
- 1,170
- References
- 52
- FWCI
- 30.54
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.