The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Interpreting Super-Resolution Networks with Local Attribution Maps
Open accessAbstract
Image super-resolution (SR) techniques have been developing rapidly, benefiting from the invention of deep networks and its successive breakthroughs. However, it is acknowledged that deep learning and deep neural networks are difficult to interpret. SR networks inherit this mysterious nature and little works make attempt to understand them. In this paper, we perform attribution analysis of SR networks, which aims at finding the input pixels that strongly influence the SR results. We propose a novel attribution approach called local attribution map (LAM), which inherits the integral gradient method yet with two unique features. One is to use the blurred image as the baseline input, and the other is to adopt the progressive blurring function as the path function. Based on LAM, we show that: (1) SR networks with a wider range of involved input pixels could achieve better performance. (2) Attention networks and non-local networks extract features from a wider range of input pixels. (3) Comparing with the range that actually contributes, the receptive field is large enough for most deep networks. (4) For SR networks, textures with regular stripes or grids are more likely to be noticed, while complex semantics are difficult to utilize. Our work opens new directions for designing SR networks and interpreting low-level vision deep models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
Gauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428
On Estimation of a Probability Density Function and Mode
Parzen, E. (1962) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177704472 · Cited by 10,585
Image Super-Resolution Using Deep Convolutional Networks
Dong, C., Loy, C. C., He, K., & Tang, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2439281 · Cited by 10,105
Self-citationAccurate Image Super-Resolution Using Very Deep Convolutional Networks
Kim, J., Lee, J. K., & Lee, K. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.182 · Cited by 7,870 · cited 1×
“[14] introduce the first SR network, plenty of deep learning based SR methods have been proposed, including deeper networks [15, 26, 44], residual architectures [30, 56], recurrent architectures [27, 52], and attention mechanism [59, 13].”
BackgroundEnhanced Deep Residual Networks for Single Image Super-Resolution
Lim, B., Son, S., Kim, H., Nah, S., & Lee, K. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.151 · Cited by 7,736
Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.207 · Cited by 7,570
Image Super-Resolution Using Very Deep Residual Channel Attention Networks
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_18 · Cited by 5,708
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370
ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Qiao, Y., & Loy, C. C. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-11021-5_5 · Cited by 4,731
Self-citationOn Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524
NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study
Agustsson, E., & Timofte, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.150 · Cited by 4,295
Residual Dense Network for Image Super-Resolution
Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00262 · Cited by 4,168
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087
Accelerating the Super-Resolution Convolutional Neural Network
Dong, C., Loy, C. C., & Tang, X. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_25 · Cited by 3,786
Self-citationSingle image super-resolution from transformed self-exemplars
Huang, J.-B., Singh, A., & Ahuja, N. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299156 · Cited by 3,472
Striving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447
Deeply-Recursive Convolutional Network for Image Super-Resolution
Kim, J., Lee, J. K., & Lee, K. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.181 · Cited by 3,025
Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution
Lai, W.-S., Huang, J.-B., Ahuja, N., & Yang, M.-H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.618 · Cited by 2,976
Learning Important Features Through Propagating Activation Differences
Shrikumar, A., Greenside, P., & Kundaje, A. (2017) · arXiv · doi:10.48550/arxiv.1704.02685 · Cited by 2,761
Image Super-Resolution via Deep Recursive Residual Network
Tai, Y., Yang, J., & Liu, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.298 · Cited by 2,415
Second-Order Attention Network for Single Image Super-Resolution
Dai, T., Cai, J., Zhang, Y., Xia, S.-T., & Zhang, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01132 · Cited by 2,010
Deep Back-Projection Networks for Super-Resolution
Haris, M., Shakhnarovich, G., & Ukita, N. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00179 · Cited by 1,599
SmoothGrad: removing noise by adding noise
Smilkov, D., Thorat, N., Kim, B., Viégas, F., & Wattenberg, M. (2017) · arXiv · doi:10.48550/arxiv.1706.03825 · Cited by 1,467 · cited 1×
“In addition to the above attribution methods, Guided Backpropagation [49] and Smooth Gradient [48] are also widely used in visualizing classification networks.”
MethodFast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network
Ahn, N., Kang, B., & Sohn, K.-A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_16 · Cited by 1,365
Sanity Checks for Saliency Maps
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294
Interpretable Explanations of Black Boxes by Meaningful Perturbation
Fong, R. C., & Vedaldi, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.371 · Cited by 1,097
Interpreting the Latent Space of GANs for Semantic Face Editing
Shen, Y., Gu, J., Tang, X., & Zhou, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00926 · Cited by 1,063
Self-citationLightweight Image Super-Resolution with Information Multi-distillation Network
Hui, Z., Gao, X., Yang, Y., & Wang, X. (2019) · Proceedings of the 27th ACM International Conference on Multimedia · doi:10.1145/3343031.3351084 · Cited by 1,062
Multi-scale Residual Network for Image Super-Resolution
Li, J., Fang, F., Mei, K., & Zhang, G. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_32 · Cited by 899
Relative Deprivation and the Gini Coefficient
Yitzhaki, S. (1979) · The Quarterly Journal of Economics · doi:10.2307/1883197 · Cited by 850
Residual Feature Distillation Network for Lightweight Image Super-Resolution
Liu, J., Tang, J., & Wu, G. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-67070-2_2 · Cited by 541
Visualizing Deep Convolutional Neural Networks Using Natural Pre-images
Mahendran, A., & Vedaldi, A. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-016-0911-8 · Cited by 496
Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
Shrikumar, A., Greenside, P., Shcherbina, A., & Kundaje, A. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.01713 · Cited by 496
Efficient Image Super-Resolution Using Pixel Attention
Zhao, H., Kong, X., He, J., Qiao, Y., & Dong, C. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-67070-2_3 · Cited by 430
Self-citationLayer-Wise Relevance Propagation for Neural Networks with Local Renormalization Layers
Binder, A., Montavon, G., Lapuschkin, S., Müller, K.-R., & Samek, W. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-44781-0_8 · Cited by 418
Image Super-Resolution With Cross-Scale Non-Local Attention and Exhaustive Self-Exemplars Mining
Mei, Y., Fan, Y., Zhou, Y., Huang, L., Huang, T. S., & Shi, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00573 · Cited by 413
Interpreting Deep Visual Representations via Network Dissection
Zhou, B., Bau, D., Oliva, A., & Torralba, A. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858759 · Cited by 330
Computing receptive fields of convolutional neural networks
Araujo, A., Norris, W., & Sim, J. (2019) · Distill · doi:10.23915/distill.00021 · Cited by 312
PIPAL: A Large-Scale Image Quality Assessment Dataset for Perceptual Image Restoration
Gu, J., Cai, H., Chen, H., Ye, X., Ren, J., & Dong, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58621-8_37 · Cited by 296
Self-citationDensely Residual Laplacian Super-Resolution
Anwar, S., & Barnes, N. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3021088 · Cited by 281
Non-Local Recurrent Network for Image Restoration
Liu, D., Wen, B., Fan, Y., Loy, C. C., & Huang, T. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.02919 · Cited by 252
Visualizing the Impact of Feature Attribution Baselines
Sturmfels, P., Lundberg, S., & Lee, S.-I. (2020) · Distill · doi:10.23915/distill.00022 · Cited by 200
Residual Non-local Attention Networks for Image Restoration
Zhang, Y., Li, K., Li, K., Zhong, B., & Fu, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.10082 · Cited by 172
Image Quality Assessment for Perceptual Image Restoration: A New Dataset, Benchmark and MetricPreprint
Gu, J., Cai, H., Chen, H., Ye, X., Ren, J., & Dong, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.15002 · Cited by 59
Self-citationPaths and consistency in additive cost sharing
Friedman, E. J. (2004) · International Journal of Game Theory · doi:10.1007/s001820400173 · Cited by 57
How to Explain Individual Classification Decisions
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1859912 · Cited by 54
Cited by
Activating More Pixels in Image Super-Resolution Transformer
Chen, X., Wang, X., Zhou, J., Qiao, Y., & Dong, C. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.02142 · Cited by 1,152
Self-citationTTST: A Top-k Token Selective Transformer for Remote Sensing Image Super-Resolution
Xiao, Y., Yuan, Q., Jiang, K., He, J., Lin, C.-W., & Zhang, L. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3349004 · Cited by 413
SRFormer: Permuted Self-Attention for Single Image Super-Resolution
Zhou, Y., Li, Z., Guo, C.-L., Bai, S., Cheng, M.-M., & Hou, Q. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01174 · Cited by 374
Efficient and Explicit Modelling of Image Hierarchies for Image Restoration
Li, Y., Fan, Y., Xiang, X., Demandolx, D., Ranjan, R., Timofte, R., & Van Gool, L. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01753 · Cited by 302
Spatially-Adaptive Feature Modulation for Efficient Image Super-Resolution
Sun, L., Dong, J., Tang, J., & Pan, J. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01213 · Cited by 289
Metadata
- Authors
- Gu, J., & Dong, C.
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 9195-9204
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3178925107
- Cited by
- 361
- References
- 55
- FWCI
- 21.22
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.