The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring
Open accessAbstract
We present an effective and efficient method that explores the properties of Transformers in the frequency domain for high-quality image deblurring. Our method is motivated by the convolution theorem that the correlation or convolution of two signals in the spatial domain is equivalent to an element-wise product of them in the frequency domain. This inspires us to develop an efficient frequency domain-based self-attention solver (FSAS) to estimate the scaled dot-product attention by an element-wise product operation instead of the matrix multiplication in the spatial domain. In addition, we note that simply using the naive feed-forward network (FFN) in Transformers does not generate good deblurred results. To overcome this problem, we propose a simple yet effective discriminative frequency domain-based FFN (DFFN), where we introduce a gated mechanism in the FFN based on the Joint Photographic Experts Group (JPEG) compression algorithm to discriminatively determine which low- and high-frequency information of the features should be preserved for latent clear image restoration. We formulate the proposed FSAS and DFFN into an asymmetrical network based on an encoder and decoder architecture, where the FSAS is only used in the decoder module for better image deblurring. Experimental results show that the proposed method performs favorably against the state-of-the-art approaches.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00061 · Cited by 5,259 · cited 1×
“To alleviate this problem, most approaches use the down-sampling strategy to reduce the spatial resolution of features [23].”
MethodSwinIR: Image Restoration Using Swin Transformer
Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., & Timofte, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw54120.2021.00210 · Cited by 4,734
Restormer: Efficient Transformer for High-Resolution Image Restoration
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., & Yang, M.-H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00564 · Cited by 4,143
Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., Torr, P. H. S., et al. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00681 · Cited by 4,008 · cited 1×
“…model the global contexts and achieves significant progress in lots of high-level vision tasks (e.g., image classification [11], object detection [1,31] and semantic segmentation [25,32]), it has been developed to solve image super-resolution [10], image deblurring [21,27] and image denoise [2,24].”
BackgroundPre-Trained Image Processing Transformer
Chen, H., Wang, Y., Guo, T., Xu, C., Deng, Y., Liu, Z., Ma, S., Xu, C., Xu, C., & Gao, W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01212 · Cited by 2,308 · cited 2×
“Moreover, our method is at least 4× faster than IPT [2] that is based on the original self-attentions.”
MethodDeep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring
Nah, S., Kim, T. H., & Lee, K. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.35 · Cited by 2,275
Uformer: A General U-Shaped Transformer for Image Restoration
Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., & Li, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01716 · Cited by 2,274
Multi-Stage Progressive Image Restoration
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01458 · Cited by 2,095
DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks
Kupyn, O., Budzan, V., Mykhailych, M., Mishkin, D., & Matas, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00854 · Cited by 1,706
Simple Baselines for Image Restoration
Chen, L., Chu, X., Zhang, X., & Sun, J. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20071-7_2 · Cited by 1,365
Scale-Recurrent Network for Deep Image Deblurring
Tao, X., Gao, H., Shen, X., Wang, J., & Jia, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00853 · Cited by 1,316
DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better
Kupyn, O., Martyniuk, T., Wu, J., & Wang, Z. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00897 · Cited by 1,148
Rethinking Coarse-to-Fine Approach in Single Image Deblurring
Cho, S.-J., Ji, S.-W., Hong, J.-P., Jung, S.-W., & Ko, S.-J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00460 · Cited by 798
Deep Stacked Hierarchical Multi-Patch Network for Image Deblurring
Zhang, H., Dai, Y., Li, H., & Koniusz, P. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00613 · Cited by 652
Dynamic Scene Deblurring Using Spatially Variant Recurrent Neural Networks
Zhang, J., Pan, J., Ren, J., Song, Y., Bao, L., Lau, R. W. H., & Yang, M.-H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00267 · Cited by 384
Self-citationCo-Scale Conv-Attentional Image Transformers
Xu, W., Xu, Y., Chang, T., & Tu, Z. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00983 · Cited by 375
Real-World Blur Dataset for Learning and Benchmarking Deblurring Algorithms
Rim, J., Lee, H., Won, J., & Cho, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58595-2_12 · Cited by 373
Dynamic Scene Deblurring With Parameter Selective Sharing and Nested Skip Connections
Gao, H., Tao, X., Shen, X., & Jia, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00397 · Cited by 372
Shen, Z., Wang, W., Lu, X., Shen, J., Ling, H., Xu, T., & Shao, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00567 · Cited by 359
Multi-class Token Transformer for Weakly Supervised Semantic Segmentation
Xu, L., Ouyang, W., Bennamoun, M., Boussaid, F., & Xu, D. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00427 · Cited by 305
Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring
Suin, M., Purohit, K., & Rajagopalan, A. N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00366 · Cited by 304
Stripformer: Strip Transformer for Fast Image Deblurring
Tsai, F.-J., Peng, Y.-T., Lin, Y.-Y., Tsai, C.-C., & Lin, C.-W. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19800-7_9 · Cited by 262
Physics-Based Generative Adversarial Models for Image Restoration and Beyond
Pan, J., Dong, J., Liu, Y., Zhang, J., Ren, J., Tang, J., Tai, Y.-W., & Yang, M.-H. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2969348 · Cited by 191
Self-citationImproving Image Restoration by Revisiting Global Information Aggregation
Chu, X., Chen, L., Chen, C., & Lu, X. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20071-7_4 · Cited by 166
BANet: A Blur-Aware Attention Network for Dynamic Scene Deblurring
Tsai, F.-J., Peng, Y., Tsai, C.-C., Lin, Y., & Lin, C. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3216216 · Cited by 161
DWDN: Deep Wiener Deconvolution Network for Non-Blind Image Deblurring
Dong, J., Roth, S., & Schiele, B. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3138787 · Cited by 115
Self-citationLearning Spatially-Variant MAP Models for Non-blind Image Deblurring
Dong, J., Roth, S., & Schiele, B. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00485 · Cited by 40
Self-citationLearning Spatially Variant Linear Representation Models for Joint Filtering
Dong, J., Pan, J., Ren, J., Lin, L., Tang, J., & Yang, M. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3102575 · Cited by 38
Self-citation
Metadata
- Authors
- Kong, L., Dong, J., Ge, J., Li, M., & Pan, J.
- Venue
- 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2023-06
- Pages
- 5886-5895
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4386075642
- Cited by
- 398
- References
- 33
- FWCI
- 45.14
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.