The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Learning A Sparse Transformer Network for Effective Image Deraining
Open accessAbstract
Transformers-based methods have achieved significant performance in image deraining as they can model the non-local information which is vital for high-quality image reconstruction. In this paper, we find that most existing Transformers usually use all similarities of the tokens from the query-key pairs for the feature aggregation. However, if the tokens from the query are different from those of the key, the self-attention values estimated from these tokens also involve in feature aggregation, which accordingly interferes with the clear image restoration. To overcome this problem, we propose an effective DeRaining network, Sparse Transformer (DRSformer) that can adaptively keep the most useful self-attention values for feature aggregation so that the aggregated features better facilitate high-quality image reconstruction. Specifically, we develop a learnable top-k selection operator to adaptively retain the most crucial attention scores from the keys for each query for better feature aggregation. Simultaneously, as the naive feed-forward network in Transformers does not model the multi-scale information that is important for latent clear image restoration, we develop an effective mixed-scale feed-forward network to generate better features for image deraining. To learn an enriched set of hybrid features, which combines local context from CNN operators, we equip our model with mixture of experts feature compensator to present a cooperation refinement deraining scheme. Extensive experimental results on the commonly used benchmarks demonstrate that the proposed method achieves favorable performance against state-of-the-art approaches. The source code and trained models are available at https://github.com/cschenxiang/DRSformer.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
Making a “Completely Blind” Image Quality Analyzer
Mittal, A., Soundararajan, R., & Bovik, A. C. (2013) · IEEE Signal Processing Letters · doi:10.1109/lsp.2012.2227726 · Cited by 6,766
No-Reference Image Quality Assessment in the Spatial Domain
Mittal, A., Moorthy, A. K., & Bovik, A. C. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2214050 · Cited by 6,064 · cited 1×
“For the rainy images without ground truth images, we use the non-reference metrics including NIQE [33] and BRISQUE [32].”
MethodAdaptive Mixtures of Local Experts
Jacobs, R. A., Jordan, M. I., Nowlan, S. J., & Hinton, G. E. (1991) · Neural Computation · doi:10.1162/neco.1991.3.1.79 · Cited by 5,384
SGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309
Restormer: Efficient Transformer for High-Resolution Image Restoration
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., & Yang, M.-H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00564 · Cited by 4,143
Scope of validity of PSNR in image/video quality assessment
Huynh-Thu, Q., & Ghanbari, M. (2008) · Electronics Letters · doi:10.1049/el:20080522 · Cited by 2,959 · cited 6×
“By adding local feature extraction and fusion at different scales, the MSFN can indeed better boost the performance, and achieve PSNR gain of 0.21 dB over GDFN. Effectiveness of MEFC .”
MethodInfluentialPre-Trained Image Processing Transformer
Chen, H., Wang, Y., Guo, T., Xu, C., Deng, Y., Liu, Z., Ma, S., Xu, C., Xu, C., & Gao, W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01212 · Cited by 2,308 · cited 12×
“Motivated by the great success of the Transformers [7] in natural language processing (NLP) [40] and high-level vision tasks [1, 28] [44, 64].”
BackgroundMethodInfluentialUformer: A General U-Shaped Transformer for Image Restoration
Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., & Li, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01716 · Cited by 2,274
Self-citationMulti-Stage Progressive Image Restoration
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01458 · Cited by 2,095
Removing Rain from Single Images via a Deep Detail Network
Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., & Paisley, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.186 · Cited by 1,235
Deep Joint Rain Detection and Removal from a Single Image
Yang, W., Tan, R. T., Feng, J., Liu, J., Guo, Z., & Yan, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.183 · Cited by 1,231
Image De-Raining Using a Conditional Generative Adversarial Network
Zhang, H., Sindagi, V., & Patel, V. M. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2920407 · Cited by 1,214
Density-Aware Single Image De-raining Using a Multi-stream Dense Network
Zhang, H., & Patel, V. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00079 · Cited by 1,066
Progressive Image Deraining Networks: A Better and Simpler Baseline
Ren, D., Zuo, W., Hu, Q., Zhu, P., & Meng, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00406 · Cited by 1,000
Gated Fusion Network for Single Image Dehazing
Ren, W., Ma, L., Zhang, J., Pan, J., Cao, X., Liu, W., & Yang, M.-H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00343 · Cited by 973
Self-citationLearning Texture Transformer Network for Image Super-Resolution
Yang, F., Yang, H., Fu, J., Lu, H., & Guo, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00583 · Cited by 966
Rain Streak Removal Using Layer Priors
Li, Y., Tan, R. T., Guo, X., Lu, J., & Brown, M. S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.299 · Cited by 952
Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining
Li, X., Wu, J., Lin, Z., Liu, H., & Zha, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_16 · Cited by 927
Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition
Kang, L.-W., Lin, C.-W., & Fu, Y.-H. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2179057 · Cited by 905
Removing Rain from a Single Image via Discriminative Sparse Coding
Luo, Y., Xu, Y., & Ji, H. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.388 · Cited by 837
Multi-Scale Progressive Fusion Network for Single Image Deraining
Jiang, K., Wang, Z., Yi, P., Chen, C., Huang, B., Luo, Y., Ma, J., & Jiang, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00837 · Cited by 795
Spatial Attentive Single-Image Deraining With a High Quality Real Rain Dataset
Wang, T., Yang, X., Xu, K., Chen, S., Zhang, Q., & Lau, R. W. H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01255 · Cited by 681
Incorporating Convolution Designs into Visual Transformers
Yuan, K., Guo, S., Liu, Z., Zhou, A., Yu, F., & Wu, W. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00062 · Cited by 611
Image Super-Resolution with Non-Local Sparse Attention
Mei, Y., Fan, Y., & Zhou, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00352 · Cited by 607
Image Dehazing Transformer with Transmission-Aware 3D Position Embedding
Guo, C., Yan, Q., Anwar, S., Cong, R., Ren, W., & Li, C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00572 · Cited by 552
Efficient Content-Based Sparse Attention with Routing Transformers
Roy, A., Saffar, M., Vaswani, A., & Grangier, D. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00353 · Cited by 509
Semi-Supervised Transfer Learning for Image Rain Removal
Wei, W., Meng, D., Zhao, Q., Xu, Z., & Wu, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00400 · Cited by 458
A Model-Driven Deep Neural Network for Single Image Rain Removal
Wang, H., Xie, Q., Zhao, Q., & Meng, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00317 · Cited by 449
Depth-Attentional Features for Single-Image Rain Removal
Hu, X., Fu, C.-W., Zhu, L., & Heng, P.-A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00821 · Cited by 390
Xiao, J., Fu, X., Liu, A., Wu, F., & Zha, Z.-J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3183612 · Cited by 389
LocalViT: Analyzing Locality in Vision Transformers
Li, Y., Zhang, K., Cao, J., Timofte, R., Magno, M., Benini, L., & Van Goo, L. (2023) · 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10342025 · Cited by 372
Single Image Deraining: From Model-Based to Data-Driven and Beyond
Yang, W., Tan, R. T., Wang, S., Fang, Y., & Liu, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2995190 · Cited by 284
Joint Convolutional Analysis and Synthesis Sparse Representation for Single Image Layer Separation
Gu, S., Meng, D., Zuo, W., & Zhang, L. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.189 · Cited by 233
Unpaired Deep Image Deraining Using Dual Contrastive Learning
Chen, X., Pan, J., Jiang, K., Li, Y., Huang, Y., Kong, C., Dai, L., & Fan, Z. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00206 · Cited by 203
Self-citationSyn2Real Transfer Learning for Image Deraining Using Gaussian Processes
Yasarla, R., Sindagi, V. A., & Patel, V. M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00280 · Cited by 200
Adaptively Sparse Transformers
Correia, G. M., Niculae, V., & Martins, A. F. T. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1223 · Cited by 198
KVT: k-NN Attention for Boosting Vision Transformers
Wang, P., Wang, X., Wang, F., Lin, M., Chang, S., Li, H., & Jin, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20053-3_17 · Cited by 198
Self-citationNFormer: Robust Person Re-identification with Neighbor Transformer
Wang, H., Shen, J., Liu, Y., Gao, Y., & Gavves, E. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00715 · Cited by 194
Explicit Sparse Transformer: Concentrated Attention Through Explicit SelectionPreprint
Zhao, G., Lin, J., Zhang, Z., Ren, X., Su, Q., & Sun, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.11637 · Cited by 191
SparseTT: Visual Tracking with Sparse Transformers
Fu, Z., Fu, Z., Liu, Q., Cai, W., & Wang, Y. (2022) · Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/127 · Cited by 176
Convolutional Sparse and Low-Rank Coding-Based Rain Streak Removal
Zhang, H., & Patel, V. M. (2017) · 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2017.145 · Cited by 165
Structure-Preserving Deraining with Residue Channel Prior Guidance
Yi, Q., Li, J., Dai, Q., Fang, F., Zhang, G., & Zeng, T. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00420 · Cited by 164
Rain Streak Removal via Dual Graph Convolutional Network
Fu, X., Qi, Q., Zha, Z.-J., Zhu, Y., & Ding, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16224 · Cited by 159
DCSFN: Deep Cross-scale Fusion Network for Single Image Rain Removal
Wang, C., Xing, X., Wu, Y., Su, Z., & Chen, J. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413820 · Cited by 143
Magic ELF: Image Deraining Meets Association Learning and Transformer
Jiang, K., Wang, Z., Chen, C., Wang, Z., Cui, L., & Lin, C. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547760 · Cited by 133
Hybrid CNN-Transformer Feature Fusion for Single Image Deraining
Chen, X., Pan, J., Lu, J., Fan, Z., & Li, H. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25111 · Cited by 123
Self-citationSuganuma, M., Liu, X., & Okatani, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00925 · Cited by 121
Memory Oriented Transfer Learning for Semi-Supervised Image Deraining
Huang, H., Yu, A., & He, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00764 · Cited by 121
Chasing Sparsity in Vision Transformers: An End-to-End Exploration
Chen, T., Cheng, Y., Gan, Z., Yuan, L., Zhang, L., & Wang, Z. (2021) · arXiv · doi:10.48550/arxiv.2106.04533 · Cited by 111 · cited 2×
“Unlike the dropout strategy of randomly abandoning the scores, an adaptive selection of the top-k contributive scores is implemented upon M, aiming to preserve the most significant components and remove the useless ones [3].”
MethodClosing the Loop: Joint Rain Generation and Removal via Disentangled Image Translation
Ye, Y., Chang, Y., Zhou, H., & Yan, L. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00209 · Cited by 103
DRT: A Lightweight Single Image Deraining Recursive Transformer
Liang, Y., Anwar, S., & Liu, Y. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00074 · Cited by 103
KNN Local Attention for Image Restoration
Lee, H., Choi, H., Sohn, K., & Min, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00218 · Cited by 91
Continual Image Deraining With Hypergraph Convolutional Networks
Fu, X., Xiao, J., Zhu, Y., Liu, A., Wu, F. H., & Zha, Z.-J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3241756 · Cited by 84
Unpaired Learning for Deep Image Deraining with Rain Direction Regularizer
Liu, Y., Yue, Z., Pan, J., & Su, Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00471 · Cited by 73
Self-citationMulti-Decoding Deraining Network and Quasi-Sparsity Based Training
Wang, Y., Ma, C., & Zeng, B. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01317 · Cited by 43
ETDNet: An Efficient Transformer Deraining Model
Qin, Q., Yan, J., Wang, Q., Wang, X., Li, M., & Wang, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3108516 · Cited by 39
Self-citationRestoring Spatially-Heterogeneous Distortions Using Mixture of Experts Network
Kim, S., Ahn, N., & Sohn, K.-A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69532-3_12 · Cited by 12
Cited by
Std-net: spatio-temporal decomposition network for video demoiréing with sparse transformers
Niu, Y., Xu, R., Lin, Z., & Liu, W. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3386604 · Cited by 12
Metadata
- Authors
- Chen, X., Li, H., Li, M., & Pan, J.
- Venue
- 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2023-06
- Pages
- 5896-5905
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4386075800
- Cited by
- 538
- References
- 63
- FWCI
- 61.06
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.