Density-Aware Single Image De-raining Using a Multi-stream Dense Network
Open accessAbstract
Single image rain streak removal is an extremely challenging problem due to the presence of non-uniform rain densities in images. We present a novel density-aware multi-stream densely connected convolutional neural network-based algorithm, called DID-MDN, for joint rain density estimation and de-raining. The proposed method enables the network itself to automatically determine the rain-density information and then efficiently remove the corresponding rain-streaks guided by the estimated rain-density label. To better characterize rain-streaks with different scales and shapes, a multi-stream densely connected de-raining network is proposed which efficiently leverages features from different scales. Furthermore, a new dataset containing images with rain-density labels is created and used to train the proposed density-aware network. Extensive experiments on synthetic and real datasets demonstrate that the proposed method achieves significant improvements over the recent state-of-the-art methods. In addition, an ablation study is performed to demonstrate the improvements obtained by different modules in the proposed method. The code can be downloaded at https://github.com/hezhangsprinter/DID-MDN.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Qualitatively, Hoa et al.'s proposed algorithm, as well as Qian's and DID-MDN, removed most raindrops successfully.
“Qualitatively, the proposed algorithm, as well as Qian’s and the DIID-MDN removed most raindrops successfully.”
From A Review of Detection and Removal of Raindrops in Automotive Vision Systems (2021) · cites this paper as [60]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · see citing works
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438 · see citing works
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696 · see citing works
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971 · see citing works
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428 · see citing works
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283 · see citing works
Single image haze removal using dark channel prior
He, K., Sun, J., & Tang, X. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.168 · Cited by 7,601 · see citing works
Residual Dense Network for Image Super-Resolution
Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00262 · Cited by 4,371 · see citing works
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Eigen, D., Puhrsch, C., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1406.2283 · Cited by 3,442 · see citing works
Holistically-Nested Edge Detection
Xie, S., & Tu, Z. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.164 · Cited by 3,401 · see citing works
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_23 · Cited by 3,034 · see citing works
AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks
Xu, T., Zhang, P., Huang, Q., Zhang, H., Gan, Z., Huang, X., & He, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00143 · Cited by 1,923 · see citing works
Self-citationSingle Image Dehazing via Multi-scale Convolutional Neural Networks
Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., & Yang, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46475-6_10 · Cited by 1,878 · see citing works
Self-citationThe One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation
Jégou, S., Drozdzal, M., Vázquez, D., Romero, A., & Bengio, Y. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.156 · Cited by 1,762 · see citing works
Context Encoding for Semantic Segmentation
Zhang, H., Dana, K., Shi, J., Zhang, Z., Wang, X., Tyagi, A., & Agrawal, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00747 · Cited by 1,510 · see citing works
Self-citationDensely Connected Pyramid Dehazing Network
Zhang, H., & Patel, V. M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00337 · Cited by 1,283 · see citing works
Self-citationDeep Joint Rain Detection and Removal from a Single Image
Yang, W., Tan, R. T., Feng, J., Liu, J., Guo, Z., & Yan, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.183 · Cited by 1,280 · see citing works
Removing Rain from Single Images via a Deep Detail Network
Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., & Paisley, J. W. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.186 · Cited by 1,273 · see citing works
Image De-Raining Using a Conditional Generative Adversarial Network
Zhang, H., Sindagi, V. A., & Patel, V. M. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2920407 · Cited by 1,232 · see citing works
Self-citationRain Streak Removal Using Layer Priors
Li, Y., Tan, R. T., Guo, X., Lu, J., & Brown, M. S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.299 · Cited by 980 · see citing works
Clearing the Skies: A Deep Network Architecture for Single-Image Rain Removal
Fu, X., Huang, J., Ding, X., Liao, Y., & Paisley, J. W. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2691802 · Cited by 939 · see citing works
Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition
Kang, L.-W., Lin, C., & Fu, Y.-H. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2179057 · Cited by 913 · see citing works
Removing Rain from a Single Image via Discriminative Sparse Coding
Luo, Y., Xu, Y., & Ji, H. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.388 · Cited by 863 · see citing works
Generating High-Quality Crowd Density Maps Using Contextual Pyramid CNNs
Sindagi, V. A., & Patel, V. M. (2017) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.206 · Cited by 734 · see citing works
Self-citationA Generalized Low-Rank Appearance Model for Spatio-temporally Correlated Rain Streaks
Chen, Y., & Hsu, C.-T. (2013) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.247 · Cited by 490 · see citing works
Face anti-spoofing using patch and depth-based CNNs
Atoum, Y., Liu, Y., Jourabloo, A., & Liu, X. (2017) · IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/btas.2017.8272713 · Cited by 456 · see citing works
Restoring an Image Taken through a Window Covered with Dirt or Rain
Eigen, D., Krishnan, D., & Fergus, R. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.84 · Cited by 454 · see citing works
MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network
Zhang, Z., Xie, Y., Xing, F., McGough, M., & Yang, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.378 · Cited by 354 · see citing works
Joint Bi-layer Optimization for Single-Image Rain Streak Removal
Zhu, L., Fu, C., Lischinski, D., & Heng, P. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.276 · Cited by 345 · see citing works
Hidden Two-Stream Convolutional Networks for Action Recognition
Zhu, Y., Lan, Z., Newsam, S. D., & Hauptmann, A. G. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20893-6_23 · Cited by 300 · see citing works
Image De-raining Using a Conditional Generative Adversarial NetworkPreprint
Zhang, H., Sindagi, V. A., & Patel, V. M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.05957 · Cited by 245 · see citing works
Self-citationSelf-Learning Based Image Decomposition With Applications to Single Image Denoising
Huang, D.-A., Kang, L., Wang, Y.-C. F., & Lin, C. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2284759 · Cited by 214 · see citing works
Utilizing Local Phase Information to Remove Rain from Video
Santhaseelan, V., & Asari, V. K. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-014-0759-8 · Cited by 182 · see citing works
Visual Depth Guided Color Image Rain Streaks Removal Using Sparse Coding
Chen, D.-Y., Chen, C.-C., & Kang, L.-W. (2014) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2014.2308627 · Cited by 180 · see citing works
Video Desnowing and Deraining Based on Matrix Decomposition
Ren, W., Tian, J., Han, Z. J., Chan, A. B., & Tang, Y. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.303 · Cited by 180 · see citing works
Convolutional Sparse and Low-Rank Coding-Based Rain Streak Removal
Zhang, H., & Patel, V. M. (2017) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2017.145 · Cited by 171 · see citing works
Self-citationA Recurrent Encoder-Decoder Network for Sequential Face Alignment
Peng, X., Feris, R., Wang, X., & Metaxas, D. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_3 · Cited by 148 · see citing works
Should We Encode Rain Streaks in Video as Deterministic or Stochastic?
Wei, W., Yi, L., Xie, Q., Zhao, Q., Meng, D., & Xu, Z. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.275 · Cited by 142 · see citing works
Joint Transmission Map Estimation and Dehazing using Deep Networks
Zhang, H., Sindagi, V. A., & Patel, V. M. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2912145 · Cited by 121 · see citing works
Self-citationContext-Aware Single Image Rain Removal
Huang, D.-A., Kang, L., Yang, M.-C., et al. (2012) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2012.92 · Cited by 115 · see citing works
Multi-style Generative Network for Real-time TransferPreprint
Zhang, H., & Dana, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.06953 · Cited by 113 · see citing works
Self-citationVideo Deblurring via Semantic Segmentation and Pixel-Wise Non-linear Kernel
Ren, W., Pan, J., Cao, X., & Yang, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.123 · Cited by 99 · see citing works
Hidden Two-Stream Convolutional Networks for Action RecognitionPreprint
Zhu, Y., Lan, Z., Newsam, S. D., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.00389 · Cited by 98 · see citing works
Differential Angular Imaging for Material Recognition
Xue, J., Zhang, H., Dana, K., & Nishino, K. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.734 · Cited by 96 · see citing works
Self-citationJoint Transmission Map Estimation and Dehazing using Deep NetworksPreprint
Zhang, H., Sindagi, V. A., & Patel, V. M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.00581 · Cited by 38 · see citing works
Self-citationMulti-Stream Deep Similarity Learning Networks for Visual Tracking
Li, K., Kong, Y., & Fu, Y. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/301 · Cited by 20 · see citing works
Reconstruction for Feature Disentanglement in Pose-invariant Face RecognitionPreprint
Peng, X., Yu, X., Sohn, K., et al. (2017) · arXiv (Cornell University) · Cited by 18 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
All in One Bad Weather Removal Using Architectural Search
Li, R., Tan, R. T., & Cheong, L.-F. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00324 · Cited by 377 · see citing works
DeepTAM: Deep Tracking and Mapping
Zhou, H., Ummenhofer, B., & Brox, T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_50 · Cited by 226 · see citing works
Physics-Based Rendering for Improving Robustness to Rain
Halder, S., Lalonde, J.-F., & Charette, R. D. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01030 · Cited by 130 · see citing works
Learning From Synthetic Photorealistic Raindrop for Single Image Raindrop Removal
Hao, Z., You, S., Li, Y., Li, K., & Lu, F. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00534 · Cited by 54 · see citing works
A Review of Detection and Removal of Raindrops in Automotive Vision Systems
Hamzeh, Y., & Rawashdeh, S. A. (2021) · Journal of Imaging · doi:10.3390/jimaging7030052 · Cited by 21 · see citing works
“benchmarked their proposed algorithm against the ones proposed by Eigen [46], Qian [49], as well as Pix2Pix [52], and DIDMDN [60] algorithms that were not designed originally for adherent raindrop removal.”
Method
- Authors
- Zhang, H., & Patel, V. M.
- Affiliations
- Rutgers, The State University of New Jersey (US)
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 695-704
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2964212750
- Cited by
- 1,066
- References
- 52
- Citations by year
- 2026: 66 · 2025: 140 · 2024: 156 · 2023: 167 · 2022: 148 · 2021: 155 · 2020: 131 · 2019: 84 · 2018: 19
- FWCI
- 29.31
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Enhancement Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Signal Denoising Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image Processing Techniques
- Keywords
- single image rain removal, rain streak removal, multi-stream network, DenseNet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
