The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Joint Rain Detection and Removal from a Single Image with Contextualized Deep Networks
Abstract
Rain streaks, particularly in heavy rain, not only degrade visibility but also make many computer vision algorithms fail to function properly. In this paper, we address this visibility problem by focusing on single-image rain removal, even in the presence of dense rain streaks and rain-streak accumulation, which is visually similar to mist or fog. To achieve this, we introduce a new rain model and a deep learning architecture. Our rain model incorporates a binary rain map indicating rain-streak regions, and accommodates various shapes, directions, and sizes of overlapping rain streaks, as well as rain accumulation, to model heavy rain. Based on this model, we construct a multi-task deep network, which jointly learns three targets: the binary rain-streak map, rain streak layers, and clean background, which is our ultimate output. To generate features that can be invariant to rain steaks, we introduce a contextual dilated network, which is able to exploit regional contextual information. To handle various shapes and directions of overlapping rain streaks, our strategy is to utilize a recurrent process that progressively removes rain streaks. Our binary map provides a constraint and thus additional information to train our network. Extensive evaluation on real images, particularly in heavy rain, shows the effectiveness of our model and architecture.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Image Super-Resolution Using Deep Convolutional Networks
Dong, C., Loy, C. C., He, K., & Tang, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2439281 · Cited by 10,105
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662206 · Cited by 8,983
Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937655 · Cited by 8,216
Multi-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738
The Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139
Image Style Transfer Using Convolutional Neural Networks
Gatys, L. A., Ecker, A. S., & Bethge, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.265 · Cited by 6,025
DehazeNet: An End-to-End System for Single Image Haze Removal
Cai, B., Xu, X., Jia, K., Qing, C., & Tao, D. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2598681 · Cited by 3,367
Scene Parsing through ADE20K Dataset
Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.544 · Cited by 3,326
Scope of validity of PSNR in image/video quality assessment
Huynh-Thu, Q., & Ghanbari, M. (2008) · Electronics Letters · doi:10.1049/el:20080522 · Cited by 2,959
Globally and locally consistent image completion
Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073659 · Cited by 2,203
LLNet: A deep autoencoder approach to natural low-light image enhancement
Lore, K. G., Akintayo, A., & Sarkar, S. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.06.008 · Cited by 1,951
Single Image Dehazing via Multi-scale Convolutional Neural Networks
Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., & Yang, M.-H. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_10 · Cited by 1,759
Make3D: Learning 3D Scene Structure from a Single Still Image
Saxena, A., Sun, M., & Ng, A. Y. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.132 · Cited by 1,736
Narasimhan, S. G., & Nayar, S. K. (2002) · International Journal of Computer Vision · doi:10.1023/a:1016328200723 · Cited by 1,720
Berman, D., Treibitz, T., & Avidan, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.185 · Cited by 1,642
Removing Rain from Single Images via a Deep Detail Network
Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., & Paisley, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.186 · Cited by 1,235
Deep Joint Rain Detection and Removal from a Single Image
Yang, W., Tan, R. T., Feng, J., Liu, J., Guo, Z., & Yan, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.183 · Cited by 1,231
Self-citationImage De-Raining Using a Conditional Generative Adversarial Network
Zhang, H., Sindagi, V., & Patel, V. M. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2920407 · Cited by 1,214
Enhancing underwater images and videos by fusion
Ancuti, C., Ancuti, C. O., Haber, T., & Bekaert, P. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247661 · Cited by 1,086
Density-Aware Single Image De-raining Using a Multi-stream Dense Network
Zhang, H., & Patel, V. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00079 · Cited by 1,066
Gated Fusion Network for Single Image Dehazing
Ren, W., Ma, L., Zhang, J., Pan, J., Cao, X., Liu, W., & Yang, M.-H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00343 · Cited by 973
Rain Streak Removal Using Layer Priors
Li, Y., Tan, R. T., Guo, X., Lu, J., & Brown, M. S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.299 · Cited by 952
Self-citationClearing the Skies: A Deep Network Architecture for Single-Image Rain Removal
Fu, X., Huang, J., Ding, X., Liao, Y., & Paisley, J. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2691802 · Cited by 940
Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition
Kang, L.-W., Lin, C.-W., & Fu, Y.-H. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2179057 · Cited by 905
Compression Artifacts Reduction by a Deep Convolutional Network
Dong, C., Deng, Y., Loy, C. C., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.73 · Cited by 883
Removing Rain from a Single Image via Discriminative Sparse Coding
Luo, Y., Xu, Y., & Ji, H. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.388 · Cited by 837
Deep Convolutional Neural Network for Image Deconvolution
Li, X., Ren, J., Liu, C., & Jia, J. (2014) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 836
Attentive Generative Adversarial Network for Raindrop Removal from A Single Image
Qian, R., Tan, R. T., Yang, W., Su, J., & Liu, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00263 · Cited by 821
Self-citationComputational Color Constancy: Survey and Experiments
Gijsenij, A., Gevers, T., & van de Weijer, J. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2118224 · Cited by 562
Schuler, C. J., Hirsch, M., Harmeling, S., & Schölkopf, B. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2481418 · Cited by 560
Funt, B., & Finlayson, G. D. (1995) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.391390 · Cited by 560
Detection and removal of rain from videos
Garg, K., & Nayar, S. K. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315077 · Cited by 515
Garg, K., & Nayar, S. K. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-006-0028-6 · Cited by 477
A Generalized Low-Rank Appearance Model for Spatio-temporally Correlated Rain Streaks
Chen, Y.-L., & Hsu, C.-T. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.247 · Cited by 458
Restoring an Image Taken through a Window Covered with Dirt or Rain
Eigen, D., Krishnan, D., & Fergus, R. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.84 · Cited by 454
Analysis of Rain and Snow in Frequency Space
Barnum, P. C., Narasimhan, S., & Kanade, T. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-008-0200-2 · Cited by 356
Rain or Snow Detection in Image Sequences Through Use of a Histogram of Orientation of Streaks
Bossu, J., Hautière, N., & Tarel, J.-P. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-011-0421-7 · Cited by 349
Deep Multi-patch Aggregation Network for Image Style, Aesthetics, and Quality Estimation
Lu, X., Lin, Z., Shen, X., Mech, R., & Wang, J. Z. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.119 · Cited by 321
Rain Removal in Video by Combining Temporal and Chromatic Properties
Zhang, X., Li, H., Qi, Y., Leow, W., & Ng, T. (2006) · 2006 IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2006.262572 · Cited by 293
Single-image deraining using an adaptive nonlocal means filter
Kim, J.-H., Lee, C., Sim, J.-Y., & Kim, C.-S. (2013) · 2013 IEEE International Conference on Image Processing · doi:10.1109/icip.2013.6738189 · Cited by 273
A comparative study of image retargeting
Rubinstein, M. H., Gutiérrez, D., Sorkine, O., et al. (2010) · ACM Transactions on Graphics · doi:10.1145/1882261.1866186 · Cited by 269
Joint Convolutional Analysis and Synthesis Sparse Representation for Single Image Layer Separation
Gu, S., Meng, D., Zuo, W., & Zhang, L. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.189 · Cited by 233
An End-to-End Compression Framework Based on Convolutional Neural Networks
Jiang, F., Tao, W., Liu, S., Ren, J., Guo, X., & Zhao, D. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2734838 · Cited by 232
Photorealistic rendering of rain streaks
Garg, K., & Nayar, S. K. (2006) · ACM Transactions on Graphics · doi:10.1145/1141911.1141985 · Cited by 227
Garg, K., & Nayar, S. K. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.253 · Cited by 225
Self-Learning Based Image Decomposition With Applications to Single Image Denoising
Huang, D.-A., Kang, L.-W., Wang, Y.-C. F., & Lin, C.-W. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2284759 · Cited by 214
Deep Retinex Decomposition for Low-Light Enhancement
Wei, C., Wang, W., Yang, W., & Liu, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.04560 · Cited by 212
Self-citationDeep Edge Guided Recurrent Residual Learning for Image Super-Resolution
Yang, W., Feng, J., Yang, J., Zhao, F., Liu, J., Guo, Z., & Yan, S. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2750403 · Cited by 210
Self-citationErase or Fill? Deep Joint Recurrent Rain Removal and Reconstruction in Videos
Liu, J., Yang, W., Yang, S., & Guo, Z. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00341 · Cited by 193
Self-citationUtilizing Local Phase Information to Remove Rain from Video
Santhaseelan, V., & Asari, V. K. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0759-8 · Cited by 174
Dmcnn: Dual-Domain Multi-Scale Convolutional Neural Network for Compression Artifacts Removal
Zhang, X., Yang, W., Hu, Y., & Liu, J. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451694 · Cited by 172
Self-citationA directional global sparse model for single image rain removal
Deng, L.-J., Huang, T., Zhao, X.-L., & Jiang, T.-X. (2018) · Applied Mathematical Modelling · doi:10.1016/j.apm.2018.03.001 · Cited by 162
Using the Shape Characteristics of Rain to Identify and Remove Rain from Video
Brewer, N., & Liu, N. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-89689-0_49 · Cited by 120
D3R-Net: Dynamic Routing Residue Recurrent Network for Video Rain Removal
Liu, J., Yang, W., Yang, S., & Guo, Z. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2869722 · Cited by 117
Self-citationExploiting image structural similarity for single image rain removal
Sun, S.-H., Fan, S.-P., & Wang, Y.-C. F. (2014) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2014.7025909 · Cited by 115
Context-Aware Single Image Rain Removal
Huang, D.-A., Kang, L., Yang, M.-C., Lin, C., & Wang, Y.-C. F. (2012) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2012.92 · Cited by 111
Seeing through water: Image restoration using model-based tracking
Tian, Y., & Narasimhan, S. G. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459440 · Cited by 107
One-for-All: Grouped Variation Network-Based Fractional Interpolation in Video Coding
Liu, J., Xia, S., Yang, W., Li, M., & Liu, D. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2882923 · Cited by 78
Self-citationBlind visual quality assessment for image super-resolution by convolutional neural network
Fang, Y., Zhang, C., Yang, W., Liu, J., & Guo, Z. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-5805-z · Cited by 67
Self-citationVideo super-resolution based on spatial-temporal recurrent residual networks
Yang, W., Feng, J., Xie, G.-S., Liu, J., Guo, Z., & Yan, S. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.09.002 · Cited by 63
Self-citationFine-Grained Quality Assessment for Compressed Images
Zhang, X., Lin, W., Wang, S., Liu, J., Ma, S., & Gao, W. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2874283 · Cited by 44
Self-citationDeep joint rain and haze removal from a single image
Shen, L. C., Yue, Z., Chen, Q., Feng, F., & Ma, J. (2018) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2018.8545729 · Cited by 36
Reference-Guided Deep Super-Resolution via Manifold Localized External Compensation
Yang, W., Xia, S., Liu, J., & Guo, Z. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2838453 · Cited by 32
Self-citationA Group Variational Transformation Neural Network for Fractional Interpolation of Video Coding
Xia, S., Yang, W., Hu, Y., Ma, S., & Liu, J. (2018) · DCC · doi:10.1109/dcc.2018.00021 · Cited by 25
Self-citationReal-time deep image super-resolution via global context aggregation and local queue jumping
Hu, Y., Liu, J., Yang, W., Deng, S., Zhang, L., & Guo, Z. (2017) · IEEE International Conference on Visual Communications and Image Processing (VCIP) · doi:10.1109/vcip.2017.8305071 · Cited by 4
Self-citation
Cited by
Pre-Trained Image Processing Transformer
Chen, H., Wang, Y., Guo, T., Xu, C., Deng, Y., Liu, Z., Ma, S., Xu, C., Xu, C., & Gao, W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01212 · Cited by 2,308
“For the rain-streak generation, we follow the method described in [79].”
MethodUformer: A General U-Shaped Transformer for Image Restoration
Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., & Li, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01716 · Cited by 2,274
Self-citationA Survey of Deep Learning-Based Object Detection
Jiao, L., Zhang, F., Liu, F., Yang, S., Li, L., Feng, Z., & Qu, R. (2019) · IEEE Access · doi:10.1109/access.2019.2939201 · Cited by 1,323
A Model-Driven Deep Neural Network for Single Image Rain Removal
Wang, H., Xie, Q., Zhao, Q., & Meng, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00317 · Cited by 449
All-In-One Image Restoration for Unknown Corruption
Li, B., Liu, X., Hu, P., Wu, Z., Lv, J., & Peng, X. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01693 · Cited by 432
Metadata
- Authors
- Yang, W., Tan, R. T., Feng, J., Guo, Z., Yan, S., & Liu, J.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2020-06-01
- Volume
- 42
- Issue
- 6
- Pages
- 1377-1393
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2913360047
- Cited by
- 439
- References
- 68
- FWCI
- 22.57
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.