Learning From Synthetic Photorealistic Raindrop for Single Image Raindrop Removal
Open accessAbstract
Raindrops adhered to camera lens or windshield are inevitable in rainy scenes and can become an issue for many computer vision systems such as autonomous driving. Because raindrop appearance is affected by too many parameters, therefore it is unlikely to find an effective model based solution. Learning based methods are also problematic, because traditional learning method cannot properly model the complex appearance. Whereas deep learning method lacks sufficiently large and realistic training data. To solve it, in our work, we propose the first photo-realistic dataset of synthetic adherent raindrops for training. The rendering is physics based with consideration of the water dynamic, geometric and photometry. The dataset contains various types of rainy scenes and particularly the rainy driving scenes. Based on the modeling of raindrop imagery, we introduce a detection network which has the awareness of the raindrop refraction as well as its blurring. Based on that, we propose the removal network that can well recover the image structure. Rigorous experiments demonstrate the state-of-the-art performance of our proposed framework.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Hoa et al.'s proposed algorithm performed better than all other algorithms in terms of SSIM and PSNR scores.
“The quantitative results showed that Hoa et al. proposed algorithm performed better than all other algorithms in terms of SSIM and PSNR scores.”
From A Review of Detection and Removal of Raindrops in Automotive Vision Systems (2021) · cites this paper as [56]
Section 5.1 reports that the complete model outperformed the compared methods on both PSNR and SSIM on the synthetic test set. The claim omits that setting: Section 5.2 says its quantitative results on the real-world dataset were on par with Qian’s method, not better than every comparator. (LLM, conf=0.86)
Experimental — accuracy not yet measuredHoa et al. developed a synthetic raindrop generator and used synthetically generated rainy scenes along with real rainy scenes to train their GAN-based raindrop removal algorithm.
“Hoa et al. [56] tried to solve this problem by developing a synthetic raindrop generator and using the synthetically generated rainy scenes, along with real raindrop real rainy scenes to train their GAN-based raindrop removal algorithm.”
From A Review of Detection and Removal of Raindrops in Automotive Vision Systems (2021) · cites this paper as [56]
Section 3 describes generating photorealistic raindrops for a synthetic training dataset, and Section 5.2 describes training on a real-world raindrop dataset. But Section 4 describes the authors’ removal method as a three-subnetwork detection-and-reconstruction architecture, not a GAN; Section 2.2 attributes the GAN-based method to Qian et al. The paper also does not say the synthetic and real scenes were combined in one training set. (LLM, conf=0.90)
Experimental — accuracy not yet measuredWhen trained using Qian et al.'s dataset with only a rough mask, Hoa et al.'s proposed algorithm produced results on-par with Qian and better than all other methods.
“Results showed that the proposed algorithm produced results that were on-par with Qian [49] and better than all other methods.”
From A Review of Detection and Removal of Raindrops in Automotive Vision Systems (2021) · cites this paper as [56]
Section 5.2 says the methods trained on Qian’s dataset using rough masks achieved quantitative results on par with Qian’s. The statement that the proposed method outperformed other methods appears in Section 5.1, but it concerns the complete model on a synthetic dataset, not the rough-mask model on Qian’s dataset. Page numbers are unavailable in our copy. (LLM, conf=0.86)
Experimental — accuracy not yet measuredHoa et al.'s proposed algorithm generated de-rained images with the least artifacts compared to other benchmarked methods.
“The proposed algorithm as well generated de-rained images with the least artifacts, as compared to other methods benchmarked.”
From A Review of Detection and Removal of Raindrops in Automotive Vision Systems (2021) · cites this paper as [56]
Qualitatively, Hoa et al.'s proposed algorithm, as well as Qian's and DID-MDN, removed most raindrops successfully.
“Qualitatively, the proposed algorithm, as well as Qian’s and the DIID-MDN removed most raindrops successfully.”
From A Review of Detection and Removal of Raindrops in Automotive Vision Systems (2021) · cites this paper as [56]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · see citing works
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392 · see citing works
Labaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,100 · see citing works
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.690 · Cited by 19,892 · see citing works
TensorFlow: A system for large-scale machine learningPreprint
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.5555/3026877.3026899 · Cited by 16,369 · see citing works
(2010) · Cited by 14,134 · see citing works
(2010) · International Conference on Machine Learning · Cited by 13,995 · see citing works
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · see citing works
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741 · see citing works
Single image haze removal using dark channel prior
He, K., Sun, J., & Tang, X. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.168 · Cited by 7,601 · see citing works
A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms
Scharstein, D., & Szeliski, R. (2002) · International Journal of Computer Vision · doi:10.1023/a:1014573219977 · Cited by 6,740 · see citing works
Online Object Tracking: A Benchmark
Wu, Y., Lim, J., & Yang, M.-H. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.312 · Cited by 4,138 · see citing works
DehazeNet: An End-to-End System for Single Image Haze Removal
Cai, B., Xu, X., Jia, K., Qing, C., & Tao, D. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2598681 · Cited by 3,367 · see citing works
Unsupervised Monocular Depth Estimation with Left-Right Consistency
Godard, C., Aodha, O. M., & Brostow, G. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.699 · Cited by 3,270 · see citing works
ECO: Efficient Convolution Operators for Tracking
Danelljan, M., Bhat, G., Khan, F. S., & Felsberg, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.733 · Cited by 2,805 · see citing works
Loss Functions for Image Restoration With Neural Networks
Zhao, H., Gallo, O., Frosio, I., & Kautz, J. (2016) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2016.2644865 · Cited by 2,659 · see citing works
A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.19 · Cited by 2,457 · see citing works
Deep Joint Rain Detection and Removal from a Single Image
Yang, W., Tan, R. T., Feng, J., Liu, J., Guo, Z., & Yan, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.183 · Cited by 1,280 · see citing works
Density-Aware Single Image De-raining Using a Multi-stream Dense Network
Zhang, H., & Patel, V. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00079 · Cited by 1,066 · see citing works
Rain Streak Removal Using Layer Priors
Li, Y., Tan, R. T., Guo, X., Lu, J., & Brown, M. S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.299 · Cited by 980 · see citing works
Self-citationRemoving Rain from a Single Image via Discriminative Sparse Coding
Luo, Y., Xu, Y., & Ji, H. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.388 · Cited by 863 · see citing works
Attentive Generative Adversarial Network for Raindrop Removal from A Single Image
Qian, R., Tan, R. T., Yang, W., Su, J., & Liu, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00263 · Cited by 821 · see citing works
Restoring an Image Taken through a Window Covered with Dirt or Rain
Eigen, D., Krishnan, D., & Fergus, R. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.84 · Cited by 454 · see citing works
User Assisted Separation of Reflections from a Single Image Using a Sparsity Prior
Levin, A., & Weiss, Y. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1106 · Cited by 391 · see citing works
A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing
Fan, Q., Yang, J., Hua, G., Chen, B., & Wipf, D. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.351 · Cited by 333 · see citing works
Exploiting Reflection Change for Automatic Reflection Removal
Li, Y., & Brown, M. S. (2013) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.302 · Cited by 191 · see citing works
Self-citationLi, X., Ren, J., Yan, Q., Liao, R., & Jia, J. (2015) · Cited by 175 · see citing works
Adherent Raindrop Modeling, Detection and Removal in Video
You, S., Tan, R. T., Kawakami, R., Mukaigawa, Y., & Ikeuchi, K. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2491937 · Cited by 169 · see citing works
Self-citationVideo-based raindrop detection for improved image registration
Roser, M., & Geiger, A. (2009) · 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops · doi:10.1109/iccvw.2009.5457650 · Cited by 126 · see citing works
Halimeh, J. C., & Roser, M. (2009) · 2009 IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2009.5164347 · Cited by 76 · see citing works
Adherent Raindrop Detection and Removal in Video
You, S., Tan, R. T., Kawakami, R., et al. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.138 · Cited by 71 · see citing works
Self-citationRealistic Modeling of Water Droplets for Monocular Adherent Raindrop Recognition Using Bézier Curves
Roser, M., Kurz, J., & Geiger, A. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-22819-3_24 · Cited by 63 · see citing works
von Bernuth, A., Volk, G., & Bringmann, O. (2018) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2018.8500494 · Cited by 21 · see citing works
Raindrop Detection and Removal from Long Range Trajectories
You, S., Tan, R. T., Kawakami, R., Mukaigawa, Y., & Ikeuchi, K. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16808-1_38 · Cited by 17 · see citing works
Self-citationWaterdrop StereoPreprint
You, S., Tan, R. T., Kawakami, R., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.00730 · Cited by 9 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 5
- I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Gen- erative adversarial nets. In Advances in neural information processing systems (NIPS), 2014. 2
- K. He, G. Gkioxari, P. Dollár, and R. Girshick. Mask r- cnn. In IEEE International Conference on Computer Vision (ICCV), 2017. 1, 4
- J. Valmadre, L. Bertinetto, J. Henriques, A. Vedaldi, and P. H. Torr. End-to-end representation learning for correla- tion filter based tracking. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. 1
- S. You, R. T. Tan, R. Kawakami, Y. Mukaigawa, and K. Ikeuchi. Adherent raindrop modeling, detectionand re- moval in video. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 38(9):1721-1733, 2016. 1, 2
- F. Yu and V. Koltun. Multi-scale context aggregation by di- lated convolutions. In International Conference on Learning Representations (ICLR), 2016. 4
Cited by
Sorted by citations
All in One Bad Weather Removal Using Architectural Search
Li, R., Tan, R. T., & Cheong, L.-F. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00324 · Cited by 377 · see citing works
A Review of Detection and Removal of Raindrops in Automotive Vision Systems
Hamzeh, Y., & Rawashdeh, S. A. (2021) · Journal of Imaging · doi:10.3390/jimaging7030052 · Cited by 21 · see citing works
“[56] tried to solve this problem by developing a synthetic raindrop generator and using the synthetically generated rainy scenes, along with real raindrop real rainy scenes to train their GAN-based raindrop removal algorithm.”
BackgroundMethodInfluentialChen, Z., Lu, L., Yuan, Z., Zhu, Y., Li, Y., Yuan, C., & Deng, W. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i2.27889 · Cited by 3 · see citing works
Self-citation
- Authors
- Hao, Z., You, S., Li, Y., Li, K., & Lu, F.
- Affiliations
- Beihang University (CN)
- Commonwealth Scientific and Industrial Research Organisation (AU)
- Tencent (China) (CN)
- Australian National University (AU)
- Peng Cheng Laboratory (CN)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
- Publisher
- IEEE
- Published
- 2019-10
- Volume
- 2019
- Pages
- 4340-4349
- ISSN
- 2473-9936 (ISSN-L)
- OpenAlex
- W2991017102
- Cited by
- 54
- References
- 41
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 8 · 2023: 13 · 2022: 10 · 2021: 7 · 2020: 7
- FWCI
- 1.96
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE International Conference on Computer Vision Workshops (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- IEEE Conference Proceedings
- Funders
- National Natural Science Foundation of China (61972012, 61732016)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Enhancement Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Signal Denoising Methods
- Keywords
- raindrop removal, photorealistic rendering, physically based rendering, raindrop detection, image restoration, autonomous driving
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
