Raindrop-Aware GAN: Unsupervised Learning for Raindrop-Contaminated Coastal Video Enhancement
Open accessAbstract
We propose an unsupervised network with adversarial learning, the Raindrop-aware GAN, which enhances the quality of coastal video images contaminated by raindrops. Raindrop removal from coastal videos faces two main difficulties: converting the degraded image into a clean one by visually removing the raindrops, and restoring the background coastal wave information in the raindrop regions. The components of the proposed network—a generator and a discriminator for adversarial learning—are trained on unpaired images degraded by raindrops and clean images free from raindrops. By creating raindrop masks and background-restored images, the generator restores the background information in the raindrop regions alone, preserving the input as much as possible. The proposed network was trained and tested on an open-access dataset and directly collected dataset from the coastal area. It was then evaluated by three metrics: the peak signal-to-noise ratio, structural similarity, and a naturalness-quality evaluator. The indices of metrics are 8.2% (+2.012), 0.2% (+0.002), and 1.6% (−0.196) better than the state-of-the-art method, respectively. In the visual assessment of the enhanced video image quality, our method better restored the image patterns of steep wave crests and breaking than the other methods. In both quantitative and qualitative experiments, the proposed method more effectively removed the raindrops in coastal video and recovered the damaged background wave information than state-of-the-art methods.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Adverse weather conditions, especially rain, degrade the quality of captured images/videos, making them difficult to use as training data.
“Adverse weather conditions, especially rain, degrades the quality of captured images/videos which makes them difficult to use as training data [61].”
From A Review of Detection and Removal of Raindrops in Automotive Vision Systems (2021) · cites this paper as [61]
Kim et al. proposed an unsupervised GAN-based video generation algorithm (Raindrop-aware GAN) to enhance raindrop-contaminated coastal video.
“Kim et al. [61] proposed an algorithm for unsupervised learning with (GAN)-based video generation (Raindrop-aware GAN) to enhance a raindrop-contaminated coastal video.”
From A Review of Detection and Removal of Raindrops in Automotive Vision Systems (2021) · cites this paper as [61]
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · see citing works
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · see citing works
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · see citing works
(2010) · International Conference on Machine Learning · Cited by 13,995 · see citing works
Making a “Completely Blind” Image Quality Analyzer
Mittal, A., Soundararajan, R., & Bovik, A. C. (2013) · IEEE Signal Processing Letters · doi:10.1109/lsp.2012.2227726 · Cited by 6,766 · see citing works
Deep learning for computer vision: A brief review
Voulodimos, A., Doulamis, N., Doulamis, A., & Protopapadakis, E. (2018) · Computational Intelligence and Neuroscience · doi:10.1155/2018/7068349 · Cited by 3,403 · see citing works
Scope of validity of PSNR in image/video quality assessment
Huynh-Thu, Q., & Ghanbari, M. (2008) · Electronics Letters · doi:10.1049/el:20080522 · Cited by 2,959 · see citing works
Attentive Generative Adversarial Network for Raindrop Removal from A Single Image
Qian, R., Tan, R. T., Yang, W., Su, J., & Liu, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00263 · Cited by 821 · see citing works
A comprehensive review of deep learning applications in hydrology and water resources
Sit, M. A., Demiray, B. Z., Xiang, Z., Ewing, G. J., Sermet, Y., & Demir, İ. (2020) · Water Science & Technology · doi:10.2166/wst.2020.369 · Cited by 631 · see citing works
The history and technical capabilities of Argus
Holman, R. A., & Stanley, J. (2007) · Coastal Engineering · doi:10.1016/j.coastaleng.2007.01.003 · Cited by 618 · see citing works
Restoring an Image Taken through a Window Covered with Dirt or Rain
Eigen, D., Krishnan, D., & Fergus, R. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.84 · Cited by 454 · see citing works
Introduction to Coastal Processes and Geomorphology
Davidson-Arnott, R. G. D., Bauer, B. O., & Houser, C. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108546126 · Cited by 276 · see citing works
cBathy: A robust algorithm for estimating nearshore bathymetry
Holman, R. A., Plant, N. G., & Holland, T. (2013) · Journal of Geophysical Research Oceans · doi:10.1002/jgrc.20199 · Cited by 254 · see citing works
Benjdira, B., Bazi, Y., Koubâa, A., & Ouni, K. (2019) · Remote Sensing · doi:10.3390/rs11111369 · Cited by 236 · see citing works
Remote Sensing of the Nearshore
Holman, R. A., & Haller, M. C. (2012) · Annual Review of Marine Science · doi:10.1146/annurev-marine-121211-172408 · Cited by 181 · see citing works
Building Change Detection in VHR SAR Images via Unsupervised Deep Transcoding
Saha, S., Bovolo, F., & Bruzzone, L. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3000296 · Cited by 179 · see citing works
Reyes, M. F., Auer, S., Merkle, N., Henry, C., & Schmitt, M. (2019) · Remote Sensing · doi:10.3390/rs11172067 · Cited by 169 · see citing works
Generative Adversarial Networks and Conditional Random Fields for Hyperspectral Image Classification
Zhong, Z., Li, J., Clausi, D. A., & Wong, A. K. C. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2915094 · Cited by 165 · see citing works
Vousdoukas, M. I., Wziatek, D., & Almeida, L. P. (2011) · Ocean Dynamics · doi:10.1007/s10236-011-0480-x · Cited by 146 · see citing works
Splinter, K. D., Harley, M. D., & Turner, I. L. (2018) · Remote Sensing · doi:10.3390/rs10111744 · Cited by 140 · see citing works
Atkinson, A. L., Power, H. E., Moura, T., et al. (2016) · Coastal Engineering · doi:10.1016/j.coastaleng.2016.10.001 · Cited by 132 · see citing works
Video-based raindrop detection for improved image registration
Roser, M., & Geiger, A. (2009) · 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops · doi:10.1109/iccvw.2009.5457650 · Cited by 126 · see citing works
On the prediction of runup, setup and swash on beaches
da Silva, P. G., Coco, G., Garnier, R., et al. (2020) · Earth-Science Reviews · doi:10.1016/j.earscirev.2020.103148 · Cited by 125 · see citing works
Nearshore subtidal bathymetry from time‐exposure video images
Aarninkhof, S., Ruessink, G., & Roelvink, D. (2005) · Journal of Geophysical Research Atmospheres · doi:10.1029/2004jc002791 · Cited by 95 · see citing works
Ocean Wavenumber Estimation From Wave-Resolving Time Series Imagery
Plant, N. G., Holland, K. T., & Haller, M. C. (2008) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2008.919821 · Cited by 87 · see citing works
A Comprehensive Review of Deep Learning Applications in Hydrology and Water ResourcesPreprint
Sit, M. A., Demiray, B. Z., Xiang, Z., Ewing, G., Sermet, Y., & Demir, İ. (2020) · doi:10.31223/osf.io/xs36g · Cited by 75 · see citing works
Unsupervised Deep Transfer Learning-Based Change Detection for HR Multispectral Images
Saha, S., Solano‐Correa, Y. T., Bovolo, F., & Bruzzone, L. (2020) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2020.2990284 · Cited by 73 · see citing works
Adherent Raindrop Detection and Removal in Video
You, S., Tan, R. T., Kawakami, R., et al. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.138 · Cited by 71 · see citing works
Benjdira, B., Bazi, Y., Koubâa, A., & Ouni, K. (2019) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) · Cited by 70 · see citing works
Video Sensing of Nearshore Bathymetry Evolution with Error Estimate
Thuan, D. H., Almar, R., Marchesiello, P., et al. (2019) · Journal of Marine Science and Engineering · doi:10.3390/jmse7070233 · Cited by 44 · see citing works
Ondoa, G. A., Bonou, F., Tomety, F. S., et al. (2017) · Water · doi:10.3390/w9060447 · Cited by 40 · see citing works
Breaking Wave Height Estimation from Timex Images: Two Methods for Coastal Video Monitoring Systems
Andriolo, U., Mendes, D., & Taborda, R. (2020) · Remote Sensing · doi:10.3390/rs12020204 · Cited by 39 · see citing works
Semi-Deraingan: A New Semi-Supervised Single Image Deraining
Wei, Y., Zhang, Z., Wang, Y., et al. (2021) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme51207.2021.9428285 · Cited by 39 · see citing works
Nearshore Wave Transformation Domains from Video Imagery
Andriolo, U. (2019) · Journal of Marine Science and Engineering · doi:10.3390/jmse7060186 · Cited by 29 · see citing works
Wave-Tracking in the Surf Zone Using Coastal Video Imagery with Deep Neural Networks
Kim, J., Kim, J., Kim, T., Huh, D., & Caires, S. (2020) · Atmosphere · doi:10.3390/atmos11030304 · Cited by 28 · see citing works
Self-citationThe Future of Nearshore Processes Research
Elko, N. A., Feddersen, F., Foster, D., et al. (2014) · 2014 AGU Fall Meeting · Cited by 27 · see citing works
Shoreline Response to a Sandy Nourishment in a Wave-Dominated Coast Using Video Monitoring
Santos, C. J., Andriolo, U., & Ferreira, J. C. (2020) · Water · doi:10.3390/w12061632 · Cited by 23 · see citing works
Exploiting remote imagery in an embayed sandy beach for the validation of a runup model framework
Valentini, N., Saponieri, A., Danisi, A., et al. (2019) · Estuarine Coastal and Shelf Science · doi:10.1016/j.ecss.2019.106244 · Cited by 20 · see citing works
Long-Term Automated Monitoring of Nearshore Wave Height From Digital Video
Gal, Y., Browne, M., & Lane, C. (2013) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2013.2272790 · Cited by 16 · see citing works
Peng, J., Xu, Y., Chen, T., & Huang, Y. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2019.12.012 · Cited by 16 · see citing works
Unsupervised deep learning based change detection in Sentinel-2 images
Saha, S., Solano‐Correa, Y. T., Bovolo, F., & Bruzzone, L. (2019) · International Workshop on the Analysis of Multitemporal Remote Sensing Images (MultiTemp) · doi:10.1109/multi-temp.2019.8866899 · Cited by 13 · see citing works
Cycle-Spinning GAN for Raindrop Removal from Images
Uzun, U., & Temizel, A. (2019) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2019.8909824 · Cited by 10 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Review of Detection and Removal of Raindrops in Automotive Vision Systems
Hamzeh, Y., & Rawashdeh, S. A. (2021) · Journal of Imaging · doi:10.3390/jimaging7030052 · Cited by 21 · see citing works
“LGG = ee1Lrreegg + ee2Lmmddsskk + ee31Lggeenn where ee1, ee2,aanndd ee3 are hyperparameters to control the importance of each loss function [61].”
BackgroundMethodInfluential
- Authors
- Kim, J., Huh, D., Kim, T., Kim, J., Yoo, J., & Shim, J.-S.
- Affiliations
- Korea Institute of Ocean Science and Technology (KR)
- Kyungpook National University (KR)
- Kyungpook National University, 80 Daehak-ro, Buk-gu, Daegu 41566, Korea
- Venue
- Remote Sensing
- Publisher
- MDPI AG
- Published
- 2020-10-21
- Volume
- 12
- Issue
- 20
- Pages
- 3461
- ISSN
- 2072-4292 (ISSN-L)
- OpenAlex
- W3094623056
- Cited by
- 4
- References
- 42
- Citations by year
- 2025: 1 · 2022: 1 · 2021: 2
- FWCI
- 0.29
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Remote Sensing (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Korea Institute of Ocean Science and Technology (PE99842)
- Ministry of Science and ICT, South Korea (2020-0-00843)
- Ministry of Oceans and Fisheries (PM61730)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Enhancement Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Signal Denoising Methods
- Keywords
- raindrop removal, unsupervised image restoration, generative adversarial networks, background restoration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
