Fast Underwater Image Enhancement for Improved Visual Perception
Open accessThe DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
In this letter, we present a conditional generative adversarial network-based model for real-time underwater image enhancement. To supervise the adversarial training, we formulate an objective function that evaluates the perceptual image quality based on its global content, color, local texture, and style information. We also present EUVP,a large-scale dataset of a paired and an unpaired collection of underwater images (of `poor' and `good' quality) that are captured using seven different cameras over various visibility conditions during oceanic explorations and human-robot collaborative experiments. In addition, we perform several qualitative and quantitative evaluations which suggest that the proposed model can learn to enhance underwater image quality from both paired and unpaired training. More importantly, the enhanced images provide improved performances of standard models for underwater object detection, human pose estimation, and saliency prediction. These results validate that it is suitable for real-time preprocessing in the autonomy pipeline by visually-guided underwater robots. The model and associated training pipelines are available athttps://github.com/xahidbuffon/funie-gan.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · cited 14×
“Moreover, the GAN-based models [25] have shown great success for style-transfer and image-to-image translation problems [16].”
BackgroundMethodInfluentialGAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
Conditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863
Single image haze removal using dark channel prior
He, K., Sun, J., & Tang, X. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.168 · Cited by 7,601
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539 · cited 1×
“We analyze the performance of standard deep visual models for underwater object detection [7], 2D human body-pose estimation [8], and visual attention-based saliency prediction [9]; although results vary depending on the image qualities of a particular test set, on an average, we observe 11-14%, 22-28%, and 26-28% imp…”
(2017) · International Conference on Machine Learning · Cited by 6,382
Image Style Transfer Using Convolutional Neural Networks
Gatys, L. A., Ecker, A. S., & Bethge, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.265 · Cited by 6,025
Learning a Deep Convolutional Network for Image Super-Resolution
Dong, C., Loy, C. C., He, K., & Tang, X. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10593-2_13 · Cited by 5,494
“Least squares generative adversarial networks,”
Mao, X., Li, Q., Xie, H., Lau, R. Y. K., Wang, Z., & Smolley, S. P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.304 · Cited by 5,437
Image Quality Metrics: PSNR vs. SSIM
Hore, A., & Ziou, D. (2010) · 2010 20th International Conference on Pattern Recognition · doi:10.1109/icpr.2010.579 · Cited by 4,943
DehazeNet: An End-to-End System for Single Image Haze Removal
Cai, B., Xu, X., Jia, K., Qing, C., & Tao, D. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2598681 · Cited by 3,367
Zhang, R., Isola, P., & Efros, A. A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_40 · Cited by 3,178
DualGAN: Unsupervised Dual Learning for Image-to-Image Translation
Yi, Z., Zhang, H., Tan, P., & Gong, M. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.310 · Cited by 2,134
Human-Visual-System-Inspired Underwater Image Quality Measures
Panetta, K., Gao, C., & Agaian, S. (2016) · IEEE Journal of Oceanic Engineering · doi:10.1109/joe.2015.2469915 · Cited by 1,718
Perceptual Generative Adversarial Networks for Small Object Detection
Li, J., Liang, X., Wei, Y., Xu, T., Feng, J., & Yan, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.211 · Cited by 953 · cited 4×
“Qualitative performance comparison of FUnIE-GAN and FUnIE-GAN-UP with learning-based methods: CycleGAN [17], UGAN-P [6], Pix2Pix [16], LS-GAN [28], Res-GAN [43], and Res-WGAN [26]; the super-scripted asterisk (∗) denotes unpaired training.”
BackgroundMethodInfluentialEnergy-based Generative Adversarial Network
Zhao, J., Mathieu, M., & LeCun, Y. (2016) · arXiv · doi:10.48550/arxiv.1609.03126 · Cited by 912 · cited 1×
“Energy-based GANs [27] also improve training stability by modeling the discriminator as an energy function, whereas the Least-Squared GAN [28] addresses the vanishing gradients problem by adopting a least-square loss function for the discriminator.”
MethodReal-World Underwater Enhancement: Challenges, Benchmarks, and Solutions Under Natural Light
Liu, R., Fan, X., Zhu, M., Hou, M., & Luo, Z. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2963772 · Cited by 863
Retinex processing for automatic image enhancement
Jobson, D. J. (2004) · Journal of Electronic Imaging · doi:10.1117/1.1636183 · Cited by 827
Enhancing Underwater Imagery Using Generative Adversarial Networks
Fabbri, C., Islam, M. J., & Sattar, J. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460552 · Cited by 808
Self-citationMotion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency
Irani, M., & Peleg, S. (1993) · Journal of Visual Communication and Image Representation · doi:10.1006/jvci.1993.1030 · Cited by 805
Underwater Single Image Color Restoration Using Haze-Lines and a New Quantitative Dataset
Berman, D., Levy, D., Avidan, S., & Treibitz, T. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2977624 · Cited by 608
DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks
Ignatov, A., Kobyshev, N., Timofte, R., & Vanhoey, K. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.355 · Cited by 582
Cheng, Z., Yang, Q., & Sheng, B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.55 · Cited by 557
Underwater Image Enhancement Using a Multiscale Dense Generative Adversarial Network
Guo, Y., Li, H., & Zhuang, P. (2020) · IEEE Journal of Oceanic Engineering · doi:10.1109/joe.2019.2911447 · Cited by 508
Deep Photo Enhancer: Unpaired Learning for Image Enhancement from Photographs with GANs
Chen, Y.-S., Wang, Y.-C., Kao, M.-H., & Chuang, Y.-Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00660 · Cited by 467
A Revised Underwater Image Formation Model
Akkaynak, D., & Treibitz, T. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00703 · Cited by 457
Underwater image enhancement via extended multi-scale Retinex
Zhang, S., Wang, T., Dong, J., & Yu, H. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.03.029 · Cited by 443
Bingham, B., Foley, B., Singh, H., Camilli, R., Delaporta, K., Eustice, R., Mallios, A., Mindell, D., Roman, C., & Sakellariou, D. (2010) · Journal of Field Robotics · doi:10.1002/rob.20350 · Cited by 250
AQUA: An Amphibious Autonomous Robot
Dudek, G., Giguère, P., Prahacs, C., Saunderson, S., Sattar, J., Torres-Méndez, L. A., Jenkin, M. R. M., German, A., Hogue, A., Ripsman, A., et al. (2007) · Computer · doi:10.1109/mc.2007.6 · Cited by 246
Self-citationSalient Object Detection Driven by Fixation Prediction
Wang, W., Shen, J., Dong, X., & Borji, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00184 · Cited by 205
Underwater Image Enhancement With a Deep Residual Framework
Liu, P., Wang, G., Qi, H., Zhang, C., Zheng, H., & Yu, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2928976 · Cited by 174
Compression Artifacts Removal Using Convolutional Neural NetworksPreprint
Svoboda, P., Hradiš, M., Barina, D., & Zemčík, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.00366 · Cited by 152
True Color Correction of Autonomous Underwater Vehicle Imagery
Bryson, M., Johnson‐Roberson, M., Pizarro, O. R., et al. (2015) · Journal of Field Robotics · doi:10.1002/rob.21638 · Cited by 129
Underwater-GAN: Underwater Image Restoration via Conditional Generative Adversarial Network
Yu, X., Qu, Y., & Hong, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-05792-3_7 · Cited by 125
Lu, H., Li, Y., & Serikawa, S. (2013) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2013.6738704 · Cited by 118
Understanding human motion and gestures for underwater human–robot collaboration
Islam, J., Ho, M., & Sattar, J. (2018) · Journal of Field Robotics · doi:10.1002/rob.21837 · Cited by 109
Self-citationModel Assisted Multi-band Fusion for Single Image Enhancement and Applications to Robot Vision
Cho, Y., Jeong, J., & Kim, A. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2843127 · Cited by 102
Islam, M. J., Fulton, M., & Sattar, J. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2882856 · Cited by 91
Self-citationJoint Rain Detection and Removal via Iterative Region Dependent Multi-Task Learning.Preprint
Yang, W., Tan, R. T., Feng, J., Liu, J., Guo, Z., & Yan, S. (2016) · arXiv (Cornell University) · Cited by 49
Real-world Underwater Enhancement: Challenging, Benchmark and Efficient Solutions.Preprint
Liu, R., Hou, M., Fan, X. N., & Luo, Z. (2019) · arXiv (Cornell University) · Cited by 5
Cited by
U-Shape Transformer for Underwater Image Enhancement
Peng, L., Zhu, C., & Bian, L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3276332 · Cited by 912
Underwater Image Enhancement via Minimal Color Loss and Locally Adaptive Contrast Enhancement
Zhang, W., Zhuang, P., Sun, H.-H., Li, G., Kwong, S., & Li, C. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3177129 · Cited by 888
Underwater Image Enhancement via Weighted Wavelet Visual Perception Fusion
Zhang, W., Zhou, L., Zhuang, P., Li, G., Pan, X., Zhao, W., & Li, C. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3299314 · Cited by 406
Twin Adversarial Contrastive Learning for Underwater Image Enhancement and Beyond
Liu, R., Jiang, Z., Yang, S., & Fan, X. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3190209 · Cited by 390
PUGAN: Physical Model-Guided Underwater Image Enhancement Using GAN With Dual-Discriminators
Cong, R., Yang, W., Zhang, W., Li, C., Guo, C.-L., Huang, Q., & Kwong, S. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3286263 · Cited by 350
Metadata
- Authors
- Islam, M. J., Xia, Y., & Sattar, J.
- Venue
- IEEE Robotics and Automation Letters
- Published
- 2020-04
- Volume
- 5
- Issue
- 2
- Pages
- 3227-3234
- ISSN
- 2377-3766 (ISSN-L)
- 2377-3774
- OpenAlex
- W2925094354
- Cited by
- 1,675
- References
- 48
- FWCI
- 51.50
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.