Low-Light Image Enhancement with Wavelet-Based Diffusion Models
The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Abstract
Diffusion models have achieved promising results in image restoration tasks, yet suffer from time-consuming, excessive computational resource consumption, and unstable restoration. To address these issues, we propose a robust and efficient Diffusion-based Low-Light image enhancement approach, dubbed DiffLL. Specifically, we present a wavelet-based conditional diffusion model (WCDM) that leverages the generative power of diffusion models to produce results with satisfactory perceptual fidelity. Additionally, it also takes advantage of the strengths of wavelet transformation to greatly accelerate inference and reduce computational resource usage without sacrificing information. To avoid chaotic content and diversity, we perform both forward diffusion and denoising in the training phase of WCDM, enabling the model to achieve stable denoising and reduce randomness during inference. Moreover, we further design a high-frequency restoration module (HFRM) that utilizes the vertical and horizontal details of the image to complement the diagonal information for better fine-grained restoration. Extensive experiments on publicly available real-world benchmarks demonstrate that our method outperforms the existing state-of-the-art methods both quantitatively and visually, and it achieves remarkable improvements in efficiency compared to previous diffusion-based methods. In addition, we empirically show that the application for low-light face detection also reveals the latent practical values of our method. Code is available at https://github.com/JianghaiSCU/Diffusion-Low-Light.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · cited 11×
“Moreover, we utilize a content loss Lcontent that combines L1 loss and SSIM loss [Wang et al. 2004] to minimize the”
MethodInfluentialXception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
Image denoising by sparse 3-D transform-domain collaborative filtering
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.901238 · Cited by 9,283 · cited 2×
“For example, LIME [Guo et al. 2016] utilized a weighted vibration model based on a prior hypothesis to adjust the estimated illumination, followed by BM3D [Dabov et al. 2007] as a post-processing step for noise removal.”
MethodDenoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712
Making a “Completely Blind” Image Quality Analyzer
Mittal, A., Soundararajan, R., & Bovik, A. C. (2013) · IEEE Signal Processing Letters · doi:10.1109/lsp.2012.2227726 · Cited by 6,766
No-Reference Image Quality Assessment in the Spatial Domain
Mittal, A., Moorthy, A. K., & Bovik, A. C. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2214050 · Cited by 6,064 · cited 7×
“For the other five unpaired datasets, we use three non-reference perceptual metrics NIQE [Mittal et al. 2012b], BRISQUE [Mittal et al. 2012a], and PI [Blau et al. 2018] for evaluation.”
BackgroundMethodInfluentialRestormer: Efficient Transformer for High-Resolution Image Restoration
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., & Yang, M.-H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00564 · Cited by 4,143
A multiscale retinex for bridging the gap between color images and the human observation of scenes
Jobson, D. J., Rahman, Z., & Woodell, G. A. (1997) · IEEE Transactions on Image Processing · doi:10.1109/83.597272 · Cited by 3,036
LIME: Low-Light Image Enhancement via Illumination Map Estimation
Guo, X., Li, Y., & Ling, H. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2639450 · Cited by 2,998
The Retinex Theory of Color Vision
Land, E. H. (1977) · Scientific American · doi:10.1038/scientificamerican1277-108 · Cited by 2,981
EnlightenGAN: Deep Light Enhancement Without Paired Supervision
Jiang, Y., Gong, X., Liu, D., Cheng, Y., Fang, C., Shen, X., Yang, J., Zhou, P., & Wang, Z. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3051462 · Cited by 2,715 · cited 9×
“In row 3, Zero-DCE, EnlightenGAN, and DDIM smooth the cloud texture, as Uformer and WeatherDiff produce gridding effects and messy content.”
BackgroundMethodInfluentialZero-Reference Deep Curve Estimation for Low-Light Image Enhancement
Guo, C., Li, C., Guo, J., Loy, C. C., Hou, J., Kwong, S., & Cong, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00185 · Cited by 2,459 · cited 2×
“…CDEF [Lei et al. 2022], 2) learning-based LLIE methods RetinexNet [Wei et al. 2018], DSLR [Lim and Kim 2020], DRBN [Yang et al. 2020a], Zero-DCE [Guo et al. 2020], MIRNet [Zamir et al. 2020], EnlightenGAN [Jiang et al. 2021], ReLLIE [Zhang et al. 2021], RUAS [Liu et al. 2021], SCI [Ma et al.…”
MethodUformer: A General U-Shaped Transformer for Image Restoration
Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., & Li, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01716 · Cited by 2,274
LLNet: A deep autoencoder approach to natural low-light image enhancement
Lore, K. G., Akintayo, A., & Sarkar, S. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.06.008 · Cited by 1,951 · cited 2×
“Following the development of learning-based image restoration methods [45, 7, 62], LLNet [39] first proposed a deep encoder-decoder network for contrast enhancement and introduced a dataset synthesis pipeline.”
MethodInfluentialRePaint: Inpainting using Denoising Diffusion Probabilistic Models
Lugmayr, A., Danelljan, M., Romero, A., Yu, F., Timofte, R., & Van Gool, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01117 · Cited by 1,778
Naturalness Preserved Enhancement Algorithm for Non-Uniform Illumination Images
Wang, S., Zheng, J., Hu, H.-M., & Li, B. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2261309 · Cited by 1,755
Palette: Image-to-Image Diffusion Models
Saharia, C., Chan, W., Chang, H., Lee, C., Ho, J., Salimans, T., Fleet, D., & Norouzi, M. (2022) · Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings · doi:10.1145/3528233.3530757 · Cited by 1,717
Perceptual Quality Assessment for Multi-Exposure Image Fusion
Ma, K., Zeng, K., & Wang, Z. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2442920 · Cited by 1,190
Toward Fast, Flexible, and Robust Low-Light Image Enhancement
Ma, L., Ma, T., Liu, R., Fan, X., & Luo, Z. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00555 · Cited by 1,099
A Weighted Variational Model for Simultaneous Reflectance and Illumination Estimation
Fu, X., Zeng, D., Huang, Y., Zhang, X.-P., & Ding, X. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.304 · Cited by 1,081
Contrast Enhancement Based on Layered Difference Representation of 2D Histograms
Lee, C., Lee, C., & Kim, C.-S. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2284059 · Cited by 1,066
Liu, R., Ma, L., Zhang, J., Fan, X., & Luo, Z. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01042 · Cited by 1,057
Learning Enriched Features for Real Image Restoration and Enhancement
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58595-2_30 · Cited by 946
Learning a convolutional neural network for non-uniform motion blur removal
Sun, J., Cao, W., Xu, Z., & Ponce, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298677 · Cited by 906
Getting to know low-light images with the Exclusively Dark dataset
Loh, Y. P., & Chan, C. S. (2019) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2018.10.010 · Cited by 843
Handbook of Image and Video Processing
Wells, P. N. T. (2001) · Physiological Measurement · doi:10.1088/0967-3334/22/1/702 · Cited by 827
Sparse Gradient Regularized Deep Retinex Network for Robust Low-Light Image Enhancement
Yang, W., Wang, W., Huang, H., Wang, S., & Liu, J. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3050850 · Cited by 816
URetinex-Net: Retinex-based Deep Unfolding Network for Low-light Image Enhancement
Wu, W., Weng, J., Zhang, P., Wang, X., Yang, W., & Jiang, J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00581 · Cited by 809
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2021) · arXiv · doi:10.48550/arxiv.2112.10752 · Cited by 737
SNR-Aware Low-light Image Enhancement
Xu, X., Wang, R., Fu, C.-W., & Jia, J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01719 · Cited by 688
Low-Light Image and Video Enhancement Using Deep Learning: A Survey
Li, C., Guo, C., Han, L., Jiang, J., Cheng, M.-M., Gu, J., & Loy, C. C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3126387 · Cited by 657
From Fidelity to Perceptual Quality: A Semi-Supervised Approach for Low-Light Image Enhancement
Yang, W., Wang, S., Fang, Y., Wang, Y., & Liu, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00313 · Cited by 644 · cited 6×
“The results presented in rows 1 and 3 show that DRBN, MIRNet, RUAS, and GDP are unable to properly improve the contrast of the images, yielding underexposed or overexposed results.”
MethodInfluentialAn Augmented Lagrangian Method for Total Variation Video Restoration
Chan, S. H., Khoshabeh, R., Gibson, K. B., Gill, P. E., & Nguyen, T. Q. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2158229 · Cited by 488
Li, J., Wang, Y., Wang, C., Tai, Y., Qian, J., Yang, J., Wang, C., Li, J., & Huang, F. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00520 · Cited by 488
Restoring Vision in Adverse Weather Conditions With Patch-Based Denoising Diffusion Models
Özdenizci, O., & Legenstein, R. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3238179 · Cited by 469
R2RNet: Low-light image enhancement via Real-low to Real-normal Network
Hai, J., Xuan, Z., Yang, R., Hao, Y., Zou, F., Lin, F., & Han, S. (2023) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2022.103712 · Cited by 427 · cited 2×
“We first compare our method with all comparison methods on the LOLv1 [Wei et al. 2018], LOLv2-real [Yang et al. 2021b], and LSRW [Hai et al. 2023] test sets.”
MethodSelf-citationDenoising Diffusion Restoration Models
Kawar, B., Elad, M., Ermon, S., & Song, J. (2022) · neural information processing systems · doi:10.52202/068431-1714 · Cited by 349
Advancing Image Understanding in Poor Visibility Environments: A Collective Benchmark Study
Yang, W., Yuan, Y., Ren, W., Liu, J., Scheirer, W. J., Wang, Z., Zhang, T., Zhong, Q., Xie, D., Pu, S., et al. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2981922 · Cited by 342
Self-citationDSLR: Deep Stacked Laplacian Restorer for Low-Light Image Enhancement
Lim, S., & Kim, W. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3039361 · Cited by 295
GAN Prior Embedded Network for Blind Face Restoration in the Wild
Yang, T., Ren, P., Xie, X., & Zhang, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00073 · Cited by 294
Generative Diffusion Prior for Unified Image Restoration and Enhancement
Fei, B., Lyu, Z., Pan, L., Zhang, J., Yang, W., Luo, T., Zhang, B., & Dai, B. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00958 · Cited by 279
Chung, H., Sim, B., & Ye, J. C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01209 · Cited by 275
Zur Theorie der orthogonalen Funktionensysteme
Haar, A. (1911) · Mathematische Annalen · doi:10.1007/bf01456927 · Cited by 256
Refusion: Enabling Large-Size Realistic Image Restoration with Latent-Space Diffusion Models
Luo, Z., Gustafsson, F., Zhao, Z., Sjölund, J., & Schön, T. B. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00169 · Cited by 254 · cited 3×
“Recently, diffusion models (DMs) [58, 61, 60] have garnered attention due to their impressive performance in image synthesis [17, 59] and restoration tasks [42, 33].”
BackgroundMethodA deep learning based image enhancement approach for autonomous driving at night
Li, G., Yang, Y., Qu, X., Cao, D., & Li, K. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106617 · Cited by 242 · cited 4×
“ject detection [34], automatic driving [30], and visual navigation [13], etc.”
BackgroundMethodInfluentialShadowDiffusion: When Degradation Prior Meets Diffusion Model for Shadow Removal
Guo, L., Wang, C., Yang, W., Huang, S., Wang, Y., Pfister, H., & Wen, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01350 · Cited by 239 · cited 1×
“ShadowDiffusion [Guo et al. 2023] designed an unrolling diffusion model that achieves robust shadow removal by progressively refining the results with degradation and generative priors.”
MethodGenerating Diverse Structure for Image Inpainting With Hierarchical VQ-VAE
Peng, J., Liu, D., Xu, S., & Li, H. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01063 · Cited by 232
Enhancement of low exposure images via recursive histogram equalization algorithms
Singh, K., Kapoor, R., & Sinha, S. K. (2015) · Optik · doi:10.1016/j.ijleo.2015.06.060 · Cited by 178 · cited 2×
“HEbased methods [57, 50, 48] take effect by changing the histogram of the image to improve the contrast.”
BackgroundMethodDR2: Diffusion-Based Robust Degradation Remover for Blind Face Restoration
Wang, Z., Zhang, Z., Zhang, X., Zheng, H., Zhou, M., Zhang, Y., & Wang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00170 · Cited by 174
Rethinking Low-Light Enhancement via Transformer-GAN
Yang, S., Zhou, D., Cao, J., & Guo, Y. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3167331 · Cited by 125 · cited 2×
“Therefore, following perceptual-driven approaches [22, 31], deep generative models such as generative adversarial networks (GANs) [19, 70] and variational autoencoders (VAEs) [79] have emerged as promising approaches for LLIE.”
BackgroundInfluentialRecurrent Exposure Generation for Low-Light Face Detection
Liang, J., Wang, J., Quan, Y., Chen, T., Liu, J., Ling, H., & Xu, Y. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3068840 · Cited by 107
ReLLIE: Deep Reinforcement Learning for Customized Low-Light Image Enhancement
Zhang, R., Guo, L., Huang, S., & Wen, B. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475410 · Cited by 91 · cited 3×
“In this section, we compare our proposed DiffLL with four categories of existing state-of-the-art methods, including 1) optimization-based LLIE methods NPE [Wang et al. 2013], SRIE [Fu et al. 2016], LIME [Guo et al. 2016], and CDEF [Lei et al. 2022], 2) learning-based LLIE methods RetinexNet [Wei et al. 2018], DSLR [L…”
MethodLearn From Unpaired Data for Image Restoration: A Variational Bayes Approach
Zheng, D., Zhang, X., Ma, K., & Bao, C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3215571 · Cited by 44
Perceptual Variousness Motion Deblurring with Light Global Context Refinement
Li, J., Tan, W., & Yan, B. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00408 · Cited by 36
Low-Light Image Enhancement Using the Cell Vibration Model
Lei, X., Fei, Z., Zhou, W., Zhou, H., & Fei, M. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3175634 · Cited by 32
A Visual Navigation System for UAV under Diverse Illumination Conditions
Jiang, H., Hao, Y., Zou, F., Lin, F., & Han, S. (2021) · Applied Artificial Intelligence · doi:10.1080/08839514.2021.1985799 · Cited by 29
Self-citationCombining Spatial and Frequency Information for Image Deblurring
Jiang, H., Ren, Y., Yu, Y., & Han, S. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3194807 · Cited by 25
Self-citation
Metadata
- Authors
- Jiang, H., Luo, A., Fan, H., Han, S., & Liu, S.
- Venue
- ACM Transactions on Graphics
- Published
- 2023-12-05
- Volume
- 42
- Issue
- 6
- Pages
- 1-14
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W4389347891
- Cited by
- 246
- References
- 61
- FWCI
- 22.37
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.