The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Deep bilateral learning for real-time image enhancement
Open accessAbstract
Performance is a critical challenge in mobile image processing. Given a reference imaging pipeline, or even human-adjusted pairs of images, we seek to reproduce the enhancements and enable real-time evaluation. For this, we introduce a new neural network architecture inspired by bilateral grid processing and local affine color transforms. Using pairs of input/output images, we train a convolutional neural network to predict the coefficients of a locally-affine model in bilateral space. Our architecture learns to make local, global, and content-dependent decisions to approximate the desired image transformation. At runtime, the neural network consumes a low-resolution version of the input image, produces a set of affine transformations in bilateral space, upsamples those transformations in an edge-preserving fashion using a new slicing node, and then applies those upsampled transformations to the full-resolution image. Our algorithm processes high-resolution images on a smartphone in milliseconds, provides a real-time viewfinder at 1080p resolution, and matches the quality of state-of-the-art approximation techniques on a large class of image operators. Unlike previous work, our model is trained off-line from data and therefore does not require access to the original operator at runtime. This allows our model to learn complex, scene-dependent transformations for which no reference implementation is available, such as the photographic edits of a human retoucher.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · cited 6×
“We compare the speed and quality of our algorithm against two modern network architectures: U-Net (adapted from [Isola et al. 2016]) and dilated convolutions [Yu and Koltun 2015].”
BackgroundMethodInfluentialDelving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662206 · Cited by 8,983
Bilateral Filtering for Gray and Color Images
Tomasi, C., & Manduchi, R. (2002) · Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271) · doi:10.1109/iccv.1998.710815 · Cited by 8,177 · cited 1×
“Similarly, many approaches have been proposed to accelerate bilateral filtering, due to the ubiquity of edge-aware image processing [Adams et al. 2010; Chen et al. 2007; Paris and Durand 2006; Tomasi and Manduchi 1998].”
BackgroundMulti-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738 · cited 2×
“far from real-time. In Figure 10, we compare our technique against a U-Net architecture [Ronneberger et al. 2015] adapted from Isola et al. [2016], and a linear network based on dilated convolutions [Yu and Koltun 2015]. We explore several settings for the depth (number of layers, 3 to 11) and the width (number of fil…”
Method
Cited by
EnlightenGAN: Deep Light Enhancement Without Paired Supervision
Jiang, Y., Gong, X., Liu, D., Cheng, Y., Fang, C., Shen, X., Yang, J., Zhou, P., & Wang, Z. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3051462 · Cited by 2,715
Learning a Deep Single Image Contrast Enhancer from Multi-Exposure Images
Cai, J., Gu, S., & Zhang, L. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2794218 · Cited by 1,234
A Deep Learning System to Screen Novel Coronavirus Disease 2019 Pneumonia
Xu, X., Jiang, X., Ma, C., Du, P., Li, X., Lv, S., Yu, L., Ni, Q., Chen, Y., Su, J., et al. (2020) · Engineering · doi:10.1016/j.eng.2020.04.010 · Cited by 1,209
Underexposed Photo Enhancement Using Deep Illumination Estimation
Wang, R., Zhang, Q., Fu, C.-W., Shen, X., Zheng, W.-S., & Jia, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00701 · Cited by 983
Learning Enriched Features for Real Image Restoration and Enhancement
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58595-2_30 · Cited by 946
Low-Light Image and Video Enhancement Using Deep Learning: A Survey
Li, C., Guo, C., Han, L., Jiang, J., Cheng, M.-M., Gu, J., & Loy, C. C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3126387 · Cited by 657
Rivenson, Y., Göröcs, Z., Günaydin, H., Zhang, Y., Wang, H., & Ozcan, A. (2017) · Optica · doi:10.1364/optica.4.001437 · Cited by 643
Learning Enriched Features for Fast Image Restoration and Enhancement
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3167175 · Cited by 531
Low-Light Image Enhancement via a Deep Hybrid Network
Ren, W., Liu, S., Ma, L., Xu, Q., Xu, X., Cao, X., Du, J., & Yang, M.-H. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2910412 · Cited by 508
LightenNet: A Convolutional Neural Network for weakly illuminated image enhancement
Li, C., Guo, J., Porikli, F., & Pang, Y. (2018) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.01.010 · Cited by 506
Deep Photo Enhancer: Unpaired Learning for Image Enhancement from Photographs with GANs
Chen, Y.-S., Wang, Y.-C., Kao, M.-H., & Chuang, Y.-Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00660 · Cited by 467
Learning to Restore Low-Light Images via Decomposition-and-Enhancement
Xu, K., Yang, X., Yin, B., & Lau, R. W. H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00235 · Cited by 449
Learning Blind Video Temporal Consistency
Lai, W.-S., Huang, J.-B., Wang, O., Shechtman, E., Yumer, E., & Yang, M.-H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_11 · Cited by 405
StereoNet: Guided Hierarchical Refinement for Real-Time Edge-Aware Depth Prediction
Khamis, S., Fanello, S., Rhemann, C., Kowdle, A., Valentin, J., & Izadi, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_35 · Cited by 401
An Experiment-Based Review of Low-Light Image Enhancement Methods
Wang, W., Wu, X., Yuan, X., & Gao, Z. (2020) · IEEE Access · doi:10.1109/access.2020.2992749 · Cited by 364
NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images
Mildenhall, B., Hedman, P., Martin-Brualla, R., Srinivasan, P. P., & Barron, J. T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01571 · Cited by 349
Self-citationAttention Guided Low-Light Image Enhancement with a Large Scale Low-Light Simulation Dataset
Lv, F., Li, Y., & Lu, F. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01466-8 · Cited by 319
DSLR: Deep Stacked Laplacian Restorer for Low-Light Image Enhancement
Lim, S., & Kim, W. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3039361 · Cited by 295
Learning Guided Convolutional Network for Depth Completion
Tang, J., Tian, F.-P., Feng, W., Li, J., & Tan, P. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3040528 · Cited by 291
Exposure: A White-Box Photo Post-Processing Framework
Hu, Y., He, H., Xu, C., Wang, B., & Lin, S. (2018) · ACM Transactions on Graphics · doi:10.1145/3181974 · Cited by 279
Fast End-to-End Trainable Guided Filter
Wu, H., Zheng, S., Zhang, J., & Huang, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00197 · Cited by 278
DeepLPF: Deep Local Parametric Filters for Image Enhancement
Moran, S., Marza, P., McDonagh, S., Parisot, S., & Slabaugh, G. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01284 · Cited by 278
Learning Image-adaptive 3D Lookup Tables for High Performance Photo Enhancement in Real-time
Zeng, H., Cai, J., Li, L., Cao, Z., & Zhang, L. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3026740 · Cited by 271
DeepISP: Toward Learning an End-to-End Image Processing Pipeline
Schwartz, E., Giryes, R., & Bronstein, A. M. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2872858 · Cited by 265
Ultra-High-Definition Image Dehazing via Multi-Guided Bilateral Learning
Zheng, Z., Ren, W., Cao, X., Hu, X., Wang, T., Song, F., & Jia, X. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01592 · Cited by 262
Deep reinforcement learning in computer vision: a comprehensive survey
Le, N., Rathour, V. S., Yamazaki, K., Luu, K., & Savvides, M. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10061-9 · Cited by 261
Single image portrait relighting
Sun, T., Barron, J. T., Tsai, Y.-T., Xu, Z., Yu, X., Fyffe, G., Rhemann, C., Busch, J., Debevec, P., & Ramamoorthi, R. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323008 · Cited by 259
Self-citationSemantic photo manipulation with a generative image prior
Bau, D., Strobelt, H., Peebles, W., Wulff, J., Zhou, B., Zhu, J.-Y., & Torralba, A. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323023 · Cited by 207
TreEnhance: A tree search method for low-light image enhancement
Cotogni, M., & Cusano, C. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2022.109249 · Cited by 40
ISP Meets Deep Learning: A Survey on Deep Learning Methods for Image Signal Processing
Santos, C. F. G. D., Arrais, R. R., Silva, J. V. S. D., Silva, M. H. M. D., Neto, W. B. G. D. A., Lopes, L. T., Bileki, G. A., Lima, I. O., Rondon, L. B., Souza, B. M. D., et al. (2025) · ACM Computing Surveys · doi:10.1145/3708516 · Cited by 15
“In the second, results were validated over the MIT-Adobe FiveK dataset [11] and compared with HDRNet [31], DPE [16], White-Box [41], Distort-and-Recover [84], and DeepUPE [117] methods where CURL scored better on PSNR and LPIPS metrics, but DeepUPE scored the best on SSIM.”
Metadata
- Authors
- Gharbi, M., Chen, J., Barron, J. T., Hasinoff, S. W., & Durand, F.
- Venue
- ACM Transactions on Graphics
- Published
- 2017-07-20
- Volume
- 36
- Issue
- 4
- Pages
- 1-12
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W2735974062
- Cited by
- 800
- References
- 47
- FWCI
- 21.90
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.