The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Learning Image-adaptive 3D Lookup Tables for High Performance Photo Enhancement in Real-time
Open accessAbstract
Recent years have witnessed the increasing popularity of learning based methods to enhance the color and tone of photos. However, many existing photo enhancement methods either deliver unsatisfactory results or consume too much computational and memory resources, hindering their application to high-resolution images (usually with more than 12 megapixels) in practice. In this paper, we learn image-adaptive 3-dimensional lookup tables (3D LUTs) to achieve fast and robust photo enhancement. 3D LUTs are widely used for manipulating color and tone of photos, but they are usually manually tuned and fixed in camera imaging pipeline or photo editing tools. We, for the first time to our best knowledge, propose to learn 3D LUTs from annotated data using pairwise or unpaired learning. More importantly, our learned 3D LUT is image-adaptive for flexible photo enhancement. We learn multiple basis 3D LUTs and a small convolutional neural network (CNN) simultaneously in an end-to-end manner. The small CNN works on the down-sampled version of the input image to predict content-dependent weights to fuse the multiple basis 3D LUTs into an image-adaptive one, which is employed to transform the color and tone of source images efficiently. Our model contains less than 600K parameters and takes less than 2 ms to process an image of 4K resolution using one Titan RTX GPU. While being highly efficient, our model also outperforms the state-of-the-art photo enhancement methods by a large margin in terms of PSNR, SSIM and a color difference metric on two publically available benchmark datasets. Code will be released at https://github.com/HuiZeng/Image-Adaptive-3DLUT.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Nonlinear total variation based noise removal algorithms
Rudin, L. I., Osher, S., & Fatemi, E. (1992) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(92)90242-f · Cited by 15,704
(2010) · Cited by 14,134
(2017) · Cited by 11,625
Improved training of Wasserstein GANs
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (2017) · arXiv · doi:10.48550/arxiv.1704.00028 · Cited by 6,800
Image Style Transfer Using Convolutional Neural Networks
Gatys, L. A., Ecker, A. S., & Bethge, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.265 · Cited by 6,025
He, K., Sun, J., & Tang, X. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.213 · Cited by 5,546
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization
Huang, X., & Belongie, S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.167 · Cited by 5,076
Reinhard, E., Adhikhmin, M., Gooch, B., & Shirley, P. (2001) · IEEE Computer Graphics and Applications · doi:10.1109/38.946629 · Cited by 2,937
Instance Normalization: The Missing Ingredient for Fast Stylization
Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.08022 · Cited by 2,775
Empirical Evaluation of Rectified Activations in Convolutional Network
Xu, B., Wang, N., Chen, T., & Li, M. (2015) · arXiv · doi:10.48550/arxiv.1505.00853 · Cited by 2,264
Contrast enhancement using brightness preserving bi-histogram equalization
Kim, Y.-T. (1997) · IEEE Transactions on Consumer Electronics · doi:10.1109/30.580378 · Cited by 1,586
Fast bilateral filtering for the display of high-dynamic-range images
Durand, F., & Dorsey, J. (2002) · Proceedings of the 29th annual conference on Computer graphics and interactive techniques · doi:10.1145/566570.566574 · Cited by 1,547
Learning a Deep Single Image Contrast Enhancer from Multi-Exposure Images
Cai, J., Gu, S., & Zhang, L. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2794218 · Cited by 1,234
Self-citationUnderexposed Photo Enhancement Using Deep Illumination Estimation
Wang, R., Zhang, Q., Fu, C.-W., Shen, X., Zheng, W.-S., & Jia, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00701 · Cited by 983
Deep bilateral learning for real-time image enhancement
Gharbi, M., Chen, J., Barron, J. T., Hasinoff, S. W., & Durand, F. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073592 · Cited by 800
Luan, F., Paris, S., Shechtman, E., & Bala, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.740 · Cited by 714
Shades of Gray and Colour Constancy
Finlayson, G. D., & Trezzi, E. (2004) · Color and Imaging Conference · doi:10.2352/cic.2004.12.1.art00008 · Cited by 614
Computational Color Constancy: Survey and Experiments
Gijsenij, A., Gevers, T., & van de Weijer, J. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2118224 · Cited by 562
Learning photographic global tonal adjustment with a database of input/output image pairs
Bychkovsky, V., Paris, S., Chan, E., & Durand, F. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995413 · Cited by 536
Learning photographic global tonal adjustment with a database of input / output image pairs
Bychkovsky, V., Paris, S., Chan, E., & Durand, F. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995332 · Cited by 529
Burst photography for high dynamic range and low-light imaging on mobile cameras
Hasinoff, S. W., Sharlet, D., Geiss, R., Adams, A., Barron, J. T., Kainz, F., Chen, J., & Levoy, M. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2980254 · Cited by 483
Deep Photo Enhancer: Unpaired Learning for Image Enhancement from Photographs with GANs
Chen, Y.-S., Wang, Y.-C., Kao, M.-H., & Chuang, Y.-Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00660 · Cited by 467
Mantiuk, R., Daly, S., & Kerofsky, L. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360667 · Cited by 333
Exposure: A White-Box Photo Post-Processing Framework
Hu, Y., He, H., Xu, C., Wang, B., & Lin, S. (2018) · ACM Transactions on Graphics · doi:10.1145/3181974 · Cited by 279
Automatic Photo Adjustment Using Deep Neural Networks
Yan, Z., Zhang, H., Wang, B., Paris, S., & Yu, Y. (2016) · ACM Transactions on Graphics · doi:10.1145/2790296 · Cited by 271
DeepISP: Toward Learning an End-to-End Image Processing Pipeline
Schwartz, E., Giryes, R., & Bronstein, A. M. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2872858 · Cited by 265
Distort-and-Recover: Color Enhancement Using Deep Reinforcement Learning
Park, J., Lee, J.-Y., Yoo, D., & Kweon, I. S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00621 · Cited by 211
Automatic Exposure Correction of Consumer Photographs
Yuan, L., & Sun, J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_55 · Cited by 168
A New In-Camera Imaging Model for Color Computer Vision and Its Application
Kim, S. J., Lin, H. T., Lu, Z., Süsstrunk, S., Lin, S., & Brown, M. S. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.58 · Cited by 167
A Software Platform for Manipulating the Camera Imaging Pipeline
Karaimer, H. C., & Brown, M. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_26 · Cited by 166
Color Vision: Perspectives from Different Disciplines
(1998) · De Gruyter eBooks · doi:10.1515/9783110806984 · Cited by 165
Aesthetic-Driven Image Enhancement by Adversarial Learning
Deng, Y., Loy, C. C., & Tang, X. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240531 · Cited by 149
CameraNet: A Two-Stage Framework for Effective Camera ISP Learning
Liang, Z., Cai, J., Cao, Z., & Zhang, L. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3051486 · Cited by 116
Self-citationUnpaired Image Enhancement Featuring Reinforcement-Learning-Controlled Image Editing Software
Kosugi, S., & Yamasaki, T. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6790 · Cited by 114
Personalization of image enhancement
Kang, S. B., Kapoor, A., & Lischinski, D. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5539850 · Cited by 87
A Learning-to-Rank Approach for Image Color Enhancement
Yan, J., Lin, S. C.-F., Kang, S. B., & Tang, X. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.382 · Cited by 82
Automatic Content-Aware Color and Tone Stylization
Lee, J., Sunkavalli, K., Lin, Z., Shen, X., & Kweon, I. S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.271 · Cited by 64
AutoStyle: Automatic Style Transfer from Image Collections to Users' Images
Liu, Y., Cohen, M., Uyttendaele, M., & Rusinkiewicz, S. M. (2014) · Computer Graphics Forum · doi:10.1111/cgf.12409 · Cited by 34
Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network
Nam, S., & Kim, S. J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.190 · Cited by 24
Cited by
Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement
Cai, Y., Bian, H., Lin, J., Wang, H., Timofte, R., & Zhang, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01149 · Cited by 896
SNR-Aware Low-light Image Enhancement
Xu, X., Wang, R., Fu, C.-W., & Jia, J. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01719 · Cited by 688
Low-Light Image and Video Enhancement Using Deep Learning: A Survey
Li, C., Guo, C., Han, L., Jiang, J., Cheng, M.-M., Gu, J., & Loy, C. C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3126387 · Cited by 657
Image-Adaptive YOLO for Object Detection in Adverse Weather Conditions
Liu, W., Ren, G., Yu, R., Guo, S., Zhu, J., & Zhang, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20072 · Cited by 604
Self-citationUltra-High-Definition Image Dehazing via Multi-Guided Bilateral Learning
Zheng, Z., Ren, W., Cao, X., Hu, X., Wang, T., Song, F., & Jia, X. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01592 · Cited by 262
Metadata
- Authors
- Zeng, H., Cai, J., Li, L., Cao, Z., & Zhang, L.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2020
- Volume
- 44
- Issue
- 4
- Pages
- 1-1
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W3088065502
- Cited by
- 271
- References
- 46
- FWCI
- 10.81
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.