The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index.
The Perception-Distortion Tradeoff
Open accessAbstract
Image restoration algorithms are typically evaluated by some distortion measure (e.g. PSNR, SSIM, IFC, VIF) or by human opinion scores that quantify perceived perceptual quality. In this paper, we prove mathematically that distortion and perceptual quality are at odds with each other. Specifically, we study the optimal probability for correctly discriminating the outputs of an image restoration algorithm from real images. We show that as the mean distortion decreases, this probability must increase (indicating worse perceptual quality). As opposed to the common belief, this result holds true for any distortion measure, and is not only a problem of the PSNR or SSIM criteria. However, as we show experimentally, for some measures it is less severe (e.g. distance between VGG features). We also show that generative-adversarial-nets (GANs) provide a principled way to approach the perception-distortion bound. This constitutes theoretical support to their observed success in low-level vision tasks. Based on our analysis, we propose a new methodology for evaluating image restoration methods, and use it to perform an extensive comparison between recent super-resolution algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 36,808
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
Cited by
ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Qiao, Y., & Loy, C. C. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-11021-5_5 · Cited by 4,731
Deep Learning for Single Image Super-Resolution: A Brief Review
Yang, W., Zhang, X., Tian, Y., Wang, W., Xue, J.-H., & Liao, Q. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2919431 · Cited by 1,154
SinGAN: Learning a Generative Model From a Single Natural Image
Shaham, T. R., Dekel, T., & Michaeli, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00467 · Cited by 890
Self-citationDesigning a Practical Degradation Model for Deep Blind Image Super-Resolution
Zhang, K., Liang, J., Van Gool, L., & Timofte, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00475 · Cited by 760
Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation
Li, C., Guo, C., & Chen, C. L. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3063604 · Cited by 719
Designing an encoder for StyleGAN image manipulation
Tov, O., Alaluf, Y., Nitzan, Y., Patashnik, O., & Cohen-Or, D. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459838 · Cited by 686
Low-Light Image and Video Enhancement Using Deep Learning: A Survey
Li, C., Guo, C., Han, L., Jiang, J., Cheng, M.-M., Gu, J., & Loy, C. C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3126387 · Cited by 657
The 2018 PIRM Challenge on Perceptual Image Super-Resolution
Blau, Y., Mechrez, R., Timofte, R., Michaeli, T., & Zelnik-Manor, L. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-11021-5_21 · Cited by 593
Self-citationMANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment
Yang, S., Wu, T., Shi, S., Lao, S., Gong, Y., Cao, M., Wang, J., & Yang, Y. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00126 · Cited by 527
RankSRGAN: Generative Adversarial Networks With Ranker for Image Super-Resolution
Zhang, W., Liu, Y., Dong, C., & Qiao, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00319 · Cited by 405
SRFlow: Learning the Super-Resolution Space with Normalizing Flow
Lugmayr, A., Danelljan, M., Van Gool, L., & Timofte, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58558-7_42 · Cited by 347
Toward Semantic Communications: Deep Learning-Based Image Semantic Coding
Huang, D., Gao, F., Tao, X., Du, Q., & Lu, J. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3221999 · Cited by 331
Image super-resolution: A comprehensive review, recent trends, challenges and applications
Lepcha, D. C., Goyal, B., Dogra, A., & Goyal, V. (2023) · Information Fusion · doi:10.1016/j.inffus.2022.10.007 · Cited by 305
The Devil Is in the Details: Window-based Attention for Image Compression
Zou, R., Song, C., & Zhang, Z. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01697 · Cited by 300
Adversarial super-resolution of climatological wind and solar data
Stengel, K., Glaws, A., Hettinger, D., & King, R. N. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1918964117 · Cited by 287
Deep Convolutional Neural Network for Multi-Modal Image Restoration and Fusion
Deng, X., & Dragotti, P. L. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2984244 · Cited by 282
Unsupervised Single Image Dehazing Using Dark Channel Prior Loss
Golts, A., Freedman, D., & Elad, M. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2952032 · Cited by 264
Deblurring via Stochastic Refinement
Whang, J., Delbracio, M., Talebi, H., Saharia, C., Dimakis, A. G., & Milanfar, P. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01581 · Cited by 259
Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
Liu, M.-Y., Huang, X., Yu, J., Wang, T.-C., & Mallya, A. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3049196 · Cited by 236
Large Scale Image Completion via Co-Modulated Generative Adversarial Networks
Zhao, S., Cui, J., Sheng, Y., Dong, Y., Liang, X., Chang, E. I., & Xu, Y. (2021) · arXiv · doi:10.48550/arxiv.2103.10428 · Cited by 224
Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff
Blau, Y., & Michaeli, T. (2019) · arXiv · doi:10.48550/arxiv.1901.07821 · Cited by 198
Self-citationDeep Learning-Based Video Coding: A Review and a Case Study
Liu, D., Li, Y., Lin, J., Li, H., & Wu, F. (2021) · ACM Computing Surveys · doi:10.1145/3368405 · Cited by 192
Face Generation and Editing With StyleGAN: A Survey
Melnik, A., Miasayedzenkau, M., Makaravets, D., Pirshtuk, D., Akbulut, E., Holzmann, D., Renusch, T., Reichert, G., & Ritter, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3350004 · Cited by 105
Stochastic Image Denoising by Sampling from the Posterior Distribution
Kawar, B., Vaksman, G., & Elad, M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw54120.2021.00213 · Cited by 42
$$\hbox {S}^2\hbox {RGAN}$$: sonar-image super-resolution based on generative adversarial network
Song, H., Wang, M., Zhang, L., Li, Y., Jiang, Z., & Yin, G. (2021) · The Visual Computer · doi:10.1007/s00371-020-01986-3 · Cited by 23
GAN-based Defect Synthesis for Anomaly Detection in Fabrics
Rippel, O., Muller, M., & Merhof, D. (2020) · 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) · doi:10.1109/etfa46521.2020.9212099 · Cited by 22
Metadata
- Authors
- Blau, Y., & Michaeli, T.
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 6228-6237
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2768814045
- Cited by
- 955
- References
- 72
- FWCI
- 24.00
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.