The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index.
A Deep Journey into Super-resolution: A Survey
Abstract
Deep convolutional networks–based super-resolution is a fast-growing field with numerous practical applications. In this exposition, we extensively compare more than 30 state-of-the-art super-resolution Convolutional Neural Networks (CNNs) over three classical and three recently introduced challenging datasets to benchmark single image super-resolution. We introduce a taxonomy for deep learning–based super-resolution networks that groups existing methods into nine categories including linear, residual, multi-branch, recursive, progressive, attention-based, and adversarial designs. We also provide comparisons between the models in terms of network complexity, memory footprint, model input and output, learning details, the type of network losses, and important architectural differences (e.g., depth, skip-connections, filters). The extensive evaluation performed shows the consistent and rapid growth in the accuracy in the past few years along with a corresponding boost in model complexity and the availability of large-scale datasets. It is also observed that the pioneering methods identified as the benchmarks have been significantly outperformed by the current contenders. Despite the progress in recent years, we identify several shortcomings of existing techniques and provide future research directions towards the solution of these open problems. Datasets and codes for evaluation are publicly available at https://github.com/saeed-anwar/SRsurvey.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,667
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
Image Super-Resolution Using Deep Convolutional Networks
Dong, C., Loy, C. C., He, K., & Tang, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2439281 · Cited by 10,105
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662206 · Cited by 8,983
Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937655 · Cited by 8,216
Accurate Image Super-Resolution Using Very Deep Convolutional Networks
Kim, J., Lee, J. K., & Lee, K. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.182 · Cited by 7,870
Enhanced Deep Residual Networks for Single Image Super-Resolution
Lim, B., Son, S., Kim, H., Nah, S., & Lee, K. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.151 · Cited by 7,736
Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.207 · Cited by 7,570
Making a “Completely Blind” Image Quality Analyzer
Mittal, A., Soundararajan, R., & Bovik, A. C. (2013) · IEEE Signal Processing Letters · doi:10.1109/lsp.2012.2227726 · Cited by 6,766
No-Reference Image Quality Assessment in the Spatial Domain
Mittal, A., Moorthy, A. K., & Bovik, A. C. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2214050 · Cited by 6,064
Image Super-Resolution Using Very Deep Residual Channel Attention Networks
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_18 · Cited by 5,708
Learning a Deep Convolutional Network for Image Super-Resolution
Dong, C., Loy, C. C., He, K., & Tang, X. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10593-2_13 · Cited by 5,494
Image Super-Resolution Via Sparse Representation
Yang, J., Wright, J., Huang, T. S., & Ma, Y. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2050625 · Cited by 5,217
Remote Sensing And Image Interpretation
LILLESAND, T. M., Kiefer, R. W., & Chipman, J. (1979) · HAL · Cited by 4,463
Residual Dense Network for Image Super-Resolution
Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00262 · Cited by 4,168
Accelerating the Super-Resolution Convolutional Neural Network
Dong, C., Loy, C. C., & Tang, X. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_25 · Cited by 3,786
Single image super-resolution from transformed self-exemplars
Huang, J.-B., Singh, A., & Ahuja, N. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299156 · Cited by 3,472
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011
Region-based convolutional networks for accurate object detection and segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2437384 · Cited by 2,912
Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding
Bevilacqua, M., Roumy, A., Guillemot, C., & Morel, M.-L. A. (2012) · Procedings of the British Machine Vision Conference 2012 · doi:10.5244/c.26.135 · Cited by 2,701
Example-based super-resolution
Freeman, W. T., Jones, T. R., & Pasztor, E. C. (2002) · IEEE Computer Graphics and Applications · doi:10.1109/38.988747 · Cited by 2,498
Image Super-Resolution via Deep Recursive Residual Network
Tai, Y., Yang, J., & Liu, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.298 · Cited by 2,415
Learning Deep CNN Denoiser Prior for Image Restoration
Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.300 · Cited by 2,116 · cited 5×
“The architecture of SRMD is similar to [28], [30], [31].”
BackgroundResultInfluentialImproving resolution by image registration
Irani, M., & Peleg, S. (1991) · CVGIP: Graphical Models and Image Processing · doi:10.1016/1049-9652(91)90045-l · Cited by 1,929
Super-resolution through neighbor embedding
Chang, H., Yeung, D.-Y., & Xiong, Y. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315043 · Cited by 1,902
MemNet: A Persistent Memory Network for Image Restoration
Tai, Y., Yang, J., Liu, X., & Xu, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.486 · Cited by 1,883
Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., et al. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01316-z · Cited by 1,738
NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results
Timofte, R., Agustsson, E., Gool, L. V., Yang, M.-H., Zhang, L., Lim, B., Son, S., Kim, H., Nah, S., Lee, K. M., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.149 · Cited by 1,441
An information fidelity criterion for image quality assessment using natural scene statistics
Sheikh, H. R., Bovik, A. C., & de Veciana, G. (2005) · IEEE Transactions on Image Processing · doi:10.1109/tip.2005.859389 · Cited by 1,399
Image Super-Resolution Using Dense Skip Connections
Tong, T., Li, G., Liu, X., & Gao, Q. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.514 · Cited by 1,373
Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network
Ahn, N., Kang, B., & Sohn, K.-A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_16 · Cited by 1,365
Edvr: video restoration with enhanced deformable convolutional networks
Wang, X., Chan, K. C. K., Yu, K., Dong, C., & Loy, C. C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00247 · Cited by 1,272
Underwater scene prior inspired deep underwater image and video enhancement
Li, C., Anwar, S., & Porikli, F. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2019.107038 · Cited by 1,252
Self-citationLearning a Single Convolutional Super-Resolution Network for Multiple Degradations
Zhang, K., Zuo, W., & Zhang, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00344 · Cited by 1,124
EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis
Sajjadi, M. S. M., Scholkopf, B., & Hirsch, M. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.481 · Cited by 1,058
Nonlinear image recovery with half-quadratic regularization
Geman, D., & Yang, C. (1995) · IEEE Transactions on Image Processing · doi:10.1109/83.392335 · Cited by 1,000
Fast and Accurate Single Image Super-Resolution via Information Distillation Network
Hui, Z., Wang, X., & Gao, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00082 · Cited by 955
Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks
Lai, W.-S., Huang, J.-B., Ahuja, N., & Yang, M.-H. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2865304 · Cited by 953
Feedback Network for Image Super-Resolution
Li, Z., Yang, J., Liu, Z., Yang, X., Jeon, G., & Wu, W. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00399 · Cited by 883
Waterloo Exploration Database: New Challenges for Image Quality Assessment Models
Ma, K., Duanmu, Z., Wu, Q., Wang, Z., Yong, H., Li, H., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2631888 · Cited by 789
Deep Networks for Image Super-Resolution with Sparse Prior
Wang, Z., Liu, D., Yang, J., Han, W., & Huang, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.50 · Cited by 784
Single-Image Super-Resolution: A Benchmark
Yang, C.-Y., Ma, C., & Yang, M.-H. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10593-2_25 · Cited by 615
Learning a no-reference quality metric for single-image super-resolution
Ma, C., Yang, C.-Y., Yang, X., & Yang, M.-H. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2016.12.009 · Cited by 605
Unsupervised Image Super-Resolution Using Cycle-in-Cycle Generative Adversarial Networks
Yuan, Y., Liu, S., Zhang, J., Zhang, Y., Dong, C., & Lin, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00113 · Cited by 594
Jo, Y., Oh, S. W., Kang, J., & Kim, S. J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00340 · Cited by 590
Super-Resolution in Medical Imaging
Greenspan, H. (2008) · The Computer Journal · doi:10.1093/comjnl/bxm075 · Cited by 564
SOD-MTGAN: Small Object Detection via Multi-Task Generative Adversarial Network
Bai, Y., Zhang, Y., Ding, M., & Ghanem, B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_13 · Cited by 487
Image upsampling via imposed edge statistics
Fattal, R. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276496 · Cited by 416
To Learn Image Super-Resolution, Use a GAN to Learn How to Do Image Degradation First
Bulat, A., Yang, J., & Tzimiropoulos, G. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_12 · Cited by 407
NTIRE 2018 Challenge on Single Image Super-Resolution: Methods and Results
Timofte, R., Gu, S., Wu, J., Van Gool, L., Zhang, L., Yang, M.-H., Haris, M., Shakhnarovich, G., Ukita, N., Hu, S., et al. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00130 · Cited by 341
PieAPP: Perceptual Image-Error Assessment Through Pairwise Preference
Prashnani, E., Cai, H., Mostofi, Y., & Sen, P. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00194 · Cited by 339
Fast Image Super-Resolution Based on In-Place Example Regression
Yang, J., Lin, Z., & Cohen, S. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.141 · Cited by 327
Natural Language Processing (Almost) from Scratch
CollobertRonan, WestonJason, BottouLéon, KarlenMichael, KavukcuogluKoray, & KuksaPavel (2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2078186 · Cited by 306
Digital image forensics via intrinsic fingerprints
Swaminathan, A., Wu, M., & Liu, K. J. R. (2008) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2007.916010 · Cited by 292
SRFeat: Single Image Super-Resolution with Feature Discrimination
Park, S.-J., Son, H., Cho, S., Hong, K.-S., & Lee, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_27 · Cited by 205
Fast Spatio-Temporal Residual Network for Video Super-Resolution
Li, S., He, F., Du, B., Zhang, L., Xu, Y., & Tao, D. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01077 · Cited by 188
Low Resolution Face Recognition Across Variations in Pose and Illumination
Mudunuri, S. P., & Biswas, S. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2469282 · Cited by 162
Embedded Block Residual Network: A Recursive Restoration Model for Single-Image Super-Resolution
Qiu, Y., Wang, R., Tao, D., & Cheng, J. S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00428 · Cited by 145
NTIRE 2019 Challenge on Real Image Super-Resolution: Methods and Results
Cai, J., Gu, S., Timofte, R., Zhang, L., Liu, X., Ding, Y., He, D., Li, C., Fu, Y., Wen, S., et al. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00274 · Cited by 97
Self-citationImage and Video Super Resolution using Recurrent Generative Adversarial Network
Thawakar, O., Patil, P. W., Dudhane, A. A., Murala, S., & Kulkarni, U. (2019) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2019.8909900 · Cited by 24
Cited by
Multi-Stage Progressive Image Restoration
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01458 · Cited by 2,095
Self-citationLearning Enriched Features for Real Image Restoration and Enhancement
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58595-2_30 · Cited by 946
Self-citationLearning Enriched Features for Fast Image Restoration and Enhancement
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3167175 · Cited by 531
Self-citationA Survey on Wearable Technology: History, State-of-the-Art and Current Challenges
Ometov, A., Shubina, V., Klus, L., Skibińska, J., Saafi, S., Pascacio, P., Flueratoru, L., Gaibor, D. Q., Chukhno, N., Chukhno, O., et al. (2021) · Computer Networks · doi:10.1016/j.comnet.2021.108074 · Cited by 506
Real-world single image super-resolution: A brief review
Chen, H., He, X., Qing, L., Wu, Y., Ren, C., Sheriff, R. E., & Zhu, C. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.09.005 · Cited by 405
Metadata
- Authors
- Anwar, S., Khan, S., & Barnes, N.
- Venue
- ACM Computing Surveys
- Published
- 2020-05-28
- Volume
- 53
- Issue
- 3
- Pages
- 1-34
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W3040726448
- Cited by
- 394
- References
- 72
- FWCI
- 19.37
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.