The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Limits on super-resolution and how to break them
Abstract
Nearly all super-resolution algorithms are based on the fundamental constraints that the super-resolution image should generate low resolution input images when appropriately warped and down-sampled to model the image formation process. (These reconstruction constraints are normally combined with some form of smoothness prior to regularize their solution.) We derive a sequence of analytical results which show that the reconstruction constraints provide less and less useful information as the magnification factor increases. We also validate these results empirically and show that, for large enough magnification factors, any smoothness prior leads to overly smooth results with very little high-frequency content. Next, we propose a super-resolution algorithm that uses a different kind of constraint in addition to the reconstruction constraints. The algorithm attempts to recognize local features in the low-resolution images and then enhances their resolution in an appropriate manner. We call such a super-resolution algorithm a hallucination or reconstruction algorithm. We tried our hallucination algorithm on two different data sets, frontal images of faces and printed Roman text. We obtained significantly better results than existing reconstruction-based algorithms, both qualitatively and in terms of RMS pixel error.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Principles of Optics: 60th Anniversary Edition
Born, M., & Wolf, E. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108769914 · Cited by 26,251
The Laplacian pyramid as a compact image code
Burt, P., & Adelson, E. (1983) · IEEE Transactions on Communications · doi:10.1109/tcom.1983.1095851 · Cited by 6,077
Neural network-based face detection
Rowley, H. A., Baluja, S., & Kanade, T. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.655647 · Cited by 3,554
The design and use of steerable filters
Freeman, W. T., & Adelson, E. H. (1991) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.93808 · Cited by 2,942
Improving resolution by image registration
Irani, M., & Peleg, S. (1991) · CVGIP: Graphical Models and Image Processing · doi:10.1016/1049-9652(91)90045-l · Cited by 1,929
Freeman, W. T., Pasztor, E. C., & Carmichael, O. T. (2000) · International Journal of Computer Vision · doi:10.1023/a:1026501619075 · Cited by 1,459
Limits on super-resolution and how to break them
Baker, S., & Kanade, T. (2002) · Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662) · doi:10.1109/cvpr.2000.854852 · Cited by 1,184
Self-citationHierarchical model-based motion estimation
Bergen, J. R., Anandan, P., Hanna, K. J., & Hingorani, R. (1992) · Lecture Notes in Computer Science · doi:10.1007/3-540-55426-2_27 · Cited by 1,154
Elad, M., & Feuer, A. (1997) · IEEE Transactions on Image Processing · doi:10.1109/83.650118 · Cited by 1,035
Extraction of high-resolution frames from video sequences
Schultz, R. R., & Stevenson, R. L. (1996) · IEEE Transactions on Image Processing · doi:10.1109/83.503915 · Cited by 978
Cited by
Image Super-Resolution Via Sparse Representation
Yang, J., Wright, J., Huang, T. S., & Ma, Y. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2050625 · Cited by 5,217
NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study
Agustsson, E., & Timofte, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.150 · Cited by 4,295
Super-resolution from a single image
Glasner, D., Bagon, S., & Irani, M. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459271 · Cited by 1,889
Sparse Representation for Color Image Restoration
Mairal, J., Elad, M., & Sapiro, G. (2008) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.911828 · Cited by 1,716
Image Alignment and Stitching: A Tutorial
Szeliski, R. (2007) · Foundations and Trends in Computer Graphics and Vision · doi:10.1561/0600000009 · Cited by 1,593
Image super-resolution as sparse representation of raw image patches
Yang, J., Wright, J., Huang, T., & Ma, Y. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587647 · Cited by 1,475
Learning a Single Convolutional Super-Resolution Network for Multiple Degradations
Zhang, K., Zuo, W., & Zhang, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00344 · Cited by 1,124
Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior
Kim, K. I., & Kwon, Y. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.25 · Cited by 956
Image super-resolution using gradient profile prior
Sun, J., Xu, Z., & Shum, H.-Y. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587659 · Cited by 916
Coupled Dictionary Training for Image Super-Resolution
Yang, J., Wang, Z., Lin, Z., Cohen, S., & Huang, T. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2192127 · Cited by 793
Deep Networks for Image Super-Resolution with Sparse Prior
Wang, Z., Liu, D., Yang, J., Han, W., & Huang, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.50 · Cited by 784
Text Detection and Recognition in Imagery: A Survey
Ye, Q., & Doermann, D. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2366765 · Cited by 780
Generalizing the Nonlocal-Means to Super-Resolution Reconstruction
Protter, M., Elad, M., Takeda, H., & Milanfar, P. (2009) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.2008067 · Cited by 707
Super-resolution: a comprehensive survey
Nasrollahi, K., & Moeslund, T. B. (2014) · Machine Vision and Applications · doi:10.1007/s00138-014-0623-4 · Cited by 684
Variational Light Field Analysis for Disparity Estimation and Super-Resolution
Wanner, S., & Goldluecke, B. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.147 · Cited by 615
Image super-resolution: the techniques, applications, and future
Yue, L., Shen, H., Li, J., Yuan, Q., Zhang, H., & Zhang, L. (2016) · Signal Processing · doi:10.1016/j.sigpro.2016.05.002 · Cited by 554
On Bayesian Adaptive Video Super Resolution
Liu, C., & Sun, D. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.127 · Cited by 526
Wave optics theory and 3-D deconvolution for the light field microscope
Broxton, M., Grosenick, L., Yang, S., Cohen, N., Andalman, A., Deisseroth, K., & Levoy, M. (2013) · Optics Express · doi:10.1364/oe.21.025418 · Cited by 516
Very Low Resolution Face Recognition Problem
Zou, W. W. W., & Yuen, P. C. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2162423 · Cited by 488
Hallucinating Face by Eigentransformation
Wang, X., & Tang, X. (2005) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2005.848171 · Cited by 482
Chatterjee, P., & Milanfar, P. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2037087 · Cited by 422
Camera-based analysis of text and documents: a survey
Liang, J., Doermann, D., & Li, H. (2005) · International Journal of Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-004-0138-z · Cited by 419
Rothganger, F., Lazebnik, S., Schmid, C., & Ponce, J. (2006) · International Journal of Computer Vision · doi:10.1007/s11263-005-3674-1 · Cited by 416
Deep Plug-And-Play Super-Resolution for Arbitrary Blur Kernels
Zhang, K., Zuo, W., & Zhang, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00177 · Cited by 408
The Light Field Camera: Extended Depth of Field, Aliasing, and Superresolution
Bishop, T. E., & Favaro, P. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.168 · Cited by 369
Super resolution using edge prior and single image detail synthesis
Tai, Y.-W., Liu, S., Brown, M. S., & Lin, S. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539933 · Cited by 354
Image up-sampling using total-variation regularization with a new observation model
Aly, H. A., & Dubois, E. (2005) · IEEE Transactions on Image Processing · doi:10.1109/tip.2005.851684 · Cited by 348
Tian, Y.-L., Kanade, T., & Cohn, J. F. (2005) · Handbook of Face Recognition · doi:10.1007/0-387-27257-7_12 · Cited by 347
Self-citationGradient Profile Prior and Its Applications in Image Super-Resolution and Enhancement
Sun, J., Sun, J., Xu, Z., & Shum, H.-Y. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2095871 · Cited by 341
A Comprehensive Survey to Face Hallucination
Wang, N., Tao, D., Gao, X., Li, X., & Li, J. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-013-0645-9 · Cited by 339
Real-World Super-Resolution via Kernel Estimation and Noise Injection
Ji, X., Cao, Y., Tai, Y., Wang, C., Li, J., & Huang, F. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00241 · Cited by 332
Hallucinating face by position-patch
Ma, X., Zhang, J., & Qi, C. (2010) · Pattern Recognition · doi:10.1016/j.patcog.2009.12.019 · Cited by 321
Ultra-Resolving Face Images by Discriminative Generative Networks
Yu, X., & Porikli, F. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_20 · Cited by 308
Iterative kernel principal component analysis for image modeling
Kwa, Franz, M. O., & Scholkopf, B. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.181 · Cited by 306
Single image super-resolution using Gaussian process regression
He, H., & Siu, W.-C. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995713 · Cited by 294
A survey on super-resolution imaging
Tian, J., & Ma, K.-K. (2011) · Signal, Image and Video Processing · doi:10.1007/s11760-010-0204-6 · Cited by 281
Video-based face recognition using adaptive hidden Markov models
Liu, X., & Cheng, T. (2003) · 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. · doi:10.1109/cvpr.2003.1211373 · Cited by 271
Robust Single Image Super-Resolution via Deep Networks With Sparse Prior
Liu, D., Wang, Z., Wen, B., Yang, J., Han, W., & Huang, T. S. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2564643 · Cited by 266
Soft Edge Smoothness Prior for Alpha Channel Super Resolution
Dai, S., Han, M., Xu, W., Wu, Y., & Gong, Y. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383028 · Cited by 266
Deep Network Cascade for Image Super-resolution
Cui, Z., Chang, H., Shan, S., Zhong, B., & Chen, X. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_4 · Cited by 255
Tian, Y., Kanade, T., & Cohn, J. F. (2011) · Handbook of Face Recognition · doi:10.1007/978-0-85729-932-1_19 · Cited by 255
Self-citationStudying Very Low Resolution Recognition Using Deep Networks
Wang, Z., Chang, S., Yang, Y., Liu, D., & Huang, T. S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.518 · Cited by 235
Özyurt, F., Sert, E., & Avcı, D. (2020) · Medical Hypotheses · doi:10.1016/j.mehy.2019.109433 · Cited by 210
Text Recognition in the Wild: A Survey
Chen, X., Jin, L., Zhu, Y., Luo, C., & Wang, T. (2022) · ACM Computing Surveys · doi:10.1145/3440756 · Cited by 205
Accurate Blur Models vs. Image Priors in Single Image Super-resolution
Efrat, N., Glasner, D., Apartsin, A., Nadler, B., & Levin, A. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.352 · Cited by 200
Grotte, M. E., Birkeland, R., Honore-Livermore, E., Bakken, S., Garrett, J. L., Prentice, E. F., Sigernes, F., Orlandic, M., Gravdahl, J. T., & Johansen, T. A. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3080175 · Cited by 91
Vision function testing for a suprachoroidal retinal prosthesis: effects of image filtering
Barnes, N., Scott, A. F., Lieby, P., Petoe, M. A., McCarthy, C., Stacey, A., Ayton, L. N., Sinclair, N. C., Shivdasani, M. N., Lovell, N. H., et al. (2016) · Journal of Neural Engineering · doi:10.1088/1741-2560/13/3/036013 · Cited by 46
Image super-resolution with densely connected convolutional networks
Kuang, P., Ma, T., Chen, Z., & Li, F. (2019) · Applied Intelligence · doi:10.1007/s10489-018-1234-y · Cited by 33
Metadata
- Authors
- Baker, S., & Kanade, T.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2002-09
- Volume
- 24
- Issue
- 9
- Pages
- 1167-1183
- ISSN
- 0162-8828 (ISSN-L)
- OpenAlex
- W2140257560
- Cited by
- 1,205
- References
- 30
- FWCI
- 35.20
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.