The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Super-Resolution Without Explicit Subpixel Motion Estimation
Abstract
The need for precise (subpixel accuracy) motion estimates in conventional super-resolution has limited its applicability to only video sequences with relatively simple motions such as global translational or affine displacements. In this paper, we introduce a novel framework for adaptive enhancement and spatiotemporal upscaling of videos containing complex activities without explicit need for accurate motion estimation. Our approach is based on multidimensional kernel regression, where each pixel in the video sequence is approximated with a 3-D local (Taylor) series, capturing the essential local behavior of its spatiotemporal neighborhood. The coefficients of this series are estimated by solving a local weighted least-squares problem, where the weights are a function of the 3-D space-time orientation in the neighborhood. As this framework is fundamentally based upon the comparison of neighboring pixels in both space and time, it implicitly contains information about the local motion of the pixels across time, therefore rendering unnecessary an explicit computation of motions of modest size. The proposed approach not only significantly widens the applicability of super-resolution methods to a broad variety of video sequences containing complex motions, but also yields improved overall performance. Using several examples, we illustrate that the developed algorithm has super-resolution capabilities that provide improved optical resolution in the output, while being able to work on general input video with essentially arbitrary motion.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1981) · Cited by 11,963
Image denoising by sparse 3-D transform-domain collaborative filtering
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.901238 · Cited by 9,283
A Review of Image Denoising Algorithms, with a New One
Buades, A., Coll, B., & Morel, J. M. (2005) · Multiscale Modeling & Simulation · doi:10.1137/040616024 · Cited by 4,083
Perception of the Visual World
(1952) · Archives of Ophthalmology · doi:10.1001/archopht.1952.01700030123016 · Cited by 3,732
Nadaraya, E. A. (1964) · Theory of Probability & Its Applications · doi:10.1137/1109020 · Cited by 3,462
Super-resolution image reconstruction: a technical overview
Park, S. C., Park, M. K., & Kang, M. G. (2003) · IEEE Signal Processing Magazine · doi:10.1109/msp.2003.1203207 · Cited by 3,282
The design and use of steerable filters
Freeman, W. T., & Adelson, E. H. (1991) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.93808 · Cited by 2,942
Wand, M. P., & Jones, M. C. (1995) · Springer eBooks · doi:10.1007/978-1-4899-4493-1 · Cited by 2,663
Example-based super-resolution
Freeman, W. T., Jones, T. R., & Pasztor, E. C. (2002) · IEEE Computer Graphics and Applications · doi:10.1109/38.988747 · Cited by 2,498
Fast and Robust Multiframe Super Resolution
Farsiu, S., Robinson, M. D., Elad, M., & Milanfar, P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2004.834669 · Cited by 2,090
Self-citationThe Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields
Black, M. J., & Anandan, P. (1996) · Computer Vision and Image Understanding · doi:10.1006/cviu.1996.0006 · Cited by 1,570
Kernel Regression for Image Processing and Reconstruction
Takeda, H., Farsiu, S., & Milanfar, P. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.888330 · Cited by 1,340
Self-citationThe Perception of the Visual World.
S., H. W., & Gibson, J. J. (1951) · The Journal of Philosophy · doi:10.2307/2021210 · Cited by 1,139
Multivariate Locally Weighted Least Squares Regression
Ruppert, D., & Wand, M. P. (1994) · The Annals of Statistics · doi:10.1214/aos/1176325632 · Cited by 1,056
Generalizing the Nonlocal-Means to Super-Resolution Reconstruction
Protter, M., Elad, M., Takeda, H., & Milanfar, P. (2009) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.2008067 · Cited by 707
Self-citationThe optic flow field: the foundation of vision
Lee, D. N. (1980) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1980.0089 · Cited by 642
A New SURE Approach to Image Denoising: Interscale Orthonormal Wavelet Thresholding
Luisier, F., Blu, T., & Unser, M. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.891064 · Cited by 623
Elad, M., & Hel-Or, Y. (2001) · IEEE Transactions on Image Processing · doi:10.1109/83.935034 · Cited by 456
Self-citationRepresenting Local Structure Using Tensors II
Knutsson, H., Westin, C.-F., & Andersson, M. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-21227-7_51 · Cited by 355
Zomet, A., Rav-Acha, A., & Peleg, S. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990535 · Cited by 285
Normalized and differential convolution
Knutsson, H., & Westin, C.-F. (2002) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1993.341081 · Cited by 242
Nguyen, N., Milanfar, P., & Golub, G. (2001) · IEEE Transactions on Image Processing · doi:10.1109/83.941854 · Cited by 229
Self-citationDeblurring Using Regularized Locally Adaptive Kernel Regression
Takeda, H., Farsiu, S., & Milanfar, P. (2008) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.918028 · Cited by 160
Self-citationPolynomial Expansion for Orientation and Motion Estimation
Farnebäck, G. (2002) · Cited by 144
Metric Learning for Kernel Regression.
Weinberger, K. Q., & Tesauro, G. J. (2007) · Cited by 140
Woods, N. A., Galatsanos, N. P., & Katsaggelos, A. K. (2005) · IEEE Transactions on Image Processing · doi:10.1109/tip.2005.860355 · Cited by 130
Statistical performance analysis of super-resolution
Robinson, D., & Milanfar, P. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.871079 · Cited by 120
Self-citationBaboulaz, L., & Dragotti, P. L. (2009) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.2009378 · Cited by 84
Least Squares and Robust Estimation of Local Image Structure
van de Weijer, J., & van den Boomgaard, R. (2005) · International Journal of Computer Vision · doi:10.1007/s11263-005-1840-0 · Cited by 38
Least Squares and Robust Estimation of Local Image Structure
van den Boomgaard, R., & van de Weijer, J. (2003) · Lecture notes in computer science · doi:10.1007/3-540-44935-3_17 · Cited by 14
Statistical Approaches to Quality Assessment for Image Restoration
Takeda, H., Seo, H. J., & Milanfar, P. (2008) · International Conference on Consumer Electronics · doi:10.1109/icce.2008.4587957 · Cited by 3
Self-citation
Cited by
Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.207 · Cited by 7,570
Edvr: video restoration with enhanced deformable convolutional networks
Wang, X., Chan, K. C. K., Yu, K., Dong, C., & Loy, C. C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00247 · Cited by 1,272
Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation
Caballero, J., Ledig, C., Aitken, A., Acosta, A., Totz, J., Wang, Z., & Shi, W. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.304 · Cited by 763
Super-resolution: a comprehensive survey
Nasrollahi, K., & Moeslund, T. B. (2014) · Machine Vision and Applications · doi:10.1007/s00138-014-0623-4 · Cited by 684
Video Super-Resolution With Convolutional Neural Networks
Kappeler, A., Yoo, S., Dai, Q., & Katsaggelos, A. K. (2016) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2016.2532323 · Cited by 647
Metadata
- Authors
- Takeda, H., Milanfar, P., Protter, M., & Elad, M.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2009-09
- Volume
- 18
- Issue
- 9
- Pages
- 1958-1975
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W2109586214
- Cited by
- 302
- References
- 31
- FWCI
- 28.22
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.