TDAN: Temporally-Deformable Alignment Network for Video Super-Resolution
Open accessThe DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Abstract
Video super-resolution (VSR) aims to restore a photo-realistic high-resolution (HR) video frame from both its corresponding low-resolution (LR) frame (reference frame) and multiple neighboring frames (supporting frames). Due to varying motion of cameras or objects, the reference frame and each support frame are not aligned. Therefore, temporal alignment is a challenging yet important problem for VSR. Previous VSR methods usually utilize optical flow between the reference frame and each supporting frame to warp the supporting frame for temporal alignment. However, both inaccurate flow and the image-level warping strategy will lead to artifacts in the warped supporting frames. To overcome the limitation, we propose a temporally-deformable alignment network (TDAN) to adaptively align the reference frame and each supporting frame at the feature level without computing optical flow. The TDAN uses features from both the reference frame and each supporting frame to dynamically predict offsets of sampling convolution kernels. By using the corresponding kernels, TDAN transforms supporting frames to align with the reference frame. To predict the HR video frame, a reconstruction network taking aligned frames and the reference frame is utilized. Experimental results demonstrate that the TDAN is capable of alleviating occlusions and artifacts for temporal alignment and the TDAN-based VSR model outperforms several recent state-of-the-art VSR networks with a comparable or even much smaller model size. The source code and pre-trained models are released in https://github.com/YapengTian/TDAN-VSR.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
(2017) · Cited by 11,625
Accurate Image Super-Resolution Using Very Deep Convolutional Networks
Kim, J., Lee, J. K., & Lee, K. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.182 · Cited by 7,870
Enhanced Deep Residual Networks for Single Image Super-Resolution
Lim, B., Son, S., Kim, H., Nah, S., & Lee, K. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.151 · Cited by 7,736
Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.207 · Cited by 7,570
Deformable Convolutional Networks
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.89 · Cited by 7,513
Image Super-Resolution Using Very Deep Residual Channel Attention Networks
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_18 · Cited by 5,708
Self-citationLearning a Deep Convolutional Network for Image Super-Resolution
Dong, C., Loy, C. C., He, K., & Tang, X. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10593-2_13 · Cited by 5,494
Cited by
Edvr: video restoration with enhanced deformable convolutional networks
Wang, X., Chan, K. C. K., Yu, K., Dong, C., & Loy, C. C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00247 · Cited by 1,272
Basicvsr++: improving video super-resolution with enhanced propagation and alignment
Chan, K. C. K., Zhou, S., Xu, X., & Loy, C. C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00588 · Cited by 604
BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond
Chan, K. C. K., Wang, X., Yu, K., Dong, C., & Loy, C. C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00491 · Cited by 604
Real-world single image super-resolution: A brief review
Chen, H., He, X., Qing, L., Wu, Y., Ren, C., Sheriff, R. E., & Zhu, C. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.09.005 · Cited by 405
VRT: A Video Restoration Transformer
Liang, J., Cao, J., Fan, Y., Zhang, K., Ranjan, R., Li, Y., Timofte, R., & Van Gool, L. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3372454 · Cited by 302
Changer: Feature Interaction is What You Need for Change Detection
Fang, S., Li, K., & Li, Z. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3277496 · Cited by 299
FVC: A New Framework towards Deep Video Compression in Feature Space
Hu, Z., Lu, G., & Xu, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00155 · Cited by 273
Understanding deformable alignment in video super-resolution
Chan, K. C. K., Wang, X., Yu, K., Dong, C., & Loy, C. C. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16181 · Cited by 162
Learning Spatial Adaptation and Temporal Coherence in Diffusion Models for Video Super-Resolution
Chen, Z., Long, F., Qiu, Z., Yao, T., Zhou, W., Luo, J., & Mei, T. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00882 · Cited by 27
Video demoiréing with relation-based temporal consistency
Dai, P., Yu, X., Ma, L., Zhang, B., Li, J., Li, W., Shen, J., & Qi, X. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01710 · Cited by 26
Metadata
- Authors
- Tian, Y., Zhang, Y., Fu, Y., & Xu, C.
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 3357-3366
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3034475761
- Cited by
- 634
- References
- 42
- FWCI
- 32.93
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.