The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Channel Attention Is All You Need for Video Frame Interpolation
Open accessAbstract
Prevailing video frame interpolation techniques rely heavily on optical flow estimation and require additional model complexity and computational cost; it is also susceptible to error propagation in challenging scenarios with large motion and heavy occlusion. To alleviate the limitation, we propose a simple but effective deep neural network for video frame interpolation, which is end-to-end trainable and is free from a motion estimation network component. Our algorithm employs a special feature reshaping operation, referred to as PixelShuffle, with a channel attention, which replaces the optical flow computation module. The main idea behind the design is to distribute the information in a feature map into multiple channels and extract motion information by attending the channels for pixel-level frame synthesis. The model given by this principle turns out to be effective in the presence of challenging motion and occlusion. We construct a comprehensive evaluation benchmark and demonstrate that the proposed approach achieves outstanding performance compared to the existing models with a component for optical flow computation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
CBAM: Convolutional Block Attention Module
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_1 · Cited by 25,853
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.207 · Cited by 7,570
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Image Super-Resolution Using Very Deep Residual Channel Attention Networks
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_18 · Cited by 5,708
Ucf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
Residual Attention Network for Image Classification
Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., Zhang, H., Wang, X., & Tang, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.683 · Cited by 4,073
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
Sun, D., Yang, X., Liu, M.-Y., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00931 · Cited by 2,648
Recurrent Models of Visual Attention
Mnih, V., Heess, N., Graves, A., & Kavukcuoglu, K. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.6247 · Cited by 2,568
Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring
Nah, S., Kim, T. H., & Lee, K. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.35 · Cited by 2,275
Self-citationA Database and Evaluation Methodology for Optical Flow
Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M. J., & Szeliski, R. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-010-0390-2 · Cited by 2,205 · cited 3×
“Video frame interpolation approaches are typically based on optical flow estimation to handle time-varying information in videos (Baker et al. 2010; Werlberger et al. 2011; Yu et al. 2013).”
BackgroundMethodSqueeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G. Z., & Wu, E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.01507 · Cited by 1,908
Unsupervised Learning of Video Representations using LSTMs
Srivastava, N., Mansimov, E., & Salakhutdinov, R. (2015) · arXiv · doi:10.48550/arxiv.1502.04681 · Cited by 1,861
Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification
Xie, S., Sun, C., Huang, J., Tu, Z., & Murphy, K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_19 · Cited by 1,415
Video Enhancement with Task-Oriented FlowPreprint
Xue, T., Chen, B., Wu, J., Wei, D., & Freeman, W. T. (2019) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 1,190
Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation
Jiang, H., Sun, D., Jampani, V., Yang, M.-H., Learned-Miller, E., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00938 · Cited by 829
Video Frame Synthesis Using Deep Voxel Flow
Liu, Z., Yeh, R. A., Tang, X., Liu, Y., & Agarwala, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.478 · Cited by 743
Flow-Guided Feature Aggregation for Video Object Detection
Zhu, X., Wang, Y., Dai, J., Yuan, L., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.52 · Cited by 729 · cited 1×
“…classi-fication (Carreira and Zisserman 2017; Simonyan and Zis-serman 2014a; Xie et al. 2018), object detection and segmentation (Cheng et al. 2017; Zhu et al. 2018; 2017), video prediction (Li et al. 2018; Reda et al. 2018), and others (Chen et al. 2017; Lai et al. 2018)—frequently depend on…”
BackgroundVideo Frame Interpolation via Adaptive Separable Convolution
Niklaus, S., Mai, L., & Liu, F. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.37 · Cited by 725
A Database and Evaluation Methodology for Optical Flow
Baker, S., Roth, S., Scharstein, D., Black, M. J., Lewis, J. P., & Szeliski, R. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408903 · Cited by 618
Depth-Aware Video Frame Interpolation
Bao, W., Lai, W.-S., Ma, C., Zhang, X., Gao, Z., & Yang, M.-H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00382 · Cited by 554
Video Frame Interpolation via Adaptive Convolution
Niklaus, S., Mai, L., & Liu, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.244 · Cited by 499
SegFlow: Joint Learning for Video Object Segmentation and Optical Flow
Cheng, J., Tsai, Y.-H., Wang, S., & Yang, M.-H. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.81 · Cited by 446
Context-Aware Synthesis for Video Frame Interpolation
Niklaus, S., & Liu, F. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00183 · Cited by 411
Learning Blind Video Temporal Consistency
Lai, W.-S., Huang, J.-B., Wang, O., Shechtman, E., Yumer, E., & Yang, M.-H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_11 · Cited by 405
Cao, C., Liu, X., Yang, Y., Yu, Y., Wang, J., Wang, Z., Huang, Y., Wang, L., Huang, C., Xu, W., et al. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.338 · Cited by 392
Bao, W., Lai, W.-S., Zhang, X., Gao, Z., & Yang, M.-H. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2941941 · Cited by 362
Deep multi-scale video prediction beyond mean square error
Mathieu, M., Couprie, C., & LeCun, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.05440 · Cited by 339
Phase-based frame interpolation for video
Meyer, S., Wang, O., Zimmer, H., Grosse, M., & Sorkine-Hornung, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298747 · Cited by 294
Towards High Performance Video Object Detection
Zhu, X., Dai, J., Yuan, L., & Wei, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00753 · Cited by 285
Coherent Online Video Style Transfer
Chen, D., Liao, J., Yuan, L., Yu, N., & Hua, G. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.126 · Cited by 281
Video (language) modeling: a baseline for generative models of natural videos
Ranzato, M., Szlam, A. D., Bruna, J., Mathieu, M., Collobert, R., & Chopra, S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6604 · Cited by 254
Learning Image Matching by Simply Watching Video
Long, G., Kneip, L., Alvarez, J. M., Li, H., Zhang, X., & Yu, Q. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_26 · Cited by 188
PhaseNet for Video Frame Interpolation
Meyer, S., Djelouah, A., McWilliams, B., Sorkine-Hornung, A., Gross, M., & Schroers, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00059 · Cited by 184
Deep Video Frame Interpolation Using Cyclic Frame Generation
Liu, Y.-L., Liao, Y.-T., Lin, Y.-Y., & Chuang, Y.-Y. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018794 · Cited by 161
SDC-Net: Video Prediction Using Spatially-Displaced Convolution
Reda, F. A., Liu, G., Shih, K. J., Kirby, R. M., Barker, J., Tarjan, D., Tao, A., & Catanzaro, B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_44 · Cited by 142
Flow-Grounded Spatial-Temporal Video Prediction from Still Images
Li, Y., Chen, F., Yang, J., Wang, Z., Lu, X., & Yang, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01240-3_37 · Cited by 139
Optical Flow Guided TV-L1 Video Interpolation and Restoration
Werlberger, M., Pock, T., Unger, M., & Bischof, H. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23094-3_20 · Cited by 91
Multi-Level Video Frame Interpolation: Exploiting the Interaction Among Different Levels
Yu, Z., Li, H., Wang, Z., Hu, Z., & Chen, C. W. (2013) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2013.2242631 · Cited by 59
Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video InterpolationPreprint
Jiang, H., Sun, D., Jampani, V., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.00080 · Cited by 44
Cited by
Real-Time Intermediate Flow Estimation for Video Frame Interpolation
Huang, Z., Zhang, T., Heng, W., Shi, B., & Zhou, S. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19781-9_36 · Cited by 326
VRT: A Video Restoration Transformer
Liang, J., Cao, J., Fan, Y., Zhang, K., Ranjan, R., Li, Y., Timofte, R., & Van Gool, L. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3372454 · Cited by 302
Metadata
- Authors
- Choi, M., Kim, H., Han, B., Xu, N., & Lee, K. M.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2020-04-03
- Volume
- 34
- Issue
- 07
- Pages
- 10663-10671
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2997150500
- Cited by
- 328
- References
- 49
- FWCI
- 15.82
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.