Convolutional Neural Network-Based Fractional-Pixel Motion Compensation
The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
Fractional-pixel motion compensation (MC) improves the efficiency of inter prediction and has been utilized extensively in video coding standards. The traditional methods of fractional-pixel MC usually follow the approach of interpolation, i.e., they adopt different kinds of filters, either fixed or adaptive, to interpolate fractional-pixel values from integer-pixel values in a reference picture. Different from the interpolation approach, in this paper, we formulate the fractional-pixel MC as an inter-picture regression problem, which is to predict the pixel values of the current to-be-coded picture from the integer-pixel values of a reference picture, given a fractional-pixel motion vector that relates the two pictures. We then propose to adopt convolutional neural network (CNN) models to approach the regression problem, inspired by the recent advances of CNN. Accordingly, we propose fractional-pixel reference generation CNN (FRCNN) for both uni-directional and bi-directional MC in video coding. We further investigate how to train FRCNN by using encoded video sequences, and empirically study the effect of different training data and different CNN structures. Moreover, we propose to integrate FRCNN into the high efficiency video coding (HEVC) scheme, and perform a comprehensive set of experiments to evaluate the effectiveness of FRCNN. The experimental results show that our proposed FRCNN achieves on average 3.9%, 2.7%, and 1.3% bits saving compared with HEVC, under low-delay P, low-delay B, and random-access configurations, respectively.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Accurate Image Super-Resolution Using Very Deep Convolutional Networks
Kim, J., Lee, J. K., & Lee, K. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.182 · Cited by 7,870
Cited by
Image and Video Compression With Neural Networks: A Review
Ma, S., Zhang, X., Jia, C., Zhao, Z., Wang, S., & Wang, S. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2910119 · Cited by 459
Deep Learning-Based Video Coding: A Review and a Case Study
Liu, D., Li, Y., Lin, J., Li, H., & Wu, F. (2021) · ACM Computing Surveys · doi:10.1145/3368405 · Cited by 192
“al. [141] propose another method, which formulates the fractional interpolation as a resolution enhancement problem. Thus, they down-sample high-resolution images to achieve training data. Yan et al. [136] consider a different formulation, treating the fractional-pixel MC as an inter-picture regression problem. They u…”
MethodSelf-citationEnhanced bi-prediction with convolutional neural network for high-efficiency video coding
Zhao, Z., Wang, S., Wang, S., Zhang, X., Ma, S., & Yang, J. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2876399 · Cited by 75
“[41], [42] enhanced the quality of interpolated pixels with CNN based half-pixel interpolation.”
Enhanced Surveillance Video Compression with Dual Reference Frames Generation
Zhao, L., Wang, S., Wang, S., Ye, Y., Ma, S., & Gao, W. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3073114 · Cited by 43
Metadata
- Authors
- Yan, N., Liu, D., Li, H., Li, B., Li, L., & Wu, F.
- Venue
- IEEE Transactions on Circuits and Systems for Video Technology
- Published
- 2019-03
- Volume
- 29
- Issue
- 3
- Pages
- 840-853
- ISSN
- 1051-8215 (ISSN-L)
- 1558-2205
- OpenAlex
- W2791933231
- Cited by
- 87
- References
- 2
- FWCI
- 8.53
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.