The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
FVC: A New Framework towards Deep Video Compression in Feature Space
Abstract
Learning based video compression attracts increasing attention in the past few years. The previous hybrid coding approaches rely on pixel space operations to reduce spatial and temporal redundancy, which may suffer from inaccurate motion estimation or less effective motion compensation. In this work, we propose a feature-space video coding network (FVC) by performing all major operations (i.e., motion estimation, motion compression, motion compensation and residual compression) in the feature space. Specifically, in the proposed deformable compensation module, we first apply motion estimation in the feature space to produce motion information (i.e., the offset maps), which will be compressed by using the auto-encoder style network. Then we perform motion compensation by using deformable convolution and generate the predicted feature. After that, we compress the residual feature between the feature from the current frame and the predicted feature from our deformable compensation module. For better frame reconstruction, the reference features from multiple previous reconstructed frames are also fused by using the nonlocal attention mechanism in the multi-frame feature fusion module. Comprehensive experimental results demonstrate that the proposed framework achieves the state-of-the-art performance on four benchmark datasets including HEVC, UVG, VTL and MCL-JCV.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Overview of the High Efficiency Video Coding (HEVC) Standard
Sullivan, G. J., Ohm, J.-R., Han, W.-J., & Wiegand, T. (2012) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2012.2221191 · Cited by 8,263
Overview of the H.264/AVC video coding standard
Wiegand, T., Sullivan, G. J., Bjontegaard, G., & Luthra, A. (2003) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2003.815165 · Cited by 8,184
Deformable Convolutional Networks
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.89 · Cited by 7,513 · cited 3×
“[9] proposed to use deformable convolution together with the learnt offset maps to enhance the modeling capability of convolution neural networks, which has achieved promising results in several videorelated tasks like action recognition [18] and video superresolution [30, 35].”
BackgroundMethodMultiscale structural similarity for image quality assessment
Wang, Z., Simoncelli, E. P., & Bovik, A. C. (2004) · The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003 · doi:10.1109/acssc.2003.1292216 · Cited by 6,253 · cited 6×
“In our proposed framework, we optimize the following Rate Distortion (RD) trade-off: where R o and R r denote the numbers of bits used to encode the offset map O t and the residual feature map R t , respectively. d ( X t , ˆ X t ) denotes the distortion between the input frame X t and the reconstructed frame ˆ X t , w…”
BackgroundMethodInfluentialCalcuation of average PSNR differences between RD-curves
Bjøntegaard, G. (2001) · Medical Entomology and Zoology · Cited by 6,090
The JPEG still picture compression standard
Wallace, G. K. (1992) · IEEE Transactions on Consumer Electronics · doi:10.1109/30.125072 · Cited by 2,851
Video Enhancement with Task-Oriented Flow
Xue, T., Chen, B., Wu, J., Wei, D., & Freeman, W. T. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-018-01144-2 · Cited by 1,393
Variational image compression with a scale hyperprior
Ballé, J., Minnen, D. C., Singh, S., Hwang, S. J., & Johnston, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.01436 · Cited by 1,318
End-to-end Optimized Image Compression
Ballé, J., Laparra, V., & Simoncelli, E. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01704 · Cited by 1,275
Edvr: video restoration with enhanced deformable convolutional networks
Wang, X., Chan, K. C. K., Yu, K., Dong, C., & Loy, C. C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00247 · Cited by 1,272 · cited 2×
“[9] proposed to use deformable convolution together with the learnt offset maps to enhance the modeling capability of convolution neural networks, which has achieved promising results in several videorelated tasks like action recognition [18] and video superresolution [30, 35].”
BackgroundMethodVideo Enhancement with Task-Oriented FlowPreprint
Xue, T., Chen, B., Wu, J., Wei, D., & Freeman, W. T. (2019) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 1,190
Full Resolution Image Compression with Recurrent Neural Networks
Toderici, G., Vincent, D., Johnston, N., Hwang, S. J., Minnen, D., Shor, J., & Covell, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.577 · Cited by 886
DVC: An End-To-End Deep Video Compression Framework
Lu, G., Ouyang, W., Xu, D., Zhang, X., Cai, C., & Gao, Z. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01126 · Cited by 739
Self-citationTDAN: Temporally-Deformable Alignment Network for Video Super-Resolution
Tian, Y., Zhang, Y., Fu, Y., & Xu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00342 · Cited by 634
Generative Adversarial Networks for Extreme Learned Image Compression
Agustsson, E., Tschannen, M., Mentzer, F., Timofte, R., & Van Gool, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00031 · Cited by 575
Johnston, N., Vincent, D., Minnen, D., Covell, M., Singh, S., Chinen, T., Jin Hwang, S., Shor, J., & Toderici, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00461 · Cited by 366
MCL-JCV: A JND-based H.264/AVC video quality assessment dataset
Wang, H., Gan, W., Hu, S., Lin, J. Y., Jin, L., Song, L., Wang, P., Katsavounidis, I., Aaron, A., & Kuo, C.-C. J. (2016) · 2016 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2016.7532610 · Cited by 319
Scale-Space Flow for End-to-End Optimized Video Compression
Agustsson, E., Minnen, D., Johnston, N., Ballé, J., Hwang, S. J., & Toderici, G. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00853 · Cited by 317
Video Compression Through Image Interpolation
Wu, C.-Y., Singhal, N., & Krähenbühl, P. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_26 · Cited by 316
Joint Autoregressive and Hierarchical Priors for Learned Image Compression
Minnen, D. C., Ballé, J., & Toderici, G. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.02736 · Cited by 310
JPEG2000: standard for interactive imaging
Taubman, D., & Marcellin, M. W. (2002) · Proceedings of the IEEE · doi:10.1109/jproc.2002.800725 · Cited by 266
Lossy Image Compression with Compressive Autoencoders
Huszár, F., Theis, L., Shi, W., & Cunningham, A. (2020) · Apollo (University of Cambridge) · doi:10.17863/cam.51995 · Cited by 237
An End-to-End Learning Framework for Video Compression
Lu, G., Zhang, X., Ouyang, W., Chen, L., Gao, Z., & Xu, D. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2988453 · Cited by 224
Self-citationRippel, O., Nair, S., Lew, C., Branson, S., Anderson, A., & Bourdev, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00355 · Cited by 223
Video Compression With Rate-Distortion Autoencoders
Habibian, A., Rozendaal, T. V., Tomczak, J., & Cohen, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00713 · Cited by 218
Learning for Video Compression With Hierarchical Quality and Recurrent Enhancement
Yang, R., Mentzer, F., Van Gool, L., & Timofte, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00666 · Cited by 218
Content Adaptive and Error Propagation Aware Deep Video Compression
Lu, G. S., Cai, C., Zhang, X., Chen, L., Ouyang, W., Xu, D., & Gao, Z. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58536-5_27 · Cited by 187
Self-citationLearning for Video Compression With Recurrent Auto-Encoder and Recurrent Probability Model
Yang, R., Mentzer, F., Gool, L. V., & Timofte, R. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3043590 · Cited by 176
Improving Deep Video Compression by Resolution-Adaptive Flow Coding
Hu, Z., Chen, Z., Xu, D., Lu, G. S., Ouyang, W., & Gu, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58536-5_12 · Cited by 161
Self-citationSpatio-temporal deformable 3D ConvNets with attention for action recognition
Li, J., Liu, X., Zhang, M., & Wang, D. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.107037 · Cited by 146
Learning Image and Video Compression Through Spatial-Temporal Energy Compaction
Cheng, Z., Sun, H., Takeuchi, M., & Katto, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01031 · Cited by 102
Non-Local ConvLSTM for Video Compression Artifact Reduction
Xu, Y., Gao, L., Tian, K., Zhou, S., & Sun, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00714 · Cited by 90
Deep Generative Video CompressionPreprint
Han, J., Lombardo, S., Schroers, C., & Mandt, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.02845 · Cited by 60
Utilising Low Complexity CNNs to Lift Non-Local Redundancies in Video Coding
Klopp, J. P., Chen, L., & Chien, S. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2991525 · Cited by 15
Cited by
Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications
Gunduz, D., Qin, Z., Aguerri, I. E., Dhillon, H. S., Yang, Z., Yener, A., Wong, K. K., & Chae, C.-B. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3223408 · Cited by 776
Metadata
- Authors
- Hu, Z., Lu, G., & Xu, D.
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 1502-1511
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3173272744
- Cited by
- 273
- References
- 36
- FWCI
- 28.82
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.