Cited by 260Video Coding and Compression TechnologiesAdvanced Data Compression TechniquesVideo Analysis and Summarization

Video Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics

Open access
Duan, Lingyu, Liu, Jiaying, Yang, Wenhan, Huang, Tiejun, Gao, Wen·2020·IEEE Transactions on Image Processing

Abstract

Video coding, which targets to compress and reconstruct the whole frame, and feature compression, which only preserves and transmits the most critical information, stand at two ends of the scale. That is, one is with compactness and efficiency to serve for machine vision, and the other is with full fidelity, bowing to human perception. The recent endeavors in imminent trends of video compression, e.g. deep learning based coding tools and end-to-end image/video coding, and MPEG-7 compact feature descriptor standards, i.e. Compact Descriptors for Visual Search and Compact Descriptors for Video Analysis, promote the sustainable and fast development in their own directions, respectively. In this paper, thanks to booming AI technology, e.g. prediction and generation models, we carry out exploration in the new area, Video Coding for Machines (VCM), arising from the emerging MPEG standardization efforts1. Towards collaborative compression and intelligent analytics, VCM attempts to bridge the gap between feature coding for machine vision and video coding for human vision. Aligning with the rising Analyze then Compress instance Digital Retina, the definition, formulation, and paradigm of VCM are given first. Meanwhile, we systematically review state-of-the-art techniques in video compression and feature compression from the unique perspective of MPEG standardization, which provides the academic and industrial evidence to realize the collaborative compression of video and feature streams in a broad range of AI applications. Finally, we come up with potential VCM solutions, and the preliminary results have demonstrated the performance and efficiency gains. Further direction is discussed as well.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tip.2020.3016485

Cites

145 of 145 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 19

  1. [18][18] Y. Xu, C. Lv, S. Li, P. Xin, S. Ma, H. Zou, W. Zhang, 'Review of development of visual neural computing,' Computer Engineering and Applications , 2017. -
  2. [37][37] L. Jiaying, X. Sifeng, , and Y. Wenhan, 'Deep reference generation with multidomain hierarchical constraints for inter prediction,' IEEE TMM , Dec. 2019. -
  3. [39][39] Y. Dai, D. Liu, Z. Zha, and F. Wu, 'A CNN-based in-loop filter with CU classification for HEVC,' in IEEE VCIP , Dec. 2018. -
  4. [77][77] A. Krizhevsky, I. Sutskever, and G. E. Hinton, 'ImageNet classification with deep convolutional neural networks,' in NeurIPS , 2012. -
  5. [78][78] K. He, X. Zhang, S. Ren, and J. Sun, 'Deep residual learning for image recognition,' in CVPR , Jun. 2016. -
  6. [94][94] Y. Choi, M. Choi, M. Kim, J. Ha, S. Kim, and J. Choo, 'StarGAN: Unified generative adversarial networks for multi-domain image-to-image translation,' in CVPR , Jun. 2018. -
  7. [95][95] W. Xian, P. Sangkloy, V. Agrawal, A. Raj, J. Lu, C. Fang, F. Yu, and J. Hays, 'TextureGAN: Controlling deep image synthesis with texture patches,' in CVPR , Jun. 2018. -
  8. [96][96] C. Ledig, L. Theis, F. Huszzr, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi, 'Photo-realistic single image super-resolution using a generative adversarial network,' in CVPR , Jul. 2017. -
  9. [98][98] J. Guo and H. Chao, 'One-to-many network for visually pleasing compression artifacts reduction,' in CVPR , Jul. 2017. -
  10. [100][100] T. Wang, M. Liu, J. Zhu, A. Tao, J. Kautz, and B. Catanzaro, 'High-resolution image synthesis and semantic manipulation with conditional GANs,' in CVPR , Jun. 2018. -
  11. [143][143] C. Liu, Y. Hu, Y. Li, S. Song, and J. Liu, 'PKU-MMD: A large scale benchmark for skeleton-based human action understanding,' in ACM MM , 2017. -
  12. [150][150] W. Yang, J. Feng, J. Yang, F. Zhao, J. Liu, Z. Guo, and S. Yan, 'Deep edge guided recurrent residual learning for image super-resolution,' IEEE TIP , Dec. 2017. -
  13. [151][151] Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, 'Residual dense network for image super-resolution,' in CVPR , Jun. 2018. -
  14. [152][152] Y. Tai, J. Yang, X. Liu, and C. Xu, 'Memnet: A persistent memory network for image restoration,' in ICCV , Oct. 2017. -
  15. [153][153] Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, 'Image super-resolution using very deep residual channel attention networks,' in ICCV , Jul. 2018. -
  16. [154][154] W. Yang, R. T. Tan, J. Feng, J. Liu, Z. Guo, and S. Yan, 'Deep joint rain detection and removal from a single image,' in CVPR , Jul. 2017. -
  17. [155][155] H. Zhang and V. M. Patel, 'Density-aware single image de-raining using a multi-stream dense network,' in CVPR , Jun. 2018. -
  18. [157][157] L. Galteri, L. Seidenari, M. Bertini, and A. Del Bimbo, 'Deep generative adversarial compression artifact removal,' in CVPR , 2017. -
  19. [158][158] O. Kupyn, V. Budzan, M. Mykhailych, D. Mishkin, and J. Matas. 'DeblurGAN: Blind motion deblurring using conditional adversarial networks,' in CVPR , Jun. 2018. -

Cited by

Sorted by citations

Authors
Duan, L., Liu, J., Yang, W., Huang, T., & Gao, W.
Affiliations
Peking University (CN)
Venue
IEEE Transactions on Image Processing
Published
2020
Volume
29
Pages
8680-8695
ISSN
  • 1057-7149 (ISSN-L)
  • 1941-0042
OpenAlex
W3082548248
Cited by
260
References
144
Citations by year
2026: 25 · 2025: 43 · 2024: 59 · 2023: 51 · 2022: 38 · 2021: 37 · 2020: 7
FWCI
18.00
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Image Processing (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Signal Processing › Video Coding and Compression Technologies
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Data Compression Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
Keywords
video coding for machines, feature compression, MPEG standardization, video compression, intelligent analytics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review