The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Scale-Space Flow for End-to-End Optimized Video Compression
Abstract
Despite considerable progress on end-to-end optimized deep networks for image compression, video coding remains a challenging task. Recently proposed methods for learned video compression use optical flow and bilinear warping for motion compensation and show competitive rate-distortion performance relative to hand-engineered codecs like H.264 and HEVC. However, these learning-based methods rely on complex architectures and training schemes including the use of pre-trained optical flow networks, sequential training of sub-networks, adaptive rate control, and buffering intermediate reconstructions to disk during training. In this paper, we show that a generalized warping operator that better handles common failure cases, e.g. disocclusions and fast motion, can provide competitive compression results with a greatly simplified model and training procedure. Specifically, we propose scale-space flow, an intuitive generalization of optical flow that adds a scale parameter to allow the network to better model uncertainty. Our experiments show that a low-latency video compression model (no B-frames) using scale-space flow for motion compensation can outperform analogous state-of-the art learned video compression models while being trained using a much simpler procedure and without any pre-trained optical flow networks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Multiscale structural similarity for image quality assessment
Wang, Z., Simoncelli, E. P., & Bovik, A. C. (2004) · The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003 · doi:10.1109/acssc.2003.1292216 · Cited by 6,253 · cited 2×
“Recently, there has been significant progress in the area of end-to-end optimized image compression, which went from barely matching JPEG [33] to methods such as [8, 26, 5] that can outperform the best hand-engineered codecs when evaluated in terms of multi-scale structural similarity (MS-SSIM) [36], PSNR, and subject…”
BackgroundMethodCalcuation of average PSNR differences between RD-curves
Bjøntegaard, G. (2001) · Medical Entomology and Zoology · Cited by 6,090
The JPEG still picture compression standard
Wallace, G. K. (1991) · Communications of the ACM · doi:10.1145/103085.103089 · Cited by 3,784
Overview of the Scalable Video Coding Extension of the H.264/AVC Standard
Schwarz, H., Marpe, D., & Wiegand, T. (2007) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2007.905532 · Cited by 3,533
Scale-Space Theory in Computer Vision
Lindeberg, T. (1994) · Springer eBooks · doi:10.1007/978-1-4757-6465-9 · Cited by 2,169
Pyramid Stereo Matching Network
Chang, J.-R., & Chen, Y.-S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00567 · Cited by 1,807
Video Enhancement with Task-Oriented Flow
Xue, T., Chen, B., Wu, J., Wei, D., & Freeman, W. T. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-018-01144-2 · Cited by 1,393
Optical Flow Estimation Using a Spatial Pyramid Network
Ranjan, A., & Black, M. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.291 · Cited by 1,344
Variational image compression with a scale hyperprior
Ballé, J., Minnen, D. C., Singh, S., Hwang, S. J., & Johnston, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.01436 · Cited by 1,318
Self-citationEnd-to-end Optimized Image Compression
Ballé, J., Laparra, V., & Simoncelli, E. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01704 · Cited by 1,275
Self-citationVideo Enhancement with Task-Oriented FlowPreprint
Xue, T., Chen, B., Wu, J., Wei, D., & Freeman, W. T. (2019) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 1,190
Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods
Bruhn, A., Weickert, J., & Schnörr, C. (2005) · International Journal of Computer Vision · doi:10.1023/b:visi.0000045324.43199.43 · Cited by 1,160
Lucas/Kanade meets Horn/Schunck : combining local and global optic flow methods
Weickert, J., Bruhn, A., & Schnörr, C. (2003) · doi:10.22028/d291-26234 · Cited by 963
Full Resolution Image Compression with Recurrent Neural Networks
Toderici, G., Vincent, D., Johnston, N., Hwang, S. J., Minnen, D., Shor, J., & Covell, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.577 · Cited by 886
Self-citationDVC: An End-To-End Deep Video Compression Framework
Lu, G., Ouyang, W., Xu, D., Zhang, X., Cai, C., & Gao, Z. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01126 · Cited by 739
Generative Adversarial Networks for Extreme Learned Image Compression
Agustsson, E., Tschannen, M., Mentzer, F., Timofte, R., & Van Gool, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00031 · Cited by 575
Self-citationConditional Probability Models for Deep Image Compression
Mentzer, F., Agustsson, E., Tschannen, M., Timofte, R., & Gool, L. V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00462 · Cited by 490
Self-citationSoft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
Agustsson, E., Mentzer, F., Tschannen, M., Cavigelli, L., Timofte, R., Benini, L., & Van Gool, L. (2017) · arXiv · doi:10.48550/arxiv.1704.00648 · Cited by 403
Self-citationMCL-JCV: A JND-based H.264/AVC video quality assessment dataset
Wang, H., Gan, W., Hu, S., Lin, J. Y., Jin, L., Song, L., Wang, P., Katsavounidis, I., Aaron, A., & Kuo, C.-C. J. (2016) · 2016 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2016.7532610 · Cited by 319
Video Compression Through Image Interpolation
Wu, C.-Y., Singhal, N., & Krähenbühl, P. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_26 · Cited by 316
Joint Autoregressive and Hierarchical Priors for Learned Image Compression
Minnen, D. C., Ballé, J., & Toderici, G. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.02736 · Cited by 310
Self-citationDensity Modeling of Images using a Generalized Normalization Transformation
Ballé, J., Laparra, V., & Simoncelli, E. P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06281 · Cited by 308
Self-citationReliable Estimation of Dense Optical Flow Fields with Large Displacements
Alvarez, L., Weickert, J., & Sánchez, J. (2000) · International Journal of Computer Vision · doi:10.1023/a:1008170101536 · Cited by 299
Variable Rate Deep Image Compression With a Conditional Autoencoder
Choi, Y., El-Khamy, M., & Lee, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00324 · Cited by 274
Rippel, O., Nair, S., Lew, C., Branson, S., Anderson, A., & Bourdev, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00355 · Cited by 223
Video Compression With Rate-Distortion Autoencoders
Habibian, A., Rozendaal, T. V., Tomczak, J., & Cohen, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00713 · Cited by 218
Uncertainty Estimates and Multi-hypotheses Networks for Optical Flow
Ilg, E., Çiçek, Ö., Galesso, S., Klein, A., Makansi, O., Hutter, F., & Brox, T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_40 · Cited by 213
End-to-end Optimized Image Compression with Attention Mechanism
Zhou, L., Sun, Z., Wu, X., & Wu, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · Cited by 75
End-to-End Learning of Video Compression using Spatio-Temporal Autoencoders
Pessoa, J., Aidos, H., Tomás, P., et al. (2020) · SiPS ... design and implementation - IEEE Workshop on Signal Processing Systems · doi:10.1109/sips50750.2020.9195249 · Cited by 33
Cited by
Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications
Gunduz, D., Qin, Z., Aguerri, I. E., Dhillon, H. S., Yang, Z., Yener, A., Wong, K. K., & Chae, C.-B. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3223408 · Cited by 776
One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
Wang, T.-C., Mallya, A., & Liu, M.-Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00991 · Cited by 444
CompressAI: a PyTorch library and evaluation platform for end-to-end compression research
Bégaint, J., Racapé, F., Feltman, S., & Pushparaja, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.03029 · Cited by 284
FVC: A New Framework towards Deep Video Compression in Feature Space
Hu, Z., Lu, G., & Xu, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00155 · Cited by 273
Notice of Removal: VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation
Luo, Z., Chen, D., Zhang, Y., Huang, Y., Wang, L., Shen, Y., Zhao, D., Zhou, J., & Tan, T. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00984 · Cited by 144
Metadata
- Authors
- Agustsson, E., Minnen, D., Johnston, N., Ballé, J., Hwang, S. J., & Toderici, G.
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 8500-8509
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3035195755
- Cited by
- 317
- References
- 30
- FWCI
- 25.10
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.