The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
End-to-End Learnt Image Compression via Non-Local Attention Optimization and Improved Context Modeling
Open accessAbstract
This article proposes an end-to-end learnt lossy image compression approach, which is built on top of the deep nerual network (DNN)-based variational auto-encoder (VAE) structure with Non-Local Attention optimization and Improved Context modeling (NLAIC). Our NLAIC 1) embeds non-local network operations as non-linear transforms in both main and hyper coders for deriving respective latent features and hyperpriors by exploiting both local and global correlations, 2) applies attention mechanism to generate implicit masks that are used to weigh the features for adaptive bit allocation, and 3) implements the improved conditional entropy modeling of latent features using joint 3D convolutional neural network (CNN)-based autoregressive contexts and hyperpriors. Towards the practical application, additional enhancements are also introduced to speed up the computational processing (e.g., parallel 3D CNN-based context prediction), decrease the memory consumption (e.g., sparse non-local processing) and reduce the implementation complexity (e.g., a unified model for variable rates without re-training). The proposed model outperforms existing learnt and conventional (e.g., BPG, JPEG2000, JPEG) image compression methods, on both Kodak and Tecnick datasets with the state-of-the-art compression efficiency, for both PSNR and MS-SSIM quality measurements. We have made all materials publicly accessible at https://njuvision.github.io/NIC for reproducible research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382
Overview of the High Efficiency Video Coding (HEVC) Standard
Sullivan, G. J., Ohm, J.-R., Han, W.-J., & Wiegand, T. (2012) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2012.2221191 · Cited by 8,263
Overview of the H.264/AVC video coding standard
Wiegand, T., Sullivan, G. J., Bjontegaard, G., & Luthra, A. (2003) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2003.815165 · Cited by 8,184
A Non-Local Algorithm for Image Denoising
Buades, A., Coll, B., & Morel, J.-M. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.38 · Cited by 7,616 · cited 1×
“, low-rank modeling [23], joint sparsity [24] and non-local means [25])”
BackgroundMultiscale structural similarity for image quality assessment
Wang, Z., Simoncelli, E. P., & Bovik, A. C. (2004) · The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003 · doi:10.1109/acssc.2003.1292216 · Cited by 6,253 · cited 2×
“MS-SSIM is reported to offer higher correlation with human perception, especially at low bit rates [15].”
BackgroundMethodCalcuation of average PSNR differences between RD-curves
Bjøntegaard, G. (2001) · Medical Entomology and Zoology · Cited by 6,090
Barlow, H. B. (1983) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(83)90030-5 · Cited by 5,424
Ahmed, N., Natarajan, T., & Rao, K. R. (1974) · IEEE Transactions on Computers · doi:10.1109/t-c.1974.223784 · Cited by 5,156
The JPEG still picture compression standard
Wallace, G. K. (1991) · Communications of the ACM · doi:10.1145/103085.103089 · Cited by 3,784
Weighted Nuclear Norm Minimization with Application to Image Denoising
Gu, S., Zhang, L., Zuo, W., & Feng, X. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.366 · Cited by 2,316 · cited 1×
“, low-rank modeling [23], joint sparsity [24] and non-local means [25])”
BackgroundRate-distortion optimization for video compression
Sullivan, G. J., & Wiegand, T. (1998) · IEEE Signal Processing Magazine · doi:10.1109/79.733497 · Cited by 1,727
Non-local sparse models for image restoration
Mairal, J., Bach, F., Ponce, J., Sapiro, G., & Zisserman, A. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459452 · Cited by 1,666
Pixel Recurrent Neural Networks
van den Oord, A., Kalchbrenner, N., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.06759 · Cited by 1,437
Variational image compression with a scale hyperprior
Ballé, J., Minnen, D. C., Singh, S., Hwang, S. J., & Johnston, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.01436 · Cited by 1,318
End-to-end Optimized Image Compression
Ballé, J., Laparra, V., & Simoncelli, E. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01704 · Cited by 1,275
Full Resolution Image Compression with Recurrent Neural Networks
Toderici, G., Vincent, D., Johnston, N., Hwang, S. J., Minnen, D., Shor, J., & Covell, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.577 · Cited by 886
Berger, T. (2003) · Wiley Encyclopedia of Telecommunications · doi:10.1002/0471219282.eot142 · Cited by 717
Generative Adversarial Networks for Extreme Learned Image Compression
Agustsson, E., Tschannen, M., Mentzer, F., Timofte, R., & Van Gool, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00031 · Cited by 575
Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism
Firat, O., Cho, K., & Bengio, Y. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1101 · Cited by 562
Conditional Probability Models for Deep Image Compression
Mentzer, F., Agustsson, E., Tschannen, M., Timofte, R., & Gool, L. V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00462 · Cited by 490
Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
Agustsson, E., Mentzer, F., Tschannen, M., Cavigelli, L., Timofte, R., Benini, L., & Van Gool, L. (2017) · arXiv · doi:10.48550/arxiv.1704.00648 · Cited by 403
Learning Convolutional Networks for Content-Weighted Image Compression
Li, M., Zuo, W., Gu, S., Zhao, D., & Zhang, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00339 · Cited by 400
Johnston, N., Vincent, D., Minnen, D., Covell, M., Singh, S., Chinen, T., Jin Hwang, S., Shor, J., & Toderici, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00461 · Cited by 366
Joint Autoregressive and Hierarchical Priors for Learned Image Compression
Minnen, D. C., Ballé, J., & Toderici, G. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.02736 · Cited by 310
Non-Local Recurrent Network for Image Restoration
Liu, D., Wen, B., Fan, Y., Loy, C. C., & Huang, T. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.02919 · Cited by 252
Practical Full Resolution Learned Lossless Image Compression
Mentzer, F., Agustsson, E., Tschannen, M., Timofte, R., & Van Gool, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01088 · Cited by 222
Residual Non-local Attention Networks for Image Restoration
Zhang, Y., Li, K., Li, K., Zhong, B., & Fu, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.10082 · Cited by 172
Efficient Nonlinear Transforms for Lossy Image Compression
Ballé, J. (2018) · Picture Coding Symposium · doi:10.1109/pcs.2018.8456272 · Cited by 78
Autoencoder Based Image Compression: Can the Learning be Quantization Independent?
Dumas, T., Roumy, A., & Guillemot, C. M. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462263 · Cited by 69
Layered Image Compression Using Scalable Auto-Encoder
Jia, C., Liu, Z., Wang, Y., Ma, S., & Gao, W. (2019) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr.2019.00087 · Cited by 30
Self-citationLearned Scalable Image Compression with Bidirectional Context Disentanglement Network
Zhang, Z., Chen, Z., Lin, J., & Li, W. (2019) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2019.00249 · Cited by 24
Extreme Image Coding via Multiscale Autoencoders with Generative Adversarial Optimization
Huang, C., Liu, H., Chen, T., Shen, Q., & Ma, Z. (2019) · IEEE International Conference on Visual Communications and Image Processing (VCIP) · doi:10.1109/vcip47243.2019.8966059 · Cited by 17
Self-citationPractical Stacked Non-local Attention Modules for Image Compression
Liu, H., Chen, T., Shen, Q., & Ma, Z. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · Cited by 16
Self-citationLearned Lossless Image Compression with A Hyperprior and Discretized Gaussian Mixture Likelihoods
Cheng, Z., Sun, H., Takeuchi, M., & Katto, J. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053413 · Cited by 15
Gated Context Model with Embedded Priors for Deep Image CompressionPreprint
Liu, H., Chen, T., Guo, P., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.10480 · Cited by 7
Self-citation
Cited by
He, D., Yang, Z., Peng, W., Ma, R., Qin, H., & Wang, Y. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00563 · Cited by 455
Self-citation
Metadata
- Authors
- Chen, T., Liu, H., Ma, Z., Shen, Q., Cao, X., & Wang, Y.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2021
- Volume
- 30
- Pages
- 3179-3191
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W3131427579
- Cited by
- 346
- References
- 42
- FWCI
- 20.81
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.