Energy Compaction-Based Image Compression Using Convolutional AutoEncoder
The DOI matches its Crossref record; 34 of the 39 references it cites were found in a registry or an index.
Abstract
Image compression has been an important research topic for many decades. Recently, deep learning has achieved great success in many computer vision tasks, and its use in image compression has gradually been increasing. In this paper, we present an energy compaction-based image compression architecture using a convolutional autoencoder (CAE) to achieve high coding efficiency. Our main contributions include three aspects: 1) we propose a CAE architecture for image compression by decomposing it into several down(up)sampling operations; 2) for our CAE architecture, we offer a mathematical analysis on the energy compaction property and we are the first work to propose a normalized coding gain metric in neural networks, which can act as a measurement of compression capability; 3) based on the coding gain metric, we propose an energy compaction-based bit allocation method, which adds a regularizer to the loss function during the training stage to help the CAE maximize the coding gain and achieve high compression efficiency. The experimental results demonstrate our proposed method outperforms BPG (HEVC-intra), in terms of the MS-SSIM quality metric. Additionally, we achieve better performance in comparison with existing bit allocation methods, and provide higher coding efficiency compared with state-of-the-art learning compression methods at high bit rates.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It can be even used to generate fake satellite images of the Earth to contain objects that do not really exist to confuse military analysts”
From Deep learning for deepfakes creation and detection: A survey (2022) · cites this paper as [17]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“Plainunitsusetwoconvolutionalfilters,whilea residualunitreplacesoneconvolutionalfilterwitharesidual block[27].”
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“To train the CAE, the model was optimized using Adam [28] with a batch size of 16.”
MethodA Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1×
“We used a subset of the ImageNet database [29], which consists of 5,500 images and cropped them into millions of 128 × 128 samples to train the CAE network.”
MethodOverview of the High Efficiency Video Coding (HEVC) Standard
Sullivan, G. J., Ohm, J.-R., Han, W.-J., & Wiegand, T. (2012) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2012.2221191 · Cited by 8,263 · cited 1×
“Conventional image compression standards, such as JPEG [1], JPEG2000 [2], and BPG, which uses the intra-coded HEVC [3], rely on the hand-crafted encoder-decoder (codec) architecture.”
MethodShi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.207 · Cited by 7,570 · cited 1×
“In [24], the authors pointed out that super resolution is achieved more efficiently by first convolving the images and then upsampling, instead of first upsampling and then convolving.”
BackgroundExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436 · cited 1×
“Inear-lieryears,theautoencoderwaswidelyappliedtodimensionality reduction,thecompactrepresentationsofimages,andgenera-tivelearningmodels[4].”
Multiscale structural similarity for image quality assessment
Wang, Z., Simoncelli, E. P., & Bovik, A. C. (2004) · The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003 · doi:10.1109/acssc.2003.1292216 · Cited by 6,253 · cited 1×
“To achieve high subjective quality in the final experiments, we used a popular MS-SSIM [31] as the distortion term defined by 1 − MS - SSIM(x , ˆx) .”
MethodThe JPEG still picture compression standard
Wallace, G. K. (1991) · Communications of the ACM · doi:10.1145/103085.103089 · Cited by 3,784 · cited 3×
“This single channel m was further expanded into a maskM ∈ R H 2n × W 2n ×K with the same dimensionality as y, and the value of the mask was clipped into the range of [0,1] as follows:”
MethodThe JPEG still picture compression standard
Wallace, G. K. (1992) · IEEE Transactions on Consumer Electronics · doi:10.1109/30.125072 · Cited by 2,851
No, P. (1986) · Cited by 1,532
Barratt, K. H. (1985) · Electronics and Power · doi:10.1049/ep.1985.0201 · Cited by 1,523 · cited 1×
“Previous digital coding theories [25] have provided theoretical solutions to the allocation of bits from the viewpoint linear system energy.”
BackgroundEnd-to-end Optimized Image Compression
Ballé, J., Laparra, V., & Simoncelli, E. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01704 · Cited by 1,275 · cited 5×
“Theis et al. [13] and Ballé et al. [14] first proposed the architecture of convolutional autoencoders for learned image compression.”
BackgroundMethodInfluentialFull Resolution Image Compression with Recurrent Neural Networks
Toderici, G., Vincent, D., Johnston, N., Hwang, S. J., Minnen, D., Shor, J., & Covell, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.577 · Cited by 886 · cited 1×
“This approach was further extended in [11] to compress full-resolution images.”
BackgroundAn overview of the JPEG 2000 still image compression standard
Rabbani, M., & Joshi, R. (2002) · Signal Processing: Image Communication · doi:10.1016/s0923-5965(01)00024-8 · Cited by 730 · cited 1×
“JPEG2000 [2] uses multi-scale DWT, uniform quantization with deadzone and an adaptive arithmetic coder.”
MethodGenerative Adversarial Networks for Extreme Learned Image Compression
Agustsson, E., Tschannen, M., Mentzer, F., Timofte, R., & Van Gool, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00031 · Cited by 575 · cited 3×
“[22] proposed to fully synthesize the unimportant regions in the decoded image from a semantic label map to achieve extremely low bit rates.”
BackgroundMethodConditional Probability Models for Deep Image Compression
Mentzer, F., Agustsson, E., Tschannen, M., Timofte, R., & Gool, L. V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00462 · Cited by 490
Real-Time Adaptive Image CompressionPreprint
Rippel, O., & Bourdev, L. D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.05823 · Cited by 484 · cited 5×
“Existing deep learning based image compression studies have already achieved promising performance, as has been reported in [17], [20], by focusing on the selection of neural network architectures and the design of differentiable quantization and entropy estimation models, which are critical components for end-to-end…”
BackgroundMethodInfluentialSoft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations
Agustsson, E., Mentzer, F., Tschannen, M., Cavigelli, L., Timofte, R., Benini, L., & Van Gool, L. (2017) · arXiv · doi:10.48550/arxiv.1704.00648 · Cited by 403 · cited 4×
“Several quantization approximations have been proposed, such as uniform noise approximation [14] and soft histogram [16].”
MethodInfluentialLearning Convolutional Networks for Content-Weighted Image Compression
Li, M., Zuo, W., Gu, S., Zhao, D., & Zhang, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00339 · Cited by 400
Johnston, N., Vincent, D., Minnen, D., Covell, M., Singh, S., Chinen, T., Jin Hwang, S., Shor, J., & Toderici, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00461 · Cited by 366
Deep Convolutional AutoEncoder-based Lossy Image Compression
Cheng, Z., Sun, H., Takeuchi, M., & Katto, J. (2018) · 2018 Picture Coding Symposium (PCS) · doi:10.1109/pcs.2018.8456308 · Cited by 241
Self-citationLossy Image Compression with Compressive Autoencoders
Huszár, F., Theis, L., Shi, W., & Cunningham, A. (2020) · Apollo (University of Cambridge) · doi:10.17863/cam.51995 · Cited by 237
Lossy Image Compression with Compressive Autoencoders
Theis, L., Shi, W., Cunningham, A., & Huszár, F. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00395 · Cited by 230 · cited 5×
“In other studies [13], [17], the derivation was replaced in the back propagation only, but it was guaranteed that the quantized value was correct in the forward propagation.”
BackgroundMethodInfluentialSanturkar, S., Budden, D., & Shavit, N. (2018) · 2018 Picture Coding Symposium (PCS) · doi:10.1109/pcs.2018.8456298 · Cited by 190
Performance evaluation of subband coding and optimization of its filter coefficients
Katto, J., & Yasuda, Y. (1991) · Journal of Visual Communication and Image Representation · doi:10.1016/1047-3203(91)90011-4 · Cited by 147
Self-citationVariable Rate Image Compression with Recurrent Neural Networks
Toderici, G. D., O’Malley, S. M., Hwang, S. J., Vincent, D., Minnen, D. C., Baluja, S., Covell, M., & Sukthankar, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06085 · Cited by 94 · cited 2×
“Mostrecentlearningbasedcompression approaches,includingrecurrentneuralnetworks[10]–[12],con-volutionalneuralnetworks[13]–[19]andgenerativeadversarial networks[20]–[22],havealladoptedtheautoencoderarchitec-ture.”
MethodClosed-Form Optimization on Saliency-Guided Image Compression for HEVC-MSP
Li, S., Xu, M., Ren, Y., & Wang, Z. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2721544 · Cited by 70 · cited 1×
“was incorporated into image compression optimization for HEVC-MSP in [5], and new color image compression implementations were proposed in [6] based on the YUV420 chroma”
MethodPerformance evaluation of subband coding and optimization of its filter coefficients
Katto, J., & Yasuda, Y. (1991) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.50249 · Cited by 65 · cited 4×
“In a linear system,Bk can be computed by the square-sum of the filter bank coefficients [26].”
BackgroundMethodInfluentialSelf-citationAn End-to-End Compression Framework Based on Convolutional Neural Networks
Wen, T., Jiang, F., Zhang, S., Ren, J., Shi, W., Zuo, W., Guo, X., & Zhao, D. (2017) · DCC · doi:10.1109/dcc.2017.54 · Cited by 32
Perceptual Quality Study on Deep Learning Based Image Compression
Cheng, Z., Akyazi, P., Sun, H., Katto, J., & Ebrahimi, T. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803824 · Cited by 30
Self-citationCheng, Z., Sun, H., Takeuchi, M., & Katto, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.00270 · Cited by 26 · cited 1×
“For example, in [7] the downsampling of the input images was applied before BPG and used super-resolution neural networks as a post filter.”
MethodSelf-citationCross-Space Distortion Directed Color Image Compression
Zhu, S., Li, M., Chen, C., Liu, S., & Zeng, B. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2749162 · Cited by 24 · cited 1×
“was incorporated into image compression optimization for HEVC-MSP in [5], and new color image compression implementations were proposed in [6] based on the YUV420 chroma”
MethodCNN-Optimized Image Compression with Uncertainty based Resource Allocation
Chen, Z., Li, Y., Liu, F., Liu, Z., Pan, X., Sun, W., Wang, Y., Zhou, Y., Zhu, H., & Liu, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · Cited by 8
Research works printed but not linked: 1
- balle, 0, Variational image compression with a hyperprior, Proc Int Conf Learn Representations, 1
Cited by
Deep learning for deepfakes creation and detection: A survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103525 · Cited by 464
Deep Learning for Deepfakes Creation and Detection: A Survey
Nguyen, T. T., Nguyen, Q. V. H., Nguyen, D. T., Nguyen, D. T., Huynh-The, T., Nahavandi, S., Nguyen, T. T., Pham, Q.-V., & Nguyen, C. M. (2019) · arXiv · doi:10.48550/arxiv.1909.11573 · Cited by 464
Ballé, J., Chou, P. A., Minnen, D., Singh, S., Johnston, N., Agustsson, E., Hwang, S. J., & Toderici, G. (2021) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3034501 · Cited by 231
Analysis Survey on Deepfake detection and Recognition with Convolutional Neural Networks
Ahmed, S. R., Sonuc, E., Ahmed, M. R., & Duru, A. D. (2022) · 2022 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA) · doi:10.1109/hora55278.2022.9799858 · Cited by 187
Deep Learning for Deepfakes Creation and Detection: A Survey
Nguyen, T. T., Nguyen, M. C., Nguyen, D. T., Nguyen, D. T., & Nahavandi, S. (2020) · Figshare · doi:10.6084/m9.figshare.12731039.v1 · Cited by 121
Metadata
- Authors
- Cheng, Z., Sun, H., Takeuchi, M., & Katto, J.
- Affiliations
- Japan Science and Technology Agency (JP)
- Venue
- IEEE Transactions on Multimedia
- Published
- 2020-04
- Volume
- 22
- Issue
- 4
- Pages
- 860-873
- ISSN
- 1520-9210 (ISSN-L)
- 1941-0077
- OpenAlex
- W2970340508
- Cited by
- 97
- References
- 39
- Citations by year
- 2026: 7 · 2025: 12 · 2024: 13 · 2023: 16 · 2022: 18 · 2021: 15 · 2020: 14
- FWCI
- 5.49
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Multimedia (publishedVersion)
- Funders
- Japan Society for the Promotion of Science (201914620)
- JST
- Precursory Research for Embryonic Science and Technology (JPMJPR19M5)
- JSPS KAKENHI (15H01684)
- Waseda University (2019Q-049)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Data Compression Techniques
- Physical Sciences › Computer Science › Signal Processing › Video Coding and Compression Technologies
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- image compression, convolutional autoencoder, energy compaction, coding gain, bit allocation, neural network compression, coding efficiency, MS-SSIM
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.