DeepCABAC: A Universal Compression Algorithm for Deep Neural Networks
Open accessAbstract
In the past decade deep neural networks (DNNs) have shown state-of-the-art performance on a wide range of complex machine learning tasks. Many of these results have been achieved while growing the size of DNNs, creating a demand for efficient compression and transmission of them. In this work we present DeepCABAC, a universal compression algorithm for DNNs that is based on applying Context-based Adaptive Binary Arithmetic Coder (CABAC) to the DNN parameters. CABAC was originally designed for the H.264/AVC video coding standard and became the state-of-the-art for the lossless compression part of video compression. DeepCABAC applies a novel quantization scheme that minimizes a rate-distortion function while simultaneously taking the impact of quantization to the DNN performance into account. Experimental results show that DeepCABAC consistently attains higher compression rates than previously proposed coding techniques for DNN compression. For instance, it is able to compress the VGG16 ImageNet model by x63.6 with no loss of accuracy, thus being able to represent the entire network with merely 9 MB. The source code for encoding and decoding can be found at https://github.com/fraunhoferhhi/DeepCABAC.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the classical congruent non-i.i.d. FL setting, performance will typically degrade if clients with varying distributions are separated into different clusters due to restricted knowledge transfer between clients in different clusters.
“In the classical congruent non-i.i.d.FL setting described in [1] where one single model can be learned, performance will typically degrade if clients with varying distributions are separated into different clusters due to the restricted knowledge transfer between clients in different clusters.”
From Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints (2021) · cites this paper as [1]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 73 · Cited by 113
Research works printed but not linked: 7
- S. Han, J. Pool, J. Tran, and W. J. Dally, "Learning both weights and connections for efficient neural networks," in Proc. Adv. Neural Inf. Process. Syst., 2015, pp. 1135-1143.
- S. Han, H. Mao, and W. J. Dally, "Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding," 2015, arXiv:1510.00149. · arXiv
- J. Redmon and A. Farhadi, "Yolov3: An incremental improvement," 2018, arXiv:1804.02767. · arXiv
- Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut, "Albert: A lite BERT for self-supervised learning of language representa- tions," 2019, arXiv:1909.11942. · arXiv
- G. Hinton, O. Vinyals, and J. Dean, "Distilling the knowledge in a neural network," 2015, arXiv:1503.02531. · arXiv
- M. Tan and Q. V. Le, "Mixconv: Mixed depthwise convolutional kernels," CoRR, vol. abs/1907.09595, 2019. [Online]. Available: http://arxiv.org/ abs/1907.09595 · arXiv
- F. Sattler, K.-R. Müller, and W. Samek, "Clustered federated learning: Model-agnostic distributed multi-task optimization under privacy con- straints," 2019, arXiv:1910.01991. · arXiv
- Authors
- Wiedemann, S., Kirchhoffer, H., Matlage, S., Haase, P., Marban, A., Marinc, T., Neumann, D., Nguyen, T., Schwarz, H., Wiegand, T., Marpe, D., & Samek, W.
- Affiliations
- Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute (DE)
- Fraunhofer Heinrich Hertz Institute, Berlin, Germany
- ORCID
- Venue
- IEEE Journal of Selected Topics in Signal Processing
- Published
- 2020-05
- Volume
- 14
- Issue
- 4
- Pages
- 700-714
- ISSN
- 1932-4553 (ISSN-L)
- 1941-0484
- OpenAlex
- W2965940031
- Cited by
- 113
- References
- 73
- Citations by year
- 2026: 5 · 2025: 13 · 2024: 19 · 2023: 29 · 2022: 21 · 2021: 16 · 2020: 10
- FWCI
- 7.33
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- IEEE Journal of Selected Topics in Signal Processing (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion, other-oa)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- arXiv (Cornell University)
- Funders
- German Ministry for Education through the Berlin Big Data Center (01IS14013A)
- Berlin Center for Machine Learning (01IS18037I)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Data Compression Techniques
- Physical Sciences › Computer Science › Signal Processing › Video Coding and Compression Technologies
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- deep neural network compression, CABAC, context-based adaptive binary arithmetic coding, rate-distortion optimization, quantization, model compression, VGG16, lossless compression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
