The DOI matches its DataCite record; 20 of the 20 references it cites were found in a registry or an index.
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
Open accessAbstract
Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems with limited hardware resources. To address this limitation, we introduce "deep compression", a three stage pipeline: pruning, trained quantization and Huffman coding, that work together to reduce the storage requirement of neural networks by 35x to 49x without affecting their accuracy. Our method first prunes the network by learning only the important connections. Next, we quantize the weights to enforce weight sharing, finally, we apply Huffman coding. After the first two steps we retrain the network to fine tune the remaining connections and the quantized centroids. Pruning, reduces the number of connections by 9x to 13x; Quantization then reduces the number of bits that represent each connection from 32 to 5. On the ImageNet dataset, our method reduced the storage required by AlexNet by 35x, from 240MB to 6.9MB, without loss of accuracy. Our method reduced the size of VGG-16 by 49x from 552MB to 11.3MB, again with no loss of accuracy. This allows fitting the model into on-chip SRAM cache rather than off-chip DRAM memory. Our compression method also facilitates the use of complex neural networks in mobile applications where application size and download bandwidth are constrained. Benchmarked on CPU, GPU and mobile GPU, compressed network has 3x to 4x layerwise speedup and 3x to 7x better energy efficiency.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“This offers an alternative compression strategy that may be more resilient than pruning, especially under high compression regimes [20].”
From LittleBit: Ultra Low-Bit Quantization via Latent Factorization (2025) · cites this paper as [20]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, Level 2 usually results in higher accuracy, as finetuning helps the quantized model to recover the accuracy degradation, particularly in ultra-low bit precision settings [85].”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [85]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The recent network pruning methods usually take an iterative training and pruning process, i.e., to remove only a small group of unimportant weights after each stage of training, and to repeat those operations [130].”
From Object Detection in 20 Years: A Survey (2023) · cites this paper as [130]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This process enables efficient inference on resource-constrained hardware, such as edge devices and embedded systems, without significantly compromising model accuracy [36-38].”
From Fine-Grained Fault Sensitivity Analysis of Vision Transformers Under Soft Errors (2025) · cites this paper as [36]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In addition, quantization approaches can address the scalability and computational demands by shifting toward quantized versions of existing models using curated, domain-specific data rather than pretraining from scratch [159].”
From Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook. (2024) · cites this paper as [159]
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Threshold pruning methods ranging from LMP [14] to dynamic weight selection via spatial correlation [21], and the recent random initializationbased pruning method [25] (performed to further sparsity the network) have achieved good results.”
From A survey of model compression strategies for object detection (2023) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The methods include pruning [40, 41], Quantization [42-44], Knowledge Distillation [45] and Neural Architecture Search [46].”
From A Survey of Techniques for Optimizing Transformer Inference (2023) · cites this paper as [41]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These challenges have motivated researchers to propose techniques for reducing transformers' size, latency and energy consumption for efficient inference for a wide range of applications.”
From A Survey of Techniques for Optimizing Transformer Inference (2023) · cites this paper as [41]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Our approach provides a clear advantage by enabling a greater compression of the teacher model compared to the previous methods that relied on pruning [10-15], knowledge distillation (KD) [1,3,5,7,8], and the combination of both [16-20].”
From Efficient and Controllable Model Compression through Sequential Knowledge Distillation and Pruning (2023) · cites this paper as [10]
The cited source says what the claim attributes to it.
In the citing paper
“Weight pruning and quantization [20] is a model compression technique that has received attention as one of the most efficient options to reduce the intensive computation and memory usage without compromising accuracy.”
From Optimized Convolutional Neural Network at the IoT edge for image detection using pruning and quantization (2024) · cites this paper as [20]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Synthetic data which does not take the internal statistics into account may not properly represent the real data distribution [85].”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [85]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Recent work has explored differential privacy as a defense against data poisoning [326], and in particular in the federated learning context [199].”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [199]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Compared to Enderich et al. [25] and SYMOG [38], where only the weights are quantized with a xed-point format, although the proposed method used two more bits for the weight parameters, the bits used for activation parameters are signi cantly reduced to 1/8, and the accuracy degradation is also less than both of them.”
From Zero-Centered Fixed-Point Quantization With Iterative Retraining for Deep Convolutional Neural Network-Based Object Detectors (2021) · cites this paper as [38]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“With network slicing, customized and optimized resources can be provided for Edge DL services, which can help reduce latency caused by access networks and support dense access to these services [77].”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [77]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“In particular, an update aggregation protocol named FedCS is developed to allow the centralized server to aggregate as many client updates as possible and to accelerate performance improvement in ML models.”
From Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing (2019) · cites this paper as [52]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Caffe: Convolutional Architecture for Fast Feature EmbeddingPreprint
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 5,299
(1989) · Cited by 3,214
Fast R-CNNPreprint
Girshick, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.08083 · Cited by 2,781
Going Deeper with Convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2014) · arXiv · doi:10.48550/arxiv.1409.4842 · Cited by 1,461
Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
Denton, R., Zaremba, W., Bruna, J., LeCun, Y., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1404.0736 · Cited by 1,414
EIE: Efficient Inference Engine on Compressed Deep Neural Network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., & Dally, W. J. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.30 · Cited by 1,077
Self-citationCompressing Deep Convolutional Networks using Vector QuantizationPreprint
Gong, Y., Liu, L., Yang, M., & Bourdev, L. D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6115 · Cited by 1,047
Compressing Neural Networks with the Hashing Trick
Chen, W., Wilson, J. T., Tyree, S., Weinberger, K. Q., & Chen, Y. (2015) · arXiv · doi:10.48550/arxiv.1504.04788 · Cited by 974
Predicting Parameters in Deep Learning
Denil, M., Shakibi, B., Dinh, L., Ranzato, M., & de Freitas, N. (2013) · arXiv · doi:10.48550/arxiv.1306.0543 · Cited by 853
(2011) · Cited by 713
Comparing Biases for Minimal Network Construction with Back-Propagation
Hanson, S. J., & Pratt, L. (1988) · neural information processing systems · Cited by 480
Fixed-point feedforward deep neural network design using weights +1, 0, and −1
Hwang, K., & Sung, W. (2014) · SiPS ... design and implementation - IEEE Workshop on Signal Processing Systems · doi:10.1109/sips.2014.6986082 · Cited by 258
Provable Bounds for Learning Some Deep RepresentationsPreprint
Arora, S., Bhaskara, A., Ge, R., & Ma, T. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1310.6343 · Cited by 251
Fixed point optimization of deep convolutional neural networks for object recognition
Anwar, S., Hwang, K., & Sung, W. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178146 · Cited by 237
Memory Bounded Deep Convolutional NetworksPreprint
Collins, M. D., & Kohli, P. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.1442 · Cited by 159
Cross-Domain Synthesis of Medical Images Using Efficient Location-Sensitive Deep Network
Nguyen, H. V., Zhou, K., & Vemulapalli, R. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24553-9_83 · Cited by 122
On the Construction of Huffman Trees.
Leeuwen, J. V. (1976) · International Colloquium on Automata, Languages and Programming · Cited by 116
Deep Fried ConvnetsPreprint
Yang, Z., Moczulski, M., Denil, M., de Freitas, N., Smola, A. J., Song, L., & Wang, Z. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.7149 · Cited by 13
Research works printed but not linked: 5
- Han, Song, Pool, Jeff, Tran, John, and Dally, William J. Learning both weights and connections for efficient neural networks. In Advances in Neural Information Processing Systems , 2015.
- Hassibi, Babak, Stork, David G, et al. Second order derivatives for network pruning: Optimal brain surgeon. Advances in neural information processing systems , pp. 164-164, 1993.
- Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. Imagenet classification with deep convolutional neural networks. In NIPS , pp. 1097-1105, 2012.
- Lin, Min, Chen, Qiang, and Yan, Shuicheng. Network in network. arXiv:1312.4400 , 2013. · arXiv
- Str¨ om, Nikko. Phoneme probability estimation with dynamic sparsely connected artificial neural networks. The Free Speech Journal , 1(5):1-41, 1997.
Cited by
Sorted by citations
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects
Li, Z., Liu, F., Yang, W., Peng, S., & Zhou, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3084827 · Cited by 5,273
“[166] proposed deep pruning for CNN to prune networks by learning essential connections.”
BackgroundPyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., & Shao, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00061 · Cited by 5,259
“, ResNet, ResNeXt), there are still some specific modules and operations designed for CNNs and not considered in this work, such as SE [22], SK [35], dilated convolution [73], model pruning [19], and NAS [60].”
MethodGhostNet: More Features From Cheap Operations
Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., & Xu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00165 · Cited by 5,222
Metadata
- Authors
- Han, S., Mao, H., & Dally, W. J.
- Affiliations
- Stanford University (US)
- Tsinghua University (CN)
- Nvidia (United Kingdom) (GB)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1510.00149
- OpenAlex
- W2119144962
- Cited by
- 4,662
- References
- 20
- Citations by year
- 2026: 35 · 2025: 270 · 2024: 497 · 2023: 593 · 2022: 621 · 2021: 736 · 2020: 701 · 2019: 581 · 2018: 394 · 2017: 186 · 2016: 40 · 2015: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- International Conference on Learning Representations
- Funders
- National Science Foundation (2312006)
- Washington State University
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Data Compression Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- neural network compression, neural network pruning, Huffman coding, weight quantization, storage reduction, embedded systems, mobile application, energy efficiency, AlexNet, VGG16, on-chip SRAM
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
