Bit Efficient Quantization for Deep Neural Networks
Open accessAbstract
Quantization for deep neural networks have afforded models for edge devices that use less on-board memory and enable efficient low-power inference. In this paper, we present a comparison of model-parameter driven quantization approaches that can achieve as low as 3-bit precision without affecting accuracy. The post-training quantization approaches are data-free, and the resulting weight values are closely tied to the dataset distribution on which the model has converged to optimality. We show quantization results for a number of state-of-art deep neural networks (DNN) using large dataset like ImageNet. To better analyze quantization results, we describe the overall range and local sparsity of values afforded through various quantization schemes. We show the methods to lower bit-precision beyond quantization limits with object class clustering.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The proposed method has an accuracy drop 0.21% less than Miyashita et al. [39], despite Miyashita et al. using 1 bit more for the activation parameter.
“However, despite using 1 bit more for the activation parameter than the proposed method, the accuracy drop of Miyashita et al. [39] is greater by 0.21% than that of the proposed method.”
From Zero-Centered Fixed-Point Quantization With Iterative Retraining for Deep Convolutional Neural Network-Based Object Detectors (2021) · cites this paper as [39]
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Natural Language Processing (almost) from ScratchPreprint
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 6,882 · see citing works
Framewise phoneme classification with bidirectional LSTM and other neural network architectures
Graves, A., & Schmidhuber, J. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.06.042 · Cited by 5,882 · see citing works
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408 · see citing works
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., & Kalenichenko, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00286 · Cited by 4,114 · see citing works
BinaryConnect: Training Deep Neural Networks with binary weights during propagations
Courbariaux, M., Bengio, Y., & David, J.-P. (2015) · arXiv · doi:10.48550/arxiv.1511.00363 · Cited by 2,180 · see citing works
Neural Architecture Search: A Survey
Elsken, T., Metzen, J. H., & Hutter, F. (2018) · arXiv · doi:10.48550/arxiv.1808.05377 · Cited by 1,763 · see citing works
DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
Zhou, S., Wu, Y., Ni, Z., Zhou, X., Wen, H., & Zou, Y. (2016) · arXiv · doi:10.48550/arxiv.1606.06160 · Cited by 1,690 · see citing works
Deep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · arXiv · doi:10.48550/arxiv.1502.02551 · Cited by 1,364 · see citing works
Incremental Network Quantization: Towards Lossless CNNs with Low-Precision WeightsPreprint
Zhou, A., Yao, A., Guo, Y., Xu, L., & Chen, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.03044 · Cited by 963 · see citing works
Quantizing deep convolutional networks for efficient inference: A whitepaper
Krishnamoorthi, R. (2018) · arXiv · doi:10.48550/arxiv.1806.08342 · Cited by 753 · see citing works
Li, F., Liu, B., Wang, X., Zhang, B., & Yan, J. (2016) · arXiv · doi:10.48550/arxiv.1605.04711 · Cited by 732 · see citing works
Data-Free Quantization Through Weight Equalization and Bias Correction
Nagel, M., Baalen, M. V., Blankevoort, T., & Welling, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00141 · Cited by 500 · see citing works
A Survey on Methods and Theories of Quantized Neural NetworksPreprint
Guo, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.04752 · Cited by 249 · see citing works
A Quantization-Friendly Separable Convolution for MobileNets
Sheng, T., Feng, C., Zhuo, S., Zhang, X., Shen, L., & Aleksic, M. (2018) · Workshop on Energy Efficient Machine Learning and Cognitive Computing for Embedded Applications (EMC2) · doi:10.1109/emc2.2018.00011 · Cited by 124 · see citing works
Relaxed Quantization for Discretized Neural Networks
Louizos, C., Reisser, M., Blankevoort, T., Gavves, E., & Welling, M. (2018) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1810.01875 · Cited by 64 · see citing works
Probabilistic Binary Neural NetworksPreprint
Peters, J. W. T., & Welling, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03368 · Cited by 43 · see citing works
BitNet: Bit-Regularized Deep Neural NetworksPreprint
Raghavan, A., Amer, M., Chai, S. M., & Taylor, G. W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.04788 · Cited by 9 · see citing works
Self-citationParajuli, S., Raghavan, A., & Chai, S. M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.04985 · Cited by 4 · see citing works
Self-citation
Flagged first, then by citations
2 of the 21 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Loss aware post-training quantization
Nahshan, Y., Chmiel, B., Baskin, C., Zheltonozhskii, E., Banner, R., Bronstein, A. M., & Mendelson, A. (2021) · Machine Learning · doi:10.1007/s10994-021-06053-z · Cited by 155 · see citing works
Deep learning in electron microscopy
Ede, J. M. (2020) · Machine Learning Science and Technology · doi:10.1088/2632-2153/abd614 · Cited by 105 · see citing works
Kim, S., & Kim, H. (2021) · IEEE Access · doi:10.1109/access.2021.3054879 · Cited by 69 · see citing works
Freire, P. J., Napoli, A., Spinnler, B., Anderson, M., Ron, D. A., Schairer, W., Bex, T., Costa, N. M. S., Turitsyn, S. K., & Prilepsky, J. E. (2023) · Journal of Lightwave Technology · doi:10.1109/jlt.2023.3234327 · Cited by 58 · see citing works
Toward Reliable Compact Modeling of Multilevel 1T-1R RRAM Devices for Neuromorphic Systems
Pérez-Bosch Quesada, E., Romero-Zaliz, R., Pérez, E., Kalishettyhalli Mahadevaiah, M., Reuben, J., Schubert, M. A., Jiménez-Molinos, F., Roldán, J. B., & Wenger, C. (2021) · Electronics · doi:10.3390/electronics10060645 · Cited by 54 · see citing works
EasyQuant: Post-training Quantization via Scale Optimization
Wu, D., Tang, Q., Zhao, Y., Zhang, M., Fu, Y., & Zhang, D. (2020) · arXiv · doi:10.48550/arxiv.2006.16669 · Cited by 44 · see citing works
Self-citationGraft: Efficient Inference Serving for Hybrid Deep Learning With SLO Guarantees via DNN Re-Alignment
Wu, J. L., Wang, L., Jin, Q., & Liu, F. (2023) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2023.3340518 · Cited by 31 · see citing works
Optimization of Multi-Level Operation in RRAM Arrays for In-Memory Computing
Pérez, E., Pérez-Ávila, A. J., Romero‐Zaliz, R., Mahadevaiah, M. K., Quesada, E. P.-B., Roldán, J. B., Jiménez-Molinos, F., & Wenger, C. (2021) · Electronics · doi:10.3390/electronics10091084 · Cited by 29 · see citing works
AdAM: Adaptive Approximate Multiplier for Fault Tolerance in DNN Accelerators
Taheri, M., Cherezova, N., Nazari, S., Azarpeyvand, A., Ghasempouri, T., Daneshtalab, M., Raik, J., & Jenihhin, M. (2024) · IEEE Transactions on Device and Materials Reliability · doi:10.1109/tdmr.2024.3523386 · Cited by 23 · see citing works
Bhalgaonkar, S., Munot, M. V., & Anuse, A. D. (2024) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2024.109180 · Cited by 22 · see citing works
Quantization-Aware NN Layers with High-throughput FPGA Implementation for Edge AI
Pistellato, M., Bergamasco, F., Bigaglia, G., Gasparetto, A., Albarelli, A., Boschetti, M., & Passerone, R. (2023) · Sensors · doi:10.3390/s23104667 · Cited by 23 · see citing works
TinyMLOps: Operational Challenges for Widespread Edge AI Adoption
Leroux, S., Simoens, P., Lootus, M., Thakore, K., & Sharma, A. (2022) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw55747.2022.00160 · Cited by 22 · see citing works
Integrated Photonic Neural Networks for Equalizing Optical Communication Signals: A Review
Silva, L. C. B., Marciano, P. R. N., Pontes, M. J., Monteiro, M. E., André, P. S. D. B., & Segatto, M. E. V. (2025) · Photonics · doi:10.3390/photonics12010039 · Cited by 18 · see citing works
Pan, Z., Li, Y., Guan, Z., Liang, M., Li, A., Wang, J., & Kou, F. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128672 · Cited by 18 · see citing works
Machine Learning Advances aiding Recognition and Classification of Indian Monuments and Landmarks
Paul, A. J., Ghose, S., Aggarwal, K., Nethaji, N., Pal, S., & Purkayastha, A. D. (2021) · IEEE Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON) · doi:10.1109/upcon52273.2021.9667619 · Cited by 18 · see citing works
Huang, Y., Chen, K., Shao, Z., Bai, Y., Huang, Y., Du, Y., Du, L., & Wang, Z. (2021) · International SoC Design Conference · doi:10.1109/isocc53507.2021.9613969 · Cited by 14 · see citing works
RCT: Resource Constrained Training for Edge AI
Huang, T., Luo, T., Yan, M., Zhou, J. T., & Goh, R. S. M. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3190451 · Cited by 13 · see citing works
Paul, A. J. (2021) · IEEE Madras Section Conference (MASCON) · doi:10.1109/mascon51689.2021.9563584 · Cited by 11 · see citing works
AI/ML for Mission Processing Onboard Satellites
Veyette, M. J., Aylor, K., Stafford, D., Herrera, M. R., Jumani, S., Lineberry, C., Macklen, C., Maxwell, E., Stiles, R., & Jenkins, M. (2022) · AIAA SciTech Forum · doi:10.2514/6.2022-1472 · Cited by 10 · see citing works
Manjari, K., Verma, M., Singal, G., & Kumar, N. (2022) · Cognitive Computation · doi:10.1007/s12559-022-10020-8 · Cited by 9 · see citing works
WinDConv: A Fused Datapath CNN Accelerator for Power-Efficient Edge Devices
Mahale, G., Udupa, P., Chandrasekharan, K. K., & Lee, S. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3013096 · Cited by 9 · see citing works
Zero and Narrow-Width Value-Aware Compression for Quantized Convolutional Neural Networks
Jang, M., Kim, J., Nam, H., & Kim, S. (2023) · IEEE Transactions on Computers · doi:10.1109/tc.2023.3315051 · Cited by 8 · see citing works
Applying convolutional neural networks for limited-memory application
Dang, X.-K., Truong, H.-N., Nguyen, V.-C., & Pham, T.-D.-A. (2020) · TELKOMNIKA (Telecommunication Computing Electronics and Control) · doi:10.12928/telkomnika.v19i1.16232 · Cited by 8 · see citing works
Development of a Lightweight Floating Object Detection Algorithm
Xian, R., Tang, L., & Liu, S. (2024) · Water · doi:10.3390/w16111633 · Cited by 7 · see citing works
Efficient Quantization Techniques for Deep Neural Networks
Jiang, C. (2021) · doi:10.1109/conf-spml54095.2021.00059 · Cited by 7 · see citing works
A Preliminary Analysis on Software Frameworks for the Development of Spiking Neural Networks
García-Vico, Á. M., & Herrera, F. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86271-8_47 · Cited by 7 · see citing works
Oh, S., Bae, C., Cho, J., Lee, S., & Jung, Y. (2021) · Sensors · doi:10.3390/s21113906 · Cited by 7 · see citing works
A Spike in Performance: Training Hybrid-Spiking Neural Networks with Quantized Activation FunctionsPreprint
Voelker, A. R., Rasmussen, D., & Eliasmith, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.03553 · Cited by 12 · see citing works
Toward Efficient Convolutional Neural Networks With Structured Ternary Patterns
Kyrkou, C. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3380827 · Cited by 6 · see citing works
Harnessing Approximate Computing for Machine Learning
Shakibhamedan, S., Aminifar, A., Vassallo, L., & TaheriNejad, N. (2024) · IEEE Computer Society Annual Symposium on VLSI · doi:10.1109/isvlsi61997.2024.00110 · Cited by 6 · see citing works
Memory Access Optimization for On-Chip Transfer Learning
Hussain, M. A., & Tsai, T. (2021) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2021.3055281 · Cited by 5 · see citing works
Compressed Object DetectionPreprint
Muhawenayo, G., & Gkioxari, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.02896 · Cited by 5 · see citing works
Nguyen, L. H., Tran, K. D., Zeng, X., & Tran, K. P. (2024) · Springer series in reliability engineering · doi:10.1007/978-3-031-71495-5_5 · Cited by 4 · see citing works
Dynamically throttleable neural networks
Liu, H., Parajuli, S., Hostetler, J., Chai, S. M., & Bhanu, B. (2022) · Machine Vision and Applications · doi:10.1007/s00138-022-01311-z · Cited by 9 · see citing works
Self-citationFixed Point Analysis Workflow for efficient Design of Convolutional Neural Networks in Hearing Aids
Klein, S., Kantic, J., & Blume, H. (2021) · Current Directions in Biomedical Engineering · doi:10.1515/cdbme-2021-2201 · Cited by 4 · see citing works
Liu, F., Wenbo, Z., Wang, Y., Changzhi, D., & Jiang, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.03903 · Cited by 4 · see citing works
Hardware-Efficient ASIC CNN Accelerator with Approximate Computing for Edge AI
Rajalakshmi, R., Shukla, V. K., Sivakumar, P., Baanu, B. B., Kumari, L. K., & Subhashini, G. K. (2025) · International Conference on Electronics and Sustainable Communication Systems (ICESC) · doi:10.1109/icesc65114.2025.11212300 · Cited by 3 · see citing works
Subtensor Quantization for Mobilenets
Dinh, T., Melnikov, A., Daskalopoulos, V., & Chai, S. M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-68238-5_10 · Cited by 3 · see citing works
Self-citationTaing, T., Khan, Y. F., Aliev, H., & Kim, H. (2025) · IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia) · doi:10.1109/icce-asia67487.2025.11263725 · Cited by 2 · see citing works
Area-efficient AdderNet hardware accelerator with merged adder tree structure
Seo, G., & Ryu, S. (2023) · IEICE Electronics Express · doi:10.1587/elex.20.20230427 · Cited by 2 · see citing works
PQ Bench: Benchmarking Pruning and Quantization Techniques
Schulze, J., Strassenburg, N., & Rabl, T. (2025) · Workshop on Data Management for End-to-End Machine Learning (DEEM) · doi:10.1145/3735654.3735940 · Cited by 2 · see citing works
DNA-TEQ: An Adaptive Exponential Quantization of Tensors for DNN Inference
Khabbazan, B., Riera, M., & González, A. M. (2023) · IEEE International Conference on High Performance Computing, Data, and Analytics · doi:10.1109/hipc58850.2023.00015 · Cited by 2 · see citing works
A Novel Sensitivity Metric For Mixed-Precision Quantization With Synthetic Data Generation
Lee, D., Cho, M., Lee, S., Song, J., & Choi, C. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506527 · Cited by 2 · see citing works
State of the Art of Machine Learning
Hossain, E. (2023) · Springer eBooks · doi:10.1007/978-3-031-46990-9_7 · Cited by 2 · see citing works
LoCaLUT: Harnessing Capacity–Computation Tradeoffs for LUT-Based Inference in DRAM-PIM
Hong, J., Shin, C., Kim, S., Noh, S. U., Kwon, T., Park, S., Kim, H., Kim, Y., & Lee, J. (2026) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca68181.2026.11408523 · Cited by 1 · see citing works
Evaluation of Quantization Techniques for Deep Neural Networks
Li, Z. (2021) · Communications in computer and information science · doi:10.1007/978-981-16-8885-0_12
In-Cache Processing with Power-of-Two Quantization for Fast CNN Inference on CPUs
Woo, J., Lee, S., Kim, S., Byeon, G., & Hong, S. (2023) · International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC) · doi:10.1109/itc-cscc58803.2023.10212854
Multi-band Convolutional Neural Network for Efficient Utilization of Model Parameters
Mondal, M., Das, B., Lall, B., Singh, P., Roy, S. D., & Joshi, S. D. (2023) · International Conference on Computer, Electronics & Electrical Engineering & their Applications (IC2E3) · doi:10.1109/ic2e357697.2023.10262420
Performance of precision auto-tuned neural networks
Ferro, Q., Graillat, S., Hilaire, T., & Jézéquel, F. (2023) · IEEE International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC) · doi:10.1109/mcsoc60832.2023.00092
Low-Complexity Vision Transformer with Adaptive Channel Partitioning Method
Lee, S., Oh, S.-S., Han, S.-H., Lim, S., Lee, J., & Sim, D. (2024) · Journal of Broadcast Engineering · doi:10.5909/jbe.2024.29.3.291
- Authors
- Nayak, P., Zhang, D., & Chai, S.
- Affiliations
- SRI International (US)
- Venue
- 2019 Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing - NeurIPS Edition (EMC2-NIPS)
- Publisher
- IEEE
- Published
- 2019-12
- Pages
- 52-56
- OpenAlex
- W3176483701
- Cited by
- 51
- References
- 21
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 8 · 2023: 10 · 2022: 6 · 2021: 15 · 2020: 7
- FWCI
- 2.28
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Workshop on Energy Efficient Machine Learning and Cognitive Computing for Embedded Applications (EMC2) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Low-power high-performance VLSI design
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- neural network quantization, low-bit precision, post-training quantization, data-free quantization, edge devices, low-power inference, weight sparsity, ImageNet, model compression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
