The DOI matches its DataCite record; 25 of the 26 references it cites were found in a registry or an index.
Fixed Point Quantization of Deep Convolutional Networks
Open accessAbstract
In recent years increasingly complex architectures for deep convolution networks (DCNs) have been proposed to boost the performance on image recognition tasks. However, the gains in performance have come at a cost of substantial increase in computation and model storage resources. Fixed point implementation of DCNs has the potential to alleviate some of these complexities and facilitate potential deployment on embedded hardware. In this paper, we propose a quantizer design for fixed point implementation of DCNs. We formulate and solve an optimization problem to identify optimal fixed point bit-width allocation across DCN layers. Our experiments show that in comparison to equal bit-width settings, the fixed point DCNs with optimized bit width allocation offer >20% reduction in the model size without any loss in accuracy on CIFAR-10 benchmark. We also demonstrate that fine-tuning can further enhance the accuracy of fixed point DCNs beyond that of the original floating point model. In doing so, we report a new state-of-the-art fixed point performance of 6.78% error-rate on CIFAR-10 benchmark.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Lin et al. [26] use the signal-to-quantization-noise ratio (SQNR) to ensure that the error resulting from the xed-point quantization is minimized and mathematically assign the bits to each layer for the xed-point quantization. However, mathematically minimizing errors does not necessarily lead to an accuracy reduction.”
From Zero-Centered Fixed-Point Quantization With Iterative Retraining for Deep Convolutional Neural Network-Based Object Detectors (2021) · cites this paper as Lin et al. [26]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Boyd, S., & Vandenberghe, L. (2004) · doi:10.1017/cbo9780511804441 · Cited by 56,601
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196
Return of the Devil in the Details: Delving Deep into Convolutional Nets
Chatfield, K., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.6 · Cited by 3,335
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_23 · Cited by 3,034
OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680
Going Deeper with Convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2014) · arXiv · doi:10.48550/arxiv.1409.4842 · Cited by 1,461
Deep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · arXiv · doi:10.48550/arxiv.1502.02551 · Cited by 1,364
Deng, L., Hinton, G. E., & Kingsbury, B. E. D. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639344 · Cited by 1,259
Compressing Deep Convolutional Networks using Vector QuantizationPreprint
Gong, Y., Liu, L., Yang, M., & Bourdev, L. D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6115 · Cited by 1,047
Compressing Neural Networks with the Hashing Trick
Chen, W., Wilson, J. T., Tyree, S., Weinberger, K. Q., & Chen, Y. (2015) · arXiv · doi:10.48550/arxiv.1504.04788 · Cited by 974
(2011) · Cited by 713
CNN Features off-the-shelf: an Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · arXiv · doi:10.48550/arxiv.1403.6382 · Cited by 607
Visualizing and Understanding Convolutional Neural NetworksPreprint
Zeiler, M. D., & Fergus, R. (2013) · arXiv (Cornell University) · Cited by 442
Neural Networks with Few MultiplicationsPreprint
Lin, Z., Courbariaux, M., Memisevic, R., & Bengio, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.03009 · Cited by 334
Image and Video Compression for Multimedia Engineering: Fundamentals, Algorithms, and Standards
Shi, Y. Q., & Sun, H. (1999) · Thammasat University Digital Collections · Cited by 272
Fixed-point feedforward deep neural network design using weights +1, 0, and −1
Hwang, K., & Sung, W. (2014) · SiPS ... design and implementation - IEEE Workshop on Signal Processing Systems · doi:10.1109/sips.2014.6986082 · Cited by 258
Fixed point optimization of deep convolutional neural networks for object recognition
Anwar, S., Hwang, K., & Sung, W. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178146 · Cited by 237
Overcoming Challenges in Fixed Point Training of Deep Convolutional NetworksPreprint
Lin, D. D., & Talathi, S. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.02241 · Cited by 36
Self-citationA Deep Neural Network Compression Pipeline: Pruning, Quantization, Huffman EncodingPreprint
Han, S., Mao, H., & Dally, W. J. (2015) · arXiv (Cornell University) · Cited by 33
Audio Coding: Theory and Applications
You, Y. (2010) · Digital Access to Libraries (Université catholique de Louvain (UCL), l'Université de Namur (UNamur) and the Université Saint-Louis (USL-B)) · doi:10.1007/978-1-4419-1754-6 · Cited by 14
Research works printed but not linked: 2
- Courbariaux, M., Bengio, Y., and David, J. Low precision arithmetic for deep learning. arXiv:1412.7024 , 2014. · arXiv
- Hinton, G., Vinyals, O., and Dean, J. Distilling knowledge in a neural network. In Proc. Deep Learning and Representation Learning NIPS Workshop , 2014.
Cited by
Sorted by citations
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
“This topic has received a lot of attention in the broader ML community [194, 120, 436, 270, 312], but these methods either do not have a straightforward mapping to federated learning, or make the training process more complex which makes it difficult to adopt.”
BackgroundA Survey of Quantization Methods for Efficient Neural Network Inference
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., & Keutzer, K. (2022) · Low-Power Computer Vision · doi:10.1201/9781003162810-13 · Cited by 1,219
“Specifically, the library leverages fixed-point quantization [112, 151, 259] with power-of-two scaling factors so that quantization and dequantization processes can be carried out efficiently with bit shifting operations.”
LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks
Zhang, D., Yang, J., Ye, D., & Hua, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_23 · Cited by 714
A comprehensive survey on model compression and acceleration
Choudhary, T., Mishra, V. K., Goswami, A., & Jagannathan, S. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09816-7 · Cited by 555
Deep Learning with Low Precision by Half-Wave Gaussian Quantization
Cai, Z., He, X., Sun, J., & Vasconcelos, N. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.574 · Cited by 498
Metadata
- Authors
- Lin, D. D., Talathi, S. S., & Annapureddy, V. S.
- Affiliations
- Qualcomm (United States) (US)
- ORCID
- Talathi, Sachin S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- Pages
- 2849-2858
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1511.06393
- OpenAlex
- W2286365479
- Cited by
- 684
- References
- 26
- Citations by year
- 2026: 1 · 2025: 21 · 2024: 36 · 2023: 48 · 2022: 71 · 2021: 109 · 2020: 141 · 2019: 130 · 2018: 86 · 2017: 31 · 2016: 10
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- International Conference on Machine Learning
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Numerical Methods and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- fixed-point quantization, deep convolutional neural networks, bit-width allocation, quantizer design, model size reduction, fine-tuning, CIFAR-10, image classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
