The DOI matches its DataCite record; 56 of the 57 references it cites were found in a registry or an index.
Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation
Open accessAbstract
Quantization techniques can reduce the size of Deep Neural Networks and improve inference latency and throughput by taking advantage of high throughput integer instructions. In this paper we review the mathematical aspects of quantization parameters and evaluate their choices on a wide range of neural network models for different application domains, including vision, speech, and language. We focus on quantization techniques that are amenable to acceleration by processors with high-throughput integer math pipelines. We also present a workflow for 8-bit quantization that is able to maintain accuracy within 1% of the floating-point baseline on all networks studied, including models that are more difficult to quantize, such as MobileNets and BERT-large.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This process enables efficient inference on resource-constrained hardware, such as edge devices and embedded systems, without significantly compromising model accuracy [36-38].”
From Fine-Grained Fault Sensitivity Analysis of Vision Transformers Under Soft Errors (2025) · cites this paper as [38]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“By leveraging the availability of training data, the QAT method provides better accuracy and more optimized networks in terms of bit-widths.”
From HADQ-Net: A Power-Efficient and Hardware-Adaptive Deep Convolutional Neural Network Translator Based on Quantization-Aware Training for Hardware Accelerators (2025) · cites this paper as [15]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It is common and recommended practice in the literature to start from a pre-trained FP32 model [12,15,17]. This approach not only improves accuracy but also reduces the number of required iterations.”
The cited source says what the claim attributes to it.
In the citing paper
“While PTQ is simpler and more efficient, QAT generally yields higher accuracy, particularly for lower bit-widths, when sufficient training data are available [14,15].”
The cited source says what the claim attributes to it.
In the citing paper
“Percentile [36] and KL divergence [37] strategies are introduced to address this problem.”
From Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey (2024) · cites this paper as [37]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“In addition to rule-based and optimization-based nonuniform quantization, clustering can also be beneficial to alleviate the information loss due to quantization.”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [256]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Product quantization [74,227,256] is an extension of vector quantization, where the weight matrix is divided into submatrices and vector quantization is applied to each submatrix.”
From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [256]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Self-citationVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
TensorFlow: A system for large-scale machine learningPreprint
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.5555/3026877.3026899 · Cited by 16,548
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273
(2010) · International Conference on Machine Learning · Cited by 13,995
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465
“Librispeech: an ASR corpus based on public domain audio books,”
Panayotov, V., Chen, G., Povey, D., & Khudanpur, S. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178964 · Cited by 6,072
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 2×
“Wu et al. [55] described how Google’s Neural Machine Translation (GNMT), which employs a Long Short Term Memory (LSTM) Recurrent Neural Network (RNN), was trained with hard range constraints on multiple tensors to be more amenable to PTQ.”
MethodInception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537
AutoAugment: Learning Augmentation Strategies From Data
Cubuk, E. D., Zoph, B., Mané, D., Vasudevan, V., & Le, Q. V. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00020 · Cited by 4,445
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408
(1989) · neural information processing systems · Cited by 3,736
Gaussian Error Linear Units (GELUs)
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1606.08415 · Cited by 3,537
Searching for activation functions
Ramachandran, P., Zoph, B., & Le, Q. V. (2017) · arXiv · doi:10.48550/arxiv.1710.05941 · Cited by 2,335
cuDNN: Efficient Primitives for Deep LearningPreprint
Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J. J., Catanzaro, B., & Shelhamer, E. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0759 · Cited by 2,058
Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
Bengio, Y., Léonard, N., & Courville, A. (2013) · arXiv · doi:10.48550/arxiv.1308.3432 · Cited by 1,910
A set of level 3 basic linear algebra subprograms
Dongarra, J. J., Croz, J. D., Hammarling, S. J., & Duff, I. (1990) · ACM Transactions on Mathematical Software · doi:10.1145/77626.79170 · Cited by 1,822
DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
Zhou, S., Wu, Y., Ni, Z., Zhou, X., Wen, H., & Zou, Y. (2016) · arXiv · doi:10.48550/arxiv.1606.06160 · Cited by 1,690
Hubara, I., Soudry, D., & Yaniv, R. E. (2016) · arXiv · doi:10.48550/arxiv.1602.02505 · Cited by 1,667
Micikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M., Kuchaiev, O., Venkatesh, G., et al. (2017) · arXiv · doi:10.48550/arxiv.1710.03740 · Cited by 1,179
Self-citationZhu, C., Han, S., Mao, H., & Dally, W. J. (2016) · arXiv · doi:10.48550/arxiv.1612.01064 · Cited by 865
Model compression via distillation and quantizationPreprint
Polino, A., Pascanu, R., & Alistarh, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05668 · Cited by 768
Bridging Nonlinearities and Stochastic Regularizers with Gaussian Error Linear UnitsPreprint
Hendrycks, D., & Gimpel, K. (2016) · arXiv (Cornell University) · Cited by 753
Quantizing deep convolutional networks for efficient inference: A whitepaper
Krishnamoorthi, R. (2018) · arXiv · doi:10.48550/arxiv.1806.08342 · Cited by 753
(2011) · Cited by 713
PACT: Parameterized Clipping Activation for Quantized Neural Networks
Choi, J., Wang, Z., Venkataramani, S., Chuang, P. I.-J., Srinivasan, V., & Gopalakrishnan, K. (2018) · arXiv · doi:10.48550/arxiv.1805.06085 · Cited by 703
Three Factors Influencing Minima in SGDPreprint
Jastrzebski, S., Kenton, Z., Arpit, D., Ballas, N., Fischer, A., Bengio, Y., & Storkey, A. (2017) · arXiv.org · doi:10.48550/arxiv.1711.04623 · Cited by 560
Data-Free Quantization Through Weight Equalization and Bias Correction
Nagel, M., Baalen, M. V., Blankevoort, T., & Welling, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00141 · Cited by 500
Zafrir, O., Boudoukh, G., Izsak, P., & Wasserblat, M. (2019) · 2019 Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing - NeurIPS Edition (EMC2-NIPS) · doi:10.1109/emc2-nips53020.2019.00016 · Cited by 442
Training deep neural networks with low precision multiplications
Courbariaux, M., Bengio, Y., & David, J.-P. (2014) · arXiv · doi:10.48550/arxiv.1412.7024 · Cited by 407
Extremely Low Bit Neural Network: Squeeze the Last Bit Out With ADMM
Leng, C., Dou, Z., Li, H., Shenghuo Zhu, B., & Jin, R. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11713 · Cited by 275
Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy
Mishra, A., & Marr, D. (2017) · arXiv · doi:10.48550/arxiv.1711.05852 · Cited by 270
Low precision storage for deep learningPreprint
Courbariaux, M., Bengio, Y., & David, J. (2014) · arXiv (Cornell University) · Cited by 142
Baskin, C., Liss, N., Schwartz, E., Zheltonozhskii, E., Giryes, R., Bronstein, A. M., & Mendelson, A. (2019) · ACM Transactions on Computer Systems · doi:10.1145/3444943 · Cited by 140
Efficient 8-Bit Quantization of Transformer Neural Machine Language Translation ModelPreprint
Bhandare, A., Sripathi, V., Karkada, D., Menon, V., Choi, S., Datta, K., & Saletore, V. A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.00532 · Cited by 134
Image Classification at Supercomputer ScalePreprint
Ying, C., Kumar, S., Chen, D., Wang, T., & Cheng, Y.-L. (2018) · arXiv.org · doi:10.48550/arxiv.1811.06992 · Cited by 133
Stochastic Gradient Methods with Layer-wise Adaptive Moments for Training of Deep NetworksPreprint
Ginsburg, B., Castonguay, P., Hrinchuk, O., Kuchaiev, O., Lavrukhin, V., Leary, R., Li, J., Nguyen, H., Zhang, Y., & Cohen, J. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11286 · Cited by 133
Jain, S. R., Gural, A., Wu, M., & Dick, C. H. (2019) · arXiv · doi:10.48550/arxiv.1903.08066 · Cited by 115
SIMPLIFYING NEURAL NETS BY DISCOVERING FLAT MINIMA
Hochreiter, S., & Schmidhuber, J. (1994) · Cited by 92
Ternary Neural Networks with Fine-Grained QuantizationPreprint
Mellempudi, N., Kundu, A., Mudigere, D., Das, D., Kaul, B., & Dubey, P. K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.01462 · Cited by 89
Discovering Low-Precision Networks Close to Full-Precision Networks for Efficient Embedded Inference
McKinstry, J. L., Esser, S. K., Appuswamy, R., Bablani, D., Arthur, J. V., Yildiz, I. B., & Modha, D. S. (2018) · arXiv · doi:10.48550/arxiv.1809.04191 · Cited by 74
Rusci, M., Capotondi, A., & Benini, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13082 · Cited by 73
Marian: Cost-effective High-Quality Neural Machine Translation in C++
Junczys-Dowmunt, M., Heafield, K., Hoang, H., Grundkiewicz, R., & Aue, A. (2018) · doi:10.18653/v1/w18-2716 · Cited by 52
Neural Network Distiller: A Python Package For DNN Compression Research
Zmora, N., Jacob, G., Zlotnik, L., Elharar, B., & Novik, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12232 · Cited by 51
Neural Network Compression Framework for Fast Model Inference
Kozlov, A., Lazarevich, I. A., Shamporov, V. A., Lyalyushkin, N., & Gorbachev, Y. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-80129-8_17 · Cited by 44
Retraining-Based Iterative Weight Quantization for Deep Neural NetworksPreprint
Lee, D., & Kim, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.11233 · Cited by 13
Flagged first, then by citations
1 of the 57 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- [2][2] Chaim Baskin, Eli Schwartz, Evgenii Zheltonozhskii, Natan Liss, Raja Giryes, Alex M Bronstein, and Avi Mendelson. Uniq: Uniform noise injection for non-uniform quantization of neural networks. arXiv preprint arXiv:1804.10969 , 2018. - · arXiv
- [8][8] Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David. BinaryConnect: Training Deep Neural Networks with binary weights during propagations. NIPS , 28:3123-3131, 2015. -
- [13][13] Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149 , 2015. - · arXiv
- [20][20] Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 2704-2713, 2018. -
- [27][27] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems , pages 1097-1105, 2012. -
- [36][36] Szymon Migacz. Nvidia 8-bit inference width tensorrt. In GPU Technology Conference , 2017. -
Cited by
Sorted by citations
A Survey of Quantization Methods for Efficient Neural Network Inference
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., & Keutzer, K. (2022) · Low-Power Computer Vision · doi:10.1201/9781003162810-13 · Cited by 1,219
“Symmetric quantization is widely adopted in practice for quantizing weights because zeroing out the zero point can lead to reduction in computational cost during inference [247], and also makes the implementation more straightforward.”
Pruning and quantization for deep neural network acceleration: A survey
Liang, T., Glossner, J., Wang, L., Shi, S., & Zhang, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.045 · Cited by 1,054
Self-citationKnowledge distillation: A good teacher is patient and consistent
Beyer, L., Zhai, X., Royer, A., Markeeva, L., Anil, R., & Kolesnikov, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01065 · Cited by 308
Emotion recognition in EEG signals using deep learning methods: A review
Jafari, M., Shoeibi, A., Khodatars, M., Bagherzadeh, S., Shalbaf, A., García, D. L., Gorriz, J. M., & Acharya, U. R. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107450 · Cited by 240
Magalhães, S. A., de Castro, L. M. G., Moreira, G., dos Santos, F. N., Cunha, M., Dias, J. M. M., & Moreira, A. P. (2021) · Sensors · doi:10.3390/s21103569 · Cited by 198
Metadata
- Authors
- Wu, H., Judd, P., Zhang, X., Isaev, M., & Micikevicius, P.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2004.09602
- OpenAlex
- W3017746288
- Cited by
- 246
- References
- 57
- Citations by year
- 2026: 4 · 2025: 30 · 2024: 68 · 2023: 65 · 2022: 41 · 2021: 32 · 2020: 6
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Numerical Methods and Algorithms
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Low-power high-performance VLSI design
- Keywords
- integer quantization, 8-bit quantization, deep neural network inference, quantization parameter, inference latency, model size reduction, MobileNet, BERT
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
