Post-training Piecewise Linear Quantization for Deep Neural Networks
Open accessThe DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
Quantization plays an important role in the energy-efficient deployment of deep neural networks on resource-limited devices. Post-training quantization is highly desirable since it does not require retraining or access to the full training dataset. The well-established uniform scheme for post-training quantization achieves satisfactory results by converting neural networks from full-precision to 8-bit fixed-point integers. However, it suffers from significant performance degradation when quantizing to lower bit-widths. In this paper, we propose a piecewise linear quantization (PWLQ) scheme to enable accurate approximation for tensor values that have bell-shaped distributions with long tails. Our approach breaks the entire quantization range into non-overlapping regions for each tensor, with each region being assigned an equal number of quantization levels. Optimal breakpoints that divide the entire range are found by minimizing the quantization error. Compared to state-of-the-art post-training quantization methods, experimental results show that our proposed method achieves superior performance on image classification, semantic segmentation, and object detection with minor overhead.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
SSD: Single Shot MultiBox Detector
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_2 · Cited by 22,128
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · arXiv · doi:10.48550/arxiv.1802.02611 · Cited by 20,063
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.690 · Cited by 19,892
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“Weight pruning [17,32,20,37], activation compression [10,9,14], knowledge distillation [21,45] and quantization [8,46,66,64,41,25] fall under this category.”
BackgroundSqueeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01083 · Cited by 9,982
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_8 · Cited by 7,356
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244
Han, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., & Kalenichenko, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00286 · Cited by 4,114
Pruning Filters for Efficient ConvNets
Li, H., Kadav, A., Durdanovic, I., Samet, H., & Graf, H. P. (2016) · arXiv · doi:10.48550/arxiv.1608.08710 · Cited by 2,928
Channel Pruning for Accelerating Very Deep Neural Networks
He, Y., Zhang, X., & Sun, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.155 · Cited by 2,647 · cited 1×
“Weight pruning [17,32,20,37], activation compression [10,9,14], knowledge distillation [21,45] and quantization [8,46,66,64,41,25] fall under this category.”
BackgroundBinaryConnect: Training Deep Neural Networks with binary weights during propagations
Courbariaux, M., Bengio, Y., & David, J.-P. (2015) · arXiv · doi:10.48550/arxiv.1511.00363 · Cited by 2,180
DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
Zhou, S., Wu, Y., Ni, Z., Zhou, X., Wen, H., & Zou, Y. (2016) · arXiv · doi:10.48550/arxiv.1606.06160 · Cited by 1,690
Deep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · arXiv · doi:10.48550/arxiv.1502.02551 · Cited by 1,364 · cited 1×
“In particular, quantization of activations and weights [15,16,57,35,6,60,62] leads to model compression and acceleration as well as to overall savings in power consumption.”
BackgroundMicikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M., Kuchaiev, O., Venkatesh, G., et al. (2017) · arXiv · doi:10.48550/arxiv.1710.03740 · Cited by 1,179 · cited 1×
“To minimize the quantization error and maintain the performance of a full-precision model, many recent studies [63,4,40,25,6,60,12,27] rely on training either from scratch (“quantization-aware” training) or by fine-tuning a pre-trained FP32 model.”
MethodQuantizing deep convolutional networks for efficient inference: A whitepaper
Krishnamoorthi, R. (2018) · arXiv · doi:10.48550/arxiv.1806.08342 · Cited by 753
PACT: Parameterized Clipping Activation for Quantized Neural Networks
Choi, J., Wang, Z., Venkataramani, S., Chuang, P. I.-J., Srinivasan, V., & Gopalakrishnan, K. (2018) · arXiv · doi:10.48550/arxiv.1805.06085 · Cited by 703
Fixed Point Quantization of Deep Convolutional Networks
Lin, D. D., Talathi, S. S., & Annapureddy, V. S. (2015) · arXiv · doi:10.48550/arxiv.1511.06393 · Cited by 684 · cited 4×
“ning uniform quantization causes signicant accuracy drop [28,62]. This is mainly because the distributions of weights and activations of pre-trained DNNs is bell-shaped such as Gaussian or Laplacian [17,35]. That is, most of the weights are clustered around zero while few of them are spread in a long tail. As a result…”
BackgroundMethodInfluentialData-Free Quantization Through Weight Equalization and Bias Correction
Nagel, M., Baalen, M. V., Blankevoort, T., & Welling, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00141 · Cited by 500
Deep Learning with Low Precision by Half-Wave Gaussian Quantization
Cai, Z., He, X., Sun, J., & Vasconcelos, N. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.574 · Cited by 498 · cited 2×
“To mitigate this issue, various quantization schemes [41, 4, 3, 43, 26, 34] are designed to take advantage of the fact that weights and activations of pre-trained DNNs typically have bell-shaped distributions with long tails.”
BackgroundMethodLow-bit Quantization of Neural Networks for Efficient Inference
Choukroun, Y., Kravchik, E., Yang, F., & Kisilev, P. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00363 · Cited by 404 · cited 6×
“From [25,7], the approximated version r̂ from uniform quantization scheme at b-bit can be defined as:”
BackgroundMethodInfluentialLearning to Quantize Deep Networks by Optimizing Quantization Intervals With Task Loss
Jung, S., Son, C., Lee, S., Son, J., Han, J.-J., Kwak, Y., Hwang, S. J., & Choi, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00448 · Cited by 385
Improving Neural Network Quantization without Retraining using Outlier Channel Splitting
Zhao, R., Hu, Y., Dotzel, J., De Sa, C., & Zhang, Z. (2019) · arXiv · doi:10.48550/arxiv.1901.09504 · Cited by 231
Fully Quantized Network for Object Detection
Li, R., Wang, Y., Liang, F., Qin, H., Yan, J., & Fan, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00292 · Cited by 214 · cited 1×
“7 We test with the top-k median and percentile-based [33] approaches and use the top-10 median method for better robustness of low-bit quantization.”
MethodSYQ: Learning Symmetric Quantization for Efficient Deep Neural Networks
Faraone, J., Fraser, N., Blott, M., & Leong, P. H. W. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00452 · Cited by 134
Post-training 4-bit quantization of convolution networks for rapid-deployment
Banner, R., Nahshan, Y., Hoffer, E., & Soudry, D. (2018) · arXiv · doi:10.48550/arxiv.1810.05723 · Cited by 115
Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorization
Meller, E., Finkelstein, A., Almog, U., & Grobman, M. (2019) · arXiv · doi:10.48550/arxiv.1902.01917 · Cited by 60 · cited 2×
“Network W/A PWLQ (Ours) QWP [28] ACIQ [2] LBQ [7] SSBD [39] QRD [31] UNIQ [3] DFQ [42]”
BackgroundMethodFighting Quantization Bias With Bias
Finkelstein, A., Almog, U., & Grobman, M. (2019) · arXiv · doi:10.48550/arxiv.1906.03193 · Cited by 41
Cited by
A Survey of Quantization Methods for Efficient Neural Network Inference
Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., & Keutzer, K. (2022) · Low-Power Computer Vision · doi:10.1201/9781003162810-13 · Cited by 1,219
“2) Post-Training Quantization: An alternative to the expensive QAT method is Post-Training Quantization (PTQ) which performs the quantization and the adjustments of the weights, without any fine-tuning [11, 24, 40, 60, 61, 68, 69, 88, 107, 140, 146, 170, 178, 273].”
Model Compression for Deep Neural Networks: A Survey
Li, Z., Li, H., & Meng, L. (2023) · Computers · doi:10.3390/computers12030060 · Cited by 296
PTQ4ViT: Post-Training Quantization for Vision Transformers with Twin Uniform Quantization
Yuan, Z., Xue, C., Chen, Y., Wu, Q., & Sun, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19775-8_12 · Cited by 163
“[8] also uses multiple quantization ranges.”
Towards Accurate Post-Training Quantization for Vision Transformer
Ding, Y., Qin, H., Yan, Q., Chai, Z., Liu, J., Wei, X., & Liu, X. (2022) · Proceedings of the 30th ACM International Conference on Multimedia · doi:10.1145/3503161.3547826 · Cited by 79
“[12] also use Piecewise Linear Quantization to make the quantized parameters better fit the bell-shaped distribution of weight and activation after scaling.”
MethodNeural gradients are near-lognormal: improved quantized and sparse training
Chmiel, B., Ben-Uri, L., Shkolnik, M., Hoffer, E., Banner, R., & Soudry, D. (2020) · arXiv · doi:10.48550/arxiv.2006.08173 · Cited by 17
“Many previous works (Banner et al., 2018b; Ye et al., 2019; Fang et al., 2020) approximate these distributions using some analytically tractable distribution (e.g. Gaussian or Laplacian distribution) and use it to analyze and optimize their proposed compression scheme.”
Metadata
- Authors
- Fang, J., Shafiee, A., Abdel-Aziz, H., Thorsley, D., Georgiadis, G., & Hassoun, J. H.
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer International Publishing
- Published
- 2020
- Pages
- 69-86
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W3108835732
- Cited by
- 148
- References
- 40
- FWCI
- 5.84
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.