Loss aware post-training quantization
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Neural network quantization enables the deployment of large models on resource-constrained devices. Current post-training quantization methods fall short in terms of accuracy for INT4 (or lower) but provide reasonable accuracy for INT8 (or above). In this work, we study the effect of quantization on the structure of the loss landscape. We show that the structure is flat and separable for mild quantization, enabling straightforward post-training quantization methods to achieve good results. We show that with more aggressive quantization, the loss landscape becomes highly non-separable with steep curvature, making the selection of quantization parameters more challenging. Armed with this understanding, we design a method that quantizes the layer parameters jointly, enabling significant accuracy improvement over current post-training quantization methods. Reference implementation is available at https://github.com/ynahshan/nn-quantization-pytorch/tree/master/lapq .
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Least squares quantization in PCM
Lloyd, S. (1982) · IEEE Transactions on Information Theory · doi:10.1109/tit.1982.1056489 · Cited by 15,836
Powell, M. J. D. (1964) · The Computer Journal · doi:10.1093/comjnl/7.2.155 · Cited by 4,632 · cited 4×
“Finally, we jointly optimize the parameters of all layers acquired on the previous step by applying a gradient-free optimization method (Powell 1964).”
MethodInfluentialQuantizing deep convolutional networks for efficient inference: A whitepaper
Krishnamoorthi, R. (2018) · arXiv · doi:10.48550/arxiv.1806.08342 · Cited by 753
Data-Free Quantization Through Weight Equalization and Bias Correction
Nagel, M., Baalen, M. V., Blankevoort, T., & Welling, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00141 · Cited by 500
Low-bit Quantization of Neural Networks for Efficient Inference
Choukroun, Y., Kravchik, E., Yang, F., & Kisilev, P. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00363 · Cited by 404 · cited 3×
“…quantization perform finer parameter assignment, such as group-wise (Mellempudi et al. 2017), channel-wise (Banner et al. 2018), pixel-wise (Fraser et al. 2018), or filter-wise (Kravchik et al. 2019) quantization, requiring special hardware support and additional computational resources.”
BackgroundMethodImproving Neural Network Quantization without Retraining using Outlier Channel Splitting
Zhao, R., Hu, Y., Dotzel, J., De Sa, C., & Zhang, Z. (2019) · arXiv · doi:10.48550/arxiv.1901.09504 · Cited by 231
SYQ: Learning Symmetric Quantization for Efficient Deep Neural Networks
Faraone, J., Fraser, N., Blott, M., & Leong, P. H. W. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00452 · Cited by 134
Post-training 4-bit quantization of convolution networks for rapid-deployment
Banner, R., Nahshan, Y., Hoffer, E., & Soudry, D. (2018) · arXiv · doi:10.48550/arxiv.1810.05723 · Cited by 115 · cited 9×
“Multiple alternative schemes for the correction of quantization bias were proposed, and those techniques are widely applied in state-of-the-art quantization approaches (Banner et al. 2018; Finkelstein et al. 2019; Nagel et al. 2019; Fang et al. 2020).”
BackgroundMethodInfluentialSelf-citationLoss-aware Weight Quantization of Deep Networks
Hou, L., & Kwok, J. T. (2018) · arXiv · doi:10.48550/arxiv.1802.08635 · Cited by 99 · cited 1×
“For example, Hou et al. (2016); Hou and Kwok (2018) have shown that we can utilize the Hessian of the loss to improve quantization of the CNNs.”
MethodBit Efficient Quantization for Deep Neural Networks
Nayak, P., Zhang, D., & Chai, S. (2019) · 2019 Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing - NeurIPS Edition (EMC2-NIPS) · doi:10.1109/emc2-nips53020.2019.00020 · Cited by 51
Cited by
BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction
Li, Y., Gong, R., Tan, X., Yang, Y., Hu, P., Zhang, Q., Yu, F., Wang, W., & Gu, S. (2021) · arXiv · doi:10.48550/arxiv.2102.05426 · Cited by 190
BinaryBERT: Pushing the Limit of BERT Quantization
Bai, H., Zhang, W., Hou, L., Shang, L., Jin, J., Jiang, X., Liu, Q., Lyu, M., & King, I. (2020) · arXiv · doi:10.48550/arxiv.2012.15701 · Cited by 42
Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformers
Frumkin, N., Gope, D., & Marculescu, D. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01557 · Cited by 14
“Complex methods involve techniques such as gradient descent[24], Hessian analysis [15], or knowledge distillation [6] to maximize the quantized model’s accuracy.”
MethodBit-shrinking: Limiting Instantaneous Sharpness for Improving Post-training Quantization
Lin, C., Peng, B., Li, Z., Tan, W., Ren, Y., Xiao, J., & Pu, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01554 · Cited by 13
“Here, we evaluate our algorithm and compare with the State-of-the-arts post-training quantization approaches, including ACIQ-Mix [1], ZeroQ [2], LAPQ [27], Bit-split [28], AdaRound [26], BRECQ [19], AdaQuant [19] and QDrop [39].”
Method
Metadata
- Authors
- Nahshan, Y., Chmiel, B., Baskin, C., Zheltonozhskii, E., Banner, R., Bronstein, A. M., & Mendelson, A.
- Venue
- Machine Learning
- Published
- 2021-10-01
- Volume
- 110
- Issue
- 11-12
- Pages
- 3245-3262
- ISSN
- 0885-6125 (ISSN-L)
- 1573-0565
- OpenAlex
- W2985339713
- Cited by
- 142
- References
- 12
- FWCI
- 7.58
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Loss aware post-training quantization. https://app.cochat.ai/papers/10.1007/s10994-021-06053-z
- The paper
- Nahshan, Y., Chmiel, B., Baskin, C., Zheltonozhskii, E., Banner, R., Bronstein, A. M., & Mendelson, A. (2021). Loss aware post-training quantization. Machine Learning. https://doi.org/10.1007/s10994-021-06053-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.