CoChat
Sign Up
PreprintCited by 5Advanced Neural Network ApplicationsDomain Adaptation and Few-Shot LearningStochastic Gradient Optimization Techniques

The DOI matches its DataCite record; the references it cites are not checked yet.

COMQ: A Backpropagation-Free Algorithm for Post-Training Quantization

Open access
Zhang, Aozhong, Yang, Zi, Wang, Naigang, Qi, Yingyong, Xin, Jack, Li, Xin, Yin, Penghang·2024·arXiv
DOI 10.48550/arxiv.2403.07134arXiv 2403.07134
Open this paper in CoChat — free account

Abstract

Post-training quantization (PTQ) has emerged as a practical approach to compress large neural networks, making them highly efficient for deployment. However, effectively reducing these models to their low-bit counterparts without compromising the original accuracy remains a key challenge. In this paper, we propose an innovative PTQ algorithm termed COMQ, which sequentially conducts coordinate-wise minimization of the layer-wise reconstruction errors. We consider the widely used integer quantization, where every quantized weight can be decomposed into a shared floating-point scalar and an integer bit-code. Within a fixed layer, COMQ treats all the scaling factor(s) and bit-codes as the variables of the reconstruction error. Every iteration improves this error along a single coordinate while keeping all other variables constant. COMQ is easy to use and requires no hyper-parameter tuning. It instead involves only dot products and rounding operations. We update these variables in a carefully designed greedy order, significantly enhancing the accuracy. COMQ achieves remarkable results in quantizing 4-bit Vision Transformers, with a negligible loss of less than 1% in Top-1 accuracy. In 4-bit INT quantization of convolutional neural networks, COMQ maintains near-lossless accuracy with a minimal drop of merely 0.3% in Top-1 accuracy.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2403.07134

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

The citation lists are queued

They are fetched later and appear here then.

References 61 · Cited by 5

Research works printed but not linked: 56

  1. [1]1. Behdin, K., Acharya, A., Gupta, A., Keerthi, S., Mazumder, R.: Quantease: Optimization-based quantization for language models-an efficient and intuitive algorithm. arXiv preprint arXiv:2309.01885 (2023) · arXiv
  2. [3]3. Chauhan, A., Tiwari, U., et al.: Post training mixed precision quantization of neural networks using first-order information. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 1343-1352 (2023)
  3. [2]2. Cai, Y., Yao, Z., Dong, Z., Gholami, A., Mahoney, M.W., Keutzer, K.: Zeroq: A novel zero shot quantization framework. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13169-13178 (2020)
  4. [4]4. Choi, J., Wang, Z., Venkataramani, S., Chuang, P.I.J., Srinivasan, V., Gopalakrishnan, K.: Pact: Parameterized clipping activation for quantized neural networks. arXiv preprint arXiv:1805.06085 (2018) · arXiv
  5. [6]6. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018) · arXiv
  6. [5]5. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A LargeScale Hierarchical Image Database. In: Preoceedings of the IEEE/CVF Computer Vision and Pattern Recognition Conference (2009)
  7. [7]7. Ding, Y., Qin, H., Yan, Q., Chai, Z., Liu, J., Wei, X., Liu, X.: Towards accurate post-training quantization for vision transformer. In: Proceedings of the 30th ACM International Conference on Multimedia. pp. 5380-5388 (2022)
  8. [9]9. Frantar, E., Alistarh, D.: Optimal brain compression: A framework for accurate post-training quantization and pruning. Advances in Neural Information Processing Systems 35 , 4475-4488 (2022)
  9. [8]8. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020) · arXiv
  10. [10]10. Frantar, E., Ashkboos, S., Hoefler, T., Alistarh, D.: Gptq: Accurate posttraining quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323 (2022) · arXiv
  11. [12]12. He, Y., Liu, L., Liu, J., Wu, W., Zhou, H., Zhuang, B.: Ptqd: Accurate post-training quantization for diffusion models. Advances in Neural Information Processing Systems 36 (2024)
  12. [11]11. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770-778 (2016)
  13. [13]13. He, Y., Zhang, X., Sun, J.: Channel pruning for accelerating very deep neural networks. In: Proceedings of the IEEE international conference on computer vision. pp. 1389-1397 (2017)
  14. [15]15. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017) · arXiv
  15. [14]14. Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Advances in neural information processing systems 33 , 6840-6851 (2020)
  16. [16]16. Hsieh, C.J., Chang, K.W., Lin, C.J., Keerthi, S.S., Sundararajan, S.: A dual coordinate descent method for large-scale linear svm. In: Proceedings of the 25th international conference on Machine learning. pp. 408-415 (2008)
  17. [17]17. Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., Bengio, Y.: Quantized neural networks: Training neural networks with low precision weights and activations. The Journal of Machine Learning Research 18 (1), 6869-6898 (2017)
  18. [18]18. Hubara, I., Nahshan, Y., Hanani, Y., Banner, R., Soudry, D.: Improving post training neural quantization: Layer-wise calibration and integer programming. arXiv preprint arXiv:2006.10518 (2020) · arXiv
  19. [19]19. Hubara, I., Nahshan, Y., Hanani, Y., Banner, R., Soudry, D.: Accurate post training quantization with small calibration sets. In: International Conference on Machine Learning. pp. 4466-4475. PMLR (2021)
  20. [20]20. Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al.: Highly accurate protein structure prediction with alphafold. Nature 596 (7873), 583-589 (2021)
  21. [21]21. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012)
  22. [22]22. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Communications of the ACM 60 (6), 84-90 (2017)
  23. [23]23. Lee, J.H., Kim, J., Kwon, S.J., Lee, D.: Flexround: Learnable rounding based on element-wise division for post-training quantization. arXiv preprint arXiv:2306.00317 (2023) · arXiv
  24. [24]24. Li, C., Peng, J., Yuan, L., Wang, G., Liang, X., Lin, L., Chang, X.: Block-wisely supervised neural architecture search with knowledge distillation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1989-1998 (2020)
  25. [25]25. Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) · arXiv
  26. [26]26. Li, Y., Gong, R., Tan, X., Yang, Y., Hu, P., Zhang, Q., Yu, F., Wang, W., Gu, S.: Brecq: Pushing the limit of post-training quantization by block reconstruction. arXiv preprint arXiv:2102.05426 (2021) · arXiv
  27. [27]27. Li, Z., Xiao, J., Yang, L., Gu, Q.: Repq-vit: Scale reparameterization for posttraining quantization of vision transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 17227-17236 (2023)
  28. [28]28. Lin, C., Peng, B., Li, Z., Tan, W., Ren, Y., Xiao, J., Pu, S.: Bit-shrinking: Limiting instantaneous sharpness for improving post-training quantization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 16196-16205 (2023)
  29. [29]29. Lin, Y., Zhang, T., Sun, P., Li, Z., Zhou, S.: Fq-vit: Post-training quantization for fully quantized vision transformer. arXiv preprint arXiv:2111.13824 (2021) · arXiv
  30. [30]30. Liu, J., Niu, L., Yuan, Z., Yang, D., Wang, X., Liu, W.: Pd-quant: Posttraining quantization based on prediction difference metric. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 2442724437 (2023)
  31. [31]31. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 10012-10022 (2021)
  32. [32]32. Liu, Z., Wang, Y., Han, K., Zhang, W., Ma, S., Gao, W.: Post-training quantization for vision transformer. Advances in Neural Information Processing Systems 34 , 28092-28103 (2021)
  33. [33]33. Nagel, M., Amjad, R.A., Van Baalen, M., Louizos, C., Blankevoort, T.: Up or down? adaptive rounding for post-training quantization. In: International Conference on Machine Learning. pp. 7197-7206. PMLR (2020)
  34. [35]35. Nesterov, Y.: Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM Journal on Optimization 22 (2), 341-362 (2012)
  35. [34]34. Nagel, M., Baalen, M.v., Blankevoort, T., Welling, M.: Data-free quantization through weight equalization and bias correction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 1325-1334 (2019)
  36. [38]38. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: Towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing systems. pp. 91-99 (2015)
  37. [37]37. Rastegari, M., Ordonez, V., Redmon, J., Farhadi, A.: Xnor-net: Imagenet classification using binary convolutional neural networks. In: European conference on computer vision. pp. 525-542. Springer (2016)
  38. [39]39. Shang, Y., Yuan, Z., Xie, B., Wu, B., Yan, Y.: Post-training quantization on diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1972-1981 (2023)
  39. [41]41. Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., Jégou, H.: Training data-efficient image transformers & distillation through attention. In: International conference on machine learning. pp. 10347-10357. PMLR (2021)
  40. [40]40. Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529 (7587), 484 (2016)
  41. [42]42. Uhlich, S., Mauch, L., Cardinaux, F., Yoshiyama, K., Garcia, J.A., Tiedemann, S., Kemp, T., Nakamura, A.: Mixed precision dnns: All you need is a good parametrization. arXiv preprint arXiv:1905.11452 (2019) · arXiv
  42. [44]44. Wang, N., Liu, C.C.C., Venkataramani, S., Sen, S., Chen, C.Y., El Maghraoui, K., Srinivasan, V.V., Chang, L.: Deep compression of pre-trained transformer models. Advances in Neural Information Processing Systems 35 , 14140-14154 (2022)
  43. [43]43. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017)
  44. [45]45. Wang, P., Chen, Q., He, X., Cheng, J.: Towards accurate post-training network quantization via bit-split and stitching. In: International Conference on Machine Learning. pp. 9847-9856. PMLR (2020)
  45. [47]47. Wang, P., Cheng, J.: Fixed-point factorized networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 4012-4020 (2017)
  46. [46]46. Wang, P., Chen, W., He, X., Chen, Q., Liu, Q., Cheng, J.: Optimization-based posttraining quantization with bit-split and stitching. IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (2), 2119-2135 (2022)
  47. [48]48. Wei, X., Gong, R., Li, Y., Liu, X., Yu, F.: Qdrop: Randomly dropping quantization for extremely low-bit post-training quantization. arXiv preprint arXiv:2203.05740 (2022) · arXiv
  48. [50]50. Wright, S.J.: Coordinate descent algorithms. Mathematical programming 151 (1), 3-34 (2015)
  49. [49]49. Wortsman, M., Dettmers, T., Zettlemoyer, L., Morcos, A., Farhadi, A., Schmidt, L.: Stable and low-precision training for large-scale vision-language models. Advances in Neural Information Processing Systems 36 (2024)
  50. [51]51. Xiao, G., Lin, J., Seznec, M., Wu, H., Demouth, J., Han, S.: Smoothquant: Accurate and efficient post-training quantization for large language models. In: International Conference on Machine Learning. pp. 38087-38099. PMLR (2023)
  51. [53]53. Yin, P., Lyu, J., Zhang, S., Osher, S., Qi, Y., Xin, J.: Understanding straightthrough estimator in training activation quantized neural nets. In: International Conference on Learning Representations (2019)
  52. [52]52. Xu, S., Li, H., Zhuang, B., Liu, J., Cao, J., Liang, C., Tan, M.: Generative lowbitwidth data free quantization. In: Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XII 16. pp. 1-17. Springer (2020)
  53. [54]54. Yin, P., Zhang, S., Lyu, J., Osher, S., Qi, Y., Xin, J.: Binaryrelax: A relaxation approach for training deep neural networks with quantized weights. SIAM Journal on Imaging Sciences 11 (4), 2205-2223 (2018)
  54. [56]56. Yuan, Z., Xue, C., Chen, Y., Wu, Q., Sun, G.: Ptq4vit: Post-training quantization framework for vision transformers with twin uniform quantization. arXiv preprint arXiv:2111.12293 (2021) · arXiv
  55. [57]57. Zhang, J., Zhou, Y., Saab, R.: Post-training quantization for neural networks with provable guarantees. SIAM Journal on Mathematics of Data Science 5 (2), 373-399 (2023)
  56. [58]58. Zhou, S., Wu, Y., Ni, Z., Zhou, X., Wen, H., Zou, Y.: Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160 (2016) · arXiv

Metadata

Authors
Zhang, A., Yang, Z., Wang, N., Qi, Y., Xin, J., Li, X., & Yin, P.
Affiliations
  • State University of New York (US)
  • IBM (United States) (US)
  • University of California, Irvine (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
Volume
13
Pages
99879-99891
ISSN
2169-3536 (ISSN-L)
DOI
10.48550/arxiv.2403.07134
arXiv
2403.07134
OpenAlex
W4411086485
Cited by
5
References
61
Citations by year
2026: 4 · 2025: 1
FWCI
1.76
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Access (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University) (cc-by)
Funders
  • National Science Foundation (2219904, 2110546, 2110836, 2348046, 2208126)
  • State University of New York
  • University at Albany
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
Keywords
post-training quantization, integer quantization, scaling factor optimization, greedy optimization, neural network compression, vision transformer, convolutional neural networks, low-bit precision

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: COMQ: A Backpropagation-Free Algorithm for Post-Training Quantization. https://app.cochat.ai/papers/10.48550/arxiv.2403.07134
The paper
Zhang, A., Yang, Z., Wang, N., Qi, Y., Xin, J., Li, X., & Yin, P. (2024). COMQ: A Backpropagation-Free Algorithm for Post-Training Quantization. arXiv. https://doi.org/10.48550/arxiv.2403.07134