CoChat
Sign Up
PreprintCited by 64Advanced Neural Network ApplicationsLow-power high-performance VLSI designAdvanced Memory and Neural Computing

Searching for Low-Bit Weights in Quantized Neural Networks

Open access
Yang, Zhaohui, Wang, Yunhe, Han, Kai, Xu, Chunjing, Xu, Chao, Tao, Dacheng, Xu, Chang·2020·arXiv
DOI 10.48550/arxiv.2009.08695arXiv 2009.08695
Open this paper in CoChat — free accountOpen PDF

Abstract

Quantized neural networks with low-bit weights and activations are attractive for developing AI accelerators. However, the quantization functions used in most conventional quantization methods are non-differentiable, which increases the optimization difficulty of quantized networks. Compared with full-precision parameters (i.e., 32-bit floating numbers), low-bit values are selected from a much smaller set. For example, there are only 16 possibilities in 4-bit space. Thus, we present to regard the discrete weights in an arbitrary quantized neural network as searchable variables, and utilize a differential method to search them accurately. In particular, each weight is represented as a probability distribution over the discrete value set. The probabilities are optimized during training and the values with the highest probability are selected to establish the desired quantized network. Experimental results on benchmarks demonstrate that the proposed method is able to produce quantized neural networks with higher performance over the state-of-the-art methods on both image classification and super-resolution tasks.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2009.08695

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • HAWQv3 introduces integer-only hardware-aware quantization with a fast integer linear programming method to find optimal bit precision under application constraints.

    “Recently, in HAWQv3, an integer-only, hardware-aware quantization was introduced [267] that proposed a fast Integer Linear Programming method to find the optimal bit precision for a given applicationspecific constraint (e.g., model size or latency).”

    From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [267]

    The Abstract (p. 1) says this paper proposes a differentiable search over discrete weight values, with probabilities optimized during training. The Introduction (p. 2) likewise describes a weight-searching algorithm, not HAWQv3 or an integer linear programming method for selecting bit precision under model-size or latency constraints. (LLM, conf=0.84)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Integer-only quantization uses low-precision integer arithmetic for all operations, allowing inference without floating-point dequantization.

    “However, in integer-only quantization, all the operations are performed using low-precision integer arithmetic [113,132,154,193,267], as illustrated in Figure 6 (Right).This permits the entire inference to be carried out with efficient integer arithmetic, without any floating point dequantization of any parameters or activations.”

    From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [267]

    Related Works (p. 2) says binary networks use low-level bit operations to accelerate inference, and Optimization Details (p. 4) says the proposed method uses quantized weights for inference. Neither passage establishes that all inference operations use low-precision integer arithmetic or that no floating-point dequantization of parameters or activations occurs. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Mixed-precision quantization is an effective and hardware-efficient approach for low-precision quantization of various neural network models.

    “Mixedprecision quantization has proved to be an effective and hardware-efficient method for low-precision quantization of different NN models.”

    From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [267]

    The Introduction (p. 1) describes low-bit quantization as reducing memory and computation costs and enabling deployment on devices with suitable hardware. The Abstract (p. 1) reports improved results on image-classification and super-resolution tasks. Neither passage establishes the claim specifically for mixed-precision quantization; the paper discusses low-bit weight searching and experiments with different bitwidths, not the hardware efficiency of mixed precision. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Mixed-precision quantization is hardware-efficient because mixed precision is applied only across operations or layers.

    “Recent work [267] has also shown that this approach is hardware-efficient as mixedprecision is only used across operations/layers.”

    From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [267]

    The Introduction (p. 1) says low-bit quantized networks have low memory and computational costs and can be deployed on devices with specific hardware. But the paper does not say that mixed-precision quantization is hardware-efficient because precision varies only across operations or layers; its method searches low-bit weight values, not that mixed-precision arrangement. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Deploying mixed-precision on T4 GPUs gives up to 50% speedup with INT4/INT8 over INT8.

    “This work also addressed the common question about hardware efficiency of mixed-precision quantization by directly deploying them on T4 GPUs, showing up to 50% speed up with mixed-precision (INT4/INT8) quantization as compared to INT8 quantization.”

    From A Survey of Quantization Methods for Efficient Neural Network Inference (2022) · cites this paper as [267]

    The Experiments section (p. 6) describes tests at different bit-widths, and the CIFAR-10 results (p. 7) report accuracy rather than deployment speed. Searches for T4, INT4/INT8, and a 50% speedup found no passage in our copy reporting the claimed GPU comparison. (LLM, conf=0.77)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 51

  1. [2][2] Joseph Bethge, Haojin Yang, Marvin Bornstein, and Christoph Meinel. Back to simplicity: How to train accurate bnns from scratch? arXiv , 2019. -
  2. [3][3] Marco Bevilacqua, Aline Roumy, Christine Guillemot, and Marie Line Alberi-Morel. Lowcomplexity single-image super-resolution based on nonnegative neighbor embedding. BMVC , 2012. -
  3. [4][4] Zhaowei Cai, Xiaodong He, Jian Sun, and Nuno Vasconcelos. Deep learning with low precision by half-wave gaussian quantization. CVPR , 2017. -
  4. [5][5] Jungwook Choi. Pact: Parameterized clipping activation for quantized neural networks. arXiv , 2018. -
  5. [6][6] Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David. Binaryconnect: training deep neural networks with binary weights during propagations. NeurIPS , 2015. -
  6. [7][7] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. CVPR , 2009. -
  7. [8][8] Emily Denton, Wojciech Zaremba, Joan Bruna, Yann LeCun, and Rob Fergus. Exploiting linear structure within convolutional networks for efficient evaluation. arXiv , 2014. -
  8. [9][9] Ruizhou Ding, Ting-Wu Chin, Zeye Liu, and Diana Marculescu. Regularizing activation distribution for training binarized deep networks. CVPR , 2019. -
  9. [10][10] Julian Faraone, Nicholas Fraser, Michaela Blott, and Philip H.W. Leong. Syq: Learning symmetric quantization for efficient deep neural networks. CVPR , 2018. -
  10. [11][11] Ross Girshick. Fast r-cnn. ICCV , 2015. -
  11. [12][12] Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan. Differentiable soft quantization: Bridging full-precision and low-bit neural networks. CVPR , 2019. -
  12. [13][13] Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan. Differentiable soft quantization: Bridging full-precision and low-bit neural networks. ICCV , 2019. -
  13. [14][14] Jiaxin Gu, Ce Li, Baochang Zhang, Jungong Han, Xianbin Cao, Jianzhuang Liu, and David Doermann. Projection convolutional neural networks for 1-bit cnns via discrete back propagation. arXiv , 2018. -
  14. [15][15] Jiaxin Gu, Junhe Zhao, Xiaolong Jiang, Baochang Zhang, Liu Jianzhuang, Guodong Guo, and Rongrong Ji. Bayesian optimized 1-bit cnns. ICCV , 2019. -
  15. [16][16] Song Han, Huizi Mao, and William J. Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. ICLR , 2016. -
  16. [17][17] Chaoyang He, Haishan Ye, Li Shen, and Tong Zhang. Milenas: Efficient neural architecture search via mixed-level reformulation. arXiv , 2020. -
  17. [18][18] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. ICCV , 2017. -
  18. [19][19] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. ICCV , 2015. -
  19. [22][22] Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv , 2017. -
  20. [23][23] Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. Binarized neural networks. NeurIPS , 2016. -
  21. [24][24] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. ICML , 2015. -
  22. [25][25] Max Jaderberg, Andrea Vedaldi, and Andrew Zisserman. Speeding up convolutional neural networks with low rank expansions. BMVC , 2014. -
  23. [26][26] Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. ECCV , 2016. -
  24. [29][29] Zefan Li, Bingbing Ni, Wenjun Zhang, Xiaokang Yang, and Wen Gao. Performance guaranteed network acceleration via high-order residual quantization. ICCV , 2017. -
  25. [31][31] Shaohui Lin, Rongrong Ji, Chao Chen, Dacheng Tao, and Jiebo Luo. Holistic cnn compression via low-rank decomposition with knowledge transfer. TPAMI , 2018. -
  26. [32][32] Xiaofan Lin, Cong Zhao, and Wei Pan. Towards accurate binary convolutional neural network. NeurIPS , 2017. -
  27. [33][33] Chunlei Liu, Wenrui Ding, Xin Xia, Baochang Zhang, Jiaxin Gu, Jianzhuang Liu, Rongrong Ji, and David Doermann. Circulant binary convolutional networks: Enhancing the performance of 1-bit dcnns with circulant back propagation. CVPR , 2019. -
  28. [34][34] Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. In ICLR , 2019. -
  29. [35][35] Zechun Liu, Baoyuan Wu, Wenhan Luo, Xiaodong Yang, Weiwei Liu, and Kwang-Ting Cheng. Bi-real net: Enhancing the performance of 1-bit cnns with improved representational capability and advanced training algorithm. ECCV , 2018. -
  30. [36][36] Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. ICLR , 2017. -
  31. [37][37] Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. Shufflenet v2: Practical guidelines for efficient cnn architecture design. ECCV , 2018. -
  32. [38][38] Hai Phan, Dang Huynh, Yihui He, Marios Savvides, and Zhiqiang Shen. Mobinet: A mobile binary network for image classification. arXiv , 2019. -
  33. [39][39] Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. Xnor-net: Imagenet classification using binary convolutional neural networks. ECCV , 2016. -
  34. [41][41] Mark B. Sandler, Andrew G. Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. CVPR , 2018. -
  35. [42][42] Samuel Schulter, Christian Leistner, and Horst Bischof. Fast and accurate image upscaling with super-resolution forests. CVPR , 2015. -
  36. [43][43] Mingzhu Shen, Xianglong Liu, Kai Han, Ruihao Gong, Yunhe Wang, and Chang Xu. Balanced binary neural networks with gated residual. arXiv , 2019. -
  37. [44][44] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. ICLR , 2015. -
  38. [45][45] Diwen Wan, Fumin Shen, Li Liu, Fan Zhu, Jie Qin, Ling Shao, and Heng Tao Shen. Tbn: Convolutional neural network with ternary inputs and binary weights. ECCV , 2018. -
  39. [46][46] Ziwei Wang, Jiwen Lu, Chenxin Tao, Jie Zhou, and Qi Tian. Learning channel-wise interactions for binary convolutional neural networks. CVPR , 2019. -
  40. [47][47] Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, and Kurt Keutzer. Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search. CVPR , 2019. -
  41. [48][48] Zuxuan Wu, Tushar Nagarajan, Abhishek Kumar, Steven Rennie, Larry S. Davis, Kristen Grauman, and Rogério Schmidt Feris. Blockdrop: Dynamic inference paths in residual networks. CVPR , 2018. -
  42. [50][50] Zhaohui Yang, Yunhe Wang, Chuanjian Liu, Hanting Chen, Chunjing Xu, Boxin Shi, Chao Xu, and Chang Xu. Legonet: Efficient convolutional neural networks with lego filters. ICML , 2019. -
  43. [51][51] Jiahui Yu and Thomas Huang. Autoslim: Towards one-shot architecture search for channel numbers. arXiv , 2019. -
  44. [52][52] Jiahui Yu, Pengchong Jin, Hanxiao Liu, Gabriel Bender, Pieter-Jan Kindermans, Mingxing Tan, Thomas S. Huang, Xiaodan Song, Ruoming Pang, and Quoc Le. Bignas: Scaling up neural architecture search with big single-stage models. arXiv , 2020. -
  45. [53][53] Xiyu Yu, Tongliang Liu, Xinchao Wang, and Dacheng Tao. On compressing deep models by low rank and sparse decomposition. CVPR , 2017. -
  46. [54][54] Dongqing Zhang, Jiaolong Yang, Dongqiangzi Ye, and Gang Hua. Lq-nets: Learned quantization for highly accurate and compact deep neural networks. ECCV , 2018. -
  47. [55][55] Feng Zheng, Cheng Deng, and Heng Huang. Binarized neural networks for resource-efficient hashing with minimizing quantization loss. IJCAI , 2019. -
  48. [56][56] Shuchang Zhou, Zekun Ni, Xinyu Zhou, He Wen, Yuxin Wu, and Yuheng Zou. Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv , 2016. -
  49. [57][57] Chenzhuo Zhu, Song Han, Huizi Mao, and William J. Dally. Trained ternary quantization. ICLR , 2017. -
  50. [59][59] Bohan Zhuang, Chunhua Shen, Mingkui Tan, Lingqiao Liu, and Ian D. Reid. Structured binary neural networks for accurate image classification and semantic segmentation. CVPR , 2018. -
  51. [60][60] Li'an Zhuo, Baochang Zhang, Linlin Yang, Hanlin Chen, Qixiang Ye, David Doermann, Rongrong Ji, and Guodong Guo. Cogradient descent for bilinear optimization. CVPR , 2020.

Cited by

Sorted by citations

Authors
Yang, Z., Wang, Y., Han, K., Xu, C., Xu, C., Tao, D., & Xu, C.
Affiliations
  • Peking University (CN)
  • Huawei Technologies (China) (CN)
  • The University of Sydney (AU)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2009.08695
arXiv
2009.08695
OpenAlex
W3087150032
Cited by
64
References
7
Citations by year
2025: 6 · 2024: 12 · 2023: 10 · 2022: 16 · 2021: 19 · 2020: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Low-power high-performance VLSI design
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
Keywords
neural network quantization, weight quantization, discrete optimization, AI accelerators, image classification, super-resolution

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review