A Survey of Quantization Methods for Efficient Neural Network Inference

In the paper

“Furthermore, the quantizer itself can also be jointly trained with the model parameters.These methods are referred to as learnable quantizers, and the quantization steps/levels are generally trained with iterative optimization [258,276] or gradient descent [125,158,264].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [258] 258. Xia Xiao, Zigeng Wang, and Sanguthevar Ra- jasekaran. Autoprune: Automatic network pruning by regularizing auxiliary parameters. In Advances in Neural Information Processing Systems, pages 13681-13691, 2019.
• ?Could not verify

  Section 3.1 mentions a Hessian-based approach used in quantization, but does not discuss training a quantizer jointly with model parameters. The Methodology section concerns pruning, and the References list quantization papers without describing learnable steps or levels. Searches for joint quantizer training and optimized quantization steps found no passage addressing the claim. (LLM, conf=0.74)

  cites [276] 276. Shixing Yu, Zhewei Yao, Amir Gholami, Zhen Dong, Michael W Mahoney, and Kurt Keutzer. Hessian-aware pruning and optimal neural implant. arXiv preprint arXiv:2101.08940, 2021. sourceExperimental — accuracy not yet measured
• Disputed

  Section 3.2, “Learning with LBC Layers” (page unavailable), says that only the learnable 1×1 filters are updated; the binary anchor weights remain unaffected. The paper therefore does not describe training quantization steps or levels jointly with model parameters by gradient descent. (LLM, conf=0.88)

  Checked by a reasoning model — may be wrong.

  cites [125] 125. Felix Juefei-Xu, Vishnu Naresh Boddeti, and Mar- ios Savvides. Local binary convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 19-28, 2017. sourceExperimental — accuracy not yet measured
• Not checked yetcites [158] 158. Wuwei Lin. Automating optimization of quantized deep learning models on cuda: https://tvm.apache.org/2019/04/29/opt-cuda- quantized, 2019.
• Partly supported

  The Introduction (p. 2) and §3.1 (p. 4) describe training quantized weight bits with gradient-based optimization, which supports the gradient-descent part of the attribution. BSQ uses that training and periodic precision adjustment to obtain a mixed-precision scheme; our copy does not describe directly learning quantization step sizes or levels as separate quantizer parameters. (LLM, conf=0.82)

  cites [264] 264. Huanrui Yang, Lin Duan, Yiran Chen, and Hai Li. Bsq: Exploring bit-level sparsity for mixed- precision neural network quantization. arXiv preprint arXiv:2102.10462, 2021. sourceExperimental — accuracy not yet measured

In the paper

“While STE is the mainstream approach [226,289], other approaches have also been explored in the literature [2,25,31,59,144,164].”

What the sources say

• Not checked yetcites [226] 226. S. M. Stigler. The History of Statistics: The Measurement of Uncertainty before 1900. Harvard University Press, Cambridge, 1986.
• Disputed

  The citation is attached to the claim that approaches other than STE have been explored. Section 4, “Binary Ensemble Neural Network,” says this paper uses STE during back-propagation; its bagging and boosting strategies are not presented as alternatives to STE. (LLM, conf=0.86)

  Checked by a reasoning model — may be wrong.

  cites [289] 289. Shilin Zhu, Xin Dong, and Hao Su. Binary ensemble neural network: More bits per network or more networks per bit? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4923-4932, 2019. sourceExperimental — accuracy not yet measured

In the paper

“Another different class of alternative methods tries to use regularization operators to enforce the weight to be quantized.This removes the need to use the non-differentiable quantization operator in Eq. 2. These are often referred to as Non-STE methods [4, 8, 39, 99, 144, 184, 283].”

What the sources say

• Partly supported

  Related Work (p. 5) describes regularization methods that encourage weights to lie near quantization grid points, separately from methods that mimic quantization during the forward pass. But this paper’s own method regularizes the ℓ₁-norm of gradients to make a model robust to later quantization; it does not use regularization to force its weights to quantized values (Methods, p. 3; Conclusion, p. 9). (LLM, conf=0.88)

  cites [4] 4. Milad Alizadeh, Arash Behboodi, Mart van Baalen, Christos Louizos, Tijmen Blankevoort, and Max Welling. Gradient l1 regularization for quantization robustness. arXiv preprint arXiv:2002.07520, 2020. sourceExperimental — accuracy not yet measured
• Supported

  The Introduction (p. 2) says ProxQuant uses regularizers that encourage weights toward quantized values. The Abstract (p. 1) contrasts it with straight-through back-propagation through a quantization mapping: ProxQuant back-propagates on full-precision weights and applies a proximal operator between gradient steps. This supports the cited example of a non-STE approach, though the paper describes encouragement toward quantized values rather than immediate enforcement. (LLM, conf=0.91)

  cites [8] 8. Yu Bai, Yu-Xiang Wang, and Edo Liberty. Prox- quant: Quantized neural networks via proximal operators. arXiv preprint arXiv:1810.00861, 2018. source
• ?Could not verify

  Section 2 (p. 2) describes quantizing an already-trained network by clustering its parameters, and Section 3 (p. 3) proposes Hessian-weighted k-means. The passages found when searching for regularization and differentiability do not address Non-STE methods, regularization that forces weights to quantized values, or avoidance of a non-differentiable operator. (LLM, conf=0.73)

  cites [39] 39. Yoojin Choi, Mostafa El-Khamy, and Jungwon Lee. Towards the limit of network quantization. arXiv preprint arXiv:1612.01543, 2016. sourceExperimental — accuracy not yet measured
• Disputed

  The Conclusion (p. 9) says the method explicitly uses a quantization step that projects full-precision weights onto quantized values; Section 3.1 (p. 4) defines that projection operator. Although Section 4 (p. 6) calls quantization a form of regularization, the paper does not describe using a regularization operator instead of quantization. (LLM, conf=0.87)

  Checked by a reasoning model — may be wrong.

  cites [99] 99. Lu Hou and James T Kwok. Loss-aware weight quantization of deep networks. arXiv preprint arXiv:1802.08635, 2018. sourceExperimental — accuracy not yet measured
• ?Could not verify

  The Related Work section (p. 2) mentions quantization and regularizers separately, but discusses the latter as a way to enforce sparsity for pruning. A search for Non-STE methods and regularization that drives weights to quantized values found no passage addressing that claim or avoidance of a non-differentiable quantization operator. (LLM, conf=0.74)

  cites [144] 144. Namhoon Lee, Thalaiyasingam Ajanthan, and Philip HS Torr. Snip: Single-shot network pruning based on connection sensitivity. arXiv preprint arXiv:1810.02340, 2018. sourceExperimental — accuracy not yet measured
• Partly supported

  Section 3.4 (p. 15) describes AdaRound using a regularizer to make continuous rounding variables converge to valid discrete choices. Section 4.1 (p. 19) explains that rounding is non-differentiable and presents STE as a workaround. The copy does not describe a general class called “Non-STE methods” or state that regularization, as a class, removes the need for the quantization operator. (LLM, conf=0.82)

  cites [184] 184. Markus Nagel, Marios Fournarakis, Rana Ali Am- jad, Yelysei Bondarenko, Mart van Baalen, and Tij- men Blankevoort. A white paper on neural network quantization. arXiv preprint arXiv:2106.08295, 2021. sourceExperimental — accuracy not yet measured
• ?Not checked — no full text of the sourcecites [283] 283. Sijie Zhao, Tao Yue, and Xuemei Hu. Distribution- aware adaptive multi-bit quantization. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9281-9290, 2021. source

In the paper

“Furthermore, inspired by the observation that many learned weights are close to zero, there have been attempts to ternarize network by constraining the weights/activations with ternary values, e.g., +1, 0 and -1, thereby explicitly permitting the quantized values to be zero [145,159].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [145] 145. Cong Leng, Zesheng Dou, Hao Li, Shenghuo Zhu, and Rong Jin. Extremely low bit neural network: Squeeze the last bit out with admm. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
• Disputed

  The Abstract (p. 1) describes this paper’s CNN as having weights and activations constrained to {-1,+1}, not ternary values. Section 3.2 (p. 4) explicitly restricts binary activations to 1 or -1, so the cited method does not permit zero as an individual quantized value. (LLM, conf=0.89)

  Checked by a reasoning model — may be wrong.

  cites [159] 159. Xiaofan Lin, Cong Zhao, and Wei Pan. Towards ac- curate binary convolutional neural network. arXiv preprint arXiv:1711.11294, 2017. sourceExperimental — accuracy not yet measured

In the paper

“Recently, in HAWQv3, an integer-only, hardware-aware quantization was introduced [267] that proposed a fast Integer Linear Programming method to find the optimal bit precision for a given applicationspecific constraint (e.g., model size or latency).”

What the sources say

• Disputed

  The Abstract (p. 1) says this paper proposes a differentiable search over discrete weight values, with probabilities optimized during training. The Introduction (p. 2) likewise describes a weight-searching algorithm, not HAWQv3 or an integer linear programming method for selecting bit precision under model-size or latency constraints. (LLM, conf=0.84)

  Checked by a reasoning model — may be wrong.

  cites [267] 267. Zhaohui Yang, Yunhe Wang, Kai Han, Chun- jing Xu, Chao Xu, Dacheng Tao, and Chang Xu. Searching for low-bit weights in quantized neural networks. Advances in neural information processing systems, 2020. sourceExperimental — accuracy not yet measured

In the paper

“Another popular branch is binarycode-based quantization [78,107,118,258,276] where a real-number vector r ∈ R n is quantized into m binary vectors by representing r ≈ m i=1 α i b i , with the scaling factors α i ∈ R and the binary vectors b i ∈ {-1, +1} n .”

What the sources say

• Partly supported

  Section 3.3 (p. 5) describes projecting 32-bit values into multiple 1-bit values, supporting the multiple-binary-representations part of the claim. It does not describe approximating a real vector as a sum of binary vectors with separate scaling factors; Section 3.2 (p. 4) instead says BoolNet replaces scaling factors with Boolean operations. (LLM, conf=0.82)

  cites [78] 78. Nianhui Guo, Joseph Bethge, Haojin Yang, Kai Zhong, Xuefei Ning, Christoph Meinel, and Yu Wang. Boolnet: Minimizing the energy con- sumption of binary neural networks. arXiv preprint arXiv:2106.06991, 2021. sourceExperimental — accuracy not yet measured
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [107] 107. Zehao Huang and Naiyan Wang. Data-driven sparse structure selection for deep neural networks. In Proceedings of the European conference on computer vision (ECCV), pages 304-320, 2018.
• ?Not checked — no full text of the sourcecites [118] 118. Herve Jegou, Matthijs Douze, and Cordelia Schmid. Product quantization for nearest neighbor search. IEEE transactions on pattern analysis and machine intelligence, 33(1):117-128, 2010. source
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [258] 258. Xia Xiao, Zigeng Wang, and Sanguthevar Ra- jasekaran. Autoprune: Automatic network pruning by regularizing auxiliary parameters. In Advances in Neural Information Processing Systems, pages 13681-13691, 2019.
• ?Could not verify

  The Introduction and Section 3.1 discuss Hessian-aware pruning and mention quantization, but neither describes representing a real vector as a sum of scaled binary vectors. Searches for binary-code-based quantization and for binary vectors with scaling factors found no passage addressing that representation; the quantization-related hits were primarily in the References. Our copy does not establish the attributed claim. (LLM, conf=0.72)

  cites [276] 276. Shixing Yu, Zhewei Yao, Amir Gholami, Zhen Dong, Michael W Mahoney, and Kurt Keutzer. Hessian-aware pruning and optimal neural implant. arXiv preprint arXiv:2101.08940, 2021. sourceExperimental — accuracy not yet measured

In the paper

“Symmetric quantization is widely adopted in practice for quantizing weights because zeroing out the zero point can lead to reduction in computational cost during inference [255], and also makes the implementation more straightforward.”

What the sources say

• ?Could not verify

  The Related Work section (p. 2) discusses computational savings from quantization generally, and Appendix A (p. 10) describes the paper’s weight-quantization method. Neither addresses symmetric quantization, zero points, or whether removing a zero point reduces inference cost or simplifies implementation. Searches for those terms found no relevant passage in our copy. (LLM, conf=0.75)

  cites [255] 255. Bichen Wu, Yanghan Wang, Peizhao Zhang, Yuan- dong Tian, Peter Vajda, and Kurt Keutzer. Mixed precision quantization of convnets via differen- tiable neural architecture search. arXiv preprint arXiv:1812.00090, 2018. sourceExperimental — accuracy not yet measured

In the paper

“However, this approach leads to sparse matrix operations, which are known to be hard to accelerate, and which are typically memory-bound [21,66].”

What the sources say

• ?Not checked — no full text of the sourcecites [21] 21. Aydin Buluc and John R Gilbert. Challenges and advances in parallel sparse matrix-matrix multipli- cation. In 2008 37th International Conference on Parallel Processing, pages 503-510. IEEE, 2008. source
• ?Could not verify

  The Introduction (p. 1) discusses hard-threshold activations and quantization, while Section 2 (p. 2) defines network weight matrices. Neither addresses unstructured pruning or whether sparse matrix operations are hard to accelerate or memory-bound; searches for those terms found no supporting passage in our copy. (LLM, conf=0.75)

  cites [66] 66. Abram L Friesen and Pedro Domingos. Deep learn- ing as a mixed convex-combinatorial optimization problem. arXiv preprint arXiv:1710.11573, 2017. sourceExperimental — accuracy not yet measured

In the paper

“Training and inference with high levels of pruning/sparsity, while maintaining state-of-the-art performance, has remained an open problem [16].”

What the sources say

• Partly supported

  Section 2.4 (p. 2) says pruning typically trades model quality for efficiency, and the Conclusion (p. 10) says inconsistent experiments make it impossible to assess the state of the art across pruning techniques. The paper does not establish that maintaining state-of-the-art performance during both training and inference at high sparsity is itself an unresolved problem; Section 3.2 (p. 3) reports that some pruned models retain or exceed their original accuracy. (LLM, conf=0.83)

  cites [16] 16. Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle, and John Guttag. What is the state of neural network pruning? arXiv preprint arXiv:2003.03033, 2020. sourceExperimental — accuracy not yet measured

In the paper

“In this approach, the clipping range is determined by considering all of the weights in convolutional filters of a layer [133], as shown in the third column of Figure 3.”

• Also stated in: § E. Quantization Granularity · plain

  “While this approach is very simple to implement, it often results in sub-optimal accuracy, as the range of each convolutional filter can be vary a lot.”

What the sources say

• Partly supported

  Appendix A.2 (p. 14) says the scaling factor requires determining the range of parameters or activations, which supports the general link between weight ranges and quantization. It does not say that one clipping range is determined from all convolutional filters in a layer. Nor does it address the claimed simplicity or accuracy drawback of layerwise quantization. (LLM, conf=0.78)

  cites [133] 133. Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. I-bert: Integer-only bert quantization. arXiv preprint arXiv:2101.01321, 2021. sourceExperimental — accuracy not yet measured

In the paper

“A typical rule-based non-uniform quantization is to use a logarithmic distribution [179,283], where the quantization steps and levels increase exponentially instead of linearly.”

What the sources say

• Partly supported

  Section 6 (Related work; page unavailable) describes a prior method using logarithmic quantization with base √2, which supports the logarithmic aspect of the claim. It does not characterize that method as a common rule-based approach or explicitly discuss how its steps and levels increase. Section 5 instead describes WRPN’s own clipping-and-rounding scheme. (LLM, conf=0.79)

  cites [179] 179. Asit Mishra, Eriko Nurvitadhi, Jeffrey J Cook, and Debbie Marr. Wrpn: Wide reduced-precision networks. arXiv preprint arXiv:1709.01134, 2017. sourceExperimental — accuracy not yet measured
• ?Not checked — no full text of the sourcecites [283] 283. Sijie Zhao, Tao Yue, and Xuemei Hu. Distribution- aware adaptive multi-bit quantization. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9281-9290, 2021. source

In the paper

“Achieving efficient, real-time NNs with optimal accuracy requires rethinking the design, training, and deployment of NN models [71].”

What the sources say

• Partly supported

  The Introduction and Conclusions support rethinking network design around hardware constraints to improve inference speed and energy efficiency without sacrificing accuracy. The paper discusses training procedures and barriers to deployment, but does not say that rethinking training and deployment is required; those parts of the attributed claim go beyond its findings. (LLM, conf=0.84)

  cites [71] 71. Amir Gholami, Kiseok Kwon, Bichen Wu, Zizheng Tai, Xiangyu Yue, Peter Jin, Sicheng Zhao, and Kurt Keutzer. SqueezeNext: Hardware-aware neural network design. Workshop paper in CVPR, 2018. sourceExperimental — accuracy not yet measured

In the paper

“However, Level 2 usually results in higher accuracy, as finetuning helps the quantized model to recover the accuracy degradation, particularly in ultra-low bit precision settings [85].”

What the sources say

• Partly supported

  Section 3 (p. 3) describes fine-tuning shared weights after quantization and reports no accuracy loss for pruned AlexNet at 8-bit convolutional and 5-bit fully connected precision. But the paper does not compare this method with ZSQ without fine-tuning or establish that fine-tuning usually yields higher accuracy, particularly at ultra-low precision; Table 6 (p. 10) reports accuracy loss at 4/2 bits. (LLM, conf=0.85)

  cites [85] 85. Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149, 2015. sourceExperimental — accuracy not yet measured

In the paper

“However, in integer-only quantization, all the operations are performed using low-precision integer arithmetic [113,132,154,193,267], as illustrated in Figure 6 (Right).This permits the entire inference to be carried out with efficient integer arithmetic, without any floating point dequantization of any parameters or activations.”

What the sources say

• ?Could not verify

  Section 3.1 (p. 4) discusses using fixed batch-normalization statistics during inference and combining normalization with scaling and shifting. Searches for integer-only quantization, low-precision integer arithmetic, and floating-point dequantization found no passage addressing those claims in our copy. (LLM, conf=0.76)

  cites [113] 113. Sergey Ioffe and Christian Szegedy. Batch nor- malization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pages 448-456. PMLR, 2015. sourceExperimental — accuracy not yet measured
• Partly supported

  The introduction (p. 2) and Section 2 (p. 2) describe inference using binary inputs and parameters, with XNOR and bit counting rather than floating- or fixed-point arithmetic. This supports inference without floating-point dequantization, but the paper distinguishes its bitwise approach from quantized networks that perform low-precision integer arithmetic; it does not say that all operations use such arithmetic. (LLM, conf=0.86)

  cites [132] 132. Minje Kim and Paris Smaragdis. Bitwise neural networks. arXiv preprint arXiv:1601.06071, 2016. sourceExperimental — accuracy not yet measured
• ?Could not verify

  The Introduction (p. 2) describes quantizing a matrix for spectral clustering and reducing its storage cost. The system model (p. 3) gives bits per matrix entry, but neither passage addresses integer-only inference. Searches for low-precision integer arithmetic and floating-point dequantization found no passage supporting the claimed execution method. (LLM, conf=0.76)

  cites [154] 154. Zhenyu Liao, Romain Couillet, and Michael W Mahoney. Sparse quantized spectral clustering. International Conference on Learning Representa- tions, 2021. sourceExperimental — accuracy not yet measured
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [193] 193. Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho. Relational knowledge distillation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3967-3976, 2019.
• Partly supported

  Related Works (p. 2) says binary networks use low-level bit operations to accelerate inference, and Optimization Details (p. 4) says the proposed method uses quantized weights for inference. Neither passage establishes that all inference operations use low-precision integer arithmetic or that no floating-point dequantization of parameters or activations occurs. (LLM, conf=0.82)

  cites [267] 267. Zhaohui Yang, Yunhe Wang, Kai Han, Chun- jing Xu, Chao Xu, Dacheng Tao, and Chang Xu. Searching for low-bit weights in quantized neural networks. Advances in neural information processing systems, 2020. sourceExperimental — accuracy not yet measured

In the paper

“For example, performing INT8 addition is 30× more energy efficient and 116× more area efficient as compared to FP32 addition [97].”

What the sources say

• ?Could not verify

  Section 7.1 (p. 59) discusses quantization and integer storage formats, while Section 7.2 (pp. 61–62) discusses accelerator energy consumption. Neither gives an INT8-versus-FP32 addition comparison or the claimed 30× energy and 116× area figures. Searches for those figures and both data types found no passage addressing the claim. (LLM, conf=0.72)

  cites [97] 97. Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, and Alexandra Peste. Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks. arXiv preprint arXiv:2102.00554, 2021. sourceExperimental — accuracy not yet measured

In the paper

“In HAWQv2, this method was extended to mixedprecision activation quantization [50], and was shown to be more than 100x faster than RL based mixed-precision methods [246].”

What the sources say

• ?Could not verify

  The cited paper’s Results (p. 8) discuss the speed and computational cost of L-OBS pruning, not HAWQv2. Searches for HAWQv2, mixed-precision activation quantization, and a 100× speed comparison with RL-based methods found no passage addressing the claim. (LLM, conf=0.77)

  cites [50] 50. Xin Dong, Shangyu Chen, and Sinno Jialin Pan. Learning to prune deep neural networks via layer-wise optimal brain surgeon. arXiv preprint arXiv:1705.07565, 2017. sourceExperimental — accuracy not yet measured
• ?Could not verify

  The citation to this paper is attached to the claim that HAWQv2 is more than 100 times faster than RL-based mixed-precision methods. Our copy's Additional information section compares AttentiveNAS training costs with RL-based neural architecture search, but does not address HAWQv2 or mixed-precision quantization. Searches for HAWQv2, activation quantization, and the claimed speed comparison found no supporting passage; page numbers were not available. (LLM, conf=0.76)

  cites [246] 246. Dilin Wang, Meng Li, Chengyue Gong, and Vikas Chandra. Attentivenas: Improving neural architec- ture search via attentive sampling. arXiv preprint arXiv:2011.09011, 2020. sourceExperimental — accuracy not yet measured

In the paper

“c) Improved Training Method: Another interesting branch of work aims for better training methods for binary/ternary models [5,20,44,73,160,164,285,288].”

What the sources say

• ?Not checked — no full text of the sourcecites [5] 5. Milad Alizadeh, Javier Fernández-Marqués, Nicholas D Lane, and Yarin Gal. An empirical study of binary neural networks' optimisation. In International Conference on Learning Representations, 2018.
• Supported

  The Introduction (p. 1) explicitly says the paper focuses on improving the training of binary networks through several methodological changes. It therefore supports the citing sentence as an example of work on better training methods for binary or ternary models; this paper addresses the binary case. (LLM, conf=0.93)

  cites [20] 20. Adrian Bulat, Georgios Tzimiropoulos, Jean Kos- saifi, and Maja Pantic. Improved training of binary networks for human pose estimation and image recognition. arXiv preprint arXiv:1904.05868, 2019. source
• Partly supported

  Section 3.1 (p. 3) describes distillation methods for training smaller student networks, and Section 4.2 (p. 7) reports that attention transfer outperformed other training approaches. The paper does not establish that these methods train binary or ternary models; its reported models instead use cheap convolutional blocks. (LLM, conf=0.82)

  cites [44] 44. Elliot J Crowley, Gavin Gray, and Amos J Storkey. Moonshine: Distilling with cheap convolutions. In NeurIPS, pages 2893-2903, 2018. sourceExperimental — accuracy not yet measured

In the paper

“Bi-Real Net [164] introduces identity shortcuts connecting activations to activations in consecutive blocks, through which 32-bit activations can be propagated.”

What the sources say

• ?Could not verify

  The Architecture section (p. 2) describes BERT transformer blocks, and the RoBERTa section (p. 6) describes changes to BERT pretraining. Searches for Bi-Real Net, identity shortcuts, and 32-bit activations found no passage in our copy addressing the cited claim. (LLM, conf=0.78)

  cites [164] 164. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019. sourceExperimental — accuracy not yet measured

In the paper

“To address this the recent work of [267] directly deploys quantized operations in hardware, and measures the actual deployment latency of each layer for different quantization bit precisions.”

What the sources say

• ?Could not verify

  The Experiments section (p. 6) tests quantized networks at different bitwidths, while the Introduction (p. 1) discusses hardware deployment only as a general possibility. Neither passage establishes that this work deployed operations on hardware or measured actual per-layer latency; searches for latency and hardware deployment found no such measurement in our copy. (LLM, conf=0.74)

  cites [267] 267. Zhaohui Yang, Yunhe Wang, Kai Han, Chun- jing Xu, Chao Xu, Dacheng Tao, and Chang Xu. Searching for low-bit weights in quantized neural networks. Advances in neural information processing systems, 2020. sourceExperimental — accuracy not yet measured

In the paper

“Previous methods of knowledge distillation focus on exploring different knowledge sources.[95,150,192] use logits (the soft probabilities) as the source of knowledge, while [3,207,269] try to leverage the knowledge from intermediate layers.”

What the sources say

• ?Could not verify

  The Introduction (p. 1) describes GELU as an activation function, and the MNIST Classification section (p. 2) compares it with other activation functions. Searches for knowledge distillation, logits, intermediate layers, and teacher–student knowledge transfer found no passage addressing the attributed claim. (LLM, conf=0.77)

  cites [95] 95. Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (GELUs). arXiv preprint arXiv:1606.08415, 2016. sourceExperimental — accuracy not yet measured
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [150] 150. Yuhang Li, Ruihao Gong, Fengwei Yu, Xin Dong, and Xianglong Liu. Dms: Differentiable dimension search for binary neural networks. International Conference on Learning Representations, 2020.
• ?Could not verify

  The Introduction (p. 1) and Section 2 (p. 2) discuss neural-network pruning, not knowledge distillation. Searches for logits, intermediate layers as knowledge sources, and teacher–student distillation found no passage in our copy addressing the claim. (LLM, conf=0.76)

  cites [192] 192. Sejun Park, Jaeho Lee, Sangwoo Mo, and Jin- woo Shin. Lookahead: a far-sighted alternative of magnitude-based pruning. arXiv preprint arXiv:2002.04809, 2020. sourceExperimental — accuracy not yet measured

In the paper

“b) Improved Loss function: Another branch of works focuses on the choice of loss function [48,98,99,251,284].”

What the sources say

• ?Could not verify

  Section 2.2 (pp. 4–5) uses training loss to update pruning scores and says the experiments use cross-entropy loss. The Discussion (p. 9) describes the contribution as finding accurate binary subnetworks by pruning randomly weighted networks. I found no passage presenting the choice or improvement of the loss function as this paper’s focus (searched: choice of loss function, improved loss function, loss-aware binarization). (LLM, conf=0.76)

  cites [48] 48. James Diffenderfer and Bhavya Kailkhura. Multi- prize lottery ticket hypothesis: Finding accurate binary neural networks by pruning a randomly weighted network. In International Conference on Learning Representations, 2021. sourceExperimental — accuracy not yet measured
• Not checked yetcites [98] 98. Mark Horowitz. 1.1 computing's energy problem (and what we can do about it). In 2014 IEEE In- ternational Solid-State Circuits Conference Digest of Technical Papers (ISSCC), pages 10-14. IEEE, 2014.
• Partly supported

  The Introduction (p. 2) and Conclusion (p. 9) say the method explicitly considers how weight quantization affects the loss. They do not describe choosing or improving the loss function itself; the work focuses on the quantization algorithm. (LLM, conf=0.84)

  cites [99] 99. Lu Hou and James T Kwok. Loss-aware weight quantization of deep networks. arXiv preprint arXiv:1802.08635, 2018. sourceExperimental — accuracy not yet measured

In the paper

“To address this, BNN+ [44] introduces a continuous approximation for the derivative of the sign function, while [198,261,272] replace the sign function with smooth, differentiable functions that gradually sharpens and approaches the sign function.”

What the sources say

• ?Could not verify

  Section 3.2 (p. 4) describes bottleneck and grouped convolutions, not BNN+ or a derivative of the sign function. A search of our copy for BNN+, sign-function derivatives, and surrogate gradients found no passage addressing the attributed claim. (LLM, conf=0.76)

  cites [44] 44. Elliot J Crowley, Gavin Gray, and Amos J Storkey. Moonshine: Distilling with cheap convolutions. In NeurIPS, pages 2893-2903, 2018. sourceExperimental — accuracy not yet measured

In the paper

“An important work here is BinaryConnect [42] which constrains the weights to either +1 or -1.”

What the sources say

• ?Could not verify

  The Abstract (p. 1) describes this paper's use of floating-point, fixed-point, and dynamic fixed-point formats, including 10-bit multiplications. The Related Works section (p. 8) does not mention BinaryConnect. Searches for BinaryConnect and weights restricted to +1 or -1 found no passage in our copy addressing the claim. (LLM, conf=0.76)

  cites [42] 42. Matthieu Courbariaux, Yoshua Bengio, and Jean- Pierre David. Training deep neural networks with low precision multiplications. arXiv preprint arXiv:1412.7024, 2014. sourceExperimental — accuracy not yet measured

In the paper

“The choices of teacher models are also well studied, where [235,273] use multiple teacher models to jointly supervise the student model, while [43,277] apply self-distillation without an extra teacher model.”

What the sources say

• ?Could not verify

  The Related Work section (pp. 2–3) and Training BinaryNets with high accuracy (pp. 4–5) discuss binary-network training and multiple binarizations, not teacher-model choices. Searches for multiple teachers, student supervision, and self-distillation found no passage in our copy addressing the attributed claim. (LLM, conf=0.76)

  cites [235] 235. Wei Tang, Gang Hua, and Liang Wang. How to train a compact binary neural network with high accuracy? In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017. sourceExperimental — accuracy not yet measured
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [273] 273. Penghang Yin, Shuai Zhang, Jiancheng Lyu, Stan- ley Osher, Yingyong Qi, and Jack Xin. Blended coarse gradient descent for full quantization of deep neural networks. Research in the Mathemati- cal Sciences, 6(1):14, 2019.
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [43] 43. Matthieu Courbariaux, Yoshua Bengio, and Jean- Pierre David. BinaryConnect: Training deep neural networks with binary weights during propagations. In Advances in neural information processing systems, pages 3123-3131, 2015.

In the paper

“One disadvantage of these exploration-based methods [246,254] is that they often require large computational resources, and their performance is typically sensitive to hyperparameters and even initialization.”

What the sources say

• Partly supported

  The Introduction and Related Work describe conventional neural architecture search as computationally expensive, and Related Work says some weight-sharing search methods are sensitive to hyperparameter choices, including random seeds. The paper does not establish these points specifically for mixed-precision methods, and it does not show that the expensive methods are also sensitive to initialization. Page numbers are unavailable in our copy. (LLM, conf=0.84)

  cites [246] 246. Dilin Wang, Meng Li, Chengyue Gong, and Vikas Chandra. Attentivenas: Improving neural architec- ture search via attentive sampling. arXiv preprint arXiv:2011.09011, 2020. sourceExperimental — accuracy not yet measured
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [254] 254. Bichen Wu, Alvin Wan, Xiangyu Yue, Peter Jin, Sicheng Zhao, Noah Golmant, Amir Gholaminejad, Joseph Gonzalez, and Kurt Keutzer. Shift: A zero flop, zero parameter alternative to spatial convolu- tions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9127-9135, 2018.

In the paper

“a) Quantization Error Minimization: The first branch of solutions aims to minimize the quantization error, i.e., the gap between the real values and the quantized values [19,34,62,103,151,158,164,169,178,218,248].”

What the sources say

• Partly supported

  Section 2.2 (p. 3) describes scaling factors used to reduce quantization error, and the Introduction (p. 1) contrasts reducing approximation error with optimizing task loss. The paper does not identify error minimization as a “first branch” of solutions; its own method emphasizes task-loss optimization instead. (LLM, conf=0.85)

  cites [19] 19. Adrian Bulat and Georgios Tzimiropoulos. Xnor- net++: Improved binary neural networks. arXiv preprint arXiv:1909.13863, 2019. sourceExperimental — accuracy not yet measured
• ?Not checked — no full text of the sourcecites [34] 34. Xiuyi Chen, Guangcan Liu, Jing Shi, Jiaming Xu, and Bo Xu. Distilled binary neural network for monaural speech separation. In 2018 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE, 2018. source
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [62] 62. Jun Fang, Ali Shafiee, Hamzah Abdel-Aziz, David Thorsley, Georgios Georgiadis, and Joseph H Has- soun. Post-training piecewise linear quantization for deep neural networks. In European Conference on Computer Vision, pages 69-86. Springer, 2020.

In the paper

“While several interesting works exist in this area [10,26,123,137,173], the proposed methods often require a lot of hyperparameter tuning, or they only work for a few NN models on relatively easy learning tasks.”

What the sources say

• Partly supported

  Section 2 (Previous Work) says that some more aggressively quantized networks lost accuracy on complex models and challenging datasets, supporting the concern about limited applicability. It does not say that 8-bit methods often require extensive hyperparameter tuning. The Abstract and Section 7 (Discussion) instead report this paper’s 8-bit method working on the large-scale ImageNet-1K dataset without harming accuracy. (LLM, conf=0.83)

  cites [10] 10. Ron Banner, Itay Hubara, Elad Hoffer, and Daniel Soudry. Scalable methods for 8-bit training of neural networks. Advances in neural information processing systems, 2018. sourceExperimental — accuracy not yet measured
• Partly supported

  The Related Work section (p. 2) says integer-based training approaches required custom tuning and lacked generalizability beyond the models discussed, including LeNet-5 and AlexNet. It does not characterize the learning tasks as relatively easy, so that part of the claim is unsupported. (LLM, conf=0.86)

  cites [26] 26. Léopold Cambier, Anahita Bhiwandiwalla, Ting Gong, Mehran Nekuii, Oguz H Elibol, and Hanlin Tang. Shifted and squeezed 8-bit floating point format for low-precision training of deep neural networks. arXiv preprint arXiv:2001.05674, 2020. sourceExperimental — accuracy not yet measured
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [123] 123. Qing Jin, Linjie Yang, and Zhenyu Liao. Adabits: Neural network quantization with adaptive bit- widths. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 2146-2156, 2020.

In the paper

“Product quantization [74,227,256] is an extension of vector quantization, where the weight matrix is divided into submatrices and vector quantization is applied to each submatrix.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [74] 74. Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan. Differentiable soft quantization: Bridging full-precision and low-bit neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4852-4861, 2019.
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [227] 227. Pierre Stock, Angela Fan, Benjamin Graham, Edouard Grave, Rémi Gribonval, Herve Jegou, and Armand Joulin. Training with quantization noise for extreme model compression. In International Conference on Learning Representations, 2021.
• ?Could not verify

  Section 3.2 (pp. 4–5) discusses per-row, per-column, and per-tensor quantization, but does not describe product quantization or applying vector quantization to weight submatrices. Searches for product quantization, vector quantization, submatrices, and codebooks found no passage in our copy that addresses the claimed procedure. (LLM, conf=0.75)

  cites [256] 256. Hao Wu, Patrick Judd, Xiaojie Zhang, Mikhail Isaev, and Paulius Micikevicius. Integer quan- tization for deep learning inference: Princi- ples and empirical evaluation. arXiv preprint arXiv:2004.09602, 2020. sourceExperimental — accuracy not yet measured

In the paper

“CMSIS-NN [136] is a library from ARM that helps quantizing and deploying NN models onto the ARM Cortex-M cores.Specifically, the library leverages fixed-point quantization [113,154,267] with power-oftwo scaling factors so that quantization and dequantization processes can be carried out efficiently with bit shifting operations.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [136] 136. Se Jung Kwon, Dongsoo Lee, Byeongwook Kim, Parichay Kapoor, Baeseong Park, and Gu-Yeon Wei. Structured compression by weight encryption for unstructured pruning and quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1909-1918, 2020.
• ?Could not verify

  The cited paper’s Section 3.1 (p. 4) discusses batch normalization during inference, including a linear transform, but not fixed-point quantization. Searches for CMSIS-NN, ARM Cortex-M, power-of-two scaling, and bit shifting found no passage addressing the attributed claim. (LLM, conf=0.76)

  cites [113] 113. Sergey Ioffe and Christian Szegedy. Batch nor- malization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pages 448-456. PMLR, 2015. sourceExperimental — accuracy not yet measured
• ?Could not verify

  The Introduction (p. 2) and Section 4.3 (pp. 9–10) discuss quantizing matrices for spectral clustering, not neural-network deployment. Searches for CMSIS-NN, ARM Cortex-M, fixed-point quantization, and power-of-two scaling found no passage addressing the attributed claim. (LLM, conf=0.76)

  cites [154] 154. Zhenyu Liao, Romain Couillet, and Michael W Mahoney. Sparse quantized spectral clustering. International Conference on Learning Representa- tions, 2021. sourceExperimental — accuracy not yet measured

In the paper

“GAP-8 [64], a RISC-V SoC (System on Chip) for edge inference with a dedicated CNN accelerator, is another example of an edge processor that only supports integer arithmetic.”

What the sources say

• ?Could not verify

  The Introduction (p. 1) says that 8-bit integer quantization is used across cloud and edge devices, but it does not identify GAP-8. Searches for GAP-8, GAP8, RISC-V, and a dedicated CNN accelerator found no passage establishing GAP-8’s design or that it only supports integer arithmetic. (LLM, conf=0.76)

  cites [64] 64. Alexander Finkelstein, Uri Almog, and Mark Grobman. Fighting quantization bias with bias. arXiv preprint arXiv:1906.03193, 2019. sourceExperimental — accuracy not yet measured

In the paper

“Moreover, support for INT4 quantization was only recently added to TVM [267].”

What the sources say

• ?Could not verify

  The Abstract (p. 1) discusses 4-bit quantization, and Related Works (p. 2) mentions platform support for network quantization generally. Neither identifies TVM or says when INT4 support was added. Searches for “TVM,” “INT4,” and recent platform support found no passage addressing the claim. (LLM, conf=0.76)

  cites [267] 267. Zhaohui Yang, Yunhe Wang, Kai Han, Chun- jing Xu, Chao Xu, Dacheng Tao, and Chang Xu. Searching for low-bit weights in quantized neural networks. Advances in neural information processing systems, 2020. sourceExperimental — accuracy not yet measured

In the paper

“[157] shows that INT8 inference of popular computer vision models, including ResNet50 [88], VGG-19 [224], and inceptionV3 [230] using TVM [32] quantization library, can achieve 3.89×, 3.32×, and 5.02× speedup on NVIDIA GTX 1080, respectively.”

What the sources say

• ?Could not verify

  The Experimental Settings (section 4.1) describe inference measurements on a GTX TITAN X and use a GTX 1080 Ti for training. The Quantitative Evaluation (section 4.2) reports filter-pruning results for ResNet-50 and VGG-16, but our copy does not report INT8 inference using TVM, VGG-19 or InceptionV3, or the three claimed speedups (searched: INT8, TVM, quantization, and the model names). (LLM, conf=0.76)

  cites [157] 157. Shaohui Lin, Rongrong Ji, Yuchao Li, Yongjian Wu, Feiyue Huang, and Baochang Zhang. Acceler- ating convolutional networks via global & dynamic filter pruning. In IJCAI, pages 2425-2432, 2018. sourceExperimental — accuracy not yet measured

In the paper

“In contrast, sub-channelwise quantization may result in significant overhead and is not currently the standard choice”

What the sources say

• ?Not checked — no full text of the sourcecites [68] 68. AE Gamal, L Hemachandra, Itzhak Shperling, and V Wei. Using simulated annealing to design good codes. IEEE Transactions on Information Theory, 33(1):116-123, 1987. source

In the paper

“Since there is no closed-form solution for minimizing the error between r and m i=1 α i b i , previous research relies on heuristic solutions.”

In the paper

“Using symmetric quantization, however, simplifies the quantization function in Eq. 2 by replacing the zero point with Z = 0:”

In the paper

“One could group multiple different channels inside a layer to calculate the clipping range (of either activations or convolution kernels).”

• Also stated in: § E. Quantization Granularity · hedged

  “This could be helpful for cases where the distribution of the parameters across a single convolution/activation varies a lot.”

• Also stated in: § E. Quantization Granularity · plain

  “However, this approach inevitably comes with the extra cost of accounting for different scaling factors.”

In the paper

“Each of these convolutional filters can have a different range of values.”

In the paper

“It enables the practitioner to adjust the clipping range for each individual kernel with negligible overhead.”

In the paper

“Different layers in a Neural Net have different impact on the loss function, and this motivates a mixed-precision approach to quantization.”

In the paper

“The high level intuition has been that the stochastic quantization may allow a NN to explore more, as compared to deterministic quantization.”

In the paper

“Generally, non-uniform quantization enables us to better capture the signal information, by assigning bits and discreitizing the range of parameters non-uniformly.”

In the paper

“Furthermore, ZeroQ [24] shows that the synthetic data can be used for sensitivity measurement as well as calibration, thereby enabling mixed-precision post-training quantization without any access to the training/validation data.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [24] 24. Yaohui Cai, Zhewei Yao, Zhen Dong, Amir Gho- lami, Michael W Mahoney, and Kurt Keutzer. Zeroq: A novel zero shot quantization framework. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13169-13178, 2020.

In the paper

“As such, the uniform quantization is currently the de-facto method due to its simplicity and its efficient mapping to hardware.”

In the paper

“Zero Shot (aka data free) quantization performs the entire quantization without any access to the training/validation data.”

In the paper

“Unlike QAT, which requires a sufficient amount of training data for retraining, PTQ has an additional advantage that it can be applied in situations where data is limited or unlabeled.”

In the paper

“Another notable work is AdaRound [181] which shows that the naive round-to-nearest method for quantization can counter-intuitively results in sub-optimal solutions, and it proposes an adaptive rounding method that better reduces the loss.”

What the sources say

• ?Not checked — no full text of the sourcecites [181] 181. Lopamudra Mukherjee, Sathya N Ravi, Jiming Peng, and Vikas Singh. A biresolution spectral framework for product quantization. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3329-3338, 2018. source

In the paper

“However, non-uniform quantization schemes are typically difficult to deploy efficiently on general computation hardware, e.g., GPU and CPU.”

In the paper

“Low-precision logic has multiple benefits over the full-precision counterpart in terms of latency, power consumption, and area efficiency.”

In the paper

“However, a major challenge with stochastic quantization methods is the overhead of creating random numbers for every single weight update, and as such they are not yet adopted widely in practice.”

In the paper

“Despite interesting works in this area, these methods often require a lot of tuning and so far STE approach is the most commonly used method.”

In the paper

“The work of [197] broadly categorizes these solutions into three branches.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [197] 197. Haotong Qin, Zhongang Cai, Mingyuan Zhang, Yifu Ding, Haiyu Zhao, Shuai Yi, Xianglong Liu, and Hao Su. Bipointnet: Binary neural network for point clouds. International Conference on Learning Representations, 2021.

In the paper

“For instance, the peak binary arithmetic on NVIDIA V100 GPUs is 8x higher than INT8.”

In the paper

“Binarization, where the quantized values are constrained to a 1-bit representation, thereby drastically reducing the memory requirement by 32×, is the most extreme quantization method.”

In the paper

“Binarized NN [107] (BNN) extends this idea by binarizing the activations as well as the weights.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [107] 107. Zehao Huang and Naiyan Wang. Data-driven sparse structure selection for deep neural networks. In Proceedings of the European conference on computer vision (ECCV), pages 304-320, 2018.

In the paper

“Ternarization also drastically reduces the inference latency by eliminating the costly matrix multiplications as binarization does.”

In the paper

“Besides the memory advantages, binary (1-bit) and ternary (2-bit) operations can often be computed efficiently with bit-wise arithmetic and can achieve significant acceleration over higher precisions, such as FP32 and INT8.”

In the paper

“Quantization, combined with efficient lowprecision logic and dedicated deep learning accelerators, has been one important driving force for the evolution of such edge processors.”

In the paper

“As a result, as our bibliography attests, the problem of quantization in NN models has been a highly active research area.”

In the paper

“For instance, Nvidia's TensorRT does not currently support sub-INT8 quantization.”

In the paper

“However, there is currently very little work exploring what are the optimal combinations of these methods.”

In the paper

“In addition, many edge processors do not have any support floating point operations, especially in micro-controllers.”

In the paper

“Edge devices often have tight resource constraints including compute, memory, and importantly power budget.”

In the paper

“Unlike Cloud TPUs that run in Google data centers with a large amount of computing resources, the Edge TPU is designed for small and low-power devices, and thereby it only supports 8-bit arithmetic.”

In the paper

“As discussed above, quantization is only one of the methods for efficient deployment of NNs.Other methods include efficient NN architecture design, co-design of hardware and NN architecture, pruning, and knowledge distillation.Quantization can be coupled with these other approaches.”

In the paper

“For example, with quantization-aware training, we might converge to a different solution, far away from the original solution with single precision parameters, but still get good accuracy.”

In the paper

“Second, most current NN models are heavily over-parameterized.So, there is ample opportunity for reducing the bit precision without impacting accuracy.”

In the paper

“However, the main difference between quantization methods for machine learning is that fundamentally we are not interested to compress the signal with minimum change/error as compared to the original signal.Instead, the goal is to find a reduced-precision representation that results in as small loss as possible.”

In the paper

“And in [189], the authors apply more clusters for more important quantization ranges to better preserve the information.”

What the sources say

• ?Not checked — no full text of the sourcecites [189] 189. BM Oliver, JR Pierce, and Claude E Shannon. The philosophy of pcm. Proceedings of the IRE, 36(11):1324-1331, 1948. source

In the paper

“With current methods, it is straightforward to quantize and deploy different NN models to INT8, without losing accuracy.”

In the paper

“However, the software for lower bit-precision quantization is not widely available, and sometimes it is non-existent.”

In the paper

“For instance, the recent work of [34] shows that changing the width of the NN architecture could reduce/remove generalization gap after quantization.”

What the sources say

• ?Not checked — no full text of the sourcecites [34] 34. Xiuyi Chen, Guangcan Liu, Jing Shi, Jiaming Xu, and Bo Xu. Distilled binary neural network for monaural speech separation. In 2018 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE, 2018. source

In the paper

“NN models must be quantized using either quantization-aware training or posttraining quantization of TensorFlow.”

In the paper

“A good example is provided by the recent NVIDIA Turing GPUs, and in particular T4 GPUs, which include the Turing Tensor Cores.Tensor Cores are specialized execution units designed for efficient lowprecision matrix multiplications.”

In the paper

“For example, in [74] the residuals after k-means product quantization are further recursively quantized.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [74] 74. Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan. Differentiable soft quantization: Bridging full-precision and low-bit neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4852-4861, 2019.

In the paper

“Thus, different layers in the NN have different impact on the loss function, and this motivates interesting approaches such mixed-precision quantization.”

In the paper

“But this accuracy degradation may be acceptable for less critical applications.”

In the paper

“Because some of the ARM Cortex-M cores do not include dedicated floatingpoint units, the models should first be quantized before deployment.”

In the paper

“We have said that quantization not only reduces the model size, but it also enables faster speed and requires less power, in particular for hardware that has lowprecision logic.”