A Survey of Quantization Methods for Efficient Neural Network Inference

May combine two claims.

The quantizer can be trained jointly with model parameters; learnable quantizers train steps and levels by iterative optimization or gradient descent.

“Furthermore, the quantizer itself can also be jointly trained with the model parameters.These methods are referred to as learnable quantizers, and the quantization steps/levels are generally trained with iterative optimization [258,276] or gradient descent [125,158,264].”

• cites [258] 258. Xia Xiao, Zigeng Wang, and Sanguthevar Ra- jasekaran. Autoprune: Automatic network pruning by regularizing auxiliary parameters. In Advances in Neural Information Processing Systems, pages 13681-13691, 2019.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [276] 276. Shixing Yu, Zhewei Yao, Amir Gholami, Zhen Dong, Michael W Mahoney, and Kurt Keutzer. Hessian-aware pruning and optimal neural implant. arXiv preprint arXiv:2101.08940, 2021. sourceCould not verify

  Section 3.1 mentions a Hessian-based approach used in quantization, but does not discuss training a quantizer jointly with model parameters. The Methodology section concerns pruning, and the References list quantization papers without describing learnable steps or levels. Searches for joint quantizer training and optimized quantization steps found no passage addressing the claim. (LLM, conf=0.74)

  Experimental — accuracy not yet measured
• cites [125] 125. Felix Juefei-Xu, Vishnu Naresh Boddeti, and Mar- ios Savvides. Local binary convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 19-28, 2017. sourceDisputed

  Section 3.2, “Learning with LBC Layers” (page unavailable), says that only the learnable 1×1 filters are updated; the binary anchor weights remain unaffected. The paper therefore does not describe training quantization steps or levels jointly with model parameters by gradient descent. (LLM, conf=0.88)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured
• cites [158] 158. Wuwei Lin. Automating optimization of quantized deep learning models on cuda: https://tvm.apache.org/2019/04/29/opt-cuda- quantized, 2019.Not checked yet

  Not checked in the free run.

• cites [264] 264. Huanrui Yang, Lin Duan, Yiran Chen, and Hai Li. Bsq: Exploring bit-level sparsity for mixed- precision neural network quantization. arXiv preprint arXiv:2102.10462, 2021. sourcePartly supported

  The Introduction (p. 2) and §3.1 (p. 4) describe training quantized weight bits with gradient-based optimization, which supports the gradient-descent part of the attribution. BSQ uses that training and periodic precision adjustment to obtain a mixed-precision scheme; our copy does not describe directly learning quantization step sizes or levels as separate quantizer parameters. (LLM, conf=0.82)

  Experimental — accuracy not yet measured

May combine two claims.

Although STE dominates, alternative methods exist in the literature.

“While STE is the mainstream approach [226,289], other approaches have also been explored in the literature [2,25,31,59,144,164].”

• cites [226] 226. S. M. Stigler. The History of Statistics: The Measurement of Uncertainty before 1900. Harvard University Press, Cambridge, 1986.Not checked yet

  Not checked in the free run.

• cites [289] 289. Shilin Zhu, Xin Dong, and Hao Su. Binary ensemble neural network: More bits per network or more networks per bit? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4923-4932, 2019. sourceDisputed

  The citation is attached to the claim that approaches other than STE have been explored. Section 4, “Binary Ensemble Neural Network,” says this paper uses STE during back-propagation; its bagging and boosting strategies are not presented as alternatives to STE. (LLM, conf=0.86)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured

May combine two claims.

Non-STE methods use regularization to force weights to quantized values, avoiding the non-differentiable quantization operator.

“Another different class of alternative methods tries to use regularization operators to enforce the weight to be quantized.This removes the need to use the non-differentiable quantization operator in Eq. 2. These are often referred to as Non-STE methods [4, 8, 39, 99, 144, 184, 283].”

• cites [4] 4. Milad Alizadeh, Arash Behboodi, Mart van Baalen, Christos Louizos, Tijmen Blankevoort, and Max Welling. Gradient l1 regularization for quantization robustness. arXiv preprint arXiv:2002.07520, 2020. sourcePartly supported

  Related Work (p. 5) describes regularization methods that encourage weights to lie near quantization grid points, separately from methods that mimic quantization during the forward pass. But this paper’s own method regularizes the ℓ₁-norm of gradients to make a model robust to later quantization; it does not use regularization to force its weights to quantized values (Methods, p. 3; Conclusion, p. 9). (LLM, conf=0.88)

  Experimental — accuracy not yet measured
• cites [8] 8. Yu Bai, Yu-Xiang Wang, and Edo Liberty. Prox- quant: Quantized neural networks via proximal operators. arXiv preprint arXiv:1810.00861, 2018. sourceSupported
• cites [39] 39. Yoojin Choi, Mostafa El-Khamy, and Jungwon Lee. Towards the limit of network quantization. arXiv preprint arXiv:1612.01543, 2016. sourceCould not verify

  Section 2 (p. 2) describes quantizing an already-trained network by clustering its parameters, and Section 3 (p. 3) proposes Hessian-weighted k-means. The passages found when searching for regularization and differentiability do not address Non-STE methods, regularization that forces weights to quantized values, or avoidance of a non-differentiable operator. (LLM, conf=0.73)

  Experimental — accuracy not yet measured
• cites [99] 99. Lu Hou and James T Kwok. Loss-aware weight quantization of deep networks. arXiv preprint arXiv:1802.08635, 2018. sourceDisputed

  The Conclusion (p. 9) says the method explicitly uses a quantization step that projects full-precision weights onto quantized values; Section 3.1 (p. 4) defines that projection operator. Although Section 4 (p. 6) calls quantization a form of regularization, the paper does not describe using a regularization operator instead of quantization. (LLM, conf=0.87)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured
• cites [144] 144. Namhoon Lee, Thalaiyasingam Ajanthan, and Philip HS Torr. Snip: Single-shot network pruning based on connection sensitivity. arXiv preprint arXiv:1810.02340, 2018. sourceCould not verify

  The Related Work section (p. 2) mentions quantization and regularizers separately, but discusses the latter as a way to enforce sparsity for pruning. A search for Non-STE methods and regularization that drives weights to quantized values found no passage addressing that claim or avoidance of a non-differentiable quantization operator. (LLM, conf=0.74)

  Experimental — accuracy not yet measured
• cites [184] 184. Markus Nagel, Marios Fournarakis, Rana Ali Am- jad, Yelysei Bondarenko, Mart van Baalen, and Tij- men Blankevoort. A white paper on neural network quantization. arXiv preprint arXiv:2106.08295, 2021. sourcePartly supported

  Section 3.4 (p. 15) describes AdaRound using a regularizer to make continuous rounding variables converge to valid discrete choices. Section 4.1 (p. 19) explains that rounding is non-differentiable and presents STE as a workaround. The copy does not describe a general class called “Non-STE methods” or state that regularization, as a class, removes the need for the quantization operator. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites [283] 283. Sijie Zhao, Tao Yue, and Xuemei Hu. Distribution- aware adaptive multi-bit quantization. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9281-9290, 2021. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Ternarization constrains weights or activations to +1, 0, and -1, explicitly allowing quantized values to be zero.

“Furthermore, inspired by the observation that many learned weights are close to zero, there have been attempts to ternarize network by constraining the weights/activations with ternary values, e.g., +1, 0 and -1, thereby explicitly permitting the quantized values to be zero [145,159].”

• cites [145] 145. Cong Leng, Zesheng Dou, Hao Li, Shenghuo Zhu, and Rong Jin. Extremely low bit neural network: Squeeze the last bit out with admm. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [159] 159. Xiaofan Lin, Cong Zhao, and Wei Pan. Towards ac- curate binary convolutional neural network. arXiv preprint arXiv:1711.11294, 2017. sourceDisputed

  The Abstract (p. 1) describes this paper’s CNN as having weights and activations constrained to {-1,+1}, not ternary values. Section 3.2 (p. 4) explicitly restricts binary activations to 1 or -1, so the cited method does not permit zero as an individual quantized value. (LLM, conf=0.89)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured

May combine two claims.

HAWQv3 introduces integer-only hardware-aware quantization with a fast integer linear programming method to find optimal bit precision under application constraints.

“Recently, in HAWQv3, an integer-only, hardware-aware quantization was introduced [267] that proposed a fast Integer Linear Programming method to find the optimal bit precision for a given applicationspecific constraint (e.g., model size or latency).”

• cites [267] 267. Zhaohui Yang, Yunhe Wang, Kai Han, Chun- jing Xu, Chao Xu, Dacheng Tao, and Chang Xu. Searching for low-bit weights in quantized neural networks. Advances in neural information processing systems, 2020. sourceDisputed

  The Abstract (p. 1) says this paper proposes a differentiable search over discrete weight values, with probabilities optimized during training. The Introduction (p. 2) likewise describes a weight-searching algorithm, not HAWQv3 or an integer linear programming method for selecting bit precision under model-size or latency constraints. (LLM, conf=0.84)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

Binary-code-based quantization represents a real vector as a sum of scaled binary vectors.

“Another popular branch is binarycode-based quantization [78,107,118,258,276] where a real-number vector r ∈ R n is quantized into m binary vectors by representing r ≈ m i=1 α i b i , with the scaling factors α i ∈ R and the binary vectors b i ∈ {-1, +1} n .”

• cites [78] 78. Nianhui Guo, Joseph Bethge, Haojin Yang, Kai Zhong, Xuefei Ning, Christoph Meinel, and Yu Wang. Boolnet: Minimizing the energy con- sumption of binary neural networks. arXiv preprint arXiv:2106.06991, 2021. sourcePartly supported

  Section 3.3 (p. 5) describes projecting 32-bit values into multiple 1-bit values, supporting the multiple-binary-representations part of the claim. It does not describe approximating a real vector as a sum of binary vectors with separate scaling factors; Section 3.2 (p. 4) instead says BoolNet replaces scaling factors with Boolean operations. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites [107] 107. Zehao Huang and Naiyan Wang. Data-driven sparse structure selection for deep neural networks. In Proceedings of the European conference on computer vision (ECCV), pages 304-320, 2018.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [118] 118. Herve Jegou, Matthijs Douze, and Cordelia Schmid. Product quantization for nearest neighbor search. IEEE transactions on pattern analysis and machine intelligence, 33(1):117-128, 2010. sourceNot checked — no full text of the source
• cites [258] 258. Xia Xiao, Zigeng Wang, and Sanguthevar Ra- jasekaran. Autoprune: Automatic network pruning by regularizing auxiliary parameters. In Advances in Neural Information Processing Systems, pages 13681-13691, 2019.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [276] 276. Shixing Yu, Zhewei Yao, Amir Gholami, Zhen Dong, Michael W Mahoney, and Kurt Keutzer. Hessian-aware pruning and optimal neural implant. arXiv preprint arXiv:2101.08940, 2021. sourceCould not verify

  The Introduction and Section 3.1 discuss Hessian-aware pruning and mention quantization, but neither describes representing a real vector as a sum of scaled binary vectors. Searches for binary-code-based quantization and for binary vectors with scaling factors found no passage addressing that representation; the quantization-related hits were primarily in the References. Our copy does not establish the attributed claim. (LLM, conf=0.72)

  Experimental — accuracy not yet measured

May combine two claims.

Symmetric quantization is widely used for weights because eliminating the zero point can reduce inference computational cost and simplify implementation.

“Symmetric quantization is widely adopted in practice for quantizing weights because zeroing out the zero point can lead to reduction in computational cost during inference [255], and also makes the implementation more straightforward.”

• cites [255] 255. Bichen Wu, Yanghan Wang, Peizhao Zhang, Yuan- dong Tian, Peter Vajda, and Kurt Keutzer. Mixed precision quantization of convnets via differen- tiable neural architecture search. arXiv preprint arXiv:1812.00090, 2018. sourceCould not verify

  The Related Work section (p. 2) discusses computational savings from quantization generally, and Appendix A (p. 10) describes the paper’s weight-quantization method. Neither addresses symmetric quantization, zero points, or whether removing a zero point reduces inference cost or simplifies implementation. Searches for those terms found no relevant passage in our copy. (LLM, conf=0.75)

  Experimental — accuracy not yet measured

May combine two claims.

Unstructured pruning produces sparse matrix operations that are difficult to accelerate and are typically memory-bound.

“However, this approach leads to sparse matrix operations, which are known to be hard to accelerate, and which are typically memory-bound [21,66].”

• cites [21] 21. Aydin Buluc and John R Gilbert. Challenges and advances in parallel sparse matrix-matrix multipli- cation. In 2008 37th International Conference on Parallel Processing, pages 503-510. IEEE, 2008. sourceNot checked — no full text of the source
• cites [66] 66. Abram L Friesen and Pedro Domingos. Deep learn- ing as a mixed convex-combinatorial optimization problem. arXiv preprint arXiv:1710.11573, 2017. sourceCould not verify

  The Introduction (p. 1) discusses hard-threshold activations and quantization, while Section 2 (p. 2) defines network weight matrices. Neither addresses unstructured pruning or whether sparse matrix operations are hard to accelerate or memory-bound; searches for those terms found no supporting passage in our copy. (LLM, conf=0.75)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

Maintaining state-of-the-art performance while training and inferring with high levels of pruning or sparsity remains an unresolved problem.

“Training and inference with high levels of pruning/sparsity, while maintaining state-of-the-art performance, has remained an open problem [16].”

• cites [16] 16. Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle, and John Guttag. What is the state of neural network pruning? arXiv preprint arXiv:2003.03033, 2020. sourcePartly supported

  Section 2.4 (p. 2) says pruning typically trades model quality for efficiency, and the Conclusion (p. 10) says inconsistent experiments make it impossible to assess the state of the art across pruning techniques. The paper does not establish that maintaining state-of-the-art performance during both training and inference at high sparsity is itself an unresolved problem; Section 3.2 (p. 3) reports that some pruned models retain or exceed their original accuracy. (LLM, conf=0.83)

  Experimental — accuracy not yet measured

May combine two claims.

Layerwise quantization uses the same clipping range for all convolutional filters in a layer, is simple to implement, but often gives sub-optimal accuracy because filter ranges can vary greatly.

“In this approach, the clipping range is determined by considering all of the weights in convolutional filters of a layer [133], as shown in the third column of Figure 3.”

• Also stated in: § E. Quantization Granularity · plain

  “While this approach is very simple to implement, it often results in sub-optimal accuracy, as the range of each convolutional filter can be vary a lot.”

• cites [133] 133. Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. I-bert: Integer-only bert quantization. arXiv preprint arXiv:2101.01321, 2021. sourcePartly supported

  Appendix A.2 (p. 14) says the scaling factor requires determining the range of parameters or activations, which supports the general link between weight ranges and quantization. It does not say that one clipping range is determined from all convolutional filters in a layer. Nor does it address the claimed simplicity or accuracy drawback of layerwise quantization. (LLM, conf=0.78)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

A common rule-based non-uniform quantization uses a logarithmic distribution, with steps and levels increasing exponentially rather than linearly.

“A typical rule-based non-uniform quantization is to use a logarithmic distribution [179,283], where the quantization steps and levels increase exponentially instead of linearly.”

• cites [179] 179. Asit Mishra, Eriko Nurvitadhi, Jeffrey J Cook, and Debbie Marr. Wrpn: Wide reduced-precision networks. arXiv preprint arXiv:1709.01134, 2017. sourcePartly supported

  Section 6 (Related work; page unavailable) describes a prior method using logarithmic quantization with base √2, which supports the logarithmic aspect of the claim. It does not characterize that method as a common rule-based approach or explicitly discuss how its steps and levels increase. Section 5 instead describes WRPN’s own clipping-and-rounding scheme. (LLM, conf=0.79)

  Experimental — accuracy not yet measured
• cites [283] 283. Sijie Zhao, Tao Yue, and Xuemei Hu. Distribution- aware adaptive multi-bit quantization. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9281-9290, 2021. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable).

Achieving efficient real-time neural networks with optimal accuracy requires rethinking how models are designed, trained, and deployed.

“Achieving efficient, real-time NNs with optimal accuracy requires rethinking the design, training, and deployment of NN models [71].”

• cites [71] 71. Amir Gholami, Kiseok Kwon, Bichen Wu, Zizheng Tai, Xiangyu Yue, Peter Jin, Sicheng Zhao, and Kurt Keutzer. SqueezeNext: Hardware-aware neural network design. Workshop paper in CVPR, 2018. sourcePartly supported

  The Introduction and Conclusions support rethinking network design around hardware constraints to improve inference speed and energy efficiency without sacrificing accuracy. The paper discusses training procedures and barriers to deployment, but does not say that rethinking training and deployment is required; those parts of the attributed claim go beyond its findings. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

May combine two claims.

Level 2 ZSQ+QAT usually gives higher accuracy because finetuning helps recover degradation, especially at ultra-low bit precision.

“However, Level 2 usually results in higher accuracy, as finetuning helps the quantized model to recover the accuracy degradation, particularly in ultra-low bit precision settings [85].”

• cites [85] 85. Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149, 2015. sourcePartly supported

  Section 3 (p. 3) describes fine-tuning shared weights after quantization and reports no accuracy loss for pruned AlexNet at 8-bit convolutional and 5-bit fully connected precision. But the paper does not compare this method with ZSQ without fine-tuning or establish that fine-tuning usually yields higher accuracy, particularly at ultra-low precision; Table 6 (p. 10) reports accuracy loss at 4/2 bits. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

May combine two claims.

Integer-only quantization uses low-precision integer arithmetic for all operations, allowing inference without floating-point dequantization.

“However, in integer-only quantization, all the operations are performed using low-precision integer arithmetic [113,132,154,193,267], as illustrated in Figure 6 (Right).This permits the entire inference to be carried out with efficient integer arithmetic, without any floating point dequantization of any parameters or activations.”

• cites [113] 113. Sergey Ioffe and Christian Szegedy. Batch nor- malization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pages 448-456. PMLR, 2015. sourceCould not verify

  Section 3.1 (p. 4) discusses using fixed batch-normalization statistics during inference and combining normalization with scaling and shifting. Searches for integer-only quantization, low-precision integer arithmetic, and floating-point dequantization found no passage addressing those claims in our copy. (LLM, conf=0.76)

  Experimental — accuracy not yet measured
• cites [132] 132. Minje Kim and Paris Smaragdis. Bitwise neural networks. arXiv preprint arXiv:1601.06071, 2016. sourcePartly supported

  The introduction (p. 2) and Section 2 (p. 2) describe inference using binary inputs and parameters, with XNOR and bit counting rather than floating- or fixed-point arithmetic. This supports inference without floating-point dequantization, but the paper distinguishes its bitwise approach from quantized networks that perform low-precision integer arithmetic; it does not say that all operations use such arithmetic. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites [154] 154. Zhenyu Liao, Romain Couillet, and Michael W Mahoney. Sparse quantized spectral clustering. International Conference on Learning Representa- tions, 2021. sourceCould not verify

  The Introduction (p. 2) describes quantizing a matrix for spectral clustering and reducing its storage cost. The system model (p. 3) gives bits per matrix entry, but neither passage addresses integer-only inference. Searches for low-precision integer arithmetic and floating-point dequantization found no passage supporting the claimed execution method. (LLM, conf=0.76)

  Experimental — accuracy not yet measured
• cites [193] 193. Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho. Relational knowledge distillation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3967-3976, 2019.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [267] 267. Zhaohui Yang, Yunhe Wang, Kai Han, Chun- jing Xu, Chao Xu, Dacheng Tao, and Chang Xu. Searching for low-bit weights in quantized neural networks. Advances in neural information processing systems, 2020. sourcePartly supported

  Related Works (p. 2) says binary networks use low-level bit operations to accelerate inference, and Optimization Details (p. 4) says the proposed method uses quantized weights for inference. Neither passage establishes that all inference operations use low-precision integer arithmetic or that no floating-point dequantization of parameters or activations occurs. (LLM, conf=0.82)

  Experimental — accuracy not yet measured

May combine two claims.

INT8 addition is 30 times more energy efficient and 116 times more area efficient than FP32 addition.

“For example, performing INT8 addition is 30× more energy efficient and 116× more area efficient as compared to FP32 addition [97].”

• cites [97] 97. Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, and Alexandra Peste. Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks. arXiv preprint arXiv:2102.00554, 2021. sourceCould not verify

  Section 7.1 (p. 59) discusses quantization and integer storage formats, while Section 7.2 (pp. 61–62) discusses accelerator energy consumption. Neither gives an INT8-versus-FP32 addition comparison or the claimed 30× energy and 116× area figures. Searches for those figures and both data types found no passage addressing the claim. (LLM, conf=0.72)

  Experimental — accuracy not yet measured

May combine two claims.

HAWQv2 extends this to mixed-precision activation quantization and is over 100 times faster than RL-based mixed-precision methods.

“In HAWQv2, this method was extended to mixedprecision activation quantization [50], and was shown to be more than 100x faster than RL based mixed-precision methods [246].”

• cites [50] 50. Xin Dong, Shangyu Chen, and Sinno Jialin Pan. Learning to prune deep neural networks via layer-wise optimal brain surgeon. arXiv preprint arXiv:1705.07565, 2017. sourceCould not verify

  The cited paper’s Results (p. 8) discuss the speed and computational cost of L-OBS pruning, not HAWQv2. Searches for HAWQv2, mixed-precision activation quantization, and a 100× speed comparison with RL-based methods found no passage addressing the claim. (LLM, conf=0.77)

  Experimental — accuracy not yet measured
• cites [246] 246. Dilin Wang, Meng Li, Chengyue Gong, and Vikas Chandra. Attentivenas: Improving neural architec- ture search via attentive sampling. arXiv preprint arXiv:2011.09011, 2020. sourceCould not verify

  The citation to this paper is attached to the claim that HAWQv2 is more than 100 times faster than RL-based mixed-precision methods. Our copy's Additional information section compares AttentiveNAS training costs with RL-based neural architecture search, but does not address HAWQv2 or mixed-precision quantization. Searches for HAWQv2, activation quantization, and the claimed speed comparison found no supporting passage; page numbers were not available. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

The third branch of solutions aims for better training methods for binary and ternary models.

“c) Improved Training Method: Another interesting branch of work aims for better training methods for binary/ternary models [5,20,44,73,160,164,285,288].”

• cites [5] 5. Milad Alizadeh, Javier Fernández-Marqués, Nicholas D Lane, and Yarin Gal. An empirical study of binary neural networks' optimisation. In International Conference on Learning Representations, 2018.Not checked — no full text of the source
• cites [20] 20. Adrian Bulat, Georgios Tzimiropoulos, Jean Kos- saifi, and Maja Pantic. Improved training of binary networks for human pose estimation and image recognition. arXiv preprint arXiv:1904.05868, 2019. sourceSupported
• cites [44] 44. Elliot J Crowley, Gavin Gray, and Amos J Storkey. Moonshine: Distilling with cheap convolutions. In NeurIPS, pages 2893-2903, 2018. sourcePartly supported

  Section 3.1 (p. 3) describes distillation methods for training smaller student networks, and Section 4.2 (p. 7) reports that attention transfer outperformed other training approaches. The paper does not establish that these methods train binary or ternary models; its reported models instead use cheap convolutional blocks. (LLM, conf=0.82)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

Bi-Real Net introduces identity shortcuts connecting activations to activations in consecutive blocks, allowing 32-bit activations to propagate.

“Bi-Real Net [164] introduces identity shortcuts connecting activations to activations in consecutive blocks, through which 32-bit activations can be propagated.”

• cites [164] 164. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019. sourceCould not verify

  The Architecture section (p. 2) describes BERT transformer blocks, and the RoBERTa section (p. 6) describes changes to BERT pretraining. Searches for Bi-Real Net, identity shortcuts, and 32-bit activations found no passage in our copy addressing the cited claim. (LLM, conf=0.78)

  Experimental — accuracy not yet measured

May combine two claims.

Work [267] deploys quantized operations directly in hardware and measures actual per-layer deployment latency for different bit precisions.

“To address this the recent work of [267] directly deploys quantized operations in hardware, and measures the actual deployment latency of each layer for different quantization bit precisions.”

• cites [267] 267. Zhaohui Yang, Yunhe Wang, Kai Han, Chun- jing Xu, Chao Xu, Dacheng Tao, and Chang Xu. Searching for low-bit weights in quantized neural networks. Advances in neural information processing systems, 2020. sourceCould not verify

  The Experiments section (p. 6) tests quantized networks at different bitwidths, while the Introduction (p. 1) discusses hardware deployment only as a general possibility. Neither passage establishes that this work deployed operations on hardware or measured actual per-layer latency; searches for latency and hardware deployment found no such measurement in our copy. (LLM, conf=0.74)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

Prior knowledge distillation methods have used logits or intermediate layers as sources of knowledge.

“Previous methods of knowledge distillation focus on exploring different knowledge sources.[95,150,192] use logits (the soft probabilities) as the source of knowledge, while [3,207,269] try to leverage the knowledge from intermediate layers.”

• cites [95] 95. Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (GELUs). arXiv preprint arXiv:1606.08415, 2016. sourceCould not verify

  The Introduction (p. 1) describes GELU as an activation function, and the MNIST Classification section (p. 2) compares it with other activation functions. Searches for knowledge distillation, logits, intermediate layers, and teacher–student knowledge transfer found no passage addressing the attributed claim. (LLM, conf=0.77)

  Experimental — accuracy not yet measured
• cites [150] 150. Yuhang Li, Ruihao Gong, Fengwei Yu, Xin Dong, and Xianglong Liu. Dms: Differentiable dimension search for binary neural networks. International Conference on Learning Representations, 2020.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [192] 192. Sejun Park, Jaeho Lee, Sangwoo Mo, and Jin- woo Shin. Lookahead: a far-sighted alternative of magnitude-based pruning. arXiv preprint arXiv:2002.04809, 2020. sourceCould not verify

  The Introduction (p. 1) and Section 2 (p. 2) discuss neural-network pruning, not knowledge distillation. Searches for logits, intermediate layers as knowledge sources, and teacher–student distillation found no passage in our copy addressing the claim. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

The second branch of solutions focuses on the choice of loss function.

“b) Improved Loss function: Another branch of works focuses on the choice of loss function [48,98,99,251,284].”

• cites [48] 48. James Diffenderfer and Bhavya Kailkhura. Multi- prize lottery ticket hypothesis: Finding accurate binary neural networks by pruning a randomly weighted network. In International Conference on Learning Representations, 2021. sourceCould not verify

  Section 2.2 (pp. 4–5) uses training loss to update pruning scores and says the experiments use cross-entropy loss. The Discussion (p. 9) describes the contribution as finding accurate binary subnetworks by pruning randomly weighted networks. I found no passage presenting the choice or improvement of the loss function as this paper’s focus (searched: choice of loss function, improved loss function, loss-aware binarization). (LLM, conf=0.76)

  Experimental — accuracy not yet measured
• cites [98] 98. Mark Horowitz. 1.1 computing's energy problem (and what we can do about it). In 2014 IEEE In- ternational Solid-State Circuits Conference Digest of Technical Papers (ISSCC), pages 10-14. IEEE, 2014.Not checked yet

  Not checked in the free run.

• cites [99] 99. Lu Hou and James T Kwok. Loss-aware weight quantization of deep networks. arXiv preprint arXiv:1802.08635, 2018. sourcePartly supported

  The Introduction (p. 2) and Conclusion (p. 9) say the method explicitly considers how weight quantization affects the loss. They do not describe choosing or improving the loss function itself; the work focuses on the quantization algorithm. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

BNN+ introduces a continuous approximation for the derivative of the sign function.

“To address this, BNN+ [44] introduces a continuous approximation for the derivative of the sign function, while [198,261,272] replace the sign function with smooth, differentiable functions that gradually sharpens and approaches the sign function.”

• cites [44] 44. Elliot J Crowley, Gavin Gray, and Amos J Storkey. Moonshine: Distilling with cheap convolutions. In NeurIPS, pages 2893-2903, 2018. sourceCould not verify

  Section 3.2 (p. 4) describes bottleneck and grouped convolutions, not BNN+ or a derivative of the sign function. A search of our copy for BNN+, sign-function derivatives, and surrogate gradients found no passage addressing the attributed claim. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

BinaryConnect constrains weights to either +1 or -1.

“An important work here is BinaryConnect [42] which constrains the weights to either +1 or -1.”

• cites [42] 42. Matthieu Courbariaux, Yoshua Bengio, and Jean- Pierre David. Training deep neural networks with low precision multiplications. arXiv preprint arXiv:1412.7024, 2014. sourceCould not verify

  The Abstract (p. 1) describes this paper's use of floating-point, fixed-point, and dynamic fixed-point formats, including 10-bit multiplications. The Related Works section (p. 8) does not mention BinaryConnect. Searches for BinaryConnect and weights restricted to +1 or -1 found no passage in our copy addressing the claim. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Teacher model choices have been studied, including using multiple teachers and self-distillation without an extra teacher.

“The choices of teacher models are also well studied, where [235,273] use multiple teacher models to jointly supervise the student model, while [43,277] apply self-distillation without an extra teacher model.”

• cites [235] 235. Wei Tang, Gang Hua, and Liang Wang. How to train a compact binary neural network with high accuracy? In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017. sourceCould not verify

  The Related Work section (pp. 2–3) and Training BinaryNets with high accuracy (pp. 4–5) discuss binary-network training and multiple binarizations, not teacher-model choices. Searches for multiple teachers, student supervision, and self-distillation found no passage in our copy addressing the attributed claim. (LLM, conf=0.76)

  Experimental — accuracy not yet measured
• cites [273] 273. Penghang Yin, Shuai Zhang, Jiancheng Lyu, Stan- ley Osher, Yingyong Qi, and Jack Xin. Blended coarse gradient descent for full quantization of deep neural networks. Research in the Mathemati- cal Sciences, 6(1):14, 2019.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [43] 43. Matthieu Courbariaux, Yoshua Bengio, and Jean- Pierre David. BinaryConnect: Training deep neural networks with binary weights during propagations. In Advances in neural information processing systems, pages 3123-3131, 2015.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Exploration-based mixed-precision methods often need large computational resources and are sensitive to hyperparameters and initialization.

“One disadvantage of these exploration-based methods [246,254] is that they often require large computational resources, and their performance is typically sensitive to hyperparameters and even initialization.”

• cites [246] 246. Dilin Wang, Meng Li, Chengyue Gong, and Vikas Chandra. Attentivenas: Improving neural architec- ture search via attentive sampling. arXiv preprint arXiv:2011.09011, 2020. sourcePartly supported

  The Introduction and Related Work describe conventional neural architecture search as computationally expensive, and Related Work says some weight-sharing search methods are sensitive to hyperparameter choices, including random seeds. The paper does not establish these points specifically for mixed-precision methods, and it does not show that the expensive methods are also sensitive to initialization. Page numbers are unavailable in our copy. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [254] 254. Bichen Wu, Alvin Wan, Xiangyu Yue, Peter Jin, Sicheng Zhao, Noah Golmant, Amir Gholaminejad, Joseph Gonzalez, and Kurt Keutzer. Shift: A zero flop, zero parameter alternative to spatial convolu- tions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9127-9135, 2018.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

The first branch of solutions aims to minimize quantization error, the gap between real and quantized values.

“a) Quantization Error Minimization: The first branch of solutions aims to minimize the quantization error, i.e., the gap between the real values and the quantized values [19,34,62,103,151,158,164,169,178,218,248].”

• cites [19] 19. Adrian Bulat and Georgios Tzimiropoulos. Xnor- net++: Improved binary neural networks. arXiv preprint arXiv:1909.13863, 2019. sourcePartly supported

  Section 2.2 (p. 3) describes scaling factors used to reduce quantization error, and the Introduction (p. 1) contrasts reducing approximation error with optimizing task loss. The paper does not identify error minimization as a “first branch” of solutions; its own method emphasizes task-loss optimization instead. (LLM, conf=0.85)

  Experimental — accuracy not yet measured
• cites [34] 34. Xiuyi Chen, Guangcan Liu, Jing Shi, Jiaming Xu, and Bo Xu. Distilled binary neural network for monaural speech separation. In 2018 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE, 2018. sourceNot checked — no full text of the source
• cites [62] 62. Jun Fang, Ali Shafiee, Hamzah Abdel-Aziz, David Thorsley, Georgios Georgiadis, and Joseph H Has- soun. Post-training piecewise linear quantization for deep neural networks. In European Conference on Computer Vision, pages 69-86. Springer, 2020.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Proposed INT8 training methods often require a lot of hyperparameter tuning or only work for a few neural network models on relatively easy learning tasks.

“While several interesting works exist in this area [10,26,123,137,173], the proposed methods often require a lot of hyperparameter tuning, or they only work for a few NN models on relatively easy learning tasks.”

• cites [10] 10. Ron Banner, Itay Hubara, Elad Hoffer, and Daniel Soudry. Scalable methods for 8-bit training of neural networks. Advances in neural information processing systems, 2018. sourcePartly supported

  Section 2 (Previous Work) says that some more aggressively quantized networks lost accuracy on complex models and challenging datasets, supporting the concern about limited applicability. It does not say that 8-bit methods often require extensive hyperparameter tuning. The Abstract and Section 7 (Discussion) instead report this paper’s 8-bit method working on the large-scale ImageNet-1K dataset without harming accuracy. (LLM, conf=0.83)

  Experimental — accuracy not yet measured
• cites [26] 26. Léopold Cambier, Anahita Bhiwandiwalla, Ting Gong, Mehran Nekuii, Oguz H Elibol, and Hanlin Tang. Shifted and squeezed 8-bit floating point format for low-precision training of deep neural networks. arXiv preprint arXiv:2001.05674, 2020. sourcePartly supported

  The Related Work section (p. 2) says integer-based training approaches required custom tuning and lacked generalizability beyond the models discussed, including LeNet-5 and AlexNet. It does not characterize the learning tasks as relatively easy, so that part of the claim is unsupported. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites [123] 123. Qing Jin, Linjie Yang, and Zhenyu Liao. Adabits: Neural network quantization with adaptive bit- widths. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 2146-2156, 2020.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Product quantization divides the weight matrix into submatrices and applies vector quantization to each submatrix.

“Product quantization [74,227,256] is an extension of vector quantization, where the weight matrix is divided into submatrices and vector quantization is applied to each submatrix.”

• cites [74] 74. Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan. Differentiable soft quantization: Bridging full-precision and low-bit neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4852-4861, 2019.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [227] 227. Pierre Stock, Angela Fan, Benjamin Graham, Edouard Grave, Rémi Gribonval, Herve Jegou, and Armand Joulin. Training with quantization noise for extreme model compression. In International Conference on Learning Representations, 2021.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [256] 256. Hao Wu, Patrick Judd, Xiaojie Zhang, Mikhail Isaev, and Paulius Micikevicius. Integer quan- tization for deep learning inference: Princi- ples and empirical evaluation. arXiv preprint arXiv:2004.09602, 2020. sourceCould not verify

  Section 3.2 (pp. 4–5) discusses per-row, per-column, and per-tensor quantization, but does not describe product quantization or applying vector quantization to weight submatrices. Searches for product quantization, vector quantization, submatrices, and codebooks found no passage in our copy that addresses the claimed procedure. (LLM, conf=0.75)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

CMSIS-NN helps quantize and deploy neural network models onto ARM Cortex-M cores using fixed-point quantization with power-of-two scaling factors.

“CMSIS-NN [136] is a library from ARM that helps quantizing and deploying NN models onto the ARM Cortex-M cores.Specifically, the library leverages fixed-point quantization [113,154,267] with power-oftwo scaling factors so that quantization and dequantization processes can be carried out efficiently with bit shifting operations.”

• cites [136] 136. Se Jung Kwon, Dongsoo Lee, Byeongwook Kim, Parichay Kapoor, Baeseong Park, and Gu-Yeon Wei. Structured compression by weight encryption for unstructured pruning and quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1909-1918, 2020.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [113] 113. Sergey Ioffe and Christian Szegedy. Batch nor- malization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pages 448-456. PMLR, 2015. sourceCould not verify

  The cited paper’s Section 3.1 (p. 4) discusses batch normalization during inference, including a linear transform, but not fixed-point quantization. Searches for CMSIS-NN, ARM Cortex-M, power-of-two scaling, and bit shifting found no passage addressing the attributed claim. (LLM, conf=0.76)

  Experimental — accuracy not yet measured
• cites [154] 154. Zhenyu Liao, Romain Couillet, and Michael W Mahoney. Sparse quantized spectral clustering. International Conference on Learning Representa- tions, 2021. sourceCould not verify

  The Introduction (p. 2) and Section 4.3 (pp. 9–10) discuss quantizing matrices for spectral clustering, not neural-network deployment. Searches for CMSIS-NN, ARM Cortex-M, fixed-point quantization, and power-of-two scaling found no passage addressing the attributed claim. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

GAP-8 is a RISC-V system on chip for edge inference with a dedicated CNN accelerator that only supports integer arithmetic.

“GAP-8 [64], a RISC-V SoC (System on Chip) for edge inference with a dedicated CNN accelerator, is another example of an edge processor that only supports integer arithmetic.”

• cites [64] 64. Alexander Finkelstein, Uri Almog, and Mark Grobman. Fighting quantization bias with bias. arXiv preprint arXiv:1906.03193, 2019. sourceCould not verify

  The Introduction (p. 1) says that 8-bit integer quantization is used across cloud and edge devices, but it does not identify GAP-8. Searches for GAP-8, GAP8, RISC-V, and a dedicated CNN accelerator found no passage establishing GAP-8’s design or that it only supports integer arithmetic. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

Support for INT4 quantization was only recently added to TVM.

“Moreover, support for INT4 quantization was only recently added to TVM [267].”

• cites [267] 267. Zhaohui Yang, Yunhe Wang, Kai Han, Chun- jing Xu, Chao Xu, Dacheng Tao, and Chang Xu. Searching for low-bit weights in quantized neural networks. Advances in neural information processing systems, 2020. sourceCould not verify

  The Abstract (p. 1) discusses 4-bit quantization, and Related Works (p. 2) mentions platform support for network quantization generally. Neither identifies TVM or says when INT4 support was added. Searches for “TVM,” “INT4,” and recent platform support found no passage addressing the claim. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

May combine two claims.

INT8 inference of ResNet50, VGG-19, and InceptionV3 using TVM can achieve 3.89×, 3.32×, and 5.02× speedup on NVIDIA GTX 1080, respectively.

“[157] shows that INT8 inference of popular computer vision models, including ResNet50 [88], VGG-19 [224], and inceptionV3 [230] using TVM [32] quantization library, can achieve 3.89×, 3.32×, and 5.02× speedup on NVIDIA GTX 1080, respectively.”

• cites [157] 157. Shaohui Lin, Rongrong Ji, Yuchao Li, Yongjian Wu, Feiyue Huang, and Baochang Zhang. Acceler- ating convolutional networks via global & dynamic filter pruning. In IJCAI, pages 2425-2432, 2018. sourceCould not verify

  The Experimental Settings (section 4.1) describe inference measurements on a GTX TITAN X and use a GTX 1080 Ti for training. The Quantitative Evaluation (section 4.2) reports filter-pruning results for ResNet-50 and VGG-16, but our copy does not report INT8 inference using TVM, VGG-19 or InceptionV3, or the three claimed speedups (searched: INT8, TVM, quantization, and the model names). (LLM, conf=0.76)

  Experimental — accuracy not yet measured

May combine two claims.

Sub-channelwise quantization can cause significant overhead and is not the current standard.

“In contrast, sub-channelwise quantization may result in significant overhead and is not currently the standard choice”

• cites [68] 68. AE Gamal, L Hemachandra, Itzhak Shperling, and V Wei. Using simulated annealing to design good codes. IEEE Transactions on Information Theory, 33(1):116-123, 1987. sourceNot checked — no full text of the source

May combine two claims.

Because minimizing the error in this binary-code approximation has no closed-form solution, earlier work used heuristics.

“Since there is no closed-form solution for minimizing the error between r and m i=1 α i b i , previous research relies on heuristic solutions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Symmetric quantization simplifies the quantization function by setting the zero point to zero.

“Using symmetric quantization, however, simplifies the quantization function in Eq. 2 by replacing the zero point with Z = 0:”

May combine two claims.

Groupwise quantization groups multiple channels within a layer to compute the clipping range, can help when parameter distributions vary across a convolution or activation, but incurs extra cost from handling different scaling factors.

“One could group multiple different channels inside a layer to calculate the clipping range (of either activations or convolution kernels).”

• Also stated in: § E. Quantization Granularity · hedged

  “This could be helpful for cases where the distribution of the parameters across a single convolution/activation varies a lot.”

• Also stated in: § E. Quantization Granularity · plain

  “However, this approach inevitably comes with the extra cost of accounting for different scaling factors.”

Classified as probably not a substantial claim.

Different convolutional filters may cover different value ranges.

“Each of these convolutional filters can have a different range of values.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Channelwise quantization lets a practitioner set the clipping range for each kernel with negligible overhead.

“It enables the practitioner to adjust the clipping range for each individual kernel with negligible overhead.”

May combine two claims.

Different layers in a neural network have different impacts on the loss function, which motivates mixed-precision quantization.

“Different layers in a Neural Net have different impact on the loss function, and this motivates a mixed-precision approach to quantization.”

Classified as probably not a substantial claim.

A neural network might explore more under stochastic quantization than under deterministic quantization.

“The high level intuition has been that the stochastic quantization may allow a NN to explore more, as compared to deterministic quantization.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Non-uniform quantization generally captures signal information better by assigning bits and discretizing parameter ranges non-uniformly.

“Generally, non-uniform quantization enables us to better capture the signal information, by assigning bits and discreitizing the range of parameters non-uniformly.”

May combine two claims.

ZeroQ shows synthetic data can be used for sensitivity measurement and calibration, enabling mixed-precision PTQ without training or validation data.

“Furthermore, ZeroQ [24] shows that the synthetic data can be used for sensitivity measurement as well as calibration, thereby enabling mixed-precision post-training quantization without any access to the training/validation data.”

• cites [24] 24. Yaohui Cai, Zhewei Yao, Zhen Dong, Amir Gho- lami, Michael W Mahoney, and Kurt Keutzer. Zeroq: A novel zero shot quantization framework. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13169-13178, 2020.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Uniform quantization is currently the de-facto method because it is simple and maps efficiently to hardware.

“As such, the uniform quantization is currently the de-facto method due to its simplicity and its efficient mapping to hardware.”

Classified as probably not a substantial claim.

Zero-shot quantization performs the whole quantization process without accessing training or validation data.

“Zero Shot (aka data free) quantization performs the entire quantization without any access to the training/validation data.”

May combine two claims.

Unlike QAT, which needs sufficient training data, PTQ can be used when data is limited or unlabeled.

“Unlike QAT, which requires a sufficient amount of training data for retraining, PTQ has an additional advantage that it can be applied in situations where data is limited or unlabeled.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

AdaRound shows that naive round-to-nearest can be sub-optimal and proposes adaptive rounding that reduces loss.

“Another notable work is AdaRound [181] which shows that the naive round-to-nearest method for quantization can counter-intuitively results in sub-optimal solutions, and it proposes an adaptive rounding method that better reduces the loss.”

• cites [181] 181. Lopamudra Mukherjee, Sathya N Ravi, Jiming Peng, and Vikas Singh. A biresolution spectral framework for product quantization. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3329-3338, 2018. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S4 (based).

Non-uniform quantization schemes are usually difficult to deploy efficiently on general hardware such as GPUs and CPUs.

“However, non-uniform quantization schemes are typically difficult to deploy efficiently on general computation hardware, e.g., GPU and CPU.”

May combine two claims.

Low-precision logic improves latency, power consumption, and area efficiency over full-precision logic.

“Low-precision logic has multiple benefits over the full-precision counterpart in terms of latency, power consumption, and area efficiency.”

May combine two claims.

Stochastic quantization methods suffer from the overhead of generating random numbers for every weight update and are not widely used in practice.

“However, a major challenge with stochastic quantization methods is the overhead of creating random numbers for every single weight update, and as such they are not yet adopted widely in practice.”

May combine two claims.

Non-STE methods often need substantial tuning, and STE remains the most commonly used approach.

“Despite interesting works in this area, these methods often require a lot of tuning and so far STE approach is the most commonly used method.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Work [197] categorizes solutions for reducing accuracy degradation in extreme quantization into three branches.

“The work of [197] broadly categorizes these solutions into three branches.”

• cites [197] 197. Haotong Qin, Zhongang Cai, Mingyuan Zhang, Yifu Ding, Haiyu Zhao, Shuai Yi, Xianglong Liu, and Hao Su. Bipointnet: Binary neural network for point clouds. International Conference on Learning Representations, 2021.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

On NVIDIA V100 GPUs, peak binary arithmetic throughput is eight times higher than INT8.

“For instance, the peak binary arithmetic on NVIDIA V100 GPUs is 8x higher than INT8.”

May combine two claims.

Binarization constrains quantized values to one bit and reduces memory requirements by a factor of 32.

“Binarization, where the quantized values are constrained to a 1-bit representation, thereby drastically reducing the memory requirement by 32×, is the most extreme quantization method.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Binarized Neural Network extends BinaryConnect by binarizing activations as well as weights.

“Binarized NN [107] (BNN) extends this idea by binarizing the activations as well as the weights.”

• cites [107] 107. Zehao Huang and Naiyan Wang. Data-driven sparse structure selection for deep neural networks. In Proceedings of the European conference on computer vision (ECCV), pages 304-320, 2018.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Ternarization reduces inference latency by eliminating costly matrix multiplications, as binarization does.

“Ternarization also drastically reduces the inference latency by eliminating the costly matrix multiplications as binarization does.”

May combine two claims.

Binary and ternary operations can often be computed efficiently with bit-wise arithmetic and can achieve significant acceleration over higher precisions such as FP32 and INT8.

“Besides the memory advantages, binary (1-bit) and ternary (2-bit) operations can often be computed efficiently with bit-wise arithmetic and can achieve significant acceleration over higher precisions, such as FP32 and INT8.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Quantization combined with efficient low-precision logic and dedicated deep learning accelerators has been an important driving force for the evolution of edge processors.

“Quantization, combined with efficient lowprecision logic and dedicated deep learning accelerators, has been one important driving force for the evolution of such edge processors.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The problem of quantization in neural network models has been a highly active research area.

“As a result, as our bibliography attests, the problem of quantization in NN models has been a highly active research area.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Nvidia's TensorRT does not currently support sub-INT8 quantization.

“For instance, Nvidia's TensorRT does not currently support sub-INT8 quantization.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

There is currently very little work exploring the optimal combinations of quantization with other compression methods.

“However, there is currently very little work exploring what are the optimal combinations of these methods.”

Probably not a substantial claim: may not pass S4 (based).

Many edge processors do not support floating point operations, especially micro-controllers.

“In addition, many edge processors do not have any support floating point operations, especially in micro-controllers.”

Probably not a substantial claim: may not pass S4 (based).

Edge devices often have tight resource constraints including compute, memory, and power budget.

“Edge devices often have tight resource constraints including compute, memory, and importantly power budget.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Google Edge TPU is a purpose-built ASIC for edge inference that only supports 8-bit arithmetic.

“Unlike Cloud TPUs that run in Google data centers with a large amount of computing resources, the Edge TPU is designed for small and low-power devices, and thereby it only supports 8-bit arithmetic.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Quantization can be coupled with other efficiency methods such as efficient architecture design, hardware and architecture co-design, pruning, and knowledge distillation.

“As discussed above, quantization is only one of the methods for efficient deployment of NNs.Other methods include efficient NN architecture design, co-design of hardware and NN architecture, pruning, and knowledge distillation.Quantization can be coupled with these other approaches.”

Probably not a substantial claim: may not pass S3 (load-bearing).

With quantization-aware training, a model may converge to a different solution far from the original single-precision solution but still achieve good accuracy.

“For example, with quantization-aware training, we might converge to a different solution, far away from the original solution with single precision parameters, but still get good accuracy.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Most current neural network models are heavily over-parameterized, so there is ample opportunity to reduce bit precision without impacting accuracy.

“Second, most current NN models are heavily over-parameterized.So, there is ample opportunity for reducing the bit precision without impacting accuracy.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In machine learning quantization, the goal is not to compress a signal with minimum change or error, but to find a reduced-precision representation that yields as small a loss as possible.

“However, the main difference between quantization methods for machine learning is that fundamentally we are not interested to compress the signal with minimum change/error as compared to the original signal.Instead, the goal is to find a reduced-precision representation that results in as small loss as possible.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Applying more clusters to more important quantization ranges better preserves information.

“And in [189], the authors apply more clusters for more important quantization ranges to better preserve the information.”

• cites [189] 189. BM Oliver, JR Pierce, and Claude E Shannon. The philosophy of pcm. Proceedings of the IRE, 36(11):1324-1331, 1948. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

With current methods, it is straightforward to quantize and deploy different neural network models to INT8 without losing accuracy.

“With current methods, it is straightforward to quantize and deploy different NN models to INT8, without losing accuracy.”

May combine two claims.

Software for lower bit-precision quantization is not widely available and is sometimes non-existent.

“However, the software for lower bit-precision quantization is not widely available, and sometimes it is non-existent.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Changing the width of the neural network architecture could reduce or remove the generalization gap after quantization.

“For instance, the recent work of [34] shows that changing the width of the NN architecture could reduce/remove generalization gap after quantization.”

• cites [34] 34. Xiuyi Chen, Guangcan Liu, Jing Shi, Jiaming Xu, and Bo Xu. Distilled binary neural network for monaural speech separation. In 2018 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE, 2018. sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Neural network models must be quantized using either quantization-aware training or post-training quantization of TensorFlow for the Edge TPU.

“NN models must be quantized using either quantization-aware training or posttraining quantization of TensorFlow.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

NVIDIA Turing GPUs, particularly T4 GPUs, include Turing Tensor Cores, which are specialized execution units for efficient low-precision matrix multiplications.

“A good example is provided by the recent NVIDIA Turing GPUs, and in particular T4 GPUs, which include the Turing Tensor Cores.Tensor Cores are specialized execution units designed for efficient lowprecision matrix multiplications.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Residuals after k-means product quantization can be recursively quantized.

“For example, in [74] the residuals after k-means product quantization are further recursively quantized.”

• cites [74] 74. Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan. Differentiable soft quantization: Bridging full-precision and low-bit neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4852-4861, 2019.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Different layers in a neural network have different impact on the loss function, which motivates mixed-precision quantization.

“Thus, different layers in the NN have different impact on the loss function, and this motivates interesting approaches such mixed-precision quantization.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

The accuracy degradation from extreme quantization may be acceptable for less critical applications.

“But this accuracy degradation may be acceptable for less critical applications.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Some ARM Cortex-M cores lack dedicated floating-point units, so models should be quantized before deployment.

“Because some of the ARM Cortex-M cores do not include dedicated floatingpoint units, the models should first be quantized before deployment.”

Classified as probably not a substantial claim. May combine two claims.

Quantization reduces model size, enables faster speed, and requires less power, especially for hardware with low-precision logic.

“We have said that quantization not only reduces the model size, but it also enables faster speed and requires less power, in particular for hardware that has lowprecision logic.”