FLightNNs: Lightweight Quantized Deep Neural Networks for Fast and Accurate Inference
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
To improve the throughput and energy efficiency of Deep Neural Networks (DNNs) on customized hardware, lightweight neural networks constrain the weights of DNNs to be a limited combination (denoted as k ϵ {1, 2}) of powers of 2. In such networks, the multiply-accumulate operation can be replaced with a single shift operation, or two shifts and an add operation. To provide even more design flexibility, the k for each convolutional filter can be optimally chosen instead of being fixed for every filter. In this paper, we formulate the selection of k to be differentiable, and describe model training for determining k-based weights on a per-filter basis. Over 46 FPGA-design experiments involving eight configurations and four data sets reveal that lightweight neural networks with a flexible k value (dubbed FLightNNs) fully utilize the hardware resources on Field Programmable Gate Arrays (FPGAs), our experimental results show that FLightNNs can achieve 2× speedup when compared to lightweight NNs with k = 2, with only 0.1% accuracy degradation. Compared to a 4-bit fixed-point quantization, FLightNNs achieve higher accuracy and up to 2× inference speedup, due to their lightweight shift operations. In addition, our experiments also demonstrate that FLightNNs can achieve higher computational energy efficiency for ASIC implementation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“We use the Adam optimizer [22] to train the network.”
MethodMobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755 · cited 1×
“Some work proposes more lightweight DNN units for faster inference on general-purpose hardware [22], while others propose hardwarefriendly DNN computation units to enable energy-efficient implementation on customized hardware [25].”
BackgroundDARTS: Differentiable architecture search
Liu, H., Simonyan, K., & Yang, Y. (2018) · arXiv · doi:10.48550/arxiv.1806.09055 · Cited by 3,248
Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks
Chen, Y.-H., Krishna, T., Emer, J. S., & Sze, V. (2017) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2016.2616357 · Cited by 3,247
MobileNetV2: Inverted Residuals and Linear BottlenecksPreprint
Sandler, M., Howard, A., Zhu, M., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.04381 · Cited by 2,255 · cited 1×
“Some work proposes more lightweight DNN units for faster inference on general-purpose hardware [22], while others propose hardwarefriendly DNN computation units to enable energy-efficient implementation on customized hardware [25].”
BackgroundBinaryConnect: Training Deep Neural Networks with binary weights during propagations
Courbariaux, M., Bengio, Y., & David, J.-P. (2015) · arXiv · doi:10.48550/arxiv.1511.00363 · Cited by 2,180 · cited 2×
“propose BinaryConnect, which uses only 1 bit for the DNN parameters, turning multiplications into XNOR operations on customized hardware [6].”
MethodOptimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., & Cong, J. (2015) · Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/2684746.2689060 · Cited by 2,158
Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
Bengio, Y., Léonard, N., & Courville, A. (2013) · arXiv · doi:10.48550/arxiv.1308.3432 · Cited by 1,910
DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
Zhou, S., Wu, Y., Ni, Z., Zhou, X., Wen, H., & Zou, Y. (2016) · arXiv · doi:10.48550/arxiv.1606.06160 · Cited by 1,690 · cited 1×
“Similar to prior work on DNN quantization [15, 16], we use the straight-through estimator (STE) [19] to compute ∂ L .”
MethodHubara, I., Soudry, D., & Yaniv, R. E. (2016) · arXiv · doi:10.48550/arxiv.1602.02505 · Cited by 1,667 · cited 1×
“2 RELATEDWORK Prior work has extensively explored approaches to reduce latency and energy consumption of DNNs on hardware, through both algorithmic [2, 6] and hardware [7, 8] efforts.”
BackgroundDeep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · International Conference on Machine Learning · Cited by 1,590
Deep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · arXiv · doi:10.48550/arxiv.1502.02551 · Cited by 1,364 · cited 2×
“show that a DNN with 16-bit fixed-point representation can achieve competitive accuracy compared to the full-precision network [11].”
BackgroundMethodThe influence of the sigmoid function parameters on the speed of backpropagation learning
Han, J., & Moraga, C. (1995) · Lecture Notes in Computer Science · doi:10.1007/3-540-59497-3_175 · Cited by 968 · cited 1×
“, ∂w q i ∂tj , we relax the indicator function д(x , tj ) = 1(x > tj ) to a sigmoid function [12], σ (.”
BackgroundDesigning Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning
Yang, T.-J., Chen, Y.-H., & Sze, V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.643 · Cited by 732
Fixed Point Quantization of Deep Convolutional Networks
Lin, D. D., Talathi, S. S., & Annapureddy, V. S. (2015) · arXiv · doi:10.48550/arxiv.1511.06393 · Cited by 684 · cited 1×
“Therefore, prior research has proposed model compression techniques including pruning and quantization to satisfy the stringent energy and speed requirements [3].”
MethodLearning Sparse Neural Networks through $L_0$ Regularization
Louizos, C., Welling, M., & Kingma, D. P. (2017) · arXiv · doi:10.48550/arxiv.1712.01312 · Cited by 628 · cited 1×
“Recent work has explored the idea of differentiable training for architecture search [17] and neural network pruning [18].”
BackgroundLearning Structured Sparsity in Deep Neural Networks
Wen, W. X., Wu, C., Wang, Y., Chen, Y., & Li, H. H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.03665 · Cited by 402
Processing-in-Memory for Energy-Efficient Neural Network Training: A Heterogeneous Approach
Liu, J., Zhao, H., Ogleari, M., Li, D., & Zhao, J. (2018) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2018.00059 · Cited by 157 · cited 1×
“Prior work has extensively explored approaches to reduce latency and energy consumption of DNNs on hardware, through both algorithmic [14, 28] and hardware [4, 30] efforts.”
Regularizing Activation Distribution for Training Binarized Deep Networks
Ding, R., Chin, T.-W., Liu, Z., & Marculescu, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01167 · Cited by 151 · cited 1×
“Future work will further improve training efficiency by using optimized training loss [7] or proper labels [5].”
BackgroundSelf-citationHyperPower: Power- and memory-constrained hyper-parameter optimization for neural networks
Stamoulis, D., Cai, E., Juan, D.-C., & Marculescu, D. (2018) · 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE) · doi:10.23919/date.2018.8341973 · Cited by 67
Self-citationHardware-Software Codesign of Accurate, Multiplier-free Deep Neural Networks
Tann, H., Hashemi, S., Bahar, R. I., & Reda, S. (2017) · Proceedings of the 54th Annual Design Automation Conference 2017 · doi:10.1145/3061639.3062259 · Cited by 63
Learning Accurate Low-Bit Deep Neural Networks with Stochastic Quantization
Dong, Y., Li, J., & Ni, R. (2017) · Procedings of the British Machine Vision Conference 2017 · doi:10.5244/c.31.189 · Cited by 50
LightNN: Filling the Gap between Conventional Deep Neural Networks and Binarized Networks
Ding, R., Liu, Z., Shi, R., Marculescu, D., & Blanton, R. D. (2017) · ACM Great Lakes Symposium on VLSI (GLSVLSI) · doi:10.1145/3060403.3060465 · Cited by 39
Self-citationUnderstanding the Impact of Label Granularity on CNN-Based Image Classification
Chen, Z., Ding, R., Chin, T.-W., & Marculescu, D. (2018) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2018.00131 · Cited by 31
Self-citationLearning Accurate Low-Bit Deep Neural Networks with Stochastic QuantizationPreprint
Dong, Y., Ni, R., Li, J., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.01001 · Cited by 24 · cited 1×
“The benefit of gradual quantization has also been observed by prior work [25] which shows that gradually imposing quantization constraints can achieve better accuracy than directly quantizing with a strict constraint.”
BackgroundRegularizing Activation Distribution for Training Binarized Deep NetworksPreprint
Ding, R., Chin, T.-W., Liu, Z., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02823 · Cited by 20 · cited 1×
“Future work will further improve training efficiency by using optimized training loss [7] or proper labels [5].”
BackgroundSelf-citationLightening the Load with Highly Accurate Storage- and Energy-Efficient LightNNs
Ding, R., Liu, Z., Blanton, R. D., & Marculescu, D. (2018) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3270689 · Cited by 17 · cited 2×
“Since shift operations are much more lightweight on customized hardware ( e.g. , FPGA or ASIC), LightNNs can achieve faster speed and lower energy consumption, and generally maintain accuracy for over-parameterized models [4, 5].”
BackgroundMethodSelf-citationUnderstanding the Impact of Label Granularity on CNN-based Image ClassificationPreprint
Chen, Z., Ding, R., Chin, T.-W., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.07012 · Cited by 4
Self-citationCo-designed systems for deep learning hardware accelerators
Brooks, D. M. (2018) · International Symposium on VLSI Design, Automation and Test · doi:10.1109/vlsi-dat.2018.8373283 · Cited by 1 · cited 1×
“and the architecture of the neural network running on it [3].”
Background
Research works printed but not linked: 2
- S. Han , J. Pool , J. Tran , and W. Dally , " Learning both weights and connections for efficient neural network," in Advances in Neural Information Processing Systems , pp. 1135 -- 1143 , 2015 . S. Han, J. Pool, J. Tran, and W. Dally, "Learning both weights and connections for efficient neural network," in Advances in Neural Information Processing Systems, pp. 1135--1143, 2015.
- A. L. Maas , A. Y. Hannun , and A. Y. Ng , " Rectifier nonlinearities improve neural network acoustic models ," in Proc. ICML , vol. 30 , no. 1 , pp. 3 , 2013 . A. L. Maas, A. Y. Hannun, and A. Y. Ng, "Rectifier nonlinearities improve neural network acoustic models," in Proc. ICML, vol. 30, no. 1, pp. 3, 2013.
Cited by
Single-Path NAS: Designing Hardware-Efficient ConvNets in Less Than 4 Hours
Stamoulis, D., Ding, R., Wang, D., Lymberopoulos, D., Priyantha, B., Liu, J., & Marculescu, D. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-46147-8_29 · Cited by 215
Self-citationA Survey of Methods for Low-Power Deep Learning and Computer Vision
Goel, A., Tung, C., Lu, Y.-H., & Thiruvathukal, G. K. (2020) · IEEE World Forum on Internet of Things (WF-IoT) · doi:10.1109/wf-iot48130.2020.9221198 · Cited by 146
“A constant bit-width for all layers can lead to poor performance [11].”
BackgroundMethodJoint architecture and knowledge distillation in CNN for Chinese text recognition
Wang, Z.-R., & Du, J. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107722 · Cited by 50
“Different from constraining the weights to +1 or − 1, Ding et al. [38,39] replaced the multiply-accumulate operation with one shift or a constrained number of shifts and adds, which can make trade-offs between accuracy and computational consumption.”
BackgroundSingle-Path Mobile AutoML: Efficient ConvNet Design and NAS Hyperparameter Optimization
Stamoulis, D. S., Ding, R., Wang, D., Lymberopoulos, D., Priyantha, N. B., Liu, J., & Marculescu, D. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2971421 · Cited by 41
“We draw inspiration from weight-based conditions that have been successfully used for quantization-related decisions [36], [37] and we use the group Lasso term .”
MethodSelf-citationAutomated Log-Scale Quantization for Low-Cost Deep Neural Networks
Oh, S., Sim, H., Lee, S., & Lee, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00080 · Cited by 43
Metadata
- Authors
- Ding, R., Liu, Z., Chin, T.-W., Marculescu, D., & Blanton, R. D. (.
- Affiliations
- Carnegie Mellon University (US)
- Venue
- Proceedings of the 56th Annual Design Automation Conference 2019
- Publisher
- ACM
- Published
- 2019-06-02
- Pages
- 1-6
- ISSN
- 0738-100X (ISSN-L)
- OpenAlex
- W2963398264
- Cited by
- 19
- References
- 27
- Citations by year
- 2025: 2 · 2024: 2 · 2023: 1 · 2022: 2 · 2021: 5 · 2020: 5 · 2019: 2
- FWCI
- 1.14
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM/IEEE Design Automation Conference (DAC) (publishedVersion)
- Funders
- National Science Foundation (1815899)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Low-power high-performance VLSI design
- Keywords
- lightweight neural networks, FPGA implementation, shift operations, energy efficiency, inference acceleration, ASIC implementation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.