Accelerating Deep Convolutional Networks using low-precision and sparsity
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
We explore techniques to significantly improve the compute efficiency and performance of Deep Convolution Networks without impacting their accuracy. To improve the compute efficiency, we focus on achieving high accuracy with extremely low-precision (2-bit) weight networks, and to accelerate the execution time, we aggressively skip operations on zero-values. We achieve the highest reported accuracy of 76.6% Top-1/93% Top-5 on the Imagenet object classification challenge with low-precision network while reducing the compute requirement by ~3× compared to a full-precision network that achieves similar accuracy. Furthermore, to fully exploit the benefits of our low-precision networks, we build a deep learning accelerator core, DLAC, that can achieve up to 1 TFLOP/mm2equivalent for single-precision floating-point operations (~2 TFLOP/mm2for half-precision), which is ~5× better than Linear Algebra Core [16] and ~4× better than previous deep learning accelerator proposal [8].
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“those accuracies are already comparable or higher than those reported on ternary weight networks42, for example 71.6% top-1 accuracy of ResNet-34 on ImageNet with first layer in FP3245.”
From Accurate deep neural network inference using computational phase-change memory (2020) · cites this paper as [45]
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 2×
“We first evaluate low-precision variant of VGG network on Cifar10 dataset, shown in Figure 3.”
MethodImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 12×
“Imagenet [9] This dataset consists of 1.”
BackgroundInfluentialDelving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · cited 1×
“The accuracy of DNNs is rapidly improving (for example, Top-5 error for Imagenet object classification challenge [9] improved by >6× in 5 years) and is close to human-level accuracy in some cases [12].”
BackgroundIdentity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665
Han, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662 · cited 2×
“In addition to the low-precision networks, DLAC can also accelerate pruned networks [11].”
BackgroundXNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408 · cited 2×
“These include approaches that reduce the weights to half-precision [7], binary value [17], and ternary (2-bit) value [14].”
BackgroundMethodDeep Speech 2: End-to-End Speech Recognition in English and Mandarin
Amodei, D., Anubhai, R., Battenberg, E., Case, C. T., Casper, J., Catanzaro, B., Chen, J., Chrzanowski, M., Coates, A., Diamos, G., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.02595 · Cited by 2,110 · cited 1×
“These include approaches that reduce the weights to half-precision [7], binary value [17], and ternary (2-bit) value [14].”
BackgroundDaDianNao: A Machine-Learning Supercomputer
Chen, Y., Luo, T., Liu, S., Zhang, S., He, L., Wang, J., Li, L., Chen, T., Xu, Z., Sun, N., et al. (2014) · 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1109/micro.2014.58 · Cited by 1,492 · cited 4×
“Also, the greater compute requirements make DNNs harder to deploy, which has led to a lot of interest recently in specialized hardware solutions, both commercially [3, 5] and in academia [10, 6, 8].”
BackgroundResultInfluentialEIE: Efficient Inference Engine on Compressed Deep Neural Network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., & Dally, W. J. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.30 · Cited by 1,077
Research works printed but not linked: 5
- 0, Training and investigating residual nets
- 0, Tensor processing unit
- 0, Wave dataflow engine
- 0, CIFAR-10 Dataset
- 0, 92 45% on cifar-10 in torch
Cited by
Pruning and quantization for deep neural network acceleration: A survey
Liang, T., Glossner, J., Wang, L., Shi, S., & Zhang, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.045 · Cited by 943
SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks
Parashar, A., Rhu, M., Mukkara, A., Puglielli, A., Venkatesan, R., Khailany, B., Emer, J., Keckler, S. W., & Dally, W. J. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080254 · Cited by 896
“The Deep Learning Accelerator Core (DLAC) architecture paper mentions that it can skip computation on zero-valued operands to improve performance, but the architecture does not appear to employ zero compression [32].”
BackgroundSCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks
Parashar, A., Rhu, M., Mukkara, A., Puglielli, A., Venkatesan, R., Khailany, B., Emer, J., Keckler, S. W., & Dally, W. J. (2017) · ACM SIGARCH Computer Architecture News · doi:10.1145/3140659.3080254 · Cited by 413
Accurate deep neural network inference using computational phase-change memory
Joshi, V., Le Gallo, M., Haefeli, S., Boybat, I., Nandakumar, S. R., Piveteau, C., Dazzi, M., Rajendran, B., Sebastian, A., & Eleftheriou, E. (2020) · Nature Communications · doi:10.1038/s41467-020-16108-9 · Cited by 490
Can FPGAs Beat GPUs in Accelerating Next-Generation Deep Neural Networks?
Nurvitadhi, E., Venkatesh, G., Sim, J., Marr, D., Huang, R., Ong Gee Hock, J., Liew, Y. T., Srivatsan, K., Moss, D., Subhaschandra, S., et al. (2017) · Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/3020078.3021740 · Cited by 467
“Ternary DNNs (i.e., Ternary Weight Networks) [4,5] have recently proposed constraining neural network weights to +1, 0, or -1, allowing for weights to be represented with just 2 bits, while simultaneously introducing more sparsity to these weights.”
BackgroundMethodInfluentialSelf-citationAccurate deep neural network inference using computational phase-change memory
Joshi, V., Gallo, M. L., Haefeli, S., Boybat, I., Nandakumar, S. R., Piveteau, C., Dazzi, M., Rajendran, B., Sebastian, A., & Eleftheriou, E. S. (2020) · RePEc: Research Papers in Economics · doi:10.3929/ethz-b-000416797 · Cited by 379
Bayesian Compression for Deep Learning
Louizos, C., Ullrich, K., & Welling, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08665 · Cited by 312
“Several studies show that this has little to no effect on network accuracy when using 16bit weights [Merolla et al., 2016, Gupta et al., 2015, Courbariaux et al., 2014, Venkatesh et al., 2016, Chai et al., 2017].”
BackgroundDRISA: a DRAM-based Reconfigurable In-Situ Accelerator
Li, S., Niu, D., Malladi, K. T., Zheng, H., Brennan, B., & Xie, Y. (2017) · Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1145/3123939.3123977 · Cited by 326
Extremely Low Bit Neural Network: Squeeze the Last Bit Out With ADMM
Leng, C., Dou, Z., Li, H., Zhu, S., & Jin, R. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11713 · Cited by 275
A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification
Rokh, B., Azarpeyvand, A., & Khanteymoori, A. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3623402 · Cited by 232
“By using the scale layer and quantization of the last layer, work [115] achieved a compression rate of approximately 4.5 times in NIN-net [153] compared to keeping the last layer in full precision while top-5 accuracy decreased only %1.”
BackgroundMethodInfluentialFlexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networks
Köster, U., Webb, T. J., Wang, X., Nassar, M., Bansal, A. K., Constable, W., Elibol, O. H., Gray, S., Hall, S., Hornof, L., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.02213 · Cited by 164
“These include quantization of weights and/or activations to 8-bit [8, 9, 10, 11], down to 4-bit, 2-bit [12, 13] or ternary [10], and ultimately all binary [7, 14, 5, 6].”
MethodKwon, Y., Lee, Y., & Rhu, M. (2019) · Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1145/3352460.3358284 · Cited by 218
“[59], [60], [61], [3], [62], [63], [64], [65], [66], [67], [68], [69], [70], [71] with recent interest on leveraging DNN sparsity for further energy-efficiency improvements [11], [4], [72], [8], [73], [74], [75], [76], [77], [5], [78], [79]. A scale-out acceleration platform for training DL algorithms was proposed by P…”
BackgroundAhmad, S., Shakeel, I., Mehfuz, S., & Ahmad, J. (2023) · Computer Science Review · doi:10.1016/j.cosrev.2023.100568 · Cited by 197
An introduction to deep learning in medical physics: advantages, potential, and challenges
Shen, C., Nguyen, D., Zhou, Z., Jiang, S. B., Dong, B., & Jia, X. (2020) · Physics in Medicine & Biology · doi:10.1088/1361-6560/ab6f51 · Cited by 185
“Classical regularizers including weight decay and sparsity have shown to be effective in many studies (Boureau and Cun, 2008; Glorot et al., 2011a; Krizhevsky et al., 2012; Venkatesh et al., 2017).”
BackgroundDeep Learning on Mobile and Embedded Devices: State-of-the-art, Challenges, and Future Directions
Chen, Y., Zheng, B., Zhang, Z., Wang, Q., Shen, C., & Zhang, Q. (2021) · ACM Computing Surveys · doi:10.1145/3398209 · Cited by 159
“Ganesh et al. [177] achieved high accuracy with binary weight networks while accelerating the execution time by aggressively skipping operations on zero-values.”
BackgroundPREMA: A Predictive Multi-Task Scheduling Algorithm For Preemptible Neural Processing Units
Choi, Y., & Rhu, M. (2020) · 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA) · doi:10.1109/hpca47549.2020.00027 · Cited by 142
“Aside from these closely related prior work, there has been a large body of studies exploring the design of architectures for ML [20], [21], [22], [1], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33] with recent interest on sparsity-optimized solutions for further energy-efficiency improvements [2], […”
BackgroundSYQ: Learning Symmetric Quantization for Efficient Deep Neural Networks
Faraone, J., Fraser, N., Blott, M., & Leong, P. H. W. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00452 · Cited by 134
Knowledge‐based radiation treatment planning: A data‐driven method survey
Momin, S., Fu, Y., Lei, Y., Roper, J. R., Bradley, J. D., Curran, W. J., Liu, T., & Yang, X. (2021) · Journal of Applied Clinical Medical Physics · doi:10.1002/acm2.13337 · Cited by 146
“Data augmentation [122], dropout layer [121], estimation based on the training and the validation curves [100], synthesizing new data based on physics principles [86] or incorporating regularizations to model parameters [132] have been used in the literature to prevent overfitting.”
MethodDave, S., Baghdadi, R., Nowatzki, T., Avancha, S., Shrivastava, A., & Li, B. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3098483 · Cited by 111
Norm matters: efficient and accurate normalization schemes in deep networks
Hoffer, E., Banner, R., Golan, I., & Soudry, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.01814 · Cited by 90
SCOPE: A Stochastic Computing Engine for DRAM-Based In-Situ Accelerator
Li, S., Glova, A. O., Hu, X., Gu, P., Niu, D., Malladi, K. T., Zheng, H., Brennan, B. A., & Xie, Y. (2018) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2018.00062 · Cited by 131
Yuan, Z., Liu, Y., Yue, J., Yang, Y., Wang, J., Feng, X., Zhao, J. P., Li, X., & Yang, H. (2019) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2019.2946771 · Cited by 92
“Making good use of sparsity can potentially lower storage and computation requirements [14], which makes CNNs more”
BackgroundTernary Neural Networks with Fine-Grained Quantization
Mellempudi, N., Kundu, A., Mudigere, D., Das, D., Kaul, B., & Dubey, P. K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.01462 · Cited by 55
“Table 1 has a comparison with previous reported results from[23] using 5 -bit weights and [21] using ternary weights.”
BackgroundMethodResultInfluentialAn Energy-Efficient Architecture for Binary Weight Convolutional Neural Networks
Wang, Y., Lin, J. T., & Wang, Z. (2017) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2017.2767624 · Cited by 98
Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations
Lai, L., Suda, N., & Chandra, V. (2017) · arXiv · doi:10.48550/arxiv.1703.03073 · Cited by 81
“The continuous research effort to reduce the data precision has led to many interesting demonstrations with 2-bit weights (Venkatesh et al., 2016) and even binary weights/activations (Courbariaux & Bengio, 2016; Rastegari et al.”
BackgroundLi, W., Sun, X., Huang, S., Jiang, H., & Yu, S. (2022) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2022.3163197 · Cited by 77
“We implement a sparsity-aware input controller that leverages the typically high sparsity level of input activations in DNN models [35].”
MethodVIBNN: Hardware Acceleration of Bayesian Neural Networks
Cai, R., Ren, A., Liu, N., Ding, C., Wang, L., Qian, X., Pedram, M., & Wang, Y. (2018) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3173162.3173212 · Cited by 83
“Despite the previous works such as [23, 25, 51] leveraging this technique, it is not trivial to investigate and apply this technique in VIBNN, because the introduced random noises may lead to a different conclusion.”
Boutros, A., Yazdanshenas, S., & Betz, V. (2018) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3242898 · Cited by 93
“Another compression technique reduces precision down to ternary [31, 49] or binary [29, 47] networks that are inherently more FPGA-friendly, and exhibit little or no accuracy degradation by increasing the size of the network as in Reference [28].”
BackgroundLazy Batching: An SLA-aware Batching System for Cloud Machine Learning Inference
Choi, Y., Kim, Y., & Rhu, M. (2021) · 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca51647.2021.00049 · Cited by 69
“While there has been lots of interest in designing energyefficient NPU architectures for training and inference in isolation [1], [2], [8], [10]–[13], [16], [18], [19], [24], [32], [37], [38], [40], [41], [43]–[51], [55], [56], [58], [62], [65]– [67], [74], [75], [80]–[82], [85], little attention has been paid in how…”
BackgroundAddNet: Deep Neural Networks Using FPGA-Optimized Multipliers
Faraone, J., Kumm, M., Hardieck, M., Zipf, P., Liu, X., Boland, D., & Leong, P. H. W. (2020) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2019.2939429 · Cited by 66
“Similarly, training highly sparse ternary networks and designing efficient CNN hardware for exploitation was described in [30].”
BackgroundMethodAdaptive approximate computing in edge AI and IoT applications: A review
Damsgaard, H. J., Grenier, A., Katare, D., Taufique, Z., Shakibhamedan, S., Troccoli, T., Chatzitsompanis, G., Kanduri, A., Ometov, A., Ding, A. Y., et al. (2024) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2024.103114 · Cited by 66
“First shown to be a viable strategy in [159], subsequent work has demonstrated reductions of weights and/or activations to 8-bit [165], 4-bit [165], 2-bit [156,157], and even binary [158] widths.”
BackgroundEfficient tensor core-based GPU kernels for structured sparsity under reduced precision
Chen, Z., Qu, Z., Liu, L., Ding, Y., & Xie, Y. (2021) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3458817.3476182 · Cited by 73
“To mitigate this issue, reduced precision and sparsity are usually exploited jointly [9, 13, 19, 27].”
BackgroundBeyond the Memory Wall: A Case for Memory-Centric HPC System for Deep Learning
Kwon, Y., & Rhu, M. (2018) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2018.00021 · Cited by 81
“A large body of prior work has explored the design of a single accelerator device architecture for deep learning inference [42], [46], [83], [84], [85], [86], [87], [88], [89], [90], [91], [92], [93], [94], [95] with an increased interest on leveraging DNN sparsity for further energy-efficiency improvements [47], [96]…”
BackgroundVIBNN: Hardware Acceleration of Bayesian Neural Networks
Cai, R., Ren, A., Liu, N., Ding, C., Wang, L., Qian, X., Pedram, M., & Wang, Y. (2018) · ACM SIGPLAN Notices · doi:10.1145/3296957.3173212 · Cited by 78
Learning Accurate Low-Bit Deep Neural Networks with Stochastic Quantization
Dong, Y., Li, J., & Ni, R. (2017) · Procedings of the British Machine Vision Conference 2017 · doi:10.5244/c.31.189 · Cited by 50
Optimizing N-dimensional, winograd-based convolution for manycore CPUs
Jia, Z., Zlateski, A., Durand, F., & Li, K. (2018) · ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming · doi:10.1145/3178487.3178496 · Cited by 59
“3D ConvNets are becoming more and more important, as they have been successfully applied to many fields [23, 30, 38, 39].”
BackgroundInfluentialFPGA-based CNN inference accelerator synthesized from multi-threaded C software
Kim, J. H., Grady, B., Lian, R., Brothers, J., & Anderson, J. H. (2017) · IEEE International System-on-Chip Conference (SOCC) · doi:10.1109/socc.2017.8226056 · Cited by 58
“Moreover, the reconfigurability of FPGAs permits an accelerator design to be adapted to incorporate new research findings as they arise, for example, the ability to achieve high recognition accuracy with 2-bit precision [6].”
MethodFu, Y., Ye, Z., Deng, J., Zheng, X., Huang, Y., Yang, W., Wang, Y., & Wang, K. (2019) · Remote Sensing · doi:10.3390/rs11141678 · Cited by 49
“Second, future studies may try to accelerate the training and inference process by using a series of model compressing methods, such as parameter pruning and sharing [71], low-rank factorization [72], network quantization [73], and knowledge distillation [74].”
BackgroundLi, W., Huang, S., Sun, X., Jiang, H., & Yu, S. (2021) · Custom Integrated Circuits Conference · doi:10.1109/cicc51472.2021.9431558 · Cited by 47
“B L[0] SL[0] B L[1] B L[126] SL[63] B L[127]”
MethodReliability Evaluation of Mixed-Precision Architectures
Fernandes dos Santos, F., Lunardi, C., Oliveira, D., Libano, F., & Rech, P. (2019) · 2019 IEEE International Symposium on High Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2019.00041 · Cited by 44
Energy-efficient, low-latency realization of neural networks through boolean logic minimization
Nazemi, M., Pasandi, G., & Pedram, M. (2019) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1145/3287624.3287722 · Cited by 49
“Examples of using other data types and/or low-bit-width representations can be found in [7–9].”
BackgroundExploration of Low Numeric Precision Deep Learning Inference Using Intel® FPGAs
Colangelo, P., Nasiri, N., Nurvitadhi, E., Mishra, A., Margala, M., & Nealis, K. (2018) · IEEE International Symposium on Field-Programmable Custom Computing Machines (FCCM) · doi:10.1109/fccm.2018.00020 · Cited by 61
Self-citationImpact of Tensor Cores and Mixed Precision on the Reliability of Matrix Multiplication in GPUs
Basso, P. M., dos Santos, F. F., & Rech, P. (2020) · IEEE Transactions on Nuclear Science · doi:10.1109/tns.2020.2977583 · Cited by 41
“Executing CNNs with low precision has been shown to significantly reduce the GPUs’ power consumption and execution time, with a negligible impact on the object detection accuracy [3].”
BackgroundMethodWhen FPGA Meets Cloud: A First Look at Performance
Wang, X., Niu, Y., Liu, F., & Xu, Z. (2020) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2020.2992548 · Cited by 52
“In [39], the authors present an FPGA-based deep neural network (DNN) model using SGEMM to perform convolutions.”
MethodCococcioni, M., Rossi, F., Ruffaldi, E., Saponara, S., & de Dinechin, B. D. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2988436 · Cited by 43
Qiu, K., Jao, N., Zhao, M., Mishra, C. S., Akbulut, G. G., Jose, S., Sampson, J., Kandemir, M., & Narayanan, V. (2020) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca47549.2020.00034 · Cited by 50
“The impact of different precisions on CNN accuracy has been extensively studied [ 24], [25], [26], [27], [28], [29], [30], [31], [32], [33].”
BackgroundUnrolling Ternary Neural Networks
Tridgell, S., Kumm, M., Hardieck, M., Boland, D., Moss, D. J. M., Zipf, P., & Leong, P. H. W. (2019) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3359983 · Cited by 40
“[Venkatesh et al. 2017] use 14nm technology to claim a peak speed of 2.5 TOps/sec with a size of 1.09mm2.”
BackgroundMethodInfluentialMixed Low-precision Deep Learning Inference using Dynamic Fixed PointPreprint
Mellempudi, N., Kundu, A., Das, D., Mudigere, D., & Kaul, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.08978 · Cited by 37
“Although, previous research in this space[13] showed that it is possible to recover some of this lost accuracy through retraining.”
BackgroundMethodResultApproximate Computing Methods for Embedded Machine Learning
Ibrahim, A., Osta, M., Alameh, M., Saleh, M., Chible, H., & Valle, M. (2018) · 2018 25th IEEE International Conference on Electronics, Circuits and Systems (ICECS) · doi:10.1109/icecs.2018.8617877 · Cited by 31
“Lowering the precision of network weights [26] Deep convolution network Reduction of the compute requirement by ∼3× 7% to 23.”
MethodIF-CNN: Image-Aware Inference Framework for CNN With the Collaboration of Mobile Devices and Cloud
Shu, G., Liu, W., Zheng, X., & Li, J. (2018) · IEEE Access · doi:10.1109/access.2018.2880196 · Cited by 30
“Recently, NVIDIA has proposed mixed-precision for both of the training and inference processes, the acceleration effect has been verified [47] in related works.”
BackgroundNeuMMU: Architectural Support for Efficient Address Translations in Neural Processing Units
Hyun, B., Kwon, Y., Choi, Y., Kim, J., & Rhu, M. (2020) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3373376.3378494 · Cited by 38
“8], [79], [80], [81], [82], [83], [84], [85], [86], [87], [88], [89] with recent interest on sparsity-optimized solutions for further energy-efficiency improvements [37], [38], [39], [90], [91], [92], [93], [94], [95], [96], [97], [98]. Our work on NPU MMUs is orthogonal to these prior art as our primary focus is on ad…”
NullaNet: Training Deep Neural Networks for Reduced-Memory-Access InferencePreprint
Nazemi, M., Pasandi, G., & Pedram, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.08716 · Cited by 30
Quantum Denoising Diffusion Models
Kölle, M., Stenzel, G., Stein, J., Zielinski, S., Ommer, B., & Linnhoff–Popien, C. (2024) · IEEE International Conference on Quantum Software (QSW) · doi:10.1109/qsw62656.2024.00023 · Cited by 51
“Adopting 16-bit float precision could notably reduce RAM usage [14, 51], considering the demonstrated success in classical machine learning and prevalent GPU support for FP16.”
BackgroundTinyMLOps: Operational Challenges for Widespread Edge AI Adoption
Leroux, S., Simoens, P., Lootus, M., Thakore, K., & Sharma, A. (2022) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw55747.2022.00160 · Cited by 37
Reconfigurable Convolutional Kernels for Neural Networks on FPGAs
Hardieck, M., Kumm, M., Möller, K., & Zipf, P. (2019) · ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA) · doi:10.1145/3289602.3293905 · Cited by 31
“Deep convolutional neural networks (CNNs) are one of the most successful machine learning approaches providing state-of-the-art classification accuracy for object detection and classification.”
BackgroundMethodInfluentialSamiee, A., Borulkar, P., DeMara, R. F., Zhao, P., & Bai, Y. D. (2019) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2019.2915589 · Cited by 25
“A recent industry-based study has exploited acceleration of deep learning at the data-center-scale seeking high cost ef fi ciency through FPGAs [13].”
BackgroundPruNet: Class-Blind Pruning Method For Deep Neural Networks
Marchisio, A., Hanif, M. A., Martina, M., & Shafique, M. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489764 · Cited by 31
Stochastic Quantization for Learning Accurate Low-Bit Deep Neural Networks
Dong, Y., Ni, R., Li, J., Chen, Y., Su, H., & Zhu, J. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01168-2 · Cited by 27
“Note that someworks (Rastegari et al. 2016;Venkatesh et al. 2016) keep the first or last layer full precision to alleviate possible”
BackgroundResultDelmás, A., Judd, P., Stuart, D. M., Poulos, Z., Mahmoud, M., Sharify, S., Nikolić, M., & Moshovos, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.03688 · Cited by 20
“In-formational inefficiency manifests in CNNs as ineffectual weights [4,5,6], activations [4,6,7], as an excess of precision, e.g., [8,9,10,11,12,13], as ineffectual activation bits [14], or in general as over-provisioning, e.g., [15].”
BackgroundZhang, J., & Bao, T. (2023) · Water · doi:10.3390/w15152839 · Cited by 18
Hyperdrive: A Multi-Chip Systolically Scalable Binary-Weight CNN Inference Engine
Andri, R., Cavigelli, L., Rossi, D., & Benini, L. (2019) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2019.2905654 · Cited by 19
“Recently, several methods to train neural networks to withstand extreme quantization have been proposed, yielding the notions of binary- and ternary-weight networks (BWNs, TWNs) and binarized neural networks (BNNs) [21–23].”
Hyperdrive: A Systolically Scalable Binary-Weight CNN Inference Engine for mW IoT End-Nodes
Andri, R., Cavigelli, L., Rossi, D., & Benini, L. (2018) · IEEE Computer Society Annual Symposium on VLSI · doi:10.1109/isvlsi.2018.00099 · Cited by 21
Scalable FPGA Accelerator for Deep Convolutional Neural Networks with Stochastic Streaming
Alawad, M., & Lin, M. (2018) · IEEE Transactions on Multi-Scale Computing Systems · doi:10.1109/tmscs.2018.2886266 · Cited by 18
“is used in ternary neural networks [27].”
Aspect-Driven Mixed-Precision Tuning Targeting GPUs
Nobre, R., Reis, L. P., Bispo, J., Carvalho, T., Cardoso, J. M. P., Cherubin, S., & Agosta, G. (2018) · doi:10.1145/3183767.3183776 · Cited by 17
“The use of data-types smaller than 32-bit can be useful in a broad selection of applications such as deep neural network training [7, 29] and inferencing [30], some image processing kernels [28] and radioastronomy [16].”
MethodA 701.7 TOPS/W Compute-in-Memory Processor With Time-Domain Computing for Spiking Neural Network
Park, K., Jeong, H., Kim, S., Shin, J., Kim, M. H., & Lee, K. J. (2024) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2024.3480350 · Cited by 15
Overview of the state of the art in embedded machine learning
Andrade, L., Prost-Boucle, A., & Pétrot, F. (2018) · Design, Automation, and Test in Europe · doi:10.23919/date.2018.8342164 · Cited by 16
“Venkatesh et al. [11] proposed to keep floating-point accumulations but to use only balanced ternary weights (possible values are − 1 , 0 and +1 ).”
BackgroundAksoy, B., Halis, H. D., & SALMAN, O. K. M. (2020) · International Journal of Engineering and Innovative Research · doi:10.47933/ijeir.772514 · Cited by 22
Faraone, J., Fraser, N. J., Gambardella, G., Blott, M., & Leong, P. H. W. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-70096-0_41 · Cited by 20
“We use similar Convolu-tional Neural Networks (CNNs) to [6] for CIFAR10 classification and achieve similar accuracies, although their network has a full precision 1st layer compared to our ternary weights.”
MethodArtificial Neural Network and Accelerator Co-design using Evolutionary Algorithms
Colangelo, P., Segal, O., Speicher, A., & Margala, M. (2019) · IEEE High Performance Extreme Computing Conference · doi:10.1109/hpec.2019.8916533 · Cited by 15
“Specifically, these reconfigurable architectures are a popular platform for both research and deployment due to their ability to change their fabric routing and resource structures to fit various workloads and optimizations by leveraging the resiliency of neural networks through low-numeric precision [17][18] and spar…”
BackgroundIrregular Convolutional Neural Networks
Ma, J., Wang, W., & Wang, L. (2017) · Asian Conference on Pattern Recognition · doi:10.1109/acpr.2017.108 · Cited by 13
Deep Residual Learning Approach forPlant Disease Recognition
Pavel, M. I., Rumi, R. I., Fairooz, F., Jahan, S., & Hossain, M. A. (2020) · EAI/Springer Innovations in Communication and Computing · doi:10.1007/978-3-030-49795-8_49 · Cited by 14
DPRed: Making Typical Activation Values Matter In Deep Learning Computing.Preprint
Delmás, A., Sharify, S., Judd, P., Nikolić, M., & Moshovos, A. (2018) · arXiv (Cornell University) · Cited by 13
nZESPA: A Near-3D-Memory Zero Skipping Parallel Accelerator for CNNs
Das, P., & Kapoor, H. K. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3022330 · Cited by 13
Application of convolutional neural networks on Intel® Xeon® processor with integrated FPGA
Colangelo, P., Lübbers, E., Huang, R., Margala, M., & Nealis, K. (2017) · IEEE High Performance Extreme Computing Conference · doi:10.1109/hpec.2017.8091025 · Cited by 17
“Until recently, few designs have been published on accelerating binary networks [8][12][13] and even fewer prototyped and benchmarked.”
BackgroundNovel Arithmetics to Accelerate Machine Learning Classifiers in Autonomous Driving Applications
Cococcioni, M., Rossi, F., Ruffaldi, E., & Saponara, S. (2019) · IEEE International Conference on Electronics, Circuits and Systems (ICECS) · doi:10.1109/icecs46596.2019.8965031 · Cited by 11
“To address the above issues new computing arithmetic styles are appearing in research [11-20] overcoming the classic fixed-point (INT) vs.”
BackgroundScalable-effort ConvNets for multilevel classification
Peluso, V., & Calimera, A. (2018) · doi:10.1145/3240765.3240845 · Cited by 11
“dynamic precision scaling [22] and pruning [25, 27].”
Releasing the Potential of Tensor Core for Unstructured SpMM using Tiled-CSR Format
Xue, Z., Wen, M., Chen, Z., Shi, Y., Tang, M., Yang, J., & Luo, Z. (2023) · IEEE International Conference on Computer Design: VLSI in Computers and Processors (ICCD) · doi:10.1109/iccd58817.2023.00076 · Cited by 13
“Lower precision uses fewer bits for storage and computation, and the low-precision units can improve the computation throughput [2].”
BackgroundRethinking Floating Point Overheads for Mixed Precision DNN AcceleratorsPreprint
Abdel-Aziz, H. A., Shafiee, A. R., Shin, J. H., Pedram, A., & Hassoun, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.11748 · Cited by 16
“are considered to maintain FP32-level accuracy (Zhu et al., 2016; Venkatesh et al., 2017).”
BackgroundBit-Slicing FPGA Accelerator for Quantized Neural Networks
Bilaniuk, O., Wagner, S. J., Savaria, Y., & David, J. (2019) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2019.8702332 · Cited by 16
“Ternary [2], two-bit [3], and six-bit [4] representations have also been seen.”
BackgroundA Camera That CNNs: Towards Embedded Neural Networks on Pixel Processor Arrays
Bose, L., Dudek, P., Chen, J., Carey, S. J., & Mayol-Cuevas, W. W. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00142 · Cited by 49
“One particular area of interest is that of using low precision weights and neuron activations [26, 18] in order to greatly decrease memory requirements and remove the majority of computational work arising from real value multiplication.”
BackgroundWRPN: Training and Inference using Wide Reduced-Precision NetworksPreprint
Mishra, A., Cook, J. J., Nurvitadhi, E., & Marr, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.03079 · Cited by 13
“A promising approach to deliver one such extremely efficient solution is through the use of low numeric precision deep learning algorithms.”
Self-citationBitNet: Bit-Regularized Deep Neural NetworksPreprint
Raghavan, A., Amer, M., Chai, S. M., & Taylor, G. W. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.04788 · Cited by 12
“In the past few years, the intriguing fact that the performance of deep neural networks is robust to parsimonious representations of parameters [33, 47], activations [30, 20] and even gradients [48] has sparked research in both machine learning [15, 10, 16] and hardware [23, 43, 18].”
BackgroundOptimizing N-dimensional, winograd-based convolution for manycore CPUs
Jia, Z., Zlateski, A., Durand, F., & Li, K. (2018) · ACM SIGPLAN Notices · doi:10.1145/3200691.3178496 · Cited by 14
IEEE-754 Half-Precision Floating-Point Low-Latency Reciprocal Square Root IP-Core
Aguilera-Galicia, C. R., Longoria‐Gandara, O., Guzman-Ramos, O. A., Pizano‐Escalante, L., & Vázquez-Castillo, J. (2018) · IEEE Latin-American Conference on Communications · doi:10.1109/latincom.2018.8613254 · Cited by 8
“Several convolutional NN accelerator have been implemented using half-precision arithmetic [14], or even more reduced FP precision [15], achieving high accuracy and performance in image classification, while reducing computing requirements.”
MethodSparTANN: sparse training accelerator for neural networks with threshold-based sparsification
Sim, H., Choi, J., & Lee, J. (2020) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1145/3370748.3406554 · Cited by 9
“Previous sparse training hardware architectures [7, 19] do essentially zero skipping, removing only what is known to be inef-fectual, but the mechanism to achieve it may vary.”
BackgroundDPRed: Making Typical Activation and Weight Values Matter In Deep Learning ComputingPreprint
Delmás, A., Sharify, S., Judd, P., Siu, K., Nikolić, M., & Moshovos, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.06732 · Cited by 11
“…and exploited various forms of informational inefficiency in DNNs such as ineffectual neurons [4, 5], activations [4, 6, 5], ineffectual weights [7, 5], an excess of precision [8, 9, 10, 11, 12, 13, 14, 15], ineffectual activation bits [16], and hyper-parameter over-provisioning, e.g., [17, 18].”
BackgroundUltra-Fast Machine Learning Inference through C Code Generation for Tangled Program Graphs
Desnos, K., Bourgoin, T., Dardaillon, M., Sourbier, N., Gesny, O., & Pelcat, M. (2022) · SiPS ... design and implementation - IEEE Workshop on Signal Processing Systems · doi:10.1109/sips55645.2022.9919237 · Cited by 10
“The need for fast and low-power AIs is the subject of many studies in the scientific literature [8], [13], [17].”
BackgroundMethodLightweight Asymmetric Dilation Network for Real-Time Semantic Segmentation
Hu, X., & Gong, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3071866 · Cited by 8
Multi-spectral cloud detection based on a multi-dimensional and multi-grained dense cascade forest
Shao, M., & Zou, Y. (2021) · Journal of Applied Remote Sensing · doi:10.1117/1.jrs.15.028507 · Cited by 8
“Considering the running environment and the scale of the multi-spectral cloud detection, ResNet-34 40,41 In the design of MDForest, three grained sliding windows with a sliding ratio of 0.125, 0.25, and 0.5 are first performed on original satellite cloud images.”
BackgroundMethodInfluentialRethinking arithmetic for deep neural networks
Constantinides, G. A. (2020) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2019.0051 · Cited by 8
An Efficient Streaming Accelerator for Low Bit-Width Convolutional Neural Networks
Chen, Q., Fu, Y., Song, W., Cheng, K., Lu, Z., Zhang, C., & Li, L. (2019) · Electronics · doi:10.3390/electronics8040371 · Cited by 6
“Recently, there is an increasing amount of research [17,18,21,26,27] focusing on the accelerators targeting on low bit-width CNNs. Wang et al. [17], Venkatesh et al. [26] and Andri et al. [18] all propose their own architecture for deep Binarized CNNs, working by “one size fits all” approach.”
BackgroundHadaNets: Flexible Quantization Strategies for Neural Networks
Akhauri, Y. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00078 · Cited by 6
“Current attempts at reducing the memory footprint of neural networks focus primarily on compression techniques [12, 6, 5, 8] or bit-width reduction [7, 11, 9, 26, 28, 19].”
Sirojan, T., Phung, B. T., & Eliathamby Ambikairajah, A. (2018) · International Conference on Information and Automation for Sustainability (ICIAfS) · doi:10.1109/iciafs.2018.8913352 · Cited by 6
“Furthermore, different data types, that require lower number of bits, can be used to represent the model parameters [13].”
BackgroundEfficient convolutional networks learning through irregular convolutional kernels
Guo, W., Ma, J., Ouyang, Y., Wang, L., & Huang, Y. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.02.065 · Cited by 6
Limits to the Energy Efficiency of CMOS Microprocessors
Ho, A., Erdil, E., & Besiroglu, T. (2023) · IEEE International Conference on Rebooting Computing (ICRC) · doi:10.1109/icrc60800.2023.10386559 · Cited by 21
“While common formats like FP32 or FP64 use higher precision, various emerging workloads are resilient to reduced precision [29], [30].”
BackgroundMethodDeploying Customized Data Representation and Approximate Computing in Machine Learning Applications
Nazemi, M., & Pedram, M. (2018) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1145/3218603.3218612 · Cited by 5
“Lop can be used to answer questions such as the following: • Using fixed-point representation, how many bits are required to represent weights, biases, and activations in a deep neural network to reach a target prediction or classification accuracy [1–3]?”
BackgroundInfluentialInstant Quantization of Neural Networks using Monte Carlo Methods
Mordido, G., Keirsbilck, M. V., & Keller, A. (2019) · Workshop on Energy Efficient Machine Learning and Cognitive Computing for Embedded Applications (EMC2) · doi:10.1109/emc2-nips53020.2019.00014 · Cited by 9
Multiple-Precision Arithmetic Implementation of the Multilevel Fast Multipole Algorithm
Kalfa, M., Ergül, Ö., & Erturk, V. B. (2023) · IEEE Transactions on Antennas and Propagation · doi:10.1109/tap.2023.3291077 · Cited by 4
“Specifically for low-precision computation, recent research efforts are mostly focused on accelerating the training of machine learning algorithms [26], [27].”
BackgroundCan GPU performance increase faster than the code error rate?
dos Santos, F. F., & Rech, P. (2024) · The Journal of Supercomputing · doi:10.1007/s11227-024-06119-4 · Cited by 4
“If the programmer chooses to use a lower precision execution, both the FIT rate and execution time are reduced significantly, achieving, on average, over 10 × (up to ∼ 25 × for LavaMD) a higher number of executions correctly completed between failures.”
Alawad, M., & Tourassi, G. D. (2019) · IEEE Computer Society Annual Symposium on VLSI · doi:10.1109/isvlsi.2019.00033 · Cited by 6
“2-bit data representation is used in ternary neural networks [10] to represent weights, however neuron values are represented by 32 bits.”
Method
Metadata
- Authors
- Venkatesh, G., Nurvitadhi, E., & Marr, D.
- Affiliations
- Intel (United States) (US)
- Venue
- 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Publisher
- IEEE
- Published
- 2017-03
- Pages
- 2861-2865
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W2529546376
- Cited by
- 123
- References
- 13
- Citations by year
- 2026: 1 · 2024: 5 · 2023: 9 · 2022: 7 · 2021: 9 · 2020: 22 · 2019: 23 · 2018: 31 · 2017: 16
- FWCI
- 7.36
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE International Conference on Acoustics Speech and Signal Processing (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Low-power high-performance VLSI design
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Keywords
- deep convolutional neural networks, low-precision arithmetic, sparse computation, computational efficiency, deep learning accelerators, energy efficiency, image classification, ImageNet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.