Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review
Open accessThe DOI matches its Crossref record; 408 of the 491 references it cites were found in a registry or an index.
Abstract
Successful integration of deep neural networks (DNNs) or deep learning (DL) has resulted in breakthroughs in many areas. However, deploying these highly accurate models for data-driven, learned, automatic, and practical machine learning (ML) solutions to end-user applications remains challenging. DL algorithms are often computationally expensive, power-hungry, and require large memory to process complex and iterative operations of millions of parameters. Hence, training and inference of DL models are typically performed on high-performance computing (HPC) clusters in the cloud. Data transmission to the cloud results in high latency, round-trip delay, security and privacy concerns, and the inability of real-time decisions. Thus, processing on edge devices can significantly reduce cloud transmission cost. Edge devices are end devices closest to the user, such as mobile phones, cyber–physical systems (CPSs), wearables, the Internet of Things (IoT), embedded and autonomous systems, and intelligent sensors. These devices have limited memory, computing resources, and power-handling capability. Therefore, optimization techniques at both the hardware and software levels have been developed to handle the DL deployment efficiently on the edge. Understanding the existing research, challenges, and opportunities is fundamental to leveraging the next generation of edge devices with artificial intelligence (AI) capability. Mainly, four research directions have been pursued for efficient DL inference on edge devices: 1) novel DL architecture and algorithm design; 2) optimization of existing DL methods; 3) development of algorithm–hardware codesign; and 4) efficient accelerator design for DL deployment. This article focuses on surveying each of the four research directions, providing a comprehensive review of the state-of-the-art tools and techniques for efficient edge inference.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These data cannot be managed by relying only on the limited functional services provided by cloud service centers (Ge et al. 2023;Shuvo et al. 2022).”
From Survey: federated learning data security and privacy-preserving in edge-Internet of Things (2024) · cites this paper as Shuvo et al. 2022
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 1×
“NLP models, such as Transformer [45] and BERT [224], have around 80% [225] and 93% [226] weight sparsity.”
MethodGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · cited 1×
“Thus, splitting the computation of convolution and dense layer between GPU and CPU can achieve a 60× speedup for object detection using YOLO in the Jetson TK1 board [283].”
MethodDensely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
(2014) · Cited by 33,419
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755 · cited 3×
“MobiFace [386] is an improvement on Mobile-FaceNet using residual bottleneck [72] with an expansion layer and fast downsampling.”
MethodLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · cited 1×
“Convolutions are simplified using 1 × 1 convolution in MobileNet [70] and Xception [56] networks.”
MethodMobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 2×
“The idea of KD was developed in [181] and generalized in [182].”
BackgroundMethodKaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · cited 1×
“Data augmentation [410], [411] and representation learning could [412] be two promising future research directions to mitigate the data inconsistency problem at the edge.”
BackgroundTensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · cited 1×
“Ten-sorFlow [255], Caffe [256], and PyTorch [257] are some of the popular frameworks.”
BackgroundShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982 · cited 1×
“In ShuffleNet [75], convolutions are applied parallelly to separate parts of the inputs reducing the number of operations.”
MethodShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01083 · Cited by 9,982
Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.-C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., Pang, R., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00140 · Cited by 9,708 · cited 1×
“MobileNet-v3 [64] released in large and small versions uses NAS and NetAdapt [65] algorithms.”
Method(2012) · Cited by 9,062
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · cited 1×
“Intelligent agents continuously learn to refine their actions over states from the environment to maximize the accumulative reward functions [41], [42].”
BackgroundTransformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
Rethinking Atrous Convolution for Semantic Image Segmentation
Chen, L.-C., Papandreou, G., Schroff, F., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1706.05587 · Cited by 7,517 · cited 1×
“Upsampling operation in CNN inserts zero weights resulting in sparsity [221].”
BackgroundLearning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2017) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.48550/arxiv.1707.07012 · Cited by 6,376
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
Learning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00907 · Cited by 6,249 · cited 1×
“NASNet [59] starts with a sequence of blocks in which the construction of blocks is determined by RL. NASNet consists of the normal block and the reduction block.”
MethodSqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016) · arXiv · doi:10.48550/arxiv.1602.07360 · Cited by 6,201 · cited 5×
“1 × 1 convolution is also used in SqueezeNet [73] to reduce the number of channels in a CNN.”
MethodInfluentialGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 1×
“Residual connections [91] and skip connections [92] can also be exploited in RNN for faster convergence and performance boost.”
BackgroundAn overview of gradient descent optimization algorithms
Ruder, S. (2016) · arXiv · doi:10.48550/arxiv.1609.04747 · Cited by 4,962 · cited 1×
“There are several optimization algorithms, such as stochastic gradient descent (SGD), ADAM, RMSProp, and AdaGrad [35].”
MethodHan, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662 · cited 5×
“Sparse weight compression is an effective technique that results in 20%–30% memory access reduction [157].”
BackgroundMethodInfluentialInception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537 · cited 2×
“Tianjic [324] and Simba [325] are two spatial architectures that utilized the concept of distributed weights [326] and weight exchange among PEs [319].”
BackgroundResultXNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215 · cited 6×
“Block diagram of the generic DNN inference accelerator [2].”
BackgroundMethodInfluentialNeural Architecture Search with Reinforcement Learning
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 3,974 · cited 1×
“Agents can be trained through RL [61] to estimate the performance of a novel architecture on unseen data.”
BackgroundSemi-Supervised Learning Literature Survey
Zhu, X. (2005) · Minds at UW (University of Wisconsin) · Cited by 3,844
Federated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766 · cited 1×
“There are research efforts to minimize the number of update rounds in federated learning to ensure communication efficiency [359], [363], [364].”
BackgroundMnasNet: Platform-Aware Neural Architecture Search for Mobile
Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00293 · Cited by 3,416 · cited 1×
“MNASNet [67] automatically generates DL models for mobile platforms.”
MethodDARTS: Differentiable architecture search
Liu, H., Simonyan, K., & Yang, Y. (2018) · arXiv · doi:10.48550/arxiv.1806.09055 · Cited by 3,248
Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks
Chen, Y.-H., Krishna, T., Emer, J. S., & Sze, V. (2017) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2016.2616357 · Cited by 3,247 · cited 5×
“RLC for activation compression, PE data gating, and zero input detection are used in [250].”
BackgroundMethodResultInfluentialFitNets: Hints for Thin Deep Nets
Romero, A., Ballas, N., Kahou, S. E., Chassang, A., Gatta, C., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.6550 · Cited by 2,859 · cited 2×
“FitNets [185] are student networks with increased depth and reduced width that make a generalized compact network.”
BackgroundMethodAn empirical comparison of supervised learning algorithms
Caruana, R., & Niculescu-Mizil, A. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143865 · Cited by 2,780 · cited 1×
“Supervised ML techniques widely used are DTs, RF, naive Bayes, SVM, KNNs, LR, and NNs, among many others [37].”
MethodLearning Efficient Convolutional Networks through Network Slimming
Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., & Zhang, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.298 · Cited by 2,761
A survey on semi-supervised learning
van Engelen, J. E., & Hoos, H. H. (2020) · Machine Learning · doi:10.1007/s10994-019-05855-6 · Cited by 2,666 · cited 2×
“Semisupervised techniques include expectation maximization with generative mixture models, transductive SVMs, graph-based methods, and so on [39], [40].”
BackgroundMethodMeasurement Reliability and Validity
(2000) · Research Methods in Applied Settings · doi:10.4324/9781410605337-29 · Cited by 2,541
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455 · cited 3×
“The EI of DL models can be categorized into three types [18]: 1) inference on the edge server; 2) inference on the edge device; and 3) collaborative inference on the edge device and edge server.”
BackgroundMethodPrivacy-preserving deep learning
Shokri, R., & Shmatikov, V. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813687 · Cited by 2,356 · cited 1×
“To mitigate the gradient optimization problem, selective SGD [362] has been proposed, which allows independent training on edge on local data and selective sharing of parameter updates to the central server.”
MethodArtificial intelligence and statistics
Yu, B., & Kumbier, K. (2018) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1700813 · Cited by 2,329
BinaryConnect: Training Deep Neural Networks with binary weights during propagations
Courbariaux, M., Bengio, Y., & David, J.-P. (2015) · arXiv · doi:10.48550/arxiv.1511.00363 · Cited by 2,180
Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1602.02830 · Cited by 2,106
1.1 Computing's energy problem (and what we can do about it)
Horowitz, M. (2014) · 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC) · doi:10.1109/isscc.2014.6757323 · Cited by 2,069
Buciluǎ, C., Caruana, R., & Niculescu-Mizil, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150464 · Cited by 2,068 · cited 1×
“The idea of KD was developed in [181] and generalized in [182].”
BackgroundZagoruyko, S., & Komodakis, N. (2016) · arXiv · doi:10.48550/arxiv.1612.03928 · Cited by 2,027
EIE: efficient inference engine on compressed deep neural network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., & Dally, W. J. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001163 · Cited by 1,937 · cited 4×
“EIE also stores the starting position of each column, which is required due to the variable length of compressed weights.”
BackgroundMethodInfluentialProgressive Neural Architecture Search
Liu, C., Zoph, B., Neumann, M., Shlens, J., Hua, W., Li, L.-J., Fei-Fei, L., Yuille, A., Huang, J., & Murphy, K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01246-5_2 · Cited by 1,833 · cited 1×
“Progressive NAS (PNAS) is reported in [66] for CNN architecture search.”
BackgroundTinyBERT: Distilling BERT for natural language understanding
Jiao, X., Yin, Y., Shang, L., Jiang, X., Chen, X., Li, L., Wang, F., & Liu, Q. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.372 · Cited by 1,786 · cited 1×
“For NLP on edge devices, a smaller transformer-based model has been designed using the KD technique named TinyBERT [188].”
MethodZhang, Y., Xiang, T., Hospedales, T. M., & Lu, H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00454 · Cited by 1,752 · cited 1×
“Various applications with impressive results have been developed for KD, such as mutual learning [190], teacher assistant [191], and self-learning [192].”
BackgroundDoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
Zhou, S., Wu, Y., Ni, Z., Zhou, X., Wen, H., & Zou, Y. (2016) · arXiv · doi:10.48550/arxiv.1606.06160 · Cited by 1,690 · cited 1×
“Quantized NNs (QNNs) [142], DoReFa-Net [143], and HWGQ-Net [144] are a few examples of DL models that use 2-bits and 1-bit precision for activations and weights, respectively.”
BackgroundHubara, I., Soudry, D., & Yaniv, R. E. (2016) · arXiv · doi:10.48550/arxiv.1602.02505 · Cited by 1,667
A State-of-the-Art Survey on Deep Learning Theory and Architectures
Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., Hasan, M., Van Essen, B. C., Awwal, A. A. S., & Asari, V. K. (2019) · Electronics · doi:10.3390/electronics8030292 · Cited by 1,620 · cited 3×
“There are available reviews of different DL techniques [5], DNNs on edge [2], [16], DL compression techniques for hardware acceleration [10], DL inference on FPGA-based platforms [17], edge intelligence [18], and edge computing [19].”
BackgroundPerformance evaluation of classification algorithms by k-fold and leave-one-out cross validation
Wong, T.-T. (2015) · Pattern Recognition · doi:10.1016/j.patcog.2015.03.009 · Cited by 1,562 · cited 2×
“Three popular resampling methods are random splitting [33], cross-validation [34], and bootstrapping.”
MethodRefaeilzadeh, P., Tang, L., & Liu, H. (2009) · Encyclopedia of Database Systems · doi:10.1007/978-0-387-39940-9_565 · Cited by 1,549
Deep Learning With Edge Computing: A Review
Chen, J., & Ran, X. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2921977 · Cited by 1,540 · cited 3×
“There are available reviews of different DL techniques [5], DNNs on edge [2], [16], DL compression techniques for hardware acceleration [10], DL inference on FPGA-based platforms [17], edge intelligence [18], and edge computing [19].”
BackgroundMethodDaDianNao: A Machine-Learning Supercomputer
Chen, Y., Luo, T., Liu, S., Zhang, S., He, L., Wang, J., Li, L., Chen, T., Xu, Z., Sun, N., et al. (2014) · 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1109/micro.2014.58 · Cited by 1,492 · cited 1×
“Tianjic [324] and Simba [325] are two spatial architectures that utilized the concept of distributed weights [326] and weight exchange among PEs [319].”
BackgroundLearning both Weights and Connections for Efficient Neural Networks
Han, S., Pool, J., Tran, J., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1506.02626 · Cited by 1,481 · cited 3×
“Studies demonstrate that these models are often overparameterized having significant redundancy [47], [48].”
BackgroundMethodAn Empirical Exploration of Recurrent Network Architectures
Józefowicz, R., Zaremba, W., & Sutskever, I. (2015) · International Conference on Machine Learning · Cited by 1,459
Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
Denton, R., Zaremba, W., Bruna, J., LeCun, Y., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1404.0736 · Cited by 1,414
A Survey on Deep Learning: Algorithms, Techniques, and Applications
Pouyanfar, S., Sadiq, S., Yan, Y., Tian, H., Tao, Y., Reyes, M. P., Shyu, M.-L., Chen, S.-C., & Iyengar, S. S. (2019) · ACM Computing Surveys · doi:10.1145/3234150 · Cited by 1,403
Deep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · arXiv · doi:10.48550/arxiv.1502.02551 · Cited by 1,364 · cited 1×
“Instead of using single-precision 32-bit floating point (FP32), the calculations and implementation of DL algorithms become much easier in 16-bit floating-point [112], 8-bit floating-point [113], and fixed-point representations [114], [115], [116], [117].”
MethodBranchyNet: Fast inference via early exiting from deep neural networks
Teerapittayanon, S., McDanel, B., & Kung, H. T. (2016) · 2016 23rd International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr.2016.7900006 · Cited by 1,326 · cited 1×
“BranchyNet [212] is an implementation of an early exiting technique that adds multiple exit points to the base network.”
BackgroundPruning Convolutional Neural Networks for Resource Efficient Inference
Molchanov, P. A., Tyree, S., Karras, T., Aila, T., & Kautz, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.06440 · Cited by 1,239
Chi, P., Li, S., Xu, C., Zhang, T., Zhao, J., Liu, Y., Wang, Y., & Xie, Y. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001140 · Cited by 1,231
GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism
Huang, Y., Cheng, Y., Bapna, A., Firat, O., Chen, M. X., Chen, D., Lee, H., Ngiam, J., Le, Q. V., Wu, Y., et al. (2018) · arXiv · doi:10.48550/arxiv.1811.06965 · Cited by 1,180
Micikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M., Kuchaiev, O., Venkatesh, G., et al. (2017) · arXiv · doi:10.48550/arxiv.1710.03740 · Cited by 1,179 · cited 1×
“Instead of using single-precision 32-bit floating point (FP32), the calculations and implementation of DL algorithms become much easier in 16-bit floating-point [112], 8-bit floating-point [113], and fixed-point representations [114], [115], [116], [117].”
MethodA Survey of Deep Active Learning
Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Gupta, B. B., Chen, X., & Wang, X. (2022) · ACM Computing Surveys · doi:10.1145/3472291 · Cited by 1,169 · cited 1×
“Active learning [421] can deal with the unlabeled data problem up to a certain extent.”
BackgroundSpeeding up Convolutional Neural Networks with Low Rank Expansions
Jaderberg, M., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.88 · Cited by 1,164
Equivalent-accuracy accelerated neural-network training using analogue memory
Ambrogio, S., Narayanan, P., Tsai, H., Shelby, R. M., Boybat, I., di Nolfo, C., Sidler, S., Giordano, M., Bodini, M., Farinha, N. C. P., et al. (2018) · Nature · doi:10.1038/s41586-018-0180-5 · Cited by 1,151 · cited 1×
“For instance, using RRAM [336], PCRAM [337], and memristors, the MVM operation can be performed fast in the analog domain within one clock cycle.”
MethodEyeriss v2: A Flexible Accelerator for Emerging Deep Neural Networks on Mobile Devices
Chen, Y.-H., Yang, T.-J., Emer, J. S., & Sze, V. (2019) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2019.2910232 · Cited by 1,106 · cited 4×
“Eyeriss v2 [237] uses the CSC format to store compressed parameters and activations.”
MethodInfluentialPower to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084 · cited 1×
“Interactive ML (iML) involves users, providing information to the learning process and monitoring the output in subsequent iterations [356], [357].”
BackgroundTowards artificial general intelligence with hybrid Tianjic chip architecture
Pei, J., Deng, L., Song, S., Zhao, M., Zhang, Y., Wu, S., Wang, G., Zou, Z., Wu, Z., He, W., et al. (2019) · Nature · doi:10.1038/s41586-019-1424-8 · Cited by 1,082 · cited 1×
“Tianjic [324] and Simba [325] are two spatial architectures that utilized the concept of distributed weights [326] and weight exchange among PEs [319].”
BackgroundModel Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey
Deng, B. L., Li, G., Han, S., Shi, L., & Xie, Y. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2020.2976475 · Cited by 996 · cited 2×
“There are available reviews of different DL techniques [5], DNNs on edge [2], [16], DL compression techniques for hardware acceleration [10], DL inference on FPGA-based platforms [17], edge intelligence [18], and edge computing [19].”
BackgroundMethodCompressing Neural Networks with the Hashing Trick
Chen, W., Wilson, J. T., Tyree, S., Weinberger, K. Q., & Chen, Y. (2015) · arXiv · doi:10.48550/arxiv.1504.04788 · Cited by 974
A review of medical image data augmentation techniques for deep learning applications
Chlap, P., Min, H., Vandenberg, N., Dowling, J., Holloway, L., & Haworth, A. (2021) · Journal of Medical Imaging and Radiation Oncology · doi:10.1111/1754-9485.13261 · Cited by 956 · cited 1×
“Data augmentation [410], [411] and representation learning could [412] be two promising future research directions to mitigate the data inconsistency problem at the edge.”
BackgroundCompressing Deep Convolutional Networks using Vector Quantization
Gong, Y., Liu, L., Yang, M., & Bourdev, L. D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6115 · Cited by 925 · cited 1×
“The number of parameters in CNN can be reduced by using vector quantization [163].”
MethodCompression of Deep Convolutional Neural Networks for Fast and Low Power Mobile ApplicationsPreprint
Kim, Y., Park, E., Yoo, S., Choi, T., Lu, Y., & Shin, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06530 · Cited by 920
Fast Algorithms for Convolutional Neural Networks
Lavin, A., & Gray, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.435 · Cited by 915 · cited 1×
“The number of floating-point multiplications and integer arithmetic operations can be reduced by a factor of 2.25 × [79] and 3.13 × [80], respectively, using the Winograd algorithm.”
MethodInfluentialFace Spoof Detection With Image Distortion Analysis
Wen, D., Han, H., & Jain, A. K. (2015) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2015.2400395 · Cited by 905 · cited 1×
“For instance, face recognition is being actively used in door unlocking, mobile banking [383], device unlocking [384], and so on.”
BackgroundCondenseNet: An Efficient DenseNet Using Learned Group Convolutions
Huang, G., Liu, S., Maaten, L. V. D., & Weinberger, K. Q. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00291 · Cited by 904 · cited 3×
“CondenseNet with 50% fewer parameters can achieve ShuffleNet-like accuracy.”
MethodInfluentialBlockchained On-Device Federated Learning
Kim, H., Park, J., Bennis, M., & Kim, S.-L. (2020) · IEEE Communications Letters · doi:10.1109/lcomm.2019.2921755 · Cited by 887 · cited 1×
“There are research efforts to minimize the number of update rounds in federated learning to ensure communication efficiency [359], [363], [364].”
BackgroundBlockchained On-Device Federated Learning
Kim, H., Park, J., Bennis, M., & Kim, S.-L. (2018) · arXiv · doi:10.48550/arxiv.1808.03949 · Cited by 882
Zhu, C., Han, S., Mao, H., & Dally, W. J. (2016) · arXiv · doi:10.48550/arxiv.1612.01064 · Cited by 865 · cited 1×
“In ternary weight nets (TWNs) [145] and trained ternary quantization (TTQ) [146], ternary weights instead of binary weights are used with 32-bit floating-point representation for activations that exhibit as low as 0.6% accuracy degradation.”
MethodInfluentialZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers
Yao, Z., Aminabadi, R. Y., Zhang, M., Wu, X., Li, C., & He, Y. (2022) · neural information processing systems · doi:10.52202/068431-1970 · Cited by 853
Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices
Teerapittayanon, S., McDanel, B., & Kung, H. T. (2017) · 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2017.226 · Cited by 843 · cited 2×
“DL models are mapped across different computing hierarchies: Distributed DNNs (DDNNs) [214].”
BackgroundSak, H., Senior, A., & Beaufays, F. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.1128 · Cited by 838 · cited 1×
“For instance, the dimen-sion of the hidden state can be minimized using the linear recurrent projection layer in LSTM to improve the parameter efficiency [89].”
MethodCheng, Y., Wang, D., Zhou, P., & Zhang, T. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2765695 · Cited by 817 · cited 1×
“Four prominent compression techniques, such as parameter pruning, low-rank approximation, compact CNN design, and knowledge distillation (KD), are summarized in [6].”
MethodNeurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge
Kang, Y., Hauswald, J., Gao, C., Rovinski, A., Mudge, T., Mars, J., & Tang, L. (2017) · ACM SIGARCH Computer Architecture News · doi:10.1145/3093337.3037698 · Cited by 806
To prune, or not to prune: exploring the efficacy of pruning for model compression
Zhu, M., & Gupta, S. (2017) · arXiv · doi:10.48550/arxiv.1710.01878 · Cited by 805 · cited 15×
“Therefore, in GRU, the input and forget gates are coupled into the update gate resulting in a simplification of LSTM.”
BackgroundInfluentialElsken, T., Metzen, J. H., & Hutter, F. (2019) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-030-05318-5_3 · Cited by 785
Convolutional networks for fast, energy-efficient neuromorphic computing
Esser, S. K., Merolla, P. A., Arthur, J. V., Cassidy, A. S., Appuswamy, R., Andreopoulos, A., Berg, D. J., McKinstry, J. L., Melano, T., Barch, D. R., et al. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1604850113 · Cited by 781 · cited 1×
“TrueNorth chip [147] also implemented the ternary weights and binary activations.”
MethodMobileFaceNets: Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devices
Chen, S., Liu, Y., Gao, X., & Han, Z. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-97909-0_46 · Cited by 771 · cited 1×
“In [385], a CNN named Mobile-FaceNets is presented that uses a global depthwise convolution filter that can perform face recognition having less than 1M parameters.”
MethodWang, L., & Yoon, K.-J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3055564 · Cited by 752 · cited 1×
“KD is a compression method that mimics the behavior of a large DL model into a lightweight model by transferring learned knowledge [178].”
MethodLi, F., Liu, B., Wang, X., Zhang, B., & Yan, J. (2016) · arXiv · doi:10.48550/arxiv.1605.04711 · Cited by 732 · cited 1×
“In ternary weight nets (TWNs) [145] and trained ternary quantization (TTQ) [146], ternary weights instead of binary weights are used with 32-bit floatingpoint representation for activations that exhibit as low as 0.”
MethodDesigning Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning
Yang, T.-J., Chen, Y.-H., & Sze, V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.643 · Cited by 732 · cited 1×
“3) Energy-Aware Pruning: This technique is developed in [108] that prunes weights depending on an estimated energy.”
MethodTime Series Data Augmentation for Deep Learning: A Survey
Wen, Q., Sun, L., Yang, F., Song, X., Gao, J., Wang, X., & Xu, H. (2020) · arXiv · doi:10.48550/arxiv.2002.12478 · Cited by 727
Time Series Data Augmentation for Deep Learning: A Survey
Wen, Q., Sun, L., Yang, F., Song, X., Gao, J., Wang, X., & Xu, H. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/631 · Cited by 727
(2011) · Cited by 713
Reinforcement Learning: State-of-the-Art
(2012) · Adaptation, Learning, and Optimization · doi:10.1007/978-3-642-27645-3 · Cited by 691
Fixed Point Quantization of Deep Convolutional Networks
Lin, D. D., Talathi, S. S., & Annapureddy, V. S. (2015) · arXiv · doi:10.48550/arxiv.1511.06393 · Cited by 684
DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing
Yao, S., Hu, S., Zhao, Y., Zhang, A., & Abdelzaher, T. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052577 · Cited by 667 · cited 3×
“In DeepSense, most computation-intensive tasks are performed by GPU, and only the padding and memory allocation are performed by the CPU.”
MethodSkipNet: Learning Dynamic Routing in Convolutional Networks
Wang, X., Yu, F., Dou, Z.-Y., Darrell, T., & Gonzalez, J. E. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_25 · Cited by 625 · cited 1×
“Instead of adding branches, skippable tiny NNs known as gates are attached to the network architecture of SkipNet [217].”
BackgroundCnvlutin: ineffectual-neuron-free deep neural network computing
Albericio, J., Judd, P., Hetherington, T., Aamodt, T., Jerger, N. E., & Moshovos, A. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001138 · Cited by 614
Cambricon-X: An accelerator for sparse neural networks
Zhang, S., Du, Z., Zhang, L., Lan, H., Liu, S., Li, L., Guo, Q., Chen, T., & Chen, Y. (2016) · 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2016.7783723 · Cited by 603 · cited 3×
“An example of static sparsity involves the locations of zero weight being known beforehand for inference in Cambricon-X [233].”
BackgroundMethodESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA
Han, S., Kang, J., Mao, H., Hu, Y., Li, X., Li, Y., Xie, D., Luo, H., Yao, S., Wang, Y., et al. (2017) · Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/3020078.3021745 · Cited by 591 · cited 6×
“The minimal gated unit (MGU) integrates the reset gate and update gate of the GRU [87].”
BackgroundMethodInfluentialNetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications
Yang, T.-J., Howard, A., Chen, B., Zhang, X., Go, A., Sandler, M., Sze, V., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_18 · Cited by 572 · cited 2×
“MobileNet-v3 [64] released in large and small versions uses NAS and NetAdapt [65] algorithms.”
MethodRevisiting Knowledge Distillation via Label Smoothing Regularization
Yuan, L., Tay, F. E., Li, G., Wang, T., & Feng, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00396 · Cited by 572 · cited 1×
“Various applications with impressive results have been developed for KD, such as mutual learning [190], teacher assistant [191], and self-learning [192].”
BackgroundLiu, Z., Wu, B., Luo, W., Yang, X., Liu, W., & Cheng, K.-T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_44 · Cited by 565 · cited 1×
“Bi-Real Net [139] is proposed to minimize the accuracy degradation of 1-bit CNN incorporating real activations after a 1-bit convolution operation.”
MethodAngel-Eye: A Complete Design Flow for Mapping CNN Onto Embedded FPGA
Guo, K., Sui, L., Qiu, J., Yu, J., Wang, J., Yao, S., Han, S., Wang, Y., & Yang, H. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2017.2705069 · Cited by 562 · cited 11×
“It is often expressed as giga operations per watt (GOPS/W) or tera operations per watt (TOPS/W), illustrating how many operations the inference engine can perform per watt of power consumption.”
BackgroundMethodInfluentialBit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Network
Sharma, H., Park, J., Suda, N., Lai, L., Chau, B., Kim, J. K., Chandra, V., & Esmaeilzadeh, H. (2018) · 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2018.00069 · Cited by 543 · cited 1×
“The PEs in the bit fusion [126] architecture are combined differently to implement spatial flexibility in bit widths.”
BackgroundArithmetic Complexity of Computations
Winograd, S. (1980) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611970364 · Cited by 535
Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons
Heo, B., Lee, M., Yun, S., & Choi, J. Y. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013779 · Cited by 522 · cited 1×
“The student model can also use other information from the teacher model, such as activations [183], neurons [184], or features of intermediate layers [185] as knowledge to improve the performance.”
MethodA Configurable Cloud-Scale DNN Processor for Real-Time AI
Fowers, J., Ovtcharov, K., Papamichael, M., Massengill, T., Liu, M., Lo, D., Alkalay, S., Haselman, M., Adams, L., Ghandi, M., et al. (2018) · 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2018.00012 · Cited by 520 · cited 1×
“The neural processing unit (NPU) is presented in [329] for the project Brainwave to provide real-time AI.”
BackgroundDeepThings: Distributed Adaptive Deep Learning Inference on Resource-Constrained IoT Edge Clusters
Zhao, Z., Barijough, K. M., & Gerstlauer, A. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2018.2858384 · Cited by 502 · cited 1×
“For example, in DeepThings [375], feature maps are partitioned into nonoverlapping tiles using fused tile partitioning (FTP) to assign them to different edge devices.”
MethodDeep Learning with Low Precision by Half-Wave Gaussian Quantization
Cai, Z., He, X., Sun, J., & Vasconcelos, N. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.574 · Cited by 498 · cited 1×
“Quantized NNs (QNNs) [142], DoReFa-Net [143], and HWGQ-Net [144] are a few examples of DL models that use 2-bits and 1-bit precision for activations and weights, respectively.”
BackgroundFPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review
Shawahna, A., Sait, S. M., & El-Maleh, A. (2019) · IEEE Access · doi:10.1109/access.2018.2890150 · Cited by 496 · cited 2×
“Due to the increased demand for EI, small and high-density FPGAs are now being considered [11], [315], [316], [317].”
BackgroundMethodBinaryNet: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1Preprint
Courbariaux, M., & Bengio, Y. (2016) · arXiv (Cornell University) · Cited by 484
Moons, B., Uytterhoeven, R., Dehaene, W., & Verhelst, M. (2017) · 2017 IEEE International Solid-State Circuits Conference (ISSCC) · doi:10.1109/isscc.2017.7870353 · Cited by 479
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7_14 · Cited by 477
Horovod: fast and easy distributed deep learning in TensorFlow
Sergeev, A., & Balso, M. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05799 · Cited by 472
Can FPGAs Beat GPUs in Accelerating Next-Generation Deep Neural Networks?
Nurvitadhi, E., Venkatesh, G., Sim, J., Marr, D., Huang, R., Ong Gee Hock, J., Liew, Y. T., Srivatsan, K., Moss, D., Subhaschandra, S., et al. (2017) · Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/3020078.3021740 · Cited by 467 · cited 1×
“To minimize on-chip memory requirement, the weight matrix can be stored in a dense format [110].”
BackgroundDeep Learning for Smart Industry: Efficient Manufacture Inspection System With Fog Computing
Li, L., Ota, K., & Dong, M. (2018) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2842821 · Cited by 466 · cited 2×
“In DeepIns, data are collected by the edge sensors and have two exit points: one at the edge server and the other one at the cloud.”
MethodDeepX: A Software Accelerator for Low-Power Deep Learning Inference on Mobile Devices
Lane, N. D., Bhattacharya, S., Georgiev, P., Forlivesi, C., Jiao, L., Qendro, L., & Kawsar, F. (2016) · 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1109/ipsn.2016.7460664 · Cited by 450 · cited 1×
“DeepX [284] uses SVD to compress layers to get compact network architecture.”
MethodFINN- R: An End-to-End Deep-Learning Framework for Fast Exploration of Quantized Neural Networks
Blott, M., Preußer, T. B., Fraser, N. J., Gambardella, G., O’brien, K., Umuroglu, Y., Leeser, M., & Vissers, K. (2018) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3242897 · Cited by 446 · cited 1×
“Xilinx provides quantization support through the FINN-R framework for inference realization on FPGA [413].”
MethodThe State of Sparsity in Deep Neural Networks
Gale, T., Elsen, E., & Hooker, S. (2019) · arXiv · doi:10.48550/arxiv.1902.09574 · Cited by 440 · cited 1×
“NLP models, such as Transformer [45] and BERT [224], have around 80% [225] and 93% [226] weight sparsity.”
MethodIncremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
Zhou, A., Yao, A., Guo, Y., Xu, L., & Chen, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.03044 · Cited by 439
A survey of FPGA-based accelerators for convolutional neural networks
Mittal, S. (2020) · Neural Computing and Applications · doi:10.1007/s00521-018-3761-1 · Cited by 429 · cited 1×
“FPGA-based acceleration of DL tensor computation [11] and execution of compact networks on FPGA [12] are also studied.”
MethodSpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning
Wang, H., Zhang, Z., & Han, S. (2021) · 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca51647.2021.00018 · Cited by 427 · cited 2×
“Lossless reduction of computation and memory access of 3.8 × and 1.1 × is reported in the NLP model SpAtten [227] by pruning unimportant tokens and heads.”
BackgroundInfluentialSCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks
Parashar, A., Rhu, M., Mukkara, A., Puglielli, A., Venkatesan, R., Khailany, B., Emer, J., Keckler, S. W., & Dally, W. J. (2017) · ACM SIGARCH Computer Architecture News · doi:10.1145/3140659.3080254 · Cited by 413
Edgenext: Efficiently amalgamated CNN-transformer architecture for mobile vision applications
Maaz, M., Shaker, A., Cholakkal, H., Khan, S., Zamir, S. W., Anwer, R. M., & Shahbaz Khan, F. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-25082-8_1 · Cited by 407
Learning Structured Sparsity in Deep Neural Networks
Wen, W. X., Wu, C., Wang, Y., Chen, Y., & Li, H. H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.03665 · Cited by 402
Lifelong Machine Learning, Second Edition
Chen, Z., & Liu, B. (2018) · Synthesis lectures on artificial intelligence and machine learning · doi:10.2200/s00832ed1v01y201802aim037 · Cited by 394
Cunningham, P., Cord, M., & Delany, S. J. (2008) · Cognitive Technologies · doi:10.1007/978-3-540-75171-7_2 · Cited by 387
Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy
Li, E., Zhou, Z., & Chen, X. (2018) · Proceedings of the 2018 Workshop on Mobile Edge Communications · doi:10.1145/3229556.3229562 · Cited by 385
NeuFlow: A runtime reconfigurable dataflow processor for vision
Farabet, C., Martini, B., Corda, B., Akselrod, P., Culurciello, E., & LeCun, Y. (2011) · CVPR 2011 WORKSHOPS · doi:10.1109/cvprw.2011.5981829 · Cited by 369 · cited 2×
“The neuFlow [288] is a general-purpose vision processor for real-time object detection, recognition, and localization.”
BackgroundMethodNeural Cache: Bit-Serial In-Cache Acceleration of Deep Neural Networks
Eckert, C., Wang, X., Wang, J., Subramaniyan, A., Iyer, R., Sylvester, D., Blaaauw, D., & Das, R. (2018) · 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2018.00040 · Cited by 369 · cited 1×
“The traditional SRAM [338], DRAM [339], and Flash [340] memories also can be modified to realize the PIM implementation of DNN algorithms.”
MethodHan, S., Shen, H., Philipose, M., Agarwal, S., Wolman, A., & Krishnamurthy, A. (2016) · Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services · doi:10.1145/2906388.2906396 · Cited by 367
NoScope: optimizing neural network queries over video at scale
Kang, D., Emmons, J., Abuzaid, F., Bailis, P., & Zaharia, M. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3137628.3137664 · Cited by 367
From high-level deep neural models to FPGAs
Sharma, H., Park, J., Mahajan, D., Amaro, E., Kim, J. K., Shao, C., Mishra, A., & Esmaeilzadeh, H. (2016) · 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2016.7783720 · Cited by 365 · cited 1×
“Frameworks such as DNNWeaver [414], DNNBuilder [415], T2S-Tensor [416], and HeteroCL [417] have been developed recently for FPGA, ASIC, and other DL hardware accelerators for efficient EI.”
MethodMinimal gated unit for recurrent neural networks
Zhou, G.-B., Wu, J., Zhang, C.-L., & Zhou, Z.-H. (2016) · International Journal of Automation and Computing · doi:10.1007/s11633-016-1006-2 · Cited by 357 · cited 1×
“The minimal gated unit (MGU) integrates the reset gate and update gate of the GRU [87].”
BackgroundSqueezeNext: Hardware-Aware Neural Network Design
Gholami, A., Kwon, K., Wu, B., Tai, Z., Yue, X., Jin, P., Zhao, S., & Keutzer, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00215 · Cited by 350 · cited 1×
“Instead of using a fire module, a two-stage bottleneck module is used in SqueezeNext [74].”
MethodDNNBuilder: an automated tool for building high-performance DNN hardware accelerators for FPGAs
Zhang, X., Wang, J., Zhu, C., Lin, Y., Xiong, J., Hwu, W.-M., & Chen, D. (2018) · Proceedings of the International Conference on Computer-Aided Design · doi:10.1145/3240765.3240801 · Cited by 345 · cited 2×
“Frameworks such as DNNWeaver [414], DNNBuilder [415], T2S-Tensor [416], and HeteroCL [417] have been developed recently for FPGA, ASIC, and other DL hardware accelerators for efficient EI.”
MethodNeural Networks with Few MultiplicationsPreprint
Lin, Z., Courbariaux, M., Memisevic, R., & Bengio, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.03009 · Cited by 344
MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks
Gordon, A., Eban, E., Nachum, O., Chen, B., Wu, H., Yang, T.-J., & Choi, E. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00171 · Cited by 341
Exploring Sparsity in Recurrent Neural NetworksPreprint
Narang, S., Elsen, E., Diamos, G., & Sengupta, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.05119 · Cited by 337
Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural NetworksPreprint
You, Z., Yan, K., Ye, J., Ma, M., & Wang, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.08174 · Cited by 336
Hello Edge: Keyword Spotting on Microcontrollers
Zhang, Y., Suda, N., Lai, L., & Chandra, V. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.07128 · Cited by 334 · cited 1×
“Hello Edge [334] implements various DL models on MCU for keyword spotting for speech-based smart devices.”
MethodFlexFlow: A Flexible Dataflow Accelerator Architecture for Convolutional Neural Networks
Lu, W., Yan, G., Li, J., Gong, S., Han, Y., & Li, X. (2017) · 2017 IEEE International Symposium on High Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2017.29 · Cited by 333 · cited 1×
“FlexFlow [311] is a scalable accelerator that exploits the parallelism of computing platforms and CNN architectures to support a flexible data flow and data reuse.”
BackgroundMoDNN: Local distributed mobile computing system for Deep Neural Network
Mao, J., Chen, X., Nixon, K. W., Krieger, C., & Chen, Y. (2017) · Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017 · doi:10.23919/date.2017.7927211 · Cited by 331 · cited 1×
“MoDNN [371] is a horizontally distributed platform for DL inference in resource-constrained edge devices.”
BackgroundDeepMon: Mobile GPU-based Deep Learning Framework for Continuous Vision Applications
Huynh, L. N., Lee, Y., & Balan, R. K. (2017) · Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3081333.3081360 · Cited by 329 · cited 1×
“DeepMon [162] uses a combination of quantization and caching to speed up the execution by reducing redundant computations of video frames.”
MethodLearning Activation Functions to Improve Deep Neural Networks
Agostinelli, F., Hoffman, M. D., Sadowski, P., & Baldi, P. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6830 · Cited by 326 · cited 1×
“Some of the common nonlinear functions are rectified nonlinear unit (ReLU), leaky ReLU, sigmoid, Tanh, and ELU [44].”
BackgroundPhased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences
Neil, D., Pfeiffer, M., & Liu, S. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.48550/arxiv.1610.09513 · Cited by 319
Zeng, L., Chen, X., Zhou, Z., Yang, L., & Zhang, J. (2021) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2020.3042320 · Cited by 313 · cited 1×
“CoEdge [376] reduces the redundant calculation by nonoverlapping partitioning with an added data transmission cost.”
BackgroundDeepEar: robust smartphone audio sensing in unconstrained acoustic environments using deep learning
Lane, N. D., Georgiev, P., & Qendro, L. (2015) · Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2750858.2804262 · Cited by 310 · cited 1×
“DeepEar [397] is DL-based audio sensing using a smartphone that used stacked RBM.”
MethodLee, J., Kim, C., Kang, S., Shin, D., Kim, S., & Yoo, H.-J. (2018) · 2018 IEEE International Solid - State Circuits Conference - (ISSCC) · doi:10.1109/isscc.2018.8310262 · Cited by 303 · cited 3×
“UNPU [124] also implements bit-serial computation with 16-bit fixed-point representation for activations and 1–16-bit variable bit width for weights.”
MethodInfluentialLike What You Like: Knowledge Distill via Neuron Selectivity Transfer
Huang, Z., & Wang, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.01219 · Cited by 300
FP-BNN: Binarized neural network on FPGA
Liang, S., Yin, S., Liu, L., Luk, W., & Wei, S. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.09.046 · Cited by 298
Aimar, A., Mostafa, H., Calabrese, E., Rios-Navarro, A., Tapiador-Morales, R., Lungu, I.-A., Milde, M. B., Corradi, F., Linares-Barranco, A., Liu, S.-C., et al. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2852335 · Cited by 292 · cited 1×
“NullHop [248] utilizes the CIS scheme for the weights and implements zero skipping for null activations.”
MethodShiDianNao: shifting vision processing closer to the sensor
Du, Z., Fasthuber, R., Chen, T., Ienne, P., Li, L., Luo, T., Feng, X., Chen, Y., & Temam, O. (2015) · ACM SIGARCH Computer Architecture News · doi:10.1145/2872887.2750389 · Cited by 290
CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs
Oliveira, A., Costa, D., Moreira, G., Martins, J., & Pinto, S. (2018) · arXiv (Cornell University) · doi:10.4230/oasics.ng-res.2025.4 · Cited by 288
Hardware-oriented Approximation of Convolutional Neural Networks
Gysel, P., Motamedi, M., & Ghiasi, S. (2016) · arXiv · doi:10.48550/arxiv.1604.03168 · Cited by 287
Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
Lebedev, V., Ganin, Y., Rakhuba, M., Oseledets, I. V., & Lempitsky, V. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6553 · Cited by 283
Multimodal Deep Learning for Activity and Context Recognition
Radu, V., Tong, C., Bhattacharya, S., Lane, N. D., Mascolo, C., Marina, M. K., & Kawsar, F. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3161174 · Cited by 281 · cited 1×
“Similarly, researchers also investigated DL training in wearable [347] and embedded [348] devices.”
BackgroundBackpropagation for energy-efficient neuromorphic computing
Esser, S. K., Appuswamy, R., Merolla, P. A., Arthur, J. V., & Modha, D. S. (2015) · neural information processing systems · Cited by 281
SparTen: A Sparse Tensor Accelerator for Convolutional Neural Networks
Gondimalla, A., Chesnut, N., Thottethodi, M., & Vijaykumar, T. N. (2019) · Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1145/3352460.3358291 · Cited by 278 · cited 1×
“Sparten’s [246] architecture is similar to SCNN with an improved multiplication reducing the overhead.”
MethodEdge Intelligence: Empowering Intelligence to the Edge of Network
Xu, D., Li, T., Li, Y., Su, X., Tarkoma, S., Jiang, T., Crowcroft, J., & Hui, P. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3119950 · Cited by 272 · cited 2×
“3 illustrates the four essential elements of edge-AI [20]: 1) edge caching; 2) edge training; 3) EI; and 4) edge offloading.”
BackgroundMesh-TensorFlow: Deep Learning for SupercomputersPreprint
Shazeer, N., Cheng, Y., Parmar, N., Tran, D., Vaswani, A., Koanantakool, P., Hawkins, P., Lee, H., Hong, M., Young, C., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.02084 · Cited by 270
Kalchbrenner, N., Danihelka, I., & Graves, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.01526 · Cited by 267 · cited 1×
“The concept of weight sharing is utilized in the grid LSTM [93] for multidimensional data processing.”
MethodProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
Cai, H., Zhu, L., & Han, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00332 · Cited by 267
ThiNet: Pruning CNN Filters for a Thinner Net
Luo, J.-H., Zhang, H., Zhou, H.-Y., Xie, C.-W., Wu, J., & Lin, W. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858232 · Cited by 262
Yang, Q., Liu, Y., Cheng, Y., Kang, Y., Chen, T., & Yu, H. (2019) · Synthesis lectures on artificial intelligence and machine learning · doi:10.2200/s00960ed2v01y201910aim043 · Cited by 261 · cited 1×
“A shared model can learn collaboratively from local training data instead of requiring to accumulate the data to a central cloud server [360].”
BackgroundAxNN: energy-efficient neuromorphic systems using approximate computing
Venkataramani, S., Ranjan, A., Roy, K., & Raghunathan, A. (2014) · Proceedings of the 2014 international symposium on Low power electronics and design · doi:10.1145/2627369.2627613 · Cited by 261 · cited 1×
“An approximate NN (AxNN) can be obtained using a quality-configurable technique exploiting neuromorphic processing [174].”
MethodMCUNet: Tiny Deep Learning on IoT Devices
Lin, J., Chen, W.-M., Lin, Y., Cohn, J., Gan, C., & Han, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.10319 · Cited by 259
Adaptive Neural Networks for Efficient Inference
Bolukbasi, T., Wang, J., Dekel, O., & Saligrama, V. (2017) · arXiv · doi:10.48550/arxiv.1702.07811 · Cited by 251 · cited 1×
“In [213], a regularization has been added to the latency term to identify whether the current input sample is to be passed to the next layer or not.”
BackgroundEvaluating Fast Algorithms for Convolutional Neural Networks on FPGAs
Lu, L., Liang, Y., Xiao, Q., & Yan, S. (2017) · 2017 IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) · doi:10.1109/fccm.2017.64 · Cited by 248
AI Benchmark: All About Deep Learning on Smartphones in 2019
Ignatov, A., Timofte, R., Kulik, A., Yang, S., Wang, K., Baum, F., Wu, M., Xu, L., & Van Gool, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00447 · Cited by 247 · cited 1×
“Benchmarking quantized DL models to accelerate on smartphones is presented in [14].”
MethodRay: A Distributed Framework for Emerging AI Applications
Moritz, P., Nishihara, R., Wang, S. F., Tumanov, A., Liaw, R., Liang, E., Elibol, M., Yang, Z., Paul, W., Jordan, M. I., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.05889 · Cited by 247
Fixed point optimization of deep convolutional neural networks for object recognition
Anwar, S., Hwang, K., & Sung, W. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178146 · Cited by 239 · cited 1×
“Instead of using single-precision 32-bit floating point (FP32), the calculations and implementation of DL algorithms become much easier in 16-bit floating-point [112], 8-bit floating-point [113], and fixed-point representations [114], [115], [116], [117].”
MethodBhattacharya, S., & Lane, N. D. (2016) · Proceedings of the 14th ACM Conference on Embedded Networked Sensor Systems CD-ROM · doi:10.1145/2994551.2994564 · Cited by 236
NeST: A Neural Network Synthesis Tool Based on a Grow-and-Prune Paradigm
Dai, X., Yin, H., & Jha, N. K. (2019) · IEEE Transactions on Computers · doi:10.1109/tc.2019.2914438 · Cited by 233 · cited 2×
“For example, in [95], in each iteration, a small percentage of the least important weights based on a predefined threshold are removed from every layer, and the models are retrained to evaluate the performance.”
BackgroundLane, N. D., Bhattacharya, S., Georgiev, P., Forlivesi, C., & Kawsar, F. (2015) · Proceedings of the 2015 International Workshop on Internet of Things towards Applications · doi:10.1145/2820975.2820980 · Cited by 228 · cited 1×
“Similarly, researchers also investigated DL training in wearable [347] and embedded [348] devices.”
BackgroundHAT: Hardware-Aware Transformers for Efficient Natural Language Processing
Wang, H., Wu, Z., Liu, Z., Cai, H., Zhu, L., Gan, C., & Han, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.686 · Cited by 227 · cited 1×
“Hardware-aware transformer designs using NAS techniques for edge-compatible and low-latency inference are presented in [68].”
MethodSqueezing Deep Learning into Mobile and Embedded Devices
Lane, N. D., Bhattacharya, S., Mathur, A., Georgiev, P., Forlivesi, C., & Kawsar, F. (2017) · IEEE Pervasive Computing · doi:10.1109/mprv.2017.2940968 · Cited by 224 · cited 1×
“For instance, DL models are trained in a smartphone for activity recognition and audio signal sensing in [346].”
MethodC-LSTM: Enabling Efficient LSTM using Structured Compression Techniques on FPGAs
Wang, S., Li, Z., Ding, C., Yuan, B., Qiu, Q., Wang, Y., & Liang, Y. (2018) · Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/3174243.3174253 · Cited by 220
Exploring the Regularity of Sparse Structure in Convolutional Neural Networks
Mao, H., Han, S., Pool, J., Li, W., Liu, X., Wang, Y., & Dally, W. J. (2017) · arXiv · doi:10.48550/arxiv.1705.08922 · Cited by 219 · cited 1×
“The group of weights can be neighboring weights, part of a filter, or the whole filter [99].”
BackgroundMAERI: Enabling Flexible Dataflow Mapping over DNN Accelerators via Reconfigurable Interconnects
Kwon, H., Samajdar, A., & Krishna, T. (2018) · ACM SIGPLAN Notices · doi:10.1145/3296957.3173176 · Cited by 217
Research works printed but not linked: 30
- Wendelken, 2022, Worldwide and U.S. IoT Cellular Connections Forecast, 2021–2025
- Gupta, ProtoNN: Compressed and accurate KNN for resource-scarce devices, 1331
- Dennis, 2020, EdgeML: Machine learning for resource-constrained edge devices
- Kumar, Resource-efficient machine learning in 2 KB RAM for the Internet of Things, 70, 1935
- 2022, AutoML
- 2022, AutoKeras
- Ryu, BitBlade: Area and energy-efficient precision-scalable neural network accelerator with bitwise summation, 1
- Hojabr, SkippyNN: An embedded stochastic-computing accelerator for convolutional neural networks, 1
- 2022, Jetson TX2 Module
- 2022, Intel® Edison Development Platform
- 2022, Open Neural Network Exchange
- 2022, Deep Learning HDL Toolbox
- 2022, X-CUBE-AI, AI Expansion Pack for STM32CubeMX
- 2022, DNNDK User Guide
- 2022, NVIDIA TensorRT
- 2022, CEVA Deep Neural Network (CDNN)
- 2021, Qualcomm Neural Processing SDK for AI
- 2022, Stratus High-Level Synthesis
- 2022, Deploy Machine Learning Models on Mobile and IoT Devices
- Rasley, DeepSpeed, 3505
- Baines, 2021, FairScale: A general purpose modular PyTorch library for high performance and large scale training
- Dai, BigDL, 50
- 2022, Tensilica DNA Processor Family for on-Device AI
- 2022, DesignWare ARC EV Processors for Embedded Vision
- 2022, AI for the Edge
- 2022, Products, Helping You Bring Local AI to Applications From Prototype to Production
- Shacham, 2022, Pixel Visual Core: Image Processing and Machine Learning on Pixel 2
- Yang, Sparse ReRAM engine: Joint exploration of activation and weight sparsity in compressed neural networks, 236
- Mao, A privacy-preserving deep learning approach for face recognition with edge computing, 1
- Wang, Not just privacy, 2407
Cited by
Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models
Wang, X., Tang, Z., Guo, J., Meng, T., Wang, C., Wang, T., & Jia, W. (2025) · ACM Computing Surveys · doi:10.1145/3724420 · Cited by 202
A Review on Edge Large Language Models: Design, Execution, and Applications
Zheng, Y., Chen, Y., Qian, B., Shi, X., Shu, Y., & Chen, J. (2025) · ACM Computing Surveys · doi:10.1145/3719664 · Cited by 169
“Moreover, the quadratic complexity of the self-attention mechanism with respect to sequence length exacerbates computational demands, creating severe throughput bottlenecks on edge Central Processing Units (CPUs), Graphics Processing Units (GPUs) or Neural Processing Units (NPUs) [156].”
Review of Lightweight Deep Convolutional Neural Networks
Chen, F., Li, S., Han, J., Ren, F., & Yang, Z. (2024) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-023-10032-z · Cited by 164
“And in [12–19] focuses on the techniques of compacting and accelerating DNN models.”
InfluentialFuture of generative adversarial networks (GAN) for anomaly detection in network security: A review
Lim, W., Yong, K. S. C., Lau, B. T., & Tan, C. C. L. (2024) · Computers & Security · doi:10.1016/j.cose.2024.103733 · Cited by 165
Edge deep learning in computer vision and medical diagnostics: a comprehensive survey
Xu, Y., Khan, T. M., Song, Y., & Meijering, E. H. W. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11033-5 · Cited by 236
Metadata
- Authors
- Shuvo, M. M. H., Islam, S. K., Cheng, J., & Morshed, B. I.
- Affiliations
- University of Missouri (US)
- Texas Tech University (US)
- Venue
- Proceedings of the IEEE
- Published
- 2023-01
- Volume
- 111
- Issue
- 1
- Pages
- 42-91
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W4312910656
- Cited by
- 436
- References
- 491
- Citations by year
- 2026: 159 · 2025: 173 · 2024: 82 · 2023: 22
- FWCI
- 22.67
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Proceedings of the IEEE (publishedVersion, cc-by)
- ThinkTech (Texas Tech University) (submittedVersion, cc-by)
- ThinkTech (Texas Tech University) (submittedVersion)
- Funders
- College of Engineering, University of Missouri, Columbia, MO, USA
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Keywords
- deep learning inference, edge computing, resource-constrained devices, inference acceleration, algorithm-hardware co-design, neural network accelerators, model compression, Internet of Things, embedded systems, energy-efficient computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.