Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review
Open accessThe DOI matches its Crossref record; 410 of the 491 references it cites were found in a registry or an index.
Abstract
Successful integration of deep neural networks (DNNs) or deep learning (DL) has resulted in breakthroughs in many areas. However, deploying these highly accurate models for data-driven, learned, automatic, and practical machine learning (ML) solutions to end-user applications remains challenging. DL algorithms are often computationally expensive, power-hungry, and require large memory to process complex and iterative operations of millions of parameters. Hence, training and inference of DL models are typically performed on high-performance computing (HPC) clusters in the cloud. Data transmission to the cloud results in high latency, round-trip delay, security and privacy concerns, and the inability of real-time decisions. Thus, processing on edge devices can significantly reduce cloud transmission cost. Edge devices are end devices closest to the user, such as mobile phones, cyber–physical systems (CPSs), wearables, the Internet of Things (IoT), embedded and autonomous systems, and intelligent sensors. These devices have limited memory, computing resources, and power-handling capability. Therefore, optimization techniques at both the hardware and software levels have been developed to handle the DL deployment efficiently on the edge. Understanding the existing research, challenges, and opportunities is fundamental to leveraging the next generation of edge devices with artificial intelligence (AI) capability. Mainly, four research directions have been pursued for efficient DL inference on edge devices: 1) novel DL architecture and algorithm design; 2) optimization of existing DL methods; 3) development of algorithm–hardware codesign; and 4) efficient accelerator design for DL deployment. This article focuses on surveying each of the four research directions, providing a comprehensive review of the state-of-the-art tools and techniques for efficient edge inference.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These data cannot be managed by relying only on the limited functional services provided by cloud service centers (Ge et al. 2023;Shuvo et al. 2022).”
From Survey: federated learning data security and privacy-preserving in edge-Internet of Things (2024) · cites this paper as Shuvo et al. 2022
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 1×
“NLP models, such as Transformer [45] and BERT [224], have around 80% [225] and 93% [226] weight sparsity.”
MethodGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · cited 1×
“Thus, splitting the computation of convolution and dense layer between GPU and CPU can achieve a 60× speedup for object detection using YOLO in the Jetson TK1 board [283].”
MethodDensely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
(2014) · Cited by 33,419
Research works printed but not linked: 30
- Wendelken, 2022, Worldwide and U.S. IoT Cellular Connections Forecast, 2021–2025
- Gupta, ProtoNN: Compressed and accurate KNN for resource-scarce devices, 1331
- Dennis, 2020, EdgeML: Machine learning for resource-constrained edge devices
- Kumar, Resource-efficient machine learning in 2 KB RAM for the Internet of Things, 70, 1935
- 2022, AutoML
- 2022, AutoKeras
- Ryu, BitBlade: Area and energy-efficient precision-scalable neural network accelerator with bitwise summation, 1
- Hojabr, SkippyNN: An embedded stochastic-computing accelerator for convolutional neural networks, 1
- 2022, Jetson TX2 Module
- 2022, Intel® Edison Development Platform
- 2022, Open Neural Network Exchange
- 2022, Deep Learning HDL Toolbox
- 2022, X-CUBE-AI, AI Expansion Pack for STM32CubeMX
- 2022, DNNDK User Guide
- 2022, NVIDIA TensorRT
- 2022, CEVA Deep Neural Network (CDNN)
- 2021, Qualcomm Neural Processing SDK for AI
- 2022, Stratus High-Level Synthesis
- 2022, Deploy Machine Learning Models on Mobile and IoT Devices
- Rasley, DeepSpeed, 3505
- Baines, 2021, FairScale: A general purpose modular PyTorch library for high performance and large scale training
- Dai, BigDL, 50
- 2022, Tensilica DNA Processor Family for on-Device AI
- 2022, DesignWare ARC EV Processors for Embedded Vision
- 2022, AI for the Edge
- 2022, Products, Helping You Bring Local AI to Applications From Prototype to Production
- Shacham, 2022, Pixel Visual Core: Image Processing and Machine Learning on Pixel 2
- Yang, Sparse ReRAM engine: Joint exploration of activation and weight sparsity in compressed neural networks, 236
- Mao, A privacy-preserving deep learning approach for face recognition with edge computing, 1
- Wang, Not just privacy, 2407
Cited by
Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models
Wang, X., Tang, Z., Guo, J., Meng, T., Wang, C., Wang, T., & Jia, W. (2025) · ACM Computing Surveys · doi:10.1145/3724420 · Cited by 202
A Review on Edge Large Language Models: Design, Execution, and Applications
Zheng, Y., Chen, Y., Qian, B., Shi, X., Shu, Y., & Chen, J. (2025) · ACM Computing Surveys · doi:10.1145/3719664 · Cited by 169
“Moreover, the quadratic complexity of the self-attention mechanism with respect to sequence length exacerbates computational demands, creating severe throughput bottlenecks on edge Central Processing Units (CPUs), Graphics Processing Units (GPUs) or Neural Processing Units (NPUs) [156].”
Review of Lightweight Deep Convolutional Neural Networks
Chen, F., Li, S., Han, J., Ren, F., & Yang, Z. (2024) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-023-10032-z · Cited by 164
“And in [12–19] focuses on the techniques of compacting and accelerating DNN models.”
InfluentialFuture of generative adversarial networks (GAN) for anomaly detection in network security: A review
Lim, W., Yong, K. S. C., Lau, B. T., & Tan, C. C. L. (2024) · Computers & Security · doi:10.1016/j.cose.2024.103733 · Cited by 165
Edge deep learning in computer vision and medical diagnostics: a comprehensive survey
Xu, Y., Khan, T. M., Song, Y., & Meijering, E. H. W. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11033-5 · Cited by 238
Putra, K. T., Arrayyan, A. Z., Hayati, N. Q., Firdaus, F., Damarjati, C., Bakar, A., & Chen, H. (2024) · IEEE Access · doi:10.1109/access.2024.3358827 · Cited by 191
“AI inferencing executed on microprocessors/microcontro llers encapsulates the actuation of pre-trained AI paradigms on resource-restricted microcontroller apparatuses [97].”
MethodGreen Edge AI: A Contemporary Survey
Mao, Y., Yu, X., Huang, K., Zhang, Y. A., & Zhang, J. (2024) · Proceedings of the IEEE · doi:10.1109/jproc.2024.3437365 · Cited by 187
“Moreover, recent developments of efficient DNN accelerators for edge AI were summarized in [8], [44], and [47].”
BackgroundSurvey: federated learning data security and privacy-preserving in edge-Internet of Things
Li, H., Ge, L., & Tian, L. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10774-7 · Cited by 112
“These data cannot be managed by relying only on the limited functional services provided by cloud service centers (Ge et al. 2023; Shuvo et al. 2022).”
Deploying AI on Edge: Advancement and Challenges in Edge Intelligence
Wang, T., Guo, J., Zhang, B., Yang, G., & Li, D. (2025) · Mathematics · doi:10.3390/math13111878 · Cited by 122
Deep Learning in Multimodal Fusion for Sustainable Plant Care: A Comprehensive Review
Yang, Z., Li, Y., Wang, R.-F., Hu, P., & Su, W. (2025) · Sustainability · doi:10.3390/su17125255 · Cited by 115
Shuvo, M. M. H., & ISLAM, S. K. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3233486 · Cited by 78
“In addition, as the model contains only 19,000 trainable parameters, the computational complexity is much simpler than similar works allowing more hardware flexibility and potential chance of edge deployment [19].”
MethodSelf-citationACE-CNN: Approximate Carry Disregard Multipliers for Energy-Efficient CNN-Based Image Classification
Shakibhamedan, S., Amirafshar, N., Baroughi, A. S., Shahhoseini, H. S., & TaheriNejad, N. (2024) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2024.3369230 · Cited by 65
“However, with the rise in implementation of deep learning algorithms, including CNNs, on-edge devices, and power-constrained hardware [6], [7], there is a growing demand for improved power efficiency.”
BackgroundAl-Haija, Q. S. A., & Droos, A. (2024) · Expert Systems · doi:10.1111/exsy.13726 · Cited by 119
Yang, Z.-Y., Xia, W.-K., Chu, H.-Q., Su, W., Wang, R.-F., & Wang, H. (2025) · Plants · doi:10.3390/plants14101481 · Cited by 77
“…transmission and enhance the intelligence of agricultural equipment [193]; • Adopting hardware acceleration technologies, such as TensorRT, Google Edge TPU, and FPGAs, to boost model performance and reduce energy consumption [194]; • Designing energy-efficient AI devices to support low-power deep…”
Artificial Intelligence in Visible Light Positioning for Indoor IoT: A Methodological Review
Rekkas, V. P., Iliadis, L. A., Sotiroudis, S. P., Boursianis, A. D., Sarigiannidis, P. G., Plets, D., Joseph, W., Wan, S., Christodoulou, C. G., Karagiannidis, G. K., et al. (2023) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2023.3327211 · Cited by 63
“DL models pose as accurate and robust approaches in various fields and applications [122].”
BackgroundReal-time data visual monitoring of triboelectric nanogenerators enabled by Deep learning
Zhang, H., Liu, T., Zou, X., Zhu, Y., Chi, M., Wu, D., Jiang, K., Zhu, S., Zhai, W., Wang, S., et al. (2024) · Nano Energy · doi:10.1016/j.nanoen.2024.110186 · Cited by 52
Data-Driven Weather Forecasting and Climate Modeling from the Perspective of Development
Wu, Y., & Xue, W. (2024) · Atmosphere · doi:10.3390/atmos15060689 · Cited by 60
Edge AI in Practice: A Survey and Deployment Framework for Neural Networks on Embedded Systems
Cordova-Cardenas, R., Amor, D., & Gutiérrez, Á. (2025) · Electronics · doi:10.3390/electronics14244877 · Cited by 93
PolyLUT: Learning Piecewise Polynomials for Ultra-Low Latency FPGA LUT-based Inference
Andronic, M., & Constantinides, G. A. (2023) · International Conference on Field-Programmable Technology (FPT) · doi:10.1109/icfpt59805.2023.00012 · Cited by 69
“Deploying these models to the cloud leads to the inability to produce results in real-time, security concerns, and increased latency and round-trip delay [2].”
BackgroundInfluentialAlshuhail, A., Alshahrani, A., Mahgoub, H., Ghaleb, M., Darem, A. A., Aljehane, N. O., Alotaibi, M., & Alzahrani, F. (2025) · Alexandria Engineering Journal · doi:10.1016/j.aej.2025.08.030 · Cited by 60
Human-centred artificial intelligence for mobile health sensing: challenges and opportunities
Dang, T., Spathis, D., Ghosh, A., & Mascolo, C. (2023) · Royal Society Open Science · doi:10.1098/rsos.230806 · Cited by 30
Machine learning inference serving models in serverless computing: a survey
Aslani, A., & Ghobaei‐Arani, M. (2025) · Computing · doi:10.1007/s00607-024-01377-9 · Cited by 31
Integrating IoT and 6G: Applications of Edge Intelligence, Challenges, and Future Directions
He, Q., Lin, J., Fang, H., Wang, X., Huang, M., Yi, X., & Yu, K. (2025) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2025.3586152 · Cited by 58
“Accelerators such as GPUs and TPUs provide powerful computing capabilities [56], supporting complex deep learning tasks, while ARM-based low-power processors, known for their energy efficiency and low cost, are particularly suitable for embedded systems and IoT devices [57].”
A Study on the Application of TensorFlow Compression Techniques to Human Activity Recognition
Contoli, C., & Lattanzi, E. (2023) · IEEE Access · doi:10.1109/access.2023.3276438 · Cited by 38
Yu, G., Deng, Z., Bao, Z., Zhang, Y., & He, B. (2023) · Bioengineering · doi:10.3390/bioengineering10111324 · Cited by 28
Xiao, T., Li, Y., Zhang, D., et al. (2024) · Journal of Hazardous Materials · doi:10.1016/j.jhazmat.2024.134723 · Cited by 28
Dong, C., Shafiq, M., Dabel, M. M. A., Sun, Y., & Tian, Z. (2023) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2023.3339468 · Cited by 31
“Offloading computation-intensive tasks in the MEC is an efficient method to provide low-latency, high-quality inference services [6], [7].”
MethodCollaborative Robotics: A Survey From Literature and Practitioners Perspectives
Montini, E., Daniele, F., Agbomemewa, L., Confalonieri, M., Cutrona, V., Bettoni, A., Rocco, P., & Ferrario, A. (2024) · Journal of Intelligent & Robotic Systems · doi:10.1007/s10846-024-02141-z · Cited by 35
“The efficient deployment of arti-ficial intelligence algorithms on resource-constrained edge devices is also a critical factor in enabling real-time decision-making and reducing latency [108].”
Physical Reservoir Computing Using van der Waals Ferroelectrics for Acoustic Keyword Spotting
Cao, Y. H., Zhang, Z., Qin, B.-W., Sang, W., Li, H., Wang, T., Tan, F., Yang, G., Zhang, X., Liu, T., et al. (2024) · ACS Nano · doi:10.1021/acsnano.4c06144 · Cited by 28
Edge-AI: A systematic review on architectures, applications, and challenges
Gauttam, H., Nain, G., Pattanaik, K. K., & Mendes, P. (2025) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2025.104375 · Cited by 42
Artificial intelligence-powered visual internet of things in smart cities: A comprehensive review
Ghati, O. E., Alaoui-Fdili, O., Chahbouni, O., Alioua, N., & Bouarifi, W. (2024) · Sustainable Computing Informatics and Systems · doi:10.1016/j.suscom.2024.101004 · Cited by 44
Pazhanivel, D. B., Narayanan, V. A., & Sivakumar, P. B. (2024) · Sensors · doi:10.3390/s24155069 · Cited by 30
“On the other Pruning [83] involves removing the least significant parameters, such as weights and biases, to reduce the complexity and memory footprint of a neural networks.”
Qu, X., Aponte, D., Banbury, C., Robinson, D. P., Ding, T., Koishida, K., Zharkov, I., & Chen, T. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01419 · Cited by 55
“One major challenge is the substantial storage space required to hold these models, which can be impractical for everyday devices such as standard PCs and even more so for resource-constrained edge devices [50].”
Patra, A., Pandey, A., Hassija, V., Chamola, V., & Mishra, R. P. (2024) · IEEE Access · doi:10.1109/access.2024.3452184 · Cited by 26
“This ensures smooth and uninterrupted experiences for users, even during peak usage [46], [47].”
BackgroundAljami, H. M., Alrowais, N. A., AlAwajy, A. M., Alhrgan, S. O., Aldwaani, R. A., Alsawadi, M. S., Saqib, N. U., Alam, S. S., & Alsubaie, R. (2026) · Computers · doi:10.3390/computers15020074 · Cited by 24
Edge Deployment of Deep Networks for Visual Detection: A Review
Zhai, H., Du, J., Ai, Y., & Hu, T. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3502539 · Cited by 35
“Currently, this task faces the following challenges [24], [25], [26], [27]. a) Computational Constraints: Object detection tasks entail high complexity, requiring extensive computations.”
An Efficient Deep Neural Network for Surface Defect Detection in Industrial Edge Sensing
Wang, J., Zou, H., Zhou, M., & Su, R. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3507954 · Cited by 30
“So there is a growing realization about the lightweight and efficient improvement [13], [14].”
Efficient Deployment of Peanut Leaf Disease Detection Models on Edge AI Devices
Lv, Z., Yang, S., Ma, S., Wang, Q., Sun, J., Du, L., Han, J., Guo, Y., & Zhang, H. (2025) · Agriculture · doi:10.3390/agriculture15030332 · Cited by 36
NeuraLUT: Hiding Neural Network Density in Boolean Synthesizable Functions
Andronic, M., & Constantinides, G. A. (2024) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl64840.2024.00028 · Cited by 47
“In response to these challenges, research efforts come from both hardware and software perspectives [4].”
BackgroundUSSC-YOLO: Enhanced Multi-Scale Road Crack Object Detection Algorithm for UAV Image
Zhang, Y., Lu, Y., Huo, Z., Li, J., Sun, Y., & Huang, H. (2024) · Sensors · doi:10.3390/s24175586 · Cited by 17
Wang, X., Xu, Z., & Sui, X. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2025.1538277 · Cited by 32
Intelligent Embedded Systems Platform for Vehicular Cyber-Physical Systems
Conrad, C., Al–Rubaye, S., & Tsourdos, A. (2023) · Electronics · doi:10.3390/electronics12132908 · Cited by 23
“[24] conduct a review of deep learning techniques applied to edge devices such as CPSs, IoT, autonomous systems, embedded hardware, and intelligent sensors.”
BackgroundJoint Device Scheduling and Resource Allocation for ISCC-Based Multiview–Multitask Inference
Wang, D., Wen, D., He, Y., Chen, Q., Zhu, G., & Yu, G. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3456569 · Cited by 22
Reducing the Power Consumption of Edge Devices Supporting Ambient Intelligence Applications
Fanariotis, A., Orphanoudakis, T. G., & Fotopoulos, V. E. (2024) · Information · doi:10.3390/info15030161 · Cited by 16
“The review performed in [6] provides a comprehensive examination of tools and techniques for efficient edge inference, a key element in AI on edge devices.”
MethodInfer-EDGE: Dynamic DNN Inference Optimization in Just-in-Time Edge-AI Implementations
Mounesan, M., Zhang, X., & Debroy, S. (2025) · IEEE/IFIP Network Operations and Management Symposium (NOMS) · doi:10.1109/noms57970.2025.11073623 · Cited by 32
Hossain, M. N., Rahman, M. M., & Ramasamy, D. (2025) · Measurement · doi:10.1016/j.measurement.2025.117618 · Cited by 26
DACBN: Dual attention convolutional broad network for fine-grained visual recognition
Chen, T., Wang, L., Liu, Y., & Yu, H. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110749 · Cited by 16
A Survey on the Datasets and Algorithms for Satellite Data Applications
Affek, M., & Szymański, J. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3424954 · Cited by 22
“Readers interested in optimization methods for miniaturizing the deep learning algorithms can start from [183].”
BackgroundHarrath, Y., Adohinzin, O., Kaabi, J., & Saathoff, M. (2025) · Computers · doi:10.3390/computers14090374 · Cited by 23
“For example, deploying AI-powered chatbots on mobile devices often relies on such techniques to ensure responsiveness and energy efficiency [237,238].”
Wang, H., Li, T., Zhang, M., Li, Q., Cui, H., Jiang, Y., & Yuan, Z. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3465434 · Cited by 17
“Edge computing offers a promising solution by processing data locally on edge devices ( e.g., intelligent cameras), which are closer to the data source, and have limited memory and computing power [8]–[10].”
Shuvo, M. M. H., Titirsha, T., Gonzalez, G. L., & ISLAM, S. K. (2024) · IEEE Access · doi:10.1109/access.2024.3485550 · Cited by 20
“Besides achieving human-level performance, the DL techniques automatically learn rich features with increased depth, eliminating manual feature engineering, as necessary in classical ML methods [18], [19], [20].”
MethodSelf-citationYan, P., & Khoei, T. T. (2025) · Franklin Open · doi:10.1016/j.fraope.2025.100256 · Cited by 24
Pan, J., Song, P.-C., Chou, J., Watada, J., & Chu, S. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3353851 · Cited by 15
“L OW-POWER resource-constrained devices are widely present in industrial environments and play an irreplace-able role [1].”
BackgroundImage Processing for Smart Agriculture Applications Using Cloud-Fog Computing
Marković, D., Stamenković, Z., Đorđević, B., & Ranđić, S. (2024) · Sensors · doi:10.3390/s24185965 · Cited by 14
“The findings reveal that ViTs, after fine-tuning, can outperform convolutional models and are more robust to adversarial attacks, making them suitable for such critical applications.”
BackgroundA Survey on Real-Time Object Detection on FPGAs
Hozhabr, S. H., & Giorgi, R. (2025) · IEEE Access · doi:10.1109/access.2025.3544515 · Cited by 25
“However, achieving an efficient network tailored to the characteristics of the target hardware is a challenging and time-consuming endeavor [247].”
Zhang, Y., Qiu, S., Du, K., Wu, S. P., Xiang, T., Zheng, K., Liu, Z., Chen, H., Ji, N., Wang, F., et al. (2026) · Nano-Micro Letters · doi:10.1007/s40820-025-02003-9 · Cited by 23
“Dynamic scheduling techniques [176] have been introduced to enable dynamic adaptation to context, improving responsiveness and energy efficiency in daily use.”
A comprehensive review of lightweight deep learning models for edge computing with future directions
Lai, N., Dewi, D. A., Maidin, S. S., Xiao, W., Zhao, S., & Hu, Q. (2026) · Discover Computing · doi:10.1007/s10791-026-10021-3 · Cited by 31
Characterizing and Modeling AI-Driven Animal Ecology Studies at the Edge
Kline, J., O’Quinn, A., Berger-Wolf, T. Y., & Stewart, C. (2024) · IEEE/ACM Symposium on Edge Computing (SEC) · doi:10.1109/sec62691.2024.00025 · Cited by 19
“Designed for remote regions with limited compute and memory resources.”
Naser, M. Z. (2025) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2025.116231 · Cited by 16
Incentive-Aware Partitioning and Offloading Scheme for Inference Services in Edge Computing
Kim, T., Kim, C. K., Lee, S.-S., et al. (2024) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2024.3359148 · Cited by 11
“Timely inference at the edge is essential in numerous use cases including security, surveillance, healthcare, and autonomous driving (e.g., to inspect visitors’ faces, or to detect heartbeat frequency) [18], [19].”
BackgroundPolyLUT-Add: FPGA-based LUT Inference with Wide Inputs
Lou, B., Rademacher, R., Boland, D., & Leong, P. H. W. (2024) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl64840.2024.00029 · Cited by 27
“Compared with Binary Neural Networks (BNNs [9]), which utilize 1-bit quantization to replace multipliers with simple XNOR gates, LUT-based neurons further optimize FPGA resource utilization using LUTs as direct inference operators.”
Energy-aware tinyML model selection on zero energy devices
Šabović, A., Fontaine, J., Poorter, E. D., & Famaey, J. (2025) · Internet of Things · doi:10.1016/j.iot.2025.101488 · Cited by 18
A deep learning detection framework for gas pipeline leaks suitable for low-power edge devices
Shi, J., Xu, Y., & Lyu, P. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.113269 · Cited by 13
Im, H., Lee, D. C., & Lee, S. (2024) · Sensors · doi:10.3390/s24092807 · Cited by 10
“Considering that the proposed IDS principally relies on traditional ML techniques, it remains sufficiently lightweight for implementation in in-vehicle edge computing devices [14,15].”
MethodPower Conversion Internet of Things: Architecture, Key Technologies and Prospects
Zhou, K., Gao, F., Hou, Z., & Meng, X. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3396559 · Cited by 11
“The research on deploying AI algorithms on resource-constrained edge devices to alleviate communication and computation pressure has garnered significant attention [36], [37], [38], [39], [40].”
BackgroundMahmudah, H., Aisjah, A. S., Arifin, S., & Prastyanto, C. A. (2025) · IEEE Access · doi:10.1109/access.2025.3562612 · Cited by 15
“However, it is essential to balance the trade-off between system performance and efficiency [74], [75], [76], [77].”
Liquid neural network in smart manufacturing: A new opportunity
Leng, J., Zhu, T., Li, J., Wang, B., Yang, B. Y., Li, X., Liu, Q., Chen, X., & Wang, L. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.104222 · Cited by 12
A Review of Intelligent Self-Powered Sensing Systems Enabling Autonomous AIoT
Cui, H., Tang, T., & Liu, H. (2025) · AI Sensors · doi:10.3390/aisens2010001 · Cited by 15
“The strategy of edge computing involves embedding part of the algorithmic model at the edge, while the cloud processes only the pre-processed data transmitted from the edge devices [112].”
FPGA Acceleration of DL-Based Real-Time DC Series Arc Fault Detection
Mao, Y., Weiss, R. J., Zhang, Y., et al. (2024) · IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) · doi:10.1109/ipdpsw63119.2024.00031 · Cited by 9
“Hardware acceleration of deep learning (DL) inference on edge devices is gaining more and more attention [12], [13].”
BackgroundReparameterization of Lightweight Transformer for On-Device Speech Emotion Recognition
Zhang, Z., Dong, Z., Xu, W., & Han, J. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3483232 · Cited by 15
Unsupervised Learning-Based Low-Complexity Integrated Sensing and Communication Precoder Design
Temiz, M., & Masouros, C. (2025) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2025.3559737 · Cited by 13
“Deep learning solutions are especially appealing since efficient neural network accelerators are already included in recent central processing units (CPUs), even in the CPUs of resource-constrained edge devices and mobile phones [23].”
Zhu, Z., Gao, Z., Zhuang, J., Huang, D., Huang, G., Wang, H., Pei, J., Zheng, J., & Liu, C. (2025) · Agriculture · doi:10.3390/agriculture15151653 · Cited by 9
Wu, Q., Ding, X., Gao, S. H., Luo, J. T., & Huang, W. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103516 · Cited by 10
Real time wire rope detection method based on Rockchip RK3588
Qian, M., Wang, Y., Liu, S., Xu, Z., Ji, Z., Chen, M., Wu, H., & Zhang, Z. (2025) · Scientific Reports · doi:10.1038/s41598-025-16043-z · Cited by 11
“Practical deployment of such algorithms poses significant challenges, as the trained models exhibit sensitivity to environmental variations when transitioned from servers to edge devices 19 .”
A Survey of AI Inference Technologies for On-Device Systems
Wang, W., Li, K., Ji, B., Liu, X., Yu, J., & Wu, Q. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3613691 · Cited by 16
“Shuvo et al. [13] reviewed accelerated inference techniques for resource-constrained edge devices, covering aspects such as model design, model compression, hardware platforms, software stacks, and distributed training and inference at the edge.”
Chen, X., Xiong, H., Chen, J., Kong, J., Zhang, Y., & Fang, X. (2025) · ACS Applied Materials & Interfaces · doi:10.1021/acsami.5c14054 · Cited by 11
Literature Review on Fault Mechanism Analysis and Diagnosis Methods for Main Pump Systems
Ma, W., Ma, S., Zou, Z., Fu, B., Ma, J., Liu, J., & Zhang, Q. (2025) · Machines · doi:10.3390/machines13111000 · Cited by 13
AI and ML empowering 5G and shaping the 6G future: Models, metrics, architectures, and applications
Tulaskar, D. P., Sindhu, B., Chakole, N., Parteki, R., Leema, A. A., Balakrishnan, P. A., Avthanka, A., Girhe, R., Kulkarni, M. B., & Bhaiyya, M. L. (2025) · ICT Express · doi:10.1016/j.icte.2025.11.024 · Cited by 9
Cost-Aware Neural Network Splitting and Dynamic Rescheduling for Edge Intelligence
Luger, D., Aral, A., & Brandić, I. (2023) · International Workshop on Edge Systems, Analytics and Networking (EdgeSys) · doi:10.1145/3578354.3592871 · Cited by 12
McEnroe, P., Wang, S., & Liyanage, M. (2024) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc58415.2024.10919742 · Cited by 9
“The inference latency of a deep learning based model can be improved by reducing the size of the model through model compression techniques [5].”
A quantized subtraction-convolution network for industrial lightweight edge interpretable diagnosis
Wu, Q., Luo, J., Huang, W., & Ding, X. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.112241 · Cited by 8
Lithology Identification of Lithium Minerals Based on TL-FMix-MobileViT Model
Jing, J., Zhang, N., Zhang, H., Liao, S., Chen, L., Chang, J., Tao, J., & Li, S. (2025) · Natural Resources Research · doi:10.1007/s11053-025-10475-0 · Cited by 7
“…fps) and parameter size were also employed as important indicators for assessing the complexity of the rock classification model (Hu et al., 2021; Shuvo et al., 2022). where TP, FP, TN, and FN are the numbers of true positives (correct predictions of positive instances as positive), false…”
Han, C., Yang, T., Cui, Z., & Sun, X. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3583764 · Cited by 11
“Second, many edge devices are resource-constrained, with limited computational power, memory, and energy resources [9], [10].”
DPU-KV: On the Benefits of DPU Offloading for In-Memory Key-Value Stores at the Edge
Kashyap, A., Li, Y., & Lu, X. (2025) · ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC) · doi:10.1145/3731545.3731571 · Cited by 9
“Additionally, edge servers can be resource-constrained [49, 67, 73], and offloading KVS tasks can help free up resources, enabling resource sharing among other 5 Finally, the DPU-offloaded KVS should be able to provide low latency and high throughput in comparison to CPU-based KVS. Limitation of…”
Khan, S., Perumal, K., Alsolai, H., & Aljohani, A. A. (2025) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2025.110761 · Cited by 13
Xu, P. J., Kan, S. X., Jin, J., Zhang, Z. J., Gu, Y. X., Zhang, B., & Zhou, Y. L. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131817 · Cited by 9
Ahmadpanah, S. H., Mirabi, M., Sobhanloo, S., Afsharfarnia, P., Fallah, D., & Bayati, M. (2025) · Ad Hoc Networks · doi:10.1016/j.adhoc.2025.104080 · Cited by 10
Artificial intelligence enabled smart digital eye wearables
RaviChandran, N., Teo, Z. L., & Ting, D. S. W. (2023) · Current Opinion in Ophthalmology · doi:10.1097/icu.0000000000000985 · Cited by 8
“These computations can be real-time and run on resource-constrained platforms, thus reducing latency and power consumption and addressing privacy-related issues [71].”
BackgroundTian, J., Song, H., Sheng, G., & Jiang, X. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3364264 · Cited by 8
“This article proposed a joint inference method to assess the transformer condition by analyzing the subsystem condition data and archival KG, where the inference refers to predicting the condition grades from unseen data using the learnable parameters learned through the training process [25].”
MethodDynamic Tsetlin Machine Accelerators for On-Chip Training Using FPGAs
Mao, G., Rahman, T., Maheshwari, S., Pattison, B., Shao, Z., Shafik, R., & Yakovlev, A. (2025) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2025.3564875 · Cited by 16
“The FPGA component, in particular, offers advantages in customization, cost, ease of prototyping, and lower implementation risk compared to ASIC solutions with better power efficiency than CPU and GPU training implementations [12].”
Efficient Integer-Only-Inference of Gradient Boosting Decision Trees on Low-Power Devices
Alsharari, M., Mai, S. T., Woods, R., & Reaño, C. (2024) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2024.3446582 · Cited by 12
“The reader is directed to [22] for an in-depth exploration.”
SynFlowFL: A Dynamic Synaptic Flow Framework for Efficient, Personalized Federated Learning
Li, D., Lin, W., Wu, W., Zhang, H., & Wang, X. (2025) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2025.3526941 · Cited by 8
Huang, W., & Deng, X. (2025) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2025.104231 · Cited by 15
Bringing IoT Intrusion Detection to the Edge
Asulba, B. A., Souto, P. F., & Almeida, L. (2024) · International Conference on Future Networks and Distributed Systems (ICFNDS) · doi:10.1145/3726122.3726166 · Cited by 10
“Although deep learning (DL) algorithms show potential in this domain [13], their high computational demands pose challenges for resource-constrained edge devices [25].”
PolyLUT: Ultra-Low Latency Polynomial Inference With Hardware-Aware Structured Pruning
Andronic, M., Li, J., & Constantinides, G. A. (2025) · IEEE Transactions on Computers · doi:10.1109/tc.2025.3586311 · Cited by 17
“Unlike cloud computing, deploying DNN models on edge devices poses substantial challenges because of their constrained computational resources [7].”
Centralized Two-Tiered Tree-Based Intrusion-Detection System (C2T-IDS)
Yassine, H. A. K., Saleh, M. A., Nakhal, B., & Chakik, A. E. (2025) · IoT · doi:10.3390/iot6040067 · Cited by 10
“Deep Neural Networks (DNNs): Require significant memory and processing power for training and inference [15].”
Deep Feature Learning From Electromyographic Signals for Gesture Recognition Systems
Zhong, W., Jiang, X., Szymaniak, K., Jabbari, M., Ma, C., & Nazarpour, K. (2025) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2025.3635419 · Cited by 14
Yang, Y., Liu, X., & Wu, Y. (2026) · Energy · doi:10.1016/j.energy.2026.139967 · Cited by 6
Cyber Physical Systems Security: Bridging Privacy, Verification, Intelligence, and Forensics
Ramya, K., Rath, S., Venkataraman, P., & Kumar, R. (2026) · IEEE Access · doi:10.1109/access.2026.3668412 · Cited by 11
“Specialized toolkits like EdgeML further provide optimized implementations of classification and sequence prediction models for low-power CPS devices [129], [130].”
Rostkowska, M., & Skrzypczyński, P. (2023) · Sensors · doi:10.3390/s23146485 · Cited by 7
“This enables real-time inference and efficient utilization of resources on edge devices, ensuring faster and more responsive image processing capabilities even with limited computing power [56].”
Background
Metadata
- Authors
- Shuvo, M. M. H., Islam, S. K., Cheng, J., & Morshed, B. I.
- Affiliations
- University of Missouri (US)
- Texas Tech University (US)
- Venue
- Proceedings of the IEEE
- Published
- 2023-01
- Volume
- 111
- Issue
- 1
- Pages
- 42-91
- ISSN
- 0018-9219 (ISSN-L)
- 1558-2256
- OpenAlex
- W4312910656
- Cited by
- 436
- References
- 491
- Citations by year
- 2026: 159 · 2025: 173 · 2024: 82 · 2023: 22
- FWCI
- 22.67
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Proceedings of the IEEE (publishedVersion, cc-by)
- ThinkTech (Texas Tech University) (submittedVersion, cc-by)
- ThinkTech (Texas Tech University) (submittedVersion)
- Funders
- College of Engineering, University of Missouri, Columbia, MO, USA
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Keywords
- deep learning inference, edge computing, resource-constrained devices, inference acceleration, algorithm-hardware co-design, neural network accelerators, model compression, Internet of Things, embedded systems, energy-efficient computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.