Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 202

The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.

Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models

Open access
Wang, Xubin, Tang, Zhiqing, Guo, Jianxiong, Meng, Tianhui, Wang, Chenhao, Wang, Tian, Jia, Weijia · 2025 · ACM Computing Surveys

Abstract

The rapid advancement of artificial intelligence (AI) technologies has led to an increasing deployment of AI models on edge and terminal devices, driven by the proliferation of the Internet of Things (IoT) and the need for real-time data processing. This survey comprehensively explores the current state, technical challenges, and future trends of on-device AI models. We define on-device AI models as those designed to perform local data processing and inference, emphasizing their characteristics such as real-time performance, resource constraints, and enhanced data privacy. The survey is structured around key themes, including the fundamental concepts of AI models, application scenarios across various domains, and technical challenges faced in edge environments. We also discuss optimization and implementation strategies, such as data preprocessing, model compression, and hardware acceleration, which are essential for effective deployment. Furthermore, we examine the impact of emerging technologies, including edge computing and foundation models, on the evolution of on-device AI models. By providing a structured overview of the challenges, solutions, and future directions, this survey aims to facilitate further research and application of on-device AI, ultimately contributing to the advancement of intelligent systems in everyday life.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3724420

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 41 references

Research works printed but not linked: 215

  1. 10.1002/spe.3033 · DOI
  2. Abdulmalik Alwarafy, Khaled A. Al-Thelaya, Mohamed Abdallah, Jens Schneider, and Mounir Hamdi. 2020. A survey on security and privacy issues in edge-computing-assisted internet of things. IEEE Internet of Things Journal 8, 6 (2020), 4004–4022. · DOI · Paper page
  3. 10.1016/j.eswa.2022.117362 · DOI · Paper page
  4. Jokubas Ausra, Micah Madrid, Rose T. Yin, Jessica Hanna, Suzanne Arnott, Jaclyn A. Brennan, Roberto Peralta, David Clausen, Jakob A. Bakall, Igor R. Efimov, and others. 2022. Wireless, fully implantable cardiac stimulation and recording with on-device computation for closed-loop pacing and defibrillation. Science Advances 8, 43 (2022), eabq7469. · DOI
  5. 10.1109/COMST.2022.3200740 · DOI · Paper page
  6. 10.1016/j.ijpe.2020.107776 · DOI · Paper page
  7. Mohammed Djameleddine Belgoumri, Mohamed Reda Bouadjenek, Sunil Aryal, and Hakim Hacid. 2024. Data quality in edge machine learning: A state-of-the-art survey. arXiv preprint arXiv:2406.02600 (2024). · arXiv
  8. 10.1109/INFOCOM41043.2020.9155403 · DOI
  9. Mélanie Bernhardt, Daniel C. Castro, Ryutaro Tanno, Anton Schwaighofer, Kerem C. Tezcan, Miguel Monteiro, Shruthi Bannur, Matthew P. Lungren, Aditya Nori, Ben Glocker, and others. 2022. Active label cleaning for improved dataset quality under resource constraints. Nature Communications 13, 1 (2022), 1161. · DOI
  10. 10.1109/TIM.2006.887319 · DOI
  11. 10.1145/3486618 · DOI
  12. Han Cai, Ligeng Zhu, and Song Han. 2018. Proxylessnas: Direct neural architecture search on target task and hardware. arXiv preprint arXiv:1812.00332 (2018). · arXiv
  13. Nicola Capodieci, Roberto Cavicchioli, Marko Bertogna, and Aingara Paramakuru. 2018. Deadline-based scheduling for GPU with preemption support. In 2018 IEEE Real-time Systems Symposium (RTSS’18). IEEE, 119–130. · DOI
  14. 10.1109/CVPR.2013.389 · DOI · Paper page
  15. 10.1007/s11263-020-01379-y · DOI
  16. H. Cheng M. Zhang and J. Q. Shi. 2024. A survey on deep neural network pruning: taxonomy comparison analysis and recommendations. IEEE Transactions on Pattern Analysis & Machine Intelligence 1 (2024) 1–20.
  17. 10.1145/3652597 · DOI
  18. Vladimir Chikin and Mikhail Antiukh. 2022. Data-free network compression via parametric non-uniform mixed precision quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 450–459.
  19. Hyungmin Cho, Jeesoo Lee, and Jaejin Lee. 2021. FARNN: FPGA-GPU hybrid acceleration platform for recurrent neural networks. IEEE Transactions on Parallel and Distributed Systems 33, 7 (2021), 1725–1738.
  20. 10.1145/3519598 · DOI
  21. Y. Chu D. Feng Z. Liu L. Zhang Z. Zhao Z. Wang and X. G. Xia. 2022. A fine-grained attention model for high accuracy operational robot guidance. IEEE Internet of Things Journal 10 2 (2022) 1066–1081. · DOI
  22. 10.1109/TCAD.2022.3197986 · DOI
  23. 10.5555/2789272.2912091 · doi:10.5555/2789272.2912091
  24. 10.1109/TVT.2020.3013990 · DOI · Paper page
  25. H. J. Damsgaard A. Grenier D. Katare Z. Taufique S. Shakibhamedan T. Troccoli and J. Nurmi. 2024. Adaptive approximate computing in Edge AI and IoT applications. Journal of Systems Architecture 150 (2024) 103114. · DOI
  26. 10.1109/JIOT.2020.2984887 · DOI · Paper page
  27. 10.1145/3450494 · DOI
  28. 10.1145/3575693.3575702 · DOI
  29. 10.1109/TMM.2018.2814346 · DOI
  30. S. Dong J. Tang K. Abbas R. Hou J. Kamruzzaman L. Rutkowski and R. Buyya. 2024. Task offloading strategies for mobile edge computing: A survey. Computer Networks 254 (2024) 110791. · DOI · Paper page
  31. 10.1109/MWC.009.2300501 · DOI · Paper page
  32. Sijing Duan, Dan Wang, Ju Ren, Feng Lyu, Ye Zhang, Huaqing Wu, and Xuemin Shen. 2022. Distributed artificial intelligence empowered by end-edge-cloud computing: A survey. IEEE Communications Surveys & Tutorials 25, 1 (2022), 591–624. · DOI · Paper page
  33. Jeremy Fowers, Kalin Ovtcharov, Michael Papamichael, Todd Massengill, Ming Liu, Daniel Lo, Shlomi Alkalay, Michael Haselman, Logan Adams, Mahdi Ghandi, and others. 2018. A configurable cloud-scale DNN processor for real-time AI. In 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA’18). IEEE, 1–14.
  34. Morgan R. Frank, David Autor, James E. Bessen, Erik Brynjolfsson, Manuel Cebrian, David J. Deming, Maryann Feldman, Matthew Groh, José Lobo, Esteban Moro, and others. 2019. Toward understanding the impact of artificial intelligence on labor. Proceedings of the National Academy of Sciences 116, 14 (2019), 6531–6539. · DOI · Paper page
  35. Yonggan Fu, Haoran You, Yang Zhao, Yue Wang, Chaojian Li, Kailash Gopalakrishnan, Zhangyang Wang, and Yingyan Lin. 2020. Fractrain: Fractionally squeezing bit savings both temporally and spatially for efficient DNN training. Advances in Neural Information Processing Systems 33 (2020), 12127–12139.
  36. 10.1109/JETCAS.2020.3040300 · DOI
  37. 10.1109/CVPR42600.2020.00197 · DOI
  38. Tong Geng, Ang Li, Tianqi Wang, Chunshu Wu, et al. 2020. O3BNN-R: An out-of-order architecture for high-performance and regularized BNN inference. IEEE Transactions on Parallel and Distributed Systems 32, 1 (2020), 199–213. · DOI
  39. 10.1145/3664200 · DOI
  40. 10.1109/TITS.2023.3275741 · DOI · Paper page
  41. H. Gu L. Zhao Z. Han G. Zheng and S. Song. 2023. AI-enhanced cloud-edge-terminal collaborative network: Survey applications and future directions. IEEE Communications Surveys & Tutorials 26 2 (2023) 1322–1385. · DOI
  42. 10.1109/INFOCOM42981.2021.9488918 · doi:10.1109/infocom42981.2021.9488918
  43. 10.1007/s11227-020-03288-w · DOI
  44. Jialin Guo, Jie Wu, Anfeng Liu, and Neal N. Xiong. 2021. LightFed: An efficient and secure federated edge learning system on model splitting. IEEE Transactions on Parallel and Distributed Systems 33, 11 (2021), 2701–2713.
  45. Yundi Guo, Beiji Zou, Ju Ren, Qingqing Liu, Deyu Zhang, and Yaoxue Zhang. 2019. Distributed and efficient object detection via interactions among devices, edge, and cloud. IEEE Transactions on Multimedia 21, 11 (2019), 2903–2915. · DOI
  46. Chirag Gupta, Arun Sai Suggala, Ankit Goyal, Harsha Vardhan Simhadri, Bhargavi Paranjape, et al. 2017. Protonn: Compressed and accurate KNN for resource-scarce devices. In International Conference on Machine Learning. PMLR, 1331–1340.
  47. MyungJoo Ham, Jijoong Moon, Geunsik Lim, et al. 2021. NNStreamer: Efficient and agile development of on-device AI systems. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP’21). IEEE, 198–207.
  48. 10.1145/3510457.3513026 · doi:10.1145/3510457.3513026
  49. Song Han, Huizi Mao, and William J. Dally. 2016. Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding. In International Conference on Learning Representations (ICLR’16).
  50. Tao Han, Khan Muhammad, Tanveer Hussain, Jaime Lloret, and Sung Wook Baik. 2020. An efficient deep learning framework for intelligent energy management in IoT networks. IEEE Internet of Things Journal 8, 5 (2020), 3170–3179. · DOI · Paper page
  51. 10.1109/TMM.2022.3192663 · DOI
  52. 10.1109/MWC.001.2000345 · DOI
  53. 10.1145/3555802 · DOI · Paper page
  54. 10.1145/3467956 · DOI
  55. Chun-Hsian Huang Wen-Tung Chen Yi-Chun Chang and Kuan-Ting Wu. 2024. An edge and trustworthy AI UAV 1276 system with self-adaptivity and hyperspectral imaging for air quality monitoring. IEEE Internet of Things Journal 11 20 (2024) 32572–32584. · DOI
  56. 10.1145/3495243.3560551 · DOI
  57. Yakun Huang, Xiuquan Qiao, Jian Tang, Pei Ren, et al. 2020. Deepadapter: A collaborative deep learning framework for the mobile web using context-aware network pruning. In IEEE Conference on Computer Communications (IEEE INFOCOM’20). IEEE, 834–843.
  58. 10.1109/TMC.2021.3122279 · DOI
  59. 10.1109/ICCAD51958.2021.9643557 · DOI
  60. Tianyu Jia, Yuhao Ju, et al. 2020. NCPU: An embedded neural CPU architecture on resource-constrained low power devices for real-time end-to-end performance. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO’20). IEEE, 1097–1109.
  61. 10.1109/TPDS.2022.3225185 · DOI
  62. 10.1038/s41586-023-05773-7 · DOI
  63. D. Katare D. Perino J. Nurmi M. Warnier M. Janssen and A. Y. Ding. 2023. A survey on approximate edge AI for energy efficient autonomous driving services. IEEE Communications Surveys & Tutorials 25 4 (2023) 2714–2754. · DOI
  64. Alexandros Kouris, Stylianos I. Venieris, Stefanos Laskaridis, and Nicholas D. Lane. 2022. Fluid batching: Exit-aware preemptive serving of early-exit neural networks on edge NPUs. arXiv preprint arXiv:2209.13443 (2022). · arXiv
  65. Bo-Han Kung, Po-Yuan Hu, Chiu-Chang Huang, Cheng-Che Lee, et al. 2020. An efficient ECG classification system using resource-saving architecture and random forest. IEEE Journal of Biomedical and Health Informatics 25, 6 (2020), 1904–1914. · DOI
  66. Matt Kusner, Stephen Tyree, Kilian Weinberger, and Kunal Agrawal. 2014. Stochastic neighbor compression. In International Conference on Machine Learning. PMLR, 622–630.
  67. W. Kwon S. Kim M. W. Mahoney J. Hassoun K. Keutzer and A. Gholami. 2022. A fast post-training pruning framework for transformers. Advances in Neural Information Processing Systems 35 (2022) 24101–24116.
  68. 10.1109/TITS.2022.3145389 · DOI · Paper page
  69. Bing Li, Songyun Qu, and Ying Wang. 2021. An automated quantization framework for high-utilization RRAM-based PIM. IEEE Transactions on Computer-aided Design of Integrated Circuits and Systems 41, 3 (2021), 583–596.
  70. 10.1109/TCSI.2024.3364093 · doi:10.1109/tcsi.2024.3364093
  71. 10.1109/TCAD.2020.3013050 · DOI
  72. 10.1109/TII.2018.2842821 · DOI · Paper page
  73. 10.1109/ICDCS54860.2022.00051 · DOI
  74. Tao Li, Yitao Ma, and Tetsuo Endoh. 2022. From algorithm to module: Adaptive and energy-efficient quantization method for edge artificial intelligence in IoT society. IEEE Transactions on Industrial Informatics 19, 8 (2022), 8953–8964.
  75. 10.1016/j.asoc.2024.111684 · DOI
  76. X. Li and S. Bi. 2024. Optimal AI model splitting and resource allocation for device-edge co-inference in multi-user wireless sensing systems. IEEE Transactions on Wireless Communications 23 9_Part_1 (2024) 11094–11108. · DOI
  77. Xiong Li, Shanpeng Liu, Fan Wu, Saru Kumari, and Joel J. P. C. Rodrigues. 2018. Privacy preserving data aggregation scheme for mobile edge computing assisted IoT applications. IEEE Internet of Things Journal 6, 3 (2018), 4755–4763.
  78. Yunsheng Li, Yinpeng Chen, Xiyang Dai, Dongdong Chen, Mengchen Liu, Lu Yuan, Zicheng Liu, Lei Zhang, and Nuno Vasconcelos. 2020. MicroNet: Towards image recognition with extremely low FLOPs. arXiv preprint arXiv:2011.12289 (2020). · arXiv
  79. Wei Yang Bryan Lim, Jer Shyuan Ng, Zehui Xiong, Jiangming Jin, et al. 2021. Decentralized edge intelligence: A dynamic resource allocation framework for hierarchical federated learning. IEEE Transactions on Parallel and Distributed Systems 33, 3 (2021), 536–550.
  80. 10.1109/JSAC.2022.3233625 · DOI
  81. 10.1109/CVPR42600.2020.00160 · DOI · Paper page
  82. 10.1109/TWC.2021.3081991 · DOI
  83. 10.1109/JPROC.2019.2915983 · DOI · Paper page
  84. Xiaochen Liu, Yurong Jiang, Puneet Jain, and Kyu-Han Kim. 2018. TAR: Enabling fine-grained targeted advertising in retail stores. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services. 323–336.
  85. 10.1109/TIFS.2019.2946476 · DOI
  86. 10.1109/JIOT.2020.3004500 · DOI · Paper page
  87. Yinghan Long, Indranil Chakraborty, et al. 2021. Complexity-aware adaptive training and inference for edge-cloud distributed AI systems. In 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS’21). IEEE, 573–583.
  88. J. Lou Z. Tang and W. Jia. 2022. Energy-efficient joint task assignment and migration in data centers: A deep reinforcement learning approach. IEEE Transactions on Network and Service Management 20 2 (2022) 961–973. · DOI
  89. J. Lou Z. Tang W. Jia W. Zhao and J. Li. 2023. Startup-aware dependent task scheduling with bandwidth constraints in edge computing. IEEE Transactions on Mobile Computing 23 2 (2023) 1586–1600.
  90. J. Lou Z. Tang S. Zhang W. Jia W. Zhao and J. Li. 2022. Cost-effective scheduling for dependent tasks with tight deadline constraints in mobile edge computing. IEEE Transactions on Mobile Computing 22 10 (2022) 5829–5845. · DOI
  91. Chunjie Luo, Xiwen He, Jianfeng Zhan, Lei Wang, Wanling Gao, and Jiahui Dai. 2020. Comparison and benchmarking of ai models and frameworks on mobile devices. arXiv preprint arXiv:2005.05085 (2020). · arXiv
  92. 10.1109/TC.2021.3053199 · DOI
  93. 10.1145/3421510 · DOI
  94. B. Lyu S. Wen K. Shi and T. Huang. 2021. Multiobjective reinforcement learning-based neural architecture search for efficient portrait parsing. IEEE Transactions on Cybernetics 53 2 (2021) 1158–1169. · DOI
  95. Bo Lyu, Hang Yuan, Longfei Lu, and Yunye Zhang. 2021. Resource-constrained neural architecture search on edge devices. IEEE Transactions on Network Science and Engineering 9, 1 (2021), 134–142. · DOI
  96. 10.1109/JIOT.2023.3268339 · DOI
  97. Lichuan Ma, Qingqi Pei, Lu Zhou, Haojin Zhu, Licheng Wang, and Yusheng Ji. 2020. Federated data cleaning: Collaborative and privacy-preserving data cleaning for edge intelligence. IEEE Internet of Things Journal 8, 8 (2020), 6757–6770.
  98. Y. Mao X. Yu K. Huang Y. J. A. Zhang and J. Zhang. 2024. Green Edge AI: A contemporary survey. Proceedings of the IEEE 112 7 (2024) 880–911. · DOI
  99. Alberto Marchisio, Beatrice Bussolino, Alessio Colucci, Maurizio Martina, Guido Masera, and Muhammad Shafique. 2020. Q-capsnets: A specialized framework for quantizing capsule networks. In 2020 57th ACM/IEEE Design Automation Conference (DAC’20). IEEE, 1–6.
  100. 10.1145/3313831.3376479 · DOI
  101. 10.1145/3457607 · DOI · Paper page
  102. S. Mehta and M. Rastegari. 2022. MobileViT: Light-weight general-purpose and mobile-friendly vision transformer. In International Conference on Learning Representations. Retrieved from https://openreview.net/forum?id=vh-0sUt8HlG
  103. 10.1109/TC.2021.3059819 · DOI
  104. Christian Meurisch and Max Mühlhäuser. 2021. Data protection in AI services: A survey. ACM Computing Surveys (CSUR) 54, 2 (2021), 1–38. · DOI
  105. Fengchun Miao, Wayne Holmes, Ronghuai Huang, et al. 2021. AI and Education: A Guidance for Policymakers. Unesco Publishing.
  106. 10.1109/JAS.2023.123561 · DOI · Paper page
  107. R. Mishra A. Gupta and H. P. Gupta. 2021. Locomotion mode recognition using sensory data with noisy labels: A deep learning approach. IEEE Transactions on Mobile Computing 22 6 (2021) 3460–3471. · DOI
  108. 10.1016/j.sysarc.2019.01.011 · DOI · Paper page
  109. Marissa Mock, Suzanne Edavettal, Christopher Langmead, and Alan Russell. 2023. AI can help to speed up drug discovery–but only if we give it the right data. Nature 621, 7979 (2023), 467–470. · DOI
  110. Alejandro Moran, Christiam F. Frasser, Miquel Roca, and Josep L. Rossello. 2019. Energy-efficient pattern recognition hardware with elementary cellular automata. IEEE Transactions on Computers 69, 3 (2019), 392–401. · DOI
  111. 10.1007/s00371-021-02200-8 · DOI
  112. F. Mou J. Lou Z. Tang Y. Wu W. Jia Y. Zhang and W. Zhao. 2025. Adaptive digital twin migration in vehicular edge computing and networks. IEEE Transactions on Vehicular Technology 74 3 (2025) 4839–4854. 10.1109/TVT.2024.3492349 · DOI
  113. 10.1145/3469029 · DOI · Paper page
  114. 10.1109/JIOT.2022.3160970 · DOI
  115. Jianyuan Ni, Raunak Sarbajna, Yang Liu, et al. 2022. Cross-modal knowledge distillation for vision-to-sensor action recognition. In 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’22). IEEE, 4448–4452.
  116. 10.1145/3460288 · DOI · Paper page
  117. 10.1145/3373376.3378534 · DOI · Paper page
  118. Anant V. Nori, Rahul Bera, et al. 2021. Reduct: Keep it close, keep it cool!: Efficient scaling of DNN inference on multi-core cpus with near-cache compute. In 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA’21). IEEE, 167–180.
  119. 10.1016/j.sysarc.2021.102062 · DOI
  120. Anton Obukhov, Maxim Rakhuba, Stamatios Georgoulis, Menelaos Kanakis, Dengxin Dai, and Luc Van Gool. 2020. T-basis: A compact representation for neural networks. In International Conference on Machine Learning. PMLR, 7392–7404.
  121. A. Paszke S. Gross F. Massa A. Lerer J. Bradbury G. Chanan T. Killeen Z. Lin N. Gimelshein L. Antiga A. Desmaison A. Köpf E. Yang Z. DeVito M. Raison A. Tejani S. Chilamkurthy B. Steiner L. Fang J. Bai and S. Chintala. 2019. PyTorch: an imperative style high-performance deep learning library. In Proceedings of the 33rd International Conference on Neural Information Processing Systems. Curran Associates Inc. Red Hook NY USA 1–12.
  122. 10.1016/j.sysarc.2022.102561 · DOI
  123. Giacomo Pedretti, Catherine E. Graves, Sergey Serebryakov, Ruibin Mao, Xia Sheng, Martin Foltin, Can Li, and John Paul Strachan. 2021. Tree-based machine learning performed in-memory with memristive analog CAM. Nature Communications 12, 1 (2021), 5806. · DOI
  124. 10.1145/3405837.3411381 · doi:10.1145/3405837.3411381
  125. 10.1145/3609387 · DOI
  126. 10.1109/TCAD.2020.3013049 · DOI
  127. Aji Gautama Putrada, Maman Abdurohman, Doan Perdana, and Hilal Hudan Nuha. 2023. Q8KNN: A novel 8-bit KNN quantization method for edge computing in smart lighting systems with NodeMCU. In Intelligent Systems Conference. Springer, 598–615.
  128. 10.1109/COMST.2020.3009103 · DOI · Paper page
  129. 10.1109/JIOT.2021.3051080 · DOI · Paper page
  130. 10.1016/j.engappai.2020.103737 · DOI
  131. Qiqi Ren, Omid Abbasi, Gunes Karabulut Kurt, Halim Yanikomeroglu, and Jian Chen. 2022. Caching and computation offloading in high altitude platform station (HAPS) assisted intelligent transportation systems. IEEE Transactions on Wireless Communications 21, 11 (2022), 9010–9024. · DOI
  132. 10.1109/TC.2019.2949042 · DOI
  133. 10.1145/3308560.3317590 · DOI · Paper page
  134. Yi Sheng, Junhuan Yang, Yawen Wu, Kevin Mao, Yiyu Shi, Jingtong Hu, Weiwen Jiang, and Lei Yang. 2022. The larger the fairer? Small neural networks can achieve fairness for edge devices. In Proceedings of the 59th ACM/IEEE Design Automation Conference. 163–168. · DOI
  135. 10.1109/COMST.2020.3007787 · DOI · Paper page
  136. You Shi, Changyan Yi, Bing Chen, Chenze Yang, Kun Zhu, and Jun Cai. 2022. Joint online optimization of data sampling rate and preprocessing mode for edge–cloud collaboration-enabled industrial IoT. IEEE Internet of Things Journal 9, 17 (2022), 16402–16417. · DOI
  137. V. Sindhwani T. N. Sainath and S. Kumar. 2015. Structured transforms for small-footprint deep learning. In Proceedings of the 29th International Conference on Neural Information Processing Systems-Volume 2. 3088–3096.
  138. 10.1109/COMST.2021.3061981 · DOI · Paper page
  139. 10.1109/JSAC.2020.3000372 · DOI · Paper page
  140. Ayush Srivastava, Oshin Dutta, Jigyasa Gupta, Sumeet Agarwal, and Prathosh AP. 2021. A variational information bottleneck based method to compress sequential networks for human action recognition. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2745–2754.
  141. Y. Sui M. Yin Y. Gong and B. Yuan. 2024. Co-exploring structured sparsification and low-rank tensor decomposition for compact DNNs. IEEE Transactions on Neural Networks and Learning Systems. 1–13. 10.1109/TNNLS.2024.3408099 · DOI
  142. 10.1109/PCCC.2015.7410342 · DOI
  143. 10.1145/3470566 · doi:10.1145/3470566
  144. 10.1145/3397271.3401125 · DOI
  145. 10.1145/3466752.3480095 · DOI
  146. T. Tambe E. Y. Yang G. G. Ko Y. Chai C. Hooper M. Donato and G. Y. Wei. 2022. A 16-nm SoC for noise-robust speech and NLP edge AI inference with Bayesian sound source separation and attention-based DNNs. IEEE Journal of Solid-State Circuits 58 2 (2022) 569–581. · DOI
  147. Chong Min John Tan and Mehul Motani. 2020. Dropnet: Reducing neural network complexity via iterative pruning. In International Conference on Machine Learning. PMLR, 9356–9366.
  148. Mingxing Tan and Quoc Le. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning. PMLR, 6105–6114.
  149. Mingxing Tan and Quoc Le. 2021. EfficientNetV2: Smaller models and faster training. In International Conference on Machine Learning. PMLR, 10096–10106.
  150. Qinqin Tang, Renchao Xie, Fei Richard Yu, Tianjiao Chen, Ran Zhang, Tao Huang, and Yunjie Liu. 2022. Collective deep reinforcement learning for intelligence sharing in the internet of intelligence-empowered edge computing. IEEE Transactions on Mobile Computing 22, 11 (2022), 6327–6342.
  151. Zhiqing Tang, Weijia Jia, Xiaojie Zhou, Wenmian Yang, and Yongjian You. 2020. Representation and reinforcement learning for task scheduling in edge computing. IEEE Transactions on Big Data 8, 3 (2020), 795–808. · DOI
  152. Z. Tang J. Lou and W. Jia. 2022. Layer dependency-aware learning scheduling algorithms for containers in mobile edge computing. IEEE Transactions on Mobile Computing 22 6 (2022) 3444–3459. · DOI
  153. Z. Tang F. Mou J. Lou W. Jia Y. Wu and W. Zhao. 2023. Multi-user layer-aware online container migration in edge-assisted vehicular networks. IEEE/ACM Transactions on Networking 32 2 (2023) 1807–1822. · DOI
  154. Z. Tang F. Mou J. Lou W. Jia Y. Wu and W. Zhao. 2024. Joint resource overbooking and container scheduling in edge computing. IEEE Transactions on Mobile Computing 23 12 (2024) 10903–10917. · DOI
  155. Z. Tang F. Zhang X. Zhou W. Jia and W. Zhao. 2022. Pricing model for dynamic resource overbooking in edge computing. IEEE Transactions on Cloud Computing 11 2 (2022) 1970–1984. · DOI
  156. Gemma Team Morgane Riviere Shreya Pathak Pier Giuseppe Sessa Cassidy Hardin Surya Bhupatiraju Léonard Hussenot Thomas Mesnard Bobak Shahriari Alexandre Ramé and others. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118 (2024). · arXiv
  157. K. Ullrich E. Meeds and M. Welling. 2017. Soft Weight-sharing for neural network compression. In International Conference on Learning Representations.
  158. Laurens Van Der Maaten, Eric Postma, Jaap Van den Herik, and others. 2009. Dimensionality reduction: A comparative. Journal of Machine Learning Research 10, 66-71 (2009), 13.
  159. Laura Verde, Nadia Brancati, Giuseppe De Pietro, Maria Frucci, and Giovanna Sannino. 2021. A deep learning approach for voice disorder detection for smart connected living environments. ACM Transactions on Internet Technology (TOIT) 22, 1 (2021), 1–16. · DOI
  160. Gaurav Verma, Yashi Gupta, Abid M. Malik, and Barbara Chapman. 2021. Performance evaluation of deep learning compilers for edge inference. In 2021 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW’21). IEEE, 858–865. · DOI
  161. 10.1109/CVPR.2019.00881 · DOI · Paper page
  162. Ke Wang, Ship Peng Xu, Chien-Ming Chen, SK Hafizul Islam, Mohammad Mehedi Hassan, Claudio Savaglio, Pasquale Pace, and Gianluca Aloi. 2021. A trusted consensus scheme for collaborative learning in the edge AI computing domain. IEEE Network 35, 1 (2021), 204–210. · DOI
  163. 10.1109/TASLP.2022.3182856 · DOI
  164. Siqi Wang, Gayathri Anantanarayanan, Yifan Zeng, Neeraj Goel, Anuj Pathania, and Tulika Mitra. 2019. High-throughput CNN inference on embedded arm big. little multicore processors. IEEE Transactions on Computer-aided Design of Integrated Circuits and Systems 39, 10 (2019), 2254–2267.
  165. Shibo Wang, Shusen Yang, and Cong Zhao. 2020. SurveilEdge: Real-time video query based on collaborative cloud-edge deep learning. In IEEE Conference on Computer Communications (IEEE INFOCOM’20). IEEE, 2519–2528.
  166. 10.1109/TII.2019.2938861 · DOI · Paper page
  167. 10.1109/TCSI.2022.3184115 · DOI
  168. 10.1145/2966986.2967068 · DOI
  169. 10.1109/TCSI.2022.3184175 · DOI
  170. Pete Warden and Daniel Situnayake. 2019. TinyML: Machine Learning with Tensorflow Lite on Arduino and Ultra-low-power Microcontrollers. O’Reilly Media.
  171. 10.1016/j.neunet.2019.11.018 · DOI
  172. 10.1007/s00778-022-00775-9 · DOI · Paper page
  173. Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, and Kurt Keutzer. 2019. FFNet: Hardware-aware efficient ConvNet design via differentiable neural architecture search. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10734–10742.
  174. Junru Wu, Yue Wang, Zhenyu Wu, Zhangyang Wang, Ashok Veeraraghavan, and Yingyan Lin. 2018. Deep k-means: Re-training and parameter sharing with harder cluster assignments for compressing deep convolutions. In International Conference on Machine Learning. PMLR, 5363–5372.
  175. Yirui Wu, Haifeng Guo, Chinmay Chakraborty, Mohammad R. Khosravi, Stefano Berretti, and Shaohua Wan. 2022. Edge computing driven low-light image dynamic enhancement for object detection. IEEE Transactions on Network Science and Engineering 10, 5 (2022), 3086–3098.
  176. 10.1016/j.sysarc.2021.101991 · DOI
  177. 10.1145/3477495.3531775 · DOI
  178. Yecheng Xiang and Hyoseung Kim. 2019. Pipelined data-parallel CPU/GPU scheduling for multi-DNN real-time inference. In 2019 IEEE Real-time Systems Symposium (RTSS’19). IEEE, 392–405. · DOI · Paper page
  179. Jinbo Xiong, Mingfeng Zhao, Md. Zakirul Alam Bhuiyan, Lei Chen, and Youliang Tian. 2019. An AI-enabled three-party game framework for guaranteed data privacy in mobile edge crowdsensing of IoT. IEEE Transactions on Industrial Informatics 17, 2 (2019), 922–933. · doi:10.1109/tii.2019.2957130
  180. Dianlei Xu, Tong Li, Yong Li, Xiang Su, Sasu Tarkoma, Tao Jiang, Jon Crowcroft, and Pan Hui. 2020. Edge intelligence: Architectures, challenges, and applications. arXiv preprint arXiv:2003.12172 (2020). · arXiv
  181. 10.1109/JPROC.2021.3119950 · DOI · Paper page
  182. Jiajun Xu, Zhiyuan Li, Wei Chen, Qun Wang, Xin Gao, Qi Cai, and Ziyuan Ling. 2024. On-device language models: A comprehensive review. arXiv preprint arXiv:2409.00088 (2024). · arXiv
  183. Zirui Xu, Fuxun Yu, Zhuwei Qin, Chenchen Liu, and Xiang Chen. 2020. Directx: Dynamic resource-aware CNN reconfiguration framework for real-time mobile applications. IEEE Transactions on Computer-aided Design of Integrated Circuits and Systems 40, 2 (2020), 246–259.
  184. 10.1145/3625687.3625793 · DOI
  185. Huanrui Yang, Minxue Tang, Wei Wen, Feng Yan, Daniel Hu, Ang Li, Hai Li, and Yiran Chen. 2020. Learning low-rank deep neural networks via singular vector orthogonality regularization and singular value sparsification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 678–679.
  186. Honghong Yang, Jinming Wen, Xiaojun Wu, Li He, and Shahid Mumtaz. 2019. An efficient edge artificial intelligence multipedestrian tracking method with rank constraint. IEEE Transactions on Industrial Informatics 15, 7 (2019), 4178–4188. · DOI · Paper page
  187. Qing Yang and Hai Li. 2020. BitSystolic: A 26.7 TOPS/W 2b~ 8b NPU with configurable data flows for edge devices. IEEE Transactions on Circuits and Systems I: Regular Papers 68, 3 (2020), 1134–1145.
  188. Haoran You, Baopu Li, Shi Huihong, Yonggan Fu, and Yingyan Lin. 2022. ShiftAddNAS: Hardware-inspired search for more accurate and efficient neural networks. In International Conference on Machine Learning. PMLR, 25566–25580.
  189. Yunxuan Yu, Tiandong Zhao, Kun Wang, and Lei He. 2020. Light-OPU: An FPGA-based overlay processor for lightweight convolutional neural networks. In Proceedings of the 2020 ACM/SIGDA International Symposium on Field-programmable Gate Arrays. 122–132.
  190. 10.1145/3636534.3649361 · DOI
  191. Muhammad Zawish, Nouman Ashraf, Rafay Iqbal Ansari, and Steven Davy. 2022. Energy-aware AI-driven framework for edge-computing-based IoT applications. IEEE Internet of Things Journal 10, 6 (2022), 5013–5023. · DOI
  192. 10.1109/MWC.004.2300479 · DOI
  193. 10.1016/j.sysarc.2020.101713 · DOI
  194. 10.1145/3081333.3081336 · DOI
  195. Daochen Zha, Zaid Pervaiz Bhat, Kwei-Herng Lai, Fan Yang, Zhimeng Jiang, Shaochen Zhong, and Xia Hu. 2023. Data-centric artificial intelligence: A survey. arXiv preprint arXiv:2303.10158 (2023). · arXiv
  196. 10.1109/JIOT.2022.3154605 · DOI
  197. Jun Zhang and Khaled B. Letaief. 2019. Mobile edge intelligence and computing for the internet of vehicles. Proceedings of the IEEE 108, 2 (2019), 246–261. · DOI · Paper page
  198. Jie Zhang, Xiaolong Wang, Dawei Li, and Yalin Wang. 2018. Dynamically hierarchy revolution: Dirnet for compressing recurrent neural network on mobile devices. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. 3089–3096.
  199. Linfeng Zhang, Jiebo Song, Anni Gao, Jingwei Chen, Chenglong Bao, and Kaisheng Ma. 2019. Be your own teacher: Improve the performance of convolutional neural networks via self distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 3713–3722.
  200. 10.1109/CVPR.2018.00716 · DOI · Paper page

200 of 215 shown

Metadata

Authors
Wang, X., Tang, Z., Guo, J., Meng, T., Wang, C., Wang, T., & Jia, W.
Venue
ACM Computing Surveys
Published
2025-04-04
Volume
57
Issue
9
Pages
1-39
ISSN
  • 0360-0300 (ISSN-L)
  • 1557-7341
OpenAlex
W4408529725
Cited by
202
References
41
FWCI
89.04
Type
journal article
Language
en
Open access
green

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models. https://app.cochat.ai/papers/10.1145/3724420
The paper
Wang, X., Tang, Z., Guo, J., Meng, T., Wang, C., Wang, T., & Jia, W. (2025). Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models. ACM Computing Surveys. https://doi.org/10.1145/3724420

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.