Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing
Open accessThe DOI matches its Crossref record; 49 of the 51 references it cites were found in a registry or an index, and 2 are not checked yet.
Abstract
As a key technology of enabling Artificial Intelligence (AI) applications in 5G era, Deep Neural Networks (DNNs) have quickly attracted widespread attention. However, it is challenging to run computation-intensive DNN-based tasks on mobile devices due to the limited computation resources. What’s worse, traditional cloud-assisted DNN inference is heavily hindered by the significant wide-area network latency, leading to poor real-time performance as well as low quality of user experience. To address these challenges, in this paper, we proposeEdgent, a framework that leverages edge computing for DNN collaborative inference through device-edge synergy.Edgentexploits two design knobs: (1) DNN partitioning that adaptively partitions computation between device and edge for purpose of coordinating the powerful cloud resource and the proximal edge resource for real-time DNN inference; (2) DNN right-sizing that further reduces computing latency via early exiting inference at an appropriate intermediate DNN layer. In addition, considering the potential network fluctuation in real-world deployment,Edgentis properly design to specialize for both static and dynamic network environment. Specifically, in a static environment where the bandwidth changes slowly,Edgentderives the best configurations with the assist of regression-based prediction models, while in a dynamic environment where the bandwidth varies dramatically,Edgentgenerates the best execution plan through the online change point…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For vertical edge inference, the dynamic computation partition and early existing scheme was proposed in [82] to accelerate the inference speed via edge deviceserver synergy.”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [82]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
Edge computing: Vision and challenges
Shi, W., Cao, J., Zhang, Q., Li, Y., & Xu, L. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2579198 · Cited by 8,733
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016) · arXiv · doi:10.48550/arxiv.1602.07360 · Cited by 6,201
A Survey on Mobile Edge Computing: The Communication Perspective
Mao, Y., You, C., Zhang, J., Huang, K., & Letaief, K. B. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2745201 · Cited by 5,607 · cited 1×
“The principal idea of edge computing [6]–[8] is to sink the cloud computing capability from the network core to the network edges (e.g., base stations and WLAN) in close proximity to end devices [9]–[14].”
BackgroundWaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911
Han, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662
Mobile Edge Computing: A Survey on Architecture and Computation Offloading
Mach, P., & Becvar, Z. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2682318 · Cited by 3,258
Measurement Reliability and Validity
(2000) · Research Methods in Applied Settings · doi:10.4324/9781410605337-29 · Cited by 2,541
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455 · cited 1×
“By hosting artificial intelligence on mobile devices, mobile DNN computation deploys DNN models close to users in order to achieve more flexible execution as well as more secure interaction [15].”
MethodSelf-citationProgressive Neural Architecture Search
Liu, C., Zoph, B., Neumann, M., Shlens, J., Hua, W., Li, L.-J., Fei-Fei, L., Yuille, A., Huang, J., & Murphy, K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01246-5_2 · Cited by 1,833
Learning IoT in Edge: Deep Learning for the Internet of Things with Edge Computing
Li, H., Ota, K., & Dong, M. (2018) · IEEE Network · doi:10.1109/mnet.2018.1700202 · Cited by 1,566 · cited 1×
“Specifically, some frameworks [34]–[36] take advantage of DNN partitioning to optimize the computation offloading between the mobile devices and the cloud, while some framework target to distribute computation workload among mobile devices [37]–[39].”
Background(2018) · International Conference on Machine Learning · Cited by 1,465
Hybrid computing using a neural network with dynamic external memory
Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J., et al. (2016) · Nature · doi:10.1038/nature20101 · Cited by 1,412 · cited 1×
“Hardware acceleration generally embraces basic DNN computation operations in hardware level design [28]–[30] while some works aim at optimizing the utilization of the existing hardware resources [31]–[33].”
BackgroundBranchyNet: Fast inference via early exiting from deep neural networks
Teerapittayanon, S., McDanel, B., & Kung, H. T. (2016) · 2016 23rd International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr.2016.7900006 · Cited by 1,326
Quantized Convolutional Neural Networks for Mobile Devices
Wu, J., Leng, C., Wang, Y., Hu, Q., & Cheng, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.521 · Cited by 1,212
FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
Umuroglu, Y., Fraser, N. J., Gambardella, G., Blott, M., Leong, P., Jahre, M., & Vissers, K. (2017) · Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/3020078.3021744 · Cited by 1,189
Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge
Kang, Y., Hauswald, J., Gao, C., Rovinski, A., Mudge, T., Mars, J., & Tang, L. (2017) · ACM SIGARCH Computer Architecture News · doi:10.1145/3093337.3037698 · Cited by 806
Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge
Kang, Y., Hauswald, J., Gao, C., Rovinski, A., Mudge, T., Mars, J., & Tang, L. (2017) · Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/3037697.3037698 · Cited by 673 · cited 1×
“Specifically, some frameworks [34]–[36] take advantage of DNN partitioning to optimize the computation offloading between the mobile devices and the cloud, while some framework target to distribute computation workload among mobile devices [37]–[39].”
BackgroundBayesian Online Changepoint Detection
Adams, R. P., & MacKay, D. J. C. (2007) · arXiv · doi:10.48550/arxiv.0710.3742 · Cited by 601
Efficient Architecture Search by Network Transformation
Cai, H., Chen, T., Zhang, W., Yu, Y., & Wang, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11709 · Cited by 557
Follow Me at the Edge: Mobility-Aware Dynamic Service Placement for Mobile Edge Computing
Ouyang, T., Zhou, Z., & Chen, X. (2018) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2018.2869954 · Cited by 519
Self-citationEfficient Neural Architecture Search via Parameter Sharing
Pham, H., Guan, M. Y., Zoph, B., Le, Q. V., & Dean, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03268 · Cited by 517
Minerva: enabling low-power, highly-accurate deep neural network accelerators
Reagen, B., Whatmough, P., Adolf, R., Rama, S., Lee, H., Lee, S. K., Hernández-Lobato, J. M., Wei, G.-Y., & Brooks, D. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001165 · Cited by 446 · cited 1×
“Hardware acceleration generally embraces basic DNN computation operations in hardware level design [28]–[30] while some works aim at optimizing the utilization of the existing hardware resources [31]–[33].”
BackgroundEdge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy
Li, E., Zhou, Z., & Chen, X. (2018) · Proceedings of the 2018 Workshop on Mobile Edge Communications · doi:10.1145/3229556.3229562 · Cited by 385
Self-citationJointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services
Eshratifar, A. E., Abrishami, M. S., & Pedram, M. (2021) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2947893 · Cited by 381
A Long Short-Term Memory Model for Answer Sentence Selection in Question Answering
Wang, D., & Nyberg, E. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) · doi:10.3115/v1/p15-2116 · Cited by 353
HTTP/2-Based Adaptive Streaming of HEVC Video Over 4G/LTE Networks
van der Hooft, J., Petrangeli, S., Wauters, T., Huysegems, R., Alface, P. R., Bostoen, T., & De Turck, F. (2016) · IEEE Communications Letters · doi:10.1109/lcomm.2016.2601087 · Cited by 336
Oboe: auto-tuning video ABR algorithms to network conditions
Akhtar, Z., Nam, Y. S., Govindan, R., Rao, S., Chen, J., Katz-Bassett, E., Ribeiro, B., Zhan, J., & Zhang, H. (2018) · Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication · doi:10.1145/3230543.3230558 · Cited by 327
DeepEar: robust smartphone audio sensing in unconstrained acoustic environments using deep learning
Lane, N. D., Georgiev, P., & Qendro, L. (2015) · Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2750858.2804262 · Cited by 310
Minerva: Enabling Low-Power, Highly-Accurate Deep Neural Network Accelerators
Reagen, B., Whatmough, P., Adolf, R., Rama, S., Lee, H., Lee, S. K., Hernandez-Lobato, J. M., Wei, G.-Y., & Brooks, D. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.32 · Cited by 246
ThriftyEdge: Resource-Efficient Edge Computing for Intelligent IoT Applications
Chen, X., Shi, Q., Yang, L., & Xu, J. (2018) · IEEE Network · doi:10.1109/mnet.2018.1700145 · Cited by 201
Self-citationMEETS: Maximal Energy Efficient Task Scheduling in Homogeneous Fog Networks
Yang, Y., Wang, K., Zhang, G., Chen, X., Luo, X., & Zhou, M.-T. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2846644 · Cited by 189
Self-citationProgressive Neural Architecture Search
Liu, C., Zoph, B., Neumann, M., Shlens, J., Hua, W., Li, L.-J., Fei-Fei, L., Yuille, A., Huang, J., & Murphy, K. (2017) · arXiv · doi:10.48550/arxiv.1712.00559 · Cited by 171
Efficient Resource Allocation for On-Demand Mobile-Edge Cloud Computing
Chen, X., Li, W., Lu, S., Zhou, Z., & Fu, X. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2846232 · Cited by 145
Self-citationDeepWear: Adaptive Local Offloading for On-Wearable Deep Learning
Xu, M., Qian, F., Zhu, M., Huang, F., Pushp, S., & Liu, X. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2893250 · Cited by 138
Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications
Kim, Y., Park, E., Yoo, S., Choi, T., Lu, Y., & Shin, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06530 · Cited by 111
AAIoT: Accelerating Artificial Intelligence in IoT Systems
Zhou, J., Wang, Y., Ota, K., & Dong, M. (2019) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2019.2894703 · Cited by 97
Zygarde: Time-Sensitive On-Device Deep Inference and Adaptation on Intermittently-Powered Systems
Islam, B., & Nirjon, S. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3411808 · Cited by 97
Kim, Y., Kim, J., Chae, D., Kim, D., & Kim, J. (2019) · Proceedings of the Fourteenth EuroSys Conference 2019 · doi:10.1145/3302424.3303950 · Cited by 93
Distilling at the Edge: A Local Differential Privacy Obfuscation Framework for IoT Data Analytics
Xu, C., Ren, J., Zhang, D., & Zhang, Y. (2018) · IEEE Communications Magazine · doi:10.1109/mcom.2018.1701080 · Cited by 79
ApNet: Approximation-Aware Real-Time Neural Network
Bateni, S., & Liu, C. (2018) · IEEE Real-Time Systems Symposium (RTSS) · doi:10.1109/rtss.2018.00017 · Cited by 77
MobiRNN: Efficient Recurrent Neural Network Execution on Mobile GPU
Cao, Q., Balasubramanian, N., & Balasubramanian, A. (2017) · Proceedings of the 1st International Workshop on Deep Learning for Mobile Systems and Applications · doi:10.1145/3089801.3089804 · Cited by 49
Software-Defined System Support for Enabling Ubiquitous Mobile Edge Computing
Jararweh, Y., Alsmirat, M. A., Al‐Ayyoub, M., Benkhelifa, E., Darabseh, A., Gupta, B. B., & Doulat, A. (2017) · The Computer Journal · doi:10.1093/comjnl/bxx019 · Cited by 47
Lane, N., Bhattacharya, S., Mathur, A., Forlivesi, C., & Kawsar, F. (2016) · Proceedings of the The 8th EAI International Conference on Mobile Computing, Applications and Services · doi:10.4108/eai.30-11-2016.2267463 · Cited by 33
Musical Chair: Efficient Real-Time Recognition Using Collaborative IoT DevicesPreprint
Hadidi, R., Cao, J., Woodward, M., Ryoo, M. S., & Kim, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.02138 · Cited by 28
An Online Intrusion Detection System to Cloud Computing Based on Neucube Algorithms
Almomani, A., Alauthman, M., Albalas, F. A., Dorgham, O. M., & Obeidat, A. A. (2018) · International Journal of Cloud Applications and Computing · doi:10.4018/ijcac.2018040105 · Cited by 24
Enabling Cooperative Inference of Deep Learning on Wearables and Smartphones.Preprint
Xu, M., Qian, F., & Pushp, S. (2017) · arXiv (Cornell University) · Cited by 19
Zygarde: Time-Sensitive On-Device Deep Intelligence on Intermittently-Powered Systems.Preprint
Islam, B., Luo, Y., & Nirjon, S. (2019) · arXiv (Cornell University) · Cited by 8
Research works printed but not linked: 5
- 2017, Augmented and Virtual Reality: The First Wave of 5G Killer Apps
- Iandola, 2016, SqueezeNet: AlexNet-level accuracy with 50 $\times$ fewer parameters and <0.5 MB model size
- Tokui, Chainer: A next-generation open source framework for deep learning, 1
- Mulhollon, 2004, Wondershaper
- Kulick, 2013, Bayesian Changepoint Detection
Cited by
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“For vertical edge inference, the dynamic computation partition and early existing scheme was proposed in [82] to accelerate the inference speed via edge device-server synergy.”
MethodSingh, R., & Gill, S. S. (2023) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2023.02.004 · Cited by 572
A comprehensive survey on model compression and acceleration
Choudhary, T., Mishra, V., Goswami, A., & Sarangapani, J. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09816-7 · Cited by 513
Communication-efficient edge ai: algorithms and systems
Shi, Y., Yang, K., Jiang, T., Zhang, J., & Letaief, K. B. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3007787 · Cited by 488
Luo, S., Chen, X., Wu, Q., Zhou, Z., & Yu, S. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3003744 · Cited by 455
Self-citation
Metadata
- Authors
- Li, E., Zeng, L., Zhou, Z., & Chen, X.
- Affiliations
- Sun Yat-sen University (CN)
- Venue
- IEEE Transactions on Wireless Communications
- Published
- 2020-01
- Volume
- 19
- Issue
- 1
- Pages
- 447-457
- ISSN
- 1536-1276 (ISSN-L)
- 1558-2248
- OpenAlex
- W2980856918
- Cited by
- 929
- References
- 50
- Citations by year
- 2026: 133 · 2025: 177 · 2024: 159 · 2023: 157 · 2022: 131 · 2021: 116 · 2020: 56
- FWCI
- 41.34
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Wireless Communications (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (U1711265, 61972432, 61802449)
- Program for Guangdong Introducing Innovative and Entrepreneurial Teams (2017ZT07X355)
- Pearl River Talent Recruitment Program (2017GC010465)
- Guangdong Natural Science Funds (2018A030313032)
- Fundamental Research Funds for the Central Universities (17lgjc40)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Keywords
- edge AI, deep neural network inference, edge computing, DNN partitioning, early exit, change point detection, low-latency inference, mobile devices, 5G
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.