Dynamic Adaptive DNN Surgery for Inference Acceleration on the Edge
The DOI matches its Crossref record; 22 of the 23 references it cites were found in a registry or an index.
Abstract
Recent advances in deep neural networks (DNNs) have substantially improved the accuracy and speed of a variety of intelligent applications. Nevertheless, one obstacle is that DNN inference imposes heavy computation burden to end devices, but offloading inference tasks to the cloud causes transmission of a large volume of data. Motivated by the fact that the data size of some intermediate DNN layers is significantly smaller than that of raw input data, we design the DNN surgery, which allows partitioned DNN processed at both the edge and cloud while limiting the data transmission. The challenge is twofold: (1) Network dynamics substantially influence the performance of DNN partition, and (2) State-of-the-art DNNs are characterized by a directed acyclic graph (DAG) rather than a chain so that partition is greatly complicated. In order to solve the issues, we design a Dynamic Adaptive DNN Surgery (DADS) scheme, which optimally partitions the DNN under different network condition. Under the lightly loaded condition, DNN Surgery Light (DSL) is developed, which minimizes the overall delay to process one frame. The minimization problem is equivalent to a min-cut problem so that a globally optimal solution is derived. In the heavily loaded condition, DNN Surgery Heavy (DSH) is developed, with the objective to maximize throughput. However, the problem is NP-hard so that DSH resorts an approximation method to achieve an approximation ratio of 3. Real-world prototype based on self-driving car video dataset is implemented, showing that compared with executing entire the DNN on the edge and cloud, DADS can improve latency up to 6.45 and 8.08 times respectively, and improve throughput up to 8.31 and 14.01 times respectively.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.690 · Cited by 19,892 · cited 4×
“For example, the input data size of tiny YOLOv2 [4] is 0 .”
BackgroundInfluentialInception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636 · cited 1×
“6 shows the DAG of the pure inception v4 network [10] in Fig.”
BackgroundAn experimental comparison of min-cut/max- flow algorithms for energy minimization in vision
Boykov, Y., & Kolmogorov, V. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.60 · Cited by 4,577
MAUI: making smartphones last longer with code offload
Cuervo, E., Balasubramanian, A., Cho, D.-K., Wolman, A., Saroiu, S., Chandra, R., & Bahl, P. (2010) · Proceedings of the 8th international conference on Mobile systems, applications, and services · doi:10.1145/1814433.1814441 · Cited by 2,281
Dynamic adaptive streaming over HTTP -: standards and design principles
Stockhammer, T. (2011) · Proceedings of the second annual ACM conference on Multimedia systems · doi:10.1145/1943552.1943572 · Cited by 1,221 · cited 1×
“This can be realized by a method similar to HTTP DASH [11].”
MethodDeep neural networks for small footprint text-dependent speaker verification
Variani, E., Lei, X., McDermott, E., Moreno, I. L., & Gonzalez-Dominguez, J. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6854363 · Cited by 1,035
Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices
Teerapittayanon, S., McDanel, B., & Kung, H. T. (2017) · 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2017.226 · Cited by 843
(2011) · Cited by 713
Cited by
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455
“[80] propose an approximation algorithm that provides worst-case performance guarantee, based on the graph min-cut method.”
MethodModel Pruning Enables Efficient Federated Learning on Edge Devices
Jiang, Y., Wang, S., Valls, V., Ko, B. J., Lee, W.-H., Leung, K. K., & Tassiulas, L. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3166101 · Cited by 589
“, knowledge distillation [36], runtime neural pruning [37], and DNN partitioning and offloading [38], do not serve our purpose.”
BackgroundUnleashing the Power of Edge-Cloud Generative AI in Mobile Networks: A Survey of AIGC Services
Xu, M., Du, H., Niyato, D., Kang, J., Xiong, Z., Mao, S., Han, Z., Jamalipour, A., Kim, D. I., Shen, X., et al. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3353265 · Cited by 449
“AIGC service provisioning based on edge intelligence is computationally and communication-intensive for resourceconstrained edge servers and mobile devices [132].”
Holistic Network Virtualization and Pervasive Network Intelligence for 6G
Shen, X., Gao, J., Wu, W., Li, M., Zhou, C., & Zhuang, W. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3135829 · Cited by 392
SPINN: synergistic progressive inference of neural networks over device and cloud
Laskaridis, S., Venieris, S. I., Almeida, M., Leontiadis, I., & Lane, N. D. (2020) · Proceedings of the 26th Annual International Conference on Mobile Computing and Networking · doi:10.1145/3372224.3419194 · Cited by 310
“on [34], partitions the CNN between a device-mapped head and a cloud-mapped tail and selects a single split point based on the device and cloud load as well as the network conditions. Similarly, DADS [22] tackles CNN offloading, but from a scheduler-centric standpoint, with the aim to yield the optimal partitioning sc…”
BackgroundTowards edge computing in intelligent manufacturing: Past, present and future
Nain, G., Pattanaik, K. K., & Sharma, G. K. (2022) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2022.01.010 · Cited by 283
Edge Intelligence in Intelligent Transportation Systems: A Survey
Gong, T., Zhu, L., Yu, F. R., & Tang, T. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3275741 · Cited by 256
Energy-Efficient Offloading for DNN-Based Smart IoT Systems in Cloud-Edge Environments
Chen, X., Zhang, J., Lin, B., Chen, Z., Wolter, K., & Min, G. (2022) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3100298 · Cited by 244
Generative AI-Driven Semantic Communication Networks: Architecture, Technologies, and Applications
Liang, C., Du, H., Sun, Y., Niyato, D. T., Kang, J., Zhao, D., & Imran, M. A. (2024) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2024.3435524 · Cited by 245
“To accommodate these resource demands, advanced strategies for computing resource allocation, such as distributed computing and edge computing, must be employed [178], [179].”
End-Edge-Cloud Collaborative Computing for Deep Learning: A Comprehensive Survey
Wang, Y., Yang, C., Lan, S., Zhu, L., & Zhang, Y. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3393230 · Cited by 237
“Inlight of the above issues, Hu et al. [174] researched the layer-level partition of DLMs characterized by DAGs.”
InfluentialMobile Edge Intelligence for Large Language Models: A Contemporary Survey
Qu, G., Chen, Q., Wei, W., Lin, Z., Chen, X., & Huang, K. (2025) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2025.3527641 · Cited by 235
“Similar problems have been considered for multi-hop split inference [235], [236], where the model splitting and/or placement is often mapped to a graph, which can be further mapped to shortest path problem [236] or min-cut problem [235].”
EdgeShard: Efficient LLM Inference via Collaborative Edge Computing
Zhang, M., Shen, X., Cao, J., Cui, Z., & Jiang, S. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3524255 · Cited by 220
“Several studies [26], [27], [28], [29] explore the distribution of DNN Authorized licensed use limited to the terms of the applicable license agreement with IEEE.”
Distributed Inference Acceleration with Adaptive DNN Partitioning and Offloading
Mohammed, T., Joe-Wong, C., Babbar, R., & Francesco, M. D. (2020) · IEEE INFOCOM 2020 - IEEE Conference on Computer Communications · doi:10.1109/infocom41043.2020.9155237 · Cited by 211
“As a consequence, distributing computations to devices nearby is a better option than offloading them entirely to the cloud [3, 7].”
BackgroundMethodInfluentialBaccour, E., Mhaisen, N., Abdellatif, A. A., Erbad, A., Mohamed, A., Hamdi, M., & Guizani, M. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3200740 · Cited by 186
“Authors in [213] designed a Dynamic Adaptive”
Cost-Driven Off-Loading for DNN-Based Applications Over Cloud, Edge, and End Devices
Lin, B., Huang, Y., Zhang, J., Hu, J., Chen, X., & Li, J. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2961237 · Cited by 159
“[8] proposed a dynamic adaptive DNN surgery scheme that allowed DNN layers to be executed in both the cloud and the edge while limiting the data transmission under different network condition.”
Li, J., Liang, W., Li, Y., Xu, Z., Jia, X., & Guo, S. (2021) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3125949 · Cited by 157
“…DNN partitioning method, which has been widely adopted for DNN model inferences as the amounts of output data in some intermediate layers of a DNN model are significantly smaller than that of its raw input data [9], thereby reducing the delay of data uploading from its IoT device to the MEC network.”
BackgroundMethodResultInfluentialAuto-Split: A General Framework of Collaborative Edge-Cloud AI
Banitalebi-Dehkordi, A., Vedula, N., Pei, J., Xia, F., Wang, L., & Zhang, Y. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467078 · Cited by 146
“DADS [27]: min-cut on un-optimized DNN.”
InfluentialWang, H., Zhang, W., Yang, D., & Xiang, Y. (2022) · IEEE Systems Journal · doi:10.1109/jsyst.2022.3193200 · Cited by 141
“Restrictions apply. space [91].”
A Survey on Collaborative DNN Inference for Edge Intelligence
Ren, W., Qu, Y., Dong, C., Jing, Y., Sun, H., Wu, Q., & Guo, S. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1391-7 · Cited by 137
“Song et al. [49] also propose a collaborative inference system based on edgent , which applies early-exit mechanism and model division technology to solve the problem of EI in task flow scenarios.”
BackgroundEdge Video Analytics: A Survey on Applications, Systems and Enabling Techniques
Xu, R., Razavi, S., & Zheng, R. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3323091 · Cited by 134
“More fine-grained partitioning is considered in several works [120], [184], [224], [225], [226], [227], [228], [229], [230], [231], [232], [233], [234] by splitting neural network layers.”
BackgroundHe, W., Guo, S., Guo, S., Qiu, X., & Qi, F. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2981338 · Cited by 133
“Hu et al. [36] characterized the DL model by a directed acyclic graph, then designed DNN Surgery Light and DNN Surgery Heavy to minimize the delay under lightly and heavily loaded conditions, but they do not consider the accurate evaluation of E2E inference delay for DL tasks in partitions.”
MethodEnergy-Aware Inference Offloading for DNN-Driven Applications in Mobile Edge Clouds
Xu, Z., Zhao, L., Liang, W., RANA, O. F., Zhou, P., Xia, Q., Xu, W., & Wu, G. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.3032443 · Cited by 128
“Also, there are already excellent studies on partitioning specific DNNs into different tasks [15], [21], [24]; we thus consider the partitioning of DNNs out of the scope of this paper.”
BackgroundDeepSlicing: Collaborative and Adaptive CNN Inference With Low Latency
Zhang, S., Zhang, S., Qian, Z., Wu, J., Jin, Y., & Lu, S. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3058532 · Cited by 127
“model partitioning [18], [19], [20], [21], [22], [23], [24], [25], data partitioning [26], [27], [28], [29], [30] and domain specific hardware/tools [31], [32].”
BackgroundDDPQN: An Efficient DNN Offloading Strategy in Local-Edge-Cloud Collaborative Environments
Xue, M., Wu, H., Peng, G., & Wolter, K. (2021) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2021.3116597 · Cited by 125
Distributed Inference with Deep Learning Models across Heterogeneous Edge Devices
Hu, C., & Li, B. (2022) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom48880.2022.9796896 · Cited by 112
“[17] proposed a dynamic adaptive method to partition the model under different network conditions, and [18] tried to reduce the transmission size by encoding the features.”
MethodSelf-citationDeng, F., Xie, Z., Mao, W., Li, B., Shan, Y., Wei, B., & Zeng, H. (2021) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2022.108054 · Cited by 106
Edge-assisted Online On-device Object Detection for Real-time Video Analytics
Hanyao, M., Jin, Y., Qian, Z., Zhang, S., & Lu, S. (2021) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom42981.2021.9488741 · Cited by 103
Multi-exit DNN Inference Acceleration based on Multi-Dimensional Optimization for Edge Intelligence
Dong, F., Wang, H., Shen, D., Huang, Z., He, Q., Zhang, J., Wen, L., & Zhang, T. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3172402 · Cited by 103
“Under such a scheme, a large DNN can be sliced into multiple partitions, where shallow partitions are processed locally on mobile device and deep partitions are offloaded to edge servers or cloud servers to complete the remaining inference [11], [12], [13].”
BackgroundAccelerating Split Federated Learning Over Wireless Communication Networks
Xu, C., Li, J., Liu, Y., Ling, Y., & Wen, M. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3327372 · Cited by 102
“Under different channel conditions, optimal DNN splitting is determined for both delay minimization and throughput maximization in [28].”
MethodHiveMind: Towards Cellular Native Machine Learning Model Splitting
Wang, S., Zhang, X., Uchiyama, H., & Matsuda, H. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118403 · Cited by 100
Towards Real-time Cooperative Deep Inference over the Cloud and Edge End Devices
Zhang, S., Li, Y., Liu, X., Guo, S., Wang, W., Wang, J., Ding, B., & Wu, D. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3397315 · Cited by 99
“described in [9] is awed and might fail to nd the correct partition under practical settings.”
BackgroundMethodInfluentialReinforcement Learning Based Energy-Efficient Collaborative Inference for Mobile Edge Computing
Xiao, Y., Xiao, L., Wan, K., Yang, H., Zhang, Y., Wu, Y., & Zhang, Y. (2022) · IEEE Transactions on Communications · doi:10.1109/tcomm.2022.3229033 · Cited by 99
“Moreover, the mobile device has to offload all the high resolution raw perception data via the wireless channel to perform inference if the deep learning model is entirely deployed on the edge server, yielding significant transmission overhead [9].”
BackgroundZhang, L., Chen, L., & Xu, J. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450051 · Cited by 94
“Compared with the layer-wise statistical modeling method adopted in [15, 16, 19], this method provides more accurate estimations because it takes into account the inter-layer optimization.”
BackgroundMethodInfluentialPyramid: Enabling Hierarchical Neural Networks with Edge Computing
He, Q., Dong, Z., Chen, F., Deng, S., Liang, W., & Yang, Y. J. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3511990 · Cited by 90
“Taking this advantage, researchers are investigating ML model partitioning to strike a trade-off between inference delay and inference accuracy [18, 22, 58].”
BackgroundIntegrating Edge Intelligence and Blockchain: What, Why, and How
Wang, X., Ren, X., Qiu, C., Xiong, Z., Yao, H., & Leung, V. C. M. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3189962 · Cited by 89
“Model early-exit [59], [60], model selection [61], model partition [62], [63] and input filter-ing [64], [65] realize inference acceleration, further speeding up the deployment on edge devices with the limited memory.”
BackgroundAccelerating Deep Learning Inference via Model Parallelism and Partial Computation Offloading
Zhou, H., Li, M., Wang, N., Min, G., & Wu, J. (2022) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2022.3222509 · Cited by 85
“Hu et al. [39] abstracted a DNN as a DAG, and transformed the minimum delay problem into a min-cut optimization problem.”
MethodXue, M., Wu, H., Li, R., Xu, M., & Jiao, P. (2021) · IEEE Transactions on Green Communications and Networking · doi:10.1109/tgcn.2021.3111731 · Cited by 84
“As a promising method, the DNN migration plan can reduce the pressure on mobile devices by migrating extremely computation-intensive execution from resource-constrained devices to cloud/edge servers, thereby achieving DNN inference acceleration [7], [8].”
MethodReal-time neural network inference on extremely weak devices: agile offloading with explainable AI
Huang, K., & Gao, W. (2022) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3495243.3560551 · Cited by 84
“Existing NN partitioning schemes [31, 34, 36, 39, 42, 65], however, need to use an expensive Local NN to enforce feature sparsity and incur unacceptable computing latency on the local device.”
MethodPerformance Evaluation of Deep Learning Compilers for Edge Inference
Verma, G., Gupta, Y., Malik, A. M., & Chapman, B. (2021) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw52791.2021.00128 · Cited by 80
“Methods [15] have been developed to distribute the DNN’s computations between the cloud and the edge.”
MethodHao, Z., Xu, G., Luo, Y., Hu, H., An, J., & Mao, S. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3183098 · Cited by 77
“[35] depicted DNNs by direct acyclic graphs and adopted graphic algorithms to find the partitioning strategy.”
A Panda? No, It's a Sloth: Slowdown Attacks on Adaptive Multi-Exit Neural Network InferencePreprint
Hong, S., Kaya, Y., Modoranu, I.-V., & Dumitraş, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.02432 · Cited by 77
“Hu et al. (2019) showed a real-world prototype of an optimal model partitioning, which is based on a self-driving car video dataset, can improve latency and throughput of partitioned models on the cloud and edge by 6.5–14 × , respectively.”
BackgroundInfluentialReconfigurable Intelligent Surface for Green Edge Inference
Hua, S., Zhou, Y., Yang, K., Shi, Y., & Wang, K. (2021) · IEEE Transactions on Green Communications and Networking · doi:10.1109/tgcn.2021.3058657 · Cited by 74
Network-Aware Optimization of Distributed Learning for Fog Computing
Wang, S., Ruan, Y., Tu, Y., Wagle, S., Brinton, C. G., & Joe‐Wong, C. (2021) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2021.3075432 · Cited by 73
Joint Optimization With DNN Partitioning and Resource Allocation in Mobile Edge Computing
Dong, C., Hu, S., Chen, X., & Wen, W. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3116665 · Cited by 70
Efficient Execution of Deep Neural Networks on Mobile Devices with NPU
Tan, T., & Cao, G. (2021) · ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1145/3412382.3458272 · Cited by 69
“Model partition techniques have been studied in [17, 22, 30, 44], where the DNN model is split into different layers and executed in different places, such as CPU, GPU or cloud server.”
MethodNetwork-Aware Optimization of Distributed Learning for Fog Computing
Tu, Y., Ruan, Y., Wagle, S., Brinton, C. G., & Joe-Wong, C. (2020) · IEEE INFOCOM 2020 - IEEE Conference on Computer Communications · doi:10.1109/infocom41043.2020.9155372 · Cited by 68
“Other works have considered splitting deep neural network layers between fog devices and an edge server for faster inference [20], [21].”
BackgroundFan, W., Gao, L. H., Su, Y., Wu, F., & Liu, Y. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3237361 · Cited by 65
“Only a few studies [17], [21], [23] modeled the task arriving characteristic and task queuing problems for DL tasks, but research works [23] and [21] are not in the edge–cloud-assisted IoT environments while research work [17] only focused on a single BS.”
BackgroundInfluentialA Comprehensive Survey on Training Acceleration for Large Machine Learning Models in IoT
Wang, H., Qu, Z., Zhou, Q., Zhang, H., Luo, B., Xu, W., Guo, S., & Li, R. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3111624 · Cited by 64
Liang, H., Zhang, Z., Hu, C., Gong, Y., & Cheng, D. (2023) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2023.3342619 · Cited by 63
“To address this issue, researchers have proposed various initiatives to expedite the distributed computation of DNNs [103].”
BackgroundSelf-citationHiTDL: High-Throughput Deep Learning Inference at the Hybrid Mobile Edge
Wu, J. L., Wang, L., Pei, Q., Cui, X., Liu, F., & Yang, T. (2022) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2022.3195664 · Cited by 62
“While the former two have been explored by prior work [16], [17], [19], the question of how the amount”
BackgroundInfluentialSelf-citationToward Carbon-Neutral Edge Computing: Greening Edge AI by Harnessing Spot and Future Carbon Markets
Ma, H., Zhou, Z., Zhang, X., & Chen, X. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3268339 · Cited by 62
Multiuser Co-Inference With Batch Processing Capable Edge Server
Shi, W., Zhou, S., Niu, Z., Jiang, M., & Geng, L. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3192613 · Cited by 61
“However, for the promising DNN hardware accelerators such as GPUs and NPUs, [19]–[21] cannot be directly applied, since the computing resource allocation and inference task scheduling would be completely different due to batch processing.”
Towards cloud-edge collaborative online video analytics with fine-grained serverless pipelines
Zhang, M., Wang, F., Zhu, Y., Liu, J., & Wang, Z. (2021) · ACM Multimedia Systems Conference (MMSys) · doi:10.1145/3458305.3463377 · Cited by 61
“In particular, by leveraging video content-aware forecasting and fine-grained workload-aware runtime scheduling, the Controller of CEVAS solves a cloud-edge partitioning problem for multiple concurrent serverless pipelines with ever-changing input workloads.”
BackgroundSu, Y., Fan, W., Gao, L. H., Qiao, L., Liu, Y., & Wu, F. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3219058 · Cited by 60
“Hu et al. [9] concentrated on those DNN models characterized by a directed acyclic graph (DAG), and they designed different surgery schemes to partition a DNN between the edge and the cloud according to different work-load conditions.”
BackgroundMethodEnergy-Aware, Device-to-Device Assisted Federated Learning in Edge Computing
Li, Y., Liang, W., Li, J., Cheng, X., Yu, D., Zomaya, A. Y., & Guo, S. (2023) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2023.3277423 · Cited by 55
Joint Optimization of DNN Partition and Scheduling for Mobile Cloud Computing
Duan, Y., & Wu, J. (2021) · Proceedings of the International Conference on Parallel Processing · doi:10.1145/3472456.3472468 · Cited by 54
Xu, X., Hu, Y., Cui, G., Qi, L., Dou, W., & Cai, Z. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3567459 · Cited by 53
“Hu et al. [23] focused on DNNs with topology of directed acyclic graph, and designed an adaptive model partition algorithm based on max-min cut theory.”
Optimizing Job Offloading Schedule for Collaborative DNN Inference
Duan, Y., & Wu, J. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2023.3276937 · Cited by 52
DynO: Dynamic Onloading of Deep Neural Networks from Cloud to Device
Almeida, M., Laskaridis, S., Venieris, S. I., Leontiadis, I., & Lane, N. D. (2022) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3510831 · Cited by 52
“[22] introduced a scheduling scheme to optimize either latency or throughput, based on the server load.”
BackgroundOver-the-Air Split Machine Learning in Wireless MIMO Networks
Yang, Y., Zhang, Z., Tian, Y., Yang, Z., Huang, C., Zhong, C., & Wong, K. (2023) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2023.3242701 · Cited by 49
“However, the above works [9], [10], [11], [12], [13], [14], [15] all consider ideal communication among the computing nodes, which ignores the communication scheme design.”
BackgroundAuditing Membership Leakages of Multi-Exit Networks
Li, Z., Liu, Y., He, X., Yu, N., Backes, M., & Zhang, Y. (2022) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3548606.3559359 · Cited by 48
“Besides, there is a growing body of applications deploying multi-exit architectures to IoT scenarios and realtime systems [24, 27, 28, 34, 60].”
BackgroundLiu, J., & Zhang, Q. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2984333 · Cited by 47
“Hu et al. [16] dynamically partitioned the DNN to optimize the overall latency and throughput for the scenarios with light workload and heavy workload, respectively.”
BackgroundPantheon: Preemptible Multi-DNN Inference on Mobile Edge GPUs
Han, L., Zhou, Z., & Li, Z. (2024) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3643832.3661878 · Cited by 46
“DNN can be represented as a directed acyclic graph (DAG) [23].”
BackgroundZhang, M., Abdi, M., Dasari, V. R., & Restuccia, F. (2025) · Computer Networks · doi:10.1016/j.comnet.2025.111531 · Cited by 45
AccuMO: Accuracy-Centric Multitask Offloading in Edge-Assisted Mobile Augmented Reality
Kong, Z. J., Xu, Q., Meng, J., & Hu, Y. (2023) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3570361.3592531 · Cited by 43
“(3) Partial offloading splits the DNN model between the mobile device and the server, at an intermediate DNN layer that is significantly smaller than the input [9, 24, 36, 39, 43, 44].”
BackgroundPArtNNer: Platform-Agnostic Adaptive Edge-Cloud DNN Partitioning for Minimizing End-to-End Latency
Ghosh, S. K., Raha, A., Raghunathan, V., & Raghunathan, A. (2023) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3630266 · Cited by 44
“N e u r o s u r g e o n [ 37 ] D ee p D e c i s i o n [ 63 ] A DD A [ 73 ] D A D S [ 31 ] J o i n t O p t i m i z e [ 15 ] R e s o u r c e - a w a r e [ 83 ] Characterization free Evidently, these challenging and diverse factors highlight the need for a platform-agnostic DNN partitioning system…”
BackgroundMethodA Decade of Video Analytics at Edge: Training, Deployment, Orchestration, and Platforms
Bai, T., Zhao, H., Huang, L., Wang, Z., Kim, D. I., & Nallanathan, A. (2025) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2025.3563377 · Cited by 43
“DAG-based methods use the Directed Acyclic Graph (DAG) to represent DNN, to optimize the distribution of computations across multiple devices [171], [172], [173], where vertices represent layers and edges represent the data flow between layers.”
Edge intelligence based Economic Dispatch for Virtual Power Plant in 5G Internet of Energy
Fang, D., Guan, X., Lin, L., Yu, P., Sun, D., & Hassan, M. M. (2019) · Computer Communications · doi:10.1016/j.comcom.2019.12.021 · Cited by 43
Scission: Performance-driven and Context-aware Cloud-Edge Distribution of Deep Neural Networks
Lockhart, L., Harvey, P., Imai, P., Willis, P., & Varghese, B. (2020) · IEEE/ACM International Conference on Utility and Cloud Computing (UCC) · doi:10.1109/ucc48980.2020.00044 · Cited by 43
“Other approaches, such as approximation-based are also considered in the literature [27], but are not considered here.”
BackgroundDistributed Assignment With Load Balancing for DNN Inference at the Edge
Xu, Y., Mohammed, T., Francesco, M. D., & Fischione, C. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3205410 · Cited by 43
“In contrast, end devices exchange data with edge servers in their communication range over a shared wireless channel (i.e., through WiFi) with a total bandwidth of B , according to [16].”
BackgroundMethodInfluentialInference Time Optimization Using BranchyNet Partitioning
Pacheco, R. G., & Couto, R. S. (2020) · IEEE Symposium on Computers and Communications (ISCC) · doi:10.1109/iscc50000.2020.9219647 · Cited by 40
“These average uplink rates used in this work are based on the values presented in [6].”
MethodInfluentialTorres, D. R., Martín, C., Rubio, B., & Díáz, M. (2021) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2021.102214 · Cited by 40
“An adaptive surgery [25] scheme dynamically splits DDNN between the Edge and the Cloud to optimize both the latency and throughput under variable network conditions.”
MethodScheduling Massive Camera Streams to Optimize Large-Scale Live Video Analytics
Rong, C., Wang, J. H., Liu, J., Wang, J., Li, F., & Huang, X. (2021) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2021.3125359 · Cited by 40
Wang, Z., Xu, H., Yang, X., Jiang, Z., & Liu, J. (2022) · Computer Networks · doi:10.1016/j.comnet.2022.109490 · Cited by 40
Enabling Low Latency Edge Intelligence based on Multi-exit DNNs in the Wild
Huang, Z., Dong, F., Shen, D., Zhang, J., Wang, H., Cai, G., & He, Q. (2021) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs51616.2021.00075 · Cited by 39
“Furthermore, DADS [4] proposed a dynamic DNN surgery scheme to split a DNN into two partitions to execute the DNN at the edge and cloud.”
Multi-Compression Scale DNN Inference Acceleration based on Cloud-Edge-End Collaboration
Qi, H., Ren, F., Wang, L., Jiang, P., Wan, S., & Deng, X. (2024) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3634704 · Cited by 39
“For DNNs, intermediate data may have smaller dimensions than the source data [19, 24].”
BackgroundStochastic Cumulative DNN Inference With RL-Aided Adaptive IoT Device-Edge Collaboration
Qu, K., Zhuang, W., Wu, W. R., Li, M., Shen, X., Li, X., & Shi, W. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3280746 · Cited by 38
“to reduce the total inference delay and (or) the total energy consumption for both communication and computing, by using delay/energy performance regression techniques, optimization models, and RL approaches [12], [13], [14], [15], [16], [17], [18], [19].”
BackgroundMethodProvisioning Edge Inference as a Service via Online Learning
Jin, Y., Jiao, L., Qian, Z., Zhang, S., Chen, N., Lu, S., & Wang, X. (2020) · IEEE International Conference on Sensing, Communication, and Networking (SECON) · doi:10.1109/secon48991.2020.9158425 · Cited by 37
“[16] minimized the overall delay for partitioned deep inference at edges.”
BackgroundDISNET: Distributed Micro-Split Deep Learning in Heterogeneous Dynamic IoT
Samikwa, E., Maio, A. D., & Braun, T. I. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3313514 · Cited by 36
“1) where the model is partitioned layer by layer, and some of the layer computations are processed in devices and some at the server or other devices [19], [20], [21].”
BackgroundCalibration-Aided Edge Inference Offloading via Adaptive Model Partitioning of Deep Neural Networks
Pacheco, R. G., Couto, R. S., & Simeone, O. (2021) · International Conference on Communications · doi:10.1109/icc42927.2021.9500760 · Cited by 35
“A related idea that has found an application for cloud offloading is DNN model partitioning [3], [7], [8].”
BackgroundMethodHao, J., Subedi, P., Ramaswamy, L. M., & Kim, I. K. (2023) · ACM Transactions on Internet Technology · doi:10.1145/3546192 · Cited by 35
“Similarly, studies involving eicient partitioning of heavier models and subsequent oloading to the cloud [40, 44, 52] show how the edgecloud hybrid model could be leveraged for DL inference tasks.”
BackgroundELASTIC: Edge Workload Forecasting based on Collaborative Cloud-Edge Deep Learning
Li, Y., Yuan, H., Fu, Z., Ma, X., Xu, M., & Wang, S. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583436 · Cited by 35
“However, the current DNN partition is mainly applied to model inference [12, 19] and considered individually for each edge site, which require an dedicated offline profiling phase to measure the network condition, the processing ability of the edge site, and the computing capacity of cloud server [43].”
MethodPDD: Partitioning DAG-Topology DNNs for Streaming Tasks
Wu, L., Gao, G., Yu, J., Zhou, F., Yang, Y., & Wang, T. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3323520 · Cited by 35
“Notably, DADS [15] is a milestone min-cut-based bi-partitioning method, which, however, fails to figure out a meaningful solution in some settings..”
BackgroundMethodAdaKnife: Flexible DNN Offloading for Inference Acceleration on Heterogeneous Mobile Devices
Liu, S., Luo, H., Li, X., Li, Y., Guo, B., Yu, Z., Wang, Y., Ma, K., Ding, Y., & Yao, Y. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3466931 · Cited by 34
SEE: Scheduling Early Exit for Mobile DNN Inference during Service Outage
Wang, Z., Bao, W., Yuan, D., Ge, L., Tran, N. H. K., & Zomaya, A. Y. (2019) · ACM International Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems (MSWiM) · doi:10.1145/3345768.3355917 · Cited by 34
Self-citationAdaptive DNN Partition in Edge Computing Environments
Miao, W., Zeng, Z., Wei, L., Li, S., Jiang, C., & Zhang, Z. (2020) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads51040.2020.00097 · Cited by 34
“The authors in [13, 14] also proposed to split between data centers and edge devices and they proposed solutions based on the max flow min cut theorem.”
MethodTeng, M., Li, X., & Zhu, K. (2023) · IEEE Transactions on Sustainable Computing · doi:10.1109/tsusc.2023.3291365 · Cited by 34
“The DNN tasks can be divided, and the optimization of DNN-based task offloading is splitting the DNN into two parts, running locally or at one edge [33], or in the cloud [34].”
MethodGraft: Efficient Inference Serving for Hybrid Deep Learning With SLO Guarantees via DNN Re-Alignment
Wu, J. L., Wang, L., Jin, Q., & Liu, F. (2023) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2023.3340518 · Cited by 34
“DADS [51] explores partitioning for DNNs based on directed acyclic graph (DAG) (e.g., GoogleNet [76]) and proposes two DNN partitioning strategies for light and heavy network conditions, respectively.”
BackgroundSelf-citationA Survey on Deep Neural Network Partition over Cloud, Edge and End DevicesPreprint
Xu, D., He, X., Su, T., & Wang, Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.10020 · Cited by 34
Context-Aware Deep Model Compression for Edge Cloud Computing
Wang, L., Xiang, L., Xu, J., Chen, J., Zhao, X., Yao, D., Wang, X., & Li, B. (2020) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs47774.2020.00101 · Cited by 33
“[5] turned the problem into a min-cut problem and found the optimal partition for DNNs represented by Directed Acyclic Graphs.”
MethodContext-Aware Compilation of DNN Training Pipelines across Edge and Cloud
Yao, D., Xiang, L., Wang, Z., Xu, J., Li, C., & Wang, X. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3494981 · Cited by 33
“Previous works have shown a wide range of possibilities to perform joint learning between the edge and cloud, such as [10, 14, 17, 18, 22, 29, 46, 51].”
BackgroundMethodTowards Transmission-Friendly and Robust CNN Models over Cloud and Device
Ding, C., Lu, Z., Juefei-Xu, F., Boddeti, V. N., Li, Y., & Cao, J. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3186496 · Cited by 33
“[27], [28], [29], [30], [31], [32], [33] have been proposed.”
Hybrid SD: Edge-Cloud Collaborative Inference for Stable Diffusion ModelsPreprint
Yan, C., Liu, S., Liu, H., Peng, X., Wang, X., Chen, F., Fu, L., & Mei, X. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.06646 · Cited by 32
“We leverage the mean WiFi bandwidth of 18.88Mbps re-ported at (Hu et al. 2019).”
MethodGuardians of the Deep Fog: Failure-Resilient DNN Inference from Edge to Cloud
Yousefpour, A., Devic, S., Nguyen, B., Kreidieh, A., Liao, A., Bayen, A. M., & Jue, J. P. (2019) · International Workshop on Challenges in Artificial Intelligence and Machine Learning for Internet of Things (AIChallengeIoT) · doi:10.1145/3363347.3363366 · Cited by 31
“Recently, some IoT application scenarios have emerged that need ongoing and long inference tasks [9, 14, 18, 20, 28, 29].”
BackgroundMethodInfluentialTrine: Cloud-Edge-Device Cooperated Real-Time Video Analysis for Household Applications
Zhao, Y., Yang, Z., He, X., Cai, X., Miao, X., & Ma, Q. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3154721 · Cited by 31
“Another direction to reduce offloading latency is model partition [49], [50], [51], [52].”
BackgroundEarly-exit deep neural networks for distorted images: providing an efficient edge offloading
Pacheco, R. G., Oliveira, F. D. V. R., & Couto, R. S. (2021) · Globecom · doi:10.1109/globecom46510.2021.9685469 · Cited by 30
“DADS [11] (Dynamic Adaptive DNN Surgery) optimally partitions a DNN model, dealing with”
Ma, X., Zhao, J., & Gong, Y. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3107465 · Cited by 30
Context-aware Adaptive Surgery: A Fast and Effective Framework for Adaptative Model Partition
Wang, H., Guo, B., Liu, J., Liu, S., Wu, Y., & Yu, Z. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3478073 · Cited by 30
“The speed up ratio is defined as the formula (9), where 𝑇 Neurosearch denotes the search time of Neurosurgeon, and 𝑇 Met search denotes the search time of the Met approach (e.g., DADS, CAS KNN , CAS Kd , CAS).”
BackgroundMethodInfluentialTowards Edge Computing Using Early-Exit Convolutional Neural Networks
Pacheco, R. G., Bochie, K., Gilbert, M. S., Couto, R. S., & Campista, M. E. M. (2021) · Information · doi:10.3390/info12100431 · Cited by 30
“propose DADS (Dynamic Adaptive DNN Surgery), a system that models DNNs as graphs and handles DNN partitioning as a minimum cut problem [20].”
Content-Aware Adaptive Device–Cloud Collaborative Inference for Object Detection
Hu, Y., Li, Z., Chen, Y., Cheng, Y., Cao, Z., & Liu, J. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3279579 · Cited by 30
“DADS [16] design a Dynamic Adaptive DNN Surgery scheme, which optimally partitions the DNN under different network condition.”
BackgroundMethod
Metadata
- Authors
- Hu, C., Bao, W., Wang, D., & Liu, F.
- Affiliations
- Hong Kong Polytechnic University (HK)
- The University of Sydney (AU)
- Venue
- IEEE INFOCOM 2019 - IEEE Conference on Computer Communications
- Publisher
- IEEE
- Published
- 2019-04
- Pages
- 1423-1431
- OpenAlex
- W2920031528
- Cited by
- 416
- References
- 23
- Citations by year
- 2026: 29 · 2025: 61 · 2024: 65 · 2023: 92 · 2022: 62 · 2021: 72 · 2020: 26 · 2019: 9
- FWCI
- 36.52
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Conference on Computer Communications (INFOCOM) (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Cloud Computing and Remote Desktop Technologies
- Keywords
- DNN partitioning, cloud-edge collaboration, inference acceleration, dynamic network conditions, directed acyclic graphs, minimum cut, approximation algorithms, throughput maximization, latency reduction, data transmission reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.