HGA-DP: Optimal Partitioning of Multimodal DNNs Enabling Real-Time Image Inference for AR-Assisted Communication Maintenance on Cloud-Edge-End Systems
Open accessAbstract
In the field of communication maintenance, Augmented Reality (AR) applications are critical for enhancing operational safety and efficiency. However, deploying the required multimodal models on resource-constrained terminal devices is challenging, as traditional cloud or on-device strategies fail to balance low latency and energy consumption. This paper proposes a Cloud-Edge-End collaborative inference framework tailored to multimodal model deployment. A subgraph partitioning strategy is introduced to systematically decompose complex multimodal models into functionally independent sub-units. Subsequently, a fine-grained performance estimation model is employed to accurately characterize both computation and communication costs across heterogeneous devices. And, a joint optimization problem is formulated to minimize end-to-end inference latency and terminal energy consumption. To solve this problem efficiently, a Hybrid Genetic Algorithm for DNN Partitioning (HGA-DP) evolved over 100 generations is designed, incorporating constraint-aware repair mechanisms and local neighborhood search to navigate the exponential search space of possible deployment combinations. Experimental results on a simulated three-tier collaborative computing platform demonstrate that, compared to traditional full on-device deployment, the proposed method reduces end-to-end inference latency by 70–80% and terminal energy consumption by 81.1%, achieving a 4.86× improvement in overall fitness score. Against the latency-optimized DADS heuristic, HGA-DP achieves 41.3% lower latency while reducing energy by 59.9%. Compared to the All-Cloud strategy, our approach delivers 71.5% latency reduction with only marginal additional terminal energy cost. This framework provides an adaptive and effective solution for real-time multimodal inference in resource-constrained scenarios, laying a foundation for intelligent, resource-aware deployment.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AR technology's use in a blended-reality approach is considered effective for administering CBT, particularly for SAD.
“While AR technology has been developed for various applications in healthcare and education, its use in a blended-reality approach is considered effective for administering CBT, particularly for SAD [28,29,30,31,32].”
From Advancing Undergraduate Student Mental Healthcare of Social Anxiety Disorder: Evaluating the Acceptance of AR-Assisted Cognitive Behavioral Therapy Through TAM-Based Constructs. (2026) · cites this paper as [29]
The Introduction and Conclusions discuss AR for communication-maintenance technicians and the deployment of multimodal models, not therapy. Searches for blended reality, CBT, and social anxiety disorder found no passage addressing whether AR is effective for administering CBT for SAD. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Mobile cloud computing: A survey
Fernando, N., Loke, S. W., & Rahayu, W. J. (2012) · Future Generation Computer Systems · doi:10.1016/j.future.2012.05.023 · Cited by 1,663 · see citing works
Joint Optimization of Radio and Computational Resources for Multicell Mobile-Edge Computing
Sardellitti, S., Scutari, G., & Barbarossa, S. (2015) · IEEE Transactions on Signal and Information Processing over Networks · doi:10.1109/tsipn.2015.2448520 · Cited by 889 · cited 1× · see citing works
“This heterogeneity means that cost models based on simple data volume are no longer sufficient, requiring a more nuanced, modality-aware approach to balancing computation and communication [31,32].”
Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices
Teerapittayanon, S., McDanel, B., & Kung, H. T. (2017) · 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2017.226 · Cited by 843 · see citing works
Kang, Y., Hauswald, J., Gao, C., Rovinski, A., Mudge, T., Mars, J., & Tang, L. (2017) · ACM SIGARCH Computer Architecture News · doi:10.1145/3093337.3037698 · Cited by 811 · see citing works
Siriwardhana, Y., Porambage, P., Liyanage, M., & Ylianttila, M. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3061981 · Cited by 697 · see citing works
Edge Computing with Artificial Intelligence: A Machine Learning Perspective
Hua, H., Li, Y., Wang, T., Dong, N., Li, W., & Cao, J. (2022) · ACM Computing Surveys · doi:10.1145/3555802 · Cited by 594 · cited 1× · see citing works
“…of the cloud for heavy, non-time-sensitive computations; the low-latency processing power of edge servers for intermediate, time-critical tasks; and the immediate data access and local processing capabilities of end devices for lightweight preprocessing or executing initial model layers [13,14].”
Deep Multimodal Representation Learning: A Survey
Guo, W., Wang, J., & Wang, S. (2019) · IEEE Access · doi:10.1109/access.2019.2916887 · Cited by 535 · see citing works
DeepThings: Distributed Adaptive Deep Learning Inference on Resource-Constrained IoT Edge Clusters
Zhao, Z., Barijough, K. M., & Gerstlauer, A. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2018.2858384 · Cited by 502 · cited 1× · see citing works
“…[15] demonstrated how model segments could be dynamically distributed across a cluster of devices to minimize execution time, while systems like DeepThings [16] and EdgePipe [17] highlighted the crucial role of the edge in creating efficient, privacy-aware, and high-throughput inference pipelines.”
Dynamic Adaptive DNN Surgery for Inference Acceleration on the Edge
Hu, C., Bao, W., Wang, D., & Liu, F. (2019) · IEEE INFOCOM 2019 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2019.8737614 · Cited by 416 · cited 1× · see citing works
“Some approaches rely on specialized graph-based heuristics, like those in DADS [25] and QDMP [26], to accelerate the search process for real-time inference.”
Firdose, S., Avgeris, M., Spatharakis, D., Santi, N., Dechouniotis, D., Violos, J., Leivadeas, A., Αθανασόπουλος, Ν., Mitton, N., & Papavassiliou, S. (2021) · Computer Networks · doi:10.1016/j.comnet.2021.108177 · Cited by 283 · see citing works
End-Edge-Cloud Collaborative Computing for Deep Learning: A Comprehensive Survey
Wang, Y., Yang, C., Lan, S.-L., Zhu, L.-H., & Zhang, Y. (2024) · IEEE Communications Surveys and Tutorials · doi:10.1109/comst.2024.3393230 · Cited by 255 · see citing works
Kar, B., Yahya, W., Lin, Y., & Ali, A. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3239579 · Cited by 213 · cited 2× · see citing works
“The success of any collaborative inference system, however, is not determined by the architecture alone, but by the intelligence of its workload distribution strategy [18,19].”
Zhang (2020) · doi:10.1145/3432192 · Cited by 126 · cited 1× · see citing works
“Some approaches rely on specialized graph-based heuristics, like those in DADS [25] and QDMP [26], to accelerate the search process for real-time inference.”
Auto-Split: A General Framework of Collaborative Edge-Cloud AI
Banitalebi-Dehkordi, A., Vedula, N., Pei, J., Xia, F., Wang, L., & Zhang, Y. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467078 · Cited by 108 · cited 2× · see citing works
“Consequently, the design of advanced and automated partitioning algorithms, often leveraging machine learning and dynamic adaptation, remains a primary focus of research in this domain [22–24].”
Distributed DNN Inference With Fine-Grained Model Partitioning in Mobile Edge Computing Networks
Li, H., Li, X., Fan, Q., He, Q., Wang, X., & Leung, V. C. M. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3357874 · Cited by 91 · cited 2× · see citing works
“Consequently, the design of advanced and automated partitioning algorithms, often leveraging machine learning and dynamic adaptation, remains a primary focus of research in this domain [22–24].”
An adaptive DNN inference acceleration framework with end–edge–cloud collaborative computing
Liu, G., Dai, F., Xu, X., Fu, X., Dou, W., Kumar, N., & Bilal, M. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.10.033 · Cited by 85 · cited 1× · see citing works
“To resolve this conflict, the Cloud-Edge-End collaborative inference architecture has emerged as a dominant paradigm [11,12].”
Towards Real-time Cooperative Deep Inference over the Cloud and Edge End Devices
Zhang, S., Li, Y., Liu, X., Guo, S., Wang, W., Wang, J., Ding, B., & Wu, D. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3397315 · Cited by 75 · see citing works
Self-citationCloud-Based or On-Device: An Empirical Study of Mobile Deep Inference
Guo, T. (2018) · IEEE International Conference on Cloud Engineering · doi:10.1109/ic2e.2018.00042 · Cited by 68 · see citing works
Zhao, Z., Wang, K., Ling, N., & Xing, G. (2021) · IEEE/ACM International Conference on Internet-of-Things Design and Implementation (IoTDI) · doi:10.1145/3450268.3453520 · Cited by 57 · cited 1× · see citing works
“Unlike search-based methods that find a single fixed plan, RL-based frameworks like Auto-Split [22], EdgeML [23], and others [24] train agents to learn adaptive partitioning policies.”
Multimodal Representation Learning: Advances, Trends and Challenges
Zhang, S., Zhai, J., Xie, B., Zhan, Y., & Wang, X. (2019) · International Conference on Machine Learning and Cybernetics · doi:10.1109/icmlc48188.2019.8949228 · Cited by 49 · cited 1× · see citing works
“Meanwhile, the functional integrity of its components is especially critical for multimodal models, where disrupting fusion blocks can degrade accuracy [10].”
Zeng, C., Wang, X., Zeng, R., Li, Y., Shi, J., & Huang, M. (2024) · Future Generation Computer Systems · doi:10.1016/j.future.2024.01.025 · Cited by 41 · see citing works
Energy-Latency-Aware Task Offloading and Approximate Computing at the Mobile Edge
Younis, A., Tran, T. X., & Pompili, D. (2019) · IEEE International Conference on Mobile Ad-Hoc and Sensor Systems (MASS) · doi:10.1109/mass.2019.00043 · Cited by 32 · cited 1× · see citing works
“The success of any collaborative inference system, however, is not determined by the architecture alone, but by the intelligence of its workload distribution strategy [18,19].”
Na, J., Zhang, H., Lian, J., & Zhang, B. (2022) · Applied Sciences · doi:10.3390/app122010619 · Cited by 27 · cited 2× · see citing works
“The core research challenge thus shifts from if a model should be distributed to how it should be optimally split to balance the trade-offs between computation and communication [12,20].”
Investigation on performance and energy efficiency of CNN-based object detection on embedded device
Oh, S., Kim, M., Kim, D., Jeong, M., & Lee, M. (2017) · International Conference on Computer Applications and Information Processing Technology (CAIPT) · doi:10.1109/caipt.2017.8320657 · Cited by 26 · cited 1× · see citing works
“Full on-device execution leads to unacceptable latency and rapid energy depletion, while complete cloud offloading incurs prohibitive transmission delays unsuitable for time-sensitive on-site tasks [3,4].”
Li, Y., Liu, Z., Kou, Z., Wang, Y., Zhang, G. Q., Li, Y., & Sun, Y. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3430103 · Cited by 21 · cited 1× · see citing works
“These policies can respond to real-time system conditions, jointly optimizing for latency, accuracy, and resource usage in dynamic environments [29].”
Generative AI-Aided Multimodal Parallel Offloading for AIGC Metaverse Service in IoT Networks
Zeng, W., Zheng, J., Gao, L. C., Niu, J., Ren, J., Wang, H., Cao, R., & Ji, S. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3535623 · Cited by 16 · cited 1× · see citing works
“This heterogeneity means that cost models based on simple data volume are no longer sufficient, requiring a more nuanced, modality-aware approach to balancing computation and communication [31,32].”
Kar, B., Yahya, W., Lin, Y., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.10628 · Cited by 14 · cited 1× · see citing works
“By partitioning a single DNN model and distributing its execution across the terminal, edge, and cloud tiers, this approach can theoretically balance latency, energy, and computational load [6,7].”
Enable Pipeline Processing of DNN Co-inference Tasks In the Mobile-Edge Cloud
Hu, S., Dong, C., & Wen, W. (2021) · International Conference on Computer and Communication Systems (ICCCS) · doi:10.1109/icccs52626.2021.9449178 · Cited by 12 · cited 1× · see citing works
“…[15] demonstrated how model segments could be dynamically distributed across a cluster of devices to minimize execution time, while systems like DeepThings [16] and EdgePipe [17] highlighted the crucial role of the edge in creating efficient, privacy-aware, and high-throughput inference pipelines.”
Parthasarathy, A., & Krishnamachari, B. (2022) · International Telecommunication Networks and Applications Conference (ITNAC) · doi:10.1109/itnac55475.2022.9998427 · Cited by 12 · cited 1× · see citing works
“For multi-device scenarios, algorithms were designed to minimize bottleneck latency in edge clusters by jointly optimizing partition and placement decisions [27].”
Partitioning multi-layer edge network for neural network collaborative computing
Li, Q., Zhou, M.-T., Ren, T.-F., Jiang, C., & Chen, Y. (2023) · EURASIP Journal on Wireless Communications and Networking · doi:10.1186/s13638-023-02284-x · Cited by 4 · cited 1× · see citing works
“…reviewed various splitting strategies, ranging from layer-wise partitioning for sequential CNNs to more sophisticated graph-based methods for complex DAG architectures, while highlighting the critical need for joint optimization of multiple objectives in heterogeneous environments [8,9].”
Multi-modal model partition strategy for end-edge collaborative inference
Huo, D., Zhou, Y., Hao, Y., Hu, L., Mo, Y., Chen, M., & Humar, I. (2025) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2025.105189 · Cited by 3 · see citing works
Sada, A. B., Khelloufi, A., Naouri, A., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.16904 · Cited by 1 · cited 1× · see citing works
“One major branch of work employs search-based metaheuristics, such as Genetic Algorithms, to find high-quality static partition plans offline [28].”
Auto-Split: A General Framework of Collaborative Edge-Cloud AIPreprint
Banitalebi-Dehkordi, A., Vedula, N., Pei, J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.13041 · cited 2×
“Consequently, the design of advanced and automated partitioning algorithms, often leveraging machine learning and dynamic adaptation, remains a primary focus of research in this domain [22–24].”
Fine-tuning Multimodal Transformers on Edge: A Parallel Split Learning ApproachPreprint
Fudala, T., Tsouvalas, V., & Meratnia, N. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.06355 · cited 1×
“These fusion points create tight structural dependencies, severely constraining where partitions can be placed without disrupting critical inter-modal alignments [30].”
Flagged first, then by citations
Research works printed but not linked: 1
- Zeng, W.; Zheng, J.; Gao, L.; Niu, J.; Ren, J.; Wang, H.; Cao, R.; Ji, S. Generative AI-Aided Multimodal Parallel Offloading for AIGC Metaverse Service in IoT Networks. IEEE Internet Things J. 2025, 12, 13273-13285.
Cited by
Sorted by citations
Zhou, Z., Zhou, Y., Ouyang, B., Man, S. S., & Chan, A. H. S. (2026) · Healthcare · doi:10.3390/healthcare14131978
- Authors
- Ye, C., Zhang, R., Li, X., Deng, W., Wang, J., & Shao, S.
- Affiliations
- State Grid Corporation of China (China) (CN)
- Beijing University of Posts and Telecommunications (CN)
- Information & Communication Company of State Grid Ningxia Electric Power Co., Ltd., Yinchuan 750001, China
- Corresponding author
- Zhang, Ruihang
- Venue
- Information
- Publisher
- MDPI AG
- Published
- 2025-12-08
- Volume
- 16
- Issue
- 12
- Pages
- 1091
- ISSN
- 2078-2489 (ISSN-L)
- OpenAlex
- W4417110405
- Cited by
- 1
- References
- 28
- Citations by year
- 2026: 1
- FWCI
- 0.42
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Information (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Ningxia Natural Science Foundation of China (2023AAC03846)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Keywords
- cloud-edge-end collaboration, hybrid genetic algorithm, subgraph partitioning, terminal energy consumption, resource-constrained devices, performance prediction model
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
