Energy Efficient Computation Offloading in Aerial Edge Networks With Multi-Agent Cooperation
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
With the high flexibility of supporting resource-intensive and time-sensitive applications, unmanned aerial vehicle (UAV)-assisted mobile edge computing (MEC) is proposed as an innovational paradigm to support the mobile users (MUs). As a promising technology, digital twin (DT) is capable of timely mapping the physical entities to virtual models, and reflecting the MEC network state in real-time. In this paper, we first propose an MEC network with multiple movable UAVs and one DT-empowered ground base station to enhance the MEC service for MUs. Considering the limited energy resource of both MUs and UAVs, we formulate an online problem of resource scheduling to minimize the weighted energy consumption of them. To tackle the difficulty of the combinational problem, we formulate it as a Markov decision process (MDP) with multiple types of agents. Since the proposed MDP has huge state space and action space, we propose a deep reinforcement learning approach based on multi-agent proximal policy optimization (MAPPO) with Beta distribution and attention mechanism to pursue the optimal computation offloading policy. Numerical results show that our proposed scheme is able to efficiently reduce the energy consumption and outperforms the benchmarks in performance, convergence speed and utilization of resources.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Energy Minimization for Wireless Communication With Rotary-Wing UAV
Zeng, Y., Xu, J., & Zhang, R. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2902559 · Cited by 2,172
6G Internet of Things: A Comprehensive Survey
Nguyen, D. C., Ding, M., Pathirana, P. N., Seneviratne, A., Li, J., Niyato, D., Dobre, O., & Poor, H. V. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3103320 · Cited by 1,452 · cited 1×
“It is expected to be empowered with the ability of providing intelligent and green communication features, and realizing the ubiquitous connection between massive objectives [1]– [3].”
UAV Communications for 5G and Beyond: Recent Advances and Future Trends
Li, B., Fei, Z., & Zhang, Y. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2887086 · Cited by 1,377 · cited 1×
“Recently, unmanned aerial vehicle (UAV) has emerged as a promising technology to assist traditional infrastructure-based MEC networks due to the representative features of high mobility, flexible deployment, and low cost [7], [8].”
Self-citationDigital Twins: A Survey on Enabling Technologies, Challenges, Trends and Future Prospects
Mihai, S., Yaqoob, M., Hung, D. V., Davis, W., Towakel, P., Raza, M., Karamanoglu, M., Barn, B., Shetve, D., Prasad, R. V., et al. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3208773 · Cited by 1,049 · cited 1×
“For the learning-based optimizing techniques, the dynamicity in networks imposes strict requirements on accurate state sensing, real-time decision-making, and the precise execution, which presents technical challenges for implementing the UAV-assisted MEC networks [11], [12].”
The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games
Yu, C., Velu, A., Vinitsky, E., Gao, J., Wang, Y., Bayen, A. M., & Wu, Y. (2022) · neural information processing systems · doi:10.52202/068431-1787 · Cited by 1,027 · cited 2×
“(cid:1) In PPO series, the generalized advantage estimation (GAE) is adopted to improve the performance, which is defined as follows As a variant of PPO specialized for multi-agent settings, MAPPO is one of the state-of-the-art MARL algorithms [34].”
Digital Twin Networks: A Survey
Wu, Y., Zhang, K., & Zhang, Y. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3079510 · Cited by 847 · cited 1×
“For the learning-based optimizing techniques, the dynamicity in networks imposes strict requirements on accurate state sensing, real-time decision-making, and the precise execution, which presents technical challenges for implementing the UAV-assisted MEC networks [11], [12].”
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829 · cited 1×
“It is expected to be empowered with the ability of providing intelligent and green communication features, and realizing the ubiquitous connection between massive objectives [1]– [3].”
Enabling Massive IoT Toward 6G: A Comprehensive Survey
Guo, F., Yu, F. R., Zhang, H., Li, X., Ji, H., & Leung, V. C. M. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3063686 · Cited by 701 · cited 1×
“It is expected to be empowered with the ability of providing multidimensional, intelligent, green, and other communication features, and realizing the ubiquitous connection between massive objectives [1], [2], [3].”
Peng, H., & Shen, X. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036962 · Cited by 521 · cited 1×
“Peng et al. [19] investigated the utilization of MARL to solve the dynamic network resource allocation in UAV-assisted vehicular edge networks.”
Multi-Agent Deep Reinforcement Learning for Task Offloading in UAV-Assisted Mobile Edge Computing
Zhao, N., Ye, Z., Pei, Y., Liang, Y., & Niyato, D. T. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3153316 · Cited by 520 · cited 1×
“[21] investigated the cooperation between multi-UAV and multi-ground MEC servers to minimize the total network utility and enhance the performance of service.”
Wang, L., Wang, K., Pan, C., Xu, W., Aslam, N., & Hanzo, L. (2021) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2020.3027695 · Cited by 510 · cited 2×
“The baseline algorithms for comparison with our proposed AB-MAPPO are listed as follows: • MADDPG: MADDPG is an off-policy algorithm with determinate policy [18].”
Lu, Y., Huang, X., Zhang, K., Maharjan, S., & Zhang, Y. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3017668 · Cited by 491 · cited 1×
“Lu et al. [24] proposed a blockchain-based federated learning framework that runs in DT for collaborative computing, improving the reliability and security of the system and enhancing data privacy.”
Reducing Offloading Latency for Digital Twin Edge Networks in 6G
Sun, W., Zhang, H., Wang, R., & Zhang, Y. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3018817 · Cited by 373 · cited 2×
“To fully exploit the capability of DT network for capturing the real-time state of the network, Sun et al. [22] investigated the latency minimization problem of DITEN via DRL method and considered the estimated deviation of DT. Sun et al. [23] developed an approach to automatically adjust the…”
Zhang, K., Cao, J., & Zhang, Y. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3088407 · Cited by 330 · cited 1×
“As a step forward, Do-Duy et al. [26] iteratively optimized the joint resources of mobile devices and multiple edge servers to minimize the total latency in industrial IoT networks, and Zhang et al. [27] combined DT technology and decomposed MARL into the design of vehicle edge computing networks.”
Adaptive Federated Learning and Digital Twin for Industrial Internet of Things
Sun, W., Lei, S., Wang, L., Liu, Z., & Zhang, Y. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3034674 · Cited by 327 · cited 1×
“…the real-time state of the network, Sun et al. [22] investigated the latency minimization problem of DITEN via DRL method and considered the estimated deviation of DT. Sun et al. [23] developed an approach to automatically adjust the aggregation frequency of federated learning in DITEN network.”
Deep Reinforcement Learning Based Dynamic Trajectory Control for UAV-Assisted Mobile Edge Computing
Wang, L., Wang, K., Pan, C., Xu, W., Aslam, N., & Nallanathan, A. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3059691 · Cited by 324 · cited 2×
“To pursue the real-time decision of UAV-assisted networks, Wang et al. [16] integrated deep reinforcement learning (DRL) method into iterative convex optimization approaches, and Dai et al. [17] studied the minimization of the execution latency in an edge-cloud heterogeneous network comprising UAVs…”
Path Planning for UAV-Mounted Mobile Edge Computing With Deep Reinforcement Learning
Liu, Q., Shi, L., Sun, L., Li, J., Ding, M., & Shu, F. S. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2982508 · Cited by 323 · cited 1×
“At the beginning of time slot n = 1 , all MUs are randomly located, and the MUs are moving according to Gaussian-Markov random model [29].”
A Survey of Computational Intelligence for 6G: Key Technologies, Applications and Trends
Ji, B., Wang, Y., Song, K., Li, C., Wen, H., Menon, V. G., & Mumtaz, S. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3052531 · Cited by 232 · cited 1×
“In order to achieve these requirements, the quality of service and application experience in 6G network need to be significantly enhanced, which involves higher requirements for the computing resources and communication resources of various devices [4].”
Digital-Twin-Assisted Task Offloading Based on Edge Collaboration in the Digital Twin Edge Network
Liu, T., Tang, L., Wang, W., Chen, Q., & Zeng, X. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3086961 · Cited by 210 · cited 2×
“Liu et al. [25] introduced edge collaboration into the DITEN network and realized intelligent offloading of tasks.”
Hu, J., Zhang, H., Song, L., Schober, R., & Poor, H. V. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3013599 · Cited by 201 · cited 1×
“Other parameter settings of simulation and hyperparameters of MAPPO are summarized in Table II and Table III, according to prior works [16], [17], [33], [36].”
Tran, D.-H., Nguyen, V.-D., Chatzinotas, S., Vu, T. X., & Ottersten, B. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3105821 · Cited by 198 · cited 1×
“To exploit the performance of UAV in special situations, Do-Duy et al. [14] and Tran et al. [15] studied the joint deployment and resource configuration in UAV-aided disaster emergency communications by using iterative methods.”
Do-Duy, T., Nguyen, L. D., Duong, T. Q., Khosravirad, S. R., & Claussen, H. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3088662 · Cited by 189 · cited 1×
“To exploit the performance of UAV in special situations, Do-Duy et al. [14] and Tran et al. [15] studied the joint deployment and resource configuration in UAV-aided disaster emergency communications by using iterative methods.”
Resource Allocation and Trajectory Design for MISO UAV-Assisted MEC Networks
Liu, B., Wan, Y., Zhou, F., Wu, Q., & Hu, R. Q. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3140833 · Cited by 182 · cited 1×
“By cooperatively carrying MEC servers, UAVs can fly closer to the MUs to offload part of the tasks and provide reliable communication links, which is a fast and cost-effective deployment way for computation offloading with low delivery delay and high data rate requirements [9], [10].”
UAV-Assisted MEC Networks With Aerial and Ground Cooperation
Xu, Y., Zhang, T., Liu, Y., Yang, D., Xiao, L., & Tao, M. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3086521 · Cited by 174 · cited 1×
“Furthermore, channel from UAV m to BS can be modeled by quasi-static block fading LoS link [31], i.e., We consider the orthogonal frequency division multiple access scheme for data transmission.”
Digital Twin Assisted Task Offloading for Aerial Edge Computing and Networks
Li, B., Liu, Y., Tan, L., Pan, H., & Zhang, Y. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3182647 · Cited by 173 · cited 1×
“Recently, Li et al. [28] focused on the DT-aided computation offloading in UAV edge networks where the double deep Q-network algorithm is explored to reduce the system energy consumption.”
Self-citationComputation Offloading in Beyond 5G Networks: A Distributed Learning Framework and Applications
Chen, X., Wu, C., Liu, Z., Zhang, N., & Ji, Y. (2021) · IEEE Wireless Communications · doi:10.1109/mwc.001.2000296 · Cited by 160 · cited 1×
“Specifically, the devices can offload their computational tasks to edge servers with powerful computing resources to pursue satisfactory energy consumption and latency [5], [6].”
Online Trajectory and Resource Optimization for Stochastic UAV-Enabled MEC Systems
Yang, Z., Bi, S., & Zhang, Y. A. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3142365 · Cited by 160 · cited 1×
“…channel power gain of from MU k to UAV m is given by where ˜ ι is the path loss exponent, ν is the NLoS attenuation, β 0 is the channel power gain at the reference distance of 1 m. Similar to [30], we assume that the change of LoS probability P ( LoS , ϑ k [ n ]) in each time slot is negligible.”
Digital Twin-Aided Intelligent Offloading With Edge Selection in Mobile Edge Computing
Do‐Duy, T., Huynh, D. V., Dobre, O. A., Canberk, B., & Duong, T. Q. (2022) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2022.3146207 · Cited by 154 · cited 2×
“As a step forward, Do-Duy et al. [26] iteratively optimized the joint resources of mobile devices and multiple edge servers to minimize the total latency in industrial IoT networks, and Zhang et al. [27] combined DT technology and decomposed MARL into the design of vehicle edge computing networks.”
Adaptive Digital Twin for Vehicular Edge Computing and Networks
Dai, Y., & Zhang, Y. (2022) · Journal of Communications and Information Networks · doi:10.23919/jcin.2022.9745481 · Cited by 146 · cited 1×
“Currently, DT has been widely used in smart city, healthcare, and intelligent manufacturing [13].”
Self-citationGao, Z., Yang, L., & Dai, Y. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3141080 · Cited by 123 · cited 1×
“The complexity of attention module is O ( I 2 V ) , where V is the length of feature-value vectors, according to [35].”
Self-citationJiang, F., Wang, K., Dong, L., Pan, C., Xu, W., & Yang, K. (2020) · IEEE Network · doi:10.1109/mnet.011.2000440 · Cited by 108 · cited 1×
“Recently, unmanned aerial vehicle (UAV) has emerged as a promising technology to assist traditional infrastructure-based MEC networks due to the representative features of high mobility, flexible deployment, and low cost [7], [8].”
Cai, T., Yang, Z., Chen, Y., Chen, W., Zheng, Z., Yu, Y., & Dai, H. (2021) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2021.3121690 · Cited by 87 · cited 1×
“Cai et al. [20] studied the data sensing and computation offloading problem in a multi-UAV-assisted MEC, and integrated attention mechanism into MARL algorithm to accelerate the training speed of the network.”
Towards Energy-Efficient Scheduling of UAV and Base Station Hybrid Enabled Mobile Edge Computing
Dai, B., Niu, J., Ren, T., Hu, Z., & Atiquzzaman, M. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3129214 · Cited by 82
Ji, J., Zhu, K., & Cai, L. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3181308 · Cited by 69 · cited 2×
“By cooperatively carrying MEC servers, UAVs can fly closer to the MUs to offload part of the tasks and provide reliable communication links, which is a fast and cost-effective deployment way for computation offloading with low delivery delay and high data rate requirements [9], [10].”
Cao, X., Yang, B. F., Huang, C., Yuen, C., Zhang, Y., Niyato, D. T., & Han, Z. (2021) · IEEE Network · doi:10.1109/mnet.011.2100049 · Cited by 34
Metadata
- Authors
- Liu, W., Li, B., Xie, W., Dai, Y., & Fei, Z.
- Venue
- IEEE Transactions on Wireless Communications
- Published
- 2023-09
- Volume
- 22
- Issue
- 9
- Pages
- 5725-5739
- ISSN
- 1536-1276 (ISSN-L)
- 1558-2248
- OpenAlex
- W4319996394
- Cited by
- 180
- References
- 35
- FWCI
- 20.21
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.