Deep reinforcement learning based computation offloading and resource allocation for MEC
The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Mobile edge computing (MEC) has the potential to enable computation-intensive applications in 5G networks. MEC can extend the computational capacity at the edge of wireless networks by migrating the computation-intensive tasks to the MEC server. In this paper, we consider a multi-user MEC system, where multiple user equipments (UEs) can perform computation offloading via wireless channels to an MEC server. We formulate the sum cost of delay and energy consumptions for all UEs as our optimization objective. In order to minimize the sum cost of the considered MEC system, we jointly optimize the offloading decision and computational resource allocation. However, it is challenging to obtain an optimal policy in such a dynamic system. Besides immediate reward, Reinforcement Learning (RL) also takes a long-term goal into consideration, which is very important to a time-variant dynamic systems, such as our considered multi-user wireless MEC system. To this end, we propose RL-based optimization framework to tackle the resource allocation in wireless MEC. Specifically, the Q-learning based and Deep Reinforcement Learning (DRL) based schemes are proposed, respectively. Simulation results show that the proposed scheme achieves significant reduction on the sum cost compared to other baselines.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Mobile Edge Computing: A Survey on Architecture and Computation Offloading
Mach, P., & Becvar, Z. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2682318 · Cited by 3,258 · cited 1×
“Therefore, the computation offloading and computational resource allocation themes have attracted great interests as a key point in the MEC system [3].”
BackgroundEfficient multi-user computation offloading for mobile-edge cloud computing
Chen, X., Jiao, L., Li, W., & Fu, X. (2016) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2015.2487344 · Cited by 2,675 · cited 1×
“But according to [14], [15], the download data rate is very high in general, and the data size of the result is much smaller than that of input data, so the delay and energy consumptions of this step are neglected in the rest of this paper.”
BackgroundCloud computing for mobile users: can offloading computation save energy?
Kumar, K., & Lu, Y.-H. (2010) · Computer · doi:10.1109/mc.2010.98 · Cited by 1,493 · cited 1×
“The work in [5] proposed general guidelines to make offloading decision for energy consumption minimization, where the communication links were easily assumed to have a fixed rate.”
BackgroundSelf-citationEnergy-Efficient Offloading for Mobile Edge Computing in 5G Heterogeneous Networks
Zhang, K., Mao, Y., Leng, S., Zhao, Q., Li, L., Peng, X., Pan, L., Maharjan, S., & Zhang, Y. (2016) · IEEE Access · doi:10.1109/access.2016.2597169 · Cited by 855 · cited 1×
“But according to [14], [15], the download data rate is very high in general, and the data size of the result is much smaller than that of input data, so the delay and energy consumptions of this step are neglected in the rest of this paper.”
BackgroundDelay-optimal computation task scheduling for mobile-edge computing systems
Liu, J., Mao, Y., Zhang, J., & Letaief, K. B. (2016) · 2016 IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2016.7541539 · Cited by 759 · cited 1×
“The work in [8] designed the single-user delay-optimal task scheduling policies based on the theory of MDP, which controlled the states of the local processing and transmission units and the task buffer queue length based on the channel state.”
MethodJaszkiewicz, A. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2002.802873 · Cited by 474
Communicating While Computing: Distributed mobile cloud computing over 5G heterogeneous networks
Barbarossa, S., Sardellitti, S., & Di Lorenzo, P. (2014) · IEEE Signal Processing Magazine · doi:10.1109/msp.2014.2334709 · Cited by 468 · cited 2×
“Since the data rates for wireless communications are not constant, the optimal binary computation offloading decision using convex optimization was proposed in [6], where the the famous Shannon-Hartley formula gave the power-rate function.”
BackgroundMobile-edge computing architecture: the role of mec in the internet of things
Sabella, D., Vaillant, A., Kuure, P., Rauschenbach, U., & Giust, F. (2016) · IEEE Consumer Electronics Magazine · doi:10.1109/mce.2016.2590118 · Cited by 413 · cited 1×
“MEC is a new paradigm in C-RAN, which can enhance the computation capacity at the edge of mobile networks by deploying high-performance servers [2].”
BackgroundEnergy-optimal mobile application execution: Taming resource-poor mobile devices with cloud clones
Wen, Y., Zhang, W., & Luo, H. (2012) · 2012 Proceedings IEEE INFOCOM · doi:10.1109/infcom.2012.6195685 · Cited by 394 · cited 1×
“We set zn = 10−27 ( f l n )2 according to the practical measurement in [13].”
MethodPeng, M., Sun, Y., Li, X., Mao, Z., & Wang, C. (2016) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2016.2548658 · Cited by 386
Cited by
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,060
“used for each mobile user to learn the optimal offloading policy from past experiences such as the approaches proposed in [101] and [102].”
BackgroundDeep Learning in Mobile and Wireless Networking: A Survey
Zhang, C., Patras, P., & Haddadi, H. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2904897 · Cited by 1,768
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
“Under the circumstances, a preclassification step can be performed before learning [253] to”
BackgroundPham, Q.-V., Fang, F., Ha, V. N., Piran, M. J., Le, M., Le, L. B., Hwang, W.-J., & Ding, Z. (2020) · IEEE Access · doi:10.1109/access.2020.3001277 · Cited by 870
Learning-Based Computation Offloading for IoT Devices With Energy Harvesting
Min, M., Xiao, L., Chen, Y., Cheng, P., Wu, D., & Zhuang, W. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2890685 · Cited by 560
Bi, S., Huang, L., Wang, H., & Zhang, Y.-J. A. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3085319 · Cited by 407
She, C., Sun, C., Gu, Z., Li, Y., Yang, C., Poor, H. V., & Vucetic, B. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3053601 · Cited by 405
“As summarized in Table 8, DRL algorithms have been applied in wireless networks for mobile networking [247]–[249], scheduler design [250]–[254], resource allocation [255], [256], [258], [260]–[262], and routing [263], [264].”
MethodDeep Reinforcement Learning for Energy-Efficient Computation Offloading in Mobile-Edge Computing
Zhou, H., Jiang, K., Liu, X., Li, X., & Leung, V. C. M. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3091142 · Cited by 350
Shakarami, A., Ghobaei-Arani, M., & Shahidinejad, A. (2020) · Computer Networks · doi:10.1016/j.comnet.2020.107496 · Cited by 331
Deep-Reinforcement-Learning-Based Offloading Scheduling for Vehicular Edge Computing
Zhan, W., Luo, C., Wang, J., Wang, C., Min, G., Duan, H., & Zhu, Q. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2978830 · Cited by 319
“Both [13] and [14] consider static environments and resort to DRL to deal with the enormous state space.”
BackgroundMethodA survey on computation offloading modeling for edge computing
Lin, H., Zeadally, S., Chen, Z., Labiod, H., & Wang, L. (2020) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2020.102781 · Cited by 305
Wang, Y., Ru, Z.-Y., Wang, K., & Huang, P.-Q. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2935466 · Cited by 302
“The parameter settings of the proposed multi-UAV enabled MEC system are given in Table I [30], [40], [41].”
MethodResource Allocation Based on Deep Reinforcement Learning in IoT Edge Computing
Xiong, X., Zheng, K., Lei, L., & Hou, L. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.2986615 · Cited by 302
“In [28], a reinforcement learning based”
Dependent Task Offloading for Edge Computing based on Deep Reinforcement Learning
Wang, J., Hu, J., Min, G., Zhan, W., Zomaya, A. Y., & Georgalas, N. (2022) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3131040 · Cited by 274
“[35] proposed a deep Q-Learning based offloading method to jointly optimize the offloading decision and computational resource allocation.”
MethodMulti-Agent DRL for Task Offloading and Resource Allocation in Multi-UAV Enabled IoT Edge Network
Seid, A. M., Boateng, G. O., Mareri, B., Sun, G., & Jiang, W. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3096673 · Cited by 285
Deep Reinforcement Learning for Mobile 5G and Beyond: Fundamentals, Applications, and Challenges
Xiong, Z., Zhang, Y., Niyato, D., Deng, R., Wang, P., & Wang, L.-C. (2019) · IEEE Vehicular Technology Magazine · doi:10.1109/mvt.2019.2903655 · Cited by 285
“In contrast to [8], the authors in [9] discussed an edge computing system with a single edge server, where multiple UEs can perform computation offloading via wireless channels to the server.”
ResultA DRL Agent for Jointly Optimizing Computation Offloading and Resource Allocation in MEC
Chen, J., Xing, H., Xiao, Z., Xu, L., & Tao, T. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3081694 · Cited by 273
Deng, X., Yin, J., Guan, P., Xiong, N. N., Zhang, L., & Mumtaz, S. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3123406 · Cited by 267
“In order to verify the advantages of the partial offloading strategy we proposed, we compare it with local computing scheme and full offloading scheme, as well as the DQN computational offloading strategy in [41].”
BackgroundMethodInfluentialA Gentle Introduction to Reinforcement Learning and its Application in Different Fields
Naeem, M., Rizvi, S. T. H., & Coronato, A. (2020) · IEEE Access · doi:10.1109/access.2020.3038605 · Cited by 257
“Different policies using deep RL to address data and computation offloading are proposed in: [186]–[191].”
MethodWang, J., Hu, J., Min, G., Zhan, W., Ni, Q., & Georgalas, N. (2019) · IEEE Communications Magazine · doi:10.1109/mcom.2019.1800971 · Cited by 254
“Although there are emerging DRL-based offloading methods [7], [8], [9], they assume that the tasks are independent.”
BackgroundChen, Z., & Wang, X. (2020) · EURASIP Journal on Wireless Communications and Networking · doi:10.1186/s13638-020-01801-6 · Cited by 212
“Specifically, in [23], system sum cost of a multi-user network is minimized in terms of execution delay and energy consumption by computational resource allocation.”
BackgroundTang, F., Mao, B., Kato, N., & Gui, G. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3089688 · Cited by 216
iRAF: A Deep Reinforcement Learning Approach for Collaborative Mobile Edge Computing IoT Networks
Chen, J., Chen, S., Wang, Q., Cao, B., Feng, G., & Hu, J. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2913162 · Cited by 225
“Hence, the DQN-based resource allocation method exhibits better performance than traditional single time slot decision-based algorithm in [16] (R2.”
MethodEdge Intelligence for Energy-Efficient Computation Offloading and Resource Allocation in 5G Beyond
Dai, Y., Zhang, K., Maharjan, S., & Zhang, Y. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3013990 · Cited by 210
“Specifically, F s j (t+ 1) = F s j (t)− ∑ i∈U y s ij ∗ f ij [29].”
Multiagent Deep Reinforcement Learning for Vehicular Computation Offloading in IoT
Zhu, X., Luo, Y., Liu, A., Bhuiyan, M. Z. A., & Zhang, S. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3040768 · Cited by 208
“[37] proposed a MEC– cloud collaboration solution, the computation task was decided to offload to the MEC server or cloud server using a DRL Fig.”
MethodNath, S., & Wu, J. (2020) · Intelligent and Converged Networks · doi:10.23919/icn.2020.0014 · Cited by 202
Tang, F., Mao, B., Kawamoto, Y., & Kato, N. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3073009 · Cited by 199
“The works of [45], [46] propose reinforcement learning based joint computation, channel and power allocation algorithms for ensuring end-to-end QoS.”
BackgroundZero Touch Management: A Survey of Network Automation Solutions for 5G and 6G Networks
Coronado, E., Behravesh, R., Subramanya, T., Fernandez-Fernandez, A., Siddiqui, M. S., Costa-Perez, X., & Riggio, R. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3212586 · Cited by 183
“…5G and beyond networks must support end-to-end network resource allocation using the concept of network slicing which, together with AI, can accelerate the 5G network performance, as shown in many research activities [190], [220], [249], [250], [251], [252], [253], [254], [255], [256], [257].”
BackgroundInfluentialPrivacy-preserved task offloading in mobile blockchain with deep reinforcement learning
Nguyen, D. C., Pathirana, P. N., Ding, M., & Seneviratne, A. (2020) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2020.3010967 · Cited by 178
“virtual MEC [18], multi-user MEC system [19], and multi-IoT networks [20], [21].”
BackgroundYou, Q., & Tang, B. (2021) · Journal of Cloud Computing Advances Systems and Applications · doi:10.1186/s13677-021-00256-4 · Cited by 192
Elgendy, I. A., Zhang, W.-Z., He, H., Gupta, B. B., & Abd El-Latif, A. A. (2021) · Wireless Networks · doi:10.1007/s11276-021-02554-w · Cited by 169
“For instance, in [27, 32], the allocation of the resources and computation task offloading have been defined as a non-linear problem to reduce the entire system cost.”
BackgroundMethodA review of optimization methods for computation offloading in edge computing networks
Sadatdiynov, K., Cui, L., Zhang, L., Huang, J. Z., Salloum, S., & Mahmud, M. S. (2022) · Digital Communications and Networks · doi:10.1016/j.dcan.2022.03.003 · Cited by 176
Resource Management at the Network Edge: A Deep Reinforcement Learning Approach
Zeng, D., Gu, L., Pan, S., Cai, J., & Guo, S. (2019) · IEEE Network · doi:10.1109/mnet.2019.1800386 · Cited by 163
“Some recent studies also introduce it into the management and control problem in information systems, for example, wireless network management [4], energy management [5, 6] and mobile data offloading [7], TCP contention window size setting [8], and flow scheduling [9].”
BackgroundUAV-Enabled Mobile Edge-Computing for IoT Based on AI: A Comprehensive Review
Yazid, Y., Ez‐zazi, I., González, A. G., Oualkadi, A. E., & Arioua, M. (2021) · Drones · doi:10.3390/drones5040148 · Cited by 166
Li, H., Xu, H., Zhou, C., Lü, X., & Han, Z. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.3003898 · Cited by 155
“In order to study the effect of the number of UTDs, the number of MEC servers, and the number of channels on the performance of the algorithm, the number of UTDs is set from 50 to 150, the number of servers is set to [ 4 , 8 , 16 , 24 , 32 ] and the number of channels is set from 50 to 150.”
BackgroundA Review on Computational Intelligence Techniques in Cloud and Edge Computing
Asim, M., Wang, Y., Wang, K., & Huang, P.-Q. (2020) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2020.3007905 · Cited by 158
“• DRL performs the best among all LBS for resource allocation, task offloading, and joint issues [145], [161], [163], [171].”
BackgroundFederated Multiagent Actor–Critic Learning for Age Sensitive Mobile-Edge Computing
Zhu, Z., Wan, S., Fan, P., & Letaief, K. B. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3078514 · Cited by 154
“The Q-learning and DQN based RL methods were applied for resource allocation [32], task offloading [33], as well as trajectory planning [34].”
Reinforcement Learning Methods for Computation Offloading: A Systematic Review
Zabihi, Z., Moghadam, A. M. E., & Rezvani, M. H. (2023) · ACM Computing Surveys · doi:10.1145/3603703 · Cited by 157
“Some studies [ 81 , 89 , 90 ] have used pre-classification to limit the space to accelerate convergence.”
MethodA Deep Learning Approach for Energy Efficient Computational Offloading in Mobile Edge Computing
Ali, Z., Jiao, L., Baker, T., Abbas, G., Abbas, Z. H., & Khaf, S. (2019) · IEEE Access · doi:10.1109/access.2019.2947053 · Cited by 143
“[18] propose a deep reinforcement learning approach for the total offloading scheme.”
BackgroundMethodA Probabilistic Approach for Cooperative Computation Offloading in MEC-Assisted Vehicular Networks
Dai, P., Hu, K., Wu, X. Y., Xing, H., Teng, F., & Yu, Z. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3017172 · Cited by 139
Qian, B., Su, J., Wen, Z., Jha, D. N., Li, Y., Guan, Y., Puthal, D., James, P., Yang, R., Zomaya, A. Y., et al. (2021) · ACM Computing Surveys · doi:10.1145/3398020 · Cited by 132
Joint Offloading and Resource Allocation for Satellite Assisted Vehicle-to-Vehicle Communication
Cui, G., Long, Y., Xu, L., & Wang, W. (2020) · IEEE Systems Journal · doi:10.1109/jsyst.2020.3017710 · Cited by 132
A Hybrid DQN and Optimization Approach for Strategy and Resource Allocation in MEC Networks
Wu, Y.-C., Dinh, T. Q., Fu, Y., Lin, C., & Quek, T. Q. S. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3057882 · Cited by 137
Kasgari, A. T. Z., Saad, W., Mozaffari, M., & Poor, H. V. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3031930 · Cited by 143
“A deep-RL based resource allocation scheme for mobile edge computing is studied in [23].”
BackgroundDeep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey
Majid, A. Y., Saaybi, S., van Rietbergen, T., François-Lavet, V., Prasad, R. R. V., & Verhoeven, C. J. M. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3264540 · Cited by 143
Santos, J., Wauters, T., Volckaert, B., & Turck, F. D. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3095358 · Cited by 137
Mobile Edge Computing With Wireless Backhaul: Joint Task Offloading and Resource Allocation
Pham, Q., Le, L. B., Chung, S., & Hwang, W. (2019) · IEEE Access · doi:10.1109/access.2018.2883692 · Cited by 120
“In terms of computation offloading for MEC systems, there have been some works using ML, especially reinforcement learning (RL) for computation offloading problems, for example, [44], [45].”
BackgroundMughees, A., Tahir, M., Sheikh, M. A., & Ahad, A. (2020) · IEEE Access · doi:10.1109/access.2020.3029903 · Cited by 126
“In [70], RL based theme is used to enhance energy efficiency.”
MethodEnergy-Aware Inference Offloading for DNN-Driven Applications in Mobile Edge Clouds
Xu, Z., Zhao, L., Liang, W., RANA, O. F., Zhou, P., Xia, Q., Xu, W., & Wu, G. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.3032443 · Cited by 128
“The input data of each inference request is randomly withdrawn from the range of ½ 10 ; 100 (cid:8) Mb [26].”
MethodHassan, S. S., Park, Y. M., Tun, Y. K., Saad, W., Han, Z., & Hong, C. S. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2023.3300314 · Cited by 124
“These works suggested MEC-based offloading techniques [18], i.e., the multi-user game model [19] and reinforcement learning [20], to increase dependability.”
MethodResource Allocation in IoT Edge Computing via Concurrent Federated Reinforcement Learning
Zhu, T., Zhou, W., Ye, D., Cheng, Z., & Li, J. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3086910 · Cited by 114
Self-citationZou, J., Hao, T., Chen, Y., & Jin, H. (2020) · IEEE Transactions on Computers · doi:10.1109/tc.2020.2987567 · Cited by 108
“But many studies such as [7], [24] choose to over-look the influence of downlink transmission because of two main reasons.”
BackgroundMethodOptimizing task offloading and resource allocation in edge-cloud networks: a DRL approach
Ullah, I., Lim, H.-K., Seok, Y.-J., & Han, Y. (2023) · Journal of Cloud Computing Advances Systems and Applications · doi:10.1186/s13677-023-00461-3 · Cited by 128
“[13] applied the deep Q-network (DQN) to jointly handle computation offloading and resource allocation in multiuser MEC, minimizing the sum cost of delay and energy consumption.”
BackgroundMethodManagement and Orchestration of Edge Computing for IoT: A Comprehensive Survey
Chiang, Y., Zhang, Y., Luo, H., Chen, T.-Y., Chen, G.-H., Chen, H.-T., Wang, Y.-J., Wei, H., & Chou, C. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3245611 · Cited by 116
“[124] proposed a deep Q-network-based computation offloading scheme in the multiuser wireless MEC system to minimize the long-term system delay and energy consumption.”
BackgroundMethodInfluentialDeep Learning at the Mobile Edge: Opportunities for 5G Networks
McClellan, M., Cervelló-Pastor, C., & Sallent, S. (2020) · Applied Sciences · doi:10.3390/app10144735 · Cited by 121
“[32] DNN Forecasts resource capacity and demands per slice using network probes [58] DNN Predicts traffic demand and distributes radio resources between slices [60] RL Minimizes cost of delay and energy consumption for multi-user wireless system Resource Allocation [61] RL Minimizes end-to-end delay for caching, offlo…”
BackgroundMethodClassification of optimization problems in fog computing
Bellendorf, J., & Mann, Z. Á. (2020) · Future Generation Computer Systems · doi:10.1016/j.future.2020.01.036 · Cited by 113
“[129] ~ ~ ~ [130] q [131] ~ ~ q ~ [132] ~ ~ ~ ~ [133] ~ [134] ~ [135] ~ [136] q [137] ~ ~ ~ ~ [138] ~ [139] ~ ~ ~ q ~ [140] ~ ~ ~ q ~ ~ [141] ~ [142] q q q ~ [143] ~ ~ ~ ~ ~ ~ [144] ~ ~ ~ q ~ [145] [146] q q q ~ [147] ~ ~ ~ ~ [19] ~ q ~ [148] ~ ~ [149] q ~ ~ [150] ~ ~ [151] ~ ~ ~ [152] q q ~ ~ ~ ~ [153] ~ ~ ~ q q ~ ~…”
Yang, L., Li, M., Si, P., Yang, R., Sun, E., & Zhang, Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3030646 · Cited by 108
“where q(t) is the computation complexity of the computation tasks, and zu is the coefficient of energy consumption [11], which can be denoted as”
Graph-Reinforcement-Learning-Based Task Offloading for Multiaccess Edge Computing
Sun, Z., Mo, Y., & Chen, Y. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3123822 · Cited by 100
“The network model with one MEC server and multiple user equipments was proposed in [15], where the user equipments have known data size, computational overhead, and maximum acceptable latency for applications.”
MethodDouble Deep Q-Network-Based Energy-Efficient Resource Allocation in Cloud Radio Access Network
Iqbal, A., Tham, M., & Chang, Y. C. (2021) · IEEE Access · doi:10.1109/access.2021.3054909 · Cited by 105
Enabling technologies for AI empowered 6G massive radio access networks
Shahjalal, M. A., Kim, W., Khalid, W., Moon, S., Khan, M. A., Liu, S., Lim, S., Kim, E., Yun, D.-W., Lee, J., et al. (2022) · ICT Express · doi:10.1016/j.icte.2022.07.002 · Cited by 98
Yang, N., Chen, S., Zhang, H., & Berry, R. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3405075 · Cited by 111
“Server Server In [66], the authors applied the Q-learning and the DQN approach to make offloading decisions in a multi-user MEC system.”
BackgroundMethodLyapunov Optimization Based Mobile Edge Computing for Internet of Vehicles Systems
Jia, Y., Zhang, C., Huang, Y., & Zhang, W. (2022) · IEEE Transactions on Communications · doi:10.1109/tcomm.2022.3206885 · Cited by 90
“, how to reasonably offload local computation from vehicles to edge MEC nodes, then becomes a key challenge in MEC-assisted IoV systems [4].”
BackgroundEdge Computation Offloading With Content Caching in 6G-Enabled IoV
Zhou, X., Bilal, M., Dou, R., Rodrigues, J. J. P. C., Zhao, Q., Dai, J., & Xu, X. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3239599 · Cited by 89
“At present, many studies have applied reinforcement learning methods such as DQN [40] and DDPG [41] in computation offloading, and achieved remarkable results.”
BackgroundSeid, A. M., Erbad, A., Abishu, H. N., Albaseer, A. M., Abdallah, M. M., & Guizani, M. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3283353 · Cited by 86
“It is complex and time consuming to address with traditional optimization techniques [47].”
BackgroundEdge computational task offloading scheme using reinforcement learning for IIoT scenario
Hossain, M. S., Nwakanma, C. I., Lee, J. M., & Kim, D. (2020) · ICT Express · doi:10.1016/j.icte.2020.06.002 · Cited by 85
Aghapour, Z., Sharifian, S., & Taheri, H. (2023) · Computer Networks · doi:10.1016/j.comnet.2023.109577 · Cited by 92
Multiuser Resource Control With Deep Reinforcement Learning in IoT Edge Computing
Lei, L., Xu, H., Xiong, X., Zheng, K., Xiang, W., & Wang, X. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2935543 · Cited by 81
“[27] and [28] focus on the offloading and resource allocation problems under deterministic task models, where a fixed number of tasks per user need to be processed either locally or offloaded to the edge server.”
BackgroundPhysical-Layer Assisted Secure Offloading in Mobile-Edge Computing
He, X., Jin, R., & Dai, H. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2979456 · Cited by 76
“Although various delay-optimal and energy-optimal offloading schemes (e.g., [4], [7]–[13] and the related references in [3]) have been developed in the literature, they barely consider the aforementioned security issue; without a proper protection mechanism, privacy-sensitive users may be deterred…”
BackgroundMasoudi, M., & Çavdar, Ç. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.2999784 · Cited by 80
Chen, Z., Zhang, L., Pei, Y., Jiang, C., & Yin, L. (2021) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2021.3093436 · Cited by 74
Hu, Z., Niu, J., Ren, T., Dai, B., Li, Q., Xu, M., & Das, S. K. (2021) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2021.3116280 · Cited by 75
Intelligent secure mobile edge computing for beyond 5G wireless networks
Lai, S., Zhao, R., Tang, S., Xia, J., Zhou, F., & Fan, L. (2021) · Physical Communication · doi:10.1016/j.phycom.2021.101283 · Cited by 72
Panda, S. K., Lin, M., & Zhou, T. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3153399 · Cited by 80
“Li et al. [16] proposed a DRL offloading approach to jointly optimize the offloading decision and resource allocations for a multiuser mobile EC (MEC) system, where multiple user equipment (UE) can perform computation offloading via wireless channels to a MEC server to minimize the sum of delay cost…”
Hao, Z., Xu, G., Luo, Y., Hu, H., An, J., & Mao, S. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3183098 · Cited by 77
“[8] use deep Q-learning (DQL) to allocate offloading decision and resources for multiple UEs, and Nath et al.”
Ye, X., Li, M., Si, P., Yang, R., Wang, Z., & Zhang, Y. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3210570 · Cited by 76
“…task n ∈ N = { 1 , 2 , . . . , N } , the energy consumption at time slot t is represented as where k is set to 10 − 27 based on the actual measurement [38], F l is the computing capacity of all vehicles, and c n ( t ) is the required CPU cycles to implement the computation task n at time slot t .”
BackgroundZhao, J., Quan, H., Xia, M., & Wang, D. (2023) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2023.3335663 · Cited by 77
“Like previous studies based on DRL algorithms [12], [13], [14], [21], [22], [23], [24], the proposed MAPTD3 algorithm has three essential elements: state space, action space, and reward function, but the difference lies that the task-slicing method is adopted to design the state space and reward…”
MethodA deep reinforcement learning for user association and power control in heterogeneous networks
Ding, H., Zhao, F., Tian, J., Li, D., & Zhang, H. (2020) · Ad Hoc Networks · doi:10.1016/j.adhoc.2019.102069 · Cited by 75
Computation offloading in Edge Computing environments using Artificial Intelligence techniques
Carvalho, G., Cabral, B., Pereira, V., & Bernardino, J. (2020) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2020.103840 · Cited by 74
A Digital Twin Approach for Self-optimization of Mobile Networks
Deng, J., Zheng, Q., Liu, G., Bai, J., Tian, K., Sun, C., Yan, Y., & Liu, Y. (2021) · IEEE Wireless Communications and Networking Conference Workshops (WCNCW) · doi:10.1109/wcncw49093.2021.9420037 · Cited by 81
“Deep reinforcement learning is used in [11] to optimize the offloading decision and resource allocation in order to minimize the total cost of system.”
MethodLi, H., Assis, K. D. R., Yan, S., & Simeonidou, D. E. (2022) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2022.3191748 · Cited by 71
“Under the same system model, Li et al. [16] designed a centralized-based DRL-based model to simultaneously resolve offloading decisions and computing resource allocations.”
MethodQin, Y., Chen, J., Jin, L., Yao, R., & Gong, Z. (2025) · Scientific Reports · doi:10.1038/s41598-024-84038-3 · Cited by 75
FedDOVe: A Federated Deep Q-learning-based Offloading for Vehicular fog computing
Sethi, V., & Pal, S. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.11.012 · Cited by 66
Deep Reinforcement Learning Based 3D-Trajectory Design and Task Offloading in UAV-Enabled MEC System
Liu, C., Zhong, Y., Wu, R., Ren, S., Du, S., & Guo, B. (2024) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2024.3469977 · Cited by 71
“The works [26], [27], [32], [33], [34], [35] have already applied DRL to help improve the performance of either conventional MEC or UAV-enabled systems.”
Heidari, A., Jamali, M. A. J., Navimipour, N. J., & Akbarpour, S. (2022) · Applied Sciences · doi:10.3390/app12168232 · Cited by 66
Load Balance Guaranteed Vehicle-to-Vehicle Computation Offloading for Min-Max Fairness in VANETs
Wu, Y., Wu, J., Chen, L., Yan, J., & Han, Y. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3109154 · Cited by 64
Joint Task Offloading and QoS-Aware Resource Allocation in Fog-Enabled Internet-of-Things Networks
Huang, X., Cui, Y., Chen, Q., & Zhang, J. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2982670 · Cited by 68
“Li et al. [22] proposed the double-dueling-deep Q -network which jointly optimize networking, caching, and computing resources to improve the performance of vehicular networks.”
BackgroundDeep reinforcement learning-based online task offloading in mobile edge computing networks
Wu, H., Geng, J., Bai, X., & Jin, S. (2023) · Information Sciences · doi:10.1016/j.ins.2023.119849 · Cited by 63
AlQerm, I., & Pan, J. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3123959 · Cited by 73
“DRL has been employed for solving decision-making re-lated problems in the context of edge computing such as computation offloading [13], [14], [15], [16], management problems in vehicular networks [17], [18], [19], [20], [21] and edge resource allocation [22], [23].”
MethodHastening Stream Offloading of Inference via Multi-Exit DNNs in Mobile Edge Computing
Liu, Z., Song, J., Qiu, C., Wang, X., Chen, X., He, Q., & Sheng, H. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3218724 · Cited by 66
“For ease of discussion, the time of result The n th task and the previous task generated on the same device returning from the edge node is ignored [44].”
BackgroundDeep reinforcement learning‐based resource allocation in multi‐access edge computing
Khani, M., Sadr, M. M., & Jamali, S. (2023) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.7995 · Cited by 74
Machine learning-based computation offloading in multi-access edge computing: A survey
Choudhury, A., Ghose, M., Islam, A., & Yogita, Y. (2024) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2024.103090 · Cited by 72
Huang, H., Ye, Q., & Zhou, Y.-T. (2021) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2021.3115054 · Cited by 65
“proposed a multi-user Deep-Q-Network (DQN) based offloading method for wireless MEC systems to mitigate both transmission delay and energy consumption [29].”
BackgroundDeepEdge: A Deep Reinforcement Learning Based Task Orchestrator for Edge Computing
Yamansavascilar, B., Baktır, A. C., Sonmez, C., Özgövde, A., & Ersoy, C. (2022) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2022.3217311 · Cited by 71
“In [19], Zhu et al. performed a similar study as [18] for task offloading by proposing a DRL agent that considers the task completion time and energy consumption.”
BackgroundMethodRetracted: A DQN-Based Frame Aggregation and Task Offloading Approach for Edge-Enabled IoMT
Yuan, X., Zhang, Z., Feng, C., Cui, Y., Garg, S., Kaddoum, G., & Yu, K. (2022) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2022.3218313 · Cited by 61
“In [31], Li et al. proposed a multi-user MEC system aimed at minimizing the sum of energy consumption and latency by Q-learning based on DRL.”
MethodRetractedShi, D., Gao, H., Wang, L., Pan, M., Han, Z., & Poor, H. V. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2983741 · Cited by 63
“[20] put forward a novel method to optimize the offloading decision and computational resources allocation based on the DRL.”
MethodSelf-citationBlockchain-Inspired Secure Computation Offloading in a Vehicular Cloud Network
Xu, S., Guo, C., Hu, R. Q., & Qian, Y. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3054866 · Cited by 61
Wireless Edge Machine Learning: Resource Allocation and Trade-Offs
Merluzzi, M., Lorenzo, P. D., & Barbarossa, S. (2021) · IEEE Access · doi:10.1109/access.2021.3066559 · Cited by 66
“Several works investigate the problem of radio and computation resource allocation for dynamic computation offloading [8]–[17].”
BackgroundAdaptive Resource Allocation for Blockchain-Based Federated Learning in Internet of Things
Zhang, J., Liu, Y., Qin, X., Xu, X., & Zhang, P. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3241318 · Cited by 59
Cui, Y., Liang, Y., & Wang, R. (2019) · IEEE Access · doi:10.1109/access.2018.2882000 · Cited by 54
“Inspired by the aforementioned works in [13]–[16], this paper proposes a multi-platform offload intelligent resource allocation algorithm combining cloud computing, mobile edge computing and local computing, aiming at minimizing delay.”
MethodLu, H., Gu, C., Luo, F., Ding, W., Zheng, S., & Shen, Y. (2020) · IEEE Access · doi:10.1109/access.2020.3036416 · Cited by 56
“It needs fewer parameters and has better global search capabilities, which can solve the more complex, high-dimensional and more realistic task [12].”
Background
Metadata
- Authors
- Li, J., Gao, H., Lv, T., & Lu, Y.
- Affiliations
- Beijing University of Posts and Telecommunications (CN)
- Venue
- 2018 IEEE Wireless Communications and Networking Conference (WCNC)
- Publisher
- IEEE
- Published
- 2018-04
- Pages
- 1-6
- ISSN
- 1525-3511 (ISSN-L)
- OpenAlex
- W2808381205
- Cited by
- 548
- References
- 17
- Citations by year
- 2026: 16 · 2025: 45 · 2024: 73 · 2023: 83 · 2022: 92 · 2021: 96 · 2020: 94 · 2019: 45 · 2018: 4
- FWCI
- 49.38
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Wireless Communications and Networking Conference (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Wireless Network Optimization
- Physical Sciences › Engineering › Aerospace Engineering › UAV Applications and Optimization
- Keywords
- mobile edge computing, computation offloading, resource allocation, deep reinforcement learning, Q-learning, multi-user systems, energy consumption, delay cost, 5G
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.