Online Learning for Offloading and Autoscaling in Energy Harvesting Mobile Edge Computing
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
Mobile edge computing (also known as fog computing) has recently emerged to enablein-situprocessing of delay-sensitive applications at the edge of mobile networks. Providing grid power supply in support of mobile edge computing, however, is costly and even infeasible (in certain rugged or under-developed areas), thus mandating on-site renewable energy as a major or even sole power supply in increasingly many scenarios. Nonetheless, the high intermittency and unpredictability of renewable energy make it very challenging to deliver a high quality of service to users in energy harvesting mobile edge computing systems. In this paper, we address the challenge of incorporating renewables into mobile edge computing and propose an efficient reinforcement learning-based resource management algorithm, which learns on-the-fly the optimal policy of dynamic workload offloading (to the centralized cloud) and edge server provisioning to minimize the long-term system cost (including both service delay and operational cost). Our online learning algorithm uses a decomposition of the (offline) value iteration and (online) reinforcement learning, thus achieving a significant improvement of learning rate and run-time performance when compared to standard reinforcement learning algorithms such as${Q}$-learning. We prove the convergence of the proposed algorithm and analytically show that the learned policy has a simple monotone structure amenable to practical implementation. Our simulation results validate the efficacy of our algorithm, which significantly improves the edge computing performance compared to fixed or myopic optimization schemes and conventional reinforcement learning algorithms.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“At this background, large state and action spaces in the whole offloading problem make the conventional learning algorithms [245] [256] [247] infeasible actually.”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [245]
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · cited 5×
“The proposed PDS-based learning algorithm is compared with three benchmark schemes: • Q-learning [10]: Q-learning is a famous model-free reinforcement learning technique for solving MDP problems.”
BackgroundMethodInfluentialReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
(1995) · Cited by 10,558
Edge computing: Vision and challenges
Shi, W., Cao, J., Zhang, Q., Li, Y., & Xu, L. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2579198 · Cited by 8,733 · cited 4×
“backbone networks [1] but also result in (sometimes intolerable) large transmission latencies that degrade the quality of service [2]–[4].”
BackgroundInfluentialMarkov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037 · cited 1×
“battery state b and the power demand action a for any given λ,e,h. This is to prove that ∀b ≤ b′ and a ≤ a′, we have C(b,a ′)− C(b,a) ≥ C(b ,a′)− C(b′,a) (32) If the above is true, then we can apply [27] (Section 4.7) to show that π(s) is non-decreasing in b. Equation (32) is equivalent to V∗(b− a ′)− V ∗(b −a) ≥ V∗(b…”
InfluentialA Survey on Mobile Edge Computing: The Communication Perspective
Mao, Y., You, C., Zhang, J., Huang, K., & Letaief, K. B. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2745201 · Cited by 5,607
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Baxter, L. A. (1995) · Technometrics · doi:10.1080/00401706.1995.10484354 · Cited by 2,415
Fog and IoT: An Overview of Research Opportunities
Chiang, M., & Zhang, T. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2584538 · Cited by 2,362 · cited 2×
“Fog computing [5] is a related concept that refers to the same computing paradigm.”
BackgroundStochastic Network Optimization with Application to Communication and Queueing Systems
Neely, M. J. (2010) · Synthesis lectures on communication networks · doi:10.2200/s00271ed1v01y201006cnt007 · Cited by 2,218
Energy Harvesting Sensor Nodes: Survey and Implications
Sudevalayam, S., & Kulkarni, P. (2011) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2011.060710.00094 · Cited by 1,981 · cited 2×
“and/or solar radiation is embraced as a major or even sole power supply for edge systems in the field, thanks to the recent advancements of energy harvesting techniques [8], [9].”
Background
Research works printed but not linked: 4
- reiss, 2011, Google Cluster-usage Traces Format+ Schema, 1
- patel, 2014, Mobile-Edge Computing—Introductory Technical White Paper
- 2017, Elia Power Generation
- rivera, 2014, Gartner Says the Internet of Things Will Transform the Data Center
Cited by
Ensemble deep learning: A review
Ganaie, M. A., Hu, M., Malik, A. K., Tanveer, M., & Suganthan, P. N. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105151 · Cited by 2,143
“With the limitation of getting the complete data in real-world problems, online/incremental learning has been applied to various tasks like learning social representations [174], fog computing [175], identifying suspicious URLs [176], etc.”
MethodConvergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
“At this background, large state and action spaces in the whole offloading problem make the conventional learning algorithms [245], [256], [247] infeasible actually.”
BackgroundPham, Q.-V., Fang, F., Ha, V. N., Piran, M. J., Le, M., Le, L. B., Hwang, W.-J., & Ding, Z. (2020) · IEEE Access · doi:10.1109/access.2020.3001277 · Cited by 870
“The literature [106] jointly studied the offloading and autoscaling policy (i.”
Computation Rate Maximization in UAV-Enabled Wireless-Powered Mobile-Edge Computing Systems
Zhou, F., Wu, Y., Hu, R. Q., & Qian, Y. (2018) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2018.2864426 · Cited by 851
Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., & Bennis, M. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2876279 · Cited by 680
Joint Service Caching and Task Offloading for Mobile Edge Computing in Dense Networks
Xu, J., Chen, L., & Zhou, P. (2018) · IEEE INFOCOM 2018 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2018.8485977 · Cited by 585
Self-citationEdge Computing with Artificial Intelligence: A Machine Learning Perspective
Hua, H., Li, Y., Wang, T., Dong, N., Li, W., & Cao, J. (2023) · ACM Computing Surveys · doi:10.1145/3555802 · Cited by 563
“[91] propose an algorithm that uses the special structure of state transitions of the considered EC system to overcome the curse-of-dimensionality problem.”
BackgroundMethodLearning-Based Computation Offloading for IoT Devices With Energy Harvesting
Min, M., Xiao, L., Chen, Y., Cheng, P., Wu, D., & Zhuang, W. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2890685 · Cited by 560
Edge-Computing-Enabled Smart Cities: A Comprehensive Survey
Khan, L. U., Yaqoob, I., Tran, N. H., Kazmi, S. M. A., Dang, T. N., & Hong, C. S. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2987070 · Cited by 475
“There are significant variations in the harvested energy from both natural sources and radio-frequency sources [133].”
Dynamic Task Offloading and Resource Allocation for Ultra-Reliable Low-Latency Edge Computing
Liu, C.-F., Bennis, M., Debbah, M., & Poor, H. V. (2019) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2898573 · Cited by 435
Sensing, Computing, and Communications for Energy Harvesting IoTs: A Survey
Ma, D., Lan, G., Hassan, M., Hu, W., & Das, S. K. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2962526 · Cited by 336
“At the edge servers, a post-decision state (PDS) based learning algorithm is proposed in [154] to obtain the optimal workload offloading (to the centralized cloud) policy, which minimizes the long-term system cost.”
Shakarami, A., Ghobaei-Arani, M., & Shahidinejad, A. (2020) · Computer Networks · doi:10.1016/j.comnet.2020.107496 · Cited by 331
Deep Learning Empowered Task Offloading for Mobile Edge Computing in Urban Informatics
Zhang, K., Zhu, Y., Leng, S., He, Y., Maharjan, S., & Zhang, Y. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2903191 · Cited by 309
A survey on computation offloading modeling for edge computing
Lin, H., Zeadally, S., Chen, Z., Labiod, H., & Wang, L. (2020) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2020.102781 · Cited by 305
AI Models for Green Communications Towards 6G
Mao, B., Tang, F., Kawamoto, Y., & Kato, N. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3130901 · Cited by 300
“Similar to [113], the authors of [250] also consider AI”
ResultDeep learning for hybrid 5G services in mobile edge computing systems: learn from a digital twin
Dong, R., She, C., Hardjawana, W., Li, Y., & Vucetic, B. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2927312 · Cited by 295
“The authors of [19] proposed an efficient reinforcement learning-based resource management algorithm to incorporate renewable energy into MEC systems.”
BackgroundDeep Reinforcement Learning for Vehicular Edge Computing: An Intelligent Offloading System
Ning, Z., Dong, P., Wang, X., Rodrigues, J. J. P. C., & Xia, F. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3317572 · Cited by 292
“The residual battery capability and renewable energy are considered in Reference [34].”
BackgroundStochastic Computation Offloading and Trajectory Scheduling for UAV-Assisted Mobile Edge Computing
Zhang, J., Zhou, L., Tang, Q., Ngai, E. C.-H., Hu, X., Zhao, H., & Wei, J. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2890133 · Cited by 287
“Therefore, the energy consumption for task execution at SMD i or the UAV during time slot t can be, respectively, denoted as [28]”
MethodIntelligent Offloading in Multi-Access Edge Computing: A State-of-the-Art Review and Framework
Cao, B., Zhang, L., Li, Y., Feng, D., & Cao, W. (2019) · IEEE Communications Magazine · doi:10.1109/mcom.2019.1800608 · Cited by 275
“in [14] propose a novel post-decision state-based reinforcement learning algorithm, which can learn “on the fly” the optimal policy of dynamic workload offloading and edge server provisioning.”
BackgroundPrice-Based Distributed Offloading for Mobile-Edge Computing With Computation Capacity Constraints
Liu, M., & Liu, Y. (2018) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2017.2780128 · Cited by 260
Deep Reinforcement Learning for Collaborative Edge Computing in Vehicular Networks
Li, M., Gao, J., Zhao, L., & Shen, X. (2020) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2020.3003036 · Cited by 251
“Meanwhile, compared to the conventional mobile cloud computing, the network delay caused by task offloading can be significantly reduced in MEC due to the proximity of the edge server to vehicles [8].”
Kai, C., Zhou, H., Yi, Y., & Huang, W. (2021) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2020.3018159 · Cited by 249
“A post-decision state based reinforcement learning algorithm was developed in [25] to learn the optimal offloading and auto scaling policy by exploiting the special structure of the considered problem.”
MethodComputation Efficiency Maximization in Wireless-Powered Mobile Edge Computing Networks
Zhou, F., & Hu, R. Q. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2970920 · Cited by 246
“Similar to the works in [6], [19] and [20], in order to clarify the issues pertaining to CE and permit reaching meaningful insights into the CE maximization problem, it is assumed that all devices are equipped with a single antenna.”
MethodResultRodrigues, T. K., Suto, K., Nishiyama, H., Liu, J., & Kato, N. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2943405 · Cited by 245
“First of all, not all regions and places, especially the ones not in urban areas, have the infrastructure to deploy a fixed server [149], [213].”
BackgroundMethodToward Computation Offloading in Edge Computing: A Survey
Jiang, C., Cheng, X., Gao, H., Zhou, X., & Wan, J. (2019) · IEEE Access · doi:10.1109/access.2019.2938660 · Cited by 235
Wang, F., Xu, J., & Cui, S. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2964765 · Cited by 210
“First, unlike [23]–[25] considering random and uncontrollable energy arrivals from ambient renewable sources (e.”
BackgroundSelf-citationChen, Z., & Wang, X. (2020) · EURASIP Journal on Wireless Communications and Networking · doi:10.1186/s13638-020-01801-6 · Cited by 212
“Furthermore, it was shown in [28] and [29] that an optimal dynamic computation offloading policy can be learned by the emerging reinforcement learning (RL) based algorithm without any prior knowledge of the MEC system.”
BackgroundUAV-Enabled Mobile Edge Computing: Offloading Optimization and Trajectory Design
Zhou, F., Wu, Y., Sun, H., & Chu, Z. (2018) · 2018 IEEE International Conference on Communications (ICC) · doi:10.1109/icc.2018.8422277 · Cited by 206
Learning-Based Privacy-Aware Offloading for Healthcare IoT With Energy Harvesting
Min, M., Wan, X., Xiao, L., Chen, Y., Xia, M., Wu, D., & Dai, H. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2875926 · Cited by 213
“For instance, a renewable-powered MEC system as investigated in [15] combines the value iteration with the RL technique to improve the edge computing performance of the mobile device for delay-sensitive applications with intermittent and unpredictable renewable energy.”
MethodFogNetSim++: A Toolkit for Modeling and Simulation of Distributed Fog Environment
Qayyum, T., Malik, A. W., Khan Khattak, M. A., Khalid, O., & Khan, S. U. (2018) · IEEE Access · doi:10.1109/access.2018.2877696 · Cited by 196
“[28] proposed a dynamic energy management framework that learn on-the-fly and dynamically manage the workload offloading mechanism.”
BackgroundHaibeh, L. A., Yagoub, M. C. E., & Jarray, A. (2022) · IEEE Access · doi:10.1109/access.2022.3152787 · Cited by 190
Collaborative Service Placement for Edge Computing in Dense Small Cell Networks
Chen, L., Shen, C., Zhou, P., & Xu, J. (2021) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2945956 · Cited by 189
“, stochastic task arrivals [10], energy efficiency [3], [11], collaborative offloading [12] etc.”
BackgroundSelf-citationMachine and Deep Learning for Resource Allocation in Multi-Access Edge Computing: A Survey
Djigal, H., Xu, J., Liu, L., & Zhang, Y. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3199544 · Cited by 166
“The simulation results show that the proposed joint offloading and scheduling algorithm outperforms the delay-optimal algorithm [307] (which only considers the delay cost of task offloading ignoring the energy consumption), the energy-optimal algorithm [303] (that only considers the energy cost ignoring delay cost), a…”
BackgroundSelf-citationWireless Powered Mobile Edge Computing Networks: A Survey
Wang, X., Li, J., Ning, Z., Song, Q., Guo, L., Guo, S., & Obaidat, M. S. (2023) · ACM Computing Surveys · doi:10.1145/3579992 · Cited by 166
“Generally, ML algorithms used for WPMEC systems include Q-learning and actor-critic, which maximize the long-term performance through continuous learning [118, 129].”
BackgroundMethodInfluentialShakarami, A., Ghobaei-Arani, M., Masdari, M., & Hosseinzadeh, M. (2020) · Journal of Grid Computing · doi:10.1007/s10723-020-09530-2 · Cited by 167
Performance Optimization in Mobile-Edge Computing via Deep Reinforcement Learning
Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., & Bennis, M. (2018) · 2018 IEEE 88th Vehicular Technology Conference (VTC-Fall) · doi:10.1109/vtcfall.2018.8690980 · Cited by 158
Wang, T., Lu, Y., Wang, J., Dai, H.-N., Zheng, X., & Jia, W. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3060484 · Cited by 159
Asheralieva, A., & Niyato, D. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2923702 · Cited by 155
“On the other hand, the methods with incomplete information [8], [13], [15] present the offloading problem as a noncooperative game where players do not share their resources.”
BackgroundMethodComputation Offloading in Beyond 5G Networks: A Distributed Learning Framework and Applications
Chen, X., Wu, C., Liu, Z., Zhang, N., & Ji, Y. (2021) · IEEE Wireless Communications · doi:10.1109/mwc.001.2000296 · Cited by 160
“put forward an online reinforcement learning (RL) algorithm [7].”
BackgroundPham, Q.-V., Leanh, T., Tran, N. H., Park, B. J., & Hong, C. S. (2018) · IEEE Access · doi:10.1109/access.2018.2882800 · Cited by 146
“Most of existing works in MEC focus on solutions for single-server MEC homogeneous networks [10], [14], [15], single-server MEC HetNets [12], [13], [20], and different aspects in multiple-server MEC networks, for example, computation offloading [21], [22], computation offloading and server provisioning [23], [24], and…”
BackgroundUAV-Enabled Mobile Edge-Computing for IoT Based on AI: A Comprehensive Review
Yazid, Y., Ez‐zazi, I., González, A. G., Oualkadi, A. E., & Arioua, M. (2021) · Drones · doi:10.3390/drones5040148 · Cited by 166
Machine Learning Meets Communication Networks: Current Trends and Future Challenges
Ahmad, I., Shahabuddin, S., Malik, H. H., Harjula, E., Leppänen, T., Lovén, L., Anttonen, A., Sodhro, A. H., Alam, M. M., Juntti, M. J., et al. (2020) · IEEE Access · doi:10.1109/access.2020.3041765 · Cited by 160
“The joint offloading and autoscaling problem in energy harvesting MEC systems has been studied in [304].”
BackgroundDynamic Edge Computation Offloading for Internet of Things With Energy Harvesting: A Learning Method
Wei, Z., Zhao, B., Su, J., & Lu, X. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2882783 · Cited by 133
“systems [18]–[21], most of them focus on the resource management problem and the computation offloading problem is seldom considered.”
BackgroundDeep Learning-Constructed Joint Transmission-Recognition for Internet of Things
Lee, C.-H., Lin, J.-W., Chen, P.-H., & Chang, Y.-C. (2019) · IEEE Access · doi:10.1109/access.2019.2920929 · Cited by 126
“In addition, computation and energy saving issues of IoT devices have recently been investigated [44]–[46], but their approaches are very different from ours.”
BackgroundDependent task placement and scheduling with function configuration in edge computing
Liu, L., Tan, H., Jiang, S. H.-C., Han, Z., Li, X., & Huang, H. (2019) · International Workshop on Quality of Service · doi:10.1145/3326285.3329055 · Cited by 136
“Chen et al. [8] proposed an offloading policy to minimize the delay to meet the low latency requirement, where energy efficiency is considered to prolong the battery life of the mobile devices [17, 26, 27].”
BackgroundReinforcement Learning Methods for Computation Offloading: A Systematic Review
Zabihi, Z., Moghadam, A. M. E., & Rezvani, M. H. (2023) · ACM Computing Surveys · doi:10.1145/3603703 · Cited by 157
“Some examples are energy cost plus monetary cost [ 99 ], energy cost plus bandwidth and buffer queue length [ 86 ], and service delay cost plus operating cost [ 100 ].”
BackgroundA Reinforcement Learning and Blockchain-Based Trust Mechanism for Edge Networks
Xiao, L., Ding, Y., Jiang, D., Huang, J., Wang, D., Li, J., & Vincent Poor, H. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.2995371 · Cited by 123
“The edge computation resource allocation scheme in [25] uses a post-decision state algorithm to determine the offloading workload and processing speed to address network congestion, reduce the service delay and operation cost in secure networks.”
MethodLearning-Based URLLC-Aware Task Offloading for Internet of Health Things
Zhou, Z., Wang, Z., Yu, H., Liao, H., Mumtaz, S., Oliveira, L., & Frascolla, V. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3020680 · Cited by 121
“literatures focus on the time-average QoS performance metrics such as time-average delay, time-average energy consumption, and mean rate stability of queue backlog [15], [16].”
BackgroundPerspectives in flow-induced vibration energy harvesting
Wang, J., Yurchenko, D., Hu, G., Zhao, L., Tang, L., & Yang, Y. (2021) · Applied Physics Letters · doi:10.1063/5.0063488 · Cited by 122
Hu, H., Wang, Q., Hu, R. Q., & Zhu, H. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3081983 · Cited by 119
Spatio–Temporal Edge Service Placement: A Bandit Learning Approach
Chen, L., Xu, J., Ren, S., & Zhou, P. (2018) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2018.2876823 · Cited by 122
Self-citationSun, J., Gu, Q., Zheng, T., Dong, P., Valera, A., & Qin, Y. (2020) · IEEE Access · doi:10.1109/access.2020.2965620 · Cited by 104
“[9] studied an energy harvesting MEC system that makes offloading and edge server provisioning decisions based on unique information such as available battery power and anticipated renewable power arrival.”
BackgroundGuo, K., Gao, R., Xia, W., & Quek, T. Q. S. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3038875 · Cited by 110
Resource Management in Mobile Edge Computing: A Comprehensive Survey
Zhang, X., & Debroy, S. (2023) · ACM Computing Surveys · doi:10.1145/3589639 · Cited by 137
A Machine Learning Approach for Task and Resource Allocation in Mobile-Edge Computing-Based Networks
Wang, S., Chen, M., Liu, X., Yin, C., Cui, S. R., & Poor, H. V. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3011286 · Cited by 113
“The work in [10] developed a resource management algorithm to minimize the long-term system energy cost.”
BackgroundZhang, X., Zhong, Y., Liu, P., Zhou, F., & Wang, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2935217 · Cited by 101
“After the MEC server performs computation, the computation results can be transmitted back to the smart mobile devices [15]–[17].”
BackgroundMethodIntelligent workload allocation in IoT–Fog–cloud architecture towards mobile edge computing
Abbasi, M., Mohammadi-Pasand, E., & Khosravi, M. R. (2021) · Computer Communications · doi:10.1016/j.comcom.2021.01.022 · Cited by 109
Cost-Effective Edge Server Placement in Wireless Metropolitan Area Networks
Zeng, F., Ren, Y., Deng, X., & Li, W. (2018) · Sensors · doi:10.3390/s19010032 · Cited by 109
“Most studies assumed that the edge servers they need have been properly deployed and merely concentrate on the offloading and resource allocation strategies [9,10].”
BackgroundAbbasi, M., Yaghoobikia, M., Rafiee, M., Jolfaei, A., & Khosravi, M. R. (2020) · Computer Communications · doi:10.1016/j.comcom.2020.02.017 · Cited by 110
Trading off Between User Coverage and Network Robustness for Edge Server Placement
Cui, G., He, Q., Chen, F., Jin, H., & Yang, Y. J. (2020) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2020.3008440 · Cited by 103
Edge AI for Internet of Medical Things: A literature review
Rocha, A. R. D., Monteiro, M. S., Mattos, C. L. C., Dias, M. L. D., Soares, J. B., Magalhães, R. P., & de Macêdo, J. A. F. (2024) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2024.109202 · Cited by 131
Li, L., Cheng, Q., Tang, X., Bai, T., Chen, W., Ding, Z., & Han, Z. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3033843 · Cited by 97
“In [26], the authors proposed an effective resource management algorithm based on RL, which learned the optimal offloading policy online.”
An Air-Ground Integration Approach for Mobile Edge Computing in IoT
Zhou, Z., Feng, J., Tan, L., He, Y., & Gong, J. (2018) · IEEE Communications Magazine · doi:10.1109/mcom.2018.1701111 · Cited by 91
“• [7] • Provide MEC services for renewable”
Zhang, Y., Liu, Y., Zhou, J., Sun, J. Y., & Li, K. (2020) · Future Generation Computer Systems · doi:10.1016/j.future.2020.05.025 · Cited by 91
Energy-Aware Computation Offloading and Transmit Power Allocation in Ultradense IoT Networks
Guo, H., Zhang, J., Liu, J., & Zhang, H. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2875535 · Cited by 94
“[38] proposed a post-decision state-based learning algorithm to learn the optimal joint offloading and autoscaling policy on-the-fly, with the objective of minimizing the long-term system cost in terms of the service delay and operational cost.”
BackgroundJazayeri, F., Shahidinejad, A., & Ghobaei‐Arani, M. (2020) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-020-02561-3 · Cited by 95
“In another study, RL was used for offloading and automatic scaling in the energy harvesting of the MEC, namely, Post-Decision State (PDS) (Xu et al. 2017).”
MethodOnline Computation Offloading and Resource Scheduling in Mobile-Edge Computing
Liu, T., Zhang, Y., Zhu, Y., Tong, W., & Yang, Y. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3051427 · Cited by 90
“[13] considered the computing capacities of edge servers are adjustable, and an efficient reinforcement learning-based algorithm is provided to manage the computing resources, with the goal to minimize the system cost.”
BackgroundSeid, A. M., Lu, J., Abishu, H. N., & Ayall, T. A. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3213352 · Cited by 91
“Recent works have been done on machine learning-based offloading with EH/transfer in IoT edge network [13], [14], [15], [16].”
BackgroundWireless Powered Mobile Edge Computing: Dynamic Resource Allocation and Throughput Maximization
Deng, X., Li, J., Shi, L. X., Wei, Z., Zhou, X., & Yuan, J. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3034479 · Cited by 103
Jazayeri, F., Shahidinejad, A., & Ghobaei‐Arani, M. (2020) · The Journal of Supercomputing · doi:10.1007/s11227-020-03476-8 · Cited by 90
“The paper [35] combined task offloading and resource provisioning as an MDP.”
BackgroundMethodDependency-Aware Application Assigning and Scheduling in Edge Computing
Liao, H., Li, X., Guo, D., Kang, W., & Li, J. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3104015 · Cited by 85
Task offloading in mobile fog computing by classification and regression tree
Rahbari, D., & Nickray, M. (2019) · Peer-to-Peer Networking and Applications · doi:10.1007/s12083-019-00721-7 · Cited by 94
“[43] joint task offloading and edge device provisioning as a Markov decision process.”
BackgroundCE-IoT: Cost-Effective Cloud-Edge Resource Provisioning for Heterogeneous IoT Applications
Zhou, Z., Yu, S., Chen, W., & Chen, X. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2994308 · Cited by 86
“[12] proposed an online learning-based strategy to jointly provision resource at the cloud side and manage the energy consumption at the power consumption at the edge side.”
BackgroundZhang, G., Yan, C., Shen, Z., & Wang, L. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2875909 · Cited by 79
“The MEC systems with renewables has been studied in [20], [32], and [33].”
BackgroundMethodSocially trusted collaborative edge computing in ultra dense networks
Chen, L., & Xu, J. (2017) · Proceedings of the Second ACM/IEEE Symposium on Edge Computing · doi:10.1145/3132211.3134451 · Cited by 78
Self-citationShahidinejad, A., & Ghobaei‐Arani, M. (2020) · Software: Practice and Experience · doi:10.1002/spe.2888 · Cited by 79
“3 Although some approaches have already been investigated by jointly the edge server provisioning and computation offloading issues, 4,5 still more effort are necessitated for managing varying workloads better, dynamically.”
BackgroundAdaptive Fog Configuration for the Industrial Internet of Things
Chen, L., Zhou, P., Gao, L., & Xu, J. (2018) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2846549 · Cited by 78
“Task offloading has been a central theme of many prior works [6], [7], which concerns what/when/how to offload workload of end devices to FNs.”
BackgroundSelf-citationGrzesik, P., & Mrozek, D. (2024) · Electronics · doi:10.3390/electronics13030640 · Cited by 75
“One of the approaches involves energy harvesting techniques based on solar panels, wind mills, or electromagnetic signals that allow the edge devices to recharge their batteries without external intervention, making these systems more resilient and autonomous [32–34].”
MethodDistributed Multi-Cloud Multi-Access Edge Computing by Multi-Agent Reinforcement Learning
Zhang, Y., Di, B., Zheng, Z., Lin, J., & Song, L. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3043038 · Cited by 75
“Note that the time interval between the publication of two tasks is much larger than the duration for obtaining the optimal scheme [37].”
BackgroundPeace: Privacy-Preserving and Cost-Efficient Task Offloading for Mobile-Edge Computing
He, X., Jin, R., & Dai, H. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2958091 · Cited by 75
“Offloading strategy for mobile devices with energy harvesting capability is examined in [6], [7].”
BackgroundWhen Edge Computing Meets Microgrid: A Deep Reinforcement Learning Approach
Munir, M. S., Abedin, S. F., Tran, N. H. K., & Hong, C. S. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2899673 · Cited by 74
“usages and the amount of stored energy at time slot t is as follows [38]:”
BackgroundReinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence
Wei, P., Guo, K., Li, Y., Wang, J., Feng, W., Jin, S., Ge, N., & Liang, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3183647 · Cited by 88
“post-decision state of CSI [122], the privacy-aware task offloading RL algorithm accelerated by a Dyna architecture is adopted to protect the privacy of user location and usage pattern [118].”
InfluentialA Survey of Computation Offloading With Task Types
Zhang, S., Yi, N., & Ma, Y. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3410896 · Cited by 102
“Therefore, in [105], the authors considered both the green power collected by EH and the energy of the battery of the mobile device itself.”
BackgroundMa, H., Huang, P., Zhou, Z., Zhang, X., & Chen, X. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3147027 · Cited by 77
“Definitely, the researches of MEC systems with energy harvesting module also has attracted great attention from researches [28], [29].”
BackgroundMethodData Offloading in Mobile Edge Computing: A Coalition and Pricing Based Approach
Zhang, T. (2017) · IEEE Access · doi:10.1109/access.2017.2785265 · Cited by 78
“In [12], energy harvesting aided MEC has been investigated, and an online learning algorithm for offloading and auto-scaling is proposed.”
BackgroundTask Offloading for UAV-based Mobile Edge Computing via Deep Reinforcement Learning
Li, J., Liu, Q., Wu, P., Shu, F., & Jin, S. (2018) · IEEE/CIC International Conference on Communications in China (ICCC) · doi:10.1109/iccchina.2018.8641189 · Cited by 68
“The work in [11] proposes an online learning algorithm to decide how to process the computing tasks in order to minimize the delay cost.”
MethodDing, Y., Liu, C., Zhou, X., Liu, Z., & Tang, Z. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2951206 · Cited by 69
Energy- and performance-aware load-balancing in vehicular fog computing
Hameed, A. R., Islam, S. U., Ahmad, I., & Munir, K. (2020) · Sustainable Computing Informatics and Systems · doi:10.1016/j.suscom.2020.100454 · Cited by 87
Shaping future low-carbon energy and transportation systems: Digital technologies and applications
Song, J., He, G., Wang, J., & Zhang, P. (2022) · iEnergy · doi:10.23919/ien.2022.0040 · Cited by 80
Wang, F., Xing, H., & Xu, J. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3011990 · Cited by 65
“, machine learning algorithms [21], [23], [33].”
Self-citationHeidari, A., Jamali, M. A. J., Navimipour, N. J., & Akbarpour, S. (2022) · Applied Sciences · doi:10.3390/app12168232 · Cited by 66
Ke, H., Wang, J., Wang, H., & Ge, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2959348 · Cited by 69
“introduced a joint offloading and edge server provisioning problem based on RL to minimize the long-term system cost and used a post-decision state(PDS) learning algorithm to implement the special structure of state transitions in MEC system [27].”
MethodMunir, M. S., Abedin, S. F., Tran, N. H. K., Han, Z., Huh, E., & Hong, C. S. (2021) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2021.3049381 · Cited by 79
Siyadatzadeh, R., Mehrafrooz, F., Ansari, M., Safaei, B., Shafique, M., Henkel, J., & Ejlali, A. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3240007 · Cited by 65
“To address the limitations of cloud computing, fog computing was introduced to mitigate the amount of delay by executing a number of tasks near the network’s edge without transmitting them to cloud servers [11].”
MethodEnd-to-End Performance-Based Autonomous VNF Placement With Adopted Reinforcement Learning
Bunyakitanon, M., Vasilakos, X., Nejabati, R., & Simeonidou, D. E. (2020) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2020.2988486 · Cited by 60
“Most solutions use SL for resource allocation such in [6], [7].”
MethodWang, Z., Lv, T., & Chang, Z. (2022) · Computer Networks · doi:10.1016/j.comnet.2021.108732 · Cited by 61
UbiPriSEQ—Deep Reinforcement Learning to Manage Privacy, Security, Energy, and QoS in 5G IoT HetNets
Mohammed, T., Albeshri, A., Katib, I., & Mehmood, R. (2020) · Applied Sciences · doi:10.3390/app10207120 · Cited by 57
A new task offloading algorithm in edge computing
Zhang, Z., Li, C., Peng, S., & Pei, X. (2021) · EURASIP Journal on Wireless Communications and Networking · doi:10.1186/s13638-021-01895-6 · Cited by 63
Edge Computing for Cyber-physical Systems: A Systematic Mapping Study Emphasizing Trustworthiness
Sánchez, J. M. G., Jörgensen, N., Törngren, M., Inam, R., Berezovskyi, A., Feng, L., Fersman, E., Ramli, M. R., & Tan, K. (2022) · ACM Transactions on Cyber-Physical Systems · doi:10.1145/3539662 · Cited by 66
“Nevertheless, other factors, such as the operational cost [71], network utility [61], quality-of-experience [49], and robustness of transmission network [9], are also considered as the objectives to be optimized in the application of edge computing system for CPS.”
BackgroundMachine learning-based computation offloading in multi-access edge computing: A survey
Choudhury, A., Ghose, M., Islam, A., & Yogita, Y. (2024) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2024.103090 · Cited by 72
Metadata
- Authors
- Xu, J., Chen, L., & Ren, S.
- Affiliations
- University of Miami (US)
- University of California, Riverside (US)
- Venue
- IEEE Transactions on Cognitive Communications and Networking
- Published
- 2017-09
- Volume
- 3
- Issue
- 3
- Pages
- 361-373
- ISSN
- 2332-7731 (ISSN-L)
- OpenAlex
- W2599106590
- Cited by
- 368
- References
- 33
- Citations by year
- 2026: 14 · 2025: 15 · 2024: 32 · 2023: 24 · 2022: 41 · 2021: 55 · 2020: 87 · 2019: 59 · 2018: 34 · 2017: 7
- FWCI
- 34.02
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Cognitive Communications and Networking (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- U.S. NSF (CNS-1551661, CNS-1565474, ECCS-1610471)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Physical Sciences › Computer Science › Information Systems › Cloud Computing and Resource Management
- Keywords
- mobile edge computing, fog computing, energy harvesting, renewable energy, workload offloading, auto-scaling, reinforcement learning, online learning, service delay, operating costs, convergence analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.