Optimized Computation Offloading Performance in Virtual Edge Computing Systems Via Deep Reinforcement Learning
Open accessThe DOI matches its Crossref record; 42 of the 43 references it cites were found in a registry or an index.
Abstract
To improve the quality of computation experience for mobile devices, mobile-edge computing (MEC) is a promising paradigm by providing computing capabilities in close proximity within a sliced radio access network (RAN), which supports both traditional communication and MEC services. Nevertheless, the design of computation offloading policies for a virtual MEC system remains challenging. Specifically, whether to execute a computation task at the mobile device or to offload it for MEC server execution should adapt to the time-varying network dynamics. This paper considers MEC for a representative mobile user in an ultradense sliced RAN, where multiple base stations (BSs) are available to be selected for computation offloading. The problem of solving an optimal computation offloading policy is modeled as a Markov decision process, where our objective is to maximize the long-term utility performance whereby an offloading decision is made based on the task queue state, the energy queue state as well as the channel qualities between mobile user and BSs. To break the curse of high dimensionality in state space, we first propose a double deep Q-network (DQN)-based strategic computation offloading algorithm to learn the optimal policy without knowing a priori knowledge of network dynamics. Then motivated by the additive structure of the utility function, a Q-function decomposition technique is combined with the double DQN, which leads to a novel learning algorithm for the solving of stochastic computation offloading. Numerical experiments show that our proposed learning algorithms achieve a significant improvement in computation offloading performance compared with the baseline policies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These attacks degrade the performance of VANET which can cause critical problems and affect the stability of the networks [10].”
From X-GEVON- a novel explainable intelligent network to detect the multiple attacks in vanet systems (2025) · cites this paper as [10]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
A Survey on Mobile Edge Computing: The Communication Perspective
Mao, Y., You, C., Zhang, J., Huang, K., & Letaief, K. B. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2745201 · Cited by 5,607
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
Mobile Edge Computing: A Survey on Architecture and Computation Offloading
Mach, P., & Becvar, Z. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2682318 · Cited by 3,258
The emergence of edge computing
Satyanarayanan, M. (2017) · Computer · doi:10.1109/mc.2017.9 · Cited by 2,832
Efficient multi-user computation offloading for mobile-edge cloud computing
Chen, X., Jiao, L., Li, W., & Fu, X. (2016) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2015.2487344 · Cited by 2,675
Self-citationWhat is the best multi-stage architecture for object recognition?
Jarrett, K., Kavukcuoglu, K., Ranzato, M. A., & LeCun, Y. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459469 · Cited by 2,153
Chen, C. L. P., & Liu, Z. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2716952 · Cited by 1,957
Dynamic computation offloading for mobile-edge computing with energy harvesting devices
Mao, Y., Zhang, J., & Letaief, K. B. (2016) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2016.2611964 · Cited by 1,705
Energy-Efficient Resource Allocation for Mobile-Edge Computation Offloading
You, C., Huang, K., Chae, H., & Kim, B.-H. (2017) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2016.2633522 · Cited by 1,503
On-line Q-learning Using Connectionist Systems
Rummery, G. A. (1994) · Cited by 1,477
Reinforcement learning for robots using neural networks
Lin, L.-J. (1992) · Defense Technical Information Center (DTIC) · Cited by 1,043
Joint Offloading and Computing Optimization in Wireless Powered Mobile-Edge Computing Systems
Wang, F., Xu, J., Wang, X., & Cui, S. (2018) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2017.2785305 · Cited by 922
Delay-optimal computation task scheduling for mobile-edge computing systems
Liu, J., Mao, Y., Zhang, J., & Letaief, K. B. (2016) · 2016 IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2016.7541539 · Cited by 759
Wang, C., Liang, C., Yu, F. R., Chen, Q., & Tang, L. (2017) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2017.2703901 · Cited by 710
Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms
Singh, S., Jaakkola, T., Littman, M. L., & Szepesvári, C. (2000) · Machine Learning · doi:10.1023/a:1007678930559 · Cited by 609
Processor design for portable systems
Burd, T. D., & Brodersen, R. W. (1996) · Journal of VLSI signal processing systems for signal, image and video technology · doi:10.1007/bf01130406 · Cited by 593
Dynamic energy-aware cloudlet-based mobile cloud computing model for green computing
Gai, K., Qiu, M., Zhao, H., Tao, L., & Zong, Z. (2016) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2015.05.016 · Cited by 489
Online Learning for Offloading and Autoscaling in Energy Harvesting Mobile Edge Computing
Xu, J., Chen, L., & Ren, S. (2017) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2017.2725277 · Cited by 368
NVS: A Substrate for Virtualizing Wireless Resources in Cellular Networks
Kokku, R., Mahindra, R., Zhang, H., & Rangarajan, S. (2012) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2011.2179063 · Cited by 329
Self-citationWireless Powered Cooperation-Assisted Mobile Edge Computing
Hu, X., Wong, K.-K., & Yang, K. (2018) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2018.2794345 · Cited by 318
Radio access network virtualization for future mobile carrier networks
Costa-Perez, X., Swetina, J., Guo, T., Mahindra, R., & Rangarajan, S. (2013) · IEEE Communications Magazine · doi:10.1109/mcom.2013.6553675 · Cited by 288
Energy-Efficient Admission of Delay-Sensitive Tasks for Mobile Edge Computing
Lyu, X., Tian, H., Ni, W., Zhang, Y., Zhang, P., & Liu, R. P. (2018) · IEEE Transactions on Communications · doi:10.1109/tcomm.2018.2799937 · Cited by 213
Latency and Reliability-Aware Task Offloading and Resource Allocation for Mobile Edge Computing
Liu, C.-F., Bennis, M., & Poor, H. V. (2017) · 2017 IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/glocomw.2017.8269175 · Cited by 201
Self-citationRelaxations of Weakly Coupled Stochastic Dynamic Programs
Adelman, D., & Mersereau, A. J. (2008) · Operations Research · doi:10.1287/opre.1070.0445 · Cited by 198
Optimal resource allocation using reinforcement learning for IoT content-centric services
Gai, K., & Qiu, M. (2018) · Applied Soft Computing · doi:10.1016/j.asoc.2018.03.056 · Cited by 192
Zhou, Y., Yu, F. R., Chen, J., & Kuo, Y. (2017) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2737028 · Cited by 165
Performance Optimization in Mobile-Edge Computing via Deep Reinforcement Learning
Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., & Bennis, M. (2018) · 2018 IEEE 88th Vehicular Technology Conference (VTC-Fall) · doi:10.1109/vtcfall.2018.8690980 · Cited by 158
Self-citationQ-decomposition for reinforcement learning agents
Russell, S., & Zimdars, A. L. (2003) · International Conference on Machine Learning · Cited by 146
Energy Delay Tradeoff in Cloud Offloading for Multi-Core Mobile Devices
Jiang, Z., & Mao, S. (2015) · IEEE Access · doi:10.1109/access.2015.2499300 · Cited by 89
Self-citationA Secure Content Caching Scheme for Disaster Backup in Fog Computing Enabled Mobile Social Networks
Su, Z., Xu, Q., Luo, J., Pu, H., Peng, Y., & Lu, R. (2018) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2849984 · Cited by 78
Wireless Resource Scheduling in Virtualized Radio Access Networks Using Stochastic Learning
Chen, X., Han, Z., Zhang, H., Xue, G., Xiao, Y., & Bennis, M. (2018) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2017.2742949 · Cited by 55
Self-citationProcessor-Network Speed Scaling for Energy–Delay Tradeoff in Smartphone Applications
Kwak, J., Choi, O., Chong, S., & Mohapatra, P. (2015) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2015.2419219 · Cited by 48
Information Prediction and Dynamic Programming-Based RAN Slicing for Mobile Edge Computing
Zhao, P., Tian, H., Fan, S., & Paulraj, A. J. (2018) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2018.2802522 · Cited by 37
Smart Resource Allocation Using Reinforcement Learning in Content-Centric Cyber-Physical Systems
Gai, K., Qiu, M., Liu, M., & Zhao, H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-73830-7_5 · Cited by 13
Multipath Transmission Scheduling in Millimeter Wave Cloud Radio Access Networks
Chen, X., Liu, P., Liu, H., Wu, C., & Ji, Y. (2018) · International Conference on Communications · doi:10.1109/icc.2018.8422564 · Cited by 6
Self-citationFast Adaptation of Activity Sensing Policies in Mobile Devices
Alsheikh, M. A., Niyato, D. T., Lin, S., Tan, H.-P., & Kim, D. I. (2016) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2016.2628966 · Cited by 3
Research works printed but not linked: 1
- 2017, Cisco Visual Networking Index Global Mobile Data Traffic Forecast Update
Cited by
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,060
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
“IoT edge environments powered by Energy Harvesting (EH) is investigated in [251], [254].”
BackgroundMethodSelf-citationEdge Intelligence: The Confluence of Edge Computing and Artificial Intelligence
Deng, S., Zhao, H., Fang, W., Yin, J., Dustdar, S., & Zomaya, A. Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2984887 · Cited by 1,147
Huang, L., Bi, S., & Zhang, Y.-J. A. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2928811 · Cited by 1,040
Pham, Q.-V., Fang, F., Ha, V. N., Piran, M. J., Le, M., Le, L. B., Hwang, W.-J., & Ding, Z. (2020) · IEEE Access · doi:10.1109/access.2020.3001277 · Cited by 870
Metadata
- Authors
- Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., & Bennis, M.
- Affiliations
- VTT Technical Research Centre of Finland (FI)
- Zhejiang University (CN)
- University of Electro-Communications (JP)
- Auburn University (US)
- National Institute of Informatics (JP)
- Venue
- IEEE Internet of Things Journal
- Published
- 2019-06
- Volume
- 6
- Issue
- 3
- Pages
- 4005-4018
- ISSN
- 2327-4662 (ISSN-L)
- 2372-2541
- OpenAlex
- W2963334314
- Cited by
- 680
- References
- 43
- Citations by year
- 2026: 33 · 2025: 50 · 2024: 73 · 2023: 97 · 2022: 142 · 2021: 120 · 2020: 114 · 2019: 47
- FWCI
- 44.14
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Internet of Things Journal (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Basic Research Program of China (2018YFB0803702)
- National Natural Science Foundation of China (61701439, 61731002)
- Zhejiang Key Research and Development Plan (2018C03056)
- National Science Foundation (CNS-1702957)
- Auburn University
- JSPS KAKENHI (JP16H02817)
- Academy of Finland (289611)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Wireless Network Optimization
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Keywords
- computation offloading, mobile edge computing, deep reinforcement learning, double deep Q-network, Markov decision process, ultra-dense networks, base station selection, task queues, utility maximization, mobile devices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.