Knowledge-Driven Service Offloading Decision for Vehicular Edge Computing: A Deep Reinforcement Learning Approach
The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
The smart vehicles construct Internet of Vehicle (IoV), which can execute various intelligent services. Although the computation capability of a vehicle is limited, multi-type of edge computing nodes provide heterogeneous resources for intelligent vehicular services. When offloading the complex service to the vehicular edge computing node, the decision for its destination should be considered according to numerous factors. This paper mostly formulate the offloading decision as a resource scheduling problem with single or multiple objective function and constraints, where some customized heuristics algorithms are explored. However, offloading multiple data dependence tasks in a complex service is a difficult decision, as an optimal solution must understand the resource requirement, the access network, the user mobility, and importantly the data dependence. Inspired by recent advances in machine learning, we propose a knowledge driven (KD) service offloading decision framework for IoV, which provides the optimal policy directly from the environment. We formulate the offloading decision for the multiple tasks as a long-term planning problem, and explore the recent deep reinforcement learning to obtain the optimal solution. It can scruple the future data dependence of the following tasks when making decision for a current task from the learned offloading knowledge. Moreover, the framework supports the pre-training at the powerful edge computing node and continually online learning when the vehicular service is executed, so that it can adapt the environment changes and can learn policy that are sensible in foresight. The simulation results show that KD service offloading decision converges quickly, adapts to different conditions, and outperforms a greedy offloading decision algorithm.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · cited 3×
“The critic network updates the parameters θ v of the value function V ( s t ; θ v ) and make the value function to approximate the real reward [20].”
BackgroundMethodInfluentialEfficient multi-user computation offloading for mobile-edge cloud computing
Chen, X., Jiao, L., Li, W., & Fu, X. (2016) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2015.2487344 · Cited by 2,675
Cheng, G., Zhou, P., & Han, J. (2016) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2601622 · Cited by 1,771
Dynamic computation offloading for mobile-edge computing with energy harvesting devices
Mao, Y., Zhang, J., & Letaief, K. B. (2016) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2016.2611964 · Cited by 1,705
Neural Adaptive Video Streaming with Pensieve
Mao, H., Netravali, R., & Alizadeh, M. (2017) · Proceedings of the Conference of the ACM Special Interest Group on Data Communication · doi:10.1145/3098822.3098843 · Cited by 1,357
Resource Management with Deep Reinforcement Learning
Mao, H., Alizadeh, M., Menache, I., & Kandula, S. (2016) · Proceedings of the 15th ACM Workshop on Hot Topics in Networks · doi:10.1145/3005745.3005750 · Cited by 1,353 · cited 1×
“Mao et al. [22] find the optimal resource management policy by DRL.”
BackgroundMobile-Edge Computing: Partial Computation Offloading Using Dynamic Voltage Scaling
Wang, Y., Sheng, M., Wang, X., Wang, L., & Li, J. (2016) · IEEE Transactions on Communications · doi:10.1109/tcomm.2016.2599530 · Cited by 1,002
Vehicular Fog Computing: A Viewpoint of Vehicles as the Infrastructures
Hou, X., Li, Y., Chen, M., Wu, D., Jin, D., & Chen, S. (2016) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2016.2532863 · Cited by 930
Han, J., Zhang, D., Cheng, G., Guo, L., & Ren, J. (2015) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2014.2374218 · Cited by 733
Zhang, K., Mao, Y., Leng, S., He, Y., & ZHANG, Y. (2017) · IEEE Vehicular Technology Magazine · doi:10.1109/mvt.2017.2668838 · Cited by 701
TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
Wen, W., Xu, C., Yan, F., Wu, C., Wang, Y., Chen, Y., & Li, H. H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.07878 · Cited by 622
Background Prior-Based Salient Object Detection via Deep Reconstruction Residual
Han, J., Zhang, D., Hu, X., Guo, L., Ren, J., & Wu, F. (2015) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2014.2381471 · Cited by 444
Experience-driven Networking: A Deep Reinforcement Learning based Approach
Xu, Z., Tang, J., Meng, J., Zhang, W., Wang, Y., Liu, C. H., & Yang, D. (2018) · IEEE INFOCOM 2018 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2018.8485853 · Cited by 439
Kato, N., Fadlullah, Z. M., Mao, B., Tang, F., Akashi, O., Inoue, T., & Mizutani, K. (2017) · IEEE Wireless Communications · doi:10.1109/mwc.2016.1600317wc · Cited by 400
Mao, B., Fadlullah, Z. M., Tang, F., Kato, N., Akashi, O., Inoue, T., & Mizutani, K. (2017) · IEEE Transactions on Computers · doi:10.1109/tc.2017.2709742 · Cited by 350
Reducing location update cost in a PCS network
Lin, Y.-B. (1997) · IEEE/ACM Transactions on Networking · doi:10.1109/90.554719 · Cited by 330
Detection of Co-salient Objects by Looking Deep and Wide
Zhang, D., Han, J., Li, C., Wang, J., & Li, X. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-016-0907-4 · Cited by 328
Self-citationWei, Y., Yu, F. R., Song, M., & Han, Z. (2018) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2017.2769644 · Cited by 301
Liu, N., Li, Z., Xu, J., Xu, Z., Lin, S., Qiu, Q., Tang, J., & Wang, Y. (2017) · 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2017.123 · Cited by 299
He, Y., Zhang, Z., Yu, F. R., Zhao, N., Yin, H., Leung, V. C. M., & Zhang, Y. (2017) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2751641 · Cited by 288
Optimal Schedule of Mobile Edge Computing for Internet of Things Using Partial Information
Lyu, X., Ni, W., Tian, H., Liu, R. P., Wang, X., Giannakis, G. B., & Paulraj, A. (2017) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2017.2760186 · Cited by 251
Luong, N. C., Xiong, Z., Wang, P., & Niyato, D. (2018) · 2018 IEEE International Conference on Communications (ICC) · doi:10.1109/icc.2018.8422743 · Cited by 229
Cao, H., & Cai, J. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2740724 · Cited by 223
When Edge Meets Learning: Adaptive Control for Resource-Constrained Distributed Machine Learning
Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K. (2018) · IEEE INFOCOM 2018 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2018.8486403 · Cited by 209
Zhou, Y., Yu, F. R., Chen, J., & Kuo, Y. (2017) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2737028 · Cited by 165
The Role of Data Analysis in the Development of Intelligent Energy Networks
Ma, Z., Xie, J., Li, H., Sun, Q., Si, Z., Zhang, J., & Guo, J. H. (2017) · IEEE Network · doi:10.1109/mnet.2017.1600319 · Cited by 145
Self-citationStochastic Analysis of a Single-Hop Communication Link in Vehicular Ad Hoc Networks
Abboud, K., & Zhuang, W. (2014) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2014.2314453 · Cited by 101
Dual-Side Optimization for Cost-Delay Tradeoff in Mobile Edge Computing
Kim, Y., Kwak, J., & Chong, S. (2017) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2762423 · Cited by 99
A deep learning approach for optimizing content delivering in cache-enabled HetNet
Lei, L., You, L., Dai, G., Vu, T. X., Yuan, D., & Chatzinotas, S. (2017) · International Symposium on Wireless Communication Systems · doi:10.1109/iswcs.2017.8108157 · Cited by 91
Prefetching-Based Data Dissemination in Vehicular Cloud Systems
Kim, R., Lim, H., & Krishnamachari, B. (2015) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2015.2388851 · Cited by 78
On the Serviceability of Mobile Vehicular Cloudlets in a Large-Scale Urban Environment
Wang, C., Li, Y., Jin, D., & Chen, S. (2016) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2016.2561293 · Cited by 50
An energy-efficient task scheduling for mobile devices based on cloud assistant
Liu, T., Chen, F., Ma, Y., & Xie, Y. (2016) · Future Generation Computer Systems · doi:10.1016/j.future.2016.02.004 · Cited by 48
A Tag-Based Integrated Diffusion Model for Personalized Location Recommendation
Zheng, Y., Wang, Y., Zhang, L., Wang, J., & Qi, Q. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-70139-4_33 · Cited by 5
Self-citation
Cited by
Edge Computing with Artificial Intelligence: A Machine Learning Perspective
Hua, H., Li, Y., Wang, T., Dong, N., Li, W., & Cao, J. (2023) · ACM Computing Surveys · doi:10.1145/3555802 · Cited by 563
Arthurs, P., Gillam, L., Krause, P., Wang, N., Halder, K., & Mouzakitis, A. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3084396 · Cited by 339
Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges
Lei, L., Tan, Y., Zheng, K., Liu, S., Zhang, K., & Shen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2988367 · Cited by 298
Wu, H., Wolter, K., Jiao, P., Deng, Y., Zhao, Y., & Xu, M. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3033521 · Cited by 289
Qiu, X., Liu, L., Chen, W., Hong, Z., & Zheng, Z. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2924015 · Cited by 287
Metadata
- Authors
- Qi, Q., Wang, J., Ma, Z., Sun, H., Cao, Y., Zhang, L., & Liao, J.
- Venue
- IEEE Transactions on Vehicular Technology
- Published
- 2019-05
- Volume
- 68
- Issue
- 5
- Pages
- 4192-4203
- ISSN
- 0018-9545 (ISSN-L)
- 1939-9359
- OpenAlex
- W2918400102
- Cited by
- 269
- References
- 35
- FWCI
- 20.79
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.