The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Dependent Task Offloading for Edge Computing based on Deep Reinforcement Learning
Open accessAbstract
Edge computing is an emerging promising computing paradigm that brings computation and storage resources to the network edge, hence significantly reducing the service latency and network traffic. In edge computing, many applications are composed of dependent tasks where the outputs of some are the inputs of others. How to offload these tasks to the network edge is a vital and challenging problem which aims to determine the placement of each running task in order to maximize the Quality-of-Service (QoS). Most of the existing studies either design heuristic algorithms that lack strong adaptivity or learning-based methods but without considering the intrinsic task dependency. Different from the existing work, we propose an intelligent task offloading scheme leveraging off-policy reinforcement learning empowered by a Sequence-to-Sequence (S2S) neural network, where the dependent tasks are represented by a Directed Acyclic Graph (DAG). To improve the training efficiency, we combine a specific off-policy policy gradient algorithm with a clipped surrogate objective. We then conduct extensive simulation experiments using heterogeneous applications modelled by synthetic DAGs. The results demonstrate that: 1) our method converges fast and steadily in training; 2) it outperforms the existing methods and approximates the optimal solution in latency and energy consumption under various scenarios.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625 · cited 7×
“To improve the performance and training efficiency, the training target function is modified based on PPO [23], which generates trajectories using the old policy πθold and updates the current policy πθ whose initial value equals πθold for several epochs.”
BackgroundMethodInfluentialAsynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2018) · Science · doi:10.1126/science.aar6404 · Cited by 3,562 · cited 1×
“Recent breakthroughs in DRL have led to many successful applications in a wide range of areas including gaming [7], robotics [8], networking [9], etc.”
BackgroundPerformance-effective and low-complexity task scheduling for heterogeneous computing
Topcuoglu, H., Hariri, S., & Wu, M.-Y. (2002) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/71.993206 · Cited by 3,475
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455
Mobile Edge Computing: A Survey on Architecture and Computation Offloading
Mach, P., & Becvar, Z. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2682318 · Cited by 3,258
High-Dimensional Continuous Control Using Generalized Advantage Estimation
Schulman, J., Moritz, P., Levine, S., Jordan, M., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1506.02438 · Cited by 2,103
Static scheduling algorithms for allocating directed task graphs to multiprocessors
Kwok, Y.-K., & Ahmad, I. (1999) · ACM Computing Surveys · doi:10.1145/344588.344618 · Cited by 1,314
Task Offloading for Mobile Edge Computing in Software Defined Ultra-Dense Network
Chen, M., & Hao, Y. (2018) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2018.2815360 · Cited by 1,068
Learning Combinatorial Optimization Algorithms over Graphs
Dai, H., Khalil, E. B., Zhang, Y., Dilkina, B., & Song, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.01665 · Cited by 1,007
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Reinforcement Learning · doi:10.1007/978-1-4615-3618-5_2 · Cited by 903
Offloading in Mobile Edge Computing: Task Allocation and Computational Frequency Scaling
Thinh, T. Q., Tang, J., La, Q. D., & Quek, T. Q. S. (2017) · IEEE Transactions on Communications · doi:10.1109/tcomm.2017.2699660 · Cited by 866
Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., & Bennis, M. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2876279 · Cited by 680
List Scheduling Algorithm for Heterogeneous Systems by an Optimistic Cost Table
Arabnejad, H., & Barbosa, J. G. (2014) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2013.57 · Cited by 653
Deep Reinforcement Learning for Task Offloading in Mobile Edge Computing Systems
Tang, M., & Wong, V. W. S. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3036871 · Cited by 584
Deep reinforcement learning based computation offloading and resource allocation for MEC
Li, J., Gao, H., Lv, T., & Lu, Y. (2018) · 2018 IEEE Wireless Communications and Networking Conference (WCNC) · doi:10.1109/wcnc.2018.8377343 · Cited by 548
Zeng, D., Gu, L., Guo, S., Cheng, Z., & Yu, S. (2016) · IEEE Transactions on Computers · doi:10.1109/tc.2016.2536019 · Cited by 426
Odessa: enabling interactive perception applications on mobile devices
Ra, M.-R., Sheth, A., Mummert, L., Pillai, P., Wetherall, D., & Govindan, R. (2011) · Proceedings of the 9th international conference on Mobile systems, applications, and services · doi:10.1145/1999995.2000000 · Cited by 361
Lin, X., Wang, Y., Xie, Q., & Pedram, M. (2015) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2014.2381227 · Cited by 257
Wang, J., Hu, J., Min, G., Zhan, W., Ni, Q., & Georgalas, N. (2019) · IEEE Communications Magazine · doi:10.1109/mcom.2019.1800971 · Cited by 254
Self-citationCao, H., & Cai, J. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2740724 · Cited by 223
A Deep Reinforcement Learning Based Offloading Game in Edge Computing
Zhan, Y., Guo, S., Li, P., & Zhang, J. (2020) · IEEE Transactions on Computers · doi:10.1109/tc.2020.2969148 · Cited by 218
Learning for Computation Offloading in Mobile Edge Computing
Dinh, T. Q., La, Q. D., Quek, T. Q. S., & Shin, H. (2018) · IEEE Transactions on Communications · doi:10.1109/tcomm.2018.2866572 · Cited by 214
Mobility-Aware Multi-User Offloading Optimization for Mobile Edge Computing
Zhan, W., Luo, C., Min, G., Wang, C., Zhu, Q., & Duan, H. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2966500 · Cited by 203
Self-citationULOOF: A User Level Online Offloading Framework for Mobile Edge Computing
Neto, J. L. D., Yu, S.-Y., Macedo, D. F., Nogueira, J. M. S., Langar, R., & Secci, S. (2018) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2018.2815015 · Cited by 162
Zou, J., Hao, T., Chen, Y., & Jin, H. (2020) · IEEE Transactions on Computers · doi:10.1109/tc.2020.2987567 · Cited by 108
Algorithmics of Cost-Driven Computation Offloading in the Edge-Cloud Environment
Du, M., Wang, Y., Ye, K., & Xu, C. (2020) · IEEE Transactions on Computers · doi:10.1109/tc.2020.2976996 · Cited by 82
Avatar: Mobile Distributed Computing in the Cloud
Borcea, C., Ding, X., Gehani, N., Curtmola, R., Khan, M. A. R., & Debnath, H. (2015) · IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (MobileCloud) · doi:10.1109/mobilecloud.2015.22 · Cited by 41
Adaptive Avatar Handoff in the Cloudlet Network
Sun, X., & Ansari, N. (2017) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2017.2701794 · Cited by 40
Reward-Oriented Task Offloading Under Limited Edge Server Power for Multiaccess Edge Computing
Song, M., Lee, Y., & Kim, K. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3065429 · Cited by 38
Trajectory-Based Off-Policy Deep Reinforcement LearningPreprint
Doerr, A., Volpp, M., Toussaint, M., Trimpe, S., & Daniel, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.05710 · Cited by 3
Metadata
- Authors
- Wang, J., Hu, J., Min, G., Zhan, W., Zomaya, A. Y., & Georgalas, N.
- Venue
- IEEE Transactions on Computers
- Published
- 2022-10-01
- Volume
- 71
- Issue
- 10
- Pages
- 2449-2461
- ISSN
- 0018-9340 (ISSN-L)
- 1557-9956
- 2326-3814
- OpenAlex
- W3217119770
- Cited by
- 274
- References
- 35
- FWCI
- 16.41
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.