The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Lyapunov-Guided Deep Reinforcement Learning for Stable Online Computation Offloading in Mobile-Edge Computing Networks
Abstract
Opportunistic computation offloading is an effective method to improve the computation performance of mobile-edge computing (MEC) networks under dynamic edge environment. In this paper, we consider a multi-user MEC network with time-varying wireless channels and stochastic user task data arrivals in sequential time frames. In particular, we aim to design an online computation offloading algorithm to maximize the network data processing capability subject to the long-term data queue stability and average power constraints. The online algorithm is practical in the sense that the decisions for each time frame are made without the assumption of knowing the future realizations of random channel conditions and data arrivals. We formulate the problem as a multi-stage stochastic mixed integer non-linear programming (MINLP) problem that jointly determines the binary offloading (each user computes the task either locally or at the edge server) and system resource allocation decisions in sequential time frames. To address the coupling in the decisions of different time frames, we propose a novel framework, named LyDROO, that combines the advantages of Lyapunov optimization and deep reinforcement learning (DRL). Specifically, LyDROO first applies Lyapunov optimization to decouple the multi-stage stochastic MINLP into deterministic per-frame MINLP subproblems. By doing so, it guarantees to satisfy all the long-term constraints by solving the per-frame subproblems that are much smaller in size. Then, LyDROO integrates model-based optimization and model-free DRL to solve the per-frame MINLP problems with very low computational complexity. Simulation results show that under various network setups, the proposed LyDROO achieves optimal computation performance while stabilizing all queues in the system. Besides, it induces very low computation time that is particularly suitable for real-time implementation in fast fading environments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,414
Combinatorial Optimization: Algorithms and Complexity.
Johnson, D. S., Papadimitriou, C. H., & Steiglitz, K. (1984) · The American Mathematical Monthly · doi:10.2307/2322374 · Cited by 5,942
A Survey on Mobile Edge Computing: The Communication Perspective
Mao, Y., You, C., Zhang, J., Huang, K., & Letaief, K. B. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2745201 · Cited by 5,607
Stochastic Network Optimization with Application to Communication and Queueing Systems
Neely, M. J. (2010) · Synthesis lectures on communication networks · doi:10.2200/s00271ed1v01y201006cnt007 · Cited by 2,218
Dynamic computation offloading for mobile-edge computing with energy harvesting devices
Mao, Y., Zhang, J., & Letaief, K. B. (2016) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2016.2611964 · Cited by 1,705
Resource Allocation and Cross-Layer Control in Wireless Networks
Georgiadis, L., Neely, M. J., & Tassiulas, L. (2005) · now publishers, Inc. eBooks · doi:10.1561/9781933019697 · Cited by 1,291
Machine Learning: An Algorithmic Perspective
Marsland, S. (2009) · Cited by 1,182
Huang, L., Bi, S., & Zhang, Y.-J. A. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2928811 · Cited by 1,040
Self-citationResource Allocation and Cross-Layer Control in Wireless Networks
Georgiadis, L., Neely, M. J., & Tassiulas, L. (2006) · Foundations and Trends® in Networking · doi:10.1561/1300000001 · Cited by 1,040
Bi, S., & Zhang, Y. J. (2018) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2018.2821664 · Cited by 870
Self-citationOffloading in Mobile Edge Computing: Task Allocation and Computational Frequency Scaling
Thinh, T. Q., Tang, J., La, Q. D., & Quek, T. Q. S. (2017) · IEEE Transactions on Communications · doi:10.1109/tcomm.2017.2699660 · Cited by 866
Energy-Optimal Mobile Cloud Computing under Stochastic Wireless Channel
Zhang, W., Wen, Y., Guan, K., Kilper, D., Luo, H., & Wu, D. O. (2013) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2013.072513.121842 · Cited by 863
Mao, Y., Zhang, J., Song, S. H., & Letaief, K. B. (2017) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2017.2717986 · Cited by 690
Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., & Bennis, M. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2876279 · Cited by 680
Deep Reinforcement Learning for Task Offloading in Mobile Edge Computing Systems
Tang, M., & Wong, V. W. S. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3036871 · Cited by 584
Liu, Y., Yu, H., Xie, S., & Zhang, Y. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2935450 · Cited by 570
Self-citationLearning-Based Computation Offloading for IoT Devices With Energy Harvesting
Min, M., Xiao, L., Chen, Y., Cheng, P., Wu, D., & Zhuang, W. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2890685 · Cited by 560
Deep reinforcement learning based computation offloading and resource allocation for MEC
Li, J., Gao, H., Lv, T., & Lu, Y. (2018) · 2018 IEEE Wireless Communications and Networking Conference (WCNC) · doi:10.1109/wcnc.2018.8377343 · Cited by 548
Energy Efficient Mobile Cloud Computing Powered by Wireless Energy Transfer
You, C., Huang, K., & Chae, H. (2016) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2016.2545382 · Cited by 495
Dynamic Task Offloading and Resource Allocation for Ultra-Reliable Low-Latency Edge Computing
Liu, C.-F., Bennis, M., Debbah, M., & Poor, H. V. (2019) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2898573 · Cited by 435
EMM: Energy-Aware Mobility Management for Mobile Edge Computing in Ultra Dense Networks
Sun, Y., Zhou, S., & Xu, J. (2017) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2017.2760160 · Cited by 420
Bi, S., Huang, L., & Zhang, Y.-J. A. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2988386 · Cited by 340
Self-citationWei, Y., Yu, F. R., Song, M., & Han, Z. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2878435 · Cited by 321
Du, J., Yu, F. R., Chu, X., Feng, J., & Lu, G. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2883156 · Cited by 302
Yan, J., Bi, S., Zhang, Y. J., & Tao, M. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2943563 · Cited by 280
Self-citationDu, J., Yu, F. R., Lu, G., Wang, J., Jiang, J., & Chu, X. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3003449 · Cited by 278
Yan, J., Bi, S., & Zhang, Y. J. A. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2993071 · Cited by 220
Self-citationReinforcement Learning-Based Mobile Offloading for Edge Computing Against Jamming and Interference
Xiao, L., Lu, X., Xu, T., Wan, X., Ji, W., & Zhang, Y. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2020.3007742 · Cited by 160
Self-citationZhang, J., Du, J., Shen, Y., & Wang, J. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3000527 · Cited by 130
Stable Online Computation Offloading via Lyapunov-guided Deep Reinforcement Learning
Bi, S., Huang, L. P., Wang, H., & Zhang, Y. A. (2021) · International Conference on Communications · doi:10.1109/icc42927.2021.9500520 · Cited by 23
Self-citation
Metadata
- Authors
- Bi, S., Huang, L., Wang, H., & Zhang, Y.-J. A.
- Venue
- IEEE Transactions on Wireless Communications
- Published
- 2021-11
- Volume
- 20
- Issue
- 11
- Pages
- 7519-7537
- ISSN
- 1536-1276 (ISSN-L)
- 1558-2248
- OpenAlex
- W3168768468
- Cited by
- 407
- References
- 32
- FWCI
- 25.74
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.