The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Joint Optimization of Caching, Computing, and Radio Resources for Fog-Enabled IoT Using Natural Actor–Critic Deep Reinforcement Learning
Abstract
The cloud-based Internet of Things (IoT) develops rapidly but suffer from large latency and backhaul bandwidth requirement, the technology of fog computing and caching has emerged as a promising paradigm for IoT to provide proximity services, and thus reduce service latency and save backhaul bandwidth. However, the performance of the fog-enabled IoT depends on the intelligent and efficient management of various network resources, and consequently the synergy of caching, computing, and communications becomes the big challenge. This paper simultaneously tackles the issues of content caching strategy, computation offloading policy, and radio resource allocation, and propose a joint optimization solution for the fog-enabled IoT. Since wireless signals and service requests have stochastic properties, we use the actor-critic reinforcement learning framework to solve the joint decision-making problem with the objective of minimizing the average end-to-end delay. The deep neural network (DNN) is employed as the function approximator to estimate the value functions in the critic part due to the extremely large state and action space in our problem. The actor part uses another DNN to represent a parameterized stochastic policy and improves the policy with the help of the critic. Furthermore, the Natural policy gradient method is used to avoid converging to the local maximum. Using the numerical simulations, we demonstrate the learning capacity of the proposed algorithm and analyze the end-to-end service latency.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
Dueling Network Architectures for Deep Reinforcement Learning
Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06581 · Cited by 3,300
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120
Natural Gradient Works Efficiently in Learning
Amari, S.-I. (1998) · Neural Computation · doi:10.1162/089976698300017746 · Cited by 2,681
Fog and IoT: An Overview of Research Opportunities
Chiang, M., & Zhang, T. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2584538 · Cited by 2,362
Dynamic computation offloading for mobile-edge computing with energy harvesting devices
Mao, Y., Zhang, J., & Letaief, K. B. (2016) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2016.2611964 · Cited by 1,705
Joint Optimization of Radio and Computational Resources for Multicell Mobile-Edge Computing
Sardellitti, S., Scutari, G., & Barbarossa, S. (2015) · IEEE Transactions on Signal and Information Processing over Networks · doi:10.1109/tsipn.2015.2448520 · Cited by 891
Decentralized Computation Offloading Game for Mobile Cloud Computing
Chen, X. (2015) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2014.2316834 · Cited by 872
A Survey of Computation Offloading for Mobile Systems
Kumar, K., Liu, J., Lu, Y.-H., & Bhargava, B. (2013) · Mobile Networks and Applications · doi:10.1007/s11036-012-0368-0 · Cited by 840
Peters, J., & Schaal, S. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.11.026 · Cited by 738
Wang, C., Liang, C., Yu, F. R., Chen, Q., & Tang, L. (2017) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2017.2703901 · Cited by 710
Self-citationMao, Y., Zhang, J., Song, S. H., & Letaief, K. B. (2017) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2017.2717986 · Cited by 690
Natural actor–critic algorithms
Bhatnagar, S., Sutton, R. S., Ghavamzadeh, M., & Lee, M. (2009) · Automatica · doi:10.1016/j.automatica.2009.07.008 · Cited by 559
Communicating While Computing: Distributed mobile cloud computing over 5G heterogeneous networks
Barbarossa, S., Sardellitti, S., & Di Lorenzo, P. (2014) · IEEE Signal Processing Magazine · doi:10.1109/msp.2014.2334709 · Cited by 468
Baktir, A. C., Ozgovde, A., & Ersoy, C. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2717482 · Cited by 425
Wireless caching: technical misconceptions and business barriers
Paschos, G., Bastug, E., Land, I., Caire, G., & Debbah, M. (2016) · IEEE Communications Magazine · doi:10.1109/mcom.2016.7537172 · Cited by 321
Computation Offloading for Service Workflow in Mobile Cloud Computing
Deng, S., Huang, L., Taheri, J., & Zomaya, A. Y. (2015) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2014.2381640 · Cited by 276
A survey on predicting the popularity of web content
Tatar, A., de Amorim, M. D., Fdida, S., & Antoniadis, P. (2014) · Journal of Internet Services and Applications · doi:10.1186/s13174-014-0008-y · Cited by 240
Joint allocation of computation and communication resources in multiuser mobile cloud computing
Barbarossa, S., Sardellitti, S., & Di Lorenzo, P. (2013) · 2013 IEEE 14th Workshop on Signal Processing Advances in Wireless Communications (SPAWC) · doi:10.1109/spawc.2013.6612005 · Cited by 189
Pricing and Resource Allocation via Game Theory for a Small-Cell Video Caching System
Li, J., Chen, H., Chen, Y., Lin, Z., Vucetic, B. S., & Hanzo, L. (2016) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2016.2577278 · Cited by 167
Deep reinforcement learning approaches for process control
Spielberg, S. P. K., Gopaluni, R. B., & Loewen, P. D. (2017) · International Symposium on Advanced Control of Industrial Processes · doi:10.1109/adconip.2017.7983780 · Cited by 161
Efficient Model Learning Methods for Actor–Critic Control
Grondman, I., Vaandrager, M., Buşoniu, L., Babuška, R., & Schuitema, E. (2011) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2011.2170565 · Cited by 126
Learning Based Frequency- and Time-Domain Inter-Cell Interference Coordination in HetNets
Simsek, M., Bennis, M., & Güvenç, İ. (2014) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2014.2374237 · Cited by 109
On the Complexity of Optimal Request Routing and Content Caching in Heterogeneous Cache Networks
Dehghan, M., Jiang, B., Seetharam, A., He, T., Salonidis, T., Kurose, J., Towsley, D., & Sitaraman, R. (2017) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2016.2636843 · Cited by 107 · cited 1×
“tent caching and request routing scheme is proposed in [19] to minimize the content-access latency, and the authors show that the joint optimization problem is NP-complete in both the”
BackgroundDistributed cache replacement for caching-enable base stations in cellular networks
Gu, J., Wang, W., Huang, A., Shan, H., & Zhang, Z. (2014) · International Conference on Communications · doi:10.1109/icc.2014.6883723 · Cited by 78
Exploring synergy between communications, caching, and computing in 5G-grade deployments
Andreev, S. D., Galinina, O., Pyattaev, A., Hošek, J., Mašek, P., Yanıkömeroğlu, H., & Koucheryavy, Y. A. (2016) · IEEE Communications Magazine · doi:10.1109/mcom.2016.7537178 · Cited by 75
Oo, T. Z., Tran, N. H. K., Saad, W., Niyato, D. T., Han, Z., & Hong, C. S. (2016) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2016.2613864 · Cited by 75
Self-citationFundamental Storage-Latency Tradeoff in Cache-Aided MIMO Interference Networks
Cao, Y., Tao, M., Xu, F., & Liu, K. (2017) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2017.2705102 · Cited by 67
Caching in Base Station with Recommendation via Q-Learning
Guo, K., Yang, C., & Liu, T. (2017) · 2017 IEEE Wireless Communications and Networking Conference (WCNC) · doi:10.1109/wcnc.2017.7925848 · Cited by 54
Cache-enabled heterogeneous cellular networks: Comparison and tradeoffs
Liu, D., & Yang, C. (2016) · International Conference on Communications · doi:10.1109/icc.2016.7510749 · Cited by 45
Network Function Virtualization
Durand, J. (2016) · Elsevier eBooks · doi:10.1016/c2014-0-02180-6 · Cited by 45
Osseiran, A., Elloumi, O., Song, J., & Monserrat, J. F. (2017) · IEEE Communications Standards Magazine · doi:10.1109/mcomstd.2017.7992936 · Cited by 39
Optimizing mobile computational offloading with delay constraints
Kao, Y.-H., & Krishnamachari, B. (2014) · Globecom · doi:10.1109/glocom.2014.7037149 · Cited by 37
Network Function Virtualization
Joshi, K., & Benson, T. (2016) · IEEE Internet Computing · doi:10.1109/mic.2016.112 · Cited by 37
Joint caching and base station activation for green heterogeneous cellular networks
Poularakis, K., Iosifidis, G., & Tassiulas, L. (2015) · International Conference on Communications · doi:10.1109/icc.2015.7248844 · Cited by 35
Cited by
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
“[207] DRL AC Schedule users and content caching to minimize the average transmission delay”
BackgroundInfluentialArtificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial
Chen, M., Challita, U., Saad, W., Yin, C., & Debbah, M. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2926625 · Cited by 1,117
Pham, Q.-V., Fang, F., Ha, V. N., Piran, M. J., Le, M., Le, L. B., Hwang, W.-J., & Ding, Z. (2020) · IEEE Access · doi:10.1109/access.2020.3001277 · Cited by 870
Edge Computing with Artificial Intelligence: A Machine Learning Perspective
Hua, H., Li, Y., Wang, T., Dong, N., Li, W., & Cao, J. (2023) · ACM Computing Surveys · doi:10.1145/3555802 · Cited by 563
Peng, H., & Shen, X. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036962 · Cited by 521
Metadata
- Authors
- Wei, Y., Yu, F. R., Song, M., & Han, Z.
- Venue
- IEEE Internet of Things Journal
- Published
- 2019-04
- Volume
- 6
- Issue
- 2
- Pages
- 2061-2073
- ISSN
- 2327-4662 (ISSN-L)
- 2372-2541
- OpenAlex
- W2898652425
- Cited by
- 321
- References
- 39
- FWCI
- 23.96
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Funders
- National Natural Science Foundation of China (61571059, 61871058)
- U.S. MURI AFOSR (18RT0073)
- National Science Foundation (CNS-1717454, CNS-1731424, CNS-1702850, CNS-1646607, ECCS-1547201)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.