The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
User Scheduling and Resource Allocation in HetNets With Hybrid Energy Supply: An Actor-Critic Reinforcement Learning Approach
Abstract
Densely deployment of various small-cell base stations in cellular networks to increase capacity will lead to heterogeneous networks (HetNets), and meanwhile, embedding the energy harvesting capabilities in base stations as an alternative energy supply is becoming a reality. How to make efficient utilization of radio resource and renewable energy is a brand-new challenge. This paper investigates the optimal policy for user scheduling and resource allocation in HetNets powered by hybrid energy with the purpose of maximizing energy efficiency of the overall network. Since wireless channel conditions and renewable energy arrival rates have stochastic properties and the environment's dynamics are unknown, the model-free reinforcement learning approach is used to learn the optimal policy through interactions with the environment. To solve our problem with continuous-valued state and action variables, a policy-gradient-based actor-critic algorithm is proposed. The actor part uses the Gaussian distribution as the parameterized policy to generate continuous stochastic actions, and the policy parameters are updated with the gradient ascent method. The critic part uses compatible function approximation to estimate the performance of the policy and helps the actor learn the gradient of the policy. The advantage function is used to further reduce the variance of the policy gradient. Using the numerical simulations, we demonstrate the convergence property of the proposed algorithm and analyze network energy efficiency.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
(1999) · Cited by 5,147
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,233
Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem
Vamvoudakis, K. G., & Lewis, F. L. (2010) · Automatica · doi:10.1016/j.automatica.2010.02.018 · Cited by 1,703
A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients
Grondman, I., Busoniu, L., Lopes, G. A. D., & Babuska, R. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2012.2218595 · Cited by 1,042
Optimal Packet Scheduling in an Energy Harvesting Communication System
Yang, J., & Ulukus, S. (2012) · IEEE Transactions on Communications · doi:10.1109/tcomm.2011.112811.100349 · Cited by 829
Peters, J., & Schaal, S. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.11.026 · Cited by 738
Cited by
Zhao, N., Liang, Y.-C., Niyato, D., Pei, Y., Wu, M., & Jiang, Y. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2933417 · Cited by 461
Morocho-Cayamcela, M. E., Lee, H., & Lim, W. (2019) · IEEE Access · doi:10.1109/access.2019.2942390 · Cited by 418
She, C., Sun, C., Gu, Z., Li, Y., Yang, C., Poor, H. V., & Vucetic, B. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3053601 · Cited by 405
Deep Deterministic Policy Gradient (DDPG)-Based Energy Harvesting Wireless Communications
Qiu, C., Hu, Y., Chen, Y., & Zeng, B. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2921159 · Cited by 383
AI-Assisted Network-Slicing Based Next-Generation Wireless Networks
Shen, X., Gao, J., Wu, W., Lyu, K., Li, M., Zhuang, W., Li, X., & Rao, J. (2020) · IEEE Open Journal of Vehicular Technology · doi:10.1109/ojvt.2020.2965100 · Cited by 342
Sensing, Computing, and Communications for Energy Harvesting IoTs: A Survey
Ma, D., Lan, G., Hassan, M., Hu, W., & Das, S. K. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2962526 · Cited by 336
Feng, J., Richard Yu, F., Pei, Q., Chu, X., Du, J., & Zhu, L. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2961707 · Cited by 332
AI Models for Green Communications Towards 6G
Mao, B., Tang, F., Kawamoto, Y., & Kato, N. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3130901 · Cited by 300
Power Allocation in Multi-User Cellular Networks: Deep Reinforcement Learning Approaches
Meng, F., Chen, P., Wu, L., & Cheng, J. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3001736 · Cited by 288
Qiu, X., Liu, L., Chen, W., Hong, Z., & Zheng, Z. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2924015 · Cited by 287
Qi, Q., Wang, J., Ma, Z., Sun, H., Cao, Y., Zhang, L., & Liao, J. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2894437 · Cited by 269
Agarwal, B., Togou, M. A., Ruffini, M., & Muntean, G.-M. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3207967 · Cited by 123
“In [126], the authors examined a UA and RA scheme for HetNets with hybrid energy supply to exploit the harvested energy across small-cells.”
BackgroundA Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access
Zhong, C., Lu, Z., Gursoy, M. C., & Velipasalar, S. (2019) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2019.2952909 · Cited by 123
“For example, the DQN was applied to seek optimal policies to jointly allocate the sub-bands and power in vehicle-to-vehicle communication [24], and an actor-critic reinforcement learning framework was proposed to jointly solve the user scheduling, and subchannel and power allocation problem in order to maximize the en…”
MethodSharma, N., & Kumar, K. (2021) · Physical Communication · doi:10.1016/j.phycom.2021.101415 · Cited by 75
Hu, Q., Liu, Y., Cai, Y., Yu, G., & Ding, Z. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3087233 · Cited by 64
“Recently, the machine learning techniques emerge as an efficient method and there are a number of studies employing the neural networks (NNs) for solving the investigated problems [16], such as precoding design [17], beam selection [18]–[20], and resource allocation [21].”
BackgroundVehicular Edge Computing via Deep Reinforcement Learning
Qi, Q., & Ma, Z. (2019) · arXiv · doi:10.48550/arxiv.1901.04290 · Cited by 17
Metadata
- Authors
- Wei, Y., Yu, F. R., Song, M., & Han, Z.
- Venue
- IEEE Transactions on Wireless Communications
- Published
- 2018-01
- Volume
- 17
- Issue
- 1
- Pages
- 680-692
- ISSN
- 1536-1276 (ISSN-L)
- OpenAlex
- W2767359317
- Cited by
- 301
- References
- 32
- FWCI
- 13.43
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.