The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
User Scheduling and Resource Allocation in HetNets With Hybrid Energy Supply: An Actor-Critic Reinforcement Learning Approach
Abstract
Densely deployment of various small-cell base stations in cellular networks to increase capacity will lead to heterogeneous networks (HetNets), and meanwhile, embedding the energy harvesting capabilities in base stations as an alternative energy supply is becoming a reality. How to make efficient utilization of radio resource and renewable energy is a brand-new challenge. This paper investigates the optimal policy for user scheduling and resource allocation in HetNets powered by hybrid energy with the purpose of maximizing energy efficiency of the overall network. Since wireless channel conditions and renewable energy arrival rates have stochastic properties and the environment's dynamics are unknown, the model-free reinforcement learning approach is used to learn the optimal policy through interactions with the environment. To solve our problem with continuous-valued state and action variables, a policy-gradient-based actor-critic algorithm is proposed. The actor part uses the Gaussian distribution as the parameterized policy to generate continuous stochastic actions, and the policy parameters are updated with the gradient ascent method. The critic part uses compatible function approximation to estimate the performance of the policy and helps the actor learn the gradient of the policy. The advantage function is used to further reduce the variance of the policy gradient. Using the numerical simulations, we demonstrate the convergence property of the proposed algorithm and analyze network energy efficiency.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
(1999) · Cited by 5,147
Neuronlike adaptive elements that can solve difficult learning control problems
Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313077 · Cited by 3,233
Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem
Vamvoudakis, K. G., & Lewis, F. L. (2010) · Automatica · doi:10.1016/j.automatica.2010.02.018 · Cited by 1,703
A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients
Grondman, I., Busoniu, L., Lopes, G. A. D., & Babuska, R. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2012.2218595 · Cited by 1,042
Optimal Packet Scheduling in an Energy Harvesting Communication System
Yang, J., & Ulukus, S. (2012) · IEEE Transactions on Communications · doi:10.1109/tcomm.2011.112811.100349 · Cited by 829
Peters, J., & Schaal, S. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.11.026 · Cited by 738
Konda, V. R., & Tsitsiklis, J. N. (2003) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012901385691 · Cited by 709
Natural actor–critic algorithms
Bhatnagar, S., Sutton, R. S., Ghavamzadeh, M., & Lee, M. (2009) · Automatica · doi:10.1016/j.automatica.2009.07.008 · Cited by 559
Cloud assisted HetNets toward 5G wireless networks
Zhang, N., Cheng, N., Gamage, A. T., Zhang, K., Mark, J. W., & Shen, X. (2015) · IEEE Communications Magazine · doi:10.1109/mcom.2015.7120046 · Cited by 315
Energy-Efficient Resource Allocation for Heterogeneous Cognitive Radio Networks with Femtocells
Xie, R., Yu, F. R., Ji, H., et al. (2012) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2012.092112.111510 · Cited by 267
Self-citationChen, X., Wu, J., Cai, Y., Zhang, H., & Chen, T. (2015) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2015.2393496 · Cited by 250
Bu, S., Yu, F. R., Chua, M. Y.-K., & Liu, X. P. (2012) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2012.052512.111766 · Cited by 224
Self-citationFundamentals of Heterogeneous Cellular Networks with Energy Harvesting
(2014) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2014.040214.131201 · Cited by 201
Optimal Control of Wake Up Mechanisms of Femtocells in Heterogeneous Networks
Saker, L., Elayoubi, S. E., Combes, R., & Chahed, T. (2012) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2012.120415 · Cited by 182
A Survey on Green Mobile Networking: From The Perspectives of Network Operators and Mobile Users
Ismail, M., Zhuang, W., Serpedin, E., & Qaraqe, K. A. (2014) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2014.2367592 · Cited by 152
Energy-Aware Traffic Offloading for Green Heterogeneous Networks
Zhang, S., Zhang, N., Zhou, S., et al. (2016) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2016.2520244 · Cited by 151
Efficient Model Learning Methods for Actor–Critic Control
Grondman, I., Vaandrager, M., Buşoniu, L., Babuška, R., & Schuitema, E. (2011) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2011.2170565 · Cited by 126
Decentralized Quasi-Newton Methods
Eisen, M., Mokhtari, A., & Ribeiro, A. (2017) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2017.2666776 · Cited by 120
Learning Based Frequency- and Time-Domain Inter-Cell Interference Coordination in HetNets
Simsek, M., Bennis, M., & Güvenç, İ. (2014) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2014.2374237 · Cited by 109
Hoang, D. T., Niyato, D. T., Wang, P., & Kim, D. I. (2015) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2015.2408610 · Cited by 76
Power Allocation for an Energy Harvesting Transmitter with Hybrid Energy Sources
Ahmed, I., Ikhlef, A., Ng, D. W. K., & Schober, R. (2013) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2013.111013.130215 · Cited by 74
A sensitivity formula for risk-sensitive cost and the actor–critic algorithm
Borkar, V. S. (2001) · Systems & Control Letters · doi:10.1016/s0167-6911(01)00152-9 · Cited by 46
Energy Allocation and Cooperation for Energy-Efficient Wireless Two-Tier Networks
Ramamonjison, R., & Bhargava, V. K. (2016) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2016.2585115 · Cited by 37
Context-aware mobility management in HetNets: A reinforcement learning approach
Simsek, M., Bennis, M., & Güvenç, İ. (2015) · IEEE Wireless Communications and Networking Conference · doi:10.1109/wcnc.2015.7127696 · Cited by 34
Sustainable communication and networking in two-tier green cellular networks
Zheng, Z., Zhang, X., Cai, L., Zhang, R., & Shen, X. (2014) · IEEE Wireless Communications · doi:10.1109/mwc.2014.6882295 · Cited by 30
Power Allocation in HetNets with Hybrid Energy Supply Using Actor-Critic Reinforcement Learning
Wei, Y., Zhang, Z., Yu, F. R., & Han, Z. (2017) · Globecom · doi:10.1109/glocom.2017.8254214 · Cited by 22
Self-citationContinuous-time on-policy neural Reinforcement Learning of working memory tasks
Zambrano, D., Roelfsema, P. R., & Bohté, S. M. (2015) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2015.7280636 · Cited by 9
Incremental Receptive Field Weighted Actor-Critic
Lee, D., & Lee, J.-J. (2012) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2012.2209660 · Cited by 8
Cited by
Zhao, N., Liang, Y.-C., Niyato, D., Pei, Y., Wu, M., & Jiang, Y. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2933417 · Cited by 461
Morocho-Cayamcela, M. E., Lee, H., & Lim, W. (2019) · IEEE Access · doi:10.1109/access.2019.2942390 · Cited by 418
She, C., Sun, C., Gu, Z., Li, Y., Yang, C., Poor, H. V., & Vucetic, B. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3053601 · Cited by 405
Deep Deterministic Policy Gradient (DDPG)-Based Energy Harvesting Wireless Communications
Qiu, C., Hu, Y., Chen, Y., & Zeng, B. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2921159 · Cited by 383
AI-Assisted Network-Slicing Based Next-Generation Wireless Networks
Shen, X., Gao, J., Wu, W., Lyu, K., Li, M., Zhuang, W., Li, X., & Rao, J. (2020) · IEEE Open Journal of Vehicular Technology · doi:10.1109/ojvt.2020.2965100 · Cited by 342
Metadata
- Authors
- Wei, Y., Yu, F. R., Song, M., & Han, Z.
- Venue
- IEEE Transactions on Wireless Communications
- Published
- 2018-01
- Volume
- 17
- Issue
- 1
- Pages
- 680-692
- ISSN
- 1536-1276 (ISSN-L)
- OpenAlex
- W2767359317
- Cited by
- 301
- References
- 32
- FWCI
- 13.43
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.