Intelligent Power Control for Spectrum Sharing in Cognitive Radios: A Deep Reinforcement Learning Approach
Open accessThe DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
We consider the problem of spectrum sharing in a cognitive radio system consisting of a primary user and a secondary user. The primary user and the secondary user work in a non-cooperative manner. Specifically, the primary user is assumed to update its transmitted power based on a pre-defined power control policy. The secondary user does not have any knowledge about the primary user's transmit power, or its power control strategy. The objective of this paper is to develop a learning-based power control method for the secondary user in order to share the common spectrum with the primary user. To assist the secondary user, a set of sensor nodes are spatially deployed to collect the received signal strength information at different locations in the wireless environment. We develop a deep reinforcement learning-based method, which the secondary user can use to intelligently adjust its transmit power such that after a few rounds of interaction with the primary user, both users can transmit their own data successfully with required qualities of service. Our experimental results show that the secondary user can interact with the primary user efficiently to reach a goal state (defined as a state in which both users can successfully transmit their data) from any initial states within a few number of steps.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Such dynamic sharing of spectrum aims at enhancing the spectrum utilization efficiency through the provision of access of under-utilized spectrum to the secondary network users [111].”
From Quantum Machine Learning for 6G Communication Networks: State-of-the-Art and Vision for the Future (2019) · cites this paper as [111]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“The Adam algorithm [28] is adopted for updating the weights θ, where the size of a minibatch is set to 256.”
MethodHuman-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Wireless Communications: Principles and Practice
Rappaport, T. S. (1996) · Cited by 17,370
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
Cognitive radio: brain-empowered wireless communications
Haykin, S. (2005) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2004.839380 · Cited by 11,916 · cited 1×
“In order to improve the spectrum efficiency, the notion of spectrum sharing with secondary users through cognitive radios is highly motivated [2].”
BackgroundPlaying Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471 · cited 1×
“Recently, deep reinforcement learning was introduced and proves its competence for challenging tasks, say Go and Atari games [32]–[34].”
BackgroundWatkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Wireless sensor network survey
Yick, J., Mukherjee, B., & Ghosal, D. (2008) · Computer Networks · doi:10.1016/j.comnet.2008.04.002 · Cited by 6,255
Deep learning for wireless physical layer: Opportunities and challenges
Wang, T., Wen, C.-K., Wang, H., Gao, F., Jiang, T., & Jin, S. (2017) · China Communications · doi:10.1109/cc.2017.8233654 · Cited by 606
Joint scheduling and power control for wireless ad hoc networks
ElBatt, T., & Ephremides, A. (2004) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2003.819032 · Cited by 569
A utility-based power-control scheme in wireless cellular systems
Xiao, M., Shroff, N. B., & Chong, E. K. P. (2003) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2003.810314 · Cited by 481 · cited 1×
“Based on [5], modified approaches with different constraints or scenarios were developed [6]–[10].”
BackgroundDeep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access
Naparstek, O., & Cohen, K. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2018.2879433 · Cited by 477
Dynamic Spectrum Access with QoS and Interference Temperature Constraints
Xing, Y., Mathur, C. N., Haleem, M. A., Chandramouli, R., & Subbalakshmi, K. P. (2007) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2007.50 · Cited by 461 · cited 1×
“Most existing works address this dynamic power control problem from an optimization perspective.”
BackgroundTomic, S., Beko, M., & Dinis, R. (2015) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2014.2334397 · Cited by 353 · cited 1×
“Note that collecting the RSS information from spatially distributed sensor nodes is a basic requirement for many applications, e.g. source localization [35].”
BackgroundGrandhi, S. A., Zander, J., & Yates, R. (1994) · Wireless Personal Communications · doi:10.1007/bf01098870 · Cited by 344 · cited 4×
“For the first strategy, the transmit power of the primary user is updated according to the classical power control algorithm [11]”
BackgroundMethodInfluentialCognitive Wireless Communication Networks
(2007) · Springer eBooks · doi:10.1007/978-0-387-68832-9 · Cited by 337 · cited 1×
“There are two paradigms for spectrum sharing, namely, spectrum overlay and spectrum underlay [3].”
BackgroundMultiantenna-Assisted Spectrum Sensing for Cognitive Radio
Wang, P., Fang, J., Han, N., & Li, H. (2010) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2009.2037912 · Cited by 335 · cited 2×
“The transparency of secondary to primary can be accomplished by letting the secondary user to perform spectrum sensing to explore idle spectrum [4] or to strictly control its transmit power such that the interference to the primary networks is under a desired threshold [5]–[7].”
BackgroundSelf-citationJoint rate and power allocation for cognitive radios in dynamic spectrum access environment
Kim, D. I., Le, L. B., & Hossain, E. (2008) · IEEE Transactions on Wireless Communications · doi:10.1109/t-wc.2008.071465 · Cited by 266 · cited 1×
“The transparency of secondary to primary can be accomplished by letting the secondary user to perform spectrum sensing to explore idle spectrum [4] or to strictly control its transmit power such that the interference to the primary networks is under a desired threshold [5]–[7].”
BackgroundA Q-learning based approach to interference avoidance in self-organized femtocell networks
Bennis, M., & Niyato, D. (2010) · 2010 IEEE Globecom Workshops · doi:10.1109/glocomw.2010.5700414 · Cited by 161 · cited 1×
“Reinforcement learning [24], also known as Q-learning, has been explored for cognitive radio applications such as dynamic spectrum access [25]–[31].”
BackgroundActive Cooperation Between Primary Users and Cognitive Radio Users in Heterogeneous Ad-Hoc Networks
Su, W., Matyjas, J. D., & Batalama, S. N. (2012) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2181841 · Cited by 118 · cited 1×
“[3], [8]–[10], also considered an active model in which some (cooperative or non-cooperative) interaction between the primary user and the secondary user are allowed to obtain improved transmission performance or economic compensations.”
BackgroundLearning to Compete for Resources in Wireless Stochastic Games
Fu, F., & van der Schaar, M. (2009) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2008.2002917 · Cited by 115
A potential game approach to distributed power control and scheduling
Heikkinen, T. (2005) · Computer Networks · doi:10.1016/j.comnet.2005.09.010 · Cited by 108 · cited 1×
“Besides optimization-based methods, power allocation from the game theory’s point of view was also studied [20]–[23].”
MethodMitliagkas, I., Sidiropoulos, N. D., & Swami, A. (2011) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2011.100811.101381 · Cited by 83 · cited 1×
“The transparency of secondary to primary can be accomplished by letting the secondary user to perform spectrum sensing to explore idle spectrum [4] or to strictly control its transmit power such that the interference to the primary networks is under a desired threshold [5]–[7].”
BackgroundChen, X., Zhao, Z., & Zhang, H. (2012) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2012.178 · Cited by 73 · cited 1×
“In [21], the power allocation problem was formulated as a noncooperative game with selfish users, where a sufficient condition for the existence of a Nash equilibrium was provided, and a stochastic power adaption with conjecture-based multiagent Q-learning approach was developed.”
BackgroundMultiagent -Learning for Aloha-Like Spectrum Access in Cognitive Radio Systems
Li, H. (2010) · EURASIP Journal on Wireless Communications and Networking · doi:10.1155/2010/876216 · Cited by 72
Self-citationRetrodirective Multi-User Wireless Power Transfer With Massive MIMO
Lee, S., Zeng, Y., & Zhang, R. (2017) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2017.2753259 · Cited by 56 · cited 1×
“Based on [11], modified approaches with different constraints or scenarios were developed [10], [12]–[16].”
BackgroundAdmission and Power Control for Spectrum Sharing Cognitive Radio Networks
Tadrous, J., Sultan, A., & Nafie, M. (2011) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2011.040411.101571 · Cited by 54 · cited 2×
“The transparency of secondary to primary can be accomplished by letting the secondary user to perform spectrum sensing to explore idle spectrum [4] or to strictly control its transmit power such that the interference to the primary networks is under a desired threshold [5]–[7].”
BackgroundReinforcement learning based distributed multiagent sensing policy for cognitive radio networks
Lundén, J., Koivunen, V., Kulkarni, S. R., & Poor, H. V. (2011) · IEEE International Symposium on Dynamic Spectrum Access Networks · doi:10.1109/dyspan.2011.5936261 · Cited by 52
Two-Sided Matching Based Cooperative Spectrum Sharing
Gao, L., Duan, L., & Huang, J. (2016) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2016.2557794 · Cited by 51 · cited 1×
“Besides optimization-based methods, power allocation from the game theory’s point of view was also studied [20]–[23].”
MethodRevenue Sharing Based Resource Allocation for Dynamic Spectrum Access Networks
Wu, Y., Zhu, Q., Huang, J., & Tsang, D. H. K. (2014) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2014.141125 · Cited by 41 · cited 3×
“According to the roles of the primary user, the operation of spectrum sharing or dynamic spectrum access can be classified into a passive primary user model and an active primary user model [3].”
BackgroundAdaptive power control algorithm in cognitive radio based on game theory
Yang, G., Li, B., Tan, X., & Wang, X. (2015) · IET Communications · doi:10.1049/iet-com.2014.1109 · Cited by 39
Distributed Power and Admission Control for Cognitive Radio Networks Using Antenna Arrays
Islam, M. H., Liang, Y., & Hoang, A. T. (2007) · doi:10.1109/dyspan.2007.40 · Cited by 37 · cited 3×
“Also, in [10], to achieve spectrum sharing, the primary user and the secondary user are allowed to interact with each other to update their respective”
BackgroundLiu, Y., Dai, Y., & Ma, S. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2428224 · Cited by 29 · cited 1×
“Other optimization-based methods were also proposed [17]–[19] in recent years.”
MethodLiu, Y., Hong, M., & Song, E. (2016) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2016.2542240 · Cited by 25 · cited 1×
“Other optimization-based methods were also proposed [17]–[19] in recent years.”
MethodOptimal Power Allocation in NOMA Systems with Imperfect Channel Estimation
Şenel, K., & Tekinay, S. (2017) · Globecom · doi:10.1109/glocom.2017.8254919 · Cited by 22
Spectrum sharing in multi-service cognitive network using reinforcement learning
Alsarhan, A., & Agarwal, A. (2009) · doi:10.1109/ukiwcws.2009.5749427 · Cited by 19
Power Control for Constrained Throughput Maximization in Spectrum Shared Networks
Tadrous, J., Sultan, A., Nafie, M., & El‐Keyi, A. (2010) · Globecom · doi:10.1109/glocom.2010.5683171 · Cited by 13
Cooperative spectrum sharing in cognitive radio networks with proactive primary system
Zhu, Q., Wu, Y., Tsang, D. H. K., & Peng, H. (2013) · IEEE/CIC International Conference on Communications in China - Workshops (CIC/ICCC) · doi:10.1109/iccchinaw.2013.6670572 · Cited by 8
Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum AccessPreprint
Naparstek, O., & Cohen, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.02613 · Cited by 4
Cited by
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,060
“The authors in [166] consider spectrum sensing and power control in non-cooperative cognitive radio networks.”
BackgroundDeep Learning in Mobile and Wireless Networking: A Survey
Zhang, C., Patras, P., & Haddadi, H. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2904897 · Cited by 1,768
Wireless Networks Design in the Era of Deep Learning: Model-Based, AI-Based, or Both?
Zappone, A., Di Renzo, M., & Debbah, M. (2019) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2924010 · Cited by 684
Wireless Networks Design in the Era of Deep Learning: Model-Based, AI-Based, or Both?
Zappone, A., Di Renzo, M., & Debbah, M. (2019) · arXiv · doi:10.48550/arxiv.1902.02647 · Cited by 685
Quantum Machine Learning for 6G Communication Networks: State-of-the-Art and Vision for the Future
Nawaz, S. J., Sharma, S. K., Wyne, S., Patwary, M. N., & Asaduzzaman, M. (2019) · IEEE Access · doi:10.1109/access.2019.2909490 · Cited by 658
“A deep reinforcement learning based power control method for spectrum sharing in cognitive radios has been proposed in [110].”
Method
Metadata
- Authors
- Li, X., Fang, J., Cheng, W., Duan, H., Chen, Z., & Li, H.
- Affiliations
- University of Electronic Science and Technology of China (CN)
- Stevens Institute of Technology (US)
- Venue
- IEEE Access
- Published
- 2018
- Volume
- 6
- Pages
- 25463-25473
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2783558323
- Cited by
- 192
- References
- 40
- Citations by year
- 2026: 6 · 2025: 10 · 2024: 17 · 2023: 25 · 2022: 34 · 2021: 37 · 2020: 31 · 2019: 28 · 2018: 4
- FWCI
- 15.48
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- National Science Foundation of China (61522104)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Cognitive Radio Networks and Spectrum Sensing
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Wireless Communication Security Techniques
- Physical Sciences › Computer Science › Computer Networks and Communications › Wireless Communication Networks Research
- Keywords
- spectrum sharing, cognitive radio, transmission power control, deep reinforcement learning, primary and secondary users, sensor nodes, quality of service
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.