The DOI matches its Crossref record; 36 of the 37 references it cites were found in a registry or an index.
A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access
Open accessAbstract
To make efficient use of limited spectral resources, we in this work propose a deep actor-critic reinforcement learning based framework for dynamic multichannel access. We consider both a single-user case and a scenario in which multiple users attempt to access channels simultaneously. We employ the proposed framework as a single agent in the single-user case, and extend it to a decentralized multi-agent framework in the multi-user scenario. In both cases, we develop algorithms for the actor-critic deep reinforcement learning and evaluate the proposed learning policies via experiments and numerical results. In the single-user model, in order to evaluate the performance of the proposed channel access policy and the framework's tolerance against uncertainty, we explore different channel switching patterns and different switching probabilities. In the case of multiple users, we analyze the probabilities of each user accessing channels with favorable channel conditions and the probability of collision. We also address a time-varying environment to identify the adaptive ability of the proposed framework. Additionally, we provide comparisons (in terms of both the average reward and time efficiency) between the proposed actor-critic deep reinforcement learning framework, Deep-Q network (DQN) based approach, random access, and the optimal policy when the channel dynamics are known.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The challenge with deep actor criticism is that both actor and critic need stable targets.”
From Deep reinforcement learning-based adaptive fuzzy control for electro-hydraulic servo system (2025) · cites this paper as [16]
In the citing paper
“It requires consistently training the critic without divergence that delays learning and the propagation of value estimations through the target network [16].”
From Deep reinforcement learning-based adaptive fuzzy control for electro-hydraulic servo system (2025) · cites this paper as [16]
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D. T., Wang, P., Liang, Y., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,181
Deep Learning in Mobile and Wireless Networking: A Survey
Zhang, C., Patras, P., & Haddadi, H. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2904897 · Cited by 1,913
Decentralized cognitive MAC for opportunistic spectrum access in ad hoc networks: A POMDP framework
Zhao, Q., Tong, L., Swami, A., & Chen, Y. (2007) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2007.070409 · Cited by 1,587
Monte-Carlo Planning in Large POMDPs
Silver, D., & Veness, J. (2010) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 886
Peters, J., & Schaal, S. (2008) · Neurocomputing · doi:10.1016/j.neucom.2007.11.026 · Cited by 738
Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access
Naparstek, O., & Cohen, K. (2018) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2018.2879433 · Cited by 504
Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networks
Wang, S., Liu, H., Gomes, P. H., & Krishnamachari, B. (2018) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2018.2809722 · Cited by 489
(2004) · doi:10.5555/1036843.1036906 · Cited by 417
On myopic sensing for multi-channel opportunistic access: structure, optimality, and performance
Zhao, Q., Krishnamachari, B., & Liu, K. (2008) · IEEE Transactions on Wireless Communications · doi:10.1109/t-wc.2008.071349 · Cited by 383
Liu, K., & Zhao, Q. Q. (2010) · IEEE Transactions on Information Theory · doi:10.1109/tit.2010.2068950 · Cited by 365
Optimality of Myopic Sensing in Multichannel Opportunistic Access
Ahmad, S. H. A., Liu, M., Javidi, T., Zhao, Q. Q., & Krishnamachari, B. (2009) · IEEE Transactions on Information Theory · doi:10.1109/tit.2009.2025561 · Cited by 364
Full spectrum sharing in cognitive radio networks toward 5G: a survey
Hu, F., Chen, B., & Zhu, K. (2018) · IEEE Access · doi:10.1109/access.2018.2802450 · Cited by 321
Wei, Y., Yu, F. R., Song, M. N., & Han, Z. (2017) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2017.2769644 · Cited by 298
Xu, Y., Wang, J., Wu, Q., Anpalagan, A., & Yao, Y. (2012) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2012.020812.110025 · Cited by 280
Thrun, S. (1999) · neural information processing systems · Cited by 241
Reinforcement Learning-Based NOMA Power Allocation in the Presence of Smart Jamming
Xiao, L., Li, Y., Dai, C., Dai, H., & Poor, H. V. (2017) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2782726 · Cited by 230
Deep Reinforcement Learning for Resource Allocation in V2V Communications
Ye, H., & Li, G. Y. (2018) · 2018 IEEE International Conference on Communications (ICC) · doi:10.1109/icc.2018.8422586 · Cited by 222
MAA*: A Heuristic Search Algorithm for Solving Decentralized POMDPsPreprint
Szer, D., Charpillet, F., & Zilberstein, S. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1207.1359 · Cited by 155
Liu, S., Hu, X., & Wang, W. (2018) · IEEE Access · doi:10.1109/access.2018.2809581 · Cited by 128
Multiagent -Learning for Aloha-Like Spectrum Access in Cognitive Radio Systems
Li, H. (2010) · EURASIP Journal on Wireless Communications and Networking · doi:10.1155/2010/876216 · Cited by 72
Stochastic Game-Theoretic Spectrum Access in Distributed and Dynamic Environment
Zheng, J., Cai, Y., Lu, N., Xu, Y., & Shen, X. (2014) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2014.2366559 · Cited by 70
Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks
Yu, Y., Wang, T., & Liew, S. C. (2018) · International Conference on Communications · doi:10.1109/icc.2018.8422168 · Cited by 62
Reinforcement Learning for Energy Harvesting Decode-and-Forward Two-Hop Communications
Ortiz, A., Al-Shatri, H., Li, X., Weber, T., & Klein, A. (2017) · IEEE Transactions on Green Communications and Networking · doi:10.1109/tgcn.2017.2703855 · Cited by 60
Deep Q-Learning Based Dynamic Resource Allocation for Self-Powered Ultra-Dense Networks
Li, H., Gao, H., Lv, T., & Lu, Y. (2018) · IEEE International Conference on Communications workshops · doi:10.1109/iccw.2018.8403505 · Cited by 60
A Restless Bandit Formulation of Opportunistic Access: Indexablity and Index Policy
Liu, K., & Zhao, Q. (2008) · doi:10.1109/sahcnw.2008.12 · Cited by 54
A Reinforcement Learning-Based Resource Allocation Scheme for Cloud Robotics
Liu, H., Liu, S., & Zheng, K. (2018) · IEEE Access · doi:10.1109/access.2018.2814606 · Cited by 52
Yau, K. A., Komisarczuk, P., & Paul, D. T. (2010) · doi:10.1109/lcn.2010.5735689 · Cited by 48
Distributed Reinforcement Learning based MAC protocols for autonomous cognitive secondary users
Bkassiny, M., Jayaweera, S. K., & Avery, K. A. (2011) · Wireless and Optical Communications Conference · doi:10.1109/wocc.2011.5872298 · Cited by 42
Online learning for multi-channel opportunistic access over unknown Markovian channels
Dai, W., Gai, Y., & Krishnamachari, B. (2014) · IEEE International Conference on Sensing, Communication, and Networking (SECON) · doi:10.1109/sahcn.2014.6990328 · Cited by 20
Zhang, Y., Zhang, Q., Cao, B., & Chen, P. (2014) · IEEE International Conference on Communication Systems (ICCS) · doi:10.1109/iccs.2014.7024826 · Cited by 12
Nassar, A. T., & Yılmaz, Y. (2018) · arXiv (Cornell University) · Cited by 11
Optimally Probing Channel in Opportunistic Spectrum Access
Wang, K., Liu, Q., Fan, Q., & Ai, Q. (2017) · IEEE Communications Letters · doi:10.1109/lcomm.2017.2778156 · Cited by 8
Flagged first, then by citations
1 of the 37 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Tan, X. L., Zhou, L., Wang, H., Sun, Y., Zhao, H., Seet, B., Wei, J., & Leung, V. C. M. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3168296 · Cited by 133
Li, Y., Zhang, W., Wang, C., Sun, J. F., & Liu, Y. (2020) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2020.2982895 · Cited by 96
Alwarafy, A., Abdallah, M. M., Çiftler, B. S., Al‐Fuqaha, A., & Hamdi, M. (2022) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2022.3153226 · Cited by 130
Mohanavel, V., Tamilselvi, M., Ramkumar, G., Prabu, R. T., & Anitha, G. (2022) · International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC) · doi:10.1109/i-smac55078.2022.9987395 · Cited by 65
Multi-Channel Opportunistic Access for Heterogeneous Networks Based on Deep Reinforcement Learning
Ye, X., Yu, Y., & Fu, L. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3099495 · Cited by 54
Metadata
- Authors
- Zhong, C., Lu, Z., Gursoy, M. C., & Velipasalar, S.
- Affiliations
- Syracuse University (US)
- Venue
- IEEE Transactions on Cognitive Communications and Networking
- Published
- 2019-12
- Volume
- 5
- Issue
- 4
- Pages
- 1125-1139
- ISSN
- 2332-7731 (ISSN-L)
- 2372-2045
- OpenAlex
- W2983432303
- Cited by
- 123
- References
- 37
- Citations by year
- 2026: 11 · 2025: 15 · 2024: 20 · 2023: 20 · 2022: 21 · 2021: 26 · 2020: 9 · 2019: 1
- FWCI
- 7.80
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Cognitive Communications and Networking (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Wireless Networks and Protocols
- Physical Sciences › Computer Science › Computer Networks and Communications › Cognitive Radio Networks and Spectrum Sensing
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Wireless Network Optimization
- Keywords
- channel switching, multi-agent reinforcement learning, collision probability, deep Q-network comparison, random access, cognitive radio
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
