The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access
Open accessAbstract
We consider the problem of dynamic spectrum access for network utility maximization in multichannel wireless networks. The shared bandwidth is divided into K orthogonal channels. In the beginning of each time slot, each user selects a channel and transmits a packet with a certain transmission probability. After each time slot, each user that has transmitted a packet receives a local observation indicating whether its packet was successfully delivered or not (i.e., ACK signal). The objective is a multi-user strategy for accessing the spectrum that maximizes a certain network utility in a distributed manner without online coordination or message exchanges between users. Obtaining an optimal solution for the spectrum access problem is computationally expensive, in general, due to the large-state space and partial observability of the states. To tackle this problem, we develop a novel distributed dynamic spectrum access algorithm based on deep multi-user reinforcement leaning. Specifically, at each time slot, each user maps its current state to the spectrum access actions based on a trained deep-Q network used to maximize the objective function. Game theoretic analysis of the system dynamics is developed for establishing design principles for the implementation of the algorithm. The experimental results demonstrate the strong performance of the algorithm.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
Dueling Network Architectures for Deep Reinforcement Learning
Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06581 · Cited by 3,300
A Survey of Dynamic Spectrum Access
Zhao, Q., & Sadler, B. M. (2007) · IEEE Signal Processing Magazine · doi:10.1109/msp.2007.361604 · Cited by 2,698
Deep Learning in Mobile and Wireless Networking: A Survey
Zhang, C., Patras, P., & Haddadi, H. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2904897 · Cited by 1,768
Decentralized cognitive MAC for opportunistic spectrum access in ad hoc networks: A POMDP framework
Zhao, Q., Tong, L., Swami, A., & Chen, Y. (2007) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2007.070409 · Cited by 1,587
Learning to Communicate with Deep Multi-Agent Reinforcement Learning
Foerster, J. N., Assael, Y. M., de Freitas, N., & Whiteson, S. (2016) · arXiv · doi:10.48550/arxiv.1605.06676 · Cited by 1,217
Deep Reinforcement Learning: An Overview
Li, Y. (2017) · arXiv · doi:10.48550/arxiv.1701.07274 · Cited by 762
Gambling in a rigged casino: The adversarial multi-armed bandit problem
Auer, P., Cesa-Bianchi, N., Freund, Y., & Schapire, R. E. (2002) · Proceedings of IEEE 36th Annual Foundations of Computer Science · doi:10.1109/sfcs.1995.492488 · Cited by 742
Deep Recurrent Q-Learning for Partially Observable MDPs
Hausknecht, M., & Stone, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.06527 · Cited by 667
Matching theory for future wireless networks: fundamentals and applications
Gu, Y., Saad, W., Bennis, M., Debbah, M., & Han, Z. (2015) · IEEE Communications Magazine · doi:10.1109/mcom.2015.7105641 · Cited by 561
Fair multiuser channel allocation for OFDMA networks using Nash bargaining solutions and coalitions
Han, Z., Ji, Z., & Liu, K. J. R. (2005) · IEEE Transactions on Communications · doi:10.1109/tcomm.2005.852826 · Cited by 540
Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networks
Wang, S., Liu, H., Gomes, P. H., & Krishnamachari, B. (2018) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2018.2809722 · Cited by 477
Optimality of Myopic Sensing in Multichannel Opportunistic Access
Ahmad, S. H. A., Liu, M., Javidi, T., Zhao, Q., & Krishnamachari, B. (2009) · IEEE Transactions on Information Theory · doi:10.1109/tit.2009.2025561 · Cited by 364
Liu, K., & Zhao, Q. (2010) · IEEE Transactions on Information Theory · doi:10.1109/tit.2010.2068950 · Cited by 347
Communication Networks An Optimization Control And Stochastic Networks Perspective
Srikant, R., & Ying, L. (2017) · Cited by 276
List-coloring based channel allocation for open-spectrum wireless networks
Wang, W., & Liu, X. (2006) · VTC-2005-Fall. 2005 IEEE 62nd Vehicular Technology Conference, 2005. · doi:10.1109/vetecf.2005.1558001 · Cited by 220
Learning in a Changing World: Restless Multiarmed Bandit With Unknown Dynamics
Liu, H., Liu, K., & Zhao, Q. Q. (2012) · IEEE Transactions on Information Theory · doi:10.1109/tit.2012.2230215 · Cited by 179
Leshem, A., Zehavi, E., & Yaffe, Y. (2012) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2012.120108 · Cited by 155
Achieving Proportional Fairness Using Local Information in Aloha Networks
Kar, K., Sarkar, S., & Tassiulas, L. (2004) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2004.835596 · Cited by 137
Network Games: Theory, Models, and Dynamics
Menache, I., & Ozdaglar, A. (2011) · Synthesis lectures on communication networks · doi:10.2200/s00330ed1v01y201101cnt009 · Cited by 121
Deep Learning in Mobile and Wireless Networking: A SurveyPreprint
Zhang, C., Patras, P., & Haddadi, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.04311 · Cited by 121
Deep Multi-User Reinforcement Learning for Dynamic Spectrum Access in Multichannel Wireless Networks
Naparstek, O., & Cohen, K. (2017) · Globecom · doi:10.1109/glocom.2017.8254101 · Cited by 116
Self-citationMochaourab, R., Holfeld, B., & Wirth, T. (2015) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2015.2414918 · Cited by 97
Active Anomaly Detection in Heterogeneous Processes
Huang, B., Cohen, K., & Zhao, Q. Q. (2018) · IEEE Transactions on Information Theory · doi:10.1109/tit.2018.2866257 · Cited by 87
Self-citationMultiagent -Learning for Aloha-Like Spectrum Access in Cognitive Radio Systems
Li, H. (2010) · EURASIP Journal on Wireless Communications and Networking · doi:10.1155/2010/876216 · Cited by 69
Multi-user lax communications: A multi-armed bandit approach
Avner, O., & Mannor, S. (2016) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2016.7524557 · Cited by 67
On Optimality of Myopic Policy for Restless Multi-Armed Bandit Problem: An Axiomatic Approach
Wang, K., & Chen, L. (2011) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2011.2170684 · Cited by 62
Game Theoretic Aspects of the Multi-Channel ALOHA Protocol in Cognitive Radio Networks
Cohen, K., Leshem, A., & Zehavi, E. (2013) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2013.131109 · Cited by 57
Self-citationAsymptotically Optimal Anomaly Detection via Sequential Testing
Cohen, K., & Zhao, Q. Q. (2015) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2015.2416674 · Cited by 53
Self-citationPrice of Anarchy for Congestion Games in Cognitive Radio Networks
Law, L. M., Huang, J., & Liu, M. (2012) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2012.083112.120371 · Cited by 46
Proactive Resource Management in LTE-U Systems: A Deep Learning Perspective.Preprint
Challita, U., Dong, L., & Saad, W. (2017) · arXiv (Cornell University) · Cited by 46
Fully Distributed Optimal Channel Assignment for Open Spectrum Access
Naparstek, O., & Leshem, A. (2013) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2013.2285512 · Cited by 43
Self-citationApproximately optimal adaptive learning in opportunistic spectrum access
Tekin, C., & Liu, M. (2012) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infcom.2012.6195522 · Cited by 37
Distributed Game-Theoretic Optimization and Management of Multichannel ALOHA Networks
Cohen, K., & Leshem, A. (2015) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2015.2431121 · Cited by 36
Self-citationCao, H., & Cai, J. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2789344 · Cited by 27
Proportionally Fair Distributed Resource Allocation in Multiband Wireless Systems
Hou, I.-H., & Gupta, P. (2013) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2013.2284494 · Cited by 24
Approximate Best-Response Dynamics in Random Interference Games
Bistritz, I., & Leshem, A. (2017) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2017.2749139 · Cited by 20
Deep Reinforcement Learning Radio Control and Signal Detection with KeRLym, a Gym RL AgentPreprint
O’Shea, T. J., & Clancy, T. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.09221 · Cited by 19
On exploiting idle channels in opportunistic multichannel ALOHA
To, T., & Choi, J. (2010) · IEEE Communications Letters · doi:10.1109/lcomm.2010.01.091494 · Cited by 18
Deep Reinforcement One-Shot Learning for Artificially Intelligent Classification SystemsPreprint
Puzanov, A., & Cohen, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.01527 · Cited by 16
Self-citation
Cited by
On the Road to 6G: Visions, Requirements, Key Technologies, and Testbeds
Wang, C.-X., You, X., Gao, X., Zhu, X., Li, Z., Zhang, C., Wang, H., Huang, Y., Chen, Y., Haas, H., et al. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3249835 · Cited by 2,336
Deep Learning for Intelligent Wireless Networks: A Comprehensive Survey
Mao, Q., Hu, F., & Hao, Q. (2018) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2846401 · Cited by 789
Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks
Nasir, Y. S., & Guo, D. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2933973 · Cited by 660
Zhao, N., Liang, Y.-C., Niyato, D., Pei, Y., Wu, M., & Jiang, Y. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2933417 · Cited by 461
Du, J., Jiang, C., Wang, J., Ren, Y., & Debbah, M. (2020) · IEEE Vehicular Technology Magazine · doi:10.1109/mvt.2020.3019650 · Cited by 372
Metadata
- Authors
- Naparstek, O., & Cohen, K.
- Venue
- IEEE Transactions on Wireless Communications
- Published
- 2019-01
- Volume
- 18
- Issue
- 1
- Pages
- 310-323
- ISSN
- 1536-1276 (ISSN-L)
- 1558-2248
- OpenAlex
- W2898754723
- Cited by
- 477
- References
- 43
- FWCI
- 32.05
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.