The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networks
Open accessAbstract
We consider a dynamic multichannel access problem, where multiple correlated channels follow an unknown joint Markov model and users select the channel to transmit data. The objective is to find a policy that maximizes the expected long-term number of successful transmissions. The problem is formulated as a partially observable Markov decision process with unknown system dynamics. To overcome the challenges of unknown dynamics and prohibitive computation, we apply the concept of reinforcement learning and implement a deep Q-network (DQN). We first study the optimal policy for fixedpattern channel switching with known system dynamics and show through simulations that DQN can achieve the same optimal performance without knowing the system statistics. We then compare the performance of DQN with a Myopic policy and a Whittle Index-based heuristic through both more general simulations as well as real data trace and show that DQN achieves near-optimal performance in more complex situations. Finally, we propose an adaptive DQN approach with the capability to adapt its learning in time-varying scenarios.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
NeXt generation/dynamic spectrum access/cognitive radio wireless networks: A survey
Akyildiz, I. F., Lee, W.-Y., Vuran, M. C., & Mohanty, S. (2006) · Computer Networks · doi:10.1016/j.comnet.2006.05.001 · Cited by 6,356
Information capacity and power control in single-cell multiuser communications
Knopp, R., & Humblet, P. A. (2002) · Proceedings IEEE International Conference on Communications ICC '95 · doi:10.1109/icc.1995.525188 · Cited by 2,059
End-to-End Training of Deep Visuomotor Policies
Levine, S., Finn, C., Darrell, T. J., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00702 · Cited by 1,344
The Complexity of Markov Decision Processes
Papadimitriou, C. H., & Tsitsiklis, J. N. (1987) · Mathematics of Operations Research · doi:10.1287/moor.12.3.441 · Cited by 1,329
Multiple Object Recognition with Visual Attention
Ba, J., Mnih, V., & Kavukcuoglu, K. (2014) · arXiv · doi:10.48550/arxiv.1412.7755 · Cited by 808
An MDP-Based Vertical Handoff Decision Algorithm for Heterogeneous Wireless Networks
Stevens-Navarro, E., Lin, Y., & Wong, V. W. S. (2008) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2007.907072 · Cited by 395
Optimality of Myopic Sensing in Multichannel Opportunistic Access
Ahmad, S. H. A., Liu, M., Javidi, T., Zhao, Q., & Krishnamachari, B. (2009) · IEEE Transactions on Information Theory · doi:10.1109/tit.2009.2025561 · Cited by 364
Self-citationOn myopic sensing for multi-channel opportunistic access: structure, optimality, and performance
Zhao, Q., Krishnamachari, B., & Liu, K. (2008) · IEEE Transactions on Wireless Communications · doi:10.1109/t-wc.2008.071349 · Cited by 360
Self-citationLiu, K., & Zhao, Q. (2010) · IEEE Transactions on Information Theory · doi:10.1109/tit.2010.2068950 · Cited by 347
Online learning in opportunistic spectrum access: A restless bandit approach
Tekin, C., & Liu, M. (2011) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infcom.2011.5935068 · Cited by 97
Opportunistic Bandwidth Sharing Through Reinforcement Learning
Venkatraman, P., Hamdaoui, B., & Guizani, M. (2010) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2010.2048766 · Cited by 66
Regret Bounds for Restless Markov Bandits
Ortner, R., Ryabko, D., Auer, P. L., & Munos, R. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34106-9_19 · Cited by 38
Logarithmic weak regret of non-Bayesian restless multi-armed bandit
Liu, H., Liu, K., & Zhao, Q. Q. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5946895 · Cited by 37
Self-citationData-Efficient Learning of Feedback Policies from Image Pixels using Deep Dynamical ModelsPreprint
Assael, J.-A. M., Wahlström, N., Schön, T. B., & Deisenroth, M. P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.02173 · Cited by 25
Online learning for multi-channel opportunistic access over unknown Markovian channels
Dai, W., Gai, Y., & Krishnamachari, B. (2014) · IEEE International Conference on Sensing, Communication, and Networking (SECON) · doi:10.1109/sahcn.2014.6990328 · Cited by 20
Self-citationZhang, Y., Zhang, Q., Cao, B., & Chen, P. (2014) · IEEE International Conference on Communication Systems (ICCS) · doi:10.1109/iccs.2014.7024826 · Cited by 12
Online learning of power allocation policies in energy harvesting communications
Sakulkar, P., & Krishnamachari, B. (2016) · International Conference on Signal Processing and Communications (SPCOM) · doi:10.1109/spcom.2016.7746654 · Cited by 6
Self-citation
Cited by
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,060
Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks
Wang, J., Jiang, C., Zhang, H., Ren, Y., Chen, K.-C., & Hanzo, L. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2965856 · Cited by 486
Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access
Naparstek, O., & Cohen, K. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2018.2879433 · Cited by 477
Zhao, N., Liang, Y.-C., Niyato, D., Pei, Y., Wu, M., & Jiang, Y. (2019) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2933417 · Cited by 461
Single and multi-agent deep reinforcement learning for AI-enabled wireless networks: A tutorial
Feriani, A., & Hossain, E. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3063822 · Cited by 410
Metadata
- Authors
- Wang, S., Liu, H., Gomes, P. H., & Krishnamachari, B.
- Venue
- IEEE Transactions on Cognitive Communications and Networking
- Published
- 2018-06
- Volume
- 4
- Issue
- 2
- Pages
- 257-265
- ISSN
- 2332-7731 (ISSN-L)
- OpenAlex
- W2963079995
- Cited by
- 477
- References
- 22
- FWCI
- 38.28
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.