Reinforcement learning based spectrum-aware routing in multi-hop cognitive radio networks
Abstract
Routing in multi-hop cognitive radio networks (CRN) should be spectrum-aware. In this paper, two adaptive reinforcement learning based spectrum-aware routing protocols are introduced. Q-Learning and Dual Reinforcement Learning are applied respectively for them. The cognitive nodes store a table of Q values that estimate the numbers of available channels on the routes and update them while routing. So they can adaptively learn good routes which have more available channels from just local information. Compared to the previous spectrumaware routing protocols in multi-hop cognitive radio networks, they are simpler and easier to implement, more cost-effective, and can avoid drawbacks in on-demand protocols but still keep adaptive and dynamic routing. Both of our protocols perform better than the spectrum-aware shortest path protocol when network load is not too low. In the meantime, spectrum-aware DRQ-routing learns the optimal routing policy 1.5 times as fast as the spectrum-aware Q-routing at low and medium network load. It also learns a routing policy which is more than seven times as good as that of spectrum-aware Q-routing at high network load.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
As activity level reaches 2.75 packet/s, over 3000 periods are necessary for DRQ-routing to converge.
“As the activity level reaches 2.75packet/s, over 3000 periods are necessary for DRQ-routing to converge.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [482]
After convergence, DRQ-routing minimizes end-to-end delay.
“Simulation results show that after convergence, DRQ-routing minimizes end-to-end delay”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [482]
DRQ-routing significantly reduces end-to-end delay compared to SP-routing at higher activity levels.
“it significantly reduces end-to-end delay compared to SP-routing at higher activity levels”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [482]
DRQ-routing convergence time at a 2 packet/s activity level is around 700sec, implying 1400 periods elapsed before convergence.
“the convergence time at a 2 packet/s activity level is around 700sec which implies that 1400 periods have elapsed before DQR-routing has converged.”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [482]
DRQ-routing is faster to converge than Q-routing (50% faster at lower activity level).
“is faster to converge than Q-routing (50% faster at lower activity level)”
From A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities (2018) · cites this paper as [482]
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225 · see citing works
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,263 · see citing works
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972 · see citing works
Cognitive Radio An Integrated Agent Architecture for Software Defined Radio
Mitola, J. (2000) · Cited by 3,509 · see citing works
Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach
Boyan, J. A., & Littman, M. L. (1994) · Cited by 694 · see citing works
Joint On-Demand Routing and Spectrum Assignment in Cognitive Radio Networks
Cheng, G., Liu, W., Li, Y., & Cheng, W. (2007) · International Conference on Communications · doi:10.1109/icc.2007.1075 · Cited by 230 · see citing works
A novel layered graph model for topology formation and routing in dynamic spectrum access networks
Xin, C., Xie, B., & Shen, C.-C. (2005) · doi:10.1109/dyspan.2005.1542647 · Cited by 203 · see citing works
Spectrum Aware On-Demand Routing in Cognitive Radio Networks
Cheng, G., Liu, W., Li, Y., & Cheng, W. (2007) · IEEE International Symposium on Dynamic Spectrum Access Networks · doi:10.1109/dyspan.2007.80 · Cited by 203 · see citing works
Spectrum Aware Routing for Multi-Hop Cognitive Radio Networks with a Single Transceiver
Ma, H., Zheng, L., Ma, X., & Luo, Y. (2008) · International Conference on Cognitive Radio Oriented Wireless Networks and Communications · doi:10.1109/crowncom.2008.4562509 · Cited by 158 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Boutaba, R., Salahuddin, M. A., Limam, N., Ayoubi, S., Shahriar, N., Estrada-Solano, F., & Caicedo, O. M. (2018) · Journal of Internet Services and Applications · doi:10.1186/s13174-018-0087-2 · Cited by 1,036 · see citing works
Reinforcement Learning Based Routing in Networks: Review and Classification of Approaches
Mammeri, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2913776 · Cited by 435 · see citing works
Routing optimization meets Machine Intelligence: A perspective for the future network
Dai, B., Cao, Y., Wu, Z., Dai, Z., Yao, R., & Xu, Y. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.06.093 · Cited by 215 · see citing works
Yau, K. A., Komisarczuk, P., & Teal, P. D. (2011) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2011.08.007 · Cited by 133 · see citing works
Toward Packet Routing With Fully Distributed Multiagent Deep Reinforcement Learning
You, X., Li, X., Xu, Y., Feng, H., Zhao, J., & Yan, H. (2020) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2020.3012832 · Cited by 109 · see citing works
Application of reinforcement learning to routing in distributed wireless networks: a review
Al-Rawi, H. A. A., Ng, M. A., & Yau, K. A. (2013) · Artificial Intelligence Review · doi:10.1007/s10462-012-9383-6 · Cited by 103 · see citing works
Artificial intelligence based cognitive routing for cognitive radio networks
Qadir, J. (2015) · Artificial Intelligence Review · doi:10.1007/s10462-015-9438-6 · Cited by 52 · see citing works
SMART: A SpectruM-Aware ClusteR-based rouTing scheme for distributed cognitive radio networks
Saleem, Y., Yau, K. A., Mohamad, H., Ramli, N., & Rehmani, M. H. (2015) · Computer Networks · doi:10.1016/j.comnet.2015.08.019 · Cited by 46 · see citing works
Routing and channel selection from cognitive radio network’s perspective: A survey
Saleem, Y., Salim, F., & Rehmani, M. H. (2014) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2014.07.015 · Cited by 46 · see citing works
Syed, A. R., Yau, K. A., Qadir, J., Mohamad, H., Ramli, N., & Keoh, S. L. (2016) · IEEE Access · doi:10.1109/access.2016.2613122 · Cited by 44 · see citing works
Routing in Distributed Cognitive Radio Networks: A Survey
Al-Rawi, H. A. A., & Yau, K. A. (2012) · Wireless Personal Communications · doi:10.1007/s11277-012-0674-7 · Cited by 40 · see citing works
Toward Packet Routing with Fully-distributed Multi-agent Deep Reinforcement Learning
You, X., Li, X., Xu, Y., Feng, H., & Zhao, J. (2019) · International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt) · doi:10.23919/wiopt47501.2019.9144110 · Cited by 39 · see citing works
Intelligent and efficient development of wireless networks: A review of cognitive radio networks
Zhang, P., Liu, Y., Feng, Z., Zhang, Q., Li, Q., & Xu, D. (2012) · Chinese Science Bulletin · doi:10.1007/s11434-012-5334-5 · Cited by 38 · see citing works
Renold, A. P., & Chandrakala, S. (2016) · Wireless Personal Communications · doi:10.1007/s11277-016-3729-3 · Cited by 31 · see citing works
Learning-Based Spectrum Selection in Cognitive Radio Ad Hoc Networks
Felice, M. D., Chowdhury, K. R., Wu, C., Bononi, L., & Meleis, W. M. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-13315-2_11 · Cited by 28 · see citing works
Fair Downlink Traffic Management for Hybrid LAA-LTE/Wi-Fi Networks
Li, Y., Zhou, T., Yang, Y., Hu, H., & Hamalainen, M. (2016) · IEEE Access · doi:10.1109/access.2016.2642121 · Cited by 26 · see citing works
Self-citationA Deep Reinforcement Learning-Based Geographic Packet Routing Optimization
Bai, Y., Zhang, X., Yu, D., Li, S., Wang, Y., Lei, S., & Tian, Z. (2022) · IEEE Access · doi:10.1109/access.2022.3213649 · Cited by 24 · see citing works
A reinforcement learning-based routing scheme for cognitive radio ad hoc networks
Al-Rawi, H. A. A., Yau, K. A., Mohamad, H., Ramli, N., & Hashim, W. (2014) · IFIP Wireless and Mobile Networking Conference (WMNC) · doi:10.1109/wmnc.2014.6878881 · Cited by 22 · see citing works
Raj, R. N., Nayak, A., & Kumar, M. S. (2019) · International Journal of Wireless Information Networks · doi:10.1007/s10776-019-00463-6 · Cited by 22 · see citing works
Zheng, K., Li, H., Qiu, R. C., & Gong, S. (2012) · 2012 International Conference on Computing, Networking and Communications (ICNC) · doi:10.1109/iccnc.2012.6167444 · Cited by 21 · see citing works
Non-cooperative reinforcement learning based routing in cognitive radio networks
Pourpeighambar, B., Dehghan, M., & Sabaei, M. (2017) · Computer Communications · doi:10.1016/j.comcom.2017.02.013 · Cited by 20 · see citing works
Quality of experience-based routing in multi-service wireless mesh networks
Matos, R., Coutinho, N., Marques, C., Sargento, S., Chakareski, J., & Kassler, A. (2012) · International Conference on Communications · doi:10.1109/icc.2012.6364944 · Cited by 20 · see citing works
Barve, S., & Kulkarni, P. A. (2014) · Mobile Networks and Applications · doi:10.1007/s11036-014-0551-6 · Cited by 19 · see citing works
Felice, M. D., Chowdhury, K. R., Meleis, W. M., & Bononi, L. (2010) · IEEE Workshop on Wireless Mesh Networks · doi:10.1109/wimesh.2010.5507904 · Cited by 18 · see citing works
Coutinho, N., Matos, R., Marques, C., Reis, A. B., Sargento, S., Chakareski, J., & Kassler, A. (2015) · Computer Networks · doi:10.1016/j.comnet.2015.06.016 · Cited by 18 · see citing works
Application of Reinforcement Learning in Cognitive Radio Networks: Models and Algorithms
Yau, K. A., Poh, G. S., Chien, S. F., & Al-Rawi, H. A. A. (2014) · The Scientific World JOURNAL · doi:10.1155/2014/209810 · Cited by 18 · see citing works
Adaptive Sensing Scheduling and Spectrum Selection in Cognitive Wireless Mesh Networks
Felice, M. D., Chowdhury, K. R., Kassler, A., & Bononi, L. (2011) · International Conference on Computer Communications and Networks (ICCCN) · doi:10.1109/icccn.2011.6006042 · Cited by 18 · see citing works
Basak, S., & Acharya, T. (2015) · Physical Communication · doi:10.1016/j.phycom.2015.12.003 · Cited by 16 · see citing works
Reinforcement Learning for Routing in Cognitive Radio Ad Hoc Networks
Al-Rawi, H. A. A., Yau, K. A., Mohamad, H., Ramli, N., & Hashim, W. (2014) · The Scientific World JOURNAL · doi:10.1155/2014/960584 · Cited by 15 · see citing works
Joint routing and channel assignment using online learning in cognitive radio networks
Pourpeighambar, B., Dehghan, M., & Sabaei, M. (2018) · Wireless Networks · doi:10.1007/s11276-018-1672-9 · Cited by 15 · see citing works
Toward Packet Routing with Fully-distributed Multi-agent Deep Reinforcement LearningPreprint
You, X., Li, X., Xu, Y., Feng, H., Zhao, J., & Yan, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.03494 · Cited by 13 · see citing works
ICC'12 WS — ViOpt: Realizing advanced video optimized wireless networks — Program
(2012) · International Conference on Communications · doi:10.1109/icc.2012.6365058 · Cited by 11 · see citing works
Multi-agent learning based routing for delay minimization in Cognitive Radio Networks
Pourpeighambar, B., Dehghan, M., & Sabaei, M. (2017) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2017.01.026 · Cited by 11 · see citing works
Q-routing: From the Algorithm to the Routing Protocol
Bitaillou, A., Parrein, B., & Andrieux, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-45778-5_5 · Cited by 10 · see citing works
Al-Rawi, H. A. A., & Yau, K. A. (2013) · IEEE Symposium on Computational Intelligence for Communication Systems and Networks (CIComms) · doi:10.1109/cicomms.2013.6582850 · Cited by 10 · see citing works
Renold, A. P., & Chandrakala, S. (2016) · Wireless Personal Communications · doi:10.1007/s11277-016-3832-5 · Cited by 10 · see citing works
A Survey of Applying Reinforcement Learning Techniques to Multicast Routing
Ashour, O., St‐Hilaire, M., Kunz, T., & Wang, M. (2019) · IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) · doi:10.1109/uemcon47517.2019.8993014 · Cited by 9 · see citing works
Shin, D.-J., & Kim, J.-J. (2021) · Applied Sciences · doi:10.3390/app11188727 · Cited by 8 · see citing works
Shamaee, M. S., Shiri, M. E., & Sabaei, M. (2018) · Wireless Personal Communications · doi:10.1007/s11277-018-5700-y · Cited by 8 · see citing works
The Stable Routing Protocol for the Cognitive Network
Zhang, Y., Guan, J., Xu, C., & Zhang, H. (2012) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/glocomw.2012.6477730 · Cited by 7 · see citing works
A Performance based Routing Classification in Cognitive Radio Networks
Barve, S., & Kulkarni, P. A. (2012) · International Journal of Computer Applications · doi:10.5120/6370-8762 · Cited by 7 · see citing works
A Probabilistic Routing Protocol Based on Priori Information for Cognitive Radio Sensor Networks
Jin, Z. L., Kim, B. S., Guan, D. H., & Cho, J. S. (2013) · Applied Mechanics and Materials · doi:10.4028/www.scientific.net/amm.303-306.210 · Cited by 7 · see citing works
Machine Learning and Cognitive Technology for Intelligent Wireless Networks.Preprint
Zhou, X., Sun, M., Li, G. Y., & Juang, B.-H. F. (2017) · arXiv (Cornell University) · Cited by 7 · see citing works
A decision tree cognitive routing scheme for cognitive radio mesh networks
Soltani, S., & Mutka, M. W. (2013) · Wireless Communications and Mobile Computing · doi:10.1002/wcm.2418 · Cited by 6 · see citing works
Load balancing routing in cognitive radio ad hoc networks
Ye, H., Tan, Z., Xu, S., & Qiao, X. (2011) · IEEE International Symposium on Microwave, Antenna, Propagation and EMC Technologies for Wireless Communications · doi:10.1109/mape.2011.6156301 · Cited by 6 · see citing works
Felice, M. D., Bedogni, L., & Bononi, L. (2019) · Springer eBooks · doi:10.1007/978-981-10-1394-2_58 · Cited by 6 · see citing works
Improved Q-Reinforcement Learning based Optimal Channel Selection in Cognitive Radio Networks
Talekar, S. A., Banait, S. S., & Patil, M. B. (2023) · International journal of Computer Networks & Communications · doi:10.5121/ijcnc.2023.15301 · Cited by 5 · see citing works
Saleem, Y., Yau, K. A., Mohamad, H., Ramli, N., & Rehmani, M. H. (2015) · International Conference on Computer, Communications, and Control Technology (I4CT) · doi:10.1109/i4ct.2015.7219529 · Cited by 5 · see citing works
Al-Rawi, H. A. A., Yau, K. A., Mohamad, H., Ramli, N., & Hashim, W. (2014) · International Symposium on Communication Systems, Networks and Digital Signal Processing (CSNDSP) · doi:10.1109/csndsp.2014.6923926 · Cited by 5 · see citing works
An Adaptive Multi-Path Routing Algorithm in Cognitive Wireless Mesh Networks
Zhang, G., Ding, C., Gu, J., & Bao, Z. (2011) · International Conference on Wireless Communications, Networking and Mobile Computing · doi:10.1109/wicom.2011.6040284 · Cited by 4 · see citing works
Evaluating machine learning-based routing algorithms on various wireless network topologies
Turlykozhayeva, D., Akhtanov, S., Zhexebay, D., Ussipov, N., Baigaliyeva, A., Wójcik, W., & Boranbayeva, N. (2024) · Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments (WILGA) · doi:10.1117/12.3058676 · Cited by 4 · see citing works
Yau, K. A., Poh, G. S., & Komisarczuk, P. (2013) · International Journal of Ad Hoc and Ubiquitous Computing · doi:10.1504/ijahuc.2013.052413 · Cited by 3 · see citing works
Zhang, L., Du, R., Hao, Z., Li, S., & Hu, Z. (2025) · International Journal of Communication Systems · doi:10.1002/dac.70039 · Cited by 2 · see citing works
An Adaptive WiFi/WiMAX Networking Platform for Cognitive Vehicular Networks
Niyato, D. T., Hossain, E., & Issariyakul, T. (2011) · Springer eBooks · doi:10.1007/978-1-4419-6172-3_12 · Cited by 2 · see citing works
Routing through Efficient Channel Assignment in Cognitive Radio Networks
Saleem, Y., Salim, F., Rehmani, M. H., & Rashid, B. (2015) · Advances in wireless technologies and telecommunication book series · doi:10.4018/978-1-4666-6571-2.ch011 · Cited by 2 · see citing works
Reliable Multicast Routing Protocol Based on Reinforcement Learning
Ashour, O., Kunz, T., & St‐Hilaire, M. (2023) · IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) · doi:10.1109/pimrc56721.2023.10293864 · Cited by 2 · see citing works
Optimization and Learning in Energy Efficient Cognitive Radio System
Zheng, K. (2012) · Cited by 1 · see citing works
Two-dimensional distributed spectrum reusing in cognitive radio network: Based on game theory
Fan, C., Li, B., Zhao, C., & Nallanathan, A. (2017) · International Conference on Communications · doi:10.1109/icc.2017.7996999 · Cited by 1 · see citing works
Routing in Distributed Cognitive Radio Networks
Al-RawiHasan, A., & Alvin, Y.-L. (2013) · Wireless Personal Communications · Cited by 1 · see citing works
Yau, K. A., Komisarczuk, P., Poh, G. S., & Martin, K. M. (2011) · International Conference on Broadband Communications, Information Technology & Biomedical Applications · doi:10.1109/ib2com.2011.6217907 · Cited by 1 · see citing works
MAC protocol design and resource management for distributed cognitive radio networks
Jha, S. C. (2013) · cIRcle (University of British Columbia) · doi:10.14288/1.0071927 · Cited by 1 · see citing works
Quantum reinforcement learning for dynamic spectrum access in cognitive radio networks
Nuuman, S. (2016) · Cited by 1 · see citing works
Multi-hop communication in cognitive radio networks: An experimental evaluation.
Syed, A. R. (2016) · Sunway Institutional Repository (Sunway University)
Turlykozhayeva, D., Akhtanov, S., Zhexebay, D., Ussipov, N., Boranbayeva, N., & Sawicki, D. (2025) · Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments (WILGA) · doi:10.1117/12.3100258
- Authors
- Xia, B., Wahab, M. H., Yang, Y., Fan, Z., & Sooriyabandara, M.
- Affiliations
- University College London (GB)
- ORCID
- Venue
- 2009 4th International Conference on Cognitive Radio Oriented Wireless Networks and Communications
- Publisher
- IEEE
- Published
- 2009-06
- Pages
- 1-5
- OpenAlex
- W2103222798
- Cited by
- 66
- References
- 9
- Citations by year
- 2025: 2 · 2024: 2 · 2023: 3 · 2022: 1 · 2021: 2 · 2020: 2 · 2019: 6 · 2018: 3 · 2017: 4 · 2016: 6 · 2015: 6 · 2014: 6 · 2013: 7 · 2012: 8
- FWCI
- 4.31
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Cognitive Radio Oriented Wireless Networks and Communications (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Cognitive Radio Networks and Spectrum Sensing
- Physical Sciences › Computer Science › Computer Networks and Communications › Mobile Ad Hoc Networks
- Physical Sciences › Computer Science › Computer Networks and Communications › Wireless Networks and Protocols
- Keywords
- cognitive radio networks, multi-hop networks, reinforcement learning, Q-learning, adaptive routing, dynamic spectrum access
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
