The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Reinforcement Learning Based Routing in Networks: Review and Classification of Approaches
Open accessAbstract
Reinforcement learning (RL), which is a class of machine learning, provides a framework by which a system can learn from its previous interactions with its environment to efficiently select its actions in the future. RL has been used in a number of application fields, including game playing, robotics and control, networks, and telecommunications, for building autonomous systems that improve themselves with experience. It is commonly accepted that RL is suitable for solving optimization problems related to distributed systems in general and to routing in networks in particular. RL also has reasonable overhead-in terms of control packets, memory and computation-compared to other optimization techniques used to solve the same problems. Since the mid-1990s, over 60 protocols have been proposed, with major or minor contributions in the field of optimal route selection to convey packets in different types of communication networks under various user QoS requirements. This paper provides a comprehensive review of the literature on the topic. The review is structured in a way that shows how network characteristics and requirements were gradually considered over time. Classification criteria are proposed to present and qualitatively compare existing RL-based routing protocols.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Wireless sensor networks: a survey
Akyildiz, I. F., Su, W., Sankarasubramaniam, Y., & Cayirci, E. (2002) · Computer Networks · doi:10.1016/s1389-1286(01)00302-4 · Cited by 17,418
Ad hoc On-Demand Distance Vector (AODV) Routing
Perkins, C., Belding-Royer, E., & Das, S. (2003) · doi:10.17487/rfc3561 · Cited by 10,686
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
(2012) · Cited by 9,062
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
Software-Defined Networking: A Comprehensive Survey
Kreutz, D., Ramos, F. M. V., Esteves Verissimo, P., Esteve Rothenberg, C., Azodolmolky, S., & Uhlig, S. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2014.2371999 · Cited by 5,050
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
A Brief Survey of Deep Reinforcement Learning
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · arXiv · doi:10.48550/arxiv.1708.05866 · Cited by 4,733
Wireless mesh networks: a survey
Akyildiz, I. F., Wang, X., & Wang, W. (2005) · Computer Networks · doi:10.1016/j.comnet.2004.12.001 · Cited by 4,007
Cooperative communication in wireless networks
Nosratinia, A., Hunter, T. E., & Hedayat, A. (2004) · IEEE Communications Magazine · doi:10.1109/mcom.2004.1341264 · Cited by 2,940
Optical Networks: A Practical Perspective
Ramaswami, R., & Sivarajan, K. N. (1998) · Cited by 2,592
Zhang, L., Afanasyev, A., Burke, J., Jacobson, V., claffy, K., Crowley, P., Papadopoulos, C., Wang, L., & Zhang, B. (2014) · ACM SIGCOMM Computer Communication Review · doi:10.1145/2656877.2656887 · Cited by 2,270
Quality-of-service routing for supporting multimedia applications
Wang, Z., & Crowcroft, J. (1996) · IEEE Journal on Selected Areas in Communications · doi:10.1109/49.536364 · Cited by 1,626
Sohil, F., Sohali, M. U., & Shabbir, J. (2022) · Statistical Theory and Related Fields · doi:10.1080/24754269.2021.1980261 · Cited by 1,624
Flying Ad-Hoc Networks (FANETs): A survey
Bekmezci, İ., Sahingoz, O. K., & Temel, Ş. (2013) · Ad Hoc Networks · doi:10.1016/j.adhoc.2012.12.004 · Cited by 1,510
A survey and comparison of peer-to-peer overlay network schemes
Lua, E. K., Crowcroft, J., Pias, M., Sharma, R., & Lim, S. (2005) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2005.1610546 · Cited by 1,496
A delay-tolerant network architecture for challenged internets
Fall, K. (2003) · Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications - SIGCOMM '03 · doi:10.1145/863956.863960 · Cited by 1,288
Deep Learning for Intelligent Wireless Networks: A Comprehensive Survey
Mao, Q., Hu, F., & Hao, Q. (2018) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2846401 · Cited by 789
Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach
Boyan, J. A., & Littman, M. L. (1994) · Cited by 738
Computational Intelligence in Wireless Sensor Networks: A Survey
Kulkarni, R. V., Forster, A., & Venayagamoorthy, G. K. (2011) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2011.040310.00002 · Cited by 695
(2004) · Wiley eBooks · doi:10.1002/0471656895 · Cited by 587
Machine Learning for Networking: Workflow, Advances and Opportunities
Wang, M., Cui, Y., Wang, X., Xiao, S., & Jiang, J. (2018) · IEEE Network · doi:10.1109/mnet.2017.1700200 · Cited by 448
Hu, T., & Fei, Y. (2010) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2010.28 · Cited by 405
FOR LEAST-SQUARES POLICY ITERATION
Schwab, D. (2016) · Cited by 372
(2009) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420085891 · Cited by 329
Lin, S.-C., Akyildiz, I. F., Wang, P., & Luo, M. (2016) · 2016 IEEE International Conference on Services Computing (SCC) · doi:10.1109/scc.2016.12 · Cited by 255
Ants and reinforcement learning: a case study in routing in dynamic networks
Subramanian, D., Druschel, P., & Chen, J. (1997) · Cited by 227
Machine Learning for Cognitive Network Management
Ayoubi, S., Limam, N., Salahuddin, M. A., Shahriar, N., Boutaba, R., Estrada-Solano, F., & Caicedo, O. M. (2018) · IEEE Communications Magazine · doi:10.1109/mcom.2018.1700560 · Cited by 217
Valadarsky, A., Schapira, M., Shahaf, D., & Tamar, A. (2017) · Proceedings of the 16th ACM Workshop on Hot Topics in Networks · doi:10.1145/3152434.3152441 · Cited by 204
DROM: Optimizing the Routing in Software-Defined Networks With Deep Reinforcement Learning
Yu, C., Lan, J., Guo, Z., & Hu, Y. (2018) · IEEE Access · doi:10.1109/access.2018.2877686 · Cited by 185
Calvet, L., Armas, J. D., Masip, D., & Juan, A. A. (2017) · Open Mathematics · doi:10.1515/math-2017-0029 · Cited by 176
Wu, C., Ohzahata, S., & Kato, T. (2013) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2013.2273945 · Cited by 152
Using Feedback in Collaborative Reinforcement Learning to Adaptively Optimize MANET Routing
Dowling, J., Curran, E., Cunningham, R., & Cahill, V. (2005) · IEEE Transactions on Systems Man and Cybernetics - Part A Systems and Humans · doi:10.1109/tsmca.2005.846390 · Cited by 148
QGeo: Q-Learning-Based Geographic Ad Hoc Routing Protocol for Unmanned Robotic Networks
Jung, W.-S., Yim, J., & Ko, Y. (2017) · IEEE Communications Letters · doi:10.1109/lcomm.2017.2656879 · Cited by 132
Application of reinforcement learning to routing in distributed wireless networks: a review
Al-Rawi, H. A. A., Ng, M. A., & Yau, K. A. (2013) · Artificial Intelligence Review · doi:10.1007/s10462-012-9383-6 · Cited by 107
Reinforcement learning for adaptive routing
Peshkin, L., & Savova, V. (2003) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2002.1007796 · Cited by 106
Adaptive Opportunistic Routing for Wireless Ad Hoc Networks
Bhorkar, A. A., Naghshvar, M., Javidi, T., & Rao, B. D. (2012) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2011.2159844 · Cited by 102
Mobilized ad-hoc networks: a reinforcement learning approach
Chang, Y.-H., Ho, T., & Kaelbling, L. P. (2004) · International Conference on Autonomic Computing · doi:10.1109/icac.2004.1301369 · Cited by 94
Adaptive Routing for Sensor Networks using Reinforcement Learning
Wang, P., & Wang, T. (2006) · The Sixth IEEE International Conference on Computer and Information Technology (CIT'06) · doi:10.1109/cit.2006.34 · Cited by 92
FROMS: Feedback Routing for Optimizing Multiple Sinks in WSN with Reinforcement Learning
Forster, A., & Murphy, A. L. (2007) · 2007 3rd International Conference on Intelligent Sensors, Sensor Networks and Information · doi:10.1109/issnip.2007.4496872 · Cited by 81
Computational Intelligence Inspired Data Delivery for Vehicle-to-Roadside Communications
Wu, C., Yoshinaga, T., Ji, Y., & Zhang, Y. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2871606 · Cited by 81
A reinforcement learning based routing protocol with QoS support for biomedical sensor networks
Liang, X., Balasingham, I., & Byun, S. (2008) · International Symposium on Applied Sciences in Biomedical and Communication Technologies · doi:10.1109/isabel.2008.4712578 · Cited by 80
SDCoR: Software Defined Cognitive Routing for Internet of Vehicles
Wang, C., Zhang, L., Li, Z., & Jiang, C. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2812210 · Cited by 71
Lin, Z., & van der Schaar, M. (2011) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2010.111910.091238 · Cited by 68
Reinforcement learning based spectrum-aware routing in multi-hop cognitive radio networks
Xia, B., Wahab, M. H., Yang, Y., Fan, Z., & Sooriyabandara, M. (2009) · 2009 4th International Conference on Cognitive Radio Oriented Wireless Networks and Communications · doi:10.1109/crowncom.2009.5189189 · Cited by 66
Q-Probabilistic Routing in Wireless Sensor Networks
Arroyo-Valles, R., Alaiz-Rodriguez, R., Guerrero-Curieses, A., & Cid-Sueiro, J. (2007) · 2007 3rd International Conference on Intelligent Sensors, Sensor Networks and Information · doi:10.1109/issnip.2007.4496810 · Cited by 65
CLIQUE: Role-Free Clustering with Q-Learning for Wireless Sensor Networks
Förster, A., & Murphy, A. L. (2009) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2009.43 · Cited by 62
Reinforcement Learning Based Geographic Routing Protocol for UWB Wireless Sensor Network
Dong, S., Agrawal, P., & Sivalingam, K. M. (2007) · Globecom · doi:10.1109/glocom.2007.127 · Cited by 59
ARBR: Adaptive reinforcement-based routing for DTN
Elwhishi, A., Ho, P.-H., Naik, K., & Shihada, B. (2010) · 2010 IEEE 6th International Conference on Wireless and Mobile Computing, Networking and Communications · doi:10.1109/wimob.2010.5645040 · Cited by 49
Energy-Efficient Power Control for Multiple-Relay Cooperative Networks Using $Q$ -Learning
Shams, F., Bacci, G., & Luise, M. (2014) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2014.2370046 · Cited by 49
QGrid: Q-learning based routing protocol for vehicular ad hoc networks
Li, R., Li, F., Li, X., & Wang, Y. (2014) · IEEE International Performance, Computing, and Communications Conference · doi:10.1109/pccc.2014.7017079 · Cited by 44
Förster, A., & Murphy, A. L. (2010) · Ad Hoc Networks · doi:10.1016/j.adhoc.2010.11.006 · Cited by 41
Constrained flooding: a robust and efficient routing framework for wireless sensor networks
Zhang, Y., & Fromherz, M. P. J. (2006) · International Conference on Advanced Information Networking and Applications (AINA) · doi:10.1109/aina.2006.132 · Cited by 32
A multi-agent reinforcement learning based routing protocol for wireless sensor networks
Liang, X., Balasingham, I., & Byun, S. (2008) · International Symposium on Wireless Communication Systems · doi:10.1109/iswcs.2008.4726117 · Cited by 32
Reinforcement Learning for Routing in Ad Hoc Networks
Nurmi, P. (2007) · International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt) · doi:10.1109/wiopt.2007.4480049 · Cited by 31
Routing in VANETs: A fuzzy constraint Q-Learning approach
Wu, C., Ohzahata, S., & Kato, T. (2012) · Globecom · doi:10.1109/glocom.2012.6503112 · Cited by 22
Reinforcement learning based routing in wireless mesh networks
Boushaba, M., Hafid, A. S., Belbekkouche, A., & Michel Gendreau, A. (2013) · Wireless Networks · doi:10.1007/s11276-013-0592-y · Cited by 21
Call admission control and routing in integrated services networks using reinforcement learning
Marbach, P., Mihatsch, O., & Tsitsiklis, J. N. (2002) · Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes · doi:10.1109/cdc.1998.760738 · Cited by 21
Q-learning enhanced gradient based routing for balancing energy consumption in WSNs
Bazyli, D., Spachos, P., & Areibi, S. M. (2016) · IEEE International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD) · doi:10.1109/camad.2016.7790324 · Cited by 16
Jafarzadeh, S. Z., & Yaghmaee, M. H. (2014) · International Conference on Computer and Knowledge Engineering · doi:10.1109/iccke.2014.6993408 · Cited by 15
Yang, J., Zhang, H., Pan, C., & Sun, W. (2013) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2013.6728518 · Cited by 11
Multicriteria reinforcement learning based on a Russian doll method for network routing
Pétrowski, A., Aissanou, F., Benyahia, I., et al. (2010) · IEEE International Conference on Intelligent Systems · doi:10.1109/is.2010.5548378 · Cited by 8
Introduction to multi-agent modified Q-learning routing for computer networks
Rudek, R., Koszałka, L., & Poźniak-Koszałka, I. (2005) · Advanced Industrial Conference on Telecommunications · doi:10.1109/aict.2005.53 · Cited by 7
Learning based gain-aware content dissemination in delay tolerant networks
Hajiaghajani, F., & Biswas, S. K. (2017) · International Conference on Communication Systems & Networks (COMSNETS) · doi:10.1109/comsnets.2017.7945377 · Cited by 7
Saleem, Y., Yau, K. A., Mohamad, H., Ramli, N., & Rehmani, M. H. (2015) · International Conference on Computer, Communications, and Control Technology (I4CT) · doi:10.1109/i4ct.2015.7219529 · Cited by 5
Cited by
(1996) · Institutional Research Information System University of Turin (University of Turin) · doi:10.1111/j · Cited by 5,486
Blockchain and Machine Learning for Communications and Networking Systems
Liu, Y., Yu, F. R., Li, X., Ji, H., & Leung, V. C. M. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2975911 · Cited by 313
Metadata
- Authors
- Mammeri, Z.
- Venue
- IEEE Access
- Published
- 2019
- Volume
- 7
- Pages
- 55916-55950
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2943717074
- Cited by
- 427
- References
- 68
- FWCI
- 17.76
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.