Deep reinforcement learning techniques for vehicular networks: Recent advances and future trends towards 6G
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Abstract Employing machine learning into 6G vehicular networks to support vehicular application services is being widely studied and a hot topic for the latest research works in the literature. This article provides a comprehensive review of research works that integrated reinforcement and deep reinforcement learning algorithms for vehicular networks management with an emphasis on vehicular telecommunications issues. Vehicular networks have become an important research area due to their specific features and applications such as standardization, efficient traffic management, road safety, and infotainment. In such networks, network entities need to make decisions to maximize network performance under uncertainty. To achieve this goal, Reinforcement Learning (RL) can effectively solve decision-making problems. However, the state and action spaces are massive and complex in large-scale wireless networks. Hence, RL may not be able to find the best strategy in a reasonable time. Therefore, Deep Reinforcement Learning (DRL) has been developed to combine RL with Deep Learning (DL) to overcome this issue. In this survey, we first present vehicular networks and give a brief overview of RL and DRL concepts. Then we review RL and especially DRL approaches to address emerging issues in 6G vehicular networks. We finally discuss and highlight some unresolved challenges for further study.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Graph neural networks: A review of methods and applications
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2020) · AI Open · doi:10.1016/j.aiopen.2021.01.001 · Cited by 6,880
Graph Neural Networks: A Review of Methods and Applications
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2018) · arXiv · doi:10.48550/arxiv.1812.08434 · Cited by 6,880
Deep Reinforcement Learning: A Brief Survey
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2743240 · Cited by 4,733
A Brief Survey of Deep Reinforcement Learning
Arulkumaran, K., Deisenroth, M. P., Brundage, M., & Bharath, A. A. (2017) · arXiv · doi:10.48550/arxiv.1708.05866 · Cited by 4,733
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
Dueling Network Architectures for Deep Reinforcement Learning
Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06581 · Cited by 3,300
A Survey of 5G Network: Architecture and Emerging Technologies
Gupta, A., & Jha, R. K. (2015) · IEEE Access · doi:10.1109/access.2015.2461602 · Cited by 2,420
Deep reinforcement learning for autonomous driving: A survey
Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Sallab, A. A. A., Yogamani, S., & Perez, P. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3054625 · Cited by 2,342
Toward 6G Networks: Use Cases and Technologies
Giordani, M., Polese, M., Mezzavilla, M., Rangan, S., & Zorzi, M. (2020) · IEEE Communications Magazine · doi:10.1109/mcom.001.1900411 · Cited by 2,241
Rainbow: Combining Improvements in Deep Reinforcement Learning
Hessel, M., Modayil, J., Van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., & Silver, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11796 · Cited by 1,846
Deep Reinforcement Learning That Matters
Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., & Meger, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11694 · Cited by 1,685
Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control
Chu, T., Wang, J., Codeca, L., & Li, Z. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2901791 · Cited by 1,046
Tariq, F., Khandaker, M., Wong, K.-K., Imran, M., Bennis, M., & Debbah, M. (2019) · arXiv · doi:10.48550/arxiv.1902.06700 · Cited by 1,014
A tutorial on 5G NR V2X communications
Garcia, M. H. C., Molina-Galan, A., Boban, M., Gozalvez, J., Coll-Perales, B., Şahin, T., & Kousaridas, A. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3057017 · Cited by 975
Blockchain-based decentralized trust management in vehicular networks
Yang, Z., Yang, K., Lei, L., Zheng, K., & Leung, V. C. M. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2836144 · Cited by 932
Deep Reinforcement Learning Based Resource Allocation for V2V Communications
Ye, H., Li, G. Y., & Juang, B.-H. F. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2897134 · Cited by 870
Chen, S., Liang, Y.-C., Sun, S., Kang, S., Cheng, W., & Peng, M. (2020) · IEEE Wireless Communications · doi:10.1109/mwc.001.1900333 · Cited by 768
Noor-A-Rahim, M., Liu, Z., Lee, H., Khyam, M. O., He, J., Pesch, D., Moessner, K., Saad, W., & Poor, H. V. (2022) · Proceedings of the IEEE · doi:10.1109/jproc.2022.3173031 · Cited by 737
Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues
Sun, Y., Peng, M., Zhou, Y., Huang, Y., & Mao, S. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2924243 · Cited by 678
Future Intelligent and Secure Vehicular Network Toward 6G: Machine-Learning Approaches
Tang, F., Kawamoto, Y., Kato, N., & Liu, J. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2954595 · Cited by 620
Spectrum Sharing in Vehicular Networks Based on Multi-Agent Reinforcement Learning
Liang, L., Ye, H., & Li, G. Y. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2933962 · Cited by 559
Navigating Occluded Intersections with Autonomous Vehicles Using Deep Reinforcement Learning
Isele, D., Rahimi, R., Cosgun, A., Subramanian, K., & Fujimura, K. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461233 · Cited by 379
Mobility-Aware Edge Caching and Computing in Vehicle Networks: A Deep Reinforcement Learning
Tan, L. T., & Hu, R. Q. (2018) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2867191 · Cited by 356
Deep Learning Empowered Task Offloading for Mobile Edge Computing in Urban Informatics
Zhang, K., Zhu, Y., Leng, S., He, Y., Maharjan, S., & Zhang, Y. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2903191 · Cited by 309
Machine learning for vehicular networks: Recent advances and application examples
Ye, H., Liang, L., Ye Li, G., Kim, J., Lu, L., & Wu, M. (2018) · IEEE Vehicular Technology Magazine · doi:10.1109/mvt.2018.2811185 · Cited by 303
Zhang, X., Peng, M., Yan, S., & Sun, Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2962715 · Cited by 282
Deep Reinforcement Learning for Resource Allocation in V2V Communications
Ye, H., & Li, G. Y. (2018) · 2018 IEEE International Conference on Communications (ICC) · doi:10.1109/icc.2018.8422586 · Cited by 222
Machine learning for next-generation intelligent transportation systems: A survey
Yuan, T., Da Rocha Neto, W., Rothenberg, C. E., Obraczka, K., Barakat, C., & Turletti, T. (2022) · Transactions on Emerging Telecommunications Technologies · doi:10.1002/ett.4427 · Cited by 179
Privacy-preserved task offloading in mobile blockchain with deep reinforcement learning
Nguyen, D. C., Pathirana, P. N., Ding, M., & Seneviratne, A. (2020) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2020.3010967 · Cited by 178
Blockchain-based distributed software-defined vehicular networks: a dueling deep q-learning approach
Zhang, D., Yu, F. R., & Yang, R. (2019) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2019.2944399 · Cited by 96
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2018) · arXiv · doi:10.48550/arxiv.1810.07862 · Cited by 17
Cited by
Machine learning for next-generation intelligent transportation systems: A survey
Yuan, T., Da Rocha Neto, W., Rothenberg, C. E., Obraczka, K., Barakat, C., & Turletti, T. (2022) · Transactions on Emerging Telecommunications Technologies · doi:10.1002/ett.4427 · Cited by 179
“…works include [6-16], which focus on specific aspects of ITS, such as vehicular networks [7], vehicle detection [8-10], safety of vehicular ad-hoc network (VANET) [6, 15], VANET performance optimization [16], autonomous vehicle control [13], and Deep Reinforcement Learning (DRL) for 6G VANETs [14].”
Annu, & Rajalakshmi, P. (2024) · IEEE Open Journal of Vehicular Technology · doi:10.1109/ojvt.2024.3368240 · Cited by 73
Network Slicing-Based Learning Techniques for IoV in 5G and Beyond Networks
Hamdi, W., Ksouri, C., Bulut, H., & Mosbah, M. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3372083 · Cited by 63
Differential Privacy in Cognitive Radio Networks: A Comprehensive Survey
Ul Hassan, M., Rehmani, M. H., Rehan, M., & Chen, J. (2022) · Cognitive Computation · doi:10.1007/s12559-021-09969-9 · Cited by 19
“Vehicular communication is not a new topic and it has been in the scientific community since ages [154].”
Background
Metadata
- Authors
- Mekrache, A., Bradai, A., Moulay, E., & Dawaliby, S.
- Venue
- Vehicular Communications
- Publisher
- Elsevier BV
- Published
- 2022-01
- Volume
- 33
- Pages
- 100398
- ISSN
- 2214-2096 (ISSN-L)
- OpenAlex
- W3195457333
- Cited by
- 110
- References
- 32
- FWCI
- 4.68
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.