The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Graph neural network and reinforcement learning for multi‐agent cooperative control of connected autonomous vehicles
Open accessAbstract
A connected autonomous vehicle (CAV) network can be defined as a set of connected vehicles including CAVs that operate on a specific spatial scope that may be a road network, corridor, or segment. The spatial scope constitutes an environment where traffic information is shared and instructions are issued for controlling the CAVs movements. Within such a spatial scope, high‐level cooperation among CAVs fostered by joint planning and control of their movements can greatly enhance the safety and mobility performance of their operations. Unfortunately, the highly combinatory and volatile nature of CAV networks due to the dynamic number of agents (vehicles) and the fast‐growing joint action space associated with multi‐agent driving tasks pose difficultly in achieving cooperative control. The problem is NP‐hard and cannot be efficiently resolved using rule‐based control techniques. Also, there is a great deal of information in the literature regarding sensing technologies and control logic in CAV operations but relatively little information on the integration of information from collaborative sensing and connectivity sources. Therefore, we present a novel deep reinforcement learning‐based algorithm that combines graphic convolution neural network with deep Q‐network to form an innovative graphic convolution Q network that serves as the information fusion module and decision processor. In this study, the spatial scope we consider for the CAV network is a multi‐lane road corridor. We demonstrate the proposed control algorithm using the application context of freeway lane‐changing at the approaches to an exit ramp. For purposes of comparison, the proposed model is evaluated vis‐à‐vis traditional rule‐based and long short‐term memory‐based fusion models. The results suggest that the proposed model is capable of aggregating information received from sensing and connectivity sources and prescribing efficient operative lane‐change decisions for multiple CAVs, in a manner that enhances safety and mobility. That way, the operational intentions of individual CAVs can be fulfilled even in partially observed and highly dynamic mixed traffic streams. The paper presents experimental evidence to demonstrate that the proposed algorithm can significantly enhance CAV operations. The proposed algorithm can be deployed at roadside units or cloud platforms or other centralized control facilities.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Semi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
(2012) · elib (German Aerospace Center) · Cited by 1,994
Graph Neural Networks for Social Recommendation
Fan, W., Ma, Y., Li, Q., He, Y., Zhao, E., Tang, J., & Yin, D. (2019) · The World Wide Web Conference · doi:10.1145/3308558.3313488 · Cited by 1,984
Traffic Flow Dynamics: Data, Models and Simulation
Treiber, M., & Kesting, A. (2013) · Springer eBooks · doi:10.1007/978-3-642-32460-4 · Cited by 1,351
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
Zhang, K., Yang, Z., & Başar, T. (2021) · Studies in Systems, Decision and Control · doi:10.1007/978-3-030-60990-0_12 · Cited by 1,264
The Complexity of Decentralized Control of Markov Decision Processes
Bernstein, D. S., Givan, R., Immerman, N., & Zilberstein, S. (2002) · Mathematics of Operations Research · doi:10.1287/moor.27.4.819.297 · Cited by 1,192
Deep Reinforcement Learning framework for Autonomous Driving
Sallab, A. E., Abdou, M., Perot, E., & Yogamani, S. (2017) · Electronic Imaging · doi:10.2352/issn.2470-1173.2017.19.avm-023 · Cited by 1,045
Chen, D., Lin, Y., Li, W., Li, P., Zhou, J., & Sun, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5747 · Cited by 1,043
Cooperative Multi-agent Control Using Deep Reinforcement Learning
Gupta, J. K., Egorov, M., & Kochenderfer, M. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-71682-4_5 · Cited by 874
Automatic Pixel‐Level Crack Detection and Measurement Using Fully Convolutional Network
Yang, X., Li, H., Yu, Y., Luo, X., Huang, T., & Yang, X. (2018) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12412 · Cited by 852
A survey and critique of multiagent deep reinforcement learning
Hernandez-Leal, P., Kartal, B., & Taylor, M. E. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09421-1 · Cited by 671
DeepInf: Social Influence Prediction with Deep Learning
Qiu, J., Tang, J., Ma, H., Dong, Y., Wang, K., & Tang, J. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220077 · Cited by 587
A dynamic ensemble learning algorithm for neural networks
Alam, K. M. R., Siddique, N., & Adeli, H. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04359-7 · Cited by 536
Protein interface prediction using graph convolutional networks
Fout, A., Ben‐Hur, A., Anderson, C., Chitsaz, H., & Zhou, W. (2017) · Digital Collections of Colorado (Colorado State University) · doi:10.25675/3.021826 · Cited by 513
A New Neural Dynamic Classification Algorithm
Rafiei, M. H., & Adeli, H. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2682102 · Cited by 501
Li, S., Zhao, X., & Zhou, G. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12433 · Cited by 490
Applications of graph theory in chemistry
Balaban, A. T. (1985) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci00047a033 · Cited by 458
Encoder–decoder network for pixel‐level road crack detection in black‐box images
Bang, S., Park, S., Kim, H., & Kim, H. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12440 · Cited by 423
Planning, Learning and Coordination in Multiagent Decision Processes
Boutilier, C. E. (1996) · Cited by 392
Automated pavement crack detection and segmentation based on two‐step convolutional neural network
Liu, J., Yang, X., Lau, S., Wang, X., Luo, S., Lee, V. C., & Ding, L. (2020) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12622 · Cited by 384
FEMa: a finite element machine for fast learning
Pereira, D. R., Piteri, M. A., Souza, A. N., Papa, J. P., & Adeli, H. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04146-4 · Cited by 378
Dynamic Wavelet Neural Network Model for Traffic Flow Forecasting
Jiang, X., & Adeli, H. (2005) · Journal of Transportation Engineering · doi:10.1061/(asce)0733-947x(2005)131:10(771 · Cited by 364
Enhanced probabilistic neural network with local decision circles: A robust classifier
Ahmadlou, M., & Adeli, H. (2010) · Integrated Computer-Aided Engineering · doi:10.3233/ica-2010-0345 · Cited by 349
Multi-agent reinforcement learning as a rehearsal for decentralized planning
Kraemer, L., & Banerjee, B. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.01.031 · Cited by 343
Enhancements of V2X communication in support of cooperative autonomous driving
Hobert, L., Festag, A., Llatser, I., Altomare, L., Visintainer, F., & Kovacs, A. (2015) · IEEE Communications Magazine · doi:10.1109/mcom.2015.7355568 · Cited by 338
Zhang, A., Wang, K. C. P., Fei, Y., Liu, Y., Chen, C., Yang, G., Li, J. Q., Yang, E., & Qiu, S. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12409 · Cited by 300
Model-free Deep Reinforcement Learning for Urban Autonomous Driving
Chen, J., Yuan, B., & Tomizuka, M. (2019) · 2019 IEEE Intelligent Transportation Systems Conference (ITSC) · doi:10.1109/itsc.2019.8917306 · Cited by 296
Zhou, M., Qu, X., & Jin, S. (2016) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2016.2606492 · Cited by 291
A Novel Machine Learning Model for Estimation of Sale Prices of Real Estate Units
Rafiei, M. H., & Adeli, H. (2016) · Journal of Construction Engineering and Management · doi:10.1061/(asce)co.1943-7862.0001047 · Cited by 287
Mean Field Multi-Agent Reinforcement Learning
Yang, Y., Luo, R., Li, M., Zhou, M. Q., Zhang, W., & Wang, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05438 · Cited by 286
Supervised Deep Restricted Boltzmann Machine for Estimation of Concrete
(2017) · ACI Materials Journal · doi:10.14359/51689560 · Cited by 278
Rafiei, M. H., & Adeli, H. (2018) · Journal of Construction Engineering and Management · doi:10.1061/(asce)co.1943-7862.0001570 · Cited by 269
Spatiotemporal intersection control in a connected and automated vehicle environment
Feng, Y., Yu, C., & Liu, H. X. (2018) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2018.02.001 · Cited by 254
A consensus-based distributed trajectory control in a signal-free intersection
Mirheli, A., Tajalli, M., Hajibabai, L., & Hajbabaie, A. (2019) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2019.01.004 · Cited by 244
Erdmann, J. (2015) · Lecture Notes in Mobility · doi:10.1007/978-3-319-15024-6_7 · Cited by 236
Enhanced probabilistic neural network with local decision circles: A robust classifier
AhmadlouMehran, & AdeliHojjat (2010) · Integrated Computer-Aided Engineering · doi:10.5555/1839615.1839621 · Cited by 236
Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learning
Palanisamy, P. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207663 · Cited by 211
Trajectory optimization of connected and autonomous vehicles at a multilane freeway merging area
Hu, X., & Sun, J. (2019) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2019.02.016 · Cited by 203
Mirheli, A., Hajibabai, L., & Hajbabaie, A. (2018) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2018.04.026 · Cited by 182
Applications of Graph Theory and Network Science to Transit Network Design
Derrible, S., & Kennedy, C. (2011) · Transport Reviews · doi:10.1080/01441647.2010.543709 · Cited by 177
Deep reinforcement learning for long‐term pavement maintenance planning
Yao, L., Dong, Q., Jiang, J., & Ni, F. (2020) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12558 · Cited by 174
Rafiei, M. H., & Adeli, H. (2017) · Soil Dynamics and Earthquake Engineering · doi:10.1016/j.soildyn.2017.05.013 · Cited by 171
Wu, Y., Chen, H., & Zhu, F. (2019) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2019.04.012 · Cited by 160
Cooperative merging control via trajectory optimization in mixed vehicular traffic
Karimi, M., Roncoli, C., Alecsandru, C., & Papageorgiou, M. (2020) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2020.102663 · Cited by 154
Graph Theory and Social Networks: A Technical Comment on Connectedness and Connectivity
Barnes, J. A. (1969) · Sociology · doi:10.1177/003803856900300205 · Cited by 153
Wavelet‐Clustering‐Neural Network Model for Freeway Incident Detection
Ghosh‐Dastidar, S., & Adeli, H. (2003) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/1467-8667.t01-1-00311 · Cited by 147
Xie, Y., Zhang, H., Gartner, N. H., & Arsava, T. (2017) · Journal of Intelligent Transportation Systems · doi:10.1080/15472450.2016.1248288 · Cited by 146
Chen, S., Leng, Y., & Labi, S. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12495 · Cited by 136
Self-citationDong, J., Chen, S., Li, Y., Du, R., Steinfeld, A. M., & Labi, S. (2021) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2021.103192 · Cited by 119
Self-citationDriving in Dense Traffic with Model-Free Reinforcement Learning
Saxena, D. M., Bae, S., Nakhaei, A., Fujimura, K., & Likhachev, M. A. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197132 · Cited by 118
Corridor level cooperative trajectory optimization with connected and automated vehicles
Yu, C., Feng, Y., Liu, H. X., Ma, W., & Yang, X. (2019) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2019.06.002 · Cited by 102
A graph deep learning method for short-term traffic forecasting on large road networks
Zhang, Y., Cheng, T., & Ren, Y. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12450 · Cited by 100
Pavement defect detection with fully convolutional network and an uncertainty framework
Zheng, T., Yuan, D., Gao, J., & Wang, Z. (2020) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12533 · Cited by 85
Dynamic Input for Deep Reinforcement Learning in Autonomous Driving
Huegle, M., Kalweit, G., Mirchevska, B., Werling, M., & Boedecker, J. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968560 · Cited by 73
Multi-agent replicator controller for sustainable vibration control of smart structures
Soto, M. G., & Adeli, H. (2017) · Journal of Vibroengineering · doi:10.21595/jve.2017.18924 · Cited by 67
Luo, X., Li, H., Yu, Y., Zhou, C., & Cao, D. (2020) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12538 · Cited by 60
Vehicle Connectivity and Automation: A Sibling Relationship
Ha, P. Y. J., Chen, S., Du, R., Dong, J., Li, Y., & Labi, S. (2020) · Frontiers in Built Environment · doi:10.3389/fbuil.2020.590036 · Cited by 52
Self-citationTang, K., Cao, Y., Chen, C., Yao, J., Tan, C., & Sun, J. (2020) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12559 · Cited by 49
Modeling the Proactive Driving Behavior of Connected Vehicles: A Cell‐Based Simulation Approach
Zhu, F., & Ukkusuri, S. V. (2017) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12289 · Cited by 46
Maeda, K., Takahashi, S., Ogawa, T., & Haseyama, M. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12451 · Cited by 40
Reinforcement learning‐based bird‐view automated vehicle control to avoid crossing traffic
Wang, Y., Hou, S., & Wang, X. (2020) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12572 · Cited by 40
A convolutional neural-network-based pedestrian counting model for various crowded scenes
Shen, J., Xiong, X., Xue, Z., & Bian, Y. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12454 · Cited by 37
Ha, P. Y. J., Chen, S., Dong, J., Du, R., Li, Y., & Labi, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.05436 · Cited by 33
Self-citationSpatio-weighted information fusion and DRL-based control for connected autonomous vehicles
Dong, J., Chen, S., Li, Y., et al. (2020) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc45102.2020.9294550 · Cited by 29
Self-citationLi, Y., Chen, S., Du, R., Ha, P. Y. J., Dong, J., & Labi, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.05440 · Cited by 21
Self-citationChen, S. (2019) · Purdue e-Pubs (Purdue University System) · doi:10.25394/pgs.8855591 · Cited by 9
Self-citationA smart car model based on autonomous intelligent agents for reducing accidents
Bourbakis, N., Alamaniotis, M., & Tsoukalas, L. H. (2017) · IEEE Transportation Electrification Conference and Expo · doi:10.1109/itec.2017.7993366 · Cited by 5
Chen, T., Wang, Z., Li, L., Li, D., Li, Z., Chang, X., Bian, P., Liang, G., Wang, Q., & Xie, T. (2025) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.ecoop.2025.6
Cited by
Machine Learning Methods for Small Data Challenges in Molecular Science
Dou, B., Zhu, Z., Merkurjev, E., Ke, L., Chen, L., Jiang, J., Zhu, Y., Liu, J., Zhang, B., & Wei, G.-W. (2023) · Chemical Reviews · doi:10.1021/acs.chemrev.3c00189 · Cited by 499
Graph Neural Networks for Intelligent Transportation Systems: A Survey
Rahmani, S., Baghbani, A., Bouguila, N., & Patterson, Z. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3257759 · Cited by 353
The impacts of connected autonomous vehicles on mixed traffic flow: A comprehensive review
Pan, Y., Wu, Y., Xu, L., Xia, C., & Olson, D. L. (2024) · Physica A: Statistical Mechanics and its Applications · doi:10.1016/j.physa.2023.129454 · Cited by 102
Huang, Z., Sheng, Z., Ma, C., & Chen, S. (2024) · Communications in Transportation Research · doi:10.1016/j.commtr.2024.100127 · Cited by 88
Self-citationHua, M., Qi, X., Chen, D., Jiang, K., Liu, Z. E., Sun, H., Zhou, Q., & Xu, H. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3574280 · Cited by 88
“For instance, research endeavors have delved into the utilization of graph-based MARL algorithms [122], such as those explored in [136] and [138], to enable e ffi - cient and adaptive message exchange among agents.”
Metadata
- Authors
- Chen, S., Dong, J., Ha, P. (. J., Li, Y., & Labi, S.
- Venue
- Computer-Aided Civil and Infrastructure Engineering
- Publisher
- Elsevier BV
- Published
- 2021-06-10
- Volume
- 36
- Issue
- 7
- Pages
- 838-857
- ISSN
- 1093-9687 (ISSN-L)
- 1467-8667
- OpenAlex
- W3167586364
- Cited by
- 261
- References
- 71
- FWCI
- 17.75
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.