A Deep Reinforcement Learning Network for Traffic Light Cycle Control
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Existing inefficient traffic light cycle control causes numerous problems, such as long delay and waste of energy. To improve efficiency, taking real-time traffic information as an input and dynamically adjusting the traffic light duration accordingly is a must. Existing works either split the traffic signal into equal duration or only leverage limited traffic information. In this paper, we study how to decide the traffic signal duration based on the collected data from different sensors. We propose a deep reinforcement learning model to control the traffic light cycle. In the model, we quantify the complex traffic scenario as states by collecting traffic data and dividing the whole intersection into small grids. The duration changes of a traffic light are the actions, which are modeled as a high-dimension Markov decision process. The reward is the cumulative waiting time difference between two cycles. To solve the model, a convolutional neural network is employed to map states to rewards. The proposed model incorporates multiple optimization elements to improve the performance, such as dueling network, target network, double Q-learning network, and prioritized experience replay. We evaluate our model via simulation on a Simulation of Urban MObility simulator. Simulation results show the efficiency of our model in controlling traffic lights.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
A tutorial survey on vehicular ad hoc networks
Hartenstein, H., & Laberteaux, K. P. (2008) · IEEE Communications Magazine · doi:10.1109/mcom.2008.4539481 · Cited by 1,619
Fadlullah, Z. M., Tang, F., Mao, B., Kato, N., Akashi, O., Inoue, T., & Mizutani, K. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2707140 · Cited by 810 · cited 1×
“unsupervised learning and reinforcement learning, which have been applied in network traffic control, such as traffic prediction and routing [21].”
BackgroundTraffic signal timing via deep reinforcement learning
Li, L., Lv, Y., & Wang, F.-Y. (2016) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2016.7508798 · Cited by 609
Reinforcement learning-based multi-agent system for network traffic signal control
Arel, I., Liu, C., Urbanik, T., & Kohls, A. G. (2010) · IET Intelligent Transport Systems · doi:10.1049/iet-its.2009.0070 · Cited by 592
Traffic light control using deep policy‐gradient and value‐function‐based reinforcement learning
Mousavi, S. S., Schukat, M., & Howley, E. (2017) · IET Intelligent Transport Systems · doi:10.1049/iet-its.2017.0153 · Cited by 354
Tang, F., Mao, B., Fadlullah, Z. M., Kato, N., Akashi, O., Inoue, T., & Mizutani, K. (2018) · IEEE Wireless Communications · doi:10.1109/mwc.2017.1700244 · Cited by 247
Adaptive Traffic Signal Control With Vehicular Ad hoc Networks
Pandit, K., Ghosal, D., Zhang, H. M., & Chuah, C.-N. (2013) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2013.2241460 · Cited by 244
Urban traffic signal control using reinforcement learning agents
Balaji, P. G., German, X., & Srinivasan, D. (2010) · IET Intelligent Transport Systems · doi:10.1049/iet-its.2009.0096 · Cited by 213
El-Tantawy, S., Abdulhai, B., & Abdelgawad, H. (2014) · Journal of Intelligent Transportation Systems · doi:10.1080/15472450.2013.810991 · Cited by 163
Holonic multi-agent system for traffic signals control
Abdoos, M., Mozayani, N., & Bazzan, A. L. C. (2013) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2013.01.007 · Cited by 160
Liu, W., Qin, G., He, Y., & Jiang, F. (2017) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2702388 · Cited by 127
Communication-Based Train Control System Performance Optimization Using Deep Reinforcement Learning
Zhu, L., He, Y., Yu, F. R., Ning, B., Tang, T., & Zhao, N. (2017) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2017.2724060 · Cited by 80
A Convolutional Neural Network for Transportation Mode Detection Based on Smartphone Platform
Liang, X., & Wang, G. (2017) · IEEE International Conference on Mobile Ad-Hoc and Sensor Systems (MASS) · doi:10.1109/mass.2017.81 · Cited by 78
Self-citationThreshold Tuning Using Stochastic Optimization for Graded Signal Control
Prashanth, L. A., & Bhatnagar, S. (2012) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2012.2209904 · Cited by 45
A Distributed Intersection Management Protocol for Safety, Efficiency, and Driver’s Comfort
Liang, X., Yan, T., Lee, J., & Wang, G. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2817459 · Cited by 43
Self-citation
Cited by
Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey
Haydari, A., & Yilmaz, Y. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3008612 · Cited by 690
Deep Learning for Intelligent Transportation Systems: A Survey of Emerging Trends
Veres, M., & Moussa, M. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2929020 · Cited by 394
Multi-Agent Deep Reinforcement Learning for Urban Traffic Light Control in Vehicular Networks
Wu, T., Zhou, P., Liu, K., Yuan, Y., Wang, X., Huang, H., & Wu, D. O. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2997896 · Cited by 307
Boukerche, A., Tao, Y., & Sun, P. (2020) · Computer Networks · doi:10.1016/j.comnet.2020.107484 · Cited by 215
Survey on artificial intelligence (AI) techniques for vehicular ad-hoc networks (VANETs)
Mchergui, A., Moulahi, T., & Zeadally, S. (2022) · Vehicular Communications · doi:10.1016/j.vehcom.2021.100403 · Cited by 165
Metadata
- Authors
- Liang, X., Du, X., Wang, G., & Han, Z.
- Venue
- IEEE Transactions on Vehicular Technology
- Published
- 2019-02
- Volume
- 68
- Issue
- 2
- Pages
- 1243-1253
- ISSN
- 0018-9545 (ISSN-L)
- 1939-9359
- OpenAlex
- W2794842204
- Cited by
- 604
- References
- 21
- FWCI
- 36.91
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.