The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Traffic light control using deep policy‐gradient and value‐function‐based reinforcement learning
Open accessAbstract
Recent advances in combining deep neural network architectures with reinforcement learning (RL) techniques have shown promising potential results in solving complex control problems with high‐dimensional state and action spaces. Inspired by these successes, in this study, the authors built two kinds of RL algorithms: deep policy‐gradient (PG) and value‐function‐based agents which can predict the best possible traffic signal for a traffic intersection. At each time step, these adaptive traffic light control agents receive a snapshot of the current state of a graphical traffic simulator and produce control signals. The PG‐based agent maps its observation directly to the control signal; however, the value‐function‐based agent first estimates values for all legal control signals. The agent then selects the optimal control action with the highest value. Their methods show promising results in a traffic network simulated in the simulation of urban mobility traffic simulator, without suffering from instability issues during the training process.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Introduction to Reinforcement Learning
Sutton, R. S., & Barto, A. G. (1998) · The MIT Press eBooks · Cited by 6,855
(1999) · Cited by 5,147
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
(2012) · elib (German Aerospace Center) · Cited by 1,994
An analysis of temporal-difference learning with function approximation
Tsitsiklis, J. N., & Van Roy, B. (1997) · IEEE Transactions on Automatic Control · doi:10.1109/9.580874 · Cited by 1,194
Reinforcement learning for robots using neural networks
Lin, L.-J. (1992) · Defense Technical Information Center (DTIC) · Cited by 1,042
Recent Advances in Hierarchical Reinforcement Learning
Barto, A. G., & Mahadevan, S. (2003) · Discrete Event Dynamic Systems · doi:10.1023/a:1022140919877 · Cited by 1,024
A tutorial survey of architectures, algorithms, and applications for deep learning
Deng, L. (2014) · APSIPA Transactions on Signal and Information Processing · doi:10.1017/atsip.2013.9 · Cited by 725
Traffic signal timing via deep reinforcement learning
Li, L., Lv, Y., & Wang, F.-Y. (2016) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2016.7508798 · Cited by 609
Reinforcement learning-based multi-agent system for network traffic signal control
Arel, I., Liu, C., Urbanik, T., & Kohls, A. G. (2010) · IET Intelligent Transport Systems · doi:10.1049/iet-its.2009.0070 · Cited by 592
Reinforcement Learning for True Adaptive Traffic Signal Control
Abdulhai, B., Pringle, R., & Karakoulas, G. J. (2003) · Journal of Transportation Engineering · doi:10.1061/(asce)0733-947x(2003)129:3(278 · Cited by 548
El-Tantawy, S., Abdulhai, B., & Abdelgawad, H. (2013) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2013.2255286 · Cited by 458
Optimizing traffic lights in a cellular automaton model for city traffic
Brockfeld, E., Barlovic, R., Schadschneider, A., & Schreckenberg, M. (2001) · Physical Review E · doi:10.1103/physreve.64.056132 · Cited by 404
Deep auto-encoder neural networks in reinforcement learning
Lange, S., & Riedmiller, M. (2010) · The 2010 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596468 · Cited by 329
Reinforcement Learning With Function Approximation for Traffic Signal Control
LA, P., & Bhatnagar, S. (2011) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2010.2091408 · Cited by 321
Recent Advances in Hierarchical Reinforcement Learning
Barto, A. G., & Mahadevan, S. (2003) · Discrete Event Dynamic Systems · doi:10.1023/a:1025696116075 · Cited by 317
A Survey of Traffic Control With Vehicular Communications
Li, L., Wen, D., & Yao, D. (2014) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2013.2277737 · Cited by 304
Deep Reinforcement Learning: An Overview
Mousavi, S. S., Schukat, M., & Howley, E. (2018) · Lecture Notes in Networks and Systems · doi:10.1007/978-3-319-56991-8_32 · Cited by 261
Self-citationModel-Free reinforcement learning with continuous action in practice
Degris, T., Pilarski, P. M., & Sutton, R. S. (2012) · 2012 American Control Conference (ACC) · doi:10.1109/acc.2012.6315022 · Cited by 239
Using a Deep Reinforcement Learning Agent for Traffic Signal Control
Genders, W., & Razavi, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01142 · Cited by 229
Urban traffic signal control using reinforcement learning agents
Balaji, P. G., German, X., & Srinivasan, D. (2010) · IET Intelligent Transport Systems · doi:10.1049/iet-its.2009.0096 · Cited by 213
Khamis, M. A., & Gomaa, W. (2014) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2014.01.007 · Cited by 204
Reinforcement learning algorithms with function approximation: Recent advances and applications
Xu, X., Zuo, L., & Huang, Z. (2014) · Information Sciences · doi:10.1016/j.ins.2013.08.037 · Cited by 183
Gradient Estimation Using Stochastic Computation Graphs
Schulman, J., Heess, N., Weber, T., & Abbeel, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.05254 · Cited by 166
Holonic multi-agent system for traffic signals control
Abdoos, M., Mozayani, N., & Bazzan, A. L. C. (2013) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2013.01.007 · Cited by 160
Wierstra, D., Förster, A., Peters, J., & Schmidhuber, J. (2009) · Logic Journal of IGPL · doi:10.1093/jigpal/jzp049 · Cited by 150
Experiments with Infinite-Horizon, Policy-Gradient Estimation
Baxter, J., Bartlett, P. L., & Weaver, L. (2001) · Journal of Artificial Intelligence Research · doi:10.1613/jair.807 · Cited by 137
Parallel Systems for Traffic Control: A Rethinking
Li, L., & Wen, D. (2015) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2015.2494625 · Cited by 32
A reinforcement learning approach for dynamic selection of virtual machines in cloud data centres
Duggan, M., Kieran, F., Duggan, J., et al. (2016) · International Conference on Innovative Computing Technology · doi:10.1109/intech.2016.7845053 · Cited by 31
Adaptive traffic control system based on Bayesian probability interpretation
Khamis, M. A., Gomaa, W. E., El-Mahdy, A., & Shoukry, A. (2012) · Japan-Egypt Conference on Electronics, Communications and Computers (JEC-ECC) · doi:10.1109/jec-ecc.2012.6186974 · Cited by 19
An Autonomous Network Aware VM Migration Strategy in Cloud Data Centres
Duggan, M., Duggan, J., Howley, E., & Barrett, E. (2016) · International Conference on Cloud and Autonomic Computing (ICCAC) · doi:10.1109/iccac.2016.9 · Cited by 15
Self-citationAutomatic abstraction controller in reinforcement learning agent via automata
Mousavi, S. S., Ghazanfari, B., Mozayani, N., & Jahed‐Motlagh, M. R. (2014) · Applied Soft Computing · doi:10.1016/j.asoc.2014.08.071 · Cited by 12
Self-citationEnhancing Nash Q-learning and Team Q-learning mechanisms by using bottlenecks
Ghazanfari, B., & Mozayani, N. (2014) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/ifs-130945 · Cited by 9
Ghazanfari, B., & Mozayani, N. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.01.030 · Cited by 8
Deep Learning is Competing with Random Forest in Computational Docking
Khamis, M. A., Galal, B., & Gomaa, W. E. (2023) · International Japan-Africa Conference on Electronics, Communications, and Computations (JAC-ECC) · doi:10.1109/jac-ecc61002.2023.10479635 · Cited by 2
Cited by
Nguyen, T. T., Nguyen, N. D., & Nahavandi, S. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2977374 · Cited by 1,306
Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey
Haydari, A., & Yilmaz, Y. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3008612 · Cited by 690
A Deep Reinforcement Learning Network for Traffic Light Cycle Control
Liang, X., Du, X., Wang, G., & Han, Z. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2890726 · Cited by 604
SleepEEGNet: Automated sleep stage scoring with sequence to sequence deep learning approach
Mousavi, S., Afghah, F., & Acharya, U. R. (2019) · PLOS ONE · doi:10.1371/journal.pone.0216456 · Cited by 531
Self-citationAerial imagery pile burn detection using deep learning: The FLAME dataset
Shamsoshoara, A., Afghah, F., Razi, A., Zheng, L., Fulé, P. Z., & Blasch, E. (2021) · Computer Networks · doi:10.1016/j.comnet.2021.108001 · Cited by 406
Metadata
- Authors
- Mousavi, S. S., Schukat, M., & Howley, E.
- Venue
- IET Intelligent Transport Systems
- Published
- 2017-09
- Volume
- 11
- Issue
- 7
- Pages
- 417-423
- ISSN
- 1751-956X (ISSN-L)
- 1751-9578
- OpenAlex
- W2611458796
- Cited by
- 354
- References
- 42
- FWCI
- 19.11
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.