The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Reinforcement Learning With Function Approximation for Traffic Signal Control
Abstract
We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e.g., the work of Abdulhai, on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhaiand Cools, as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of AbdulhaiWe observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (1994) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316887 · Cited by 6,024
Konda, V. R., & Tsitsiklis, J. N. (2003) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012901385691 · Cited by 709
Optimizing networks of traffic signals in real time-the SCOOT method
Robertson, D. I., & Bretherton, R. D. (1991) · IEEE Transactions on Vehicular Technology · doi:10.1109/25.69966 · Cited by 624
Natural actor–critic algorithms
Bhatnagar, S., Sutton, R. S., Ghavamzadeh, M., & Lee, M. (2009) · Automatica · doi:10.1016/j.automatica.2009.07.008 · Cited by 559
Self-citationReinforcement Learning for True Adaptive Traffic Signal Control
Abdulhai, B., Pringle, R., & Karakoulas, G. J. (2003) · Journal of Transportation Engineering · doi:10.1061/(asce)0733-947x(2003)129:3(278 · Cited by 548
Asynchronous stochastic approximation and Q-learning
Tsitsiklis, J. N. (1994) · Machine Learning · doi:10.1007/bf00993306 · Cited by 453
Neural Networks for Real-Time Traffic Signal Control
Srinivasan, D., Choy, M. C., & Cheu, R. L. (2006) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2006.874716 · Cited by 382
Self-Organizing Traffic Lights: A Realistic Simulation
Cools, S.-B., Gershenson, C., & D’Hooghe, B. (2013) · Advanced Information and Knowledge Processing · doi:10.1007/978-1-4471-5113-5_3 · Cited by 236
Distributed Geometric Fuzzy Multiagent Urban Traffic Signal Control
Gokulan, B. P., & Srinivasan, D. (2010) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2010.2050688 · Cited by 193
Medina, J., Galan, M., & Royo, E. R. (2009) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2009.2034383 · Cited by 158
Traffic-responsive signal timing for system-wide traffic control
Spall, J. C., & Chin, D. C. (1997) · Transportation Research Part C Emerging Technologies · doi:10.1016/s0968-090x(97)00012-0 · Cited by 154
Simulation and optimization of traffic in a city
Wiering, M., Vreeken, J., van Veenen, J., & Koopman, A. (2004) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2004.1336426 · Cited by 133
Q-Learning with Linear Function Approximation
Melo, F. S., & Ribeiro, I. (2007) · Springer eBooks · doi:10.1007/978-3-540-72927-3_23 · Cited by 125
Driver Behavior Analysis During ACC Activation and Deactivation in a Real Traffic Environment
Pauwelussen, J. J. A., & Feenstra, P. J. (2010) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2010.2043099 · Cited by 77
Self-Organizing Traffic Lights: A Realistic Simulation
Cools, S.-B., Gershenson, C., & D’Hooghe, B. (2007) · Advanced information and knowledge processing · doi:10.1007/978-1-84628-982-8_3 · Cited by 76
de Oliveira, D., Bazzan, A. L. C., da Silva, B. C., Basso, E. W., Rossetti, R. J. F., Oliveira, E., da Silva, R., Lamb, L. C., & Nunes, L. (2006) · CEUR Workshop Proceedings · Cited by 60
Adaptive Dynamic Programming for Multi-intersections Traffic Signal Intelligent Control
Li, T., Zhao, D., & Yi, J. (2008) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2008.4732718 · Cited by 47
Evaluation of system-wide traffic signal control using stochastic optimization and neural networks
Chin, D. C., Spall, J. C., & Smith, R. H. (1999) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.1999.786341 · Cited by 28
Evaluation of an adaptive traffic control technique with underlying system changes
Smith, R. H., & Chin, D. C. (1995) · Winter Simulation Conference · doi:10.1145/224401.224783 · Cited by 10
Cited by
Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control
Chu, T., Wang, J., Codeca, L., & Li, Z. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2901791 · Cited by 1,046
Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey
Haydari, A., & Yilmaz, Y. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3008612 · Cited by 690
Traffic signal timing via deep reinforcement learning
Li, L., Lv, Y., & Wang, F.-Y. (2016) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2016.7508798 · Cited by 609
A review of cooperative multi-agent deep reinforcement learning
Oroojlooy, A., & Hajinezhad, D. (2023) · Applied Intelligence · doi:10.1007/s10489-022-04105-y · Cited by 482
CoLight: Learning Network-level Cooperation for Traffic Signal Control
Wei, H., Xu, N., Zhang, H., Zheng, G., Zang, X., Chen, C., Zhang, W., Zhu, Y., Xu, K., & Li, Z. (2019) · Proceedings of the 28th ACM International Conference on Information and Knowledge Management · doi:10.1145/3357384.3357902 · Cited by 460
Metadata
- Authors
- LA, P., & Bhatnagar, S.
- Venue
- IEEE Transactions on Intelligent Transportation Systems
- Published
- 2011-06
- Volume
- 12
- Issue
- 2
- Pages
- 412-421
- ISSN
- 1524-9050 (ISSN-L)
- 1558-0016
- OpenAlex
- W2125001944
- Cited by
- 321
- References
- 21
- FWCI
- 15.40
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.