The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Reinforcement Learning With Function Approximation for Traffic Signal Control
Abstract
We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e.g., the work of Abdulhai, on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhaiand Cools, as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of AbdulhaiWe observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Puterman, M. L. (1994) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316887 · Cited by 6,024
Konda, V. R., & Tsitsiklis, J. N. (2003) · SIAM Journal on Control and Optimization · doi:10.1137/s0363012901385691 · Cited by 709
Optimizing networks of traffic signals in real time-the SCOOT method
Robertson, D. I., & Bretherton, R. D. (1991) · IEEE Transactions on Vehicular Technology · doi:10.1109/25.69966 · Cited by 624
Natural actor–critic algorithms
Bhatnagar, S., Sutton, R. S., Ghavamzadeh, M., & Lee, M. (2009) · Automatica · doi:10.1016/j.automatica.2009.07.008 · Cited by 559
Self-citationReinforcement Learning for True Adaptive Traffic Signal Control
Abdulhai, B., Pringle, R., & Karakoulas, G. J. (2003) · Journal of Transportation Engineering · doi:10.1061/(asce)0733-947x(2003)129:3(278 · Cited by 548
Asynchronous stochastic approximation and Q-learning
Tsitsiklis, J. N. (1994) · Machine Learning · doi:10.1007/bf00993306 · Cited by 453
Neural Networks for Real-Time Traffic Signal Control
Srinivasan, D., Choy, M. C., & Cheu, R. L. (2006) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2006.874716 · Cited by 382
Self-Organizing Traffic Lights: A Realistic Simulation
Cools, S.-B., Gershenson, C., & D’Hooghe, B. (2013) · Advanced Information and Knowledge Processing · doi:10.1007/978-1-4471-5113-5_3 · Cited by 236
Cited by
Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control
Chu, T., Wang, J., Codeca, L., & Li, Z. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2901791 · Cited by 1,046
Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey
Haydari, A., & Yilmaz, Y. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3008612 · Cited by 690
Traffic signal timing via deep reinforcement learning
Li, L., Lv, Y., & Wang, F.-Y. (2016) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2016.7508798 · Cited by 609
A review of cooperative multi-agent deep reinforcement learning
Oroojlooy, A., & Hajinezhad, D. (2023) · Applied Intelligence · doi:10.1007/s10489-022-04105-y · Cited by 482
CoLight: Learning Network-level Cooperation for Traffic Signal Control
Wei, H., Xu, N., Zhang, H., Zheng, G., Zang, X., Chen, C., Zhang, W., Zhu, Y., Xu, K., & Li, Z. (2019) · Proceedings of the 28th ACM International Conference on Information and Knowledge Management · doi:10.1145/3357384.3357902 · Cited by 460
Metadata
- Authors
- LA, P., & Bhatnagar, S.
- Venue
- IEEE Transactions on Intelligent Transportation Systems
- Published
- 2011-06
- Volume
- 12
- Issue
- 2
- Pages
- 412-421
- ISSN
- 1524-9050 (ISSN-L)
- 1558-0016
- OpenAlex
- W2125001944
- Cited by
- 321
- References
- 21
- FWCI
- 15.40
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Reinforcement Learning With Function Approximation for Traffic Signal Control. https://app.cochat.ai/papers/10.1109/tits.2010.2091408
- The paper
- LA, P., & Bhatnagar, S. (2011). Reinforcement Learning With Function Approximation for Traffic Signal Control. IEEE Transactions on Intelligent Transportation Systems. https://doi.org/10.1109/tits.2010.2091408
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.