The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Cooperative Adaptive Cruise Control: A Reinforcement Learning Approach
Open accessAbstract
Recently, improvements in sensing, communicating, and computing technologies have led to the development of driver-assistance systems (DASs). Such systems aim at helping drivers by either providing a warning to reduce crashes or doing some of the control tasks to relieve a driver from repetitive and boring tasks. Thus, for example, adaptive cruise control (ACC) aims at relieving a driver from manually adjusting his/her speed to maintain a constant speed or a safe distance from the vehicle in front of him/her. Currently, ACC can be improved through vehicle-to-vehicle communication, where the current speed and acceleration of a vehicle can be transmitted to the following vehicles by intervehicle communication. This way, vehicle-to-vehicle communication with ACC can be combined in one single system called cooperative adaptive cruise control (CACC). This paper investigates CACC by proposing a novel approach for the design of autonomous vehicle controllers based on modern machine-learning techniques. More specifically, this paper shows how a reinforcement-learning approach can be used to develop controllers for the secure longitudinal following of a front vehicle. This approach uses function approximation techniques along with gradient-descent learning algorithms as a means of directly modifying a control policy to optimize its performance. The experimental results, through simulation, show that this design approach can result in efficient behavior for CACC.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883
Dynamic Noncooperative Game Theory
(1982) · Mathematics in Science and Engineering · doi:10.1016/s0076-5392(08)x6169-7 · Cited by 4,057
The Handbook of brain theory and neural networks
(1996) · Choice Reviews Online · doi:10.5860/choice.33-4484 · Cited by 3,870
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping
Ng, A. Y., Harada, D., & Russell, S. (1999) · Cited by 1,744
Reinforcement learning and adaptive dynamic programming for feedback control
Lewis, F. L., & Vrabie, D. (2009) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2009.933854 · Cited by 1,558
The Impact of Cooperative Adaptive Cruise Control on Traffic-Flow Characteristics
van Arem, B., van Driel, C. J. G., & Visser, R. (2006) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2006.884615 · Cited by 1,490
Smart cars on smart roads: problems of control
Varaiya, P. (1993) · IEEE Transactions on Automatic Control · doi:10.1109/9.250509 · Cited by 1,383
Reinforcement learning of motor skills with policy gradients
Peters, J., & Schaal, S. (2008) · Neural Networks · doi:10.1016/j.neunet.2008.02.003 · Cited by 831
Kiencke, U. (2003) · Encyclopedia of Physical Science and Technology · doi:10.1016/b0-12-227410-5/00048-x · Cited by 663
Policy gradient reinforcement learning for fast quadrupedal locomotion
Kohl, N., & Stone, P. (2004) · IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004 · doi:10.1109/robot.2004.1307456 · Cited by 564
Infinite-Horizon Policy-Gradient Estimation
Baxter, J., & Bartlett, P. L. (2001) · Cited by 525
Policy Gradient Methods for Robotics
Peters, J., & Schaal, S. (2006) · 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2006.282564 · Cited by 498
Friend-or-Foe Q-learning in General-Sum Games
Littman, M. L. (2001) · International Conference on Machine Learning · Cited by 427
The Communicative Multiagent Team Decision Problem: Analyzing Teamwork Theories and Models
Pynadath, D. V., & Tambe, M. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1024 · Cited by 376
A comprehensive review of the development of adaptive cruise control systems
Xiao, L., & Gao, F. (2010) · Vehicle System Dynamics · doi:10.1080/00423110903365910 · Cited by 360
Infinite-Horizon Policy-Gradient Estimation
Baxter, J., & Bartlett, P. L. (2001) · Journal of Artificial Intelligence Research · doi:10.1613/jair.806 · Cited by 343
(1997) · Transportation Research Part A: Policy and Practice · doi:10.1016/s0965-8564(97)88297-3 · Cited by 299
Intelligent Vehicle Technology and Trends
Bishop, R. (2005) · Cited by 295
Effect of adaptive cruise control systems on traffic flow
Davis, L. C. (2004) · Physical Review E · doi:10.1103/physreve.69.066110 · Cited by 284
Autonomous helicopter control using reinforcement learning policy search methods
Bagnell, J. A., & Schneider, J. G. (2002) · Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164) · doi:10.1109/robot.2001.932842 · Cited by 276
Communication decisions in multi-agent cooperation: model and experiments
Xuan, P., Lesser, V., & Zilberstein, S. (2001) · Proceedings of the fifth international conference on Autonomous agents · doi:10.1145/375735.376469 · Cited by 204
Situation Assessment for Automatic Lane-Change Maneuvers
Schubert, R., Schulze, K., & Wanielik, G. (2010) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2010.2049353 · Cited by 193
The BATmobile: towards a Bayesian automated taxi
Forbes, J., Huang, T. S., Kanazawa, K., & Russell, S. (1995) · Cited by 170
Advanced Driver Assistance Systems from Autonomous to Cooperative Approach
Piao, J., & McDonald, M. (2008) · Transport Reviews · doi:10.1080/01441640801987825 · Cited by 169
Experiments with Infinite-Horizon, Policy-Gradient Estimation
Baxter, J., Bartlett, P. L., & Weaver, L. (2001) · Journal of Artificial Intelligence Research · doi:10.1613/jair.807 · Cited by 137
Exploring the solution space of beaconing in VANETs
van Eenennaam, M., Wolterink, W. K., Karagiannis, G., & Heijenk, G. (2009) · IEEE Vehicular Networking Conference (VNC) · doi:10.1109/vnc.2009.5416370 · Cited by 129
A collaborative driving system based on multiagent modelling and simulations
Hallé, S., & Chaib-draa, B. (2005) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2005.07.004 · Cited by 126
Self-citationKok, J. R., & Vlassis, N. A. (2004) · International Conference on Machine Learning · doi:10.1145/1015330.1015410 · Cited by 124
Human-Centered Design of an Acc-With-Braking and Forward-Crash-Warning System
Fancher, P. S., Bareket, Z., & Ervin, R. D. (2001) · Vehicle System Dynamics · doi:10.1076/vesd.36.2.203.3557 · Cited by 107
Neural Network Vision for Robot Driving
Pomerleau, D. (1997) · Springer eBooks · doi:10.1007/978-1-4615-6325-9_4 · Cited by 103
Game Theoretic Control for Robot Teams
Emery, R., Gordon, G. J., Schneider, J., & Thrun, S. (2006) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2005.1570273 · Cited by 85
Reinforcement Learning in POMDPs with Function Approximation
Kimura, H., Miyazaki, K., & Kobayashi, S. (1997) · International Conference on Machine Learning · Cited by 71
A new reinforcement learning vehicle control architecture for vision-based road following
Oh, S., Lee, J., & Choi, D.-H. (2000) · IEEE Transactions on Vehicular Technology · doi:10.1109/25.845116 · Cited by 66
Reinforcement Learning by Stochastic Hill Climbing on Discounted Reward
Kimura, H., Yamamura, M., & Kobayashi, S. (1995) · Elsevier eBooks · doi:10.1016/b978-1-55860-377-6.50044-x · Cited by 62
Radar and vision data fusion for hybrid adaptive cruise control on highways
Hofmann, U., Rieder, A., & Dickmanns, E. D. (2003) · Machine Vision and Applications · doi:10.1007/s00138-002-0093-y · Cited by 60
Towards On-the-Road Implementation of Cooperative Adaptive Cruise Control
Naus, G. G., Vugts, R. R., Ploeg, J., et al. (2009) · TU/e Research Portal · Cited by 49
An Introduction to Demo 2000: The Cooperative Driving Scenario
Tsugawa, S. (2000) · IEEE Intelligent Systems · doi:10.1109/mis.2000.10015 · Cited by 42
Bonnet, C., & Fritz, H. (2000) · PROCEEDINGS OF THE 7TH WORLD CONGRESS ON INTELLIGENT SYSTEMS · Cited by 28
Reinforcement learning for autonomous vehicles
Forbes, J. R. N., & Russell, S. (2002) · Cited by 27
Distributed reinforcement learning for a traffic engineering application
Pendrith, M. D. (2000) · Adaptive Agents and Multi-Agents Systems · doi:10.1145/336595.337554 · Cited by 23
Graphical Models in Local, Asymmetric Multi-Agent Markov Decision Processes
Dolgov, D., & Durfee, E. H. (2004) · doi:10.1109/aamas.2004.133 · Cited by 23
Road-following with continuous learning
Yu, G., & Sethi, I. K. (2002) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.1995.528317 · Cited by 19
AUTOPIA architecture for automatic driving and maneuvering
Naranjo, J. E., González, C., de Pedro, T., Garcia, R. A., Alonso, J., & Sotelo, M. Á. (2006) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2006.1707389 · Cited by 17
Learning Agents for Collaborative Driving
Desjardins, C., Laumônier, J., & Chaib-draa, B. (2009) · Advances in mechatronics and mechanical engineering (AMME) book series · doi:10.4018/978-1-60566-226-8.ch011 · Cited by 11
Self-citationArchitecture and Design of a Multi-Layered Cooperative Adaptive Cruise Control System
Desjardins, C., Grégoire, P.-L., Laumônier, J., & Chaib-draa, B. (2007) · SAE technical papers on CD-ROM/SAE technical paper series · doi:10.4271/2007-01-1315 · Cited by 2
Self-citation
Cited by
A Multiagent Approach to Autonomous Intersection Management
Dresner, K., & Stone, P. (2008) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2502 · Cited by 1,304
Enabling Massive IoT Toward 6G: A Comprehensive Survey
Guo, F., Yu, F. R., Zhang, H., Li, X., Ji, H., & Leung, V. C. M. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3063686 · Cited by 701
Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles
Aradi, S. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3024655 · Cited by 639
Dey, K. C., Yan, L., Wang, X., Wang, Y., Shen, H., Chowdhury, M., Yu, L., Qiu, C., & Soundararaj, V. (2016) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2015.2483063 · Cited by 554
A comprehensive review of the development of adaptive cruise control systems
Xiao, L., & Gao, F. (2010) · Vehicle System Dynamics · doi:10.1080/00423110903365910 · Cited by 360
Metadata
- Authors
- Desjardins, C., & Chaib-draa, B.
- Venue
- IEEE Transactions on Intelligent Transportation Systems
- Published
- 2011-12
- Volume
- 12
- Issue
- 4
- Pages
- 1248-1260
- ISSN
- 1524-9050 (ISSN-L)
- 1558-0016
- OpenAlex
- W2129660761
- Cited by
- 344
- References
- 46
- FWCI
- 16.34
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.