The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning
Open accessAbstract
Abstract Autonomous vehicles promise to improve traffic safety while, at the same time, increase fuel efficiency and reduce congestion. They represent the main trend in future intelligent transportation systems. This paper concentrates on the planning problem of autonomous vehicles in traffic. We model the interaction between the autonomous vehicle and the environment as a stochastic Markov decision process (MDP) and consider the driving style of an expert driver as the target to be learned. The road geometry is taken into consideration in the MDP model in order to incorporate more diverse driving styles. The desired, expert-like driving behavior of the autonomous vehicle is obtained as follows: First, we design the reward function of the corresponding MDP and determine the optimal driving strategy for the autonomous vehicle using reinforcement learning techniques. Second, we collect a number of demonstrations from an expert driver and learn the optimal driving strategy based on data using inverse reinforcement learning. The unknown reward function of the expert driver is approximated using a deep neural-network (DNN). We clarify and validate the application of the maximum entropy principle (MEP) to learn the DNN reward function, and provide the necessary derivations for using the maximum entropy principle to learn a parameterized feature (reward) function. Simulated results demonstrate the desired driving behaviors of an autonomous vehicle using both the reinforcement learning and inverse reinforcement learning techniques.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
An Introduction to Support Vector Machines and Other Kernel-based Learning Methods
Cristianini, N., & Shawe-Taylor, J. (2000) · doi:10.1017/cbo9780511801389 · Cited by 13,855
Information theory and statistical mechanics
Jaynes, E. T. (1957) · Physical Review · doi:10.1103/physrev.106.620 · Cited by 12,931
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Playing Atari with Deep Reinforcement Learning
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013) · arXiv · doi:10.48550/arxiv.1312.5602 · Cited by 9,471
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Multilayer feedforward networks are universal approximators
HornikK., StinchcombeM., & WhiteH. (1989) · Neural Networks · doi:10.5555/70405.70408 · Cited by 9,081
Approximation capabilities of multilayer feedforward networks
Hornik, K. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90009-t · Cited by 6,091
Watkins, C. (1989) · OpenGrey (Institut de l'Information Scientifique et Technique) · doi:10.17863/cam.132164 · Cited by 5,297
Sampling-based algorithms for optimal motion planning
Karaman, S., & Frazzoli, E. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364911406761 · Cited by 5,275
The unscented Kalman filter for nonlinear estimation
Wan, E. A., & Van Der Merwe, R. (2002) · Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373) · doi:10.1109/asspcc.2000.882463 · Cited by 4,001
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1023/a:1022676722315 · Cited by 3,907
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
An Introduction to Support Vector Machines and Other Kernel‐based Learning Methods
(2001) · Kybernetes · doi:10.1108/k.2001.30.1.103.6 · Cited by 3,308
Apprenticeship learning via inverse reinforcement learning
Abbeel, P., & Ng, A. Y. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015430 · Cited by 2,769
Markov games as a framework for multi-agent reinforcement learning
Littman, M. L. (1994) · Machine Learning Proceedings 1994 · doi:10.1016/b978-1-55860-335-6.50027-1 · Cited by 2,614
A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles
Paden, B., Cap, M., Yong, S. Z., Yershov, D., & Frazzoli, E. (2016) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2016.2578706 · Cited by 2,585
Bellman, R. (1957) · Indiana University Mathematics Journal · doi:10.1512/iumj.1957.6.56038 · Cited by 2,408
Maximum Entropy Inverse Reinforcement Learning
Ziebart, B. D., Maas, A. L., Bagnell, J. A., & Dey, A. K. (2018) · doi:10.1184/r1/6555512 · Cited by 1,869
(2000) · doi:10.2460/ajvr.67.2.323 · Cited by 1,621
Predictive Active Steering Control for Autonomous Vehicle Systems
Falcone, P., Borrelli, F., Asgari, J., Tseng, H. E., & Hrovat, D. (2007) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2007.894653 · Cited by 1,408
Video-Based Lane Estimation and Tracking for Driver Assistance: Survey, System, and Evaluation
McCall, J. C., & Trivedi, M. M. (2006) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2006.869595 · Cited by 1,028
Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues
Odentrantz, J. (2000) · Technometrics · doi:10.1080/00401706.2000.10485734 · Cited by 973
Nash Q-Learning for General-Sum Stochastic Games
Hu, J., Wellman, M. P., & Boutilier, C. (2003) · Cited by 923
Residual Algorithms: Reinforcement Learning with Function Approximation
Baird, L. (1995) · Machine Learning Proceedings 1995 · doi:10.1016/b978-1-55860-377-6.50013-x · Cited by 918
Real-Time Motion Planning With Applications to Autonomous Urban Driving
Kuwata, Y., Karaman, S., Teo, J., Frazzoli, E., How, J. P., & Fiore, G. (2009) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2008.2012116 · Cited by 889
Katrakazas, C., Quddus, M., Chen, W.-H., & Deka, L. (2015) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2015.09.011 · Cited by 800
Vision and navigation for the Carnegie-Mellon Navlab
Thorpe, C., Hebert, M. H., Kanade, T., & Shafer, S. A. (1988) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.3900 · Cited by 776
Kitani, K. M., Ziebart, B. D., Bagnell, J. A., & Hebert, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_15 · Cited by 701
Perception, Planning, Control, and Coordination for Autonomous Vehicles
Pendleton, S., Andersen, H., Du, X., Shen, X., Meghjani, M., Eng, Y., Rus, D., & Ang, M. (2017) · Machines · doi:10.3390/machines5010006 · Cited by 657
Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
Finn, C., Levine, S., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.00448 · Cited by 549
Theory of the backpropagation neural network
Hecht-Nielsen, R. (1988) · Neural Networks · doi:10.1016/0893-6080(88)90469-8 · Cited by 472
Friend-or-Foe Q-learning in General-Sum Games
Littman, M. L. (2001) · International Conference on Machine Learning · Cited by 427
Overview of Supervised Learning
Hastie, T., Friedman, J. H., & Tibshirani, R. J. (2001) · Springer series in statistics · doi:10.1007/978-0-387-21606-5_2 · Cited by 333
Di Cairano, S., Tseng, H. E., Bernardini, D., & Bemporad, A. (2013) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2012.2198886 · Cited by 326
Gaussian Process Regression Analysis for Functional Data
Shi, J. Q., & Choi, T. (2011) · Chapman & Hall/CRC eBooks · doi:10.1201/b11038 · Cited by 322
Aerodynamic parameter estimation from flight data applying extended and unscented Kalman filter
Chowdhary, G., & Jategaonkar, R. (2010) · Aerospace Science and Technology · doi:10.1016/j.ast.2009.10.003 · Cited by 282
Nonlinear Inverse Reinforcement Learning with Gaussian Processes
Levine, S., Popović, Z., & Koltun, V. (2011) · Cited by 280
Maximum Entropy Deep Inverse Reinforcement Learning
Wulfmeier, M., Ondrúška, P., & Posner, I. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.04888 · Cited by 272
Interdisciplinary Computing in Java Programming
Wang, S.-C. (2003) · Springer eBooks · doi:10.1007/978-1-4615-0377-4 · Cited by 260
Path Planning Based on Bézier Curve for Autonomous Ground Vehicles
Choi, J.-W., Curry, R., & Elkaim, G. (2008) · Advances in Electrical and Electronics Engineering - IAENG Special Edition of the World Congress on Engineering and Computer Science 2008 · doi:10.1109/wcecs.2008.27 · Cited by 228
Autonomous reinforcement learning on raw visual input data in a real world application
Lange, S., Riedmiller, M., & Voigtlander, A. (2012) · The 2012 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2012.6252823 · Cited by 202
Shim, T., Adireddy, G., & Yuan, H. (2012) · Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering · doi:10.1177/0954407011430275 · Cited by 178
Probabilistic online POMDP decision making for lane changes in fully automated driving
Ulbrich, S., & Maurer, M. (2013) · 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013) · doi:10.1109/itsc.2013.6728533 · Cited by 168
Continuous Inverse Optimal Control with Locally Optimal Examples
Levine, S., & Koltun, V. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.4617 · Cited by 158
The relative frequency of unsafe driving acts in serious traffic crashes
Hendricks, D. L. ;. F. (2001) · PsycEXTRA Dataset · doi:10.1037/e440112008-001 · Cited by 103
Integrated chassis control system to enhance vehicle stability
Ghoneim, Y. A., Lin, W. C., Sidlosky, D. M., et al. (2000) · International Journal of Vehicle Design · doi:10.1504/ijvd.2000.001887 · Cited by 95
Fuzzy Q-learning: a new approach for fuzzy dynamic programming
Berenji, H. R. (1994) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzzy.1994.343737 · Cited by 89
Probabilistic MDP-behavior planning for cars
Brechtel, S., Gindele, T., & Dillmann, R. (2011) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2011.6082928 · Cited by 86
PANS: a portable navigation platform
Jochem, T. M., Pomerleau, D., Kumar, B. V. K. V., & Armstrong, J. (2002) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.1995.528266 · Cited by 79
Li, N. I., Oyler, D. W., Zhang, M.-X., Yildiz, Y., Girard, A., & Kolmanovsky, I. (2016) · IEEE Conference on Decision and Control · doi:10.1109/cdc.2016.7798354 · Cited by 55
Human Behavior Modeling with Maximum Entropy Inverse Optimal Control
Ziebart, B. D., Maas, A. L., Bagnell, J. A., & Dey, A. K. (2009) · Cited by 48
You, C., Lǚ, J., & Tsiotras, P. (2017) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2017.2727547 · Cited by 48
Self-citationMaximum Entropy Distribution Estimation with Generalized Regularization
Dudı́k, M., & Schapire, R. E. (2006) · Lecture notes in computer science · doi:10.1007/11776420_12 · Cited by 47
Long-term Planning by Short-term PredictionPreprint
Shalev‐Shwartz, S., Ben-Zrihem, N., Cohen, A., & Shashua, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.01580 · Cited by 41
A decision-making method for autonomous vehicles based on simulation and reinforcement learning
Zheng, R., Liu, C., & Guo, Q. (2013) · International Conference on Machine Learning and Cybernetics · doi:10.1109/icmlc.2013.6890495 · Cited by 32
LTV-MPC based path planning of an autonomous vehicle via convex optimization
Mousavi, M. A., Heshmati, Z., & Moshiri, B. (2013) · Iranian Conference on Electrical Engineering · doi:10.1109/iraniancee.2013.6599610 · Cited by 31
Strategic Decision-Making Process in Advanced Driver Assistance Systems
Ardelt, M., Waldmann, P., Homm, F., & Kaempchen, N. (2010) · IFAC Proceedings Volumes · doi:10.3182/20100712-3-de-2013.00006 · Cited by 23
A Comparison of learning algorithms on the Arcade Learning EnvironmentPreprint
Defazio, A., & Graepel, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.8620 · Cited by 22
Maximum Entropy Semi-Supervised Inverse Reinforcement LearningPreprint
Audiffren, J., Valko, M., Lazaric, A., & Ghavamzadeh, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.20074 · Cited by 19
You, C., & Tsiotras, P. (2016) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.1109/acc.2016.7526607 · Cited by 15
Self-citation
Cited by
Digital twin: values, challenges and enablers from a modeling perspective
Rasheed, A., San, O., & Kvamsdal, T. (2020) · IEEE Access · doi:10.1109/access.2020.2970143 · Cited by 1,952
A Survey on Trajectory-Prediction Methods for Autonomous Driving
Huang, Y., Du, J., Yang, Z., Zhou, Z., Zhang, L., & Chen, H. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3167103 · Cited by 677
Artificial intelligence applications in the development of autonomous vehicles: a survey
Ma, Y., Wang, Z., Yang, H., & Yang, L. (2020) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2020.1003021 · Cited by 639
Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles
Aradi, S. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3024655 · Cited by 639
Gupta, A., Anpalagan, A., Guan, L., & Khwaja, A. S. (2021) · Array · doi:10.1016/j.array.2021.100057 · Cited by 565
Metadata
- Authors
- You, C., Lu, J., Filev, D., & Tsiotras, P.
- Venue
- Robotics and Autonomous Systems
- Publisher
- Elsevier BV
- Published
- 2019-04
- Volume
- 114
- Pages
- 1-18
- ISSN
- 0921-8890 (ISSN-L)
- OpenAlex
- W2909906617
- Cited by
- 290
- References
- 63
- FWCI
- 14.27
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.