Watch this: Scalable cost-function learning for path planning in urban environments
Open accessAbstract
In this work, we present an approach to learn cost maps for driving in complex urban environments from a large number of demonstrations of human driving behaviour. The learned cost maps are constructed directly from raw sensor measurements, bypassing the effort of manually designing cost maps as well as features. When deploying the cost maps, the trajectories generated not only replicate human-like driving behaviour but are also demonstrably robust against systematic errors in putative robot configuration. To achieve this we deploy a Maximum Entropy based, non-linear IRL framework which uses Fully Convolutional Neural Networks (FCNs) to represent the cost model underlying expert driving behaviour. Using a deep, parametric approach enables us to scale efficiently to large datasets and complex behaviours while being run-time independent of dataset extent during deployment. We demonstrate scalability and performance on an ambitious dataset collected over the course of one year including more than 25k demonstration trajectories extracted from over 120km of driving and 13 different drivers. We evaluate against a carefully designed cost map and, in addition, demonstrate robustness to systematic errors by learning precise cost-maps even in the presence of system calibration perturbations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A solution to the biased simulation problem is to use Imitation Learning methods, such as Inverse Reinforcement Learning (IRL), to learn from human driving demonstrations without needing to explore unsafe actions.
“A solution here is to use Imitation Learning methods, such as Inverse Reinforcement Learning (IRL) [27], to learn from human driving demonstrations without needing to explore unsafe actions.”
From A survey of deep learning techniques for autonomous driving (2020) · cites this paper as [27]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273 · see citing works
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382 · see citing works
A survey of robot learning from demonstration
Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2008.10.024 · Cited by 3,227 · see citing works
Apprenticeship learning via inverse reinforcement learning
Abbeel, P., & Ng, A. Y. (2004) · doi:10.1145/1015330.1015430 · Cited by 2,873 · see citing works
Stanley: The robot that won the DARPA Grand Challenge
Thrun, S., Montemerlo, M., Dahlkamp, H., Stavens, D., Aron, A., Diebel, J., Fong, P., Gale, J., Halpenny, M., Hoffmann, G., et al. (2006) · Journal of Field Robotics · doi:10.1002/rob.20147 · Cited by 2,237 · see citing works
Principles of Robot Motion: Theory, Algorithms, and Implementations
Choset, H., & Latombe, J. (2005) · Medical Entomology and Zoology · Cited by 2,021 · see citing works
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941 · see citing works
Maximum Entropy Inverse Reinforcement Learning
Ziebart, B. D., Maas, A. L., Bagnell, J. A., & Dey, A. K. (2018) · doi:10.1184/r1/6555512 · Cited by 1,868 · see citing works
Autonomous driving in urban environments: Boss and the Urban Challenge
Urmson, C., Anhalt, J., Bagnell, D., Baker, C., Bittner, R., Clark, M. N., Dolan, J., Duggins, D., Galatali, T., Geyer, C., et al. (2008) · Journal of Field Robotics · doi:10.1002/rob.20255 · Cited by 1,555 · see citing works
Kitani, K. M., Ziebart, B. D., Bagnell, J. A., & Hebert, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_15 · Cited by 701 · see citing works
Guided Cost Learning: Deep Inverse Optimal Control via Policy OptimizationPreprint
Finn, C., Levine, S., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.00448 · Cited by 589 · see citing works
Learning driving styles for autonomous vehicles from demonstration
Kuderer, M., Gulati, S., & Burgard, W. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139555 · Cited by 468 · see citing works
Scaling Learning Algorithms toward AI
Bengio, Y., & LeCun, Y. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7496.003.0016 · Cited by 457 · see citing works
Socially compliant mobile robot navigation via inverse reinforcement learning
Kretzschmar, H., Spies, M., Sprunk, C., & Burgard, W. (2016) · The International Journal of Robotics Research · doi:10.1177/0278364915619772 · Cited by 454 · see citing works
Nonlinear Inverse Reinforcement Learning with Gaussian Processes
Levine, S., Popović, Z., & Koltun, V. (2011) · Cited by 276 · see citing works
Learning to search: Functional gradient techniques for imitation learning
Ratliff, N., Silver, D., & Bagnell, J. A. (2009) · Autonomous Robots · doi:10.1007/s10514-009-9121-3 · Cited by 198 · see citing works
DeepDriving: Learning Affordance for Direct Perception in Autonomous DrivingPreprint
Chen, C., Seff, A., Kornhauser, A. L., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.00256 · Cited by 154 · see citing works
Apprenticeship learning for motion planning with application to parking lot navigation
Abbeel, P., Dolgov, D., Ng, A. Y., & Thrun, S. (2008) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2008.4651222 · Cited by 117 · see citing works
Imitation learning for locomotion and manipulation
Ratliff, N., Bagnell, J. A., & Srinivasa, S. S. (2007) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/ichr.2007.4813899 · Cited by 104 · see citing works
Neural Network Based Autonomous Navigation
Pomerleau, D. (1990) · Kluwer international series in engineering and computer science · doi:10.1007/978-1-4613-1533-9_5 · Cited by 81 · see citing works
Deep Inverse Reinforcement Learning.Preprint
Wulfmeier, M., Ondrúška, P., & Posner, I. (2015) · arXiv (Cornell University) · Cited by 35 · see citing works
Self-citationInverse reinforcement learning with locally consistent reward functions
Nguyen, Q. P., Low, K. H., & Jaillet, P. (2015) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 31 · see citing works
PlanIt: A crowdsourcing approach for learning to plan paths from large scale preference feedback
Jain, A., Das, D., Gupta, J. K., & Saxena, A. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139281 · Cited by 27 · see citing works
Churchill, W., & Newman, P. M. (2012) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2012.6338716 · Cited by 20 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- Michael Montemerlo, Jan Becker, Suhrid Bhat, Hendrik Dahlkamp, Dmitri Dolgov, Scott Ettinger, Dirk Haehnel, Tim Hilden, Gabe Hoffmann, Burkhard Huhnke, et al. Junior: The stanford entry in the urban challenge. Journal of field Robotics, 25(9):569-597, 2008.
- Pierre Sermanet, Raia Hadsell Marco Scoffier, Matt Grimes, Jan Ben, Ayse Erkan, Chris Crudele, Urs Muller, and Yann Lecun. A multi- range architecture for collision-free off-road robot navigation. Journal of Field Robotics.
- Steven M LaValle. Planning algorithms. Cambridge university press, 2006.
- Kyriacos Shiarlis, Joao Messias, Maarten van Someren, and Shimon Whiteson. Inverse Reinforcement Learning from Failure. In Robotics: Science an Systems Conference: Workshop on Learning from Demon- stration. 2015.
Cited by
Sorted by citations
A survey of deep learning techniques for autonomous driving
Grigorescu, S., Trasnea, B., Cocias, T., & Macesanu, G. (2020) · Journal of Field Robotics · doi:10.1002/rob.21918 · Cited by 1,809 · see citing works
Interpretable Goal-based Prediction and Planning for Autonomous Driving
Albrecht, S. V., Brewitt, C., Wilhelm, J., Gyevnar, B., Eiras, F., Dobre, M., & Ramamoorthy, S. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9560849 · Cited by 12 · see citing works
- Authors
- Wulfmeier, M., Wang, D. Z., & Posner, I.
- Affiliations
- University of Oxford (GB)
- Venue
- 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
- Publisher
- IEEE
- Published
- 2016-10
- Pages
- 2089-2095
- ISSN
- 2153-0858 (ISSN-L)
- OpenAlex
- W2486052342
- Cited by
- 116
- References
- 28
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 10 · 2023: 13 · 2022: 20 · 2021: 15 · 2020: 15 · 2019: 14 · 2018: 13 · 2017: 5 · 2016: 4
- FWCI
- 6.47
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/M019918/1, EP/I005021/1, EP/K034472/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotic Path Planning Algorithms
- Keywords
- inverse reinforcement learning, path planning, urban driving, maximum entropy, fully convolutional network, autonomous driving, sensor measurements, scalability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
