CoChat
Sign Up
PreprintCited by 33Autonomous Vehicle Technology and SafetyRobotic Path Planning AlgorithmsReinforcement Learning in Robotics

Parting with Misconceptions about Learning-based Vehicle Motion Planning

Open access
Dauner, Daniel, Hallgarten, Marcel, Geiger, Andreas, Chitta, Kashyap·2023·arXiv
DOI 10.48550/arxiv.2306.07962arXiv 2306.07962
Open this paper in CoChat — free accountOpen PDF

Abstract

The release of nuPlan marks a new era in vehicle motion planning research, offering the first large-scale real-world dataset and evaluation schemes requiring both precise short-term planning and long-horizon ego-forecasting. Existing systems struggle to simultaneously meet both requirements. Indeed, we find that these tasks are fundamentally misaligned and should be addressed independently. We further assess the current state of closed-loop planning in the field, revealing the limitations of learning-based methods in complex real-world scenarios and the value of simple rule-based priors such as centerline selection through lane graph search algorithms. More surprisingly, for the open-loop sub-task, we observe that the best results are achieved when using only this centerline as scene context (i.e., ignoring all information regarding the map and other agents). Combining these insights, we propose an extremely simple and efficient planner which outperforms an extensive set of competitors, winning the nuPlan planning challenge 2023.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2306.07962

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 79

  1. F. Codevilla, M. Miiller, A. López, V. Koltun, and A. Dosovitskiy. End-to-end driving via conditional imitation learning. In Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2018.
  2. F. Codevilla, E. Santana, A. M. López, and A. Gaidon. Exploring the limitations of behavior cloning for autonomous driving. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2019.
  3. N. Rhinehart, R. McAllister, K. Kitani, and S. Levine. PRECOG: prediction conditioned on goals in visual multi-agent settings. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2019.
  4. W. Zeng, W. Luo, S. Suo, A. Sadat, B. Yang, S. Casas, and R. Urtasun. End-to-end interpretable neural motion planner. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019.
  5. K. Chitta, A. Prakash, B. Jaeger, Z. Yu, K. Renz, and A. Geiger. Transfuser: Imitation with transformer-based sensor fusion for autonomous driving. IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI), 2022.
  6. M. Bojarski, D. D. Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Mon- fort, U. Muller, J. Zhang, X. Zhang, J. Zhao, and K. Zieba. End to end learning for self-driving cars. arXiv.org, 1604.07316, 2016.
  7. A. Kendall, J. Hawke, D. Janz, P. Mazur, D. Reda, J. M. Allen, V. D. Lam, A. Bewley, and A. Shah. Learning to drive in a day. arXiv.org, abs/1807.00412, 2018. · arXiv
  8. M. Bansal, A. Krizhevsky, and A. S. Ogale. Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst. In Proc. Robotics: Science and Systems (RSS), 2019.
  9. H. Caesar, J. Kabzan, K. S. Tan, W. K. Fong, E. M. Wolff, A. H. Lang, L. Fletcher, O. Beijbom, and S. Omari. nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) Workshops, 2021.
  10. O. Scheel, L. Bergamini, M. Wolczyk, B. Osiński, and P. Ondruska. Urban driver: Learning to drive from real-world demonstrations using policy gradients. In Proc. Conf. on Robot Learning (CoRL), 2021.
  11. K. Renz, K. Chitta, O.-B. Mercea, A. S. Koepke, Z. Akata, and A. Geiger. Plant: Explainable planning transformers via object-level representations. In Proc. Conf. on Robot Learning (CoRL), 2022.
  12. M. Hallgarten, M. Stoll, and A. Zell. From Prediction to Planning With Goal Conditioned Lane Graph Traversals. arXiv.org, 2302.07753, 2023.
  13. A. Prakash, K. Chitta, and A. Geiger. Multi-modal fusion transformer for end-to-end au- tonomous driving. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2021.
  14. K. Chitta, A. Prakash, and A. Geiger. Neat: Neural attention fields for end-to-end autonomous driving. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2021.
  15. F. Codevilla, A. M. Lopez, V. Koltun, and A. Dosovitskiy. On offline evaluation of vision-based driving models. In Proc. of the European Conf. on Computer Vision (ECCV), 2018.
  16. A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun. CARLA: An open urban driving simulator. In Proc. Conf. on Robot Learning (CoRL), 2017.
  17. M. Treiber, A. Hennecke, and D. Helbing. Congested traffic states in empirical observations and microscopic simulations. Physical review E, 2000.
  18. Y. Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang, L. Lu, X. Jia, Q. Liu, J. Dai, Y. Qiao, and H. Li. Planning-oriented autonomous driving. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2023.
  19. S. Thrun, M. Montemerlo, H. Dahlkamp, D. Stavens, A. Aron, J. Diebel, P. Fong, J. Gale, M. Halpenny, G. Hoffmann, K. Lau, C. M. Oakley, M. Palatucci, V. R. Pratt, P. Stang, S. Stro- hband, C. Dupont, L. Jendrossek, C. Koelen, C. Markey, C. Rummel, J. van Niekerk, E. Jensen, P. Alessandrini, G. R. Bradski, B. Davies, S. Ettinger, A. Kaehler, A. V. Nefian, and P. Mahoney. Stanley: The robot that won the DARPA grand challenge. Journal of Field Robotics (JFR), 23 (9):661-692, 2006.
  20. A. Bacha, C. Bauman, R. Faruque, M. Fleming, C. Terwelp, C. Reinholtz, D. Hong, A. Wicks, T. Alberi, D. Anderson, et al. Odin: Team victortango's entry in the darpa urban challenge. Journal of Field Robotics (JFR), 25(8), 2008.
  21. J. J. Leonard, J. P. How, S. J. Teller, M. Berger, S. Campbell, G. A. Fiore, L. Fletcher, E. Frazzoli, A. S. Huang, S. Karaman, O. Koch, Y. Kuwata, D. Moore, E. Olson, S. Peters, J. Teo, R. Truax, M. R. Walter, D. Barrett, A. Epstein, K. Maheloni, K. Moyer, T. Jones, R. Buckley, M. E. Antone, R. Galejs, S. Krishnamurthy, and J. Williams. A perception-driven autonomous urban vehicle. Journal of Field Robotics (JFR), 25(10):727-774, 2008.
  22. C. Urmson, J. Anhalt, D. Bagnell, C. Baker, R. Bittner, M. Clark, J. Dolan, D. Duggins, T. Galatali, C. Geyer, et al. Autonomous driving in urban environments: Boss and the urban challenge. Journal of Field Robotics (JFR), 25(8), 2008.
  23. C. Chen, A. Seff, A. L. Kornhauser, and J. Xiao. Deepdriving: Learning affordance for direct perception in autonomous driving. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), pages 2722-2730, 2015.
  24. A. Sauer, N. Savinov, and A. Geiger. Conditional affordance learning for driving in urban environments. In Proc. Conf. on Robot Learning (CoRL), 2018.
  25. H. Fan, F. Zhu, C. Liu, L. Zhang, L. Zhuang, D. Li, W. Zhu, J. Hu, H. Li, and Q. Kong. Baidu apollo EM motion planner. arXiv.org, 1807.08048, 2018.
  26. A. Sadat, M. Ren, A. Pokrovsky, Y. Lin, E. Yumer, and R. Urtasun. Jointly learnable behavior and trajectory planning for self-driving vehicles. In Proc. IEEE International Conf. on Intelligent Robots and Systems (IROS), 2019.
  27. A. Kesting, M. Treiber, and D. Helbing. General lane-changing model mobil for car-following models. Transportation Research Record, 1999(1), 2007.
  28. R. Chekroun, T. Gilles, M. Toromanoff, S. Hornauer, and F. Moutarde. Mbappe: Mcts-built- around prediction for planning explicitly. arXiv.org, 2309.08452, 2023.
  29. A. Cui, A. Sadat, S. Casas, R. Liao, and R. Urtasun. Lookout: Diverse multi-future prediction and planning for self-driving. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2021.
  30. S. Casas, A. Sadat, and R. Urtasun. Mp3: A unified model to map, perceive, predict and plan. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2021.
  31. A. Sadat, S. Casas, M. Ren, X. Wu, P. Dhawan, and R. Urtasun. Perceive, predict, and plan: Safe motion planning through interpretable semantic representations. In Proc. of the European Conf. on Computer Vision (ECCV), 2020.
  32. Z. Huang, P. Karkus, B. Ivanovic, Y. Chen, M. Pavone, and C. Lv. Dtpp: Differentiable joint conditional prediction and cost evaluation for tree policy planning in autonomous driving. arXiv.org, 2310.05885, 2023.
  33. T. Phan-Minh, F. Howington, T.-S. Chu, M. S. Tomov, R. E. Beaudoin, S. U. Lee, N. Li, C. Dicle, S. Findler, F. Suarez-Ruiz, et al. Driveirl: Drive in real life with inverse reinforcement learning. In Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2023.
  34. P. Karkus, B. Ivanovic, S. Mannor, and M. Pavone. Diffstack: A differentiable and modular control stack for autonomous vehicles. In Proc. Conf. on Robot Learning (CoRL), 2022.
  35. M. H. Danesh, P. Cai, and D. Hsu. Leader: Learning attention over driving behaviors for planning under uncertainty. In Proc. Conf. on Robot Learning (CoRL), 2022.
  36. Z. Huang, H. Liu, J. Wu, and C. Lv. Differentiable integrated motion prediction and planning with learnable cost function for autonomous driving. arXiv.org, 2207.10422, 2022.
  37. B. Wei, M. Ren, W. Zeng, M. Liang, B. Yang, and R. Urtasun. Perceive, attend, and drive: Learning spatial attention for safe self-driving. Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2021.
  38. S. Hu, L. Chen, P. Wu, H. Li, J. Yan, and D. Tao. St-p3: End-to-end vision-based autonomous driving via spatial-temporal feature learning. In Proc. of the European Conf. on Computer Vision (ECCV), 2022.
  39. H. Song, W. Ding, Y. Chen, S. Shen, M. Y. Wang, and Q. Chen. Pip: Planning-informed trajectory prediction for autonomous driving. In Proc. of the European Conf. on Computer Vision (ECCV), 2020.
  40. N. Rhinehart, J. He, C. Packer, M. A. Wright, R. McAllister, J. E. Gonzalez, and S. Levine. Contingencies from observations: Tractable contingency planning with learned behavior models. In Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2021.
  41. Y. Chen, P. Karkus, B. Ivanovic, X. Weng, and M. Pavone. Tree-structured policy planning with learned behavior models. In Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2023.
  42. Z. Huang, H. Liu, and C. Lv. Gameformer: Game-theoretic modeling and learning of transformer-based interactive prediction and planning for autonomous driving. arXiv.org, 2303.05760, 2023.
  43. L. Chen, P. Wu, K. Chitta, B. Jaeger, A. Geiger, and H. Li. End-to-end autonomous driving: Challenges and frontiers. arXiv.org, 2306.16927, 2023.
  44. N. Rhinehart, R. McAllister, and S. Levine. Deep imitative models for flexible inference, planning, and control. In Proc. of the International Conf. on Learning Representations (ICLR), 2020.
  45. A. Filos, P. Tigas, R. McAllister, N. Rhinehart, S. Levine, and Y. Gal. Can autonomous vehicles identify, recover from, and adapt to distribution shifts? In Proc. of the International Conf. on Machine learning (ICML), 2020.
  46. H. Xu, Y. Gao, F. Yu, and T. Darrell. End-to-end learning of driving models from large-scale video datasets. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 3530-3538, 2017.
  47. D. Chen, B. Zhou, V. Koltun, and P. Krähenbühl. Learning by cheating. In Proc. Conf. on Robot Learning (CoRL), 2019.
  48. E. Ohn-Bar, A. Prakash, A. Behl, K. Chitta, and A. Geiger. Learning situational driving. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2020.
  49. A. Behl, K. Chitta, A. Prakash, E. Ohn-Bar, and A. Geiger. Label efficient visual abstractions for autonomous driving. In Proc. IEEE International Conf. on Intelligent Robots and Systems (IROS), 2020.
  50. P. Wu, X. Jia, L. Chen, J. Yan, H. Li, and Y. Qiao. Trajectory-guided control prediction for end- to-end autonomous driving: A simple yet strong baseline. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
  51. D. Chen and P. Krähenbühl. Learning from all vehicles. In CVPR, 2022.
  52. B. Jaeger, K. Chitta, and A. Geiger. Hidden biases of end-to-end driving models. Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2023.
  53. N. Hanselmann, K. Renz, K. Chitta, A. Bhattacharyya, and A. Geiger. King: Generating safety-critical driving scenarios for robust imitation via kinematics gradients. In Proc. of the European Conf. on Computer Vision (ECCV), 2022.
  54. M. Vitelli, Y. Chang, Y. Ye, A. Ferreira, M. Wołczyk, B. Osiński, M. Niendorf, H. Grimmett, Q. Huang, A. Jain, et al. Safetynet: Safe planning for real-world self-driving vehicles using machine-learned policies. In Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2022.
  55. S. Pini, C. S. Perone, A. Ahuja, A. S. R. Ferreira, M. Niendorf, and S. Zagoruyko. Safe real-world autonomous driving by learning to predict and plan with a mixture of experts. In Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2023.
  56. J. Cheng, Y. Chen, X. Mei, B. Yang, B. Li, and M. Liu. Rethinking imitation-based planner for autonomous driving. arXiv.org, 2309.10443, 2023.
  57. H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2020.
  58. J.-T. Zhai, Z. Feng, J. Du, Y. Mao, J.-J. Liu, Z. Tan, Y. Zhang, X. Ye, and J. Wang. Rethinking the open-loop evaluation of end-to-end autonomous driving in nuscenes. arXiv.org, 2305.10430, 2023.
  59. M. Althoff, M. Koschi, and S. Manzinger. Commonroad: Composable benchmarks for motion planning on roads. In Proc. IEEE Intelligent Vehicles Symposium (IV), 2017.
  60. L. Bergamini, Y. Ye, O. Scheel, L. Chen, C. Hu, L. D. Pero, B. Osinski, H. Grimmett, and P. Ondruska. Simnet: Learning reactive self-driving simulations from real-world observations. In Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2021.
  61. S. Suo, S. Regalado, S. Casas, and R. Urtasun. Trafficsim: Learning to simulate realistic multi-agent behaviors. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2021.
  62. Z. Zhong, D. Rempe, D. Xu, Y. Chen, S. Veer, T. Che, B. Ray, and M. Pavone. Guided conditional diffusion for controllable traffic simulation. In Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2023.
  63. D. Xu, Y. Chen, B. Ivanovic, and M. Pavone. Bits: Bi-level imitation for traffic simulation. In Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2023.
  64. Z. Zhang, A. Liniger, D. Dai, F. Yu, and L. Van Gool. TrafficBots: Towards world models for autonomous driving simulation and motion prediction. In Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2023.
  65. L. Feng, Q. Li, Z. Peng, S. Tan, and B. Zhou. Trafficgen: Learning to generate diverse and realistic traffic scenarios. In Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2023.
  66. S. Ross, G. J. Gordon, and D. Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In Conference on Artificial Intelligence and Statistics (AISTATS), 2011.
  67. A. Prakash, A. Behl, E. Ohn-Bar, K. Chitta, and A. Geiger. Exploring data aggregation in policy learning for vision-based urban autonomous driving. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2020.
  68. S. Poddar, C. Mavrogiannis, and S. S. Srinivasa. From crowd motion prediction to robot navigation in crowds. arXiv preprint arXiv:2303.01424, 2023. · arXiv
  69. C. Schöller, V. Aravantinos, F. Lay, and A. Knoll. What the constant velocity model can teach us about pedestrian motion prediction. IEEE Robotics and Automation Letters, 5(2):1696-1703, 2020.
  70. H. Wu, T. Phong, C. Yu, P. Cai, S. Zheng, and D. Hsu. What truly matters in trajectory prediction for autonomous driving? arXiv preprint arXiv:2306.15136, 2023. · arXiv
  71. Y. Tassa, N. Mansard, and E. Todorov. Control-limited differential dynamic programming. In Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2014.
  72. R. Rajamani. Vehicle dynamics and control. Springer Science & Business Media, 2011.
  73. P. Polack, F. Altché, B. d'Andréa Novel, and A. de La Fortelle. The kinematic bicycle model: A consistent model for planning feasible trajectories for autonomous vehicles? In Proc. IEEE Intelligent Vehicles Symposium (IV), 2017.
  74. N. Deo, E. M. Wolff, and O. Beijbom. Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversals. In Proc. Conf. on Robot Learning (CoRL), 2021.
  75. N. Nayakanti, R. Al-Rfou, A. Zhou, K. Goel, K. S. Refaat, and B. Sapp. Wayformer: Motion forecasting via simple and efficient attention networks. In Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2023.
  76. A. Cui, S. Casas, K. Wong, S. Suo, and R. Urtasun. Gorela: Go relative for viewpoint-invariant motion forecasting. In Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2023.
  77. Y. Hu, K. Li, P. Liang, J. Qian, Z. Yang, H. Zhang, W. Shao, Z. Ding, W. Xu, and Q. Liu. Imitation with spatial-temporal heatmap: 2nd place solution for nuplan challenge. arXiv.org, 2306.15700, 2023.
  78. C. R. Qi, Y. Zhou, M. Najibi, P. Sun, K. Vo, B. Deng, and D. Anguelov. Offboard 3d object detection from point cloud sequences. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2021.
  79. O. Scheel, L. Bergamini, M. Wolczyk, B. Osinski, and P. Ondruska. Urban driver: Learning to drive from real-world demonstrations using policy gradients. In Proc. Conf. on Robot Learning (CoRL), 2021.

Cited by

Sorted by citations

Authors
Dauner, D., Hallgarten, M., Geiger, A., & Chitta, K.
Affiliations
  • Tübingen AI Center (DE)
  • University of Tübingen (DE)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2306.07962
arXiv
2306.07962
OpenAlex
W4380715592
Cited by
33
References
0
Citations by year
2025: 10 · 2024: 21 · 2023: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • International Max Planck Research School for Environmental, Cellular and Molecular Microbiology
  • Deutsche Forschungsgemeinschaft (206/4-1, 390727645)
  • Bundesministerium für Bildung und Forschung (01IS18039A, FKZ: 01IS18039A)
  • International Max Planck Research School for Advanced Methods in Process and Systems Engineering
  • H2020 European Research Council (850533)
Topics
  • Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Robotic Path Planning Algorithms
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords
vehicle motion planning, learning-based planning, closed-loop planning, autonomous driving, scene context

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review