The DOI matches its Crossref record; 214 of the 214 references it cites were found in a registry or an index.
End-to-End Autonomous Driving: Challenges and Frontiers
Open accessAbstract
The autonomous driving community has witnessed a rapid growth in approaches that embrace an end-to-end algorithm framework, utilizing raw sensor input to generate vehicle motion plans, instead of concentrating on individual tasks such as detection and motion prediction. End-to-end systems, in comparison to modular pipelines, benefit from joint feature optimization for perception and planning. This field has flourished due to the availability of large-scale datasets, closed-loop evaluation, and the increasing need for autonomous driving algorithms to perform effectively in challenging scenarios. In this survey, we provide a comprehensive analysis of more than 270 papers, covering the motivation, roadmap, methodology, challenges, and future trends in end-to-end autonomous driving. We delve into several critical challenges, including multi-modality, interpretability, causal confusion, robustness, and world models, amongst others. Additionally, we discuss current advancements in foundation models and visual pre-training, as well as how to incorporate these techniques within the end-to-end driving framework.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Generative adversarial networks
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020) · Communications of the ACM · doi:10.1145/3422622 · Cited by 18,384
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00371 · Cited by 11,953
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489
nuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636
3D Gaussian Splatting for Real-Time Radiance Field Rendering
Kerbl, B., Kopanas, G., Leimkuehler, T., & Drettakis, G. (2023) · ACM Transactions on Graphics · doi:10.1145/3592433 · Cited by 6,025
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_24 · Cited by 5,351
Congested traffic states in empirical observations and microscopic simulations
Treiber, M., Hennecke, A., & Helbing, D. (2000) · Physical Review E · doi:10.1103/physreve.62.1805 · Cited by 4,762
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Rajamani, R. (2012) · Mechanical Engineering Series · doi:10.1007/978-1-4614-1433-9 · Cited by 4,401
CARLA: An open urban driving simulator
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017) · arXiv · doi:10.48550/arxiv.1711.03938 · Cited by 3,718
Scalability in Perception for Autonomous Driving: Waymo Open Dataset
Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., Caine, B., et al. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00252 · Cited by 3,437
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435
Microscopic Traffic Simulation using SUMO
Lopez, P. A., Wiessner, E., Behrisch, M., Bieker-Walz, L., Erdmann, J., Flotterod, Y.-P., Hilbrich, R., Lucken, L., Rummel, J., & Wagner, P. (2018) · 2018 21st International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569938 · Cited by 3,297
End to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198
Domain randomization for transferring deep neural networks from simulation to the real world
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202133 · Cited by 3,124 · cited 1×
“Domainrandomization[265], [266],[267]isalsoasimpleandeffectivesim-to-realtechnique forRLpolicylearning,whichisfurtheradaptedforend-to-end autonomousdriving[188],[268].”
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117
Class-Balanced Loss Based on Effective Number of Samples
Cui, Y., Jia, M., Lin, T.-Y., Song, Y., & Belongie, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00949 · Cited by 3,094
A systematic study of the class imbalance problem in convolutional neural networks
Buda, M., Maki, A., & Mazurowski, M. A. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.07.011 · Cited by 3,088 · cited 1×
“Various methods mitigate this issue with data processing, including over-sampling [246], [247], under-sampling [248], [249], and data augmentation [250], [251].”
MethodExploratory Undersampling for Class-Imbalance Learning
Liu, X.-Y., Wu, J., & Zhou, Z.-H. (2009) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2008.2007853 · Cited by 2,465
Deep reinforcement learning for autonomous driving: A survey
Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Sallab, A. A. A., Yogamani, S., & Perez, P. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3054625 · Cited by 2,342
Shortcut learning in deep neural networks
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00257-z · Cited by 2,224
The Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3912 · Cited by 2,161
Generative adversarial imitation learning
Ho, J., & Ermon, S. (2016) · arXiv · doi:10.48550/arxiv.1606.03476 · Cited by 1,942
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555
Zhang, X. (2017) · Encyclopedia of Machine Learning and Data Mining · doi:10.1007/978-1-4899-7687-1_79 · Cited by 1,434
Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Yu, Q., & Dai, J. (2022) · arXiv · doi:10.48550/arxiv.2203.17270 · Cited by 1,419
Self-citationSim-to-Real Transfer of Robotic Control with Dynamics Randomization
Peng, X. B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460528 · Cited by 1,309 · cited 1×
“Domainrandomization[265], [266],[267]isalsoasimpleandeffectivesim-to-realtechnique forRLpolicylearning,whichisfurtheradaptedforend-to-end autonomousdriving[188],[268].”
Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation
Liu, Z., Tang, H., Amini, A., Yang, X., Mao, H., Rus, D. L., & Han, S. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160968 · Cited by 1,307
Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sunderhauf, N., Reid, I., Gould, S., & van den Hengel, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00387 · Cited by 1,187
Dynamic Few-Shot Visual Learning Without Forgetting
Gidaris, S., & Komodakis, N. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00459 · Cited by 1,128
DINOv2: Learning Robust Visual Features without Supervision
Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.07193 · Cited by 1,078 · cited 1×
“Recent advancements in foundation models in both language [166], [167], [274] and vision [273], [275], [276] have proved that large-scale data and model capacity can unleash the immense potential of AI in high-level reasoning tasks.”
BackgroundPlanning-oriented Autonomous Driving
Hu, Y., Yang, J., Chen, L., Li, K., Sima, C., Zhu, X., Chai, S., Du, S., Lin, T., Wang, W., et al. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01712 · Cited by 931
Self-citationEnd-to-End Learning of Driving Models from Large-Scale Video Datasets
Xu, H., Gao, Y., Yu, F., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.376 · Cited by 884
Block-NeRF: Scalable Large Scene Neural View Synthesis
Tancik, M., Casser, V., Yan, X., Pradhan, S., Mildenhall, B. P., Srinivasan, P., Barron, J. T., & Kretzschmar, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00807 · Cited by 781
KNN approach to unbalanced data distributions: a case study involving information extraction
Ziang, J. (2003) · International Conference on Machine Learning · Cited by 772
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2021) · arXiv · doi:10.48550/arxiv.2112.10752 · Cited by 737
ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst
Bansal, M., Krizhevsky, A., & Ogale, A. (2019) · Robotics: Science and Systems XV · doi:10.15607/rss.2019.xv.031 · Cited by 731
Kendall, A., Hawke, J., Janz, D., Mazur, P., Reda, D., Allen, J.-M., Lam, V.-D., Bewley, A., & Shah, A. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793742 · Cited by 694 · cited 2×
“Notable progress has been achieved with the development of deep neural networks, both in imitation learning [15], [16] and reinforcement learning [4], [17], [18], [19].”
BackgroundMotion Planning for Autonomous Driving: The State of the Art and Future Perspectives
Teng, S., Hu, X., Deng, P., Li, B., Li, Y., Ai, Y., Yang, D., Li, L., Xuanyuan, Z., Zhu, F., et al. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3274536 · Cited by 590
Self-citationMulti-Modal Fusion Transformer for End-to-End Autonomous Driving
Prakash, A., Chitta, K., & Geiger, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00700 · Cited by 584
Self-citationChatGPT for Robotics: Design Principles and Model Abilities
Vemprala, S. H., Bonatti, R., Bucker, A., & Kapoor, A. (2024) · IEEE Access · doi:10.1109/access.2024.3387941 · Cited by 570
DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection
Li, Y., Yu, A. W., Meng, T., Caine, B., Ngiam, J., Peng, D., Shen, J., Lu, Y., Zhou, D., Le, Q. V., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01667 · Cited by 563
Procedural content generation for games: A survey
Hendrikx, M., Meijer, S., Van Der Velden, J., & Iosup, A. (2013) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/2422956.2422957 · Cited by 558
GPT-Driver: Learning to Drive with GPTPreprint
Mao, J., Qian, Y., Ye, J., Zhao, H., & Wang, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.01415 · Cited by 533
Exploring the Limitations of Behavior Cloning for Autonomous Driving
Codevilla, F., Santana, E., Lopez, A., & Gaidon, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00942 · Cited by 509
An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detection
Lee, Y., Hwang, J.-W., Lee, S., Bae, Y., & Park, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00103 · Cited by 508
EVA: Exploring the Limits of Masked Visual Representation Learning at Scale
Fang, Y., Wang, W., Xie, B., Sun, Q., Wu, L., Wang, X., Huang, T., Wang, X., & Cao, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01855 · Cited by 506
TransFuser: Imitation With Transformer-Based Sensor Fusion for Autonomous Driving
Chitta, K., Prakash, A., Jaeger, B., Yu, Z., Renz, K., & Geiger, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3200245 · Cited by 505
Self-citationDriveGPT4: Interpretable End-to-End Autonomous Driving Via Large Language Model
Xu, Z., Zhang, Y., Xie, E., Zhao, Z., Guo, Y., Wong, K.-Y. K., Li, Z., & Zhao, H. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3440097 · Cited by 443
Klampfl, L., & Wotawa, F. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.10 · Cited by 431
Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly- Throughs
Turki, H., Ramanan, D., & Satyanarayanan, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01258 · Cited by 423
A Survey of Embodied AI: From Simulators to Research Tasks
Duan, J., Yu, S., Tan, H. L., Zhu, H., & Tan, C. (2022) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2022.3141105 · Cited by 420
Polack, P., Altche, F., d'Andrea-Novel, B., & de La Fortelle, A. (2017) · 2017 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2017.7995816 · Cited by 408
End-to-end Interpretable Neural Motion Planner
Zeng, W., Luo, W., Suo, S., Sadat, A., Yang, B., Casas, S., & Urtasun, R. (2021) · arXiv · doi:10.48550/arxiv.2101.06679 · Cited by 406
End-To-End Interpretable Neural Motion Planner
Zeng, W., Luo, W., Suo, S., Sadat, A., Yang, B., Casas, S., & Urtasun, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00886 · Cited by 398
HDMapNet: An Online HD Map Construction and Evaluation Framework
Li, Q., Wang, Y., Wang, Y., & Zhao, H. (2022) · 2022 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra46639.2022.9812383 · Cited by 374
Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention
Kim, J., & Canny, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.320 · Cited by 366
A framework for behavioural cloning
Bain, M., & Sammut, C. (2000) · Machine Intelligence 15 · doi:10.1093/oso/9780198538677.003.0006 · Cited by 360
Query-Efficient Imitation Learning for End-to-End Simulated Driving
Zhang, J., & Cho, K. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10857 · Cited by 347
TOUCHDOWN: Natural Language Navigation and Spatial Reasoning in Visual Street Environments
Chen, H., Suhr, A., Misra, D., Snavely, N., & Artzi, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01282 · Cited by 343
The DARPA Urban Challenge: Autonomous Vehicles in City Traffic
(2009) · Springer Tracts in Advanced Robotics · doi:10.1007/978-3-642-03991-1 · Cited by 341
Textual Explanations for Self-Driving Vehicles
Kim, J., Rohrbach, A., Darrell, T., Canny, J., & Akata, Z. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_35 · Cited by 337
Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?
Li, Z., Yu, Z., Lan, S., Li, J., Kautz, J., Lü, T., & Álvarez, J. M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01408 · Cited by 333
Virtual to Real Reinforcement Learning for Autonomous Driving
Pan, X., You, Y., Wang, Z., & Lu, C. (2017) · Procedings of the British Machine Vision Conference 2017 · doi:10.5244/c.31.11 · Cited by 326
Causal Confusion in Imitation LearningPreprint
de Haan, P., Jayaraman, D., & Levine, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11979 · Cited by 323
Interpretable End-to-End Urban Autonomous Driving With Latent Deep Reinforcement Learning
Chen, J., Li, S. E., & Tomizuka, M. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3046646 · Cited by 322
Multimodal end-to-end autonomous driving
Xiao, Y., Codevilla, F., Gurram, A., Urfalioglu, O., & Lopez, A. M. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3013234 · Cited by 319
Guided Conditional Diffusion for Controllable Traffic Simulation
Zhong, Z., Rempe, D., Xu, D., Chen, Y., Veer, S., Che, T., Ray, B., & Pavone, M. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161463 · Cited by 312
Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving
Jia, X., Wu, P., Chen, L., Xie, J., He, C., Yan, J., & Li, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02105 · Cited by 306
Self-citationTORCS, The Open Racing Car Simulator
Espié, E., Guionneau, C., Wymann, B., Christos Dimitrakakis, A., Coulom, R., & Sumner, A. J. (2005) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 294
A General Framework for Uncertainty Estimation in Deep Learning
Loquercio, A., Segu, M., & Scaramuzza, D. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2974682 · Cited by 291
A Survey of End-to-End Driving: Architectures and Training Methods
Tampuu, A., Matiisen, T., Semikin, M., Fishman, D., & Muhammad, N. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3043505 · Cited by 291
CIRL: Controllable Imitative Reinforcement Learning for Vision-Based Self-driving
Liang, X., Wang, T., Yang, L., & Xing, E. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_36 · Cited by 285
The 2005 DARPA Grand Challenge: The Great Robot Race
(2007) · Springer Tracts in Advanced Robotics · doi:10.1007/978-3-540-73429-1 · Cited by 276
A General Framework for Uncertainty Estimation in Deep Learning
Loquercio, A., Segù, M., & Scaramuzza, D. (2020) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-197704 · Cited by 261
ST-P3: End-to-End Vision-Based Autonomous Driving via Spatial-Temporal Feature Learning
Hu, S., Chen, L., Wu, P., Li, H., Yan, J., & Tao, D. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19839-7_31 · Cited by 259
Self-citationAgile Autonomous Driving using End-to-End Deep Imitation Learning
Pan, Y., Cheng, C.-A., Saigol, K., Lee, K., Yan, X., Theodorou, E., & Boots, B. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.056 · Cited by 257
Talk2Car: Taking Control of Your Self-Driving Car
Deruyttere, T., Vandenhende, S., Grujicic, D., Gool, L. V., & Moens, M.-F. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1215 · Cited by 253
MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning
Li, Q., Peng, Z., Feng, L., Zhang, Q., Xue, Z., & Zhou, B. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3190471 · Cited by 246
A Survey on Imitation Learning Techniques for End-to-End Autonomous Vehicles
Le Mero, L., Yi, D., Dianati, M., & Mouzakitis, A. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3144867 · Cited by 243
Scenic: a language for scenario specification and scene generation
Fremont, D. J., Dreossi, T., Ghosh, S., Yue, X., Sangiovanni-Vincentelli, A. L., & Seshia, S. A. (2019) · Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/3314221.3314633 · Cited by 241
A Survey of Deep RL and IL for Autonomous Driving Policy Learning
Zhu, Z., & Zhao, H. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3134702 · Cited by 237
LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World
Manivasagam, S., Wang, S., Wong, K., Zeng, W., Sazanovich, M., Tan, S., Yang, B., Ma, W.-C., & Urtasun, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01118 · Cited by 233
DriveLM: Driving with Graph Visual Question Answering
Sima, C., Renz, K., Chitta, K., Chen, L., Zhang, H., Xie, C., Beißwenger, J., Luo, P., Geiger, A., & Li, H. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-72943-0_15 · Cited by 229
Self-citationMP3: A Unified Model to Map, Perceive, Predict and Plan
Casas, S., Sadat, A., & Urtasun, R. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01417 · Cited by 228
Explainability of Deep Vision-Based Autonomous Driving Systems: Review and Challenges
Zablocki, É., Ben-Younes, H., Pérez, P., & Cord, M. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01657-x · Cited by 227
Explainability of deep vision-based autonomous driving systems: Review and challenges
Zablocki, É., Ben-Younes, H., Pérez, P., & Cord, M. (2021) · arXiv · doi:10.48550/arxiv.2101.05307 · Cited by 227
Imitation Learning: Progress, Taxonomies and Challenges
Zheng, B., Verma, S., Zhou, J. J., Tsang, I. W.-H., & Chen, F. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3213246 · Cited by 226
Generalized Predictive Model for Autonomous Driving
Yang, J., Gao, S., Qiu, Y., Chen, L., Li, T., Dai, B., Chitta, K., Wu, P., Zeng, J., Luo, P., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01389 · Cited by 226
Self-citationCLIPort: What and Where Pathways for Robotic Manipulation
Shridhar, M., Manuelli, L., & Fox, D. (2021) · arXiv · doi:10.48550/arxiv.2109.12098 · Cited by 223
Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation
Kundu, A., Genova, K., Yin, X., Fathi, A., Pantofaru, C., Guibas, L., Tagliasacchi, A., Dellaert, F., & Funkhouser, T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01253 · Cited by 223
NEAT: Neural Attention Fields for End-to-End Autonomous Driving
Chitta, K., Prakash, A., & Geiger, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01550 · Cited by 222
Self-citationEnd-to-End Urban Driving by Imitating a Reinforcement Learning Coach
Zhang, Z., Liniger, A., Dai, D., Yu, F., & Van Gool, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01494 · Cited by 218
TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarios
Feng, L., Li, Q., Peng, Z., Tan, S., & Zhou, B. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160296 · Cited by 217
Urban Driving with Conditional Imitation Learning
Hawke, J., Shen, R., Gurău, C., Sharma, S., Reda, D., Nikolov, N., Mazur, P., Micklethwaite, S., Griffiths, N., Shah, A., et al. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197408 · Cited by 212
Reward (Mis)design for autonomous driving
Knox, W. B., Allievi, A., Banzhaf, H., Schmitt, F., & Stone, P. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2022.103829 · Cited by 210
End-to-End Model-Free Reinforcement Learning for Urban Driving Using Implicit Affordances
Toromanoff, M., Wirbel, E., & Moutarde, F. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00718 · Cited by 208
Jia, X., Gao, Y., Chen, L., Yan, J., Liu, P. L., & Li, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00731 · Cited by 208
Self-citationLearning Robust Control Policies for End-to-End Autonomous Driving From Data-Driven Simulation
Amini, A., Gilitschenski, I., Phillips, J., Moseyko, J., Banerjee, R., Karaman, S., & Rus, D. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2966414 · Cited by 204
Hidden Biases of End-to-End Driving Models
Jaeger, B., Chitta, K., & Geiger, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00757 · Cited by 195
Self-citationA Survey of Zero-shot Generalisation in Deep Reinforcement Learning
Kirk, R., Zhang, A., Grefenstette, E., & Rocktäschel, T. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14174 · Cited by 193 · cited 1×
“Formalizing this task for the end-to-end driving domain and incorporating techniques from the zero-shot/few-shot learning literature are the key steps toward achieving this [269], [270].”
MethodLearning to drive from a world on rails
Chen, D., Koltun, V., & Krähenbühl, P. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01530 · Cited by 192
Chen, D., & Krahenbuhl, P. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01671 · Cited by 189
Huang, Z., Lv, C., Xing, Y., & Wu, J. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3003121 · Cited by 188
Wu, J., Huang, Z., & Lv, C. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3185159 · Cited by 188
KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients
Hanselmann, N., Renz, K., Chitta, K., Bhattacharyya, A., & Geiger, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19839-7_20 · Cited by 187
Self-citationAdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles
Wang, J., Pun, A., Tu, J., Manivasagam, S., Sadat, A., Casas, S., Ren, M., & Urtasun, R. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00978 · Cited by 186
Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representations
Sadat, A., Casas, S., Ren, M., Wu, X., Dhawan, P., & Urtasun, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58592-1_25 · Cited by 183
LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving
Cui, A., Casas, S., Sadat, A., Liao, R., & Urtasun, R. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01580 · Cited by 180
LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action
Shah, D., Osinski, B., Ichter, B., & Levine, S. (2022) · arXiv · doi:10.48550/arxiv.2207.04429 · Cited by 179
LMDrive: Closed-Loop End-to-End Driving with Large Language Models
Shao, H., Hu, Y., Wang, L., Song, G., Waslander, S. L., Liu, Y., & Li, H. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01432 · Cited by 174
Self-citationVisual Point Cloud Forecasting Enables Scalable Autonomous Driving
Yang, Z., Chen, L., Sun, Y., & Li, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01390 · Cited by 171
Self-citationUniSim: A Neural Closed-Loop Sensor Simulator
Yang, Z., Chen, Y., Wang, J., Manivasagam, S., Ma, W.-C., Yang, A. J., & Urtasun, R. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00140 · Cited by 170
Delving Into the Devils of Bird’s-Eye-View Perception: A Review, Evaluation and Recipe
Li, H., Sima, C., Dai, J., Wang, W., Lu, L., Wang, H., Zeng, J., Li, Z., Yang, J., Deng, H., et al. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3333838 · Cited by 170
Self-citationNuScenes-QA: A Multi-Modal Visual Question Answering Benchmark for Autonomous Driving Scenario
Qian, T., Chen, J., Zhuo, L., Jiao, Y., & Jiang, Y.-G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28253 · Cited by 168
Generating Adversarial Driving Scenarios in High-Fidelity Simulators
Abeysirigoonawardena, Y., Shkurti, F., & Dudek, G. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793740 · Cited by 167
Stochastic Model Predictive Control With a Safety Guarantee for Automated Driving
Brüdigam, T., Olbrich, M., Wollherr, D., & Leibold, M. (2021) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2021.3074645 · Cited by 163
Synthetic Datasets for Autonomous Driving: A Survey
Song, Z., He, Z., Li, X., Ma, Q., Ming, R., Mao, Z., Pei, H., Peng, L., Hu, J., Yao, D., et al. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3331024 · Cited by 162
Enhancing Photorealism Enhancement
Richter, S. R., Alhaija, H. A., & Koltun, V. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3166687 · Cited by 159
Sensor Fusion for Joint 3D Object Detection and Semantic Segmentation
Meyer, G. P., Charland, J., Hegde, D., Laddha, A., & Vallespi-Gonzalez, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00162 · Cited by 158
Learning to Drive by Imitation: An Overview of Deep Behavior Cloning Methods
Ly, A. O., & Akhloufi, M. A. (2020) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2020.3002505 · Cited by 153
Tong, W., Sima, C., Wang, T., Chen, L., Wu, S., Deng, H., Gu, Y., Lu, L., Luo, P., Lin, D., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00772 · Cited by 151
Self-citationComputer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art
Janai, J., Güney, F., Behl, A., & Geiger, A. (2017) · arXiv · doi:10.48550/arxiv.1704.05519 · Cited by 141
Self-citationSafe Local Motion Planning with Self-Supervised Freespace Forecasting
Hu, P., Huang, A., Dolan, J. M., Held, D., & Ramanan, D. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01254 · Cited by 138
ADAPT: Action-aware Driving Caption Transformer
Jin, B., Liu, X., Zheng, Y., Li, P., Zhao, H., Zhang, T., Zheng, Y., Zhou, G., & Liu, J. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160326 · Cited by 138
BITS: Bi-level Imitation for Traffic Simulation
Xu, D., Chen, Y., Ivanovic, B., & Pavone, M. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161167 · Cited by 136
DSDNet: Deep Structured Self-driving Network
Zeng, W., Wang, S., Liao, R., Chen, Y., Yang, B., & Urtasun, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58589-1_10 · Cited by 134
Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Method
Ding, W., Chen, B., Xu, M., & Zhao, D. X. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340696 · Cited by 132
Multimodal Safety-Critical Scenarios Generation for Decision-Making Algorithms Evaluation
Ding, W., Chen, B., Li, B., Eun, K. J., & Zhao, D. X. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3058873 · Cited by 132
GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Driving
Chen, Y., Rong, F., Duggal, S., Wang, S., Yan, X., Manivasagam, S., Xue, S., Yumer, E., & Urtasun, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00715 · Cited by 132
LiDAR-Video Driving Dataset: Learning Driving Policies Effectively
Chen, Y., Wang, J., Li, J., Lu, C., Luo, Z., Xue, H., & Wang, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00615 · Cited by 131
Explainable Object-Induced Action Decision for Autonomous Vehicles
Xu, Y., Yang, X., Gong, L., Lin, H.-C., Wu, T.-Y., Li, Y., & Vasconcelos, N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00954 · Cited by 129 · cited 1×
“Kim et al. [33] and Xu et al. [228] develop datasets pairing driving videos or images with descriptions and explanations, and propose end-to-end models with both control and explanation outputs.”
BackgroundEvaluating Uncertainty Quantification in End-to-End Autonomous Driving ControlPreprint
Michelmore, R., Kwiatkowska, M., & Gal, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.06817 · Cited by 129
End-to-End Driving Via Conditional Imitation Learning
Codevilla, F., Muller, M., Lopez, A., Koltun, V., & Dosovitskiy, A. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460487 · Cited by 129
Does computer vision matter for action?
Zhou, B., Krähenbühl, P., & Koltun, V. (2019) · Science Robotics · doi:10.1126/scirobotics.aaw6661 · Cited by 126
Differentiable Raycasting for Self-Supervised Occupancy Forecasting
Khurana, T., Hu, P., Dave, A., Ziglar, J., Held, D., & Ramanan, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19839-7_21 · Cited by 124
Multi-task Learning with Attention for End-to-end Autonomous Driving
Ishihara, K., Kanervisto, A., Miura, J., & Hautamäki, V. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00325 · Cited by 123
Exploring Data Aggregation in Policy Learning for Vision-Based Urban Autonomous Driving
Prakash, A., Behl, A., Ohn-Bar, E., Chitta, K., & Geiger, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01178 · Cited by 122
Self-citationReasonNet: End-to-End Driving with Temporal and Global Reasoning
Shao, H., Wang, L., Chen, R., Waslander, S. L., Li, H., & Liu, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01319 · Cited by 122
Self-citationWang, Y., He, J., Fan, L., Li, H., Chen, Y., & Zhang, Z. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01397 · Cited by 119
Self-citationGrounding Human-To-Vehicle Advice for Self-Driving Vehicles
Kim, J., Misu, T., Chen, Y.-T., Tawari, A., & Canny, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01084 · Cited by 117
Deep Object-Centric Policies for Autonomous Driving
Wang, D., Devin, C., Cai, Q.-Z., Yu, F., & Darrell, T. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794224 · Cited by 115
Learning to Drive from Simulation without Real World Labels
Bewley, A., Rigley, J., Liu, Y., Hawke, J., Shen, R., Lam, V.-D., & Kendall, A. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793668 · Cited by 114
Sensor fusion for semantic segmentation of urban scenes
Zhang, R., Candra, S. A., Vetter, K., & Zakhor, A. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139439 · Cited by 109
SimNet: Learning Reactive Self-driving Simulations from Real-world Observations
Bergamini, L., Ye, Y., Scheel, O., Chen, L., Hu, C. C., Pero, L. D., Osiński, B., Grimmett, H., & Ondrúška, P. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561666 · Cited by 108
Self-citationDid We Test All Scenarios for Automated and Autonomous Driving Systems?
Hauer, F., Schmidt, T., Holzmüller, B., & Pretschner, A. (2019) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2019.8917326 · Cited by 106
SceneGen: Learning to Generate Realistic Traffic Scenes
Tan, S., Wong, K., Wang, S., Manivasagam, S., Ren, M., & Urtasun, R. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00095 · Cited by 104 · cited 1×
“They can then generate novel scenarios that resemble the real world but were not included in the original data [96], [97], [98], [99].”
BackgroundOn Offline Evaluation of Vision-Based Driving Models
Codevilla, F., López, A. M., Koltun, V., & Dosovitskiy, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_15 · Cited by 103
MultiNet: Multi-Modal Multi-Task Learning for Autonomous Driving
Chowdhuri, S., Pankaj, T., & Zipser, K. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00164 · Cited by 101
TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction
Zhang, Z., Liniger, A., Dai, D., Yu, F., & Gool, L. V. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161243 · Cited by 100
Amini, A., Wang, T.-H. J., Gilitschenski, I., Schwarting, W., Liu, Z., Han, S., Karaman, S., & Rus, D. L. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812276 · Cited by 97
Ohn-Bar, E., Prakash, A., Behl, A., Chitta, K., & Geiger, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01131 · Cited by 95
Self-citationBeyond Grand Theft Auto V for Training, Testing and Enhancing Deep Learning in Self Driving CarsPreprint
Martinez, M., Sitawarin, C., Finch, K., Meincke, L., Yablonski, A., & Kornhauser, A. L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.01397 · Cited by 93
A Review of End-to-End Autonomous Driving in Urban Environments
Coelho, D., & de Oliveira, M. A. R. (2022) · IEEE Access · doi:10.1109/access.2022.3192019 · Cited by 92
A Survey on Deep Domain Adaptation for LiDAR Perception
Triess, L. T., Dreißig, M., Rist, C. B., & Zöllner, J. M. (2021) · IEEE Intelligent Vehicles Symposium Workshops (IV Workshops) · doi:10.1109/ivworkshops54471.2021.9669228 · Cited by 84
Cai, P., Wang, S., Sun, Y., & Liu, M. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2994027 · Cited by 82
GRI: General Reinforced Imitation and Its Application to Vision-Based Autonomous Driving
Chekroun, R., Toromanoff, M., Hornauer, S., & Moutarde, F. (2023) · Robotics · doi:10.3390/robotics12050127 · Cited by 81
Learning to Steer by Mimicking Features from Heterogeneous Auxiliary Networks
Hou, Y., Ma, Z., Liu, C., & Loy, C. C. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018433 · Cited by 76
Domain Adaptation In Reinforcement Learning Via Latent Unified State Representation
Xing, J., Nagata, T., Chen, K., Zou, X., Neftci, E., & Krichmar, J. L. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17251 · Cited by 75
End-to-End Autonomous Driving With Semantic Depth Cloud Mapping and Multi-Agent
Natan, O., & Miura, J. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3185303 · Cited by 72
Explaining Autonomous Driving by Learning End-to-End Visual Attention
Cultrera, L., Seidenari, L., Becattini, F., Pala, P., & Bimbo, A. D. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00178 · Cited by 71
VTGNet: A Vision-Based Trajectory Generation Network for Autonomous Vehicles in Urban Environments
Cai, P., Sun, Y., Wang, H., & Liu, M. (2020) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2020.3033878 · Cited by 61
Visual Explanation by Attention Branch Network for End-to-end Learning-based Self-driving
Mori, K., Fukui, H., Murase, T., Hirakawa, T., Yamashita, T., & Fujiyoshi, H. (2019) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2019.8813900 · Cited by 57
Deductive Reinforcement Learning for Visual Autonomous Urban Driving Navigation
Huang, C., Zhang, R., Ouyang, M., Wei, P., Lin, J., Su, J., & Lin, L. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3109284 · Cited by 57
Driving behavior explanation with multi-level fusion
Ben-younes, H., Zablocki, É., Pérez, P., & Cord, M. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108421 · Cited by 52
X-Align: Cross-Modal Cross-View Alignment for Bird’s-Eye-View Segmentation
Borse, S., Klingner, M., Kumar, V. R., Cai, H., Almuzairee, A., Yogamani, S. K., & Porikli, F. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00330 · Cited by 52
Is Deep Reinforcement Learning Really Superhuman on Atari? Leveling the playing fieldPreprint
Toromanoff, M., Wirbel, É., & Moutarde, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.04683 · Cited by 50
SLEDGE: Synthesizing Driving Environments with Generative Models and Rule-Based Traffic
Chitta, K., Dauner, D., & Geiger, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73235-5_4 · Cited by 50
Self-citationGao, Z., Mu, Y., Chen, C., Duan, J., Luo, P., Lu, Y., & Li, S. E. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3400227 · Cited by 49
Li, Z., Motoyoshi, T., Sasaki, K., Ogata, T., & Sugano, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.11100 · Cited by 48
Advisable Learning for Self-Driving Vehicles by Internalizing Observation-to-Action Rules
Kim, J., Moon, S., Rohrbach, A., Darrell, T., & Canny, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00968 · Cited by 45
Rethinking Closed-Loop Training for Autonomous Driving
Zhang, W., Guo, R., Zeng, W., Xiong, Y., Dai, B., Hu, R., Ren, M., & Urtasun, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19842-7_16 · Cited by 45
Generating Landmark Navigation Instructions from Maps as a Graph-to-Text Problem
Schumann, R., & Riezler, S. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.41 · Cited by 44
Learning End-to-end Autonomous Driving using Guided Auxiliary Supervision
Mehta, A., Subramanian, A., & Subramanian, A. (2018) · Indian Conference on Computer Vision, Graphics and Image Processing · doi:10.1145/3293353.3293364 · Cited by 43
Label Efficient Visual Abstractions for Autonomous Driving
Behl, A., Chitta, K., Prakash, A., Ohn-Bar, E., & Geiger, A. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340641 · Cited by 40
Self-citationZhang, J., Huang, Z., & Ohn-Bar, E. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00754 · Cited by 38
MMFN: Multi-Modal-Fusion-Net for End-to-End Driving
Zhang, Q., Tang, M., Geng, R., Chen, F., Ren, X., & Wang, L. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981775 · Cited by 37
Dynamic Conditional Imitation Learning for Autonomous Driving
Eraqi, H. M., Moustafa, M., & Honer, J. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3214079 · Cited by 36
Ground then Navigate: Language-guided Navigation in Dynamic Scenes
Jain, K., Chhangani, V., Tiwari, A., Krishna, K. M., & Gandhi, V. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160614 · Cited by 36
Scaling Vision-Based End-to-End Autonomous Driving with Multi-View Attention Learning
Xiao, Y., Codevilla, F., Porres, D., & López, A. M. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341506 · Cited by 31
UrbanGIRAFFE: Representing Urban Scenes as Compositional Generative Neural Feature Fields
Yang, Y., Yang, Y., Guo, H., Xiong, R., Wang, Y., & Liao, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00844 · Cited by 31
Visual-based Autonomous Driving Deployment from a Stochastic and Uncertainty-aware Perspective
Tai, L., Peng, Y., Chen, Y., Liu, C., Ye, H., & Liu, M. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968307 · Cited by 30
Self-Imitation Learning by Planning
Luo, S., Kasaei, H., & Schomaker, L. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561411 · Cited by 29
Ahmed, M., Abobakr, A., Lim, C. P., & Nahavandi, S. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3115235 · Cited by 29
End-to-End Interactive Prediction and Planning with Optical Flow Distillation for Autonomous Driving
Wang, H., Cai, P., Fan, R., Sun, Y., & Liu, M. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00252 · Cited by 28
Talk to the Vehicle: Language Conditioned Autonomous Navigation of Self Driving Cars
Sriram, N. N., Maniar, T., Jayaganesh, K., Gandhi, V., Bhowmick, B., & Krishna, K. M. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967929 · Cited by 27
Learning to Drive by Watching YouTube Videos: Action-Conditioned Contrastive Policy Pretraining
Zhang, Q., Peng, Z., & Zhou, B. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19809-0_7 · Cited by 25
SelfD: Self-Learning Large-Scale Driving Policies From the Web
Zhang, J., Zhu, R., & Ohn-Bar, E. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01680 · Cited by 25
MixGAIL: Autonomous Driving Using Demonstrations with Mixed Qualities
Lee, G., Kim, D., Oh, W., Lee, K., & Oh, S. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341104 · Cited by 23
Carl-Lead: Lidar-based End-to-End Autonomous Driving with Contrastive Deep Reinforcement LearningPreprint
Cai, P., Wang, S., Wang, H., & Liu, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.08473 · Cited by 23
Cited by
DriveGPT4: Interpretable End-to-End Autonomous Driving Via Large Language Model
Xu, Z., Zhang, Y., Xie, E., Zhao, Z., Guo, Y., Wong, K.-Y. K., Li, Z., & Zhao, H. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3440097 · Cited by 443
Atakishiyev, S., Salameh, M., Yao, H., & Goebel, R. (2021) · arXiv · doi:10.48550/arxiv.2112.11561 · Cited by 336
Atakishiyev, S., Salameh, M., Yao, H., & Goebel, R. (2024) · IEEE Access · doi:10.1109/access.2024.3431437 · Cited by 323
Recent Advancements in End-to-End Autonomous Driving Using Deep Learning: A Survey
Chib, P. S., & Singh, P. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3318070 · Cited by 301
VLAAD: Vision and Language Assistant for Autonomous Driving
Park, S., Lee, M., Kang, J., Choi, H., Park, Y., Cho, J., Lee, A., & Kim, D. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00107 · Cited by 49
Metadata
- Authors
- Chen, L., Wu, P., Chitta, K., Jaeger, B., Geiger, A., & Li, H.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2024-12
- Volume
- 46
- Issue
- 12
- Pages
- 10164-10183
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W4401109681
- Cited by
- 610
- References
- 214
- FWCI
- 64.76
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.