End-to-End Driving Via Conditional Imitation Learning
Open accessAbstract
Deep networks trained on demonstrations of human driving have learned to follow roads and avoid obstacles. However, driving policies trained via imitation learning cannot be controlled at test time. A vehicle trained end-to-end to imitate an expert cannot be guided to take a specific turn at an upcoming intersection. This limits the utility of such systems. We propose to condition imitation learning on high-level command input. At test time, the learned driving policy functions as a chauffeur that handles sensorimotor coordination but continues to respond to navigational commands. We evaluate different architectures for conditional imitation learning in vision-based driving. We conduct experiments in realistic three-dimensional simulations of urban driving and on a 1/5 scale robotic truck that is trained to drive in a residential area. Both systems drive based on visual input yet remain responsive to high-level navigational commands.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
IL using real-world demonstration data has become one of the most popular and robust control algorithms in research and industry.
“In this era, IL using real-world demonstration data has become one of the most popular and robust control algorithms in research and industry [59,60].”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [60]
The Introduction (p. 1) describes imitation learning from human driving demonstrations as a promising approach receiving renewed interest, and Related Work (p. 2) lists applications across several tasks. The paper does not establish that it is among the most popular control algorithms in research and industry; Training Data Distribution (p. 4) also cautions that policies trained only on natural demonstrations are typically unstable, rather than broadly robust. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
CARLA: An open urban driving simulator
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017) · arXiv · doi:10.48550/arxiv.1711.03938 · Cited by 3,718 · see citing works
Self-citationA survey of robot learning from demonstration
Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2008.10.024 · Cited by 3,227 · see citing works
End to End Learning for Self-Driving Cars
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016) · arXiv · doi:10.48550/arxiv.1604.07316 · Cited by 3,198 · see citing works
Reinforcement learning in robotics: A survey
Kober, J., Bagnell, J. A., & Peters, J. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913495721 · Cited by 3,160 · see citing works
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., & Singh, S. (1999) · Artificial Intelligence · doi:10.1016/s0004-3702(99)00052-1 · Cited by 3,119 · see citing works
A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles
Paden, B. A., Čáp, M., Yong, S. Z., Yershov, D. S., & Frazzoli, E. (2016) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2016.2578706 · Cited by 2,601 · see citing works
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941 · see citing works
DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving
Chen, C., Seff, A., Kornhauser, A., & Xiao, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.312 · Cited by 1,876 · see citing works
Maximum Entropy Inverse Reinforcement Learning
Ziebart, B. D., Maas, A. L., Bagnell, J. A., & Dey, A. K. (2018) · doi:10.1184/r1/6555512 · Cited by 1,868 · see citing works
ALVINN, an autonomous land vehicle in a neural network
Pomerleau, D. A. (2018) · doi:10.1184/r1/6603146 · Cited by 1,403 · see citing works
Recent Advances in Hierarchical Reinforcement Learning
Barto, A. G., & Mahadevan, S. (2003) · Discrete Event Dynamic Systems · doi:10.1023/a:1022140919877 · Cited by 1,043 · see citing works
Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic MotivationPreprint
Kulkarni, T. D., Narasimhan, K., Saeedi, A., & Tenenbaum, J. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.06057 · Cited by 895 · see citing works
Learning and generalization of motor skills by learning from demonstration
Pastor, P., Hoffmann, H., Asfour, T., & Schaal, S. (2009) · 2009 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.2009.5152385 · Cited by 702 · see citing works
Autonomous Helicopter Aerobatics through Apprenticeship Learning
Abbeel, P., Coates, A., & Ng, A. (2010) · Int. J. Robotics Res. · doi:10.1177/0278364910371999 · Cited by 682 · see citing works
Universal Value Function Approximators
Schaul, T., Horgan, D., Gregor, K., & Silver, D. (2015) · Cited by 677 · see citing works
A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots
Giusti, A., Guzzi, J., Ciresan, D. C., He, F.-L., Rodriguez, J. P., Fontana, F., Faessler, M., Forster, C., Schmidhuber, J., Caro, G. D., et al. (2016) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2015.2509024 · Cited by 665 · see citing works
Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation
Tellex, S., Kollar, T., Dickerson, S., Walter, M. R., Banerjee, A. G., Teller, S. J., & Roy, N. (2011) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v25i1.7979 · Cited by 654 · see citing works
Levine, S., & Koltun, V. (2013) · Cited by 481 · see citing works
Self-citation(2005) · neural information processing systems · Cited by 446 · see citing works
Learning monocular reactive UAV control in cluttered natural environments
Ross, S., Melik-Barkhudarov, N., Shankar, K. S., Wendel, A., Dey, D., Bagnell, J. A., & Hebert, M. (2013) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2013.6630809 · Cited by 410 · see citing works
Robot learning from demonstration by constructing skill trees
Konidaris, G., Kuindersma, S., Grupen, R. A., & Barto, A. G. (2011) · The International Journal of Robotics Research · doi:10.1177/0278364911428653 · Cited by 287 · see citing works
Ziebart, B. D., Maas, A. L., Dey, A. K., & Bagnell, J. A. (2008) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/1409635.1409678 · Cited by 271 · see citing works
Shared Autonomy via Hindsight Optimization
Javdani, S., Srinivasa, S. S., & Bagnell, A. (2015) · Robotics: Science and Systems · doi:10.15607/rss.2015.xi.032 · Cited by 169 · see citing works
Reinforcement learning to adjust parametrized motor primitives to new situations
Kober, J., Wilhelm, A., Öztop, E., & Peters, J. (2012) · Autonomous Robots · doi:10.1007/s10514-012-9290-3 · Cited by 163 · see citing works
DeepDriving: Learning Affordance for Direct Perception in Autonomous DrivingPreprint
Chen, C., Seff, A., Kornhauser, A. L., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.00256 · Cited by 154 · see citing works
Learning from Demonstration for Autonomous Navigation in Complex Unstructured Terrain
Silver, D., Bagnell, J. A., & Stentz, A. (2010) · The International Journal of Robotics Research · doi:10.1177/0278364910369715 · Cited by 142 · see citing works
Multi-task policy search for robotics
Deisenroth, M. P., Englert, P. A. J., Peters, J., & Fox, D. (2014) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2014.6907421 · Cited by 127 · see citing works
Query-Efficient Imitation Learning for End-to-End Simulated Driving
Zhang, J., & Cho, K. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10857 · Cited by 111 · see citing works
Imitation learning for locomotion and manipulation
Ratliff, N., Bagnell, J. A., & Srinivasa, S. S. (2007) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/ichr.2007.4813899 · Cited by 104 · see citing works
Learning Semantic Maps from Natural Language Descriptions
Walter, M. R., Hemachandra, S. M., Homberg, B. S., Tellex, S., & Teller, S. J. (2013) · Robotics: Science and Systems · doi:10.15607/rss.2013.ix.004 · Cited by 84 · see citing works
Learning from Unscripted Deictic Gesture and Language for Human-Robot Interactions
Matuszek, C., Bo, L., Zettlemoyer, L., & Fox, D. (2014) · Maryland Shared Open Access Repository (USMAI Consortium) · doi:10.13016/m2rn30b52 · Cited by 81 · see citing works
Learning models for following natural language directions in unknown environments
Hemachandra, S. M., Duvallet, F., Howard, T. M., Roy, N., Stentz, A., & Walter, M. R. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139984 · Cited by 75 · see citing works
Recent Advances in Hierarchical Reinforcement Learning
Barto, A. G., & Mahadevan, S. (2003) · Cited by 74 · see citing works
Learning Parameterized SkillsPreprint
Da Silva, B., Konidaris, G., & Barto, A. G. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6398 · Cited by 64 · see citing works
Learning to Act by Predicting the FuturePreprint
Dosovitskiy, A., & Koltun, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01779 · Cited by 60 · see citing works
Self-citationModel-based imitation learning by probabilistic trajectory matching
Englert, P. A. J., Paraschos, A., Peters, J., & Deisenroth, M. P. (2013) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2013.6630832 · Cited by 50 · see citing works
Real-time natural language corrections for assistive robotic manipulators
Broad, A., Arkin, J., Ratliff, N., Howard, T. M., & Argall, B. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364917706418 · Cited by 26 · see citing works
Learning Models for Following Natural Language Directions in Unknown EnvironmentsPreprint
Hemachandra, S., Duvallet, F., Howard, T. M., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.05079 · Cited by 15 · see citing works
Franke, U. (2017) · doi:10.1002/9781118868065.ch2 · Cited by 10 · see citing works
Flagged first, then by citations
1 of the 41 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- [5][5] A. Broad, J. Arkin, N. Ratliff, T. Howard, and B. Argall. Realtime natural language corrections for assistive robotic manipulators. International Journal of Robotics Research , 2017. -
- [21][21] M. Laskey, A. Dragan, J. Lee, K. Goldberg, and R. Fox. Dart: Optimizing noise injection in imitation learning. In Conference on Robot Learning (CoRL) , 2017. -
Cited by
Sorted by citations
CARLA: An open urban driving simulator
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017) · arXiv · doi:10.48550/arxiv.1711.03938 · Cited by 3,718 · see citing works
Self-citationGibson Env: Real-World Perception for Embodied Agents
Xia, F., Zamir, A. R., He, Z., Sax, A., Malik, J., & Savarese, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00945 · Cited by 763 · see citing works
ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst
Bansal, M., Krizhevsky, A., & Ogale, A. S. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.031 · Cited by 743 · see citing works
Multimodal end-to-end autonomous driving
Xiao, Y., Codevilla, F., Gurram, A., Urfalioglu, O., & Lopez, A. M. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3013234 · Cited by 319 · see citing works
Self-citationLi, G., Yang, Y., Li, S., Qu, X., Lyu, N., & Li, S. E. (2022) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2021.103452 · Cited by 308 · see citing works
A review of motion planning algorithms for intelligent robots
Zhou, C., Huang, B., & Fränti, P. (2021) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-021-01867-z · Cited by 273 · see citing works
Learning by CheatingPreprint
Chen, D., Zhou, B., Koltun, V., & Krähenbühl, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.12294 · Cited by 256 · see citing works
Self-citationMP3: A Unified Model to Map, Perceive, Predict and Plan
Casas, S., Sadat, A., & Urtasun, R. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01417 · Cited by 228 · see citing works
Explainability of Deep Vision-Based Autonomous Driving Systems: Review and Challenges
Zablocki, É., Ben-Younes, H., Pérez, P., & Cord, M. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01657-x · Cited by 227 · see citing works
Wu, J., Huang, Z., Hu, Z., & Lv, C. (2023) · Engineering · doi:10.1016/j.eng.2022.05.017 · Cited by 205 · see citing works
End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners
Hecker, S., Dai, D., & Van Gool, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_27 · Cited by 191 · see citing works
Urban Driving with Conditional Imitation Learning
Hawke, J., Shen, R., Gurău, C., Sharma, S., Reda, D., Nikolov, N., Mazur, P., Micklethwaite, S., Griffiths, N., Shah, A., et al. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9197408 · Cited by 146 · see citing works
Variational End-to-End Navigation and Localization
Amini, A., Rosman, G., Karaman, S., & Rus, D. L. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793579 · Cited by 124 · see citing works
Navigating to objects in the real world
Gervet, T., Chintala, S., Batra, D., Malik, J., & Chaplot, D. S. (2023) · Science Robotics · doi:10.1126/scirobotics.adf6991 · Cited by 125 · see citing works
VR-Goggles for Robots: Real-to-Sim Domain Adaptation for Visual Control
Zhang, J., Tai, L., Yun, P., Xiong, Y., Liu, M., Boedecker, J., & Burgard, W. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2894216 · Cited by 109 · see citing works
“s experiments are relateively expensive, we only compare NoGoggles and VR-Goggles in the outdoor autonomous driving scenario. We take the driving policy trained through conditional imitation learning [28] as in Section IV-B. This policy takes as inputs the first person view RGB image and a highlevel command, which fall…”
MethodLikmeta, A., Metelli, A. M., Tirinzoni, A., Giol, R., Restelli, M., & Romano, D. (2020) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2020.103568 · Cited by 106 · see citing works
On Offline Evaluation of Vision-Based Driving Models
Codevilla, F., López, A. M., Koltun, V., & Dosovitskiy, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01267-0_15 · Cited by 103 · see citing works
Self-citationA Survey of Deep Learning Applications to Autonomous Vehicle Control
Kuutti, S., Bowden, R., Jin, Y., Barber, P., & Fallah, S. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2962338 · Cited by 96 · see citing works
Colas, C., Karch, T., Sigaud, O., & Oudeyer, P. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13554 · Cited by 101 · see citing works
Improvisation through Physical Understanding: Using Novel Objects As Tools with Visual Foresight
Xie, A., Ebert, F., Levine, S., & Finn, C. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.001 · Cited by 66 · see citing works
Explaining Autonomous Driving by Learning End-to-End Visual Attention
Cultrera, L., Seidenari, L., Becattini, F., Pala, P., & Bimbo, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00178 · Cited by 66 · see citing works
Run-Time Monitoring of Machine Learning for Robotic Perception: A Survey of Emerging Trends
Rahman, Q. M., Corke, P., & Dayoub, F. (2021) · IEEE Access · doi:10.1109/access.2021.3055015 · Cited by 65 · see citing works
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to ImitationPreprint
Tai, L., Zhang, J., Liu, M., Boedecker, J., & Burgard, W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.07139 · Cited by 95 · see citing works
Reinforcement Learning based Control of Imitative Policies for Near-Accident Driving
Cao, Z., Bıyık, E., Wang, W. Z., Raventos, A., Gaidon, A., Rosman, G., & Sadigh, D. (2020) · Robotics: Science and Systems · doi:10.15607/rss.2020.xvi.039 · Cited by 65 · see citing works
Exploring the Limitations of Behavior Cloning for Autonomous Driving
Codevilla, F., Santana, E., López, A. M., & Gaidon, A. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00942 · Cited by 562 · see citing works
Self-citationWang, X., Qi, X., Wang, P., & Yang, J. (2021) · Autonomous Intelligent Systems · doi:10.1007/s43684-021-00015-x · Cited by 55 · see citing works
ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints
Shah, D., & Levine, S. (2022) · Robotics: Science and Systems · doi:10.15607/rss.2022.xviii.019 · Cited by 54 · see citing works
Vision-Based Goal-Conditioned Policies for Underwater Navigation in the Presence of Obstacles
Manderson, T., Higuera, J. C. G., Wapnick, S., Tremblay, J.-F., Shkurti, F., Meger, D., & Dudek, G. (2020) · Robotics: Science and Systems · doi:10.15607/rss.2020.xvi.048 · Cited by 50 · see citing works
Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research
Gulino, C., Fu, J., Luo, W., Tucker, G., Bronstein, E., Lu, Y., Harb, J., Pan, X., Wang, Y., Chen, X., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.08710 · Cited by 48 · see citing works
Modified DDPG car-following model with a real-world human driving experience with CARLA simulator
Li, D., & Okhrin, O. (2023) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2022.103987 · Cited by 51 · see citing works
de Morais, G. A. P., Marcos, L. B., Bueno, J. N. A. D., de Resende, N. F., Terra, M. H., & Grassi, V. (2020) · Control Engineering Practice · doi:10.1016/j.conengprac.2020.104630 · Cited by 47 · see citing works
Measurement-Robust Control Barrier Functions: Certainty in Safety with Uncertainty in State
Cosner, R. K., Singletary, A., Taylor, A. J., Molnár, T. G., Bouman, K. L., & Ames, A. D. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636584 · Cited by 46 · see citing works
Advisable Learning for Self-Driving Vehicles by Internalizing Observation-to-Action Rules
Kim, J., Moon, S., Rohrbach, A., Darrell, T., & Canny, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00968 · Cited by 45 · see citing works
CADRE: A Cascade Deep Reinforcement Learning Framework for Vision-Based Autonomous Urban Driving
Zhao, Y., Wu, K., Xu, Z., Che, Z., Lu, Q., Tang, J., & Liu, C. H. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20259 · Cited by 46 · see citing works
VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigation
Karnan, H., Warnell, G., Xiao, X., & Stone, P. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9812316 · Cited by 42 · see citing works
Zhou, J., Wang, R., Liu, X., Jiang, Y., Jiang, S., Tao, J., Miao, J., & Song, S. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636795 · Cited by 40 · see citing works
On the Geometry of Adversarial ExamplesPreprint
Khoury, M., & Hadfield-Menell, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00525 · Cited by 40 · see citing works
On failures of RGB cameras and their effects in autonomous driving applications
Secci, F., & Ceccarelli, A. (2020) · 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE) · doi:10.1109/issre5003.2020.00011 · Cited by 42 · see citing works
A deep learning gated architecture for UGV navigation robust to sensor failures
Patel, N., Choromanska, A., Krishnamurthy, P., & Khorrami, F. (2019) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2019.03.001 · Cited by 36 · see citing works
Autonomous Vehicle Control: End-to-End Learning in Simulated Urban Environments
Haavaldsen, H., Aasboe, M., & Lindseth, F. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-35664-4_4 · Cited by 36 · see citing works
Jha, S., Tsai, T. K., Hari, S. K. S., Sullivan, M. B., Kalbarczyk, Z., Keckler, S. W., & Iyer, R. K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.01024 · Cited by 34 · see citing works
Li, Z., Motoyoshi, T., Sasaki, K., Ogata, T., & Sugano, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.11100 · Cited by 33 · see citing works
End to End Vehicle Lateral Control Using a Single Fisheye Camera
Toromanoff, M., Wirbel, É., Wilhelm, F., Vejarano, C., Perrotton, X., & Moutarde, F. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594090 · Cited by 32 · see citing works
OpenBot: Turning Smartphones into Robots
Muller, M., & Koltun, V. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561788 · Cited by 30 · see citing works
Self-citationSoftware-Defined Active LiDARs for Autonomous Driving: A Parallel Intelligence-Based Adaptive Model
Liu, Y., Sun, B., Tian, Y., Wang, X., Zhu, Y., Huai, R., & Shen, Y. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3289540 · Cited by 29 · see citing works
Onishi, T., Motoyoshi, T., Suga, Y., Hiroki, M. O. R. I., & Ogata, T. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852322 · Cited by 24 · see citing works
Toward explainable and advisable model for self‐driving cars
Kim, J., Rohrbach, A., Akata, Z., Moon, S., Misu, T., Chen, Y., Darrell, T., & Canny, J. (2021) · Applied AI Letters · doi:10.1002/ail2.56 · Cited by 24 · see citing works
PILOT: Efficient Planning by Imitation Learning and Optimisation for Safe Autonomous Driving
Pulver, H., Eiras, F., Carozza, L., Hawasly, M., Albrecht, S. V., & Ramamoorthy, S. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636862 · Cited by 24 · see citing works
End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning
Capasso, A. P., Maramotti, P., Dell’Eva, A., & Broggi, A. (2021) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv48863.2021.9575135 · Cited by 23 · see citing works
Lesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., & Díaz-Rodríguez, N. (2019) · Information Fusion · doi:10.1016/j.inffus.2019.12.004 · Cited by 531 · see citing works
A novel reservoir simulation model based on physics informed neural networks
Liu, A. C. Y., Li, J., Bi, J., Chen, Z., Wang, Y., Lu, C., Jin, Y., & Lin, B. (2024) · Physics of Fluids · doi:10.1063/5.0239376 · Cited by 22 · see citing works
Learning to Plan with Logical Automata
Arak, B., Vodrahalli, K., Leech, T., Vasile, C.-I., Donahue, M., & Rus, D. L. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.064 · Cited by 22 · see citing works
Simple and Effective Synthesis of Indoor 3D Scenes
Koh, J. Y., Agrawal, H., Batra, D., Tucker, R. P., Waters, A., Lee, H., Yang, Y., Baldridge, J., & Anderson, P. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25199 · Cited by 22 · see citing works
PLOP: Probabilistic poLynomial Objects trajectory Planning for autonomous drivingPreprint
Buhet, T., Wirbel, É., Bursuc, A., & Perrotton, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.08744 · Cited by 20 · see citing works
Kadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 19 · see citing works
“In this era, IL using real-world demonstration data has become one of the most popular and robust control algorithms in research and industry [59,60].”
MethodSADA: Semantic Adversarial Diagnostic Attacks for Autonomous Applications
Hamdi, A., Mueller, M., & Ghanem, B. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6722 · Cited by 20 · see citing works
Wang, Y., Zheng, K., Tian, D., Duan, X., & Zhou, J. (2021) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1900637 · Cited by 20 · see citing works
Driving Through Ghosts: Behavioral Cloning with False Positives
Bühler, A., Gaidon, A., Cramariuc, A., Ambruş, R., Rosman, G., & Burgard, W. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340639 · Cited by 18 · see citing works
Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations
Mandlekar, A., Xu, D., Martín-Martín, R., Savarese, S., & Fei-Fei, L. (2020) · arXiv · doi:10.48550/arxiv.2003.06085 · Cited by 17 · see citing works
Chen, J., Wu, T., Shi, M., & Jiang, W. (2020) · Sensors · doi:10.3390/s20195626 · Cited by 16 · see citing works
Uncertainties in Neural Networks : A System Identification Approach
Malmström, M. (2021) · Linköping studies in science and technology. Thesis · doi:10.3384/lic.diva-174720 · Cited by 16 · see citing works
Multi-Task End-to-End Self-Driving Architecture for CAV Platoons
Huch, S., Öngel, A., Betz, J., & Lienkamp, M. (2021) · Sensors · doi:10.3390/s21041039 · Cited by 15 · see citing works
Learning Interaction-aware Guidance Policies for Motion Planning in Dense Traffic ScenariosPreprint
Brito, B., Agarwal, A., & Alonso–Mora, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.04538 · Cited by 15 · see citing works
Simulator Coupled with Distributed Co-Simulation Protocol for Automated Driving Tests
Meyer, M., Sauter, L., Granrath, C., Hadj-amor, H., & Andert, J. (2021) · Automotive Innovation · doi:10.1007/s42154-021-00161-1 · Cited by 14 · see citing works
Adversarial Training with Voronoi ConstraintsPreprint
Khoury, M., & Hadfield-Menell, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.01019 · Cited by 14 · see citing works
RLfOLD: Reinforcement Learning from Online Demonstrations in Urban Autonomous Driving
Coelho, D., Oliveira, M., & Santos, V. M. F. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i10.29049 · Cited by 13 · see citing works
Wang, S., Fujii, H., & Yoshimura, S. (2022) · Physica A Statistical Mechanics and its Applications · doi:10.1016/j.physa.2022.128172 · Cited by 13 · see citing works
Deep 3D Dynamic Object Detection towards Successful and Safe Navigation for Full Autonomous Driving
Wijesekara, P. A. D. S. N. (2022) · The Open Transportation Journal · doi:10.2174/18744478-v16-e2208191 · Cited by 13 · see citing works
CCIL: Context-conditioned imitation learning for urban driving
Guo, K., Jing, W., Chen, J., & Pan, J. (2023) · Robotics: Science and Systems · doi:10.15607/rss.2023.xix.101 · Cited by 13 · see citing works
ObserveNet Control: A Vision-Dynamics Learning Approach to Predictive Control in Autonomous Vehicles
Ginerică, C., Zaha, M. V., Gogianu, F., Buşoniu, L., Trăsnea, B., & Grigorescu, S. M. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3096157 · Cited by 13 · see citing works
Robotics: Science and Systems XV
(2019) · doi:10.15607/rss.2019.xv · Cited by 13 · see citing works
Learning Driving Models with a Surround-View Camera System and a Route Planner.Preprint
Hecker, S., Dai, D., & Gool, L. V. (2018) · arXiv (Cornell University) · Cited by 12 · see citing works
Reinforced Intelligence Through Active Interaction in Real World: A Survey on Embodied AI
Kwon, W., Baek, S., Baek, J., Shin, W., Gwak, M., Park, P., & Lee, S. (2025) · International Journal of Control Automation and Systems · doi:10.1007/s12555-025-0127-1 · Cited by 13 · see citing works
Simitate: A Hybrid Imitation Learning Benchmark
Memmesheimer, R., Kramer, I., Seib, V., & Paulus, D. W. R. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968029 · Cited by 11 · see citing works
Deep Reinforcement Learning for a Self-Driving Vehicle Operating Solely on Visual Information
Audinys, R., Šlikas, Ž., Radkevičius, J., Šutas, M., & Ostreika, A. (2025) · Electronics · doi:10.3390/electronics14050825 · Cited by 10 · see citing works
End-to-end driving simulation via angle branched networkPreprint
Wang, Q., Chen, L. F., & Tian, W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.07545 · Cited by 10 · see citing works
Carl-Lead: Lidar-based End-to-End Autonomous Driving with Contrastive Deep Reinforcement LearningPreprint
Cai, P., Wang, S., Wang, H., & Liu, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.08473 · Cited by 10 · see citing works
EARLY FUSION for Goal Directed Robotic Vision
Walsman, A., Bisk, Y., Gabriel, S., Misra, D. K., Artzi, Y., Choi, Y., & Fox, D. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968165 · Cited by 9 · see citing works
Human-in-the-Loop Learning Methods Toward Safe DL-Based Autonomous Systems: A Review
Rajendran, P. T., Espinoza, H., Delaborde, A., & Mraidha, C. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-83906-2_20 · Cited by 9 · see citing works
Multi-Agent Advisor Q-Learning
Subramanian, S. G., Taylor, M. E., Larson, K. S., & Crowley, M. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13445 · Cited by 9 · see citing works
Hybrid Imitative Planning with Geometric and Predictive Costs in Off-road Environments
Dashora, N., Shin, D., Shah, D., Leopold, H. A., Fan, D. D., Agha–mohammadi, A., Rhinehart, N., & Levine, S. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811540 · Cited by 8 · see citing works
A review of motion planning algorithms for intelligent roboticsPreprint
Zhou, C., Huang, B., & Fränti, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.02376 · Cited by 8 · see citing works
Practical Self-Driving Cars: Survey of the State-of-the-ArtPreprint
Saha, D., & De, S. (2022) · Preprints.org · doi:10.20944/preprints202202.0123.v1 · Cited by 8 · see citing works
Human-guided reinforcement learning: methodology and application to autonomous driving
Wu, J. (2023) · doi:10.32657/10356/169780 · Cited by 8 · see citing works
DeepGoal: Learning to drive with driving intention from human control demonstration
Ma, H., Wang, Y., Xiong, R., Kodagoda, S., & Tang, L. (2020) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2020.103477 · Cited by 7 · see citing works
Arizala, A., Lattarulo, R. A., Zubizarreta, A., & Perez, J. (2021) · International Conference on System Theory, Control and Computing (ICSTCC) · doi:10.1109/icstcc52150.2021.9607221 · Cited by 7 · see citing works
Learning from Interventions Using Hierarchical Policies for Safe Learning
Bi, J., Dhiman, V., Xiao, T., & Xu, C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i06.6602 · Cited by 7 · see citing works
Araki, B., Vodrahalli, K., Leech, T., Vasile, C.-I., Donahue, M., & Rus, D. L. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i06.6559 · Cited by 6 · see citing works
Learning to Drive in the NGSIM Simulator Using Proximal Policy Optimization
Zhou, Y., & Chen, Y. (2023) · Journal of Advanced Transportation · doi:10.1155/2023/4127486 · Cited by 6 · see citing works
MultiNet: Multi-Modal Multi-Task Learning for Autonomous DrivingPreprint
Chowdhuri, S., Pankaj, T., & Zipser, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.05581 · Cited by 9 · see citing works
Action-Based Representation Learning for Autonomous DrivingPreprint
Xiao, Y., Codevilla, F., Pal, C., & López, A. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.09417 · Cited by 6 · see citing works
Self-citationImitation Learning of Path-Planned Driving Using Disparity-Depth Images
Hornauer, S., Zipser, K., & Yu, S. X. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11021-5_33 · Cited by 5 · see citing works
Behavior Planning for Autonomous Driving: Methodologies, Applications, and Future Orientation
Kassem, N. (2023) · MSA Engineering Journal · doi:10.21608/msaeng.2023.291918 · Cited by 5 · see citing works
ARC: Adversarially Robust Control Policies for Autonomous Vehicles
Kuutti, S., Fallah, S., & Bowden, R. (2021) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc48978.2021.9564579 · Cited by 5 · see citing works
Driving Behavior Modeling Based on Consistent Variable Selection in a PWARX Model
Nwadiuto, J., Okuda, H., & Suzuki, T. (2021) · Applied Sciences · doi:10.3390/app11114938 · Cited by 5 · see citing works
Autonomous UAV Navigation with Domain Adaptation.Preprint
Yoo, J., Hong, Y., & Yoon, S. (2017) · arXiv (Cornell University) · Cited by 5 · see citing works
Imitative Planning using Conditional Normalizing FlowPreprint
Agarwal, S., Sikchi, H., Gulino, C., Wilkinson, E., & Gautam, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.16162 · Cited by 5 · see citing works
Mei, X., Sun, Y., Chen, Y., Liu, C., & Liu, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.00675 · Cited by 5 · see citing works
Arnéz, F., Espinoza, H., Radermacher, A., & Terrier, F. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-83906-2_17 · Cited by 4 · see citing works
Visual Odometry for Self-Driving with Multihypothesis and Network Prediction
Chen, Y., Mei, Y., & Wan, S.-F. (2021) · Mathematical Problems in Engineering · doi:10.1155/2021/1930881 · Cited by 5 · see citing works
- Authors
- Codevilla, F., Muller, M., Lopez, A., Koltun, V., & Dosovitskiy, A.
- Affiliations
- Universitat Autònoma de Barcelona (ES)
- King Abdullah University of Science and Technology (SA)
- Intel (United States) (US)
- Venue
- 2018 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2018-05
- Pages
- 4693-4700
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2760878839
- Cited by
- 129
- References
- 41
- Citations by year
- 2025: 3 · 2024: 9 · 2023: 11 · 2022: 18 · 2021: 29 · 2020: 22 · 2019: 23 · 2018: 10 · 2017: 3 · 2016: 1
- FWCI
- 21.53
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- King Abdullah University of Science and Technology Repository (King Abdullah University of Science and Technology) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Centres de Recerca de Catalunya
- Generalitat de Catalunya
- Topics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- end-to-end driving, vision-based driving, autonomous vehicles, sensorimotor coordination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
