The DOI matches its Crossref record; 109 of the 109 references it cites were found in a registry or an index.
TransFuser: Imitation With Transformer-Based Sensor Fusion for Autonomous Driving
Open accessAbstract
How should we integrate representations from complementary sensors for autonomous driving? Geometry-based fusion has shown promise for perception (e.g., object detection, motion forecasting). However, in the context of end-to-end driving, we find that imitation learning based on existing sensor fusion methods underperforms in complex driving scenarios with a high density of dynamic agents. Therefore, we propose TransFuser, a mechanism to integrate image and LiDAR representations using self-attention. Our approach uses transformer modules at multiple resolutions to fuse perspective view and bird's eye view feature maps. We experimentally validate its efficacy on a challenging new benchmark with long routes and dense traffic, as well as the official leaderboard of the CARLA urban driving simulator. At the time of submission, TransFuser outperforms all prior work on the CARLA leaderboard in terms of driving score by a large margin. Compared to geometry-based fusion, TransFuser reduces the average collisions per kilometer by 48%.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
Self-citationDecoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01167 · Cited by 8,494
PointPillars: Fast Encoders for Object Detection From Point Clouds
Lang, A. H., Vora, S., Caesar, H., Zhou, L., Yang, J., & Beijbom, O. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01298 · Cited by 4,673
CARLA: An open urban driving simulator
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017) · arXiv · doi:10.48550/arxiv.1711.03938 · Cited by 3,718
Multi-view 3D Object Detection Network for Autonomous Driving
Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.691 · Cited by 3,450
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.1078 · Cited by 3,317
Frustum PointNets for 3D Object Detection from RGB-D Data
Qi, C. R., Liu, W., Wu, C., Su, H., & Guibas, L. J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00102 · Cited by 2,655
Ros, G., Sellart, L., Materzynska, J., Vazquez, D., & Lopez, A. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.352 · Cited by 2,338 · cited 1×
“In practice, the information missing in the LiDAR must be recovered from other sensors on the vehicle, such as RGB cameras [5], [6], [7], [8], [9], [10], [11], [12], [13].”
Adaptive Control Processes: A Guided Tour.
Page, E. S., & Bellman, R. (1962) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2343225 · Cited by 2,185
Designing Network Design Spaces
Radosavovic, I., Kosaraju, R. P., Girshick, R., He, K., & Dollar, P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01044 · Cited by 2,084
A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Ross, S., Gordon, G. J., & Bagnell, J. A. (2018) · Carnegie Mellon University · doi:10.1184/r1/6550949 · Cited by 1,941
Playing for Data: Ground Truth from Computer Games
Richter, S. R., Vineet, V., Roth, S., & Koltun, V. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_7 · Cited by 1,815
Joint 3D Proposal Generation and Object Detection from View Aggregation
Ku, J., Mozifian, M., Lee, J., Harakeh, A., & Waslander, S. L. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594049 · Cited by 1,697
Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Yu, Q., & Dai, J. (2022) · arXiv · doi:10.48550/arxiv.2203.17270 · Cited by 1,419
Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation
Liu, Z., Tang, H., Amini, A., Yang, X., Mao, H., Rus, D. L., & Han, S. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160968 · Cited by 1,307
PointPainting: Sequential Fusion for 3D Object Detection
Vora, S., Lang, A. H., Helou, B., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00466 · Cited by 1,236 · cited 3×
“Most sensor fusion works consider perception tasks, e.g. object detection [14], [15], [16], [18], [19], [20], [21], [22], [23], [47], [48], [49], [50], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60] and motion forecasting [24], [25], [26], [27], [28], [29], [30], [49], [61], [62].”
3DSSD: Point-Based 3D Single Stage Object Detector
Yang, Z., Sun, Y., Liu, S., & Jia, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01105 · Cited by 1,176
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · arXiv · doi:10.48550/arxiv.1604.01685 · Cited by 1,174
VideoBERT: A Joint Model for Video and Language Representation Learning
Sun, C., Myers, A., Vondrick, C., Murphy, K., & Schmid, C. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00756 · Cited by 1,172
3D Bounding Box Estimation Using Deep Learning and Geometry
Mousavian, A., Anguelov, D., Flynn, J., & Kosecka, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.597 · Cited by 1,129
VirtualWorlds as Proxy for Multi-object Tracking Analysis
Gaidon, A., Wang, Q., Cabon, Y., & Vig, E. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.470 · Cited by 1,079
SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints
Sadeghian, A., Kosaraju, V., Sadeghian, A., Hirose, N., Rezatofighi, H., & Savarese, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00144 · Cited by 1,065
TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers
Bai, X., Hu, Z., Zhu, X., Huang, Q., Chen, Y., Fu, H., & Tai, C.-L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00116 · Cited by 990 · cited 1×
“Most sensor fusion works consider perception tasks, e.g. object detection [14], [15], [16], [18], [19], [20], [21], [22], [23], [47], [48], [49], [50], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60] and motion forecasting [24], [25], [26], [27], [28], [29], [30], [49], [61], [62].”
Deep Continuous Fusion for Multi-sensor 3D Object Detection
Liang, M., Yang, B., Wang, S., & Urtasun, R. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_39 · Cited by 974
End-to-End Learning of Driving Models from Large-Scale Video Datasets
Xu, H., Gao, Y., Yu, F., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.376 · Cited by 884
PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation
Xu, D., Anguelov, D., & Jain, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00033 · Cited by 856
ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst
Bansal, M., Krizhevsky, A., & Ogale, A. (2019) · Robotics: Science and Systems XV · doi:10.15607/rss.2019.xv.031 · Cited by 731
Multi-Task Multi-Sensor Fusion for 3D Object Detection
Liang, M., Yang, B., Chen, Y., Hu, R., & Urtasun, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00752 · Cited by 723
(2018) · arXiv (Cornell University) · Cited by 679
STGAT: Modeling Spatial-Temporal Interactions for Human Trajectory Prediction
Huang, Y., Bi, H., Li, Z., Mao, T., & Wang, Z. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00637 · Cited by 658
Luo, W., Yang, B., & Urtasun, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00376 · Cited by 644
AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting
Yuan, Y., Weng, X., Ou, Y., & Kitani, K. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00967 · Cited by 631
Multi-Modal Fusion Transformer for End-to-End Autonomous Driving
Prakash, A., Chitta, K., & Geiger, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00700 · Cited by 584
Self-citationCLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection
Pang, S., Morris, D., & Radha, H. (2020) · 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341791 · Cited by 563
Exploring the Limitations of Behavior Cloning for Autonomous Driving
Codevilla, F., Santana, E., Lopez, A., & Gaidon, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00942 · Cited by 509
The Trajectron: Probabilistic Multi-Agent Trajectory Modeling With Dynamic Spatiotemporal Graphs
Ivanovic, B., & Pavone, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00246 · Cited by 467
LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving
Meyer, G. P., Laddha, A., Kee, E., Vallespi-Gonzalez, C., & Wellington, C. K. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01296 · Cited by 374
Towards Large-Pose Face Frontalization in the Wild
Yin, X., Yu, X., Sohn, K., Liu, X., & Chandraker, M. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.430 · Cited by 353
PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings
Rhinehart, N., Mcallister, R., Kitani, K., & Levine, S. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00291 · Cited by 345
FUTR3D: A Unified Sensor Fusion Framework for 3D Detection
Chen, X., Zhang, T., Wang, Y., Wang, Y., & Zhao, H. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00022 · Cited by 328
Multimodal end-to-end autonomous driving
Xiao, Y., Codevilla, F., Gurram, A., Urfalioglu, O., & Lopez, A. M. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3013234 · Cited by 319
Cross-view Transformers for real-time Map-view Semantic Segmentation
Zhou, B., & Krahenbuhl, P. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01339 · Cited by 311
ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text DataPreprint
Qi, D., Su, L., Song, J. L., Cui, E., Bharti, T., & Sacheti, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.07966 · Cited by 294
Improving 3D Object Detection with Channel-wise Transformer
Shenga, H., Cai, S., Liu, Y., Deng, B., Huang, J., Hua, X.-S., & Zhao, M.-J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00274 · Cited by 282
Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior
Rempe, D., Philion, J., Guibas, L. J., Fidler, S., & Litany, O. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01679 · Cited by 265
Xie, E., Yu, Z., Zhou, D., Philion, J., Anandkumar, A., Fidler, S., Luo, P., & Álvarez, J. M. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.05088 · Cited by 264
Self-citationDepth Completion From Sparse LiDAR Data With Depth-Normal Constraints
Xu, Y., Zhu, X., Shi, J., Zhang, G., Bao, H., & Li, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00290 · Cited by 246
BEVSegFormer: Bird's Eye View Semantic Segmentation From Arbitrary Camera Rigs
Peng, L., Chen, Z., Fu, Z., Liang, P., & Cheng, E. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00588 · Cited by 231
EPNet++: Cascade Bi-Directional Fusion for Multi-Modal 3D Object Detection
Liu, Z., Huang, T., Li, B., Chen, X., Wang, X., & Bai, X. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3228806 · Cited by 228
NEAT: Neural Attention Fields for End-to-End Autonomous Driving
Chitta, K., Prakash, A., & Geiger, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01550 · Cited by 222
Self-citationEnd-to-End Urban Driving by Imitating a Reinforcement Learning Coach
Zhang, Z., Liniger, A., Dai, D., Yu, F., & Van Gool, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01494 · Cited by 218
End-to-End Model-Free Reinforcement Learning for Urban Driving Using Implicit Affordances
Toromanoff, M., Wirbel, E., & Moutarde, F. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00718 · Cited by 208
CAT-Det: Contrastively Augmented Transformer for Multimodal 3D Object Detection
Zhang, Y., Chen, J., & Huang, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00098 · Cited by 200
Temporal-Channel Transformer for 3D Lidar-Based Video Object Detection for Autonomous Driving
Yuan, Z., Song, X., Bai, L., Wang, Z., & Ouyang, W. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3082763 · Cited by 196
Learning to drive from a world on rails
Chen, D., Koltun, V., & Krähenbühl, P. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01530 · Cited by 192
Chen, D., & Krahenbuhl, P. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01671 · Cited by 189
Huang, Z., Lv, C., Xing, Y., & Wu, J. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3003121 · Cited by 188
KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients
Hanselmann, N., Renz, K., Chitta, K., Bhattacharyya, A., & Geiger, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19839-7_20 · Cited by 187
Self-citationAdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles
Wang, J., Pun, A., Tu, J., Manivasagam, S., Sadat, A., Casas, S., Ren, M., & Urtasun, R. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00978 · Cited by 186
Saha, A., Méndez, O., Russell, C., & Bowden, R. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811901 · Cited by 186
Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representations
Sadat, A., Casas, S., Ren, M., Wu, X., Dhawan, P., & Urtasun, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58592-1_25 · Cited by 183
PnPNet: End-to-End Perception and Prediction With Tracking in the Loop
Liang, M., Yang, B., Zeng, W., Chen, Y., Hu, R., Casas, S., & Urtasun, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01157 · Cited by 182
LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving
Cui, A., Casas, S., Sadat, A., Liao, R., & Urtasun, R. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01580 · Cited by 180
SpAGNN: Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data
Casas, S., Gulino, C., Liao, R., & Urtasun, R. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9196697 · Cited by 176
Li, M., Wang, Y.-X., & Ramanan, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58536-5_28 · Cited by 167
End-to-end Contextual Perception and Prediction with Interaction Transformer
Li, L. L., Yang, B., Liang, M., Zeng, W., Ren, M., Segal, S., & Urtasun, R. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341392 · Cited by 164
Semi-Parametric Image Synthesis
Qi, X., Chen, Q., Jia, J., & Koltun, V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00918 · Cited by 160
Sensor Fusion for Joint 3D Object Detection and Semantic Segmentation
Meyer, G. P., Charland, J., Hegde, D., Laddha, A., & Vallespi-Gonzalez, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00162 · Cited by 158
Brain-Inspired Cognitive Model With Attention for Self-Driving Cars
Chen, S., Zhang, S., Shang, J., Chen, B., & Zheng, N. (2017) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2017.2717451 · Cited by 156
CAR-Net: Clairvoyant Attentive Recurrent Network
Sadeghian, A., Legros, F., Voisin, M., Vesel, R., Alahi, A., & Savarese, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_10 · Cited by 153
Choosing smartly: Adaptive multimodal fusion for object detection in changing environments
Mees, O., Eitel, A., & Burgard, W. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759048 · Cited by 143
SafetyNet: Safe Planning for Real-World Self-Driving Vehicles Using Machine-Learned Policies
Vitelli, M., Chang, Y., Ye, Y., Ferreira, A., Wołczyk, M., Osiński, B., Niendorf, M., Grimmett, H., Huang, Q., Jain, A., et al. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811576 · Cited by 139
Does computer vision matter for action?
Zhou, B., Krähenbühl, P., & Koltun, V. (2019) · Science Robotics · doi:10.1126/scirobotics.aaw6661 · Cited by 126
3D Object Detection Using Scale Invariant and Feature Reweighting Networks
Zhao, X., Liu, Z., Hu, R., & Huang, K. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019267 · Cited by 124
Exploring Data Aggregation in Policy Learning for Vision-Based Urban Autonomous Driving
Prakash, A., Behl, A., Ohn-Bar, E., Chitta, K., & Geiger, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01178 · Cited by 122
Self-citationProbabilistic Future Prediction for Video Scene Understanding
Hu, A., Cotter, F., Mohan, N. C., Gurău, C., & Kendall, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_45 · Cited by 113
Scalable Active Learning for Object Detection
Haussmann, E., Fenzi, M., Chitta, K., Ivanecky, J., Xu, H., Roy, D., Mittel, A., Koumchatzky, N., Farabet, C., & Alvarez, J. M. (2020) · 2020 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/iv47402.2020.9304793 · Cited by 110
Self-citationMonocular Plan View Networks for Autonomous Driving
Wang, D., Devin, C., Cai, Q.-Z., Krähenbühl, P., & Darrell, T. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967897 · Cited by 101
Ohn-Bar, E., Prakash, A., Behl, A., Chitta, K., & Geiger, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01131 · Cited by 95
Self-citationSemantic-aware Grad-GAN for Virtual-to-Real Urban Scene AdaptionPreprint
Li, P., Liang, X., Jia, D., & Xing, E. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.01726 · Cited by 93
Yang, W., Li, Q., Liu, W., Yu, Y., Ma, Y., He, S., & Pan, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01528 · Cited by 89
Multi-View Fusion of Sensor Data for Improved Perception and Prediction in Autonomous Driving
Fadadu, S., Pandey, S., Hegde, D. K., Shi, Y., Chou, F., Djuric, N., & Vallespi-Gonzalez, C. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00335 · Cited by 88
STINet: Spatio-Temporal-Interactive Network for Pedestrian Detection and Trajectory Prediction
Zhang, Z., Gao, J., Mao, J., Liu, Y., Anguelov, D., & Li, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01136 · Cited by 87
End-to-End Autonomous Driving With Semantic Depth Cloud Mapping and Multi-Agent
Natan, O., & Miura, J. (2022) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3185303 · Cited by 87
GRI: General Reinforced Imitation and Its Application to Vision-Based Autonomous Driving
Chekroun, R., Toromanoff, M., Hornauer, S., & Moutarde, F. (2023) · Robotics · doi:10.3390/robotics12050127 · Cited by 81
Looking to Relations for Future Trajectory Forecast
Choi, C., & Dariush, B. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00101 · Cited by 76
RoIFusion: 3D Object Detection From LiDAR and Vision
Chen, C., Fragonara, L. Z., & Tsourdos, A. (2021) · IEEE Access · doi:10.1109/access.2021.3070379 · Cited by 75
Training Data Subset Search With Ensemble Active Learning
Chitta, K., Álvarez, J. M., Haußmann, E., & Farabet, C. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3133268 · Cited by 69
Self-citationLabel Efficient Visual Abstractions for Autonomous Driving
Behl, A., Chitta, K., Prakash, A., Ohn-Bar, E., & Geiger, A. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9340641 · Cited by 56
Self-citationMultiXNet: Multiclass Multistage Multimodal Motion Prediction
Djuric, N., Cui, H., Su, Z., Wu, S., Wang, H., Chou, F.-C., Martin, L. S., Feng, S., Hu, R., Xu, Y., et al. (2021) · 2021 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/iv48863.2021.9575718 · Cited by 53
Perceive, Attend, and Drive: Learning Spatial Attention for Safe Self-Driving
Wei, B. T., Ren, M., Zeng, W., Liang, M., Yang, B., & Urtasun, R. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9561904 · Cited by 52
Driver attention prediction based on convolution and transformers
Gou, C., Zhou, Y., & Li, D. (2022) · The Journal of Supercomputing · doi:10.1007/s11227-021-04151-2 · Cited by 52
Depth Completion via Inductive Fusion of Planar LIDAR and Monocular Camera
Fu, C., Dong, C., Mertz, C., & Dolan, J. M. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341385 · Cited by 50
LaserFlow: Efficient and Probabilistic Object Detection and Motion Forecasting
Meyer, G. P., Charland, J., Pandey, S., Laddha, A., Gautam, S., Vallespi-Gonzalez, C., & Wellington, C. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3047793 · Cited by 46
MVLidarNet: Real-Time Multi-Class Scene Understanding for Autonomous Driving Using Multiple Views
Chen, K., Oldja, R., Smolyanskiy, N., Birchfield, S. T., Popov, A., Wehr, D., Eden, İ., & Pehserl, J. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341450 · Cited by 42
Large-Scale Visual Active Learning with Deep Probabilistic EnsemblesPreprint
Chitta, K., Álvarez, J. M., & Lesnikowski, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.03575 · Cited by 38
Self-citationChen, Y., Dong, C., Palanisamy, P., Mudalige, P., Muelling, K., & Dolan, J. M. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968565 · Cited by 38
Laddha, A., Gautam, S., Palombo, S., Pandey, S., & Vallespi-Gonzalez, C. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00321 · Cited by 37
Loukkal, A., Grandvalet, Y., Drummond, T. W., & Li, Y. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00010 · Cited by 34
A Review of 3D Object Detection for Autonomous Driving of Electric Vehicles
Dai, D., Chen, Z., Bao, P., & Wang, J. (2021) · World Electric Vehicle Journal · doi:10.3390/wevj12030139 · Cited by 34
SDVTracker: Real-Time Multi-Sensor Association and Tracking for Self-Driving Vehicles
Gautam, S., Meyer, G. P., Vallespi-Gonzalez, C., & Becker, B. C. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00336 · Cited by 16
Leveraging Smooth Attention Prior for Multi-Agent Trajectory Prediction
Cao, Z., Bıyık, E., Rosman, G., & Sadigh, D. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811718 · Cited by 13
Affordance-based Reinforcement Learning for Urban DrivingPreprint
Agarwal, T., Arora, H., & Schneider, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.05970 · Cited by 10
Khalil, Y. H., & Mouftah, H. T. (2021) · IEEE Access · doi:10.1109/access.2021.3123169 · Cited by 5
Investigating the Effect of Sensor Modalities in Multi-Sensor Detection-Prediction ModelsPreprint
Mohta, A., Chou, F., Becker, B. C., Vallespi-Gonzalez, C., & Djuric, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.03279 · Cited by 3
Cited by
Planning-oriented Autonomous Driving
Hu, Y., Yang, J., Chen, L., Li, K., Sima, C., Zhu, X., Chai, S., Du, S., Lin, T., Wang, W., et al. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01712 · Cited by 931
End-to-End Autonomous Driving: Challenges and Frontiers
Chen, L., Wu, P., Chitta, K., Jaeger, B., Geiger, A., & Li, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3435937 · Cited by 610
Self-citationRecent Advancements in End-to-End Autonomous Driving Using Deep Learning: A Survey
Chib, P. S., & Singh, P. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3318070 · Cited by 301
Li, Q., Jia, X., Wang, S., & Yan, J. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-72995-9_9 · Cited by 42
Dynamic alignment and fusion of multimodal physiological patterns for stress recognition
Zhang, X., Wei, X., Zhou, Z., Zhao, Q., Zhang, S., Yang, Y., Li, R., & Hu, B. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3290177 · Cited by 38
Metadata
- Authors
- Chitta, K., Prakash, A., Jaeger, B., Yu, Z., Renz, K., & Geiger, A.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2023-11-01
- Volume
- 45
- Issue
- 11
- Pages
- 12878-12895
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W4292347911
- Cited by
- 505
- References
- 109
- FWCI
- 22.71
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.