CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection
Open accessAbstract
The perception system in autonomous vehicles is responsible for detecting and tracking the surrounding objects. This is usually done by taking advantage of several sensing modalities to increase robustness and accuracy, which makes sensor fusion a crucial part of the perception system. In this paper, we focus on the problem of radar and camera sensor fusion and propose a middle-fusion approach to exploit both radar and camera data for 3D object detection. Our approach, called CenterFusion, first uses a center point detection network to detect objects by identifying their center points on the image. It then solves the key data association problem using a novel frustum-based method to associate the radar detections to their corresponding object's center point. The associated radar detections are used to generate radar-based feature maps to complement the image features, and regress to object properties such as depth, rotation and velocity. We evaluate CenterFusion on the challenging nuScenes dataset, where it improves the overall nuScenes Detection Score (NDS) of the state-of-the-art camera-based algorithm by more than 12%. We further show that CenterFusion significantly improves the velocity estimation accuracy without using any additional temporal information. The code is available at https://github.com/mrnabati/CenterFusion.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
CRAFT adapts radar-camera correlation in polar coordinates, utilizing cross-attention for robust fusion.
“The CRAFT [100] model adapts radar-camera correlation in polar coordinates, utilizing cross-attention for robust fusion.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [100]
Section 4 (p. 4) identifies the paper’s model as CenterFusion and describes associating radar detections with object centers for middle fusion. The Introduction (p. 2) specifies frustum-based association and radar feature maps projected onto the image plane, rather than the claim’s CRAFT model with polar-coordinate correlation and cross-attention. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · see citing works
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634 · see citing works
Multi-view 3D Object Detection Network for Autonomous Driving
Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.691 · Cited by 3,450 · see citing works
SECOND: Sparsely Embedded Convolutional Detection
Yan, Y., Mao, Y., & Li, B. (2018) · Sensors · doi:10.3390/s18103337 · Cited by 3,352 · see citing works
Frustum PointNets for 3D Object Detection from RGB-D Data
Qi, C. R., Liu, W., Wu, C., Su, H., & Guibas, L. J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00102 · Cited by 2,655 · see citing works
Joint 3D Proposal Generation and Object Detection from View Aggregation
Ku, J., Mozifian, M., Lee, J., Harakeh, A., & Waslander, S. L. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594049 · Cited by 1,697 · see citing works
Yu, F., Wang, D., Shelhamer, E., & Darrell, T. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00255 · Cited by 1,592 · see citing works
Feng, D., Haase-Schutz, C., Rosenbaum, L., Hertlein, H., Glaser, C., Timm, F., Wiesbeck, W., & Dietmayer, K. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.2972974 · Cited by 1,425 · see citing works
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017) · arXiv · doi:10.48550/arxiv.1708.02002 · Cited by 1,227 · see citing works
3D Bounding Box Estimation Using Deep Learning and Geometry
Mousavian, A., Anguelov, D., Flynn, J., & Kosecka, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.597 · Cited by 1,129 · see citing works
PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation
Xu, D., Anguelov, D., & Jain, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00033 · Cited by 856 · see citing works
Multi-Task Multi-Sensor Fusion for 3D Object Detection
Liang, M., Yang, B., Chen, Y., Hu, R., & Urtasun, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00752 · Cited by 723 · see citing works
Vehicle Detection from 3D Lidar Using Fully Convolutional Network
Li, B., Zhang, T., & Xia, T. (2016) · Robotics: Science and Systems XII · doi:10.15607/rss.2016.xii.042 · Cited by 577 · see citing works
Stacked Hourglass Networks for Human Pose EstimationPreprint
Newell, A., Yang, K., & Deng, J. H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.06937 · Cited by 531 · see citing works
Disentangling Monocular 3D Object Detection
Simonelli, A., Bulo, S. R., Porzi, L., Lopez-Antequera, M., & Kontschieder, P. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00208 · Cited by 520 · see citing works
A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detection
Nobis, F., Geisslinger, M., Weber, M., Betz, J., & Lienkamp, M. (2019) · Sensor Data Fusion: Trends, Solutions, Applications (SDF) · doi:10.1109/sdf.2019.8916629 · Cited by 360 · see citing works
3D-RCNN: Instance-Level 3D Object Reconstruction via Render-and-Compare
Kundu, A., Li, Y., & Rehg, J. M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00375 · Cited by 331 · see citing works
VoxelNet: End-to-End Learning for Point Cloud Based 3D Object DetectionPreprint
Zhou, Y., & Tuzel, O. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.06396 · Cited by 318 · see citing works
Zhou, X., Wang, D., & Krähenbühl, P. (2019) · arXiv · doi:10.48550/arxiv.1904.07850 · Cited by 288 · see citing works
Vehicle Detection from 3D Lidar Using Fully Convolutional NetworkPreprint
Li, B., Zhang, T., & Xia, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.07916 · Cited by 275 · see citing works
Distant Vehicle Detection Using Radar and Vision
Chadwick, S., Maddern, W., & Newman, P. M. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794312 · Cited by 237 · see citing works
Orthographic feature transform for monocular 3d object detection
Roddick, T., Kendall, A., & Cipolla, R. (2018) · arXiv · doi:10.48550/arxiv.1811.08188 · Cited by 211 · see citing works
Rrpn: Radar region proposal network for object detection in autonomous vehicles
Nabati, R., & Qi, H. (2019) · 2019 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2019.8803392 · Cited by 197 · see citing works
Self-citationPointRCNN: 3D Object Proposal Generation and Detection from Point CloudPreprint
Shi, S., Wang, X., & Li, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.04244 · Cited by 162 · see citing works
Sensor Fusion for Joint 3D Object Detection and Semantic Segmentation
Meyer, G. P., Charland, J., Hegde, D. K., Laddha, A., & Vallespi-Gonzalez, C. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00162 · Cited by 158 · see citing works
RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects
Yang, B., Guo, R. B., Liang, M., Casas, S., & Urtasun, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58523-5_29 · Cited by 150 · see citing works
Sensor fusion for semantic segmentation of urban scenes
Zhang, R., Candra, S. A., Vetter, K., & Zakhor, A. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139439 · Cited by 108 · see citing works
3D object tracking using RGB and LIDAR data
Asvadi, A., Girão, P., Peixoto, P., & Nunes, U. (2016) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2016.7795718 · Cited by 77 · see citing works
Multi-View 3D Object Detection Network for Autonomous DrivingPreprint
Chen, X., Ma, H., Wan, J., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07759 · Cited by 65 · see citing works
InfoFocus: 3D Object Detection for Autonomous Driving with Dynamic Information Modeling
Wang, J., Lan, S., Gao, M., & Davis, L. S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58607-2_24 · Cited by 62 · see citing works
PointFusion: Deep Sensor Fusion for 3D Bounding Box EstimationPreprint
Xu, D., Anguelov, D., & Jain, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10871 · Cited by 55 · see citing works
Radar-Camera Sensor Fusion for Joint Object Detection and Distance Estimation in Autonomous VehiclesPreprint
Nabati, R., & Qi, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.08428 · Cited by 51 · see citing works
Self-citation3D Bounding Box Estimation Using Deep Learning and GeometryPreprint
Mousavian, A., Anguelov, D., Flynn, J., & Košecká, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00496 · Cited by 25 · see citing works
3D Fully Convolutional Network for Vehicle Detection in Point CloudPreprint
Li, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.08069 · Cited by 23 · see citing works
Camera and LiDAR Fusion for On-road Vehicle Tracking with Reinforcement Learning
Fang, Y., Zhao, H., Zha, H., et al. (2019) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2019.8814028 · Cited by 14 · see citing works
RadarNet: Exploiting Radar for Robust Perception of Dynamic ObjectsPreprint
Yang, B., Guo, R. B., Liang, M. X., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.14366 · Cited by 11 · see citing works
PIXOR: Real-time 3D Object Detection from Point CloudsPreprint
Yang, B., Luo, W., & Urtasun, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.06326 · Cited by 7 · see citing works
InfoFocus: 3D Object Detection for Autonomous Driving with Dynamic Information ModelingPreprint
Wang, J., Lan, S., Gao, M., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08556 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [2][2] Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. arXiv preprint arXiv:1903.11027 , 2019. - · arXiv
- [6][6] David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep network. In Advances in neural information processing systems , pages 2366-2374, 2014. -
- [9][9] Ross Girshick. Fast r-cnn. 2015 IEEE International Conference on Computer Vision (ICCV) , Dec 2015. -
Cited by
Sorted by citations
FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection
Wang, T., Zhu, X., Pang, J., & Lin, D. (2021) · arXiv · doi:10.48550/arxiv.2104.10956 · Cited by 683 · see citing works
CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perception
Kim, Y., Shin, J., Kim, S., Lee, I.-J., Choi, J. W., & Kum, D. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01615 · Cited by 155 · see citing works
3D Object Detection from Images for Autonomous Driving: A Survey
Ma, X., Ouyang, W., Simonelli, A., & Ricci, E. (2022) · arXiv · doi:10.48550/arxiv.2202.02980 · Cited by 147 · see citing works
RCBEVDet: Radar-Camera Fusion in Bird's Eye View for 3D Object Detection
Lin, Z., Liu, Z., Xia, Z., Wang, X., Wang, Y., Qi, S., Dong, Y., Dong, N., Zhang, L., & Zhu, C. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01414 · Cited by 153 · see citing works
CRAFT: Camera-Radar 3D Object Detection with Spatio-Contextual Fusion Transformer
Kim, Y., Kim, S., Choi, J. W., & Kum, D. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25198 · Cited by 130 · see citing works
Emerging trends in autonomous vehicle perception: Multimodal fusion for 3D object detection
Alaba, S. Y., Gurbuz, A. C., & Ball, J. E. (2024) · World Electric Vehicle Journal · doi:10.3390/wevj15010020 · Cited by 82 · see citing works
Radar Camera Fusion via Representation Learning in Autonomous Driving
Dong, X., Zhuang, B., Mao, Y., & Liu, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00183 · Cited by 62 · see citing works
Towards Robust 3D Object Detection with LiDAR and 4D Radar Fusion in Various Weather Conditions
Chae, Y., Kim, H., & Yoon, K.-J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01436 · Cited by 49 · see citing works
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 41 · see citing works
“Traditional SLAM systems include LiDAR-based[146, 147] and vision-based [148, 149] methods.”
Multi-Sem Fusion: Multimodal Semantic Fusion for 3-D Object Detection
Xu, S., Li, F., Song, Z., Fang, J., Wang, S., & Yang, Z.-X. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3387732 · Cited by 26 · see citing works
- Authors
- Nabati, R., & Qi, H.
- Affiliations
- University of Tennessee at Knoxville (US)
- ORCID
- Qi, Hairong
- Venue
- 2021 IEEE Winter Conference on Applications of Computer Vision (WACV)
- Publisher
- IEEE
- Published
- 2021-01
- Pages
- 1526-1535
- ISSN
- 2472-6737 (ISSN-L)
- OpenAlex
- W3105073457
- Cited by
- 461
- References
- 39
- Citations by year
- 2026: 90 · 2025: 91 · 2024: 111 · 2023: 93 · 2022: 55 · 2021: 21
- FWCI
- 69.35
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Winter Conference on Applications of Computer Vision (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Aerospace Engineering › Radar Systems and Signal Processing
- Keywords
- 3D object detection, autonomous vehicles, velocity estimation, nuScenes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
