RCBEVDet: Radar-Camera Fusion in Bird's Eye View for 3D Object Detection
Open accessAbstract
Three-dimensional object detection is one of the key tasks in autonomous driving. To reduce costs in practice, low-cost multi-view cameras for 3D object detection are proposed to replace the expansive LiDAR sensors. However, relying solely on cameras is difficult to achieve highly accurate and robust 3D object detection. An effective solution to this issue is combining multi-view cameras with the economical millimeter-wave radar sensor to achieve more reliable multi-modal 3D object detection. In this paper, we introduce RCBEVDet, a radar-camera fusion 3D object detection method in the bird's eye view (BEV). Specifically, we first design RadarBEVNet for radar BEV feature extraction. RadarBEVNet consists of a dual-stream radar backbone and a Radar Cross-Section (RCS) aware BEV encoder. In the dual-stream radar backbone, a point-based encoder and a transformer-based encoder are proposed to extract radar features, with an injection and extraction module to facilitate communication between the two encoders. The RCS-aware BEV encoder takes RCS as the object size prior to scattering the point feature in BEV. Besides, we present the Cross-Attention Multi-layer Fusion module to automatically align the multi-modal BEV feature from radar and camera with the deformable attention mechanism, and then fuse the feature with channel and spatial fusion layers. Experimental results show that RCBEVDet achieves new state-of-the-art radar-camera fusion results on nuScenes and view-of-delft (VoD) 3D object detection benchmarks. Furthermore, RCBEVDet achieves better 3D detection results than all real-time camera-only and radar-camera 3D object detectors with a faster inference speed at 21∼28 FPS. The source code will be released at https://github.com/VDIGPKU/RCBEVDet.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
RCM-Fusion combines radar and camera data at both feature and instance layers, refining 3D schemes using radar points.
“RCM-Fusion [103] combines radar and camera data at both the feature and instance layers, refining 3D schemes using radar points.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [103]
Cites
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · see citing works
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles, R., Su, H., Mo, K., & Guibas, L. J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.16 · Cited by 9,767 · see citing works
nuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152 · cited 2× · see citing works
“More recently, the emergence of multi-view camera datasets [2, 37] has led to the development of multi-view 3D object detection methods [10, 19, 23, 33, 41], which can be briefly divided into two categories, i.e. , geometry-based methods and transformer-based methods.”
BackgroundPointPillars: Fast Encoders for Object Detection From Point Clouds
Lang, A. H., Vora, S., Caesar, H., Zhou, L., Yang, J., & Beijbom, O. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01298 · Cited by 4,673 · see citing works
Scalability in Perception for Autonomous Driving: Waymo Open Dataset
Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., Caine, B., et al. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00252 · Cited by 3,437 · see citing works
SECOND: Sparsely Embedded Convolutional Detection
Yan, Y., Mao, Y., & Li, B. (2018) · Sensors · doi:10.3390/s18103337 · Cited by 3,352 · see citing works
Center-based 3D Object Detection and Tracking
Yin, T., Zhou, X., & Krähenbühl, P. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01161 · Cited by 2,197 · see citing works
Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Yu, Q., & Dai, J. (2022) · arXiv · doi:10.48550/arxiv.2203.17270 · Cited by 1,419 · see citing works
Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation
Liu, Z., Tang, H., Amini, A., Yang, X., Mao, H., Rus, D. L., & Han, S. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160968 · Cited by 1,307 · see citing works
Self-citationLift, Splat, Shoot: Encoding Images from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D
Philion, J., & Fidler, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58568-6_12 · Cited by 1,271 · see citing works
Shi, S., Wang, Z., Shi, J., Wang, X., & Li, H. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2977026 · Cited by 951 · see citing works
Self-citationBEVDepth: Acquisition of Reliable Depth for Multi-View 3D Object Detection
Li, Y., Ge, Z., Yu, G., Yang, J., Wang, Z., Shi, Y., Sun, J., & Li, Z. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i2.25233 · Cited by 784 · see citing works
FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection
Wang, T., Zhu, X., Pang, J., & Lin, D. (2021) · 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw54120.2021.00107 · Cited by 683 · see citing works
FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection
Wang, T., Zhu, X., Pang, J., & Lin, D. (2021) · arXiv · doi:10.48550/arxiv.2104.10956 · Cited by 683 · see citing works
PETR: Position Embedding Transformation for Multi-view 3D Object Detection
Liu, Y., Wang, T., Zhang, X., & Sun, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19812-0_31 · Cited by 664 · see citing works
Seeing Through Fog Without Seeing Fog: Deep Multimodal Sensor Fusion in Unseen Adverse Weather
Bijelic, M., Gruber, T., Mannan, F., Kraus, F., Ritter, W., Dietmayer, K., & Heide, F. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01170 · Cited by 647 · see citing works
Stereo R-CNN Based 3D Object Detection for Autonomous Driving
Li, P., Chen, X., & Shen, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00783 · Cited by 597 · see citing works
BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object DetectionPreprint
Huang, J., & Huang, G. (2022) · arXiv.org · doi:10.48550/arxiv.2203.17054 · Cited by 554 · see citing works
CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection
Nabati, R., & Qi, H. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00157 · Cited by 461 · see citing works
PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images
Liu, Y., Yan, J., Jia, F., Li, S., Gao, A., Wang, T., & Zhang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00302 · Cited by 400 · see citing works
Multi-Class Road User Detection With 3+1D Radar in the View-of-Delft Dataset
Palffy, A., Pool, E. A. I., Baratam, S., Kooij, J. F. P., & Gavrila, D. M. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3147324 · Cited by 322 · see citing works
MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationships
Chen, Y., Tai, L., Sun, K., & Li, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01211 · Cited by 316 · see citing works
Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection
Zhu, B., Jiang, Z., Zhou, X., Li, Z., & Yu, G. (2019) · arXiv · doi:10.48550/arxiv.1908.09492 · Cited by 297 · see citing works
BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-ViewPreprint
Huang, J., Huang, G., Zhu, Z. D., Ye, Y., & Du, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.11790 · Cited by 292 · see citing works
Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection
Wang, S., Liu, Y., Wang, T., Li, Y., & Zhang, X. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00335 · Cited by 292 · see citing works
Monocular 3D Object Detection with Pseudo-LiDAR Point Cloud
Weng, X., & Kitani, K. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00114 · Cited by 290 · see citing works
Delving into Localization Errors for Monocular 3D Object Detection
Ma, X., Zhang, Y., Xu, D., Zhou, D., Yi, S., Li, H., & Ouyang, W. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00469 · Cited by 272 · see citing works
Geometry Uncertainty Projection Network for Monocular 3D Object Detection
Lu, Y., Ma, X., Yang, L., Zhang, T., Liu, Y., Chu, Q., Yan, J., & Ouyang, W. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00310 · Cited by 265 · see citing works
Small and Short-Range Radar Systems
Charvat, G. L. (2014) · CRC Press eBooks · doi:10.1201/b16718 · Cited by 166 · see citing works
Towards Deep Radar Perception for Autonomous Driving: Datasets, Methods, and Challenges
Zhou, Y., Liu, L., Zhao, H., López‐Benítez, M., Yu, L., & Yue, Y. (2022) · Sensors · doi:10.3390/s22114208 · Cited by 164 · see citing works
CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perception
Kim, Y., Shin, J., Kim, S., Lee, I.-J., Choi, J. W., & Kum, D. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01615 · Cited by 155 · see citing works
SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videos
Liu, H., Teng, Y., Lu, T., Wang, H., & Wang, L. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01703 · Cited by 154 · see citing works
RCFusion: Fusing 4-D Radar and Camera With Bird’s-Eye View Features for 3-D Object Detection
Zheng, L., Li, S., Tan, B., Yang, L., Chen, S., Huang, L., Bai, J., Zhu, X., & Ma, Z. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3280525 · Cited by 152 · see citing works
Zhou, T., Chen, J., Shi, Y., Jiang, K., Yang, M., & Yang, D. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3240287 · Cited by 151 · see citing works
RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects
Yang, B., Guo, R. B., Liang, M., Casas, S., & Urtasun, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58523-5_29 · Cited by 150 · see citing works
CRAFT: Camera-Radar 3D Object Detection with Spatio-Contextual Fusion Transformer
Kim, Y., Kim, S., Choi, J. W., & Kum, D. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25198 · Cited by 130 · see citing works
Towards Generalization Across Depth for Monocular 3D Object Detection
Simonelli, A., Bulò, S. R., Porzi, L., Ricci, E., & Kontschieder, P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58542-6_46 · Cited by 70 · see citing works
Kim, Y.-S., Choi, J. W., & Kum, D. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341177 · Cited by 69 · see citing works
PVGNet: A Bottom-Up One-Stage 3D Object Detector with Integrated Multi-Level Features
Miao, Z., Chen, J., Pan, H., Zhang, R., Liu, K., Hao, P., Zhu, J., Wang, Y., & Zhan, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00329 · Cited by 58 · see citing works
Self-citationExploiting Temporal Relations on Radar Perception for Autonomous Driving
Li, P., Wang, P. P., Berntorp, K., & Liu, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01656 · Cited by 58 · see citing works
Chen, Z., Li, Z., Zhang, S., Fang, L., Jiang, Q., & Zhao, F. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547859 · Cited by 57 · see citing works
Temporal Enhanced Training of Multi-view 3D Object Detector via Historical Object Prediction
Zong, Z., Jiang, D., Song, G., Xue, Z., Su, J., Li, H., & Liu, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00350 · Cited by 57 · see citing works
Ulrich, M., Braun, S., Köhler, D., Niederlöhner, D., Faion, F., Gläser, C., & Blume, H. (2022) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc55140.2022.9922457 · Cited by 54 · see citing works
RADIANT: Radar-Image Association Network for 3D Object Detection
Long, Y., Kumar, A., Morris, D., Liu, X., Castro, M., & Chakravarty, P. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i2.25270 · Cited by 47 · see citing works
Monocular Depth Estimation With Multi-Scale Feature Fusion
Xu, X., Chen, Z., & Yin, F. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3067498 · Cited by 35 · see citing works
BEVPoolv2: A Cutting-edge Implementation of BEVDet Toward DeploymentPreprint
Huang, J., & Huang, G. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.17111 · Cited by 25 · see citing works
MVFusion: Multi-View 3D Object Detection with Semantic-aligned Radar and Camera FusionPreprint
Wu, Z., Chen, G., Gan, Y., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.10511 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 41 · see citing works
“A generalized SLAM pipeline involves sensor data acquisition, feature extraction, data association, motion estimation (odometry), map initialization, loop closure detection, and map optimization, as illustrated in Figure 10.”
- Authors
- Lin, Z., Liu, Z., Xia, Z., Wang, X., Wang, Y., Qi, S., Dong, Y., Dong, N., Zhang, L., & Zhu, C.
- Affiliations
- Peking University (CN)
- University of Electronic Science and Technology of China (CN)
- Chongqing Changan Automobile Co., Ltd.
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 14928-14937
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402754159
- Cited by
- 153
- References
- 47
- Citations by year
- 2026: 67 · 2025: 73 · 2024: 13
- FWCI
- 84.11
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, public-domain)
- arXiv (Cornell University) (public-domain)
- Funders
- National Key R&D Program of China (2022ZD0160305)
- Topics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Aerospace Engineering › Radar Systems and Signal Processing
- Keywords
- 3D object detection, radar-camera fusion, bird's-eye view, millimeter-wave radar, multi-view cameras, radar cross section, deformable attention, cross-attention fusion, autonomous driving
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
