Towards Robust 3D Object Detection with LiDAR and 4D Radar Fusion in Various Weather Conditions
Abstract
Detecting objects in 3D under various (normal and adverse) weather conditions is essential for safe autonomous driving systems. Recent approaches have focused on employing weather-insensitive 4D radar sensors and leveraging them with other modalities, such as LiDAR. However, they fuse multi-modal information without considering the sensor characteristics and weather conditions, and lose some height information which could be useful for localizing 3D objects. In this paper, we propose a novel framework for robust LiDAR and 4D radar-based 3D object detection. Specifically, we propose a 3D-LRF module that considers the distinct patterns they exhibit in 3D space (e.g., precise 3D mapping of LiDAR and wide-range, weather-insensitive measurement of 4D radar) and extract fusion features based on their 3D spatial relationship. Then, our weather-conditional radar-flow gating network modulates the information flow of fusion features depending on weather conditions, and obtains enhanced feature that effectively incorporates the strength of two domains under various weather conditions. The extensive experiments demonstrate that our model achieves SoTA performance for 3D object detection under various weather conditions.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Goal-oriented navigation capability has been validated in Habitat's ObjectNav task, showing effective goal localisation in unknown environments.
“This capability has been extensively validated in Habitat's ObjectNav task [108,109,17,110], demonstrating the effectiveness of achieving goal localisation in unknown environments.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [110]
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · see citing works
nuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152 · cited 1× · see citing works
“Recently, several research has focused on addressing these challenges by employing radar sensors capable of handling various weather conditions [28], and has released datasets containing diverse weather environments [4, 28, 35].”
PointPillars: Fast Encoders for Object Detection From Point Clouds
Lang, A. H., Vora, S., Caesar, H., Zhou, L., Yang, J., & Beijbom, O. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01298 · Cited by 4,673 · see citing works
Center-based 3D Object Detection and Tracking
Yin, T., Zhou, X., & Krähenbühl, P. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01161 · Cited by 2,197 · see citing works
Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation
Liu, Z., Tang, H., Amini, A., Yang, X., Mao, H., Rus, D. L., & Han, S. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160968 · Cited by 1,307 · see citing works
TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers
Bai, X., Hu, Z., Zhu, X., Huang, Q., Chen, Y., Fu, H., & Tai, C.-L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00116 · Cited by 990 · see citing works
A Survey on 3D Object Detection Methods for Autonomous Driving Applications
Arnold, E. H., Al-Jarrah, O. Y., Dianati, M., Fallah, S., Oxtoby, D., & Mouzakitis, A. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2892405 · Cited by 720 · see citing works
Mvx-net: Multimodal voxelnet for 3d object detection
Sindagi, V. A., Zhou, Y., & Tuzel, O. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794195 · Cited by 554 · see citing works
PointAugmenting: Cross-Modal Augmentation for 3D Object Detection
Wang, C., Ma, C., Zhu, M. B., & Yang, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01162 · Cited by 481 · see citing works
CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection
Nabati, R., & Qi, H. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00157 · Cited by 461 · see citing works
2023 IEEE/CVF International Conference on Computer Vision (ICCV)
(2023) · doi:10.1109/iccv51070.2023 · Cited by 362 · see citing works
Multi-Class Road User Detection With 3+1D Radar in the View-of-Delft Dataset
Palffy, A., Pool, E. A. I., Baratam, S., Kooij, J. F. P., & Gavrila, D. M. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3147324 · Cited by 322 · see citing works
BEVFusion: A Simple and Robust LiDAR-Camera Fusion FrameworkPreprint
Liang, T., Xie, H., Yu, K., Xia, Z., Lin, Z., Wang, Y., Tang, T., Wang, B., & Tang, Z. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.13790 · Cited by 310 · see citing works
Multi-Modal 3D Object Detection in Autonomous Driving: A Survey and Taxonomy
Wang, L., Zhang, X., Song, Z., Bi, J., Zhang, G., Wei, H., Tang, L., Yang, L., Li, J., Jia, C., et al. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3264658 · Cited by 276 · see citing works
RADIATE: A Radar Dataset for Automotive Perception in Bad Weather
Sheeny, M., Pellegrin, E. D., Mukherjee, S., Ahrabian, A., Wang, S., & Wallace, A. M. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9562089 · Cited by 257 · see citing works
Virtual Sparse Convolution for Multimodal 3D Object Detection
Wu, H. Y., Wen, C., Shi, S., Li, X. S., & Wang, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02074 · Cited by 243 · see citing works
Xie, L., Xiang, C. J., Yu, Z., Xu, G., Yang, Z., Cai, D., & He, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6933 · Cited by 238 · see citing works
MonoDTR: Monocular 3D Object Detection with Depth-Aware Transformer
Huang, K., Wu, T.-H., Su, H.-T., & Hsu, W. H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00398 · Cited by 238 · see citing works
Robust Multimodal Vehicle Detection in Foggy Weather Using Complementary Lidar and Radar Signals
Qian, K., Zhu, S., Zhang, X., & Li, L. E. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00051 · Cited by 206 · see citing works
LoGoNet: Towards Accurate 3D Object Detection with Local-to-Global Cross- Modal Fusion
Li, X. S., Ma, T., Hou, Y., Shi, B., Yang, Y., Liu, Y., Wu, X., Chen, Q., Li, Y., Qiao, Y., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01681 · Cited by 187 · see citing works
Multi-modal Sensor Fusion for Auto Driving Perception: A SurveyPreprint
Huang, K., Shi, B., Li, X., Li, X., Huang, S., & Li, Y. (2022) · arXiv (Cornell University) · doi:10.4230/lipics.ecrts.2025.6 · Cited by 181 · see citing works
CenterFormer: Center-Based Transformer for 3D Object Detection
Zhou, Z., Zhao, X., Wang, Y., Wang, P., & Foroosh, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19839-7_29 · Cited by 174 · see citing works
CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perception
Kim, Y., Shin, J., Kim, S., Lee, I.-J., Choi, J. W., & Kum, D. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01615 · Cited by 155 · see citing works
CNN Based Road User Detection Using the 3D Radar Cube
Palffy, A., Dong, J., Kooij, J. F. P., et al. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2967272 · Cited by 154 · see citing works
RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects
Yang, B., Guo, R. B., Liang, M., Casas, S., & Urtasun, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58523-5_29 · Cited by 150 · see citing works
Multimodal Virtual Point 3D DetectionPreprint
Yin, T., Zhou, X., & Krähenbühl, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.06881 · Cited by 130 · see citing works
CRAFT: Camera-Radar 3D Object Detection with Spatio-Contextual Fusion Transformer
Kim, Y., Kim, S., Choi, J. W., & Kum, D. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25198 · Cited by 130 · see citing works
SMURF: Spatial Multi-Representation Fusion for 3D Object Detection With 4D Imaging Radar
Liu, J., Zhao, Q., Xiong, W., Huang, T., Han, Q., & Zhu, B. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3322729 · Cited by 126 · see citing works
3-D Object Detection for Multiframe 4-D Automotive Millimeter-Wave Radar Point Cloud
Tan, B., Ma, Z., Zhu, X., Li, S., Zheng, L., Chen, S., Huang, L., & Bai, J. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3219643 · Cited by 123 · see citing works
Frustum-PointPillars: A Multi-Stage Approach for 3D Object Detection using RGB Camera and LiDAR
Paigwar, A., González, D., Erkent, Ö., & Laugier, C. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00327 · Cited by 108 · see citing works
Bansal, K., Rungta, K., Zhu, S., & Bharadia, D. (2020) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3384419.3430783 · Cited by 79 · see citing works
Song, Z., Wei, H., Bai, L., Yang, L., & Jia, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00311 · Cited by 76 · see citing works
Kim, Y.-S., Choi, J. W., & Kum, D. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341177 · Cited by 69 · see citing works
Graph Convolutional Networks for 3D Object Detection on Radar Data
Meyer, M., Kuschk, G., & Tomforde, S. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00340 · Cited by 69 · see citing works
FusionRCNN: LiDAR-Camera Fusion for Two-Stage 3D Object Detection
Xu, X., Dong, S., Xu, T., Ding, L., Wang, J., Jiang, P., Song, L., & Li, J. (2023) · Remote Sensing · doi:10.3390/rs15071839 · Cited by 68 · see citing works
Radar Voxel Fusion for 3D Object Detection
Nobis, F., Shafiei, E., Karle, P., Betz, J., & Lienkamp, M. (2021) · Applied Sciences · doi:10.3390/app11125598 · Cited by 61 · see citing works
CramNet: Camera-Radar Fusion with Ray-Constrained Cross-Attention for Robust 3D Object Detection
Hwang, J.-J., Kretzschmar, H., Manela, J., Rafferty, S., Armstrong-Crews, N., Chen, T., & Anguelov, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19839-7_23 · Cited by 59 · see citing works
InterFusion: Interaction-based 4D radar and LiDAR fusion for 3D object detection
Wang, L., Zhang, X., Xv, B., Zhang, J., Fu, R., Wang, X., Zhu, L., Ren, H., Lu, P., Li, J., et al. (2022) · 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9982123 · Cited by 57 · see citing works
Modality-Agnostic Learning for Radar-Lidar Fusion in Vehicle Detection
Li, Y.-J., Park, J., O’Toole, M., & Kitani, K. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00099 · Cited by 53 · see citing works
SSDA3D: Semi-supervised Domain Adaptation for 3D Object Detection from Point Cloud
Wang, Y., Yin, J., Li, W., Pascal Frossard, B., Yang, R., & Shen, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25370 · Cited by 51 · see citing works
High Dimensional Frustum PointNet for 3D Object Detection from Camera, LiDAR, and Radar
Wang, L., Chen, T., Anklam, C., & Goldlüecke, B. (2020) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv47402.2020.9304655 · Cited by 41 · see citing works
SA-BEV: Generating Semantic-Aware Bird’s-Eye-View Feature for Multi-view 3D Object Detection
Zhang, J., Zhang, Y., Liu, Q., & Wang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00310 · Cited by 41 · see citing works
DeepFusion: A Robust and Modular 3D Object Detector for Lidars, Cameras and Radars
Drews, F., Feng, D., Faion, F., Rosenbaum, L., Ulrich, M., & Gläser, C. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981778 · Cited by 34 · see citing works
Improved 3D Object Detector Under Snowfall Weather Condition Based on LiDAR Point Cloud
Lin, J., Yin, H., Yan, J., Wan-cheng, G., Zhang, H., & Rigoll, G. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3188985 · Cited by 31 · see citing works
A Recurrent CNN for Online Object Detection on Raw Radar Frames
Decourt, C., Rufin Vanrullen, A., Salle, D., & Oberlin, T. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3404076 · Cited by 30 · see citing works
Paek, D.-H., Kong, S.-H., & Wijaya, K. T. (2023) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv55152.2023.10186820 · Cited by 24 · see citing works
Camera and LiDAR analysis for 3D object detection in foggy weather conditions
Minh, N. A., Duthon, P., Salmane, P. H., Khoudour, L., Crouzil, A., & Velastín, S. A. (2022) · International Conference on Pattern Recognition Systems (ICPRS) · doi:10.1109/icprs54038.2022.9854073 · Cited by 20 · see citing works
Source-free Unsupervised Domain Adaptation for 3D Object Detection in Adverse Weather
Hegde, D., Kilic, V., Sindagi, V. A., Cooper, A. B., Foster, M. A., & Patel, V. M. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161341 · Cited by 17 · see citing works
LossDistillNet: 3D Object Detection in Point Cloud Under Harsh Weather Conditions
Do, A. T., & Yoo, M. (2022) · IEEE Access · doi:10.1109/access.2022.3197765 · Cited by 13 · see citing works
Enhancing Multi-modal Features Using Local Self-attention for 3D Object Detection
Li, H., Zhang, Z., Zhao, X. F., Wang, Y., Shen, Y., Pu, S., & Mao, H. Z. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20080-9_31 · Cited by 8 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 41 · see citing works
“In recent years, with the development of transformer-based deep learning techniques, some methods have introduced attention mechanisms to model temporal and cross-modal relationships.”
- Authors
- Chae, Y., Kim, H., & Yoon, K.-J.
- Affiliations
- Korea Advanced Institute of Science and Technology (KR)
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 15162-15172
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402727857
- Cited by
- 49
- References
- 52
- Citations by year
- 2026: 25 · 2025: 22 · 2024: 2
- FWCI
- 27.02
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Funders
- Ministry of Trade, Industry & Energy
- Defense Acquisition Program Administration (912768601)
- National Research Foundation of Korea (NRF2022R1A2B5B03002636)
- Topics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Aerospace Engineering › Radar Systems and Signal Processing
- Keywords
- 3D object detection, LiDAR, 4D radar, sensor fusion, weather robustness, adverse weather, autonomous driving
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
