IS-Fusion: Instance-Scene Collaborative Fusion for Multimodal 3D Object Detection
Open accessAbstract
Bird's eye view (BEV) representation has emerged as a dominant solution for describing 3D space in autonomous driving scenarios. However, objects in the BEV representation typically exhibit small sizes, and the associated point cloud context is inherently sparse, which leads to great challenges for reliable 3D perception. In this paper, we propose IS-Fusion, an innovative multimodal fusion framework that jointly captures the Instance- and Scene-level contextual information. IS-Fusion essentially differs from existing approaches that only focus on the BEV scene-level fusion by explicitly incorporating instance-level multimodal information, thus facilitating the instance-centric tasks like 3D object detection. It comprises a Hierarchical Scene Fusion (HSF) module and an Instance-Guided Fusion (IGF) module. HSF applies Point-to-Grid and Grid-to-Region transformers to capture the multimodal scene context at different granularities. IGF mines instance candidates, explores their relationships, and aggregates the local multimodal context for each instance. These instances then serve as guidance to enhance the scene feature and yield an instance-aware BEV representation. On the challenging nuScenes benchmark, IS-Fusion outperforms all the published multimodal works to date. Code is available at: https://github.com/yinjunbo/IS-Fusion.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
BIRANet improves target detection using Attentive FPN with spatial and channel scSE modules.
“BIRANet [93] improves target detection using Attentive FPN with spatial and channel scSE modules.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [93]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · see citing works
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · see citing works
The Hungarian method for the assignment problem
Kuhn, H. W. (1955) · Naval Research Logistics Quarterly · doi:10.1002/nav.3800020109 · Cited by 13,120 · see citing works
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522 · see citing works
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles, R., Su, H., Mo, K., & Guibas, L. J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.16 · Cited by 9,767 · see citing works
nuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152 · cited 4× · see citing works
“The image encoder is pre-trained on the nuIm-age dataset [2] following current approaches [1, 46, 77].”
BackgroundMethodInfluentialCSPNet: A New Backbone that can Enhance Learning Capability of CNN
Wang, C.-Y., Mark Liao, H.-Y., Wu, Y.-H., Chen, P.-Y., Hsieh, J.-W., & Yeh, I.-H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00203 · Cited by 5,265 · see citing works
VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
Zhou, Y., & Tuzel, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00472 · Cited by 4,813 · see citing works
PointPillars: Fast Encoders for Object Detection From Point Clouds
Lang, A. H., Vora, S., Caesar, H., Zhou, L., Yang, J., & Beijbom, O. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01298 · Cited by 4,673 · see citing works
Multi-view 3D Object Detection Network for Autonomous Driving
Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.691 · Cited by 3,450 · see citing works
SECOND: Sparsely Embedded Convolutional Detection
Yan, Y., Mao, Y., & Li, B. (2018) · Sensors · doi:10.3390/s18103337 · Cited by 3,352 · see citing works
Cyclical Learning Rates for Training Neural Networks
Smith, L. N. (2017) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2017.58 · Cited by 2,931 · see citing works
Zhao, H., Jiang, L., Jia, J., Torr, P. H. S., & Koltun, V. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01595 · Cited by 2,847 · see citing works
Frustum PointNets for 3D Object Detection from RGB-D Data
Qi, C. R., Liu, W., Wu, C., Su, H., & Guibas, L. J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00102 · Cited by 2,655 · see citing works
PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection
Shi, S., Guo, C., Jiang, L., Wang, Z., Shi, J., Wang, X., & Li, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01054 · Cited by 2,341 · see citing works
Center-based 3D Object Detection and Tracking
Yin, T., Zhou, X., & Krähenbühl, P. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01161 · Cited by 2,197 · see citing works
Deformable DETR: Deformable Transformers for End-to-End Object Detection
Zhu, X., Su, W., Lu, L., Li, B., Wang, X., & Dai, J. (2020) · arXiv · doi:10.48550/arxiv.2010.04159 · Cited by 1,837 · cited 1× · see citing works
“…( e.g ., D grid cells) around q ′ ins are considered to save computation costs, following the deformable attention in [92]: where f conv ( · ) is a 3 × 3 convolution operation to align the feature space between q ′ ins and B ′ F , and ˜ q ins ∈ { ˜ q 1 ins , ˜ q 2 ins , . . . , ˜ q K ins }…”
Background3D Semantic Segmentation with Submanifold Sparse Convolutional Networks
Graham, B., Engelcke, M., & Maaten, L. V. D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00961 · Cited by 1,788 · see citing works
Joint 3D Proposal Generation and Object Detection from View Aggregation
Ku, J., Mozifian, M., Lee, J., Harakeh, A., & Waslander, S. L. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594049 · Cited by 1,697 · see citing works
Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Yu, Q., & Dai, J. (2022) · arXiv · doi:10.48550/arxiv.2203.17270 · Cited by 1,419 · see citing works
Self-citationBevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation
Liu, Z., Tang, H., Amini, A., Yang, X., Mao, H., Rus, D. L., & Han, S. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160968 · Cited by 1,307 · see citing works
Deep Hough Voting for 3D Object Detection in Point Clouds
Qi, C. R., Litany, O., He, K., & Guibas, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00937 · Cited by 1,244 · see citing works
PointPainting: Sequential Fusion for 3D Object Detection
Vora, S., Lang, A. H., Helou, B., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00466 · Cited by 1,236 · see citing works
3DSSD: Point-Based 3D Single Stage Object Detector
Yang, Z., Sun, Y., Liu, S., & Jia, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01105 · Cited by 1,176 · see citing works
Wang, Y., Chao, W.-L., Garg, D., Hariharan, B., Campbell, M., & Weinberger, K. Q. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00864 · Cited by 1,154 · see citing works
Deep Continuous Fusion for Multi-sensor 3D Object Detection
Liang, M., Yang, B., Wang, S., & Urtasun, R. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_39 · Cited by 993 · see citing works
TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers
Bai, X., Hu, Z., Zhu, X., Huang, Q., Chen, Y., Fu, H., & Tai, C.-L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00116 · Cited by 990 · see citing works
Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud
Shi, W., & Rajkumar, R. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00178 · Cited by 979 · see citing works
STD: Sparse-to-Dense 3D Object Detector for Point Cloud
Yang, Z., Sun, Y., Liu, S., Shen, X., & Jia, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00204 · Cited by 875 · see citing works
PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation
Xu, D., Anguelov, D., & Jain, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00033 · Cited by 856 · see citing works
Multi-Task Multi-Sensor Fusion for 3D Object Detection
Liang, M., Yang, B., Chen, Y., Hu, R., & Urtasun, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00752 · Cited by 723 · see citing works
PETR: Position Embedding Transformation for Multi-view 3D Object Detection
Liu, Y., Wang, T., Zhang, X., & Sun, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19812-0_31 · Cited by 664 · see citing works
Stereo R-CNN Based 3D Object Detection for Autonomous Driving
Li, P., Chen, X., & Shen, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00783 · Cited by 597 · see citing works
DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection
Li, Y., Yu, A. W., Meng, T., Caine, B., Ngiam, J., Peng, D., Shen, J., Lu, Y., Zhou, D., Le, Q. V., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01667 · Cited by 563 · see citing works
Self-citationVoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking
Chen, Y., Liu, J., Zhang, X., Qi, X., & Jia, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02076 · Cited by 490 · see citing works
VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking
Chen, Y., Liu, J., Zhang, X., Qi, X., & Jia, J. (2023) · arXiv · doi:10.48550/arxiv.2303.11301 · Cited by 486 · see citing works
PointAugmenting: Cross-Modal Augmentation for 3D Object Detection
Wang, C., Ma, C., Zhu, M. B., & Yang, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01162 · Cited by 481 · see citing works
Chen, Y., Liu, S., Shen, X., & Jia, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00987 · Cited by 406 · see citing works
FUTR3D: A Unified Sensor Fusion Framework for 3D Detection
Chen, X., Zhang, T., Wang, Y., Wang, Y., & Zhao, H. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00022 · Cited by 337 · see citing works
Focal Sparse Convolutional Networks for 3D Object Detection
Chen, Y., Li, Y., Zhang, X., Sun, J. Z., & Jia, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00535 · Cited by 332 · see citing works
BEVFusion: A Simple and Robust LiDAR-Camera Fusion FrameworkPreprint
Liang, T., Xie, H., Yu, K., Xia, Z., Lin, Z., Wang, Y., Tang, T., Wang, B., & Tang, Z. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.13790 · Cited by 310 · see citing works
Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection
Zhu, B., Jiang, Z., Zhou, X., Li, Z., & Yu, G. (2019) · arXiv · doi:10.48550/arxiv.1908.09492 · Cited by 297 · see citing works
PillarNet: Real-Time and High-Performance Pillar-Based 3D Object Detection
Shi, G., Li, R., & Ma, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20080-9_3 · Cited by 255 · see citing works
Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds
He, C., Li, R., Li, S., & Zhang, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00823 · Cited by 249 · see citing works
Virtual Sparse Convolution for Multimodal 3D Object Detection
Wu, H. Y., Wen, C., Shi, S., Li, X. S., & Wang, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02074 · Cited by 243 · see citing works
Fusionpainting: Multimodal fusion with adaptive attention for 3d object detection
Xu, S., Zhou, D., Fang, J., Yin, J., Bin, Z., & Zhang, L. (2021) · 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) · doi:10.1109/itsc48978.2021.9564951 · Cited by 237 · see citing works
Self-citationEfficient DETR: Improving End-to-End Object Detector with Dense Prior
Yao, Z., Ai, J., Li, B., & Zhang, C. (2021) · arXiv · doi:10.48550/arxiv.2104.01318 · Cited by 190 · see citing works
LoGoNet: Towards Accurate 3D Object Detection with Local-to-Global Cross- Modal Fusion
Li, X. S., Ma, T., Hou, Y., Shi, B., Yang, Y., Liu, Y., Wu, X., Chen, Q., Li, Y., Qiao, Y., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01681 · Cited by 187 · see citing works
Point Density-Aware Voxels for LiDAR 3D Object Detection
Hu, J. S. K., Kuai, T., & Waslander, S. L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00828 · Cited by 180 · see citing works
Segment Any Point Cloud Sequences by Distilling Vision Foundation ModelsPreprint
Liu, Y.-C., Kong, L., Cen, J., Chen, R.-N., Zhang, W., Pan, L., Chen, K., & Liu, Z.-W. (2023) · Neural Information Processing Systems · doi:10.48550/arxiv.2306.09347 · Cited by 175 · see citing works
Self-citationCLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP
Chen, R., Liu, Y., Kong, L., Zhu, X., Ma, Y., Li, Y., Hou, Y., Qiao, Y., & Wang, W. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00678 · Cited by 161 · see citing works
Self-citationSparseFusion: Fusing Multi-Modal Sparse Representations for Multi-Sensor 3D Object Detection
Xie, Y., Xu, C., Rakotosaona, M., Rim, P., Tombari, F., Keutzer, K., Tomizuka, M., & Zhan, W. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01613 · Cited by 147 · see citing works
Depth-conditioned Dynamic Message Propagation for Monocular 3D Object Detection
Wang, L., Du, L., Ye, X., Fu, Y., Guo, G., Xue, X., Feng, J., & Zhang, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00052 · Cited by 146 · see citing works
Deformable feature aggregation for dynamic multi-modal 3D object detection
Chen, Z., Li, Z., Zhang, S., Fang, L., Jiang, Q., & Zhao, F. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20074-8_36 · Cited by 136 · see citing works
Towards A Weakly Supervised Framework for 3D Point Cloud Object Detection and Annotation
Meng, Q., Wang, W., Zhou, T., Shen, J., Jia, Y., & Gool, L. V. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3063611 · Cited by 134 · see citing works
Self-citationMultimodal Virtual Point 3D DetectionPreprint
Yin, T., Zhou, X., & Krähenbühl, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.06881 · Cited by 130 · see citing works
Unifying Voxel-based Representation with Transformer for 3D Object Detection
Li, Y., Chen, Y., Qi, X., Li, Z., Sun, J., & Jia, J. (2022) · arXiv · doi:10.48550/arxiv.2206.00630 · Cited by 125 · see citing works
Voxel Field Fusion for 3D Object Detection
Li, Y., Qi, X., Chen, Y., Wang, L., Li, Z., Sun, J. Z., & Jia, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00119 · Cited by 124 · see citing works
Joint 3D Instance Segmentation and Object Detection for Autonomous Driving
Zhou, D., Fang, J., Song, X., Liu, L., Yin, J., Dai, Y., Li, H., & Yang, R. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00191 · Cited by 109 · see citing works
Self-citationUniTR: A Unified and Efficient Multi-Modal Transformer for Bird’s-Eye-View Representation
Wang, H., Tang, H., Shi, S., Li, A., Li, Z., Schiele, B., & Wang, L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00625 · Cited by 100 · see citing works
Graph R-CNN: Towards Accurate 3D Object Detection with Semantic-Decorated Local Graph
Yang, H., Liu, Z., Wu, X., Wang, W., Qian, W., He, X., & Cai, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20074-8_38 · Cited by 98 · see citing works
Self-citationYin, J., Shen, J., Gao, X., Crandall, D., & Yang, R. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3125981 · Cited by 95 · see citing works
Self-citationOcTr: Octree-Based Transformer for 3D Object Detection
Zhou, C., Zhang, Y., Chen, J., & Huang, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00500 · Cited by 88 · see citing works
Semi-supervised 3D Object Detection with Proficient Teachers
Yin, J., Fang, J., Zhou, D., Zhang, L., Xu, C., Shen, J., & Wang, W. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19839-7_42 · Cited by 85 · see citing works
Self-citationSong, Z., Wei, H., Bai, L., Yang, L., & Jia, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00311 · Cited by 76 · see citing works
Bird’s-Eye-View Scene Graph for Vision-Language Navigation
Liu, R., Wang, X., Wang, W., & Yang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01007 · Cited by 69 · see citing works
Self-citation4D-Net for Learned Multi-Modal Alignment
Piergiovanni, A., Casser, V., Ryoo, M. S., & Angelova, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01515 · Cited by 64 · see citing works
ObjectFusion: Multi-modal 3D Object Detection with Object-Centric Fusion
Cai, Q., Pan, Y., Yao, T., Ngo, C., & Mei, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01656 · Cited by 63 · see citing works
PVGNet: A Bottom-Up One-Stage 3D Object Detector with Integrated Multi-Level Features
Miao, Z., Chen, J., Pan, H., Zhang, R., Liu, K., Hao, P., Zhu, J., Wang, Y., & Zhan, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00329 · Cited by 58 · see citing works
SSDA3D: Semi-supervised Domain Adaptation for 3D Object Detection from Point Cloud
Wang, Y., Yin, J., Li, W., Pascal Frossard, B., Yang, R., & Shen, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25370 · Cited by 51 · see citing works
Self-citationLi, X., Yin, J., Li, W., Xu, C., Yang, R., & Shen, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i4.28105 · Cited by 51 · see citing works
Self-citationBRNet: Exploring Comprehensive Features for Monocular Depth Estimation
Han, W., Yin, J., Jin, X., Xiangdong, D., & Shen, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19839-7_34 · Cited by 46 · see citing works
Self-citationClustering based Point Cloud Representation Learning for 3D Analysis
Feng, T., Wang, W., Wang, X., Yang, Y., & Zheng, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00761 · Cited by 45 · see citing works
Self-citationSelf-Supervised Monocular Depth Estimation by Direction-aware Cumulative Convolution Network
Han, W., Yin, J., & Shen, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00791 · Cited by 35 · see citing works
Self-citationMSF: Motion-guided Sequential Fusion for Efficient 3D Object Detection from Point Cloud Sequences
He, C., Li, R., Zhang, Y., Li, S., & Lei Zhang, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00503 · Cited by 30 · see citing works
LSK3DNet: Towards Effective and Efficient 3D Perception with Large Sparse Kernels
Feng, T., Wang, W., Ma, F., & Yang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01413 · Cited by 30 · see citing works
Self-citationSee More and Know More: Zero-shot Point Cloud Segmentation via Multi-modal Visual Data
Lu, Y., Jiang, Q., Chen, R., Hou, Y., Zhu, X., & Ma, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01981 · Cited by 29 · see citing works
Self-citationInterpretable3D: An Ad-Hoc Interpretable Classifier for 3D Point Clouds
Feng, T., Quan, R., Wang, X., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i2.27944 · Cited by 23 · see citing works
LWSIS: LiDAR-Guided Weakly Supervised Instance Segmentation for Autonomous Driving
Xiang, L., Yin, J., Shi, B., et al. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i2.25228 · Cited by 16 · see citing works
Self-citationTowards Label-free Scene Understanding by Vision Foundation ModelsPreprint
Chen, R., Liu, Y., Kong, L., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.03899 · Cited by 15 · see citing works
Self-citationCurricular Object Manipulation in LiDAR-based Object Detection
Zhu, Z., Meng, Q., Wang, X., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00115 · Cited by 13 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 41 · see citing works
“Future research will likely focus on improving scalability, real-time capabilities, and developing unified frameworks that integrate localization with high-level perception and decision-making tasks [145].”
- Authors
- Yin, J., Shen, J., Chen, R., Li, W., Yang, R., Frossard, P., & Wang, W.
- Affiliations
- Beijing Institute of Technology (CN)
- University of Macau (MO)
- University of Hong Kong (HK)
- Inceptio
- École Polytechnique Fédérale de Lausanne (EPFL)
- Zhejiang University (CN)
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 14905-14915
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402703030
- Cited by
- 133
- References
- 83
- Citations by year
- 2026: 60 · 2025: 58 · 2024: 15
- FWCI
- 27.31
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- arXiv (Cornell University)
- Funders
- FDCT (0154/2022/A3,0102/2023/RIA2,001/2024/SKL,MYRG-CRG2022-00013-IOTSC-ICI,SRG2022-00023-IOTSC)
- National Natural Science Foundation of China (62372405)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- multimodal fusion, 3D object detection, bird's-eye view, point clouds, autonomous driving perception, nuScenes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
