The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completion
Open accessAbstract
Current LiDAR-only 3D detection methods inevitably suffer from the sparsity of point clouds. Many multi-modal methods are proposed to alleviate this issue, while different representations of images and point clouds make it difficult to fuse them, resulting in suboptimal performance. In this paper, we present a novel multi-modal framework SFD (Sparse Fuse Dense), which utilizes pseudo point clouds generated from depth completion to tackle the issues mentioned above. Different from prior works, we propose a new RoI fusion strategy 3D-GAF (3D Grid-wise Attentive Fusion) to make fuller use of information from different types of point clouds. Specifically, 3D-GAF fuses 3D RoI features from the pair of point clouds in a grid-wise attentive way, which is more fine- grained and more precise. In addition, we propose a SynAugment (Synchronized Augmentation) to enable our multi-modal framework to utilize all data augmentation approaches tailored to LiDAR-only methods. Lastly, we customize an effective and efficient feature extractor CPConv (Color Point Convolution) for pseudo point clouds. It can explore 2D image features and 3D geometric features of pseudo point clouds simultaneously. Our method holds the highest entry on the KITTI car 3D object detection leaderboard††On the date of CVPR deadline, i.e., Nov.16, 2021, demonstrating the effectiveness of our SFD. Code will be made publicly available.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,667
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
PointPillars: Fast Encoders for Object Detection From Point Clouds
Lang, A. H., Vora, S., Caesar, H., Zhou, L., Yang, J., & Beijbom, O. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01298 · Cited by 4,673
Multi-view 3D Object Detection Network for Autonomous Driving
Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.691 · Cited by 3,450
SECOND: Sparsely Embedded Convolutional Detection
Yan, Y., Mao, Y., & Li, B. (2018) · Sensors · doi:10.3390/s18103337 · Cited by 3,352
Li, X., Wang, W., Hu, X., & Yang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00060 · Cited by 3,134
PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud
Shi, S., Wang, X., & Li, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00086 · Cited by 3,093
PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection
Shi, S., Guo, C., Jiang, L., Wang, Z., Shi, J., Wang, X., & Li, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01054 · Cited by 2,341
Center-based 3D Object Detection and Tracking
Yin, T., Zhou, X., & Krahenbuhl, P. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01161 · Cited by 2,082
Joint 3D Proposal Generation and Object Detection from View Aggregation
Ku, J., Mozifian, M., Lee, J., Harakeh, A., & Waslander, S. L. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594049 · Cited by 1,697
PointPainting: Sequential Fusion for 3D Object Detection
Vora, S., Lang, A. H., Helou, B., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00466 · Cited by 1,236 · cited 3×
“Some works [34,38] that project image segmentation scores to raw clouds can also use multi-modal data augmentation, but the image information carried by raw clouds is sparse due to the sparse correspondence between images and point clouds.”
BackgroundWang, Y., Chao, W.-L., Garg, D., Hariharan, B., Campbell, M., & Weinberger, K. Q. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00864 · Cited by 1,154 · cited 1×
“In the imagebased 3D object detection, there are some works [36, 46] that use depth estimation to generate pseudo point clouds.”
MethodVoxel R-CNN: Towards High Performance Voxel-based 3D Object Detection
Deng, J., Shi, S., Li, P., Zhou, W., Zhang, Y., & Li, H. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16207 · Cited by 1,048
Deep Continuous Fusion for Multi-sensor 3D Object Detection
Liang, M., Yang, B., Wang, S., & Urtasun, R. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_39 · Cited by 974
Shi, S., Wang, Z., Shi, J., Wang, X., & Li, H. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2977026 · Cited by 918
STD: Sparse-to-Dense 3D Object Detector for Point Cloud
Yang, Z., Sun, Y., Liu, S., Shen, X., & Jia, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00204 · Cited by 869
PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation
Xu, D., Anguelov, D., & Jain, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00033 · Cited by 856
Structure Aware Single-Stage 3D Object Detection From Point Cloud
He, C., Zeng, H., Huang, J., Hua, X.-S., & Zhang, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01189 · Cited by 651
CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection
Pang, S., Morris, D., & Radha, H. (2020) · 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341791 · Cited by 563
Mvx-net: Multimodal voxelnet for 3d object detection
Sindagi, V. A., Zhou, Y., & Tuzel, O. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794195 · Cited by 554
Wang, Z., & Jia, K. (2019) · 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968513 · Cited by 534
Yoo, J. H., Kim, Y., Kim, J., & Choi, J. W. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58583-9_43 · Cited by 520
Voxel Transformer for 3D Object Detection
Mao, J., Xue, Y., Niu, M., Bai, H., Feng, J., Liang, X., Xu, H., & Xu, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00315 · Cited by 520
Epnet: Enhancing point features with image semantics for 3d object detection
Huang, T., Liu, Z., Chen, X., & Bai, X. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58555-6_3 · Cited by 518
PointAugmenting: Cross-Modal Augmentation for 3D Object Detection
Wang, C., Ma, C., Zhu, M., & Yang, X. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01162 · Cited by 474
SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloud
Zheng, W., Tang, W., Jiang, L., & Fu, C.-W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01426 · Cited by 425
Chen, Y., Liu, S., Shen, X., & Jia, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00987 · Cited by 402
TANet: Robust 3D Object Detection from Point Clouds with Triple Attention
Liu, Z., Zhao, X., Huang, T., Hu, R., Zhou, Y., & Bai, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6837 · Cited by 392
PENet: Towards Precise and Efficient Image Guided Depth Completion
Hu, M., Wang, S., Li, B., Ning, S., Fan, L., & Gong, X. (2021) · 2021 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48506.2021.9561035 · Cited by 338
CIA-SSD: Confident IoU-Aware Single-Stage Object Detector From Point Cloud
Zheng, W., Tang, W., Chen, S., Jiang, L., & Fu, C.-W. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i4.16470 · Cited by 313
Improving 3D Object Detection with Channel-wise Transformer
Shenga, H., Cai, S., Liu, Y., Deng, B., Huang, J., Hua, X.-S., & Zhao, M.-J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00274 · Cited by 282
RangeDet: In Defense of Range View for LiDAR-based 3D Object Detection
Fan, L., Xiong, X., Wang, F., Wang, N., & Zhang, Z. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00291 · Cited by 257
HVNet: Hybrid Voxel Network for LiDAR Based 3D Object Detection
Ye, M., Xu, S., & Cao, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00170 · Cited by 242
LiDAR R-CNN: An Efficient and Universal 3D Object Detector
Li, Z., Wang, F., & Wang, N. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00746 · Cited by 242
Xie, L., Xiang, C., Yu, Z., Xu, G., Yang, Z., Cai, D., & He, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6933 · Cited by 237
Self-citationPyramid R-CNN: Towards Better Performance and Adaptability for 3D Object Detection
Mao, J., Niu, M., Bai, H., Liang, X., Xu, H., & Xu, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00272 · Cited by 187
SPG: Unsupervised Domain Adaptation for 3D Object Detection via Semantic Point Generation
Xu, Q., Zhou, Y., Wang, W., Qi, C. R., & Anguelov, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01516 · Cited by 160
Depth Completion with Twin Surface Extrapolation at Occlusion Boundaries
Imran, S., Liu, X., & Morris, D. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00261 · Cited by 134
3D Object Detection Using Scale Invariant and Feature Reweighting Networks
Zhao, X., Liu, Z., Hu, R., & Huang, K. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019267 · Cited by 124
Sparse Auxiliary Networks for Unified Monocular Depth Prediction and Completion
Guizilini, V., Ambruş, R., Burgard, W., & Gaidon, A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01093 · Cited by 78
To the Point: Efficient 3D Object Detection in the Range Image with Graph Convolution Kernels
Chai, Y., Sun, P., Ngiam, J., Wang, W., Caine, B., Vasudevan, V., Zhang, X., & Anguelov, D. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01574 · Cited by 71
DenseLiDAR: A Real-Time Pseudo Dense Depth Guided Depth Completion Network
Gu, J., Xiang, Z., Ye, Y., & Wang, L. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3060396 · Cited by 58
Ku, J., Pon, A., Walsh, S., & Waslander, S. L. (2019) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968242 · Cited by 31
Multi-Modality Cut and Paste for 3D Object Detection.Preprint
Zhang, W., Wang, Z., & Loy, C. C. (2020) · arXiv (Cornell University) · Cited by 18
Real-Time Anchor-Free Single-Stage 3D Detection with IoU-AwarenessPreprint
Ge, R., Ding, Z., Hu, Y., Shao, W., Huang, L., Li, K., & Liu, Q. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.14342 · Cited by 11
Cited by
Multi-Modal 3D Object Detection in Autonomous Driving: A Survey and Taxonomy
Wang, L., Zhang, X., Song, Z., Bi, J., Zhang, G., Wei, H., Tang, L., Yang, L., Li, J., Jia, C., et al. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3264658 · Cited by 269
A Comprehensive Survey on Deep Learning Multi-Modal Fusion: Methods, Technologies and Applications
Jiao, T., Guo, C., Feng, X., Chen, Y., & Song, J. (2024) · Computers, Materials & Continua · doi:10.32604/cmc.2024.053204 · Cited by 158
Multi-Sem Fusion: Multimodal Semantic Fusion for 3-D Object Detection
Xu, S., Li, F., Song, Z., Fang, J., Wang, S., & Yang, Z.-X. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3387732 · Cited by 26
Metadata
- Authors
- Wu, X., Peng, L., Yang, H., Xie, L., Huang, C., Deng, C., Liu, H., & Cai, D.
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 5408-5417
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3217335336
- Cited by
- 277
- References
- 46
- FWCI
- 23.95
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.