The DOI matches its Crossref record; 83 of the 91 references it cites were found in a registry or an index.
FocalFormer3D : Focusing on Hard Instance for 3D Object Detection
Open accessAbstract
False negatives (FN) in 3D object detection, e.g., missing predictions of pedestrians, vehicles, or other obstacles, can lead to potentially dangerous situations in autonomous driving. While being fatal, this issue is understudied in many current 3D detection methods. In this work, we propose Hard Instance Probing (HIP), a general pipeline that identifies FN in a multi-stage manner and guides the models to focus on excavating difficult instances. For 3D object detection, we instantiate this method as FocalFormer3D, a simple yet effective detector that excels at excavating difficult objects and improving prediction recall. FocalFormer3D features a multi-stage query generation to discover hard objects and a box-level transformer decoder to efficiently distinguish objects from massive object candidates. Experimental results on the nuScenes and Waymo datasets validate the superior performance of FocalFormer3D. The advantage leads to strong performance on both detection and tracking, in both LiDAR and multi-modal settings. Notably, FocalFormer3D achieves a 70.5 mAP and 73.9 NDS on nuScenes detection benchmark, while the nuScenes tracking benchmark shows 72.1 AMOTA, both ranking 1st place on the nuScenes LiDAR leaderboard. Our code is available at https://github.com/NVlabs/FocalFormer3D.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In contrast, radarcamera fusion enhances motion detection by leveraging radar velocity information, with RCBEVDet++ [263] demonstrating greater stability in dynamic scenes, such as high-speed driving and urban intersections.”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [263]
Citations
Cites
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 1×
“This allows for box-level query interaction and iterative box refinements, binding the object queries with sufficient box context through RoIAlign [20, 21] on the bird’s eye view to perform relative bounding box refinements.”
MethodMicrosoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · cited 1×
“Following TransFusion [4], we use a pre-trained ResNet-50 model on COCO [80] and nuIm-age [62] dataset as the image model and freeze its weights during training.”
MethodFocal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · cited 1×
“Recent research [65–67] has further explored soft-sampling, such as adjusting the loss distribution to mitigate foreground-background imbalance issues.”
BackgroundHe, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755 · cited 1×
“The generation of multi-stage BEV features is accomplished in a cascade manner using a lightweight inversed residual block [68] between stages.”
MethodGirshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285 · cited 2×
“To enhance the efficiency of object query processing, we employ deformable attention [17] instead of computationally intensive modules such as cross attention [59] or box attention [70].”
BackgroundMethodAre we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · cited 1×
“…detected objects ( Positive predictions) at k -th stage are We are then allowed to classify the ground-truth objects according to their assigned candidates: where an object matching metric σ ( · , · ) (e.g. Intersection over Union [60, 61] and center distance [62]) and a prede-fined threshold η .”
BackgroundPointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric SpacePreprint
Qi, C. R., Yi, L., Su, H., & Guibas, L. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02413 · Cited by 10,099 · cited 1×
“…LiDAR-based [1– 3, 12, 13, 22–29], or Camera-based [30–37], or Multi-Modal [4–8 , 38– 45] 3D object detectors Different from dense detectors, point-based 3D detectors [11, 50–52] directly process point clouds via Point-Net [53, 54] and perform grouping or predictions on the sparse representations.”
BackgroundPointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles, R., Su, H., Mo, K., & Guibas, L. J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.16 · Cited by 10,029 · cited 1×
“…LiDAR-based [1– 3, 12, 13, 22–29], or Camera-based [30–37], or Multi-Modal [4–8 , 38– 45] 3D object detectors Different from dense detectors, point-based 3D detectors [11, 50–52] directly process point clouds via Point-Net [53, 54] and perform grouping or predictions on the sparse representations.”
BackgroundCascade R-CNN: Delving Into High Quality Object Detection
Cai, Z., & Vasconcelos, N. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00644 · Cited by 7,370 · cited 1×
“Motivated by cascade-style decoder head for object detection [16–18], we propose a pipeline to probe false negative samples progressively, which signifi-cantly improves the recall rate Fig.”
MethodnuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152 · cited 3×
“Following TransFusion [4], we use a pre-trained ResNet-50 model on COCO [80] and nuIm-age [62] dataset as the image model and freeze its weights during training.”
BackgroundMethodDeformable DETR: Deformable Transformers for End-to-End Object Detection
Zhu, X., Su, W., Lu, L., Li, B., Wang, X., & Dai, J. (2020) · arXiv · doi:10.48550/arxiv.2010.04159 · Cited by 6,027 · cited 3×
“To enhance the efficiency of object query processing, we employ deformable attention [17] instead of computationally intensive modules such as cross attention [59] or box attention [70].”
MethodVoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
Zhou, Y., & Tuzel, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00472 · Cited by 4,813 · cited 3×
“The computation time is mostly taken up by the sparse convolution-based backbone network (VoxelNet [1, 22]), which takes 78ms.”
BackgroundMethodPointPillars: Fast Encoders for Object Detection From Point Clouds
Lang, A. H., Vora, S., Caesar, H., Zhou, L., Yang, J., & Beijbom, O. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01298 · Cited by 4,673 · cited 1×
“Some Pillar-based detectors [2, 23, 48, 49] collapse the height dimension and utilize 2D CNNs for efficient 3D detection.”
MethodMulti-view 3D Object Detection Network for Autonomous Driving
Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.691 · Cited by 3,450
Scalability in Perception for Autonomous Driving: Waymo Open Dataset
Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., Caine, B., et al. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00252 · Cited by 3,437 · cited 2×
“Waymo Open Dataset [61] has a wider detection range of 150 m × 150 m compared to the nuScenes dataset.”
BackgroundMethodSECOND: Sparsely Embedded Convolutional Detection
Yan, Y., Mao, Y., & Li, B. (2018) · Sensors · doi:10.3390/s18103337 · Cited by 3,352 · cited 2×
“The computation time is mostly taken up by the sparse convolution-based backbone network (VoxelNet [1, 22]), which takes 78ms.”
BackgroundMethodPointRCNN: 3D Object Proposal Generation and Detection From Point Cloud
Shi, S., Wang, X., & Li, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00086 · Cited by 3,093 · cited 2×
“There is a rich yet growing literature on BEV-based 3D detection, including multi-modal fusion [4–10], second-stage refinements (surface point pooling [ 3], RoIPool [11– 14], and cross attention modules [4, 15]).”
BackgroundTraining Region-Based Object Detectors with Online Hard Example Mining
Shrivastava, A., Gupta, A., & Girshick, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.89 · Cited by 2,686 · cited 1×
“The most relevant topic close to our approach is hard example mining [63, 64], which samples hard examples during training.”
BackgroundPV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection
Shi, S., Guo, C., Jiang, L., Wang, Z., Shi, J., Wang, X., & Li, H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01054 · Cited by 2,341
Center-based 3D Object Detection and Tracking
Yin, T., Zhou, X., & Krähenbühl, P. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01161 · Cited by 2,315 · cited 9×
“The neural network makes Positive or Negative predictions given a set of initial object candidates A = { a i , i = 1 , 2 , ... } , which is not limited to anchors [58], point-based anchors [3], and object queries [59].”
BackgroundMethodInfluential3D Semantic Segmentation with Submanifold Sparse Convolutional Networks
Graham, B., Engelcke, M., & Maaten, L. V. D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00961 · Cited by 1,788
Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Yu, Q., & Dai, J. (2022) · arXiv · doi:10.48550/arxiv.2203.17270 · Cited by 1,419
Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation
Liu, Z., Tang, H., Amini, A., Yang, X., Mao, H., Rus, D. L., & Han, S. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160968 · Cited by 1,307 · cited 2×
“Moreover, our single model FocalFormer3D-F with double-flip testing results performs even better than the BEVFusion [6] with model ensembling.”
MethodLift, Splat, Shoot: Encoding Images from Arbitrary Camera Rigs by Implicitly Unprojecting to 3D
Philion, J., & Fidler, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58568-6_12 · Cited by 1,271 · cited 1×
“Unlike heavy lift-splat-shot [32] camera encoders used in BEV-Fusion [5, 6], the multi-view camera images are projected onto a pre-defined voxel space and fused with LiDAR BEV feature.”
MethodPIXOR: Real-time 3D Object Detection from Point Clouds
Yang, B., Luo, W., & Urtasun, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00798 · Cited by 1,249
Deep Hough Voting for 3D Object Detection in Point Clouds
Qi, C. R., Litany, O., He, K., & Guibas, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00937 · Cited by 1,244 · cited 1×
“…LiDAR-based [1– 3, 12, 13, 22–29], or Camera-based [30–37], or Multi-Modal [4–8 , 38– 45] 3D object detectors Different from dense detectors, point-based 3D detectors [11, 50–52] directly process point clouds via Point-Net [53, 54] and perform grouping or predictions on the sparse representations.”
BackgroundPointPainting: Sequential Fusion for 3D Object Detection
Vora, S., Lang, A. H., Helou, B., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00466 · Cited by 1,236
3DSSD: Point-Based 3D Single Stage Object Detector
Yang, Z., Sun, Y., Liu, S., & Jia, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01105 · Cited by 1,176 · cited 1×
“…LiDAR-based [1– 3, 12, 13, 22–29], or Camera-based [30–37], or Multi-Modal [4–8 , 38– 45] 3D object detectors Different from dense detectors, point-based 3D detectors [11, 50–52] directly process point clouds via Point-Net [53, 54] and perform grouping or predictions on the sparse representations.”
BackgroundSelf-citationDINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection
Zhang, H., Li, F., Liu, S., Zhang, L., Su, H., Zhu, J., Ni, L. M., & Shum, H.-Y. (2022) · arXiv · Cited by 1,154 · cited 1×
“To address this, we generate noisy queries from ground-truth objects [18, 77, 78], enabling effective training of the model from scratch.”
MethodDN-DETR: Accelerate DETR Training by Introducing Query DeNoising
Li, F., Zhang, H., Liu, S., Guo, J., Ni, L. M., & Zhang, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01325 · Cited by 1,102 · cited 3×
“To address this, we generate noisy queries from ground-truth objects [18, 77, 78], enabling effective training of the model from scratch.”
BackgroundMethodVoxel R-CNN: Towards High Performance Voxel-based 3D Object Detection
Deng, J., Shi, S., Li, P., Zhou, W., Zhang, Y., & Li, H. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16207 · Cited by 1,048 · cited 1×
“Modern 3D object detectors, either LiDAR-based [1– 3, 12, 13, 22–29], or Camera-based [30–37], or Multi-Modal [4–8 , 38– 45] 3D object detectors Different from dense detectors, point-based 3D detectors [11, 50–52] directly process point clouds via Point-Net [53, 54] and perform grouping or…”
BackgroundDeep Continuous Fusion for Multi-sensor 3D Object Detection
Liang, M., Yang, B., Wang, S., & Urtasun, R. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_39 · Cited by 1,005 · cited 1×
“Modern 3D object detectors, either LiDAR-based [1– 3, 12, 13, 22–29], or Camera-based [30–37], or Multi-Modal [4–8 , 38– 45] 3D object detectors Different from dense detectors, point-based 3D detectors [11, 50–52] directly process point clouds via Point-Net [53, 54] and perform grouping or…”
BackgroundTransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers
Bai, X., Hu, Z., Zhu, X., Huang, Q., Chen, Y., Fu, H., & Tai, C.-L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00116 · Cited by 990 · cited 9×
“The object candidates obtained from the multi-stage heatmap encoder can be treated as positional object queries [4, 69].”
BackgroundMethodInfluentialSelf-citationSTD: Sparse-to-Dense 3D Object Detector for Point Cloud
Yang, Z., Sun, Y., Liu, S., Shen, X., & Jia, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00204 · Cited by 888
Self-citationPETR: Position Embedding Transformation for Multi-view 3D Object Detection
Liu, Y., Wang, T., Zhang, X., & Sun, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19812-0_31 · Cited by 764
Multi-Task Multi-Sensor Fusion for 3D Object Detection
Liang, M., Yang, B., Chen, Y., Hu, R., & Urtasun, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00752 · Cited by 723
Self-citation3D Multi-Object Tracking: A Baseline and New Evaluation Metrics
Weng, X., Wang, J., Held, D., & Kitani, K. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341164 · Cited by 604
Shi, S., Jiang, L., Deng, J., Wang, Z., Guo, C., Shi, J., Wang, X., & Li, H. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01710-9 · Cited by 602
Gradient Harmonized Single-Stage Detector
Li, B., Liu, Y., & Wang, X. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018577 · Cited by 595
DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection
Li, Y., Yu, A. W., Meng, T., Caine, B., Ngiam, J., Peng, D., Shen, J., Lu, Y., Zhou, D., Le, Q. V., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01667 · Cited by 563 · cited 1×
“Recent multi-modal detectors [5–7, 39, 42, 57] follow the similar paradigm of BEV detectors and incorporate the multi-view image features by physical projection or learnable alignments between LiDAR and cameras.”
MethodYoo, J. H., Kim, Y., Kim, J., & Choi, J. W. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58583-9_43 · Cited by 520
VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking
Chen, Y., Liu, J., Zhang, X., Qi, X., & Jia, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02076 · Cited by 518 · cited 1×
“Modern 3D object detectors, either LiDAR-based [1– 3, 12, 13, 22–29], or Camera-based [30–37], or Multi-Modal [4–8 , 38– 45] 3D object detectors Different from dense detectors, point-based 3D detectors [11, 50–52] directly process point clouds via Point-Net [53, 54] and perform grouping or…”
BackgroundSelf-citationBritish Machine Vision Conference 1999
Pridmore, T. (2002) · Image and Vision Computing · doi:10.1016/s0262-8856(02)00002-1 · Cited by 505
PointAugmenting: Cross-Modal Augmentation for 3D Object Detection
Wang, C., Ma, C., Zhu, M. B., & Yang, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01162 · Cited by 495 · cited 1×
“Recent multi-modal detectors [5–7, 39, 42, 57] follow the similar paradigm of BEV detectors and incorporate the multi-view image features by physical projection or learnable alignments between LiDAR and cameras.”
MethodVoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking
Chen, Y., Liu, J., Zhang, X., Qi, X., & Jia, J. (2023) · arXiv · doi:10.48550/arxiv.2303.11301 · Cited by 486
Self-citationChen, Y., Liu, S., Shen, X., & Jia, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00987 · Cited by 411 · cited 1×
“Modern 3D object detectors, either LiDAR-based [1– 3, 12, 13, 22–29], or Camera-based [30–37], or Multi-Modal [4–8 , 38– 45] 3D object detectors Different from dense detectors, point-based 3D detectors [11, 50–52] directly process point clouds via Point-Net [53, 54] and perform grouping or…”
BackgroundSelf-citationFUTR3D: A Unified Sensor Fusion Framework for 3D Detection
Chen, X., Zhang, T., Wang, Y., Wang, Y., & Zhao, H. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00022 · Cited by 380 · cited 1×
“Recent multi-modal detectors [5–7, 39, 42, 57] follow the similar paradigm of BEV detectors and incorporate the multi-view image features by physical projection or learnable alignments between LiDAR and cameras.”
MethodFocal Sparse Convolutional Networks for 3D Object Detection
Chen, Y., Li, Y., Zhang, X., Sun, J. Z., & Jia, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00535 · Cited by 346
Self-citationEmbracing Single Stride 3D Object Detector with Sparse Transformer
Fan, L., Pang, Z., Zhang, T., Wang, Y.-X., Zhao, H., Wang, F., Wang, N., & Zhang, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00827 · Cited by 317 · cited 2×
“SST [55] applies sparse regional attention and avoids downsampling for small-object detection.”
BackgroundGroup-Free 3D Object Detection via Transformers
Liu, Z., Zhang, Z., Cao, Y., Hu, H., & Tong, X. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00294 · Cited by 315 · cited 1×
“There is a rich yet growing literature on BEV-based 3D detection, including multi-modal fusion [4–10], second-stage refinements (surface point pooling [ 3], RoIPool [11– 14], and cross attention modules [4, 15]).”
BackgroundBEVFusion: A Simple and Robust LiDAR-Camera Fusion FrameworkPreprint
Liang, T., Xie, H., Yu, K., Xia, Z., Lin, Z., Wang, Y., Tang, T., Wang, B., & Tang, Z. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.13790 · Cited by 310 · cited 2×
“Recent multi-modal detectors [5–7, 39, 42, 57] follow the similar paradigm of BEV detectors and incorporate the multi-view image features by physical projection or learnable alignments between LiDAR and cameras.”
MethodClass-balanced Grouping and Sampling for Point Cloud 3D Object Detection
Zhu, B., Jiang, Z., Zhou, X., Li, Z., & Yu, G. (2019) · arXiv · doi:10.48550/arxiv.1908.09492 · Cited by 297
BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-ViewPreprint
Huang, J., Huang, G., Zhu, Z. D., Ye, Y., & Du, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.11790 · Cited by 292
PillarNet: Real-Time and High-Performance Pillar-Based 3D Object Detection
Shi, G., Li, R., & Ma, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20080-9_3 · Cited by 270
Prime Sample Attention in Object Detection
Cao, Y., Chen, K., Loy, C. C., & Lin, D. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01160 · Cited by 258 · cited 1×
“Recent research [65–67] has further explored soft-sampling, such as adjusting the loss distribution to mitigate foreground-background imbalance issues.”
BackgroundLiDAR R-CNN: An Efficient and Universal 3D Object Detector
Li, Z., Wang, F., & Wang, N. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00746 · Cited by 251 · cited 1×
“There is a rich yet growing literature on BEV-based 3D detection, including multi-modal fusion [4–10], second-stage refinements (surface point pooling [ 3], RoIPool [11– 14], and cross attention modules [4, 15]).”
BackgroundFusionpainting: Multimodal fusion with adaptive attention for 3d object detection
Xu, S., Zhou, D., Fang, J., Yin, J., Bin, Z., & Zhang, L. (2021) · 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) · doi:10.1109/itsc48978.2021.9564951 · Cited by 237
DSGN: Deep Stereo Geometry Network for 3D Object Detection
Chen, Y., Liu, S., Shen, X., & Jia, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01255 · Cited by 227
Self-citationLearning and example selection for object and pattern detection
Sung, K.-K., & Poggio, T. (1996) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 205
Efficient DETR: Improving End-to-End Object Detector with Dense Prior
Yao, Z., Ai, J., Li, B., & Zhang, C. (2021) · arXiv · doi:10.48550/arxiv.2104.01318 · Cited by 190 · cited 2×
“Unlike previous methods that used center point features as the query embedding [4, 69], we model the object candidates as box-level queries.”
MethodCenterFormer: Center-Based Transformer for 3D Object Detection
Zhou, Z., Zhao, X., Wang, Y., Wang, P., & Foroosh, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19839-7_29 · Cited by 183
RSN: Range Sparse Net for Efficient, Accurate LiDAR 3D Object Detection
Sun, P., Wang, W., Chai, Y., Elsayed, G. F., Bewley, A., Zhang, X., Sminchisescu, C., & Anguelov, D. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00567 · Cited by 167
SimpleTrack: Understanding and Rethinking 3D Multi-object Tracking
Pang, Z., Li, Z., & Wang, N. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25056-9_43 · Cited by 167 · cited 1×
“Interested readers can refer to the original paper [81] for more comprehensive details.”
BackgroundAFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Clouds
Hu, Y., Ding, Z., Ge, R., Shao, W., Huang, L., Li, K., & Liu, Q. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19980 · Cited by 137
Deformable feature aggregation for dynamic multi-modal 3D object detection
Chen, Z., Li, Z., Zhang, S., Fang, L., Jiang, Q., & Zhao, F. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20074-8_36 · Cited by 136
Multimodal Virtual Point 3D DetectionPreprint
Yin, T., Zhou, X., & Krähenbühl, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.06881 · Cited by 131
Voxel Field Fusion for 3D Object Detection
Li, Y., Qi, X., Chen, Y., Wang, L., Li, Z., Sun, J. Z., & Jia, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00119 · Cited by 128 · cited 1×
“Modern 3D object detectors, either LiDAR-based [1– 3, 12, 13, 22–29], or Camera-based [30–37], or Multi-Modal [4–8 , 38– 45] 3D object detectors Different from dense detectors, point-based 3D detectors [11, 50–52] directly process point clouds via Point-Net [53, 54] and perform grouping or…”
BackgroundSelf-citationUnifying Voxel-based Representation with Transformer for 3D Object Detection
Li, Y., Chen, Y., Qi, X., Li, Z., Sun, J., & Jia, J. (2022) · arXiv · doi:10.48550/arxiv.2206.00630 · Cited by 125 · cited 1×
“Recent multi-modal detectors [5–7, 39, 42, 57] follow the similar paradigm of BEV detectors and incorporate the multi-view image features by physical projection or learnable alignments between LiDAR and cameras.”
MethodSelf-citationBEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous DrivingPreprint
Zhang, Y., Zhu, Z. D., Zheng, W., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.09743 · Cited by 110
DeepInteraction: 3D Object Detection via Modality Interaction
Yang, Z., Chen, J., Miao, Z., Li, W., Zhu, X., & Zhang, L. (2022) · arXiv · doi:10.48550/arxiv.2208.11112 · Cited by 108 · cited 1×
“There is a rich yet growing literature on BEV-based 3D detection, including multi-modal fusion [4–10], second-stage refinements (surface point pooling [ 3], RoIPool [11– 14], and cross attention modules [4, 15]).”
BackgroundAFDet: Anchor Free One Stage 3D Object DetectionPreprint
Ge, R., Ding, Z., Hu, Y., Wang, Y., Chen, S., Huang, L., & Li, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12671 · Cited by 107 · cited 1×
“Some Pillar-based detectors [2, 23, 48, 49] collapse the height dimension and utilize 2D CNNs for efficient 3D detection.”
MethodDeepInteraction: 3D Object Detection Via Modality Interaction
Yang, Z., Chen, J., Miao, Z., et al. (2022) · neural information processing systems · doi:10.52202/068431-0145 · Cited by 104
Learnable Online Graph Representations for 3D Multi-Object Tracking
Zaech, J.-N., Liniger, A., Dai, D., Danelljan, M., & Gool, L. V. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3145952 · Cited by 93
Score refinement for confidence-based 3D multi-object tracking
Benbarka, N., Schröder, J., & Zell, A. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636032 · Cited by 78
LidarMultiNet: Towards a Unified Multi-Task Network for LiDAR Perception
Ye, D., Zhou, Z., Chen, W., Xie, Y., Wang, Y., Wang, P., & Foroosh, H. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25429 · Cited by 78
4D-Net for Learned Multi-Modal Alignment
Piergiovanni, A., Casser, V., Ryoo, M. S., & Angelova, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01515 · Cited by 67 · cited 1×
“Modern 3D object detectors, either LiDAR-based [1– 3, 12, 13, 22–29], or Camera-based [30–37], or Multi-Modal [4–8 , 38– 45] 3D object detectors Different from dense detectors, point-based 3D detectors [11, 50–52] directly process point clouds via Point-Net [53, 54] and perform grouping or…”
BackgroundFully Sparse 3D Object Detection
Fan, L., Wang, F., Wang, N., & Zhang, Z. (2022) · neural information processing systems · doi:10.52202/068431-0026 · Cited by 65
DSGN++: Exploiting Visual-Spatial Relation for Stereo-Based 3D Detectors
Chen, Y., Huang, S., Liu, S., Yu, B., & Jia, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3197236 · Cited by 45 · cited 1×
“Modern 3D object detectors, either LiDAR-based [1– 3, 12, 13, 22–29], or Camera-based [30–37], or Multi-Modal [4–8 , 38– 45] 3D object detectors Different from dense detectors, point-based 3D detectors [11, 50–52] directly process point clouds via Point-Net [53, 54] and perform grouping or…”
BackgroundSelf-citationBoxeR: Box-Attention for 2D and 3D Transformers
Nguyen, D.-K., Ju, J., Booij, O., Oswald, M. R., & Snoek, C. G. M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00473 · Cited by 41 · cited 1×
“To enhance the efficiency of object query processing, we employ deformable attention [17] instead of computationally intensive modules such as cross attention [59] or box attention [70].”
MethodFully Sparse 3D Object DetectionPreprint
Fan, L., Wang, F., Wang, N., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.10035 · Cited by 40 · cited 1×
“FSD [56] instead further recognize instances directly on sparse representations obtained by SST [55] and Spar-seConv for long-range detection.”
BackgroundKoh, J., Lee, J., Lee, Y., Kim, J., & Choi, J. W. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25200 · Cited by 32
Research works printed but not linked: 4
- Roddick, Orthographic feature transform for monocular 3d object detection
- 10.1007/978-3-031-20077-9_1 · DOI · Paper page
- 10.1109/tpami.2023.3333838/mm1 · DOI
- Zhou, End-to-end multi-view fusion for 3d object detection in lidar point clouds, 923
Cited by
Sorted by citations
Robustness-Aware 3D Object Detection in Autonomous Driving: A Review and Outlook
Song, Z., Liu, L., Jia, F., Luo, Y., Jia, C., Zhang, G., Yang, L., & Wang, L. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3439557 · Cited by 219
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 105
“However, in the real world, the presence of low-quality data seriously affects the performance of multimodal learning, making it difficult for models to work stably in complex environments.”
AlignFusionNet: Efficient Cross-Modal Alignment and Fusion for 3D Semantic Occupancy Prediction
Xu, Z., Qi, L., Du, H.-E., Yang, J., & Chen, Z. (2025) · IEEE Access · doi:10.1109/access.2025.3589858 · Cited by 36
“Although this approach can predict the location and size of the target and generate a compact 3D bounding box [8], it often eliminates detailed features of the detected objects.”
A review of object tracking methods: From general field to autonomous vehicles
Cao, J., Zhang, H., Jin, L., Lv, J., Hou, G., & Zhang, C. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127635 · Cited by 36
SparseLIF: High-Performance Sparse LiDAR-Camera Fusion for 3D Object Detection
Zhang, H., Liang, L., Zeng, P., Song, X., & Wang, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72761-0_7 · Cited by 35
Metadata
- Authors
- Chen, Y., Yu, Z., Chen, Y., Lan, S., Anandkumar, A., Jia, J., & Alvarez, J. M.
- Affiliations
- Chinese University of Hong Kong (HK)
- Nvidia (United Kingdom) (GB)
- Caltech
- NVIDIA
- Venue
- 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2023-10-01
- Pages
- 8360-8371
- OpenAlex
- W4390872444
- Semantic Scholar
- ef80a308b548f582ac52419dffaa48f53db8a4f8
- Cited by
- 119
- References
- 91
- Citations by year
- 2026: 35 · 2025: 53 · 2024: 30 · 2023: 1
- FWCI
- 12.73
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Keywords
- 3D object detection, false negative reduction, LiDAR, autonomous driving, object tracking, multimodal detection, nuScenes, Waymo
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
