Disentangling Monocular 3D Object Detection
Open accessAbstract
In this paper we propose an approach for monocular 3D object detection from a single RGB image, which leverages a novel disentangling transformation for 2D and 3D detection losses and a novel, self-supervised confidence score for 3D bounding boxes. Our proposed loss disentanglement has the twofold advantage of simplifying the training dynamics in the presence of losses with complex interactions of parameters, and sidestepping the issue of balancing independent regression terms. Our solution overcomes these issues by isolating the contribution made by groups of parameters to a given loss, without changing its nature. We further apply loss disentanglement to another novel, signed Intersection-over-Union criterion-driven loss for improving 2D detection results. Besides our methodological innovations, we critically review the AP metric used in KITTI3D, which emerged as the most important dataset for comparing 3D detection results. We identify and resolve a flaw in the 11-point interpolated AP metric, affecting all previously published detection results and particularly biases the results of monocular 3D detection. We provide extensive experimental evaluations and ablation studies and set a new state-of-the-art on the KITTI3D Car class.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The detection challenge enabled lidar based and camera based detection works that improved over the state-of-the-art at the time of initial release by 40% and 81%.
“The detection challenge enabled lidar based and camera based detection works such as [90, 70], that improved over the state-of-the-art at the time of initial release [51, 69] by 40% and 81% (Table 4).”
From nuScenes: A Multimodal Dataset for Autonomous Driving (2020) · cites this paper as [70]
Cites
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · see citing works
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · see citing works
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581 · see citing works
nuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152 · see citing works
PointPillars: Fast Encoders for Object Detection From Point Clouds
Lang, A. H., Vora, S., Caesar, H., Zhou, L., Yang, J., & Beijbom, O. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01298 · Cited by 4,673 · see citing works
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196 · see citing works
Deep Learning for Generic Object Detection: A Survey
Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., & Pietikäinen, M. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01247-4 · Cited by 2,965 · see citing works
Frustum PointNets for 3D Object Detection from RGB-D Data
Qi, C. R., Liu, W., Wu, C., Su, H., & Guibas, L. J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00102 · Cited by 2,655 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
nuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152 · see citing works
“PP: PointPillars [51], MDIS: MonoDIS [70].”
BackgroundMethodInfluentialPointPainting: Sequential Fusion for 3D Object Detection
Vora, S., Lang, A. H., Helou, B., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00466 · Cited by 1,236 · see citing works
FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection
Wang, T., Zhu, X., Pang, J., & Lin, D. (2021) · arXiv · doi:10.48550/arxiv.2104.10956 · Cited by 683 · see citing works
Categorical Depth Distribution Network for Monocular 3D Object Detection
Reading, C., Harakeh, A., Chae, J., & Waslander, S. L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00845 · Cited by 595 · see citing works
CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection
Nabati, R., & Qi, H. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00157 · Cited by 461 · see citing works
Is Pseudo-Lidar needed for Monocular 3D Object detection?
Park, D., Ambrus, R., Guizilini, V., Li, J., & Gaidon, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00313 · Cited by 334 · see citing works
MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationships
Chen, Y., Tai, L., Sun, K., & Li, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01211 · Cited by 316 · see citing works
HVNet: Hybrid Voxel Network for LiDAR Based 3D Object Detection
Ye, M., Xu, S., & Cao, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00170 · Cited by 242 · see citing works
Geometry-based Distance Decomposition for Monocular 3D Object Detection
Shi, X., Ye, Q., Chen, X., Chen, C., Chen, Z., & Kim, T.-K. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01489 · Cited by 158 · see citing works
CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perception
Kim, Y., Shin, J., Kim, S., Lee, I.-J., Choi, J. W., & Kum, D. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01615 · Cited by 155 · see citing works
3D Object Detection from Images for Autonomous Driving: A Survey
Ma, X., Ouyang, W., Simonelli, A., & Ricci, E. (2022) · arXiv · doi:10.48550/arxiv.2202.02980 · Cited by 147 · see citing works
Self-citationDepth-conditioned Dynamic Message Propagation for Monocular 3D Object Detection
Wang, L., Du, L., Ye, X., Fu, Y., Guo, G., Xue, X., Feng, J., & Zhang, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00052 · Cited by 146 · see citing works
M3DSSD: Monocular 3D Single Stage Object Detector
Luo, S., Dai, H., Shao, L., & Ding, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00608 · Cited by 93 · see citing works
Learning Multi-Object Tracking and Segmentation From Automatic Annotations
Porzi, L., Hofinger, M., Ruiz, I., Serrat, J., Bulo, S. R., & Kontschieder, P. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00688 · Cited by 82 · see citing works
Self-citationOcclusion-Aware Self-Supervised Monocular 6D Object Pose Estimation
Wang, G., Manhardt, F., Liu, X., Ji, X., & Tombari, F. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3136301 · Cited by 69 · see citing works
Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Yu, Q., & Dai, J. (2022) · arXiv · doi:10.48550/arxiv.2203.17270 · Cited by 1,419 · see citing works
“For the 3D object detection task, early methods are similar to 2D detection methods [1, 28, 48, 39, 52], which usually predict the 3D bounding boxes based on 2D bounding boxes.”
MethodMulti-Sem Fusion: Multimodal Semantic Fusion for 3-D Object Detection
Xu, S., Li, F., Song, Z., Fang, J., Wang, S., & Yang, Z.-X. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3387732 · Cited by 26 · see citing works
- Authors
- Simonelli, A., Bulo, S. R., Porzi, L., Lopez-Antequera, M., & Kontschieder, P.
- Affiliations
- University of Trento (IT)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 1991-1999
- OpenAlex
- W2947648124
- Cited by
- 520
- References
- 8
- Citations by year
- 2026: 20 · 2025: 66 · 2024: 87 · 2023: 93 · 2022: 92 · 2021: 92 · 2020: 66 · 2019: 4
- FWCI
- 20.70
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Iris (University of Trento) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- monocular 3D object detection, car detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
