MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationships
Open accessAbstract
Monocular 3D object detection is an essential component in autonomous driving while challenging to solve, especially for those occluded samples which are only partially visible. Most detectors consider each 3D object as an independent training target, inevitably resulting in a lack of useful information for occluded samples. To this end, we propose a novel method to improve the monocular 3D object detection by considering the relationship of paired samples. This allows us to encode spatial constraints for partially-occluded objects from their adjacent neighbors. Specifically, the proposed detector computes uncertainty-aware predictions for object locations and 3D distances for the adjacent object pairs, which are subsequently jointly optimized by nonlinear least squares. Finally, the one-stage uncertainty-aware prediction structure and the post-optimization module are dedicatedly integrated for ensuring the run-time efficiency. Experiments demonstrate that our method yields the best performance on KITTI 3D detection benchmark, by outperforming state-of-the-art competitors by wide margins, especially for the hard samples.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Averaged over the three object classes, relative gains over the prior best method [65] are 103.7% at the 0.5m threshold and 35.5% at the 1.0m threshold.
“comparing to the previous best method [65], the relative improvements averaged across the 3 object classes are 103.7% and 35.5% for 0.5m and 1.0m thresholds, respectively.”
From Is Pseudo-Lidar needed for Monocular 3D Object detection? (2021) · cites this paper as [65]
On nuScenes, DD3D beats all other methods, improving mAP by 17% over the prior best published result (41.8% vs 35.8% mAP) and by 13% over the best unpublished result.
“DD3D outperforms all other methods with a 17% improvement in mAP compared to the previously best published method [65] (41.8% vs. 35.8%mAP) as well as a 13% improvement compared to the best (unpublished) method.”
From Is Pseudo-Lidar needed for Monocular 3D Object detection? (2021) · cites this paper as [65]
Cites
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · see citing works
FCOS: Fully Convolutional One-Stage Object Detection
Tian, Z., Shen, C., Chen, H., & He, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00972 · Cited by 7,013 · see citing works
VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
Zhou, Y., & Tuzel, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00472 · Cited by 4,813 · see citing works
Multi-view 3D Object Detection Network for Autonomous Driving
Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.691 · Cited by 3,450 · see citing works
Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
Eigen, D., Puhrsch, C., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1406.2283 · Cited by 3,442 · see citing works
PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud
Shi, S., Wang, X., & Li, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00086 · Cited by 3,093 · see citing works
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
Kendall, A., & Gal, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.04977 · Cited by 3,088 · see citing works
Frustum PointNets for 3D Object Detection from RGB-D Data
Qi, C. R., Liu, W., Wu, C., Su, H., & Guibas, L. J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00102 · Cited by 2,655 · see citing works
G2o: A general framework for graph optimization
Kummerle, R., Grisetti, G., Strasdat, H., Konolige, K., & Burgard, W. (2011) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2011.5979949 · Cited by 1,983 · see citing works
Yu, F., Wang, D., Shelhamer, E., & Darrell, T. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00255 · Cited by 1,592 · see citing works
Wang, Y., Chao, W.-L., Garg, D., Hariharan, B., Campbell, M., & Weinberger, K. Q. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00864 · Cited by 1,154 · see citing works
3D Bounding Box Estimation Using Deep Learning and Geometry
Mousavian, A., Anguelov, D., Flynn, J., & Kosecka, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.597 · Cited by 1,129 · see citing works
Monocular 3D Object Detection for Autonomous Driving
Chen, X., Kundu, K., Zhang, Z., Ma, H., Fidler, S., & Urtasun, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.236 · Cited by 1,019 · see citing works
Deep Continuous Fusion for Multi-sensor 3D Object Detection
Liang, M., Yang, B., Wang, S., & Urtasun, R. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_39 · Cited by 993 · see citing works
Exploring Visual Relationship for Image Captioning
Yao, T., Pan, Y., Li, Y., & Mei, T. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_42 · Cited by 962 · see citing works
(2015) · Cited by 656 · see citing works
Stereo R-CNN Based 3D Object Detection for Autonomous Driving
Li, P., Chen, X., & Shen, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00783 · Cited by 597 · see citing works
M3D-RPN: Monocular 3D Region Proposal Network for Object Detection
Brazil, G., & Liu, X. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00938 · Cited by 591 · see citing works
Disentangling Monocular 3D Object Detection
Simonelli, A., Bulo, S. R., Porzi, L., Lopez-Antequera, M., & Kontschieder, P. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00208 · Cited by 520 · see citing works
Detecting Visual Relationships with Deep Relational Networks
Dai, B., Zhang, Y., & Lin, D. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.352 · Cited by 519 · see citing works
Scene Graph Generation from Objects, Phrases and Region Captions
Li, Y., Ouyang, W., Zhou, B., Wang, K., & Wang, X. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.142 · Cited by 518 · see citing works
Choi, J., Chun, D., Kim, H., & Lee, H.-J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00059 · Cited by 505 · see citing works
SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimation
Liu, Z., Wu, Z., & Tóth, R. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00506 · Cited by 457 · see citing works
Chabot, F., Chaouch, M., Rabarisoa, J., Teulière, C., & Château, T. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.198 · Cited by 442 · see citing works
Deep Continuous Fusion for Multi-Sensor 3D Object DetectionPreprint
Liang, M. X., Yang, B., Wang, S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.10992 · Cited by 428 · see citing works
3D Object Proposals Using Stereo Imagery for Accurate Object Class Detection
Chen, X., Kundu, K., Zhu, Y., Ma, H., Fidler, S., & Urtasun, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2706685 · Cited by 424 · see citing works
RTM3D: Real-Time Monocular 3D Detection from Object Keypoints for Autonomous Driving
Li, P., Zhao, H., Liu, P., & Cao, F. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_38 · Cited by 374 · see citing works
Multi-level Fusion Based 3D Object Detection from Monocular Images
Xu, B., & Chen, Z. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00249 · Cited by 358 · see citing works
Accurate Monocular 3D Object Detection via Color-Embedded 3D Reconstruction for Autonomous Driving
Ma, X., Wang, Z., Li, H., Zhang, P., Ouyang, W., & Fan, X. N. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00695 · Cited by 333 · see citing works
3D-RCNN: Instance-Level 3D Object Reconstruction via Render-and-Compare
Kundu, A., Li, Y., & Rehg, J. M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00375 · Cited by 331 · see citing works
Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstruction
Ku, J., Pon, A., & Waslander, S. L. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01214 · Cited by 306 · see citing works
ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shape
Manhardt, F., Kehl, W., & Gaidon, A. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00217 · Cited by 305 · see citing works
Subcategory-Aware Convolutional Neural Networks for Object Proposals and Detection
Yu, X., Choi, W., Lin, Y., & Savarese, S. (2017) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2017.108 · Cited by 303 · see citing works
MonoGRNet: A Geometric Reasoning Network for Monocular 3D Object Localization
Qin, Z., Wang, J., & Lu, Y. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018851 · Cited by 290 · see citing works
Zhou, X., Wang, D., & Krähenbühl, P. (2019) · arXiv · doi:10.48550/arxiv.1904.07850 · Cited by 288 · see citing works
Feng, D., Rosenbaum, L., & Dietmayer, K. (2018) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569814 · Cited by 260 · see citing works
RoarNet: A Robust 3D Object Detection based on RegiOn Approximation Refinement
Shin, K., Kwon, Y. P., & Tomizuka, M. (2019) · 2019 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2019.8813895 · Cited by 221 · see citing works
Orthographic feature transform for monocular 3d object detection
Roddick, T., Kendall, A., & Cipolla, R. (2018) · arXiv · doi:10.48550/arxiv.1811.08188 · Cited by 211 · see citing works
Triangulation Learning Network: From Monocular to Stereo 3D Object Detection
Qin, Z., Wang, J., & Lu, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00780 · Cited by 134 · see citing works
MonoLoco: Monocular 3D Pedestrian Localization and Uncertainty Estimation
Bertoni, L., Kreiss, S., & Alahi, A. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00696 · Cited by 112 · see citing works
Reconstructing vehicles from a single image: Shape priors for road scene understanding
Murthy, J. K., Krishna, G. V. S., Chhaya, F., et al. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989089 · Cited by 83 · see citing works
Pham, C. C., & Jeon, J. W. (2017) · Signal Processing Image Communication · doi:10.1016/j.image.2017.02.007 · Cited by 82 · see citing works
Monocular 3D Object Detection via Geometric Reasoning on Keypoints
Barabanau, I., Artemov, A., Burnaev, E. V., & Murashkin, V. (2020) · International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) · doi:10.5220/0009102506520659 · Cited by 73 · see citing works
RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous DrivingPreprint
Li, P., Zhao, H., Liu, P., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.03343 · Cited by 49 · see citing works
Geometry and uncertainty in deep learning for computer vision
Kendall, A. (2019) · Apollo (University of Cambridge) · doi:10.17863/cam.35260 · Cited by 47 · see citing works
Capturing Object Detection Uncertainty in Multi-Layer Grid Maps
Wirges, S., Reith‐Braun, M., Lauer, M., et al. (2019) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2019.8814073 · Cited by 37 · see citing works
Monocular 3D Object Detection Leveraging Accurate Proposals and Shape ReconstructionPreprint
Ku, J., Pon, A., & Waslander, S. L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01690 · Cited by 20 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection
Wang, T., Zhu, X., Pang, J., & Lin, D. (2021) · arXiv · doi:10.48550/arxiv.2104.10956 · Cited by 683 · see citing works
Categorical Depth Distribution Network for Monocular 3D Object Detection
Reading, C., Harakeh, A., Chae, J., & Waslander, S. L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00845 · Cited by 595 · see citing works
Is Pseudo-Lidar needed for Monocular 3D Object detection?
Park, D., Ambrus, R., Guizilini, V., Li, J., & Gaidon, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00313 · Cited by 334 · see citing works
Geometry-based Distance Decomposition for Monocular 3D Object Detection
Shi, X., Ye, Q., Chen, X., Chen, C., Chen, Z., & Kim, T.-K. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01489 · Cited by 158 · see citing works
3D Object Detection from Images for Autonomous Driving: A Survey
Ma, X., Ouyang, W., Simonelli, A., & Ricci, E. (2022) · arXiv · doi:10.48550/arxiv.2202.02980 · Cited by 147 · see citing works
Depth-conditioned Dynamic Message Propagation for Monocular 3D Object Detection
Wang, L., Du, L., Ye, X., Fu, Y., Guo, G., Xue, X., Feng, J., & Zhang, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00052 · Cited by 146 · see citing works
RCBEVDet: Radar-Camera Fusion in Bird's Eye View for 3D Object Detection
Lin, Z., Liu, Z., Xia, Z., Wang, X., Wang, Y., Qi, S., Dong, Y., Dong, N., Zhang, L., & Zhu, C. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01414 · Cited by 153 · see citing works
M3DSSD: Monocular 3D Single Stage Object Detector
Luo, S., Dai, H., Shao, L., & Ding, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00608 · Cited by 93 · see citing works
- Authors
- Chen, Y., Tai, L., Sun, K., & Li, M.
- Affiliations
- Alibaba Group (China) (CN)
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 12090-12099
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3034479628
- Cited by
- 316
- References
- 47
- Citations by year
- 2026: 23 · 2025: 55 · 2024: 68 · 2023: 55 · 2022: 57 · 2021: 55 · 2020: 3
- FWCI
- 12.97
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- monocular 3D object detection, occluded objects, uncertainty-aware prediction, nonlinear least squares, one-stage detectors, autonomous driving, KITTI dataset, 3D bounding box regression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
