M3D-RPN: Monocular 3D Region Proposal Network for Object Detection
Open accessAbstract
Understanding the world in 3D is a critical component of urban autonomous driving. Generally, the combination of expensive LiDAR sensors and stereo RGB imaging has been paramount for successful 3D object detection algorithms, whereas monocular image-only methods experience drastically reduced performance. We propose to reduce the gap by reformulating the monocular 3D detection problem as a standalone 3D region proposal network. We leverage the geometric relationship of 2D and 3D perspectives, allowing 3D boxes to utilize well-known and powerful convolutional features generated in the image-space. To help address the strenuous 3D parameter estimations, we further design depth-aware convolutional layers which enable location specific feature development and in consequence improved 3D scene understanding. Compared to prior work in monocular 3D detection, our method consists of only the proposed 3D region proposal network rather than relying on external networks, data, or multiple stages. M3D-RPN is able to significantly improve the performance of both monocular 3D Object Detection and Bird's Eye View tasks within the KITTI urban autonomous driving dataset, while efficiently using a shared multi-class model.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Incorporating uncertainty in estimating the 3D box or its depth greatly improves detection accuracy.
“Incorporating uncertainty in estimating the 3D box (or its depth) has also been shown to greatly improve detection accuracy [59,9,58].”
From Is Pseudo-Lidar needed for Monocular 3D Object detection? (2021) · cites this paper as [59]
A disentangled loss for 3D box regression helps stabilize training.
“In [59], the authors proposed a disentangled loss for 3D box regression that helps stabilize training.”
From Is Pseudo-Lidar needed for Monocular 3D Object detection? (2021) · cites this paper as [59]
Pseudo-lidar suffers a larger validation-to-test accuracy drop (30.1% to 18.6% BEV AP) than other methods.
“While the loss in accuracy between KITTI-3D val and the test sets is consistent with other methods [59,4,3], PL suffers particularly more (30.1% → 18.6% BEV AP), when compared to other”
From Is Pseudo-Lidar needed for Monocular 3D Object detection? (2021) · cites this paper as [59]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · see citing works
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · see citing works
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works
(2016) · Cited by 31,590 · see citing works
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · see citing works
(2017) · Cited by 11,625 · see citing works
Cascade R-CNN: Delving Into High Quality Object Detection
Cai, Z., & Vasconcelos, N. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00644 · Cited by 7,370 · see citing works
Multi-view 3D Object Detection Network for Autonomous Driving
Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.691 · Cited by 3,450 · see citing works
PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud
Shi, S., Wang, X., & Li, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00086 · Cited by 3,093 · see citing works
Frustum PointNets for 3D Object Detection from RGB-D Data
Qi, C. R., Liu, W., Wu, C., Su, H., & Guibas, L. J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00102 · Cited by 2,655 · see citing works
UnitBox: An Advanced Object Detection Network
Yu, J., Jiang, Y., Wang, Z., Cao, Z., & Huang, T. (2016) · Proceedings of the 24th ACM international conference on Multimedia · doi:10.1145/2964284.2967274 · Cited by 1,731 · see citing works
A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection
Cai, Z., Fan, Q., Feris, R. S., & Vasconcelos, N. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_22 · Cited by 1,605 · see citing works
PIXOR: Real-time 3D Object Detection from Point Clouds
Yang, B., Luo, W., & Urtasun, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00798 · Cited by 1,249 · see citing works
3D Bounding Box Estimation Using Deep Learning and Geometry
Mousavian, A., Anguelov, D., Flynn, J., & Kosecka, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.597 · Cited by 1,129 · see citing works
Monocular 3D Object Detection for Autonomous Driving
Chen, X., Kundu, K., Zhang, Z., Ma, H., Fidler, S., & Urtasun, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.236 · Cited by 1,019 · see citing works
Object Recognition with Gradient-Based Learning
LeCun, Y., Haffner, P., Bottou, L., & Bengio, Y. (1999) · Lecture Notes in Computer Science · doi:10.1007/3-540-46805-6_19 · Cited by 947 · see citing works
CNN-SLAM: Real-Time Dense Monocular SLAM with Learned Depth Prediction
Tateno, K., Tombari, F., Laina, I., & Navab, N. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.695 · Cited by 800 · see citing works
(2015) · Cited by 656 · see citing works
Repulsion Loss: Detecting Pedestrians in a Crowd
Wang, X., Xiao, T., Jiang, Y., Shao, S., Sun, J., & Shen, C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00811 · Cited by 620 · see citing works
Yang, F., Choi, W., & Lin, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.234 · Cited by 608 · see citing works
Deep Continuous Fusion for Multi-Sensor 3D Object DetectionPreprint
Liang, M. X., Yang, B., Wang, S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.10992 · Cited by 428 · see citing works
Multi-level Fusion Based 3D Object Detection from Monocular Images
Xu, B., & Chen, Z. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00249 · Cited by 358 · see citing works
DetNet: Design Backbone for Object Detection
Li, Z., Peng, C., Yu, G., Zhang, X., Deng, Y. S., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01240-3_21 · Cited by 338 · see citing works
Parallel Feature Pyramid Network for Object Detection
Kim, S.-W., Kook, H.-K., Sun, J.-Y., Kang, M.-C., & Ko, S.-J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01228-1_15 · Cited by 326 · see citing works
Accurate Single Stage Detector Using Recurrent Rolling Convolution
Ren, J., Chen, X., Liu, J., Sun, W., Pang, J., Yan, Q., Tai, Y.-W., & Xu, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.87 · Cited by 305 · see citing works
Subcategory-Aware Convolutional Neural Networks for Object Proposals and Detection
Yu, X., Choi, W., Lin, Y., & Savarese, S. (2017) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2017.108 · Cited by 303 · see citing works
Illuminating Pedestrians via Simultaneous Detection and Segmentation
Brazil, G., Yin, X. P., & Liu, X. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.530 · Cited by 290 · see citing works
Self-citationLearning Efficient Single-Stage Pedestrian Detectors by Asymptotic Localization Fitting
Liu, W., Liao, S., Hu, W., Liang, X., & Chen, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_38 · Cited by 265 · see citing works
Single-Shot Object Detection with Enriched Semantics
Zhang, Z., Qiao, S., Xie, C., Shen, W., Wang, B., & Yuille, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00609 · Cited by 236 · see citing works
Deep Feature Pyramid Reconfiguration for Object Detection
Kong, T., Sun, F., Huang, W., & Liu, H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01228-1_11 · Cited by 234 · see citing works
Bi-box Regression for Pedestrian Detection and Occlusion Estimation
Zhou, C., & Yuan, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_9 · Cited by 219 · see citing works
Womg, A., Shafiee, M. J., Li, F., et al. (2018) · Conference on Computer and Robot Vision (CRV) · doi:10.1109/crv.2018.00023 · Cited by 177 · see citing works
A General Pipeline for 3D Detection of Vehicles
Du, X., Ang, M. H., Karaman, S., & Rus, D. L. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8461232 · Cited by 161 · see citing works
Dynamic Zoom-in Network for Fast Object Detection in Large Images
Gao, M., Yu, R., Li, A., Morariu, V. I., & Davis, L. S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00724 · Cited by 148 · see citing works
Behl, A., Jafari, O. H., Mustikovela, S. K., Alhaija, H. A., Rother, C., & Geiger, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.281 · Cited by 145 · see citing works
Localization Recall Precision (LRP): A New Performance Metric for Object Detection
Öksüz, K., Çam, B. C., Akbaş, E., & Kalkan, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_31 · Cited by 136 · see citing works
LiDAR-Video Driving Dataset: Learning Driving Policies Effectively
Chen, Y., Wang, J., Li, J., Lu, C., Luo, Z., Xue, H., & Wang, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00615 · Cited by 131 · see citing works
Context Refinement for Object Detection
Chen, Z., Huang, S., & Tao, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_5 · Cited by 123 · see citing works
Pedestrian Detection With Autoregressive Network Phases
Brazil, G., & Liu, X. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00740 · Cited by 85 · see citing works
Self-citationSnapNet-R: Consistent 3D Multi-view Semantic Labeling for Robotics
Guerry, J., Boulch, A., Saux, B. L., Moras, J., Plyer, A., & Filliat, D. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.85 · Cited by 82 · see citing works
Gradient Feature Selection for Online Boosting
Liu, X., & Yu, T. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4408912 · Cited by 80 · see citing works
Self-citationIlluminating Pedestrians via Simultaneous Detection & SegmentationPreprint
Brazil, G., Yin, X. P., & Liu, X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.08564 · Cited by 61 · see citing works
Self-citationWong, A., Shafiee, M. J., Li, F., et al. (2018) · arXiv · doi:10.48550/arxiv.1802.06488 · Cited by 41 · see citing works
SurfConv: Bridging 3D and 2D Convolution for RGBD Images
Chu, H., Ma, W.-C., Kundu, K., Urtasun, R., & Fidler, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00317 · Cited by 30 · see citing works
Monocular Video-Based Trailer Coupler Detection Using Multiplexer Convolutional Neural Network
Atoum, Y., Roth, J., Bliss, M., Zhang, W., & Liu, X. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.584 · Cited by 12 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 2
- Ross Girshick. Fast R-CNN. In ICCV. IEEE, 2015. 4, 7
- Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kil- ian Q Weinberger. Densely connected convolutional net- works. In CVPR. IEEE, 2017. 5
Cited by
Sorted by citations
FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection
Wang, T., Zhu, X., Pang, J., & Lin, D. (2021) · arXiv · doi:10.48550/arxiv.2104.10956 · Cited by 683 · see citing works
Categorical Depth Distribution Network for Monocular 3D Object Detection
Reading, C., Harakeh, A., Chae, J., & Waslander, S. L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00845 · Cited by 595 · see citing works
Is Pseudo-Lidar needed for Monocular 3D Object detection?
Park, D., Ambrus, R., Guizilini, V., Li, J., & Gaidon, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00313 · Cited by 334 · see citing works
MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationships
Chen, Y., Tai, L., Sun, K., & Li, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01211 · Cited by 316 · see citing works
Geometry-based Distance Decomposition for Monocular 3D Object Detection
Shi, X., Ye, Q., Chen, X., Chen, C., Chen, Z., & Kim, T.-K. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01489 · Cited by 158 · see citing works
3D Object Detection from Images for Autonomous Driving: A Survey
Ma, X., Ouyang, W., Simonelli, A., & Ricci, E. (2022) · arXiv · doi:10.48550/arxiv.2202.02980 · Cited by 147 · see citing works
Depth-conditioned Dynamic Message Propagation for Monocular 3D Object Detection
Wang, L., Du, L., Ye, X., Fu, Y., Guo, G., Xue, X., Feng, J., & Zhang, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00052 · Cited by 146 · see citing works
M3DSSD: Monocular 3D Single Stage Object Detector
Luo, S., Dai, H., Shao, L., & Ding, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00608 · Cited by 93 · see citing works
ViewRefer: Grasp the Multi-view Knowledge for 3D Visual Grounding
Guo, Z., Tang, Y., Zhang, R., Wang, D., Wang, Z., Zhao, B., & Li, X. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01410 · Cited by 44 · see citing works
Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Yu, Q., & Dai, J. (2022) · arXiv · doi:10.48550/arxiv.2203.17270 · Cited by 1,419 · see citing works
“For the 3D object detection task, early methods are similar to 2D detection methods [1, 28, 48, 39, 52], which usually predict the 3D bounding boxes based on 2D bounding boxes.”
Method
- Authors
- Brazil, G., & Liu, X.
- Affiliations
- Michigan State University (US)
- ORCID
- Liu, Xiaoming
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 9286-9295
- OpenAlex
- W2960641820
- Cited by
- 591
- References
- 47
- Citations by year
- 2026: 41 · 2025: 83 · 2024: 114 · 2023: 108 · 2022: 113 · 2021: 94 · 2020: 35 · 2019: 3
- FWCI
- 19.97
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Army Research Office (W911NF, W911NF-18-1-0330)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- monocular 3D object detection, bird's-eye view, autonomous driving, urban driving, KITTI dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
