CoChat
Sign Up
PreprintCited by 12Advanced Neural Network ApplicationsAutonomous Vehicle Technology and SafetyAdvanced Vision and Imaging

The DOI matches its DataCite record; 4 of the 4 references it cites were found in a registry or an index.

Accurate Monocular Object Detection via Color-Embedded 3D Reconstruction for Autonomous Driving

Open access
Ma, Xinzhu, Wang, Zhihui, Li, Haojie, Zhang, Pengbo, Fan, Xin, Ouyang, Wanli·2019·arXiv
DOI 10.48550/arxiv.1903.11444arXiv 1903.11444
Open this paper in CoChat — free accountOpen PDF

Abstract

In this paper, we propose a monocular 3D object detection framework in the domain of autonomous driving. Unlike previous image-based methods which focus on RGB feature extracted from 2D images, our method solves this problem in the reconstructed 3D space in order to exploit 3D contexts explicitly. To this end, we first leverage a stand-alone module to transform the input data from 2D image plane to 3D point clouds space for a better input representation, then we perform the 3D detection using PointNet backbone net to obtain objects 3D locations, dimensions and orientations. To enhance the discriminative capability of point clouds, we propose a multi-modal feature fusion module to embed the complementary RGB cue into the generated point clouds representation. We argue that it is more effective to infer the 3D bounding boxes from the generated 3D scene space (i.e., X,Y, Z space) compared to the image plane (i.e., R,G,B image plane). Evaluation on the challenging KITTI dataset shows that our approach boosts the performance of state-of-the-art monocular approach by a large margin.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1903.11444

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Depth estimation is necessary for 3D perception from cameras, which is possible with a single camera though stereo or multi-view systems are more robust.

    “Depth estimation is therefore necessary, which is in fact possible with a single camera [158] though stereo or multi-view systems are more robust [159].”

    From A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2020) · cites this paper as [159]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 39

  1. [1][1] Florian Chabot, Mohamed Chaouch, Jaonary Rabarisoa, C´ eline Teuli` ere, and Thierry Chateau. Deep manta: A coarse-to-fine many-task network for joint 2d and 3d vehicle analysis from monocular image. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 2040-2049, 2017. 1, 2 -
  2. [2][2] Jia-Ren Chang and Yong-Sheng Chen. Pyramid stereo matching network. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 5410-5418, 2018. 7 -
  3. [3][3] Xiaozhi Chen, Kaustav Kundu, Ziyu Zhang, Huimin Ma, Sanja Fidler, and Raquel Urtasun. Monocular 3d object detection for autonomous driving. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 2147-2156, 2016. 1, 2, 5, 6 -
  4. [4][4] Xiaozhi Chen, Kaustav Kundu, Yukun Zhu, Andrew G Berneshawi, Huimin Ma, Sanja Fidler, and Raquel Urtasun. 3d object proposals for accurate object class detection. In Advances in Neural Information Processing Systems , pages 424-432, 2015. 1, 2, 5, 6, 11 -
  5. [5][5] Xiaozhi Chen, Huimin Ma, Ji Wan, Bo Li, and Tian Xia. Multi-view 3d object detection network for autonomous driving. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , volume 1, page 3, 2017. 1, 3, 5 -
  6. [7][7] Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2012. 2, 5 -
  7. [8][8] Ross Girshick. Fast r-cnn. In The IEEE International Conference on Computer Vision (ICCV) , pages 1440-1448, 2015. 1, 2 -
  8. [9][9] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 580-587, 2014. 1 -
  9. [10][10] Cl´ ement Godard, Oisin Mac Aodha, and Gabriel J Brostow. Unsupervised monocular depth estimation with leftright consistency. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , volume 2, page 7, 2017. 7 -
  10. [11][11] Kaiming He, Georgia Gkioxari, Piotr Doll´ ar, and Ross Girshick. Mask r-cnn. In The IEEE International Conference on Computer Vision (ICCV) , pages 2980-2988. IEEE, 2017. 1 -
  11. [13][13] Qiangui Huang, Weiyue Wang, and Ulrich Neumann. Recurrent slice networks for 3d segmentation of point clouds. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 2626-2635, 2018. 3, 7 -
  12. [14][14] Bo Li. 3d fully convolutional network for vehicle detection in point cloud. In IEEE International Conference on Intelligent Robots and Systems (IROS) , pages 1513-1518. IEEE, 2017. 1 -
  13. [15][15] Buyu Li, Wanli Ouyang, Lu Sheng, Xingyu Zeng, and Xiaogang Wang. Gs3d: An efficient 3d object detection framework for autonomous driving. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2019. 1 -
  14. [16][16] Ming Liang, Bin Yang, Shenlong Wang, and Raquel Urtasun. Deep continuous fusion for multi-sensor 3d object detection. In Proceedings of the European Conference on Computer Vision (ECCV) , pages 641-656, 2018. 7 -
  15. [17][17] Tsung-Yi Lin, Priyal Goyal, Ross Girshick, Kaiming He, and Piotr Doll´ ar. Focal loss for dense object detection. IEEE transactions on pattern analysis and machine intelligence , 2018. 1 -
  16. [18][18] Li Liu, Wanli Ouyang, Xiaogang Wang, Paul Fieguth, Jie Chen, Xinwang Liu, and Matti Pietik¨ ainen. Deep learning for generic object detection: A survey. arXiv preprint arXiv:1809.02165 , 2018. 1 - · arXiv
  17. [19][19] Wenjie Luo, Bin Yang, and Raquel Urtasun. Fast and furious: Real time end-to-end 3d detection, tracking and motion forecasting with a single convolutional net. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 3569-3577, 2018. 1 -
  18. [20][20] Fabian Manhardt, Wadim Kehl, and Adrien Gaidon. Roi10d: Monocular lifting of 2d detection to 6d pose and metric shape. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2019. 3, 5, 6, 7 -
  19. [21][21] Daniel Maturana and Sebastian Scherer. Voxnet: A 3d convolutional neural network for real-time object recognition. In IEEE International Conference on Intelligent Robots and Systems (IROS) , pages 922-928. IEEE, 2015. 1 -
  20. [22][22] Nikolaus Mayer, Eddy Ilg, Philip Hausser, Philipp Fischer, Daniel Cremers, Alexey Dosovitskiy, and Thomas Brox. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4040-4048, 2016. 7 -
  21. [23][23] Arsalan Mousavian, Dragomir Anguelov, John Flynn, and Jana Koˇ seck´ a. 3d bounding box estimation using deep learning and geometry. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 5632-5640. IEEE, 2017. 1, 2, 5, 6 -
  22. [24][24] Wanli Ouyang, Kun Wang, Xin Zhu, and Xiaogang Wang. Chained cascade network for object detection. In The IEEE International Conference on Computer Vision (ICCV) , Oct 2017. 1 -
  23. [25][25] Wanli Ouyang and Xiaogang Wang. Joint deep learning for pedestrian detection. In Proceedings of the IEEE International Conference on Computer Vision , pages 2056-2063, 2013. 2 -
  24. [26][26] Wanli Ouyang, Hui Zhou, Hongsheng Li, Quanquan Li, Junjie Yan, and Xiaogang Wang. Jointly learning deep features, deformable parts, occlusion and classification for pedestrian detection. IEEE transactions on pattern analysis and machine intelligence , 40(8):1874-1887, 2017. 2 -
  25. [27][27] Charles R. Qi, Wei Liu, Chenxia Wu, Hao Su, and Leonidas J. Guibas. Frustum pointnets for 3d object detection from rgb-d data. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2018. 1, 3, 4, 5, 6, 11 -
  26. [28][28] Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , July 2017. 3, 4, 7 -
  27. [29][29] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In Advances in Neural Information Processing Systems , pages 5099-5108, 2017. 3, 7 -
  28. [30][30] Zengyi Qin, Jinglu Wang, and Yan Lu. Monogrnet: A geometric reasoning network for monocular 3d object localization. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 33, pages 8851-8858, 2019. 5, 6 -
  29. [31][31] Jimmy Ren, Xiaohao Chen, Jianbo Liu, Wenxiu Sun, Jiahao Pang, Qiong Yan, Yu-Wing Tai, and Li Xu. Accurate single stage detector using recurrent rolling convolution. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , July 2017. 11 -
  30. [32][32] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems , pages 91-99, 2015. 1 -
  31. [33][33] Zhile Ren and Erik B Sudderth. Three-dimensional object detection and layout prediction using clouds of oriented gradients. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 1525-1533, 2016. 1 -
  32. [34][34] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 , 2014. 1 - · arXiv
  33. [35][35] Shuran Song and Jianxiong Xiao. Sliding shapes for 3d object detection in depth images. In European conference on computer vision , pages 634-651. Springer, 2014. 1 -
  34. [36][36] Shuyang Sun, Jiangmiao Pang, Jianping Shi, Shuai Yi, and Wanli Ouyang. Fishnet: A versatile backbone for image, region, and pixel level prediction. In Advances in Neural Information Processing Systems , pages 754-764, 2018. 1 -
  35. [38][38] Bin Xu and Zhenzhong Chen. Multi-level fusion based 3d object detection from monocular images. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2018. 1, 2, 5, 6, 11 -
  36. [40][40] Dan Xu, Elisa Ricci, Wanli Ouyang, Xiaogang Wang, and Nicu Sebe. Monocular depth estimation using multi-scale continuous crfs as sequential deep networks. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2018. 2, 3 -
  37. [41][41] Bin Yang, Wenjie Luo, and Raquel Urtasun. Pixor: Realtime 3d object detection from point clouds. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2018. 1 -
  38. [42][42] Xingyu Zeng, Wanli Ouyang, Junjie Yan, Hongsheng Li, Tong Xiao, Kun Wang, Yu Liu, Yucong Zhou, Bin Yang, Zhe Wang, et al. Crafting gbd-net for object detection. IEEE transactions on pattern analysis and machine intelligence , 40(9):2109-2123, 2017. 1 -
  39. [43][43] Yin Zhou and Oncel Tuzel. Voxelnet: End-to-end learning for point cloud based 3d object detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2018. 3, 11

Cited by

Sorted by citations

Metadata

Authors
Ma, X., Wang, Z., Li, H., Zhang, P., Fan, X., & Ouyang, W.
Affiliations
Dalian University of Technology (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2019
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1903.11444
arXiv
1903.11444
OpenAlex
W2924441208
Cited by
12
References
4
Citations by year
2024: 1 · 2023: 1 · 2022: 1 · 2021: 1 · 2020: 8
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Natural Science Foundation of China (61733002, 61572096, 61772108)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
Keywords
monocular 3D object detection, autonomous driving, point clouds, PointNet, multimodal fusion, 3D bounding box estimation, KITTI dataset

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review