Complex-YOLO: Real-time 3D Object Detection on Point Clouds
Open accessThe DOI matches its DataCite record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Lidar based 3D object detection is inevitable for autonomous driving, because it directly links to environmental understanding and therefore builds the base for prediction and motion planning. The capacity of inferencing highly sparse 3D data in real-time is an ill-posed problem for lots of other application areas besides automated vehicles, e.g. augmented reality, personal robotics or industrial automation. We introduce Complex-YOLO, a state of the art real-time 3D object detection network on point clouds only. In this work, we describe a network that expands YOLOv2, a fast 2D standard object detector for RGB images, by a specific complex regression strategy to estimate multi-class 3D boxes in Cartesian space. Thus, we propose a specific Euler-Region-Proposal Network (E-RPN) to estimate the pose of the object by adding an imaginary and a real fraction to the regression network. This ends up in a closed complex space and avoids singularities, which occur by single angle estimations. The E-RPN supports to generalize well during training. Our experiments on the KITTI benchmark suite show that we outperform current leading methods for 3D object detection specifically in terms of efficiency. We achieve state of the art results for cars, pedestrians and cyclists by being more than five times faster than the fastest competitor. Further, our model is capable of estimating all eight KITTI-classes, including Vans, Trucks or sitting pedestrians simultaneously with high accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
(2016) · Cited by 31,590
SSD: Single Shot MultiBox Detector
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2015) · arXiv · doi:10.48550/arxiv.1512.02325 · Cited by 22,443
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897 · cited 13×
“Complex-Yolo achieves accurate results by being five times faster than the most effective competitor on the KITTI benchmark [1].”
BackgroundMethodInfluentialPointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles, R. Q., Su, H., Kaichun, M., & Guibas, L. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.16 · Cited by 9,237
Dynamic Graph CNN for Learning on Point Clouds
Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., & Solomon, J. M. (2019) · ACM Transactions on Graphics · doi:10.1145/3326362 · Cited by 7,199
VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
Zhou, Y., & Tuzel, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00472 · Cited by 4,813
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196
Multi-view 3D Object Detection Network for Autonomous Driving
Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.691 · Cited by 3,450
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Qi, C. R., Su, H., Mo, K., & Guibas, L. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.00593 · Cited by 2,817
Frustum PointNets for 3D Object Detection from RGB-D Data
Qi, C. R., Liu, W., Wu, C., Su, H., & Guibas, L. J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00102 · Cited by 2,655
PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
Qi, C. R., Yi, L., Su, H., & Guibas, L. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02413 · Cited by 2,114
Joint 3D Proposal Generation and Object Detection from View Aggregation
Ku, J., Mozifian, M., Lee, J., Harakeh, A., & Waslander, S. L. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594049 · Cited by 1,697
A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection
Cai, Z., Fan, Q., Feris, R. S., & Vasconcelos, N. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_22 · Cited by 1,605 · cited 1×
“In general, object detection and classification based on deep learning is a well known task and widely established for 2D bounding box regression on images [12] [13] [14] [15] [16] [17] [18] [19] [20] [21].”
BackgroundMonocular 3D Object Detection for Autonomous Driving
Chen, X., Kundu, K., Zhang, Z., Ma, H., Fidler, S., & Urtasun, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.236 · Cited by 1,019
(2015) · Cited by 656
Vehicle Detection from 3D Lidar Using Fully Convolutional Network
Li, B., Zhang, T., & Xia, T. (2016) · Robotics: Science and Systems XII · doi:10.15607/rss.2016.xii.042 · Cited by 577 · cited 1×
“. We used batch normalization for regularization and a linear activation f(x) = xfor the last layer of our CNN, apart from that the leaky rectified linear activation: f(x) = ˆ x; x>0 0:1x;otherwise (8) Complex-YOLO: Real-time 3D Object Detection on Point Clouds 9 area 40m 80marea 40m oV sample detection spatial dist…”
MethodVote3Deep: Fast object detection in 3D point clouds using efficient convolutional neural networks
Engelcke, M., Rao, D., Wang, D. Z., Tong, C. H., & Posner, I. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989161 · Cited by 549
Rich feature hierarchies for accurate object detection and semantic segmentationPreprint
Girshick, R., Donahue, J., Darrell, T. J., et al. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1311.2524 · Cited by 514
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2016) · arXiv · doi:10.48550/arxiv.1612.08242 · Cited by 488
3D fully convolutional network for vehicle detection in point cloud
Li, B. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8205955 · Cited by 488
3D Object Proposals Using Stereo Imagery for Accurate Object Class Detection
Chen, X., Kundu, K., Zhu, Y., Ma, H., Fidler, S., & Urtasun, R. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2706685 · Cited by 421
Voting for Voting in Online Point Cloud Object Detection
Zeng Wang, D., & Posner, I. (2015) · Robotics: Science and Systems XI · doi:10.15607/rss.2015.xi.035 · Cited by 337 · cited 1×
“jei ^b ˚ is always located on the unit circle with jzj= 1 and jz^j= 1, we minimize the absolute value of the squared error to get a real loss: L Euler = coord XS2 i=0 XB j=0 1obj ij (e ib ˚ e ^ ˚)2 (6) = coord XS2 i=0 XB j=0 1obj ij (t im ^t im) 2+ (t re t^ re) (7) Where coordis a scaling factor to ensure stable conve…”
BackgroundData-driven 3D Voxel Patterns for object category recognition
Xiang, Y., Choi, W., Lin, Y., & Savarese, S. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298800 · Cited by 319
VoxelNet: End-to-End Learning for Point Cloud Based 3D Object DetectionPreprint
Zhou, Y., & Tuzel, O. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.06396 · Cited by 318
Accurate Single Stage Detector Using Recurrent Rolling Convolution
Ren, J., Chen, X., Liu, J., Sun, W., Pang, J., Yan, Q., Tai, Y.-W., & Xu, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.87 · Cited by 305 · cited 1×
“s Lidar based only. In general, object detection and classification based on deep learning is a well known task and widely established for 2D bounding box regression on images [12] [13] [14] [15] [16] [17] [18] [19] [20] [21]. Research focus was mainly a trade-off between accuracy and efficiency. In regard to automated…”
BackgroundGirshick, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.08083 · Cited by 276
Vehicle Detection from 3D Lidar Using Fully Convolutional NetworkPreprint
Li, B., Zhang, T., & Xia, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.07916 · Cited by 275
Joint 3D Proposal Generation and Object Detection from View AggregationPreprint
Ku, J., Mozifian, M., Lee, J., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.02294 · Cited by 134
3D ShapeNets for 2.5D Object Recognition and Next-Best-View PredictionPreprint
Wu, Z., Song, S., Khosla, A., Tang, X., & Xiao, J. (2014) · arXiv (Cornell University) · Cited by 97
3D ShapeNets: A Deep Representation for Volumetric ShapesPreprint
Wu, Z., Song, S., Khosla, A., et al. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.5670 · Cited by 91
Multi-View 3D Object Detection Network for Autonomous DrivingPreprint
Chen, X., Ma, H., Wan, J., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07759 · Cited by 65
Biternion Nets: Continuous Head Pose Regression from Discrete Training Labels
Beyer, L., Hermans, A., & Leibe, B. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24947-6_13 · Cited by 60
3D Object Proposals using Stereo Imagery for Accurate Object Class DetectionPreprint
Chen, X., Kundu, K., Zhu, Y., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.07711 · Cited by 49
Accurate Single Stage Detector Using Recurrent Rolling ConvolutionPreprint
Ren, J., Chen, X., Liu, J., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.05776 · Cited by 45
3D Fully Convolutional Network for Vehicle Detection in Point CloudPreprint
Li, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.08069 · Cited by 23
A Unified Multi-scale Deep Convolutional Neural Network for Fast Object DetectionPreprint
Cai, Z., Fan, Q., Feris, R., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.07155 · Cited by 4
Rich feature hierarchies for accurate object detection and semantic segmentation. arXiv e-printsPreprint
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2013)
Frustum PointNets for 3D Object Detection from RGB-D Data
Qi, C. R., Liu, W., Wu, C., et al. (2017) · arXiv · doi:10.48550/arxiv.1711.08488
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., et al. (2015) · arXiv · doi:10.48550/arxiv.1506.02640
Vote3Deep: Fast Object Detection in 3D Point Clouds Using Efficient Convolutional Neural Networks
Engelcke, M., Rao, D., Wang, D. Z., Tong, C. H., & Posner, I. (2016) · arXiv · doi:10.48550/arxiv.1609.06666
Cited by
PointPillars: Fast Encoders for Object Detection From Point Clouds
Lang, A. H., Vora, S., Caesar, H., Zhou, L., Yang, J., & Beijbom, O. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01298 · Cited by 4,673
SECOND: Sparsely Embedded Convolutional Detection
Yan, Y., Mao, Y., & Li, B. (2018) · Sensors · doi:10.3390/s18103337 · Cited by 3,352
PointPainting: Sequential Fusion for 3D Object Detection
Vora, S., Lang, A. H., Helou, B., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00466 · Cited by 1,236
“For methods using hand-coded features [29, 21], some extra work is required to modify the feature encoder.”
BackgroundMethodA Survey on 3D Object Detection Methods for Autonomous Driving Applications
Arnold, E., Al-Jarrah, O. Y., Dianati, M., Fallah, S., Oxtoby, D., & Mouzakitis, A. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2892405 · Cited by 713
Automotive LiDAR Technology: A Survey
Roriz, R., Cabral, J., & Gomes, T. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3086804 · Cited by 442
Metadata
- Authors
- Simon, M., Milz, S., Amende, K., & Gross, H.-M.
- Affiliations
- Technische Universität Ilmenau (DE)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1803.06199
- OpenAlex
- W2793066252
- Semantic Scholar
- 34c61cbf8185bd30dd51c450d7e2816a7b560e37
- Cited by
- 163
- References
- 28
- Citations by year
- 2026: 8 · 2025: 14 · 2024: 11 · 2023: 16 · 2022: 22 · 2021: 26 · 2020: 27 · 2019: 34 · 2018: 5
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Keywords
- 3D object detection, point clouds, LiDAR, YOLOv2, real-time detection, autonomous driving, pose estimation, KITTI dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.