The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
A Survey on 3D Object Detection Methods for Autonomous Driving Applications
Abstract
An autonomous vehicle (AV) requires an accurate perception of its surrounding environment to operate reliably. The perception system of an AV, which normally employs machine learning (e.g., deep learning), transforms sensory data into semantic information that enables autonomous driving. Object detection is a fundamental function of this perception system, which has been tackled by several works, most of them using 2D detection methods. However, the 2D methods do not provide depth information, which is required for driving tasks, such as path planning, collision avoidance, and so on. Alternatively, the 3D object detection methods introduce a third dimension that reveals more detailed object's size and location information. Nonetheless, the detection accuracy of such methods needs to be improved. To the best of our knowledge, this is the first survey on 3D object detection methods used for autonomous driving applications. This paper presents an overview of 3D object detection methods and prevalently used sensors and datasets in AVs. It then discusses and categorizes the recent works based on sensors modalities into monocular, point cloud-based, and fusion methods. We then summarize the results of the surveyed works and identify the research gaps and future research directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.690 · Cited by 19,892
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · cited 1×
“This curve can be summarized by a single metric called Average Precision (AP) [68]:”
MethodAre we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles, R. Q., Su, H., Kaichun, M., & Guibas, L. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.16 · Cited by 9,237
Dynamic Graph CNN for Learning on Point Clouds
Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., & Solomon, J. M. (2019) · ACM Transactions on Graphics · doi:10.1145/3326362 · Cited by 7,199
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016) · arXiv · doi:10.48550/arxiv.1602.07360 · Cited by 6,201 · cited 1×
“For this reason, point cloud (PCL) projection methods first transform the 3D points into a 2D image via plane [47], cylindrical [48] or spherical [34] projections that can then be processed using standard 2D object detection models such as [49].”
MethodVoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
Zhou, Y., & Tuzel, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00472 · Cited by 4,813
3D ShapeNets: A deep representation for volumetric shapes
Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298801 · Cited by 4,550
CARLA: An open urban driving simulator
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017) · arXiv · doi:10.48550/arxiv.1711.03938 · Cited by 3,718 · cited 1×
“CARLA [35] is an open-source simulation tool for autonomous driving that allows flexible environmental setup and sensor configuration.”
MethodGeometric Deep Learning: Going beyond Euclidean data
Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., & Vandergheynst, P. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2693418 · Cited by 3,661
Multi-view Convolutional Neural Networks for 3D Shape Recognition
Su, H., Maji, S., Kalogerakis, E., & Learned-Miller, E. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.114 · Cited by 3,480
VoxNet: A 3D Convolutional Neural Network for real-time object recognition
Maturana, D., & Scherer, S. (2015) · 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353481 · Cited by 3,458
Multi-view 3D Object Detection Network for Autonomous Driving
Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.691 · Cited by 3,450
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435
Frustum PointNets for 3D Object Detection from RGB-D Data
Qi, C. R., Liu, W., Wu, C., Su, H., & Guibas, L. J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00102 · Cited by 2,655
Joint 3D Proposal Generation and Object Detection from View Aggregation
Ku, J., Mozifian, M., Lee, J., Harakeh, A., & Waslander, S. L. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594049 · Cited by 1,697
3D Bounding Box Estimation Using Deep Learning and Geometry
Mousavian, A., Anguelov, D., Flynn, J., & Kosecka, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.597 · Cited by 1,129 · cited 2×
“Since no depth information is available, most approaches first detect 2D candidates before predicting a 3D bounding box that contains the object using neural networks [39], geometrical constraints [40] or 3D model matching [41], [42].”
BackgroundVirtualWorlds as Proxy for Multi-object Tracking Analysis
Gaidon, A., Wang, Q., Cabon, Y., & Vig, E. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.470 · Cited by 1,079
Monocular 3D Object Detection for Autonomous Driving
Chen, X., Kundu, K., Zhang, Z., Ma, H., Fidler, S., & Urtasun, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.236 · Cited by 1,019
Wu, B., Wan, A., Yue, X., & Keutzer, K. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8462926 · Cited by 967
Sivaraman, S., & Trivedi, M. M. (2013) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2013.2266661 · Cited by 908
End-to-End Learning of Driving Models from Large-Scale Video Datasets
Xu, H., Gao, Y., Yu, F., & Darrell, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.376 · Cited by 884
Autonomous vehicle perception: The technology of today and tomorrow
Van Brummelen, J., O’Brien, M., Gruyer, D., & Najjaran, H. (2018) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2018.02.012 · Cited by 869
Kuutti, S., Fallah, S., Katsaros, K., Dianati, M., Mccullough, F., & Mouzakitis, A. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2018.2812300 · Cited by 801
Self-citationPerception, Planning, Control, and Coordination for Autonomous Vehicles
Pendleton, S., Andersen, H., Du, X., Shen, X., Meghjani, M., Eng, Y., Rus, D., & Ang, M. (2017) · Machines · doi:10.3390/machines5010006 · Cited by 657
(2015) · Cited by 656
Vehicle Detection from 3D Lidar Using Fully Convolutional Network
Li, B., Zhang, T., & Xia, T. (2016) · Robotics: Science and Systems XII · doi:10.15607/rss.2016.xii.042 · Cited by 577
Vote3Deep: Fast object detection in 3D point clouds using efficient convolutional neural networks
Engelcke, M., Rao, D., Wang, D. Z., Tong, C. H., & Posner, I. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989161 · Cited by 549
3D fully convolutional network for vehicle detection in point cloud
Li, B. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8205955 · Cited by 488
Chabot, F., Chaouch, M., Rabarisoa, J., Teuliere, C., & Chateau, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.198 · Cited by 446
Vehicle Detection Techniques for Collision Avoidance Systems: A Review
Mukhtar, A., Xia, L., & Tang, T. B. (2015) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2015.2409109 · Cited by 427
Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks
Zhang, Y., Pezeshki, M., Brakel, P., Zhang, S., Laurent, C., Bengio, Y., & Courville, A. (2016) · Interspeech 2016 · doi:10.21437/interspeech.2016-1446 · Cited by 352
Data-driven 3D Voxel Patterns for object category recognition
Xiang, Y., Choi, W., Lin, Y., & Savarese, S. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298800 · Cited by 319
Subcategory-Aware Convolutional Neural Networks for Object Proposals and Detection
Xiang, Y., Choi, W., Lin, Y., & Savarese, S. (2017) · 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2017.108 · Cited by 298
Feng, D., Rosenbaum, L., & Dietmayer, K. (2018) · arXiv · doi:10.48550/arxiv.1804.05132 · Cited by 260
Feng, D., Rosenbaum, L., & Dietmayer, K. (2018) · 2018 21st International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569814 · Cited by 259
Calibration between Color Camera and 3D LIDAR Instruments with a Polygonal Planar Board
Park, Y., Yun, S., Won, C., Cho, K., Um, K., & Sim, S. (2014) · Sensors · doi:10.3390/s140305333 · Cited by 231
Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications
Müller, M., Casser, V., Lahoud, J., Smith, N., & Ghanem, B. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1073-7 · Cited by 178
The Role of Machine Vision for Intelligent Vehicles
Ranft, B., & Stiller, C. (2016) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2016.2551553 · Cited by 168
A General Pipeline for 3D Detection of Vehicles
Du, X., Ang, M. H., Karaman, S., & Rus, D. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461232 · Cited by 160
LiDAR and Camera Calibration Using Motions Estimated by Sensor Fusion Odometry
Ishikawa, R., Oishi, T., & Ikeuchi, K. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593360 · Cited by 158
Brain-Inspired Cognitive Model With Attention for Self-Driving Cars
Chen, S., Zhang, S., Shang, J., Chen, B., & Zheng, N. (2017) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2017.2717451 · Cited by 156
Fusing LIDAR and images for pedestrian detection using convolutional neural networks
Schlosser, J., Chow, C. K., & Kira, Z. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487370 · Cited by 150
Complex-YOLO: Real-time 3D Object Detection on Point CloudsPreprint
Simón, M., Milz, S., Amende, K., & Groß, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.06199 · Cited by 141
DeepTLR: A single deep convolutional network for detection and classification of traffic lights
Weber, M., Wolf, P., & Zöllner, J. M. (2016) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2016.7535408 · Cited by 112
What could move? Finding cars, pedestrians and bicyclists in 3D laser data
Wang, D. Z., Posner, I., & Newman, P. M. (2012) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2012.6224734 · Cited by 104
Laser-based segment classification using a mixture of bag-of-words
Behley, J., Steinhage, V., & Cremers, A. B. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6696957 · Cited by 104
Pham, C. C., & Jeon, J. W. (2017) · Signal Processing Image Communication · doi:10.1016/j.image.2017.02.007 · Cited by 94
Yu, S.-L., Westfechtel, T., Hamada, R., Ohno, K., & Tadokoro, S. (2017) · IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR) · doi:10.1109/ssrr.2017.8088147 · Cited by 67
Garanderie, G. P. D. L., Atapour–Abarghouei, A., & Breckon, T. P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01261-8_48 · Cited by 65
Azim, A., & Aycard, O. (2014) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2014.6856558 · Cited by 40
BirdNet: A 3D Object Detection Framework from LiDAR Information
Beltran, J., Guindel, C., Moreno, F. M., Cruzado, D., Garcia, F., & De La Escalera, A. (2018) · 2018 21st International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569311 · Cited by 37
Towards End-to-End Speech Recognition
Palaz, D. (2016) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · doi:10.5075/epfl-thesis-7054 · Cited by 15
Pose Estimation Errors, the Ultimate Diagnosis
Redondo-Cabrera, C., López-Sastre, R. J., Yu, X., Tuytelaars, T., & Savarese, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46478-7_8 · Cited by 15
Cited by
Feng, D., Haase-Schutz, C., Rosenbaum, L., Hertlein, H., Glaser, C., Timm, F., Wiesbeck, W., & Dietmayer, K. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.2972974 · Cited by 1,425
A Survey of Deep Learning-Based Object Detection
Jiao, L., Zhang, F., Liu, F., Yang, S., Li, L., Feng, Z., & Qu, R. (2019) · IEEE Access · doi:10.1109/access.2019.2939201 · Cited by 1,323
Deep Learning-Based Vehicle Behavior Prediction for Autonomous Driving Applications: A Review
Mozaffari, S., Al-Jarrah, O. Y., Dianati, M., Jennings, P., & Mouzakitis, A. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3012034 · Cited by 671
Self-citationDeep Learning for Safe Autonomous Driving: Current Challenges and Future Directions
Muhammad, K., Ullah, A., Lloret, J., Ser, J. D., & de Albuquerque, V. H. C. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3032227 · Cited by 659
Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review
Li, Y., Ma, L., Zhong, Z., Liu, F., Chapman, M. A., Cao, D., & Li, J. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3015992 · Cited by 611
Metadata
- Authors
- Arnold, E., Al-Jarrah, O. Y., Dianati, M., Fallah, S., Oxtoby, D., & Mouzakitis, A.
- Venue
- IEEE Transactions on Intelligent Transportation Systems
- Published
- 2019-10
- Volume
- 20
- Issue
- 10
- Pages
- 3782-3795
- ISSN
- 1524-9050 (ISSN-L)
- 1558-0016
- OpenAlex
- W2911486422
- Cited by
- 713
- References
- 60
- FWCI
- 30.28
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.