The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Fusion of 3D LIDAR and Camera Data for Object Detection in Autonomous Vehicle Applications
Abstract
It is vital that autonomous vehicles acquire accurate and real-time information about objects in their vicinity, which fully guarantees the safety of the passengers and vehicle in various environments. Three-dimensional light detection and ranging (3D LIDAR) sensors can directly obtain the position and geometric structure of an object within its detection range, whereas the use of vision cameras is most suitable for object recognition. Accordingly, in this paper, we present a novel object detection and identification method that fuses the complementary information obtained by two types of sensors. First, we utilise 3D LIDAR data to generate accurate object-region proposals. Then, these candidates are mapped onto the image space from which regions of interest (ROI) of the proposals are selected and input to a convolutional neural network (CNN) for further object recognition. To precisely identify the sizes of all the objects, we combine the features of the last three layers of the CNN to extract multi-scale features from the ROIs. The evaluation results obtained on the KITTI dataset demonstrate that: (1) unlike sliding windows that produce thousands of candidate object-region proposals, 3D LIDAR provides an average of 86 real candidates per frame and the minimal recall rate is better than 95%, which greatly decreases the extraction time; (2) The average processing time for each frame of the proposed method is only 66.79 ms, which meets the real-time demand of autonomous vehicles; (3) The average identification accuracies of our method for cars and pedestrians at a moderate level of difficulty are 89.04% and 78.18%, respectively, which is better than those of most previous methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
(2016) · Cited by 31,590
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.690 · Cited by 19,892
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles, R. Q., Su, H., Kaichun, M., & Guibas, L. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.16 · Cited by 9,237
Selective Search for Object Recognition
Uijlings, J. R. R., van de Sande, K. E. A., Gevers, T., & Smeulders, A. W. M. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0620-5 · Cited by 5,980
VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
Zhou, Y., & Tuzel, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00472 · Cited by 4,813
Multi-view 3D Object Detection Network for Autonomous Driving
Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.691 · Cited by 3,450
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_23 · Cited by 3,034
Region-based convolutional networks for accurate object detection and segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2437384 · Cited by 2,912
Edge Boxes: Locating Object Proposals from Edges
Zitnick, C. L., & Dollár, P. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_26 · Cited by 2,656
Frustum PointNets for 3D Object Detection from RGB-D Data
Qi, C. R., Liu, W., Wu, C., Su, H., & Guibas, L. J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00102 · Cited by 2,655
Fast Feature Pyramids for Object Detection
Dollar, P., Appel, R., Belongie, S., & Perona, P. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2300479 · Cited by 1,955
Joint 3D Proposal Generation and Object Detection from View Aggregation
Ku, J., Mozifian, M., Lee, J., Harakeh, A., & Waslander, S. L. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594049 · Cited by 1,697
Junior: The Stanford entry in the Urban Challenge
Montemerlo, M., Becker, J., Bhat, S., Dahlkamp, H., Dolgov, D., Ettinger, S., Haehnel, D., Hilden, T., Hoffmann, G., Huhnke, B., et al. (2008) · Journal of Field Robotics · doi:10.1002/rob.20258 · Cited by 1,027
Deep Continuous Fusion for Multi-sensor 3D Object Detection
Liang, M., Yang, B., Wang, S., & Urtasun, R. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_39 · Cited by 974
A method for unconstrained convex minimization problem with the rate of convergence o(1/k^2)
Nesterov, Y. E. (1983) · Medical Entomology and Zoology · Cited by 865
Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation
Pont-Tuset, J., Arbelaez, P., T.Barron, J., Marques, F., & Malik, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2537320 · Cited by 581
Vehicle Detection from 3D Lidar Using Fully Convolutional Network
Li, B., Zhang, T., & Xia, T. (2016) · Robotics: Science and Systems XII · doi:10.15607/rss.2016.xii.042 · Cited by 577
Vote3Deep: Fast object detection in 3D point clouds using efficient convolutional neural networks
Engelcke, M., Rao, D., Wang, D. Z., Tong, C. H., & Posner, I. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989161 · Cited by 549
Wang, Z., & Jia, K. (2019) · 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968513 · Cited by 534
3D fully convolutional network for vehicle detection in point cloud
Li, B. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8205955 · Cited by 488
Model based vehicle detection and tracking for autonomous urban driving
Petrovskaya, A., & Thrun, S. (2009) · Autonomous Robots · doi:10.1007/s10514-009-9115-1 · Cited by 430
A multi-sensor fusion system for moving object detection and tracking in urban driving environments
Cho, H., Seo, Y.-W., Kumar, B. V. K. V., & Rajkumar, R. R. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6907100 · Cited by 428
3D Object Proposals Using Stereo Imagery for Accurate Object Class Detection
Chen, X., Kundu, K., Zhu, Y., Ma, H., Fidler, S., & Urtasun, R. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2706685 · Cited by 421
Voting for Voting in Online Point Cloud Object Detection
Zeng Wang, D., & Posner, I. (2015) · Robotics: Science and Systems XI · doi:10.15607/rss.2015.xi.035 · Cited by 337 · cited 2×
“[13] encoded the point clouds into 3D feature grid.”
BackgroundMethodComplexer-YOLO: Real-Time 3D Object Detection and Tracking on Semantic Point Clouds
Simon, M., Amende, K., Kraus, A., Honer, J., Samann, T., Kaulbersch, H., Milz, S., & Gross, H. M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00158 · Cited by 304
CMS-RCNN: Contextual Multi-Scale Region-Based CNN for Unconstrained Face Detection
Zhu, C., Zheng, Y., Luu, K., & Savvides, M. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-61657-5_3 · Cited by 250
Multimodal vehicle detection: fusing 3D-LIDAR and color camera data
Asvadi, A., Garrote, L., Premebida, C., Peixoto, P., & J. Nunes, U. (2018) · Pattern Recognition Letters · doi:10.1016/j.patrec.2017.09.038 · Cited by 225
Automatic apple recognition based on the fusion of color and 3D feature for robotic fruit picking
Tao, Y., & Zhou, J. (2017) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2017.09.019 · Cited by 171
Road-Segmentation-Based Curb Detection Method for Self-Driving via a 3D-LiDAR Sensor
Zhang, Y., Wang, J., Wang, X., & Dolan, J. M. (2018) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2018.2789462 · Cited by 169
Fusing LIDAR and images for pedestrian detection using convolutional neural networks
Schlosser, J., Chow, C. K., & Kira, Z. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487370 · Cited by 150
Efficient Online Segmentation for Sparse 3D Laser Scans
Bogoslavskyi, I., & Stachniss, C. (2017) · PFG – Journal of Photogrammetry Remote Sensing and Geoinformation Science · doi:10.1007/s41064-016-0003-y · Cited by 137
Robust curb detection and vehicle localization in urban environments
Hata, A., Osório, F. S., & Wolf, D. F. (2014) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2014.6856405 · Cited by 132
Object Detection and Classification by Decision-Level Fusion for Intelligent Vehicle Systems
Oh, S.-I., & Kang, H.-B. (2017) · Sensors · doi:10.3390/s17010207 · Cited by 120 · cited 2×
“features coming from data from different sensors [39-43] and (3) fusion on the decision level which combines the classified results from the data of each sensor [2, 44].”
BackgroundSensor Fusion Methodology for Vehicle Detection
García, F., Martín, D., de la Escalera, A., & Armingol, J. M. (2017) · IEEE Intelligent Transportation Systems Magazine · doi:10.1109/mits.2016.2620398 · Cited by 110
Leveraging Heteroscedastic Aleatoric Uncertainties for Robust Real-Time LiDAR 3D Object Detection
Feng, D., Rosenbaum, L., Timm, F., & Dietmayer, K. (2019) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2019.8814046 · Cited by 88
LMNet: Real-time Multiclass Object Detection on CPU Using 3D LiDAR
Minemura, K., Liau, H., Monrroy, A., & Kato, S. (2018) · 2018 3rd Asia-Pacific Conference on Intelligent Robot Systems (ACIRS) · doi:10.1109/acirs.2018.8467245 · Cited by 64
Segmentation of dense range information in complex urban scenes
Schoenberg, J. R., Nathan, A., & Campbell, M. (2010) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2010.5651749 · Cited by 52
Density Enhancement-Based Long-Range Pedestrian Detection Using 3-D Range Data
Li, K., Wang, X., Xu, Y., & Wang, J. (2015) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2015.2502325 · Cited by 52
Cited by
Domain Adaptive Object Detection for Autonomous Driving under Foggy Weather
Li, J., Xu, R., Ma, J., Zou, Q., Ma, J., & Yu, H. (2023) · 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv56688.2023.00068 · Cited by 160
“With the computer vision and artificial intelligence techniques, object detection plays a critical role in autonomous driving to understand the surrounding driving scenarios [54, 59].”
BackgroundSelf-citationData Fusion for Intelligent Crowd Monitoring and Management Systems: A Survey
Li, X., Yu, Q., Alzahrani, B., Barnawi, A., Alhindi, A., Alghazzawi, D., & Miao, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3060631 · Cited by 49
Object Detection Performance Evaluation for Autonomous Vehicles in Sandy Weather Environments
Aloufi, N., Alnori, A., Thayananthan, V., & Basuhail, A. (2023) · Applied Sciences · doi:10.3390/app131810249 · Cited by 27
Metadata
- Authors
- Zhao, X., Sun, P., Xu, Z., Min, H., & Yu, H.
- Venue
- IEEE Sensors Journal
- Published
- 2020-05-01
- Volume
- 20
- Issue
- 9
- Pages
- 4901-4913
- ISSN
- 1530-437X (ISSN-L)
- 1558-1748
- 2379-9153
- OpenAlex
- W3000171824
- Cited by
- 321
- References
- 47
- FWCI
- 17.16
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.