The DOI matches its Crossref record; 137 of the 137 references it cites were found in a registry or an index.
Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review
Open accessAbstract
Autonomous vehicles were experiencing rapid development in the past few years. However, achieving full autonomy is not a trivial task, due to the nature of the complex and dynamic driving environment. Therefore, autonomous vehicles are equipped with a suite of different sensors to ensure robust, accurate environmental perception. In particular, the camera-LiDAR fusion is becoming an emerging research theme. However, so far there has been no critical review that focuses on deep-learning-based camera-LiDAR fusion methods. To bridge this gap and motivate future research, this article devotes to review recent deep-learning-based data fusion approaches that leverage both image and point cloud. This review gives a brief overview of deep learning on image and point cloud data processing. Followed by in-depth reviews of camera-LiDAR fusion methods in depth completion, object detection, semantic segmentation, tracking and online cross-sensor calibration, which are organized based on their respective fusion levels. Furthermore, we compare these methods on publicly available datasets. Finally, we identified gaps and over-looked challenges between current academic researches and real-world applications. Based on these observations, we provide our insights and point out promising research directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Are we ready for autonomous driving? The KITTI vision benchmark suite
Geiger, A., Lenz, P., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248074 · Cited by 14,897
Mean shift: a robust approach toward feature space analysis
Comaniciu, D., & Meer, P. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.1000236 · Cited by 11,305
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles, R. Q., Su, H., Kaichun, M., & Guibas, L. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.16 · Cited by 9,237
Learning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00907 · Cited by 6,249
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879
VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
Zhou, Y., & Tuzel, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00472 · Cited by 4,813
PointPillars: Fast Encoders for Object Detection From Point Clouds
Lang, A. H., Vora, S., Caesar, H., Zhou, L., Yang, J., & Beijbom, O. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01298 · Cited by 4,673
Advances In Deep Learning On Graphs (Gsp'18 Workshop)
Defferrard, M. (2016) · Figshare · doi:10.5281/zenodo.1286817 · Cited by 4,613
3D ShapeNets: A deep representation for volumetric shapes
Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298801 · Cited by 4,550
ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes
Dai, A., Chang, A. X., Savva, M., Halber, M., Funkhouser, T., & Niessner, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.261 · Cited by 4,368
Spectral Networks and Locally Connected Networks on Graphs
Bruna, J., Zaremba, W., Szlam, A., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6203 · Cited by 3,983
Multi-view Convolutional Neural Networks for 3D Shape Recognition
Su, H., Maji, S., Kalogerakis, E., & Learned-Miller, E. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.114 · Cited by 3,480
VoxNet: A 3D Convolutional Neural Network for real-time object recognition
Maturana, D., & Scherer, S. (2015) · 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353481 · Cited by 3,458
Multi-view 3D Object Detection Network for Autonomous Driving
Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.691 · Cited by 3,450
KPConv: Flexible and Deformable Convolution for Point Clouds
Thomas, H., Qi, C. R., Deschaud, J.-E., Marcotegui, B., Goulette, F., & Guibas, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00651 · Cited by 3,212
PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud
Shi, S., Wang, X., & Li, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00086 · Cited by 3,093
Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors
Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer, I., Wojna, Z., Song, Y., Guadarrama, S., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.351 · Cited by 2,743
Frustum PointNets for 3D Object Detection from RGB-D Data
Qi, C. R., Liu, W., Wu, C., Su, H., & Guibas, L. J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00102 · Cited by 2,655
RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds
Hu, Q., Yang, B., Xie, L., Rosa, S., Guo, Y., Wang, Z., Trigoni, N., & Markham, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01112 · Cited by 2,275
PointConv: Deep Convolutional Networks on 3D Point Clouds
Wu, W., Qi, Z., & Fuxin, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00985 · Cited by 1,850
ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
Paszke, A., Chaurasia, A., Kim, S., & Culurciello, E. (2016) · arXiv · doi:10.48550/arxiv.1606.02147 · Cited by 1,826
Joint 3D Proposal Generation and Object Detection from View Aggregation
Ku, J., Mozifian, M., Lee, J., Harakeh, A., & Waslander, S. L. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594049 · Cited by 1,697
OctNet: Learning Deep 3D Representations at High Resolutions
Riegler, G., Ulusoy, A. O., & Geiger, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.701 · Cited by 1,589
Learning Rich Features from RGB-D Images for Object Detection and Segmentation
Gupta, S., Girshick, R., Arbeláez, P., & Malik, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_23 · Cited by 1,528
Volumetric and Multi-view CNNs for Object Classification on 3D Data
Qi, C. R., Su, H., NieBner, M., Dai, A., Yan, M., & Guibas, L. J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.609 · Cited by 1,504
Feng, D., Haase-Schutz, C., Rosenbaum, L., Hertlein, H., Glaser, C., Timm, F., Wiesbeck, W., & Dietmayer, K. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.2972974 · Cited by 1,425
Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs
Simonovsky, M., & Komodakis, N. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.11 · Cited by 1,266
Deep Convolutional Networks on Graph-Structured Data
Henaff, M., Bruna, J., & LeCun, Y. (2015) · arXiv · doi:10.48550/arxiv.1506.05163 · Cited by 1,255
PointPainting: Sequential Fusion for 3D Object Detection
Vora, S., Lang, A. H., Helou, B., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00466 · Cited by 1,236
3D Bounding Box Estimation Using Deep Learning and Geometry
Mousavian, A., Anguelov, D., Flynn, J., & Kosecka, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.597 · Cited by 1,129
Escape from Cells: Deep Kd-Networks for the Recognition of 3D Point Cloud Models
Klokov, R., & Lempitsky, V. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.99 · Cited by 1,039
Deep Continuous Fusion for Multi-sensor 3D Object Detection
Liang, M., Yang, B., Wang, S., & Urtasun, R. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_39 · Cited by 974
Uhrig, J., Schneider, N., Schneider, L., Franke, U., Brox, T., & Geiger, A. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00012 · Cited by 924
PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation
Xu, D., Anguelov, D., & Jain, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00033 · Cited by 856
SPLATNet: Sparse Lattice Networks for Point Cloud Processing
Su, H., Jampani, V., Sun, D., Maji, S., Kalogerakis, E., Yang, M.-H., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00268 · Cited by 797
Multi-Task Multi-Sensor Fusion for 3D Object Detection
Liang, M., Yang, B., Chen, Y., Hu, R., & Urtasun, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00752 · Cited by 723
Mogelmose, A., Trivedi, M. M., & Moeslund, T. B. (2012) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2012.2209421 · Cited by 684
Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images
Song, S., & Xiao, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.94 · Cited by 681
Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image
Ma, F., & Karaman, S. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460184 · Cited by 650
Pointwise Convolutional Neural Networks
Hua, B.-S., Tran, M.-K., & Yeung, S.-K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00109 · Cited by 615
Mvx-net: Multimodal voxelnet for 3d object detection
Sindagi, V. A., Zhou, Y., & Tuzel, O. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794195 · Cited by 554
Point convolutional neural networks by extension operators
Atzmon, M., Maron, H., & Lipman, Y. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201301 · Cited by 542
Wang, Z., & Jia, K. (2019) · 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8968513 · Cited by 534
Cross Modal Distillation for Supervision Transfer
Gupta, S., Hoffman, J., & Malik, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.309 · Cited by 530 · cited 2×
“3) Feature-level Fusion: Early attempts [75] [76] of multimodal fusion are done in pixel-wise, where 3D geometry is converted to image format or appended as additional channels of an image.”
BackgroundMulti-Sensor Fusion in Automated Driving: A Survey
Wang, Z., Wu, Y., & Niu, Q. (2020) · IEEE Access · doi:10.1109/access.2019.2962554 · Cited by 526
Sequence-to-Sequence Prediction of Vehicle Trajectory via LSTM Encoder-Decoder Architecture
Park, S. H., Kim, B., Kang, C. M., Chung, C. C., & Choi, J. W. (2018) · 2018 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2018.8500658 · Cited by 504
3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans
Hou, J., Dai, A., & NieBner, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00455 · Cited by 483
Self-Supervised Sparse-to-Dense: Self-Supervised Depth Completion from LiDAR and Monocular Camera
Ma, F., Cavalheiro, G. V., & Karaman, S. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8793637 · Cited by 460
LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving
Meyer, G. P., Laddha, A., Kee, E., Vallespi-Gonzalez, C., & Wellington, C. K. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01296 · Cited by 374
LIDAR–camera fusion for road detection using fully convolutional neural networks
Caltagirone, L., Bellone, M., Svensson, L., & Wahde, M. (2019) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2018.11.002 · Cited by 370
Sliding Shapes for 3D Object Detection in Depth Images
Song, S., & Xiao, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10599-4_41 · Cited by 359
Deep Parametric Continuous Convolutional Neural Networks
Wang, S., Suo, S., Ma, W.-C., Pokrovsky, A., & Urtasun, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00274 · Cited by 359
Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network
Cheng, X., Wang, P., & Yang, R. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_7 · Cited by 357
3DMV: Joint 3D-Multi-view Prediction for 3D Semantic Scene Segmentation
Dai, A., & Nießner, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_28 · Cited by 328
PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation
Jiang, M., Wu, Y., Zhao, T., Zhao, Z., & Lu, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.00652 · Cited by 318
Taylor expansion of the accumulated rounding error
Linnainmaa, S. (1976) · BIT · doi:10.1007/bf01931367 · Cited by 315
Complexer-YOLO: Real-Time 3D Object Detection and Tracking on Semantic Point Clouds
Simon, M., Amende, K., Kraus, A., Honer, J., Samann, T., Kaulbersch, H., Milz, S., & Gross, H. M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00158 · Cited by 304
Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation
Jaritz, M., Charette, R. D., Wirbel, E., Perrotton, X., & Nashashibi, F. (2018) · 2018 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00017 · Cited by 299
Self-supervised model adaptation for multimodal semantic segmentation
Valada, A., Mohan, R., & Burgard, W. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01188-y · Cited by 293
Learning Guided Convolutional Network for Depth Completion
Tang, J., Tian, F.-P., Feng, W., Li, J., & Tan, P. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3040528 · Cited by 291
RegNet: Multimodal sensor registration using deep neural networks
Schneider, N., Piewak, F., Stiller, C., & Franke, U. (2017) · 2017 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2017.7995968 · Cited by 283
Sparse and Noisy LiDAR Completion with RGB Guidance and Uncertainty
Van Gansbeke, W., Neven, D., De Brabandere, B., & Van Gool, L. (2019) · 2019 16th International Conference on Machine Vision Applications (MVA) · doi:10.23919/mva.2019.8757939 · Cited by 264
A review of recent advances in lane detection and departure warning system
Narote, S. P., Bhujbal, P. N., Narote, A. S., & Dhane, D. M. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.08.014 · Cited by 260
Robust Multi-Modality Multi-Object Tracking
Zhang, W., Zhou, H., Sun, S., Wang, Z., Shi, J., & Loy, C. C. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00245 · Cited by 250
Cheng, X., Wang, P., Guan, C., & Yang, R. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6635 · Cited by 245
CalibNet: Geometrically Supervised Extrinsic Calibration using 3D Spatial Transformer Networks
Iyer, G., Ram, R. K., Murthy, J. K., & Krishna, K. M. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593693 · Cited by 244
Automatic Extrinsic Calibration of Vision and Lidar by Maximizing Mutual Information
Pandey, G., McBride, J. R., Savarese, S., & Eustice, R. M. (2015) · Journal of Field Robotics · doi:10.1002/rob.21542 · Cited by 243
Mobile Laser Scanned Point-Clouds for Road Object Detection and Extraction: A Review
Ma, L., Li, Y., Li, J., Wang, C., Wang, R., & Chapman, M. A. (2018) · Remote Sensing · doi:10.3390/rs10101531 · Cited by 230
Self-citationRoarNet: A Robust 3D Object Detection based on RegiOn Approximation Refinement
Shin, K., Kwon, Y. P., & Tomizuka, M. (2019) · 2019 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2019.8813895 · Cited by 221
Guo, J., Kurup, U., & Shah, M. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2926042 · Cited by 221
Confidence Propagation through CNNs for Guided Sparse Depth Regression
Eldesokey, A., Felsberg, M., & Khan, F. S. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2929170 · Cited by 217
Boscaini, D., Masci, J., Melzi, S., Bronstein, M. M., Castellani, U., & Vandergheynst, P. (2015) · Computer Graphics Forum · doi:10.1111/cgf.12693 · Cited by 207
PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things
Narita, G., Seno, T., Ishikawa, T., & Kaji, Y. (2019) · 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros40897.2019.8967890 · Cited by 206
Progressive LiDAR adaptation for road detection
Chen, Z., Zhang, J., & Tao, D. (2019) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2019.1911459 · Cited by 205
Aligning 3D models to RGB-D images of cluttered scenes
Gupta, S., Arbelaez, P., Girshick, R., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299105 · Cited by 199
Pedestrian detection combining RGB and dense LIDAR data
Premebida, C., Carreira, J., Batista, J., & Nunes, U. (2014) · 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2014.6943141 · Cited by 192
The Overlooked Elephant of Object Detection: Open Set
Dhamija, A. R., Gunther, M., Ventura, J., & Boult, T. E. (2020) · 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv45572.2020.9093355 · Cited by 178
Xing, Y., Lv, C., Chen, L., Wang, H., Wang, H., Cao, D., Velenis, E., & Wang, F.-Y. (2018) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2018.7511063 · Cited by 176
Self-citationAttribute Learning in Large-Scale Datasets
Russakovsky, O., & Fei-Fei, L. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35749-7_1 · Cited by 173
Multi-View PointNet for 3D Scene Understanding
Jaritz, M., Gu, J., & Su, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00494 · Cited by 170
Hybrid conditional random field based camera-LIDAR fusion for road detection
Xiao, L., Wang, R., Dai, B., Fang, Y., Liu, D., & Wu, T. (2018) · Information Sciences · doi:10.1016/j.ins.2017.04.048 · Cited by 166
Octree Guided CNN With Spherical Kernels for 3D Point Clouds
Lei, H., Akhtar, N., & Mian, A. S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00986 · Cited by 164
A General Pipeline for 3D Detection of Vehicles
Du, X., Ang, M. H., Karaman, S., & Rus, D. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8461232 · Cited by 160
Modeling Local Geometric Structure of 3D Point Clouds Using Geo-CNN
Lan, S., Yu, R., Yu, G., & Davis, L. S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00109 · Cited by 159
Sensor Fusion for Joint 3D Object Detection and Semantic Segmentation
Meyer, G. P., Charland, J., Hegde, D., Laddha, A., & Vallespi-Gonzalez, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00162 · Cited by 158
Fusing LIDAR and images for pedestrian detection using convolutional neural networks
Schlosser, J., Chow, C. K., & Kira, Z. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487370 · Cited by 150
Ma, L., Li, Y., Li, J., Tan, W., Yu, Y., & Chapman, M. A. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2961060 · Cited by 148
Self-citationEnd-to-end Learning of Multi-sensor 3D Tracking by Detection
Frossard, D., & Urtasun, R. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8462884 · Cited by 146
TGNet: Geometric Graph CNN on 3-D Point Cloud Segmentation
Li, Y., Ma, L., Zhong, Z., Cao, D., & Li, J. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2958517 · Cited by 143
Self-citationIPOD: Intensive Point-based Object Detector for Point Cloud
Yang, Z., Sun, Y., Liu, S., Shen, X., & Jia, J. (2018) · arXiv · doi:10.48550/arxiv.1812.05276 · Cited by 132
Finding multiple lanes in urban road networks with vision and lidar
Huang, A. S., Moore, D., Antone, M. E., Olson, E., & Teller, S. J. (2009) · Autonomous Robots · doi:10.1007/s10514-009-9113-3 · Cited by 131
Track to Reconstruct and Reconstruct to Track
Luiten, J., Fischer, T., & Leibe, B. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2969183 · Cited by 131
3D Object Detection Using Scale Invariant and Feature Reweighting Networks
Zhao, X., Liu, Z., Hu, R., & Huang, K. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019267 · Cited by 124
Deep Multi-Sensor Lane Detection
Bai, M., Máttyus, G., Homayounfar, N., Wang, S., Lakshmikanth, S. K., & Urtasun, R. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594388 · Cited by 119
Life and death decisions of autonomous vehicles
Bigman, Y., & Gray, K. J. (2020) · Nature · doi:10.1038/s41586-020-1987-4 · Cited by 119
CRF based road detection with multi-sensor fusion
Xiao, L., Dai, B., Liu, D., Hu, T., & Wu, T. (2015) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2015.7225685 · Cited by 118
Yu, Y., Li, J., Wen, C., Guan, H., Luo, H., & Wang, C. (2016) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2016.01.005 · Cited by 115
Deep Unsupervised Learning of 3D Point Clouds via Graph Topology Inference and Filtering
Chen, S., Duan, C., Yang, Y., Li, D., Feng, C., & Tian, D. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2957935 · Cited by 98
Self-citationTraffic sign detection in MLS acquired point clouds for geometric and image-based semantic inventory
Soilán, M., Riveiro, B., Martínez-Sánchez, J., & Arias, P. (2016) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2016.01.019 · Cited by 97
Traffic Light Recognition With High Dynamic Range Imaging and Deep Learning
Wang, J.-G., & Zhou, L. (2018) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2018.2849505 · Cited by 97
Noise-Aware Unsupervised Deep Lidar-Stereo Fusion
Cheng, X., Zhong, Y., Dai, Y., Ji, P., & Li, H. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00650 · Cited by 94
Cresceptron: a self-organizing neural network which grows adaptively
Weng, J., Ahuja, N., & Huang, T. S. (2003) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1992.287150 · Cited by 92
A Unified Point-Based Framework for 3D Segmentation
Chiang, H.-Y., Lin, Y.-L., Liu, Y.-C., & Hsu, W. H. (2019) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00026 · Cited by 84
High-Precision Depth Estimation Using Uncalibrated LiDAR and Stereo Fusion
Park, K., Kim, S., & Sohn, K. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2891788 · Cited by 83
Fusing Bird’s Eye View LIDAR Point Cloud and Front View Camera Image for 3D Object Detection
Wang, Z., Zhan, W., & Tomizuka, M. (2018) · 2018 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2018.8500387 · Cited by 81
Weakly Supervised Metric Learning for Traffic Sign Recognition in a LIDAR-Equipped Vehicle
Tan, M., Wang, B., Wu, Z., Wang, J., & Pan, G. (2016) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2015.2506182 · Cited by 81
Identifying Unknown Instances for Autonomous DrivingPreprint
Wong, K., Wang, S., Ren, M., Liang, M., & Urtasun, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.11296 · Cited by 81
Road terrain detection: Avoiding common obstacle detection assumptions using sensor fusion
Shinzato, P. Y., Wolf, D. F., & Stiller, C. (2014) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2014.6856454 · Cited by 78
Scanet: Spatial-channel Attention Network for 3D Object Detection
Lu, H., Chen, X., Zhang, G., Zhou, Q., Ma, Y., & Zhao, Y. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682746 · Cited by 78
Zeng, W., & Gevers, T. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11015-4_24 · Cited by 72
Groh, F., Wieschollek, P., & Lensch, H. P. A. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20887-5_7 · Cited by 70
Traffic Sign Occlusion Detection Using Mobile Laser Scanning Point Clouds
Huang, P., Cheng, M., Chen, Y., Luo, H., Wang, C., & Li, J. (2017) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2016.2639582 · Cited by 67
3D Bird’s-Eye-View Instance Segmentation
Elich, C., Engelmann, F., Kontogianni, T., & Leibe, B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33676-9_4 · Cited by 60
The computational solution of optimal control problems with time lag
Dreyfus, S. E. (1973) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1973.1100330 · Cited by 59
García, Á. A., Soilán, M., Álvarez-García, J. A., & Riveiro, B. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.07.042 · Cited by 57
Surrounding Vehicle Detection Using an FPGA Panoramic Camera and Deep CNNs
Chen, L. F., Zou, Q., Pan, Z., et al. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2949005 · Cited by 57
Self-citationEarly Fusion of Camera and Lidar for robust road detection based on U-Net FCN
Wulff, F., Schäufele, B., Sawade, O., Becker, D., Henke, B., & Radusch, I. (2018) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2018.8500549 · Cited by 49
Plug-and-Play: Improve Depth Prediction via Sparse Data Propagation
Wang, T.-H. J., Wang, F.-E., Lin, J.-T., Tsai, Y., Chiu, W.-C., & Sun, M. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794404 · Cited by 45
Visual odometry driven online calibration for monocular LiDAR-camera systems
Chien, H.-J., Klette, R., Schneider, N., & Franke, U. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7900068 · Cited by 44
Fast Road Detection by CNN-Based Camera–Lidar Fusion and Spherical Coordinate Transformation
Lee, J.-S., & Park, T.-H. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.2988302 · Cited by 43
LiStereo: Generate Dense Depth Maps from LIDAR and Stereo Imagery
Zhang, J., Ramanagopal, M. S., Vasudevan, R., & Johnson‐Roberson, M. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196628 · Cited by 42
Self-driving cars: A city perspective
Duarte, F. (2019) · Science Robotics · doi:10.1126/scirobotics.aav9843 · Cited by 26
Guan, H., Yu, Y., Peng, D., Zang, Y., Lu, J. Y., Li, A., & Li, J. (2019) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2019.2939354 · Cited by 25
From Here to There: Video Inbetweening Using Direct 3D ConvolutionsPreprint
Li, Y., Roblek, D., & Tagliasacchi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10240 · Cited by 24
Self-citationDeng, Z., & Zhou, L. (2017) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2017.2723902 · Cited by 23
A Novel Approach for Detecting Road Based on Two-Stream Fusion Fully Convolutional Network
Lv, X., Liu, Z., Xin, J., & Zheng, N. (2018) · 2018 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2018.8500551 · Cited by 21
Multi-Stage Residual Fusion Network for LIDAR-Camera Road Detection
Yu, D., Xiong, H., Xu, Q. Y., Wang, J., & Li, K. (2019) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2019.8813983 · Cited by 21
Guo, J., Kurup, U., & Shah, M. (2018) · arXiv · doi:10.48550/arxiv.1811.11277 · Cited by 14
Through-the-Lens Multi-camera Synchronisation and Frame-Drop Detection for 3D Reconstruction
İmre, E., Guillemaut, J., & Hilton, A. (2012) · doi:10.1109/3dimpvt.2012.31 · Cited by 6
Toward the Ghosting Phenomenon in a Stereo-Based Map With a Collaborative RGB-D Repair
Zhu, J., Fan, L., Tian, W., Chen, L. F., Cao, D., & Wang, F. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2019.2909734 · Cited by 5
Self-citation
Cited by
Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys
Chen, L., Li, Y., Huang, C., Li, B., Xing, Y., Tian, D., Li, L., Hu, Z., Na, X., Li, Z., et al. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2022.3223131 · Cited by 511
Self-citationMulti-Modal 3D Object Detection in Autonomous Driving: A Survey and Taxonomy
Wang, L., Zhang, X., Song, Z., Bi, J., Zhang, G., Wei, H., Tang, L., Yang, L., Li, J., Jia, C., et al. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3264658 · Cited by 269
Yao, S., Guan, R., Huang, X., Li, Z., Sha, X., Yue, Y., Lim, E. G., Seo, H., Man, K. L., Zhu, X., et al. (2024) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3307157 · Cited by 265
Disparity-Based Multiscale Fusion Network for Transportation Detection
Chen, J., Wang, Q., Peng, W., Xu, H., Li, X., & Xu, W. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3161977 · Cited by 166
A Comprehensive Survey on Deep Learning Multi-Modal Fusion: Methods, Technologies and Applications
Jiao, T., Guo, C., Feng, X., Chen, Y., & Song, J. (2024) · Computers, Materials & Continua · doi:10.32604/cmc.2024.053204 · Cited by 158
Metadata
- Authors
- Cui, Y., Chen, R., Chu, W., Chen, L., Tian, D., Li, Y., & Cao, D.
- Venue
- IEEE Transactions on Intelligent Transportation Systems
- Published
- 2022-02
- Volume
- 23
- Issue
- 2
- Pages
- 722-739
- ISSN
- 1524-9050 (ISSN-L)
- 1558-0016
- OpenAlex
- W3015248322
- Cited by
- 550
- References
- 137
- FWCI
- 28.61
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.