The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detection
Open accessAbstract
Object detection in camera images, using deep learning has been proven successfully in recent years. Rising detection rates and computationally efficient network structures are pushing this technique towards application in production vehicles. Nevertheless, the sensor quality of the camera is limited in severe weather conditions and through increased sensor noise in sparsely lit areas and at night. Our approach enhances current 2D object detection networks by fusing camera data and projected sparse radar data in the network layers. The proposed CameraRadarFusion Net (CRF-Net) automatically learns at which level the fusion of the sensor data is most beneficial for the detection result. Additionally, we introduce BlackIn, a training strategy inspired by Dropout, which focuses the learning on a specific sensor type. We show that the fusion network is able to outperform a state-of-the-art image-only network for two different datasets. The code for this research will be made available to the public at: https://github.com/TUMFTM/CameraRadarFusionNet.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
(2014) · Cited by 33,419
(2016) · Cited by 31,590
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
nuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152
Multi-view 3D Object Detection Network for Autonomous Driving
Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.691 · Cited by 3,450
Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors
Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer, I., Wojna, Z., Song, Y., Guadarrama, S., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.351 · Cited by 2,743
Joint 3D Proposal Generation and Object Detection from View Aggregation
Ku, J., Mozifian, M., Lee, J., Harakeh, A., & Waslander, S. L. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594049 · Cited by 1,697
Feng, D., Haase-Schutz, C., Rosenbaum, L., Hertlein, H., Glaser, C., Timm, F., Wiesbeck, W., & Dietmayer, K. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.2972974 · Cited by 1,425
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
He, Y., Lin, J., Liu, Z., Wang, H., Li, L.-J., & Han, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_48 · Cited by 1,400
A Toolbox for Easily Calibrating Omnidirectional Cameras
Scaramuzza, D., Martinelli, A., & Siegwart, R. (2006) · 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2006.282372 · Cited by 510
A toolbox for easily calibrating omnidirectional cameras
Scaramuzza, D., Martinelli, A., & Siegwart, R. (2006) · doi:10.3929/ethz-a-005704721 · Cited by 405
LIDAR–camera fusion for road detection using fully convolutional neural networks
Caltagirone, L., Bellone, M., Svensson, L., & Wahde, M. (2019) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2018.11.002 · Cited by 370
Sensors and Sensor Fusion in Autonomous Vehicles
Kocic, J., Jovicic, N., & Drndarevic, V. (2018) · 2018 26th Telecommunications Forum (TELFOR) · doi:10.1109/telfor.2018.8612054 · Cited by 296
Survey on Ranging Sensors and Cooperative Techniques for Relative Positioning of Vehicles
De Ponte Müller, F. (2017) · Sensors · doi:10.3390/s17020271 · Cited by 258
Distant Vehicle Detection Using Radar and Vision
Chadwick, S., Maddern, W., & Newman, P. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794312 · Cited by 233
Learn to combine modalities in multimodal deep learning
Liu, K., Li, Y., Xu, N., & Natarajan, P. (2018) · arXiv · doi:10.48550/arxiv.1805.11730 · Cited by 136
Automotive radar and camera fusion using Generative Adversarial Networks
Lekić, V., & Babić, Z. (2019) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2019.04.002 · Cited by 88
Jha, H., Lodhi, V., & Chakravarty, D. (2019) · International Conference on Signal Processing and Integrated Networks (SPIN) · doi:10.1109/spin.2019.8711717 · Cited by 67
BlackOut: Speeding up Recurrent Neural Network Language Models With Very Large VocabulariesPreprint
Ji, S., Vishwanathan, S. V. N., Satish, N. R., Anderson, M. J., & Dubey, P. K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06909 · Cited by 62
Sensor fusion for vehicle tracking with camera and radar sensor
Kim, K.-E., Lee, C.-J., Pae, D.-S., & Lim, M. (2017) · International Conference on Control, Automation and Systems (ICCAS) · doi:10.23919/iccas.2017.8204375 · Cited by 57
Radar and vision fusion for the real-time obstacle detection and identification
Zhang, X., Zhou, M., Qiu, P., Huang, Y., & Li, J. (2019) · Industrial Robot the international journal of robotics research and application · doi:10.1108/ir-06-2018-0113 · Cited by 51
Radar-vision fusion for object classification
Ji, Z., & Prokhorov, D. V. (2008) · Cited by 49
Frontal object perception for Intelligent Vehicles based on radar and camera fusion
Han, S., Wang, X., Xu, L., Sun, H., & Zheng, N. (2016) · Chinese Control Conference (CCC) · doi:10.1109/chicc.2016.7553978 · Cited by 42
Multi-Stage Residual Fusion Network for LIDAR-Camera Road Detection
Yu, D., Xiong, H., Xu, Q. Y., Wang, J., & Li, K. (2019) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2019.8813983 · Cited by 18
Cited by
Sensor and Sensor Fusion Technology in Autonomous Vehicles: A Review
Yeong, D. J., Velasco-Hernandez, G., Barry, J., & Walsh, J. (2021) · Sensors · doi:10.3390/s21062140 · Cited by 1,032
CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection
Nabati, R., & Qi, H. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00157 · Cited by 461
FUTR3D: A Unified Sensor Fusion Framework for 3D Detection
Chen, X., Zhang, T., Wang, Y., Wang, Y., & Zhao, H. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00022 · Cited by 328
Multi-Class Road User Detection With 3+1D Radar in the View-of-Delft Dataset
Palffy, A., Pool, E., Baratam, S., Kooij, J. F. P., & Gavrila, D. M. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3147324 · Cited by 315
Autonomous Vehicles Enabled by the Integration of IoT, Edge Intelligence, 5G, and Blockchain
Biswas, A., & Wang, H.-C. (2023) · Sensors · doi:10.3390/s23041963 · Cited by 274
Metadata
- Authors
- Nobis, F., Geisslinger, M., Weber, M., Betz, J., & Lienkamp, M.
- Venue
- 2019 Sensor Data Fusion: Trends, Solutions, Applications (SDF)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 1-7
- OpenAlex
- W2989940015
- Cited by
- 335
- References
- 34
- FWCI
- 33.67
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.