RCDPT: Radar-Camera Fusion Dense Prediction Transformer
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Recently, transformer networks have outperformed traditional deep neural networks in natural language processing and show a large potential in many computer vision tasks compared to convolutional backbones. In the original transformer, readout tokens are used as designated vectors for aggregating information from other tokens. However, the performance of using readout tokens in a vision transformer is limited. Therefore, we propose a novel fusion strategy to integrate radar data into a dense prediction transformer network by reassembling camera representations with radar representations. Instead of using readout tokens, radar representations contribute additional depth information to a monocular depth estimation model and improve performance. We further investigate different fusion approaches that are commonly used for integrating additional modality in a dense prediction transformer network. The experiments are conducted on the nuScenes dataset, which includes camera images, lidar, and radar data. The results show that our proposed method yields better performance than the commonly used fusion strategies and outperforms existing convolutional depth estimation models that fuse camera images and radar.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
nuScenes: A Multimodal Dataset for Autonomous Driving
Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01164 · Cited by 7,152
Vision Transformers for Dense Prediction
Ranftl, R., Bochkovskiy, A., & Koltun, V. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01196 · Cited by 2,373 · cited 9×
“The overall structure from DPT [1] is utilized, and multi-modal reassemble layers is proposed.”
BackgroundMethodInfluentialDeep Ordinal Regression Network for Monocular Depth Estimation
Fu, H., Gong, M., Wang, C., Batmanghelich, K., & Tao, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00214 · Cited by 1,991
Pyramid Stereo Matching Network
Chang, J.-R., & Chen, Y.-S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00567 · Cited by 1,807
GA-Net: Guided Aggregation Net for End-To-End Stereo Matching
Zhang, F., Prisacariu, V., Yang, R., & Torr, P. H. S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00027 · Cited by 811
Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image
Ma, F., & Karaman, S. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8460184 · Cited by 650
Multi-Modal Fusion Transformer for End-to-End Autonomous Driving
Prakash, A., Chitta, K., & Geiger, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00700 · Cited by 584
Cited by
TC–Radar: Transformer–CNN Hybrid Network for Millimeter-Wave Radar Object Detection
Jia, F., Li, C., Bi, S., Qian, J., Wei, L., & Sun, G. (2024) · Remote Sensing · doi:10.3390/rs16162881 · Cited by 26
Transformer-based sensor fusion for autonomous vehicles: A comprehensive review
Abdulmaksoud, A., & Ahmed, R. (2025) · IEEE Access · doi:10.1109/access.2025.3545032 · Cited by 25
“RCDPT [67] presents an elegant method for fusing radar and camera data for depth estimation.”
Deng, K., Xing, L., Wu, H., Ma, H., Ling, Y., & Gao, J. (2025) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-025-00302-0 · Cited by 25
Robust tracking and detection based on radar camera fusion filtering in urban autonomous driving
Baek, S.-Y., Kim, J.-H., & Yi, K. (2024) · Intelligent Service Robotics · doi:10.1007/s11370-024-00563-0 · Cited by 15
Image-Based Lunar Hazard Detection in Low Illumination Simulated Conditions via Vision Transformers
Ghilardi, L., & Furfaro, R. (2023) · Sensors · doi:10.3390/s23187844 · Cited by 14
“The readout token is generally responsible for aggregating information from other tokens [30].”
Background
Metadata
- Authors
- Lo, C.-C., & Vandewalle, P.
- Affiliations
- KU Leuven (BE)
- Venue
- ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Publisher
- IEEE
- Published
- 2023-06-04
- Pages
- 1-5
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W4372260119
- Cited by
- 18
- References
- 21
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 9 · 2023: 2
- FWCI
- 1.93
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE International Conference on Acoustics Speech and Signal Processing (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Lirias (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- KU Leuven
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Keywords
- radar-camera fusion, vision transformer, dense prediction, monocular depth estimation, multi-sensor fusion, LiDAR, nuScenes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: RCDPT: Radar-Camera Fusion Dense Prediction Transformer. https://app.cochat.ai/papers/10.1109/icassp49357.2023.10096129
- The paper
- Lo, C.-C., & Vandewalle, P. (2023). RCDPT: Radar-Camera Fusion Dense Prediction Transformer. ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). https://doi.org/10.1109/icassp49357.2023.10096129
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.