CoChat
Sign Up
Cited by 130Advanced Neural Network ApplicationsAutonomous Vehicle Technology and SafetyMultimodal Machine Learning Applications

CRAFT: Camera-Radar 3D Object Detection with Spatio-Contextual Fusion Transformer

Open access
Kim, Youngseok, Kim, Sanmin, Choi, Jun Won, Kum, Dongsuk·2023·Proceedings of the AAAI Conference on Artificial Intelligence
DOI 10.1609/aaai.v37i1.25198
Open this paper in CoChat — free accountOpen PDF

Abstract

Camera and radar sensors have significant advantages in cost, reliability, and maintenance compared to LiDAR. Existing fusion methods often fuse the outputs of single modalities at the result-level, called the late fusion strategy. This can benefit from using off-the-shelf single sensor detection algorithms, but late fusion cannot fully exploit the complementary properties of sensors, thus having limited performance despite the huge potential of camera-radar fusion. Here we propose a novel proposal-level early fusion approach that effectively exploits both spatial and contextual properties of camera and radar for 3D object detection. Our fusion framework first associates image proposal with radar points in the polar coordinate system to efficiently handle the discrepancy between the coordinate system and spatial properties. Using this as a first stage, following consecutive cross-attention based feature fusion layers adaptively exchange spatio-contextual information between camera and radar, leading to a robust and attentive fusion. Our camera-radar fusion approach achieves the state-of-the-art 41.1% mAP and 52.3% NDS on the nuScenes test set, which is 8.7 and 10.8 points higher than the camera-only baseline, as well as yielding competitive performance on the LiDAR method.

AI snapshot

Written from the paper’s full text.

TL;DR

CRAFT is a proposal-level early camera-radar fusion framework for 3D object detection that uses soft polar association and cross-attention to exchange spatial and contextual information. It reports 41.1% mAP and 52.3% NDS on the nuScenes test set, described as 8.7 and 10.8 points above the camera-only baseline.

Key findings

  • On nuScenes test, CRAFT achieves 41.1% mAP and 52.3% NDS, with the abstract stating this is 8.7 and 10.8 points higher than the camera-only baseline.
  • Ablations on the car class show the fusion transformer improves AP to 41.3 versus 34.7 for radar-only FP and 38.9 for concatenation FC; Soft Polar Association reaches 41.3 AP versus 39.5 for Ball Query; polar coordinate improves 0.5m AP by +13.6 and mean AP by +6.5 over Cartesian.
  • The method improves more for distant objects, with a 32.2% improvement for objects farther than 35m, and is robust to radar sparsity: +6.2% when no or few radar points are available and +28.7% when more radar points are provided.

Methods

  • CRAFT first uses a camera 3D detector (CenterNet with DLA-34) to generate image proposals and a PointNet++-based extractor for radar point features. Soft Polar Association then associates radar points to proposals using uncertainty-aware adaptive thresholds in polar coordinates, followed by consecutive cross-attention encoders (Image-to-Radar and Radar-to-Image Feature Encoders) and fusion detection heads that predict fusion score, polar-coordinate offsets, center-ness, and velocity.
  • Evaluation uses nuScenes (700/150/150 train/val/test scenes) with six surrounding cameras and five radar point clouds; metrics are mAP, NDS, and true-positive errors. Training uses 24 epochs, batch size 32, learning rate 2×10⁻⁴ on 4 RTX 3090 GPUs, with the camera detector pre-trained and frozen during fusion training.
  • Comparisons are reported on nuScenes test and val sets, with ablations on the val set for the car class and analyses by object distance and number of radar points.

Limitations

  • Stated: the camera 3D detector is a lightweight proof of concept, proposing a camera 3D detector is out of scope, and a more advanced camera detector is left for future work. Ablation experiments report only the car class because gains for other classes are less consistent and easier to fluctuate.
  • Evident from the text: evaluation is limited to nuScenes with single-frame input and no test-time augmentation for fusion; the camera detector is frozen during fusion training, and radar sparsity and measurement ambiguity are handled but not eliminated.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1609/aaai.v37i1.25198

Cites

59 of 59 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 4 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    CRAFT surpasses all compared camera-radar and single-frame camera approaches on the nuScenes test set.

    “As shown in Table 1, our CRAFT outperforms all competing camera-radar and single-frame camera methods on test set.”
    Quote in the paperSubstantial claim
  • Findingown data

    SPA provides the best trade-off between recall and precision for effective camera-radar feature fusion.

    “Our SPA enables the best trade-off between recall and precision for effective camera-radar feature fusion.”
    Quote in the paperSubstantial claim
  • Findingreasoning

    For distant objects beyond 35m, the fusion method achieves a significant improvement of 32.2%.

    “As the depth estimation becomes more inaccurate on distant objects farther than 35m, our fusion method benefits more from the radar measurement and achieves a significant improvement of 32.2%.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Depth estimation becomes less accurate for distant objects beyond 35m.
  • Findingnone given

    CRAFT's inference speed is more than twice as fast as that of the other methods.

    “Importantly, our CRAFT has more than two times faster inference speed compared to other methods.”
    Quote in the paperSubstantial claim
  • Findingown data

    CRAFT achieves consistent improvements across different numbers of radar points.

    “CRAFT achieves consistent improvements over a various number of radar points”
    Quote in the paperSubstantial claim
  • May combine two claims. 17
  • Classified as probably not a substantial claim. 1

  • May combine two claims.

    Findingown data

    The fusion transformer (FT) yields a large AP gain over radar-only (FP, +6.6%) and naive concatenation (FC, +2.4%), particularly at the strict 0.5m threshold.

    “Table 3a shows that our fusion transformer (FT) brings large AP improvement over radar-only (FP, +6.6%) and naive concatenation (FC, +2.4%), especially at the strict 0.5m threshold.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On the nuScenes val set, CRAFT improves over CenterFusion by +17.2% versus 4.0% on Car and +17.3% versus 2.2% on Bus.

    “Although CenterFusion (Nabati and Qi 2021) and ours have a similar camera 3D detector architecture, our CRAFT achieves a remarkable performance boost (+17.2% vs. 4.0% on Car, +17.3% vs. 2.2% on Bus)”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Nabati and Qi 2021

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Camera-radar fusion achieves significant improvements over the camera-only baseline, with larger gains for distant objects.

    “Our camera-radar fusion achieves significant improvements over the camera-only baseline and brings larger improvements for distant objects, as shown in Fig. 4a.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The method performs robustly when no or few radar points are available (+6.2%) and shows greater improvement when more radar points are provided (+28.7%).

    “Our method yields robust performance although radar points are not available or only a few (+6.2%) and shows better performance improvement when more radar points are provided (+28.7%).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Combining image features with attention-based fusion helps improve recall and reduce localization error.

    “It shows that using both image feature and attention-based fusion is beneficial for improving recall and reducing localization error.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The simple coordinate transformation makes it easier for the network to learn spatial information and effectively reduces the localization error of image proposals.

    “Thus, the simple coordinate transformation makes the network learn spatial information easier and effectively reduces the localization error of image proposal.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Late fusion strategies fail to fully exploit sensor complementarity, resulting in limited performance.

    “late fusion cannot fully exploit the complementary properties of sensors, thus having limited performance despite the huge potential of camera-radar fusion.”
    • Also stated in: § 1 Introduction · plain
      “late fusion strategies that rely on heuristics and postprocessing techniques suffer from performance-reliability trade-offs, especially when two sensor predictions disagree; thus, late fusion cannot exploit the full potential of each sensor.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The soft association strategy and cross-attention based feature fusion method effectively exploit spatial characteristics and robustly handle radar measurement ambiguity.

    “We propose the soft association strategy between image proposal and radar points and cross-attention based feature fusion method, which can effectively exploit spatial characteristics and robustly handle radar measurement ambiguity.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Radar measurement ambiguities include missed detections on objects due to low radar cross-section and false positives in the background due to multi-path effects, and these must be considered.

  • May combine two claims.

    Premisereasoning

    Learning-based early fusion for camera and radar has greater potential than late fusion, but it requires a deep understanding of each sensor's characteristics.

    “a learning-based early fusion strategy that fuses the intermediate information of each sensor at an early stage has much higher potential, but it requires an in-depth understanding of each sensor's characteristics to find the”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisereasoning

    Radar has high resolution and accuracy in the radial direction, with 0.4 m resolution and ±0.1 m accuracy.

    “the radar has high resolution and accuracy in the radial direction (0.4m and ±0.1m), which is measured by Fast Fourier Transformation FFT.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given

    The adaptive-threshold association maximizes the use of informative foreground radar points while excluding most clutter points in the azimuthal direction.

    “In this way, the proposed association using an adaptive threshold can maximize the chance of using informative foreground radar points while excluding most clutter points in the azimuthal direction.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Radar measurement ambiguities include missed detections on objects due to low radar cross-section and false positives in the background due to multi-path effects, and these must be considered.

  • May combine two claims.

    Interpretationnone given

    CRAFT's proposal-level early fusion framework reduces the mismatch between coordinate systems and handles radar measurement ambiguities.

    “We investigate the characteristics of camera-radar and propose a proposal-level early fusion framework that can mitigate the coordinate system discrepancy and measurement ambiguities.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Radar measurement ambiguities include missed detections on objects due to low radar cross-section and false positives in the background due to multi-path effects, and these must be considered.

  • May combine two claims.

    Interpretationnone given

    The largest performance gains arise from better localization and velocity estimation, which respectively improve recall at strict thresholds and NDS scores.

    “The most performance gain comes from the improved localization and velocity estimation, which improve the recall at strict thresholds (e.g., 0.5m or 1m) and NDS scores, respectively.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    Choosing a fusion strategy suited to sensor characteristics is important.

    “which shows the importance of an appropriate fusion strategy considering sensor characteristics.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Radar azimuthal measurement is inaccurate, with 4.5° resolution and ±1° accuracy, corresponding to about 4 m and ±1 m at 50 m distance.

    “the azimuthal measurement obtained by digital beamforming using multiple receive antennas (Johnson and Dudgeon 1992) is inaccurate (4.5 • and ±1 • , approximately 4m and ±1m at 50m distance).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Johnson and Dudgeon 1992

      Not checked in the free run.

  • May combine two claims.

    Study design and conductreasoning

    To ensure a fair comparison, the reported methods use only single-frame input and are trained solely on the nuScenes dataset, removing effects of multi-sequence input and large-scale depth pre-training.

    “To eliminate the effect of multi-sequence input and large-scale depth pre-training and compare with previous methods fairly, we report methods with single frame input trained on nuScenes dataset only.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The proposed framework is effective and robust for camera-radar 3D detection.

    “In this paper, we have proposed an effective and robust camera-radar 3D detection framework.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    Extensive experiments validate the framework's design choices and show its advantages over a camera-only baseline.

    “Extensive experiments validate the design choice of our framework and show the advantages over a camera-only baseline.”
    Quote in the paperSubstantial claim
18 of 18

  • The paper relies on this without support.

    Camera and radar sensors have significant advantages in cost, reliability, and maintenance compared to LiDAR.

    “Camera and radar sensors have significant advantages in cost, reliability, and maintenance compared to LiDAR.”

  • The paper relies on this without support.

    The rich semantic information of camera and long-range detection with weather condition robustness of radar are essential attributes that LiDAR cannot provide.

    “the rich semantic information of the camera and long-range detection with weather condition robustness of radar are essential attributes that LiDAR cannot provide.”

  • The paper relies on this without support.

    Learning-based camera-radar 3D object detection has not been thoroughly explored.

    “learning-based camera-radar 3D object detection (Lim et al. 2019;Kim, Choi, and Kum 2020;Nabati and Qi 2021) for autonomous driving has not been thoroughly explored.”

  • The paper relies on this without support.

    Radar measurement ambiguities include missed detections on objects due to low radar cross-section and false positives in the background due to multi-path effects, and these must be considered.

    “Measurement ambiguities are the cases when radar points are occasionally missed on objects (false negatives due to the low radar cross-section) or shown in the background (false positives due to radar multi-path problem), thus these have to be considered.”

  • The paper relies on this without support.

    Existing camera-radar 3D detectors have not thoroughly considered the spatial properties of camera and radar.

    “Thus, existing camera-radar 3D detectors have not thoroughly considered the spatial properties of camera and radar.”

  • The paper relies on this without support.

    Depth estimation becomes less accurate for distant objects beyond 35m.

    “As the depth estimation becomes more inaccurate on distant objects farther than 35m”
6 of 6

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Simple-BEV combines camera and radar features through rasterised radar information and bilinear sampling for 3D bird's-eye view representation.

    “Simple-BEV [99] combines camera and radar features through rasterised radar information and bilinear sampling for 3D bird's-eye view representation.”

    From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [99]

    The Camera-Radar Fusion section discusses other fusion methods but does not mention Simple-BEV. Searches for Simple-BEV, rasterised radar information, and bilinear sampling found no passage in our copy that addresses the claimed method. (LLM, conf=0.75)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • PointNet++ is cited to the PointNet reference

    Minor

    The methods overview cites Qi et al. 2017a for PointNet++, but the paper's reference list identifies 2017a as PointNet. PointNet++ is listed under 2017b.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 5

  1. Kim, J.; Kim, Y.; and Kum, D. 2020. Low-level sensor fu- sion network for 3D vehicle detection using radar range- azimuth heatmap and monocular image. In Proceedings of the Asian Conference on Computer Vision (ACCV), 388- 402.
  2. Lim, T.-Y.; Ansari, A.; Major, B.; Fontijne, D.; Hamilton, M.; Gowaikar, R.; and Subramanian, S. 2019. Radar and camera early fusion for vehicle detection in advanced driver assistance systems. In Advances in Neural Information Pro- cessing Systems Workshops (NeurIPSW).
  3. Long, Y.; Morris, D.; Liu, X.; Castro, M.; Chakravarty, P.; and Narayanan, P. 2021a. Full-Velocity Radar Returns by Radar-Camera Fusion. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision (ICCV), 16198- 16207.
  4. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, Ł.; and Polosukhin, I. 2017. At- tention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), 6000-6010.
  5. Xiong, R.; Yang, Y.; He, D.; Zheng, K.; Zheng, S.; Xing, C.; Zhang, H.; Lan, Y.; Wang, L.; and Liu, T. 2020. On layer normalization in the transformer architecture. In Proceed- ings of the International Conference on Machine Learning (ICML), 10524-10533.

Cited by

Sorted by citations

Related

Authors
Kim, Y., Kim, S., Choi, J. W., & Kum, D.
Affiliations
  • Korea Advanced Institute of Science and Technology (KR)
  • Hanyang University (KR)
ORCID
Venue
Proceedings of the AAAI Conference on Artificial Intelligence
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Published
2023-06-26
Volume
37
Issue
1
Pages
1160-1168
ISSN
  • 2159-5399 (ISSN-L)
  • 2374-3468
DOI
10.1609/aaai.v37i1.25198
OpenAlex
W4382469124
Cited by
130
References
59
Citations by year
2026: 31 · 2025: 48 · 2024: 36 · 2023: 15
FWCI
13.84
Type
conference paper
Language
en
Open access
Locations
  • Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • KAIST Institutional Repository (KAIST) (submittedVersion)
  • AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Ministry of Science and ICT, South Korea (2021-0-00951)
  • Institute for Information and Communications Technology Promotion (2021-0-00951)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
radar-camera fusion, 3D object detection, feature-level fusion, cross-attention, transformer models, sensor fusion, autonomous driving perception, nuScenes

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-08.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-08.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review