CoChat
Sign Up
Cited by 990Autonomous Vehicle Technology and SafetyAdvanced Neural Network ApplicationsAdvanced Vision and Imaging

TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers

Bai, Xuyang, Hu, Zeyu, Zhu, Xinge, Huang, Qingqiu, Chen, Yilun, Fu, Hangbo, Tai, Chiew-Lan·2022·2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
DOI 10.1109/cvpr52688.2022.00116
Open this paper in CoChat — free account

Abstract

LiDAR and camera are two important sensors for 3D object detection in autonomous driving. Despite the increasing popularity of sensor fusion in this field, the robustness against inferior image conditions, e.g., bad illumination and sensor misalignment, is under-explored. Existing fusion methods are easily affected by such conditions, mainly due to a hard association of LiDAR points and image pixels, established by calibration matrices. We propose TransFusion, a robust solution to LiDAR-camera fusion with a soft-association mechanism to handle inferior image conditions. Specifically, our TransFusion consists of convolutional backbones and a detection head based on a transformer decoder. The first layer of the decoder predicts initial bounding boxes from a LiDAR point cloud using a sparse set of object queries, and its second decoder layer adaptively fuses the object queries with useful image features, leveraging both spatial and contextual relationships. The attention mechanism of the transformer enables our model to adaptively determine where and what information should be taken from the image, leading to a robust and effective fusion strategy. We additionally design an image-guided query initialization strategy to deal with objects that are difficult to detect in point clouds. TransFusion achieves state-of-the-art performance on large-scale datasets. We provide extensive experiments to demonstrate its robustness against degenerated image quality and calibration errors. We also extend the proposed method to the 3D tracking task and achieve the 1st place in the leader-board of nuScenes tracking, showing its effectiveness and generalization capability. [code release]

Integrity signals

Registry

Crossref · 10.1109/cvpr52688.2022.00116

Cites

64 of 64 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Encoder-decoder architectures and Transformer-based structures are widely used to model dependencies between modalities.

    “For example, encoder-decoder architectures and Transformer-based structures are widely used to model dependencies between modalities [12,13];”

    From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [13]

  • TransFusion introduces a soft association fusion framework using a Transformer decoder for high-precision 3D detection.

    “TransFusion [13] introduces a soft association fusion framework using a Trans-”

    From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [13]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Bai, X., Hu, Z., Zhu, X., Huang, Q., Chen, Y., Fu, H., & Tai, C.-L.
Affiliations
  • Hong Kong University of Science and Technology (HK)
  • ADS, IAS BU, Huawei
  • City University of Hong Kong (HK)
ORCID
Venue
2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2022-06
Pages
1080-1089
ISSN
1063-6919 (ISSN-L)
DOI
10.1109/cvpr52688.2022.00116
OpenAlex
W4312707458
Semantic Scholar
54fbb9300530d1deea596ad19807adedf1dc89dc
Cited by
990
References
64
Citations by year
2026: 228 · 2025: 292 · 2024: 272 · 2023: 185 · 2022: 13
FWCI
195.65
Type
conference paper
Language
en
Open access
closed
Licence
Locations
IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
Funders
City University of Hong Kong (7005729)
Topics
  • Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
Keywords
LiDAR-camera fusion, 3D object detection, autonomous driving, transformer decoder, calibration error, sensor misalignment, image quality robustness, object queries, 3D tracking

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review