Cited by 216Advanced Neural Network ApplicationsAutonomous Vehicle Technology and Safety3D Shape Modeling and Analysis

Perception-Aware Multi-Sensor Fusion for 3D LiDAR Semantic Segmentation

Zhuang, Zhuangwei, Li, Rong, Jia, Kui, Wang, Qicheng, Li, Yuanqing, Tan, Mingkui·2021·2021 IEEE/CVF International Conference on Computer Vision (ICCV)

Abstract

3D LiDAR (light detection and ranging) semantic segmentation is important in scene understanding for many applications, such as auto-driving and robotics. For example, for autonomous cars equipped with RGB cameras and LiDAR, it is crucial to fuse complementary information from different sensors for robust and accurate segmentation. Existing fusion-based methods, however, may not achieve promising performance due to the vast difference between the two modalities. In this work, we investigate a collaborative fusion scheme called perception-aware multi-sensor fusion (PMF) to exploit perceptual information from two modalities, namely, appearance information from RGB images and spatio-depth information from point clouds. To this end, we first project point clouds to the camera coordinates to provide spatio-depth information for RGB images. Then, we propose a two-stream network to extract features from the two modalities, separately, and fuse the features by effective residual-based fusion modules. Moreover, we propose additional perception-aware losses to measure the perceptual difference between the two modalities. Extensive experiments on two benchmark data sets show the superiority of our method. For example, on nuScenes, our PMF outperforms the state-of-the-art method by 0.8% in mIoU.

Integrity signals

Registry

Crossref · 10.1109/iccv48922.2021.01597

Cites

62 of 62 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

3 of the 65 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Zhuang, Z., Li, R., Jia, K., Wang, Q., Li, Y., & Tan, M.
Affiliations
  • South China University of Technology (CN)
  • Shenzhen Youjia Innov Tech Co., Ltd
Venue
2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher
IEEE
Published
2021-10
Pages
16260-16270
OpenAlex
W3173732446
Cited by
216
References
65
Citations by year
2026: 26 · 2025: 58 · 2024: 68 · 2023: 47 · 2022: 16 · 2021: 1
FWCI
10.79
Type
conference paper
Language
en
Open access
closed
Locations
IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
Funders
  • Research and Development
  • Ministry of Science and Technology
  • Fundamental Research Funds for the Central Universities
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
  • Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
Keywords
multi-sensor fusion, RGB-LiDAR fusion, point cloud projection, two-stream network, autonomous driving, nuScenes

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review