Cited by 77Advanced Neural Network ApplicationsAdvanced Vision and ImagingDomain Adaptation and Few-Shot Learning

Exploring RGB+Depth Fusion for Real-Time Object Detection

Open access
Ophoff, Tanguy, Van Beeck, Kristof, Goedemé, Toon·2019·Sensors

Abstract

In this paper, we investigate whether fusing depth information on top of normal RGB data for camera-based object detection can help to increase the performance of current state-of-the-art single-shot detection networks. Indeed, depth sensing is easily acquired using depth cameras such as a Kinect or stereo setups. We investigate the optimal manner to perform this sensor fusion with a special focus on lightweight single-pass convolutional neural network (CNN) architectures, enabling real-time processing on limited hardware. For this, we implement a network architecture allowing us to parameterize at which network layer both information sources are fused together. We performed exhaustive experiments to determine the optimal fusion point in the network, from which we can conclude that fusing towards the mid to late layers provides the best results. Our best fusion models significantly outperform the baseline RGB network in both accuracy and localization of the detections.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/s19040866

Cites

36 of 36 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Experimental results show that using depth information, optical flow information, and LiDAR point clouds in addition to conventional RGB data can improve the performance of one-stage based detection systems.

    “According to experimental results, it has been shown that the use of depth information [183, 184], optical flow information [185], and LiDAR point clouds [186] in addition to conventional RGB data can improve the performance of one-stage based detection systems.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. [14]Girshick R. Fast R-CNN Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV) Santiago, Chile 7–13 December 2015 1440 1448
  2. [18]Redmon J. Farhadi A. Yolov3: An incremental improvement arXiv 2018 1804.02767

Cited by

Sorted by citations

Authors
Ophoff, T., Van Beeck, K., & Goedemé, T.
Affiliations
  • KU Leuven (BE)
  • EAVISE, KU Leuven, 2860 Sint-Katelijne-Waver, Belgium
Corresponding author
Ophoff, Tanguy
Venue
Sensors
Publisher
MDPI AG
Published
2019-02-19
Volume
19
Issue
4
Pages
866
ISSN
1424-8220 (ISSN-L)
OpenAlex
W2916052001
Cited by
77
References
35
Citations by year
2026: 8 · 2025: 8 · 2024: 8 · 2023: 10 · 2022: 16 · 2021: 17 · 2020: 8 · 2019: 2
FWCI
4.80
Type
journal article
Language
en
Open access
Locations
  • Sensors (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, other-oa)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Lirias (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Agentschap Innoveren en Ondernemen
  • Fonds Wetenschappelijk Onderzoek (S003817N)
  • KU Leuven
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
RGB-D fusion, sensor fusion, object detection, single-shot detection, depth camera, lightweight convolutional neural network, real-time detection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review