Cited by 247Visual Attention and Saliency DetectionAdvanced Image Fusion TechniquesAdvanced Neural Network Applications

RGB-T Salient Object Detection via Fusing Multi-Level CNN Features

Zhang, Qiang, Huang, Nianchang, Yao, Lin, Zhang, Dingwen, Shan, Caifeng, Han, Jungong·2020·IEEE Transactions on Image Processing

Abstract

RGB-induced salient object detection has recently witnessed substantial progress, which is attributed to the superior feature learning capability of deep convolutional neural networks (CNNs). However, such detections suffer from challenging scenarios characterized by cluttered backgrounds, low-light conditions and variations in illumination. Instead of improving RGB based saliency detection, this paper takes advantage of the complementary benefits of RGB and thermal infrared images. Specifically, we propose a novel end-to-end network for multi-modal salient object detection, which turns the challenge of RGB-T saliency detection to a CNN feature fusion problem. To this end, a backbone network (e.g., VGG-16) is first adopted to extract the coarse features from each RGB or thermal infrared image individually, and then several adjacent-depth feature combination (ADFC) modules are designed to extract multi-level refined features for each single-modal input image, considering that features captured at different depths differ in semantic information and visual details. Subsequently, a multi-branch group fusion (MGF) module is employed to capture the cross-modal features by fusing those features from ADFC modules for a RGB-T image pair at each level. Finally, a joint attention guided bi-directional message passing (JABMP) module undertakes the task of saliency prediction via integrating the multi-level fused features from MGF modules. Experimental results on several public RGB-T salient object detection datasets demonstrate the superiorities of our proposed algorithm over the state-of-the-art approaches, especially under challenging conditions, such as poor illumination, complex background and low contrast.

Integrity signals

Registry

Crossref · 10.1109/tip.2019.2959253

Cites

77 of 77 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A multi-branch group fusion module is proposed to fuse features from RGB and thermal infrared images at different levels in [182], since semantic information and visual details differ at different levels.

    “For example, a multi-branch group fusion module is proposed to fuse features from RGB and thermal infrared images at different levels in [182], since the semantic information and visual details differ at different levels.”
  • By using them with RGB images as input, cross-modal perceiving has attracted increasing attention in real-world applications, e.g., scene parsing for autonomous driving [181], [85], object detection and tracking in low-light scenarios [182], [183], and action recognition [184].

    “By using them with RGB images as input, cross-modal perceiving has attracted increasing attention in real-world applications, e.g., scene parsing for autonomous driving [181], [85], object detection and tracking in low-light scenarios [182], [183], and action recognition [184].”

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Zhang, Q., Huang, N., Yao, L., Zhang, D., Shan, C., & Han, J.
Affiliations
  • Xidian University (CN)
  • Philips (Netherlands) (NL)
  • University of Warwick (GB)
Venue
IEEE Transactions on Image Processing
Published
2020
Volume
29
Pages
3321-3335
ISSN
  • 1057-7149 (ISSN-L)
  • 1941-0042
OpenAlex
W2995936506
Cited by
247
References
77
Citations by year
2026: 23 · 2025: 37 · 2024: 37 · 2023: 52 · 2022: 49 · 2021: 38 · 2020: 11
FWCI
10.96
Type
journal article
Language
en
Open access
closed
Locations
  • IEEE Transactions on Image Processing (publishedVersion)
  • PubMed (publishedVersion)
  • Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
Funders
  • National Natural Science Foundation of China (61773301, 61876140, 61971004)
  • China Postdoctoral Support Scheme for Innovative Talents (BX20180236)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
  • Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
thermal infrared images, RGB-thermal fusion, multi-level feature fusion, cross-modal feature fusion, background clutter, low-light conditions, illumination variation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review