The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Learning Selective Self-Mutual Attention for RGB-D Saliency Detection
Abstract
Saliency detection on RGB-D images is receiving more and more research interests recently. Previous models adopt the early fusion or the result fusion scheme to fuse the input RGB and depth data or their saliency maps, which incur the problem of distribution gap or information loss. Some other models use the feature fusion scheme but are limited by the linear feature fusion methods. In this paper, we propose to fuse attention learned in both modalities. Inspired by the Non-local model, we integrate the self-attention and each other's attention to propagate long-range contextual dependencies, thus incorporating multi-modal information to learn attention and propagate contexts more accurately. Considering the reliability of the other modality's attention, we further propose a selection attention to weight the newly added attention term. We embed the proposed attention module in a two-stream CNN for RGB-D saliency detection. Furthermore, we also propose a residual fusion module to fuse the depth decoder features into the RGB stream. Experimental results on seven benchmark datasets demonstrate the effectiveness of the proposed model components and our final saliency model. Our code and saliency maps are available at https://github.com/nnizhang/S2MA.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898
(2017) · Cited by 11,625
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596
Dual Attention Network for Scene Segmentation
Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., & Lu, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00326 · Cited by 7,351
CCNet: Criss-Cross Attention for Semantic Segmentation
Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., & Liu, W. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00069 · Cited by 3,045
GCNet: Non-Local Networks Meet Squeeze-Excitation Networks and Beyond
Cao, Y., Xu, J., Lin, S., Wei, F., & Hu, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00246 · Cited by 2,307
Structure-Measure: A New Way to Evaluate Foreground Maps
Fan, D.-P., Cheng, M.-M., Liu, Y., Li, T., & Borji, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.487 · Cited by 2,092
DenseASPP for Semantic Segmentation in Street Scenes
Yang, M., Yu, K., Zhang, C., Li, Z., & Yang, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00388 · Cited by 1,674
Enhanced-alignment Measure for Binary Foreground Map Evaluation
Fan, D.-P., Gong, C., Cao, Y., Ren, B., Cheng, M.-M., & Borji, A. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/97 · Cited by 1,607
Object-Contextual Representations for Semantic Segmentation
Yuan, Y., Chen, X., & Wang, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58539-6_11 · Cited by 1,285
A Simple Pooling-Based Design for Real-Time Salient Object Detection
Liu, J.-J., Hou, Q., Cheng, M.-M., Feng, J., & Jiang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00404 · Cited by 1,100
PiCANet: Learning Pixel-Wise Contextual Attention for Saliency Detection
Liu, N., Han, J., & Yang, M.-H. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00326 · Cited by 863
Self-citationDHSNet: Deep Hierarchical Saliency Network for Salient Object Detection
Liu, N., & Han, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.80 · Cited by 833
Self-citationDual Attention Networks for Multimodal Reasoning and Matching
Nam, H., Ha, J.-W., & Kim, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.232 · Cited by 726
Asymmetric Non-Local Neural Networks for Semantic Segmentation
Zhu, Z., Xu, M., Bai, S., Huang, T., & Bai, X. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00068 · Cited by 721
Expectation-Maximization Attention Networks for Semantic Segmentation
Li, X., Zhong, Z., Wu, J., Yang, Y., Lin, Z., & Liu, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00926 · Cited by 704
Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks
Fan, D.-P., Lin, Z., Zhang, Z., Zhu, M., & Cheng, M.-M. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2996406 · Cited by 702
RGBD Salient Object Detection: A Benchmark and Algorithms
Peng, H., Li, B., Xiong, W., Hu, W., & Ji, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_7 · Cited by 668
STC: A Simple to Complex Framework for Weakly-Supervised Semantic Segmentation
Wei, Y., Liang, X., Chen, Y., Shen, X., Cheng, M.-M., Feng, J., Zhao, Y., & Yan, S. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2636150 · Cited by 576
Depth saliency based on anisotropic center-surround difference
Ju, R., Ge, L., Geng, W., Ren, T., & Wu, G. (2014) · 2014 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2014.7025222 · Cited by 523
Shifting More Attention to Video Salient Object Detection
Fan, D.-P., Wang, W., Cheng, M.-M., & Shen, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00875 · Cited by 510
Attentive Feedback Network for Boundary-Aware Salient Object Detection
Feng, M., Lu, H., & Ding, E. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00172 · Cited by 497
Depth-Induced Multi-Scale Recurrent Attention Network for Saliency Detection
Piao, Y., Ji, W., Li, J., Zhang, M., & Lu, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00735 · Cited by 497
Leveraging stereopsis for saliency analysis
Niu, Y., Geng, Y., Li, X., & Liu, F. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247708 · Cited by 479
Saliency Detection on Light Field
Li, N., Ye, J., Ji, Y., Ling, H., & Yu, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.359 · Cited by 470
CNNs-Based RGB-D Saliency Detection via Cross-View Transfer and Multiview Fusion
Han, J., Chen, H., Liu, N., Yan, C., & Li, X. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2761775 · Cited by 423
Self-citationContrast Prior and Fluid Pyramid Integration for RGBD Salient Object Detection
Zhao, J.-X., Cao, Y., Fan, D.-P., Cheng, M.-M., Li, X.-Y., & Zhang, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00405 · Cited by 412
Chen, H., Li, Y., & Su, D. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2018.08.007 · Cited by 368
JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection
Fu, K., Fan, D.-P., Ji, G.-P., & Zhao, Q. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00312 · Cited by 365
RGBD Salient Object Detection via Deep Fusion
Qu, L., He, S., Zhang, J., Tian, J., Tang, Y., & Yang, Q. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2682981 · Cited by 360
Depth Enhanced Saliency Detection Method
Cheng, Y., Fu, H., Wei, X., Xiao, J., & Cao, X. (2014) · Proceedings of International Conference on Internet Multimedia Computing and Service · doi:10.1145/2632856.2632866 · Cited by 350
A Deep Network Solution for Attention and Aesthetics Aware Photo Cropping
Wang, W., Shen, J., & Ling, H. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2840724 · Cited by 348
CCNet: Criss-Cross Attention for Semantic Segmentation
Huang, Z., Wang, X., Wei, Y., Huang, L., Shi, H., Liu, W., & Huang, T. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.11721 · Cited by 338
Progressively Complementarity-Aware Fusion Network for RGB-D Salient Object Detection
Chen, H., & Li, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00322 · Cited by 336
Three-Stream Attention-Aware Network for RGB-D Salient Object Detection
Chen, H., & Li, Y. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2891104 · Cited by 324
Adaptive Fusion for RGB-D Salient Object Detection
Wang, N., & Gong, X. (2019) · IEEE Access · doi:10.1109/access.2019.2913107 · Cited by 258
Song, H., Liu, Z., Du, H., Sun, G., Le Meur, O., & Ren, T. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2711277 · Cited by 244
Cong, R., Lei, J., Zhang, C., Huang, Q., Cao, X., & Hou, C. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2557347 · Cited by 235
Local Background Enclosure for RGB-D Salient Object Detection
Feng, D., Barnes, N., You, S., & McCarthy, C. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.257 · Cited by 216
RGBD salient object detection via deep fusion
Qu, L., He, S., Zhang, J., Tian, J., Tang, Y., & Yang, Q. (2017) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 195
Salient object detection for RGB-D image by single stream recurrent convolution neural network
Liu, Z., Shi, S., Duan, Q., Zhang, W., & Zhao, P. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.07.012 · Cited by 173
Depth really Matters: Improving Visual Salient Region Detection with Depth
Desingh, K., K, M. K., Rajan, D., & Jawahar, C. (2013) · Procedings of the British Machine Vision Conference 2013 · doi:10.5244/c.27.98 · Cited by 171
A Three-Pathway Psychobiological Framework of Salient Object Detection Using Stereoscopic Technology
Li, G., & Zhu, C. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.355 · Cited by 171
Multi-modal Attention Network Learning for Semantic Source Code Retrieval
Wan, Y., Shu, J., Sui, Y., Xu, G., Zhao, Z., Wu, J., & Yu, P. (2019) · 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00012 · Cited by 161
Self-Supervised Moving Vehicle Tracking With Stereo Sound
Gan, C., Zhao, H., Chen, P., Cox, D., & Torralba, A. B. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00715 · Cited by 157
Rother, C., Bordeaux, L., Hamadi, Y., & Blake, A. (2006) · ACM Transactions on Graphics · doi:10.1145/1141911.1141965 · Cited by 152
Discovering Class-Specific Pixels for Weakly-Supervised Semantic Segmentation
Chaudhry, A., Dokania, P. K., & Torr, P. H. S. (2017) · British Machine Vision Conference (BMVC) · doi:10.5244/c.31.20 · Cited by 151
Salient object detection for RGB-D image via saliency evolution
Guo, J., Ren, T.-W., & Bei, J. (2016) · IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2016.7552907 · Cited by 138
Ciptadi, A., Hermans, T., & Rehg, J. M. (2013) · British Machine Vision Conference (BMVC) · doi:10.5244/c.27.112 · Cited by 124
Gan, C., Li, Y., Li, H., Sun, C., & Gong, B. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.201 · Cited by 116 · cited 1×
“In the future, our proposed S 2 MA module can also be used for other multi-modal tasks, such as video saliency detection [15], visual questions and segmentation answers [20], and audio-visual tasks [21].”
MethodShigematsu, R., Feng, D. D., You, S., & Barnes, N. M. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.323 · Cited by 88
Cited by
Liu, N., Zhang, N., Wan, K., Shao, L., & Han, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00468 · Cited by 549
Self-citationSalient Object Detection via Integrity Learning
Zhuge, M., Fan, D.-P., Liu, N., Zhang, D., Xu, D., & Shao, L. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3179526 · Cited by 403
Self-citationSwinNet: Swin Transformer Drives Edge-Aware RGB-D and RGB-T Salient Object Detection
Liu, Z., Tan, Y., He, Q., & Xiao, Y. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3127149 · Cited by 384
Hierarchical Alternate Interaction Network for RGB-D Salient Object Detection
Li, G., Liu, Z., Chen, M., Bai, Z., Lin, W., & Ling, H. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3062689 · Cited by 341
ECFFNet: Effective and Consistent Feature Fusion Network for RGB-T Salient Object Detection
Zhou, W., Guo, Q., Lei, J., Yu, L., & Hwang, J.-N. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3077058 · Cited by 340
Metadata
- Authors
- Liu, N., Zhang, N., & Han, J.
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 13753-13762
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3034320133
- Cited by
- 335
- References
- 61
- FWCI
- 17.19
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.