The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Multi-modal fusion network with multi-scale multi-path and cross-modal interactions for RGB-D salient object detection
Abstract
Abstract Paired RGB and depth images are becoming popular multi-modal data adopted in computer vision tasks. Traditional methods based on Convolutional Neural Networks (CNNs) typically fuse RGB and depth by combining their deep representations in a late stage with only one path, which can be ambiguous and insufficient for fusing large amounts of cross-modal data. To address this issue, we propose a novel multi-scale multi-path fusion network with cross-modal interactions (MMCI), in which the traditional two-stream fusion architecture with single fusion path is advanced by diversifying the fusion path to a global reasoning one and another local capturing one and meanwhile introducing cross-modal interactions in multiple layers. Compared to traditional two-stream architectures, the MMCI net is able to supply more adaptive and flexible fusion flows, thus easing the optimization and enabling sufficient and efficient fusion. Concurrently, the MMCI net is equipped with multi-scale perception ability (i.e., simultaneously global and local contextual reasoning). We take RGB-D saliency detection as an example task. Extensive experiments on three benchmark datasets show the improvement of the proposed MMCI net over other state-of-the-art methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1993.341010 · Cited by 14,265
Proceedings of IEEE International Conference on Computer Vision
(1995) · Proceedings of IEEE International Conference on Computer Vision · doi:10.1109/iccv.1995.466933 · Cited by 12,860
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465
A model of saliency-based visual attention for rapid scene analysis
Itti, L., Koch, C., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,271
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 3,752
Learning to detect natural image boundaries using local brightness, color, and texture cues
Martin, D. R., Fowlkes, C. C., & Malik, J. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.1273918 · Cited by 2,420
Neural Networks: Tricks of the Trade: Second Edition
(2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8 · Cited by 2,116
Stochastic Gradient Descent Tricks
Bottou, L. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_25 · Cited by 1,929
Building high-level features using large scale unsupervised learning
Le, Q. V. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639343 · Cited by 1,695
Learning Rich Features from RGB-D Images for Object Detection and Segmentation
Gupta, S., Girshick, R., Arbeláez, P., & Malik, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_23 · Cited by 1,528
British Machine Vision Conference (BMVC)
Damen, D., & Hogg, D. (2007) · Cited by 1,213
Angelucci (2002) · doi:10.1146/annurev.ne.08.030185.002203 · Cited by 1,170
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012
Müller, A., Sejdinović, D., & Piechocki, R. J. (2012) · Cited by 1,027
2014 IEEE International Conference on Robotics and Automation (ICRA)
(2014) · doi:10.1109/icra18334.2014 · Cited by 998
2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2014)
Wada, T. (2015) · Journal of the Robotics Society of Japan · doi:10.7210/jrsj.33.100 · Cited by 958
Srivastava, R. K., Greff, K., & Schmidhuber, J. (2015) · arXiv · doi:10.48550/arxiv.1505.00387 · Cited by 912
Proceedings of the 22nd ACM international conference on Multimedia
(2014) · doi:10.1145/2647868 · Cited by 725
2015 IEEE International Conference on Image Processing (ICIP)
(2015) · doi:10.1109/icip30522.2015 · Cited by 717
RGBD Salient Object Detection: A Benchmark and Algorithms
Peng, H., Li, B., Xiong, W., Hu, W., & Ji, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_7 · Cited by 668
Multimodal deep learning for robust RGB-D object recognition
Eitel, A., Springenberg, J. T., Spinello, L., Riedmiller, M., & Burgard, W. (2015) · 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353446 · Cited by 631
Advanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey
Han, J., Zhang, D., Cheng, G., Liu, N., & Xu, D. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2749125 · Cited by 621
Depth saliency based on anisotropic center-surround difference
Ju, R., Ge, L., Geng, W., Ren, T., & Wu, G. (2014) · 2014 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2014.7025222 · Cited by 523
Co-Saliency Detection via a Self-Paced Multiple-Instance Learning Framework
Zhang, D., Meng, D., & Han, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2567393 · Cited by 508
Leveraging stereopsis for saliency analysis
Niu, Y., Geng, Y., Li, X., & Liu, F. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247708 · Cited by 479
Background Prior-Based Salient Object Detection via Deep Reconstruction Residual
Han, J., Zhang, D., Hu, X., Guo, L., Ren, J., & Wu, F. (2015) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2014.2381471 · Cited by 444
CNNs-Based RGB-D Saliency Detection via Cross-View Transfer and Multiview Fusion
Han, J., Chen, H., Liu, N., Yan, C., & Li, X. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2761775 · Cited by 423
Self-citationUnsupervised Feature Learning for RGB-D Based Object Recognition
Bo, L., Ren, X., & Fox, D. (2013) · Springer Tracts in Advanced Robotics · doi:10.1007/978-3-319-00065-7_27 · Cited by 360
RGBD Salient Object Detection via Deep Fusion
Qu, L., He, S., Zhang, J., Tian, J., Tang, Y., & Yang, Q. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2682981 · Cited by 360
Depth Enhanced Saliency Detection Method
Cheng, Y., Fu, H., Wei, X., Xiao, J., & Cao, X. (2014) · Proceedings of International Conference on Internet Multimedia Computing and Service · doi:10.1145/2632856.2632866 · Cited by 350
Schwarz, M., Schulz, H., & Behnke, S. (2015) · 2015 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2015.7139363 · Cited by 312
Song, H., Liu, Z., Du, H., Sun, G., Le Meur, O., & Ren, T. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2711277 · Cited by 244
Cong, R., Lei, J., Zhang, C., Huang, Q., Cao, X., & Hou, C. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2557347 · Cited by 235
Exploiting global priors for RGB-D saliency detection
Ren, J., Gong, X., Yu, L., Zhou, W., & Yang, M. Y. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301391 · Cited by 217
Local Background Enclosure for RGB-D Salient Object Detection
Feng, D., Barnes, N., You, S., & McCarthy, C. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.257 · Cited by 216
RGBD salient object detection via deep fusion
Qu, L., He, S., Zhang, J., Tian, J., Tang, Y., & Yang, Q. (2017) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 195
Saliency Detection for Stereoscopic Images
Fang, Y., Wang, J., Narwaria, M., Callet, P. L., & Lin, W. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2305100 · Cited by 171
Depth really Matters: Improving Visual Salient Region Detection with Depth
Desingh, K., K, M. K., Rajan, D., & Jawahar, C. (2013) · Procedings of the British Machine Vision Conference 2013 · doi:10.5244/c.27.98 · Cited by 171
Human action recognition in RGB-D videos using motion sequence information and deep learning
Ijjina, E. P., & Chalavadi, K. M. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.07.013 · Cited by 168
Bottom-Up Saliency Detection Model Based on Human Visual Sensitivity and Amplitude Spectrum
Fang, Y., Lin, W., Lee, B.-S., Lau, C.-T., Chen, Z., & Lin, C.-W. (2012) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2011.2169775 · Cited by 162
Category-Independent Object-Level Saliency Detection
Jia, Y., & Han, M. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.221 · Cited by 138
Khatib, O., Kumar, V., & Sukhatme, G. (2013) · Springer tracts in advanced robotics · doi:10.1007/978-3-642-28572-1 · Cited by 129
Discriminative Multi-modal Feature Fusion for RGBD Indoor Scene Recognition
Zhu, H., Weibel, J.-B., & Lu, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.324 · Cited by 126
Ciptadi, A., Hermans, T., & Rehg, J. M. (2013) · British Machine Vision Conference (BMVC) · doi:10.5244/c.27.112 · Cited by 124
MMSS: Multi-modal Sharable and Specific Feature Learning for RGB-D Object Recognition
Wang, A., Cai, J., Lu, J., & Cham, T. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.134 · Cited by 108
Multi-modal deep feature learning for RGB-D object detection
Xu, X., Li, Y., Wu, G., & Luo, J. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.07.026 · Cited by 108
Self-citationSalient region detection for stereoscopic images
Fan, X., Liu, Z., & Sun, G. (2014) · doi:10.1109/icdsp.2014.6900706 · Cited by 82
Multi-modal uniform deep learning for RGB-D person re-identification
Ren, L., Lu, J., Feng, J., & Zhou, J. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.06.037 · Cited by 59
Visual Attention Modeling for Stereoscopic Video: A Benchmark and Computational Model
Fang, Y., Zhang, C., Li, J., Lei, J., Silva, M. P. D., & Callet, P. L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2721112 · Cited by 57
Motion segment decomposition of RGB-D sequences for human behavior understanding
Devanne, M., Berretti, S., Pala, P., Wannous, H., Daoudi, M., & Bimbo, A. D. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.07.041 · Cited by 51
Cited by
Rethinking RGB-D Salient Object Detection: Models, Data Sets, and Large-Scale Benchmarks
Fan, D.-P., Lin, Z., Zhang, Z., Zhu, M., & Cheng, M.-M. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2996406 · Cited by 702
Collaborative Multi-Robot Search and Rescue: Planning, Coordination, Perception, and Active Vision
Queralta, J. P., Taipalmaa, J., Can Pullinen, B., Sarker, V. K., Nguyen Gia, T., Tenhunen, H., Gabbouj, M., Raitoharju, J., & Westerlund, T. (2020) · IEEE Access · doi:10.1109/access.2020.3030190 · Cited by 526
Depth-Induced Multi-Scale Recurrent Attention Network for Saliency Detection
Piao, Y., Ji, W., Li, J., Zhang, M., & Lu, H. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00735 · Cited by 497
Du, G., Wang, K., Lian, S., & Zhao, K. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-020-09888-5 · Cited by 475
UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders
Zhang, J., Fan, D.-P., Dai, Y., Anwar, S., Saleh, F. S., Zhang, T., & Barnes, N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00861 · Cited by 423
Metadata
- Authors
- Chen, H., Li, Y., & Su, D.
- Venue
- Pattern Recognition
- Publisher
- Elsevier BV
- Published
- 2019-02
- Volume
- 86
- Pages
- 376-385
- ISSN
- 0031-3203 (ISSN-L)
- OpenAlex
- W2887522866
- Cited by
- 368
- References
- 54
- FWCI
- 21.81
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.