The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
CrossFuse: A novel cross attention mechanism based infrared and visible image fusion approach
Open accessAbstract
Multimodal visual information fusion aims to integrate the multi-sensor data into a single image which contains more complementary information and less redundant features. However the complementary information is hard to extract, especially for infrared and visible images which contain big similarity gap between these two modalities. The common cross attention modules only consider the correlation, on the contrary, image fusion tasks need focus on complementarity (uncorrelation). Hence, in this paper, a novel cross attention mechanism (CAM) is proposed to enhance the complementary information. Furthermore, a two-stage training strategy based fusion scheme is presented to generate the fused images. For the first stage, two auto-encoder networks with same architecture are trained for each modality. Then, with the fixed encoders, the CAM and a decoder are trained in the second stage. With the trained CAM, features extracted from two modalities are integrated into one fused feature in which the complementary information is enhanced and the redundant features are reduced. Finally, the fused image can be generated by the trained decoder. The experimental results illustrate that our proposed fusion method obtains the SOTA fusion performance compared with the existing fusion networks. The codes are available at https://github.com/hli1221/CrossFuse
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
CCNet: Criss-Cross Attention for Semantic Segmentation
Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., & Liu, W. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00069 · Cited by 3,045
GPT-3: Its Nature, Scope, Limits, and Consequences
Floridi, L., & Chiriatti, M. (2020) · Minds and Machines · doi:10.1007/s11023-020-09548-1 · Cited by 2,519
FusionGAN: A generative adversarial network for infrared and visible image fusion
Ma, J., Yu, W., Liang, P., Li, C., & Jiang, J. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.09.004 · Cited by 2,267
U2Fusion: A Unified Unsupervised Image Fusion Network
Xu, H., Ma, J., Jiang, J., Guo, X., & Ling, H. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3012548 · Cited by 2,127
DenseFuse: A Fusion Approach to Infrared and Visible Images
Li, H., & Wu, X.-J. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2887342 · Cited by 2,032
Self-citationImage Fusion With Guided Filtering
Li, S., Kang, X., & Hu, J. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2244222 · Cited by 1,781
In-fibre Bragg grating sensors
Rao, Y.-J. (1997) · Measurement Science and Technology · doi:10.1088/0957-0233/8/4/002 · Cited by 1,606
IFCNN: A general image fusion framework based on convolutional neural network
Zhang, Y., Liu, Y., Sun, P., Yan, H., Zhao, X., & Zhang, L. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.07.011 · Cited by 1,550
SwinFusion: Cross-domain Long-range Learning for General Image Fusion via Swin Transformer
Ma, J., Tang, L., Fan, F., Huang, J., Mei, X., & Ma, Y. (2022) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2022.105686 · Cited by 1,526
Information measure for performance of image fusion
Qu, G., Zhang, D., & Yan, P. (2002) · Electronics Letters · doi:10.1049/el:20020212 · Cited by 1,427
Liu, J., Fan, X., Huang, Z., Wu, G., Liu, R., Zhong, W., & Luo, Z. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00571 · Cited by 1,336
A wavelet-based image fusion tutorial
Pajares, G., & Manuel de la Cruz, J. (2004) · Pattern Recognition · doi:10.1016/j.patcog.2004.03.010 · Cited by 1,328
Multi-focus image fusion with a deep convolutional neural network
Liu, Y., Chen, X., Peng, H., & Wang, Z. (2017) · Information Fusion · doi:10.1016/j.inffus.2016.12.001 · Cited by 1,327
Multispectral pedestrian detection: Benchmark dataset and baseline
Hwang, S., Park, J., Kim, N., Choi, Y., & Kweon, I. S. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298706 · Cited by 1,249
RFN-Nest: An end-to-end residual fusion network for infrared and visible images
Li, H., Wu, X.-J., & Kittler, J. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.02.023 · Cited by 1,173
Self-citation2010 20th International Conference on Pattern Recognition
(2010) · doi:10.1109/icpr16097.2010 · Cited by 1,171
Image Fusion With Convolutional Sparse Representation
Liu, Y., Chen, X., Ward, R. K., & Jane Wang, Z. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2618776 · Cited by 986
Infrared and Visible Image Fusion via Decoupling Network
Wang, X., Guan, Z., Yu, S., Cao, J., & Li, Y. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3216413 · Cited by 969
CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion
Zhao, Z., Bai, H., Zhang, J., Zhang, Y., Xu, S., Lin, Z., Timofte, R., & Van Gool, L. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00572 · Cited by 948
Image fusion meets deep learning: A survey and perspective
Zhang, H., Xu, H., Tian, X., Jiang, J., & Ma, J. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.06.008 · Cited by 834
2007 IEEE International Conference on Image Processing
(2007) · doi:10.1109/icip10338.2007 · Cited by 791
Assessment of image fusion procedures using entropy, image quality, and multispectral classification
Van Aardt, J. (2008) · Journal of Applied Remote Sensing · doi:10.1117/1.2945910 · Cited by 747
Ma, J., Zhao, J., Jiang, J., Zhou, H., & Guo, X. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-018-1117-z · Cited by 601
DATFuse: Infrared and Visible Image Fusion via Dual Attention Transformer
Tang, W., He, F., Liu, Y., Duan, Y., & Si, T. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3234340 · Cited by 571
An effective CNN and Transformer complementary network for medical image segmentation
Yuan, F., Zhang, Z., & Fang, Z. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2022.109228 · Cited by 539
A new image quality metric for image fusion: The sum of the correlations of differences
Aslantas, V., & Bendes, E. (2015) · AEU - International Journal of Electronics and Communications · doi:10.1016/j.aeue.2015.09.004 · Cited by 507
LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images
Li, H., Xu, T., Wu, X.-J., Lu, J., & Kittler, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3268209 · Cited by 493
Self-citationSuperFusion: A Versatile Image Registration and Fusion Network with Semantic Awareness
Tang, L., Deng, Y., Ma, Y., Huang, J., & Ma, J. (2022) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2022.106082 · Cited by 488
YDTR: Infrared and Visible Image Fusion via Y-Shape Dynamic Transformer
Tang, W., He, F., & Liu, Y. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3192661 · Cited by 485
A non-reference image fusion metric based on mutual information of image features
Haghighat, M. B. A., Aghagolzadeh, A., & Seyedarabi, H. (2011) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2011.07.012 · Cited by 473
MATR: Multimodal Medical Image Fusion via Multiscale Adaptive Transformer
Tang, W., He, F., Liu, Y., & Duan, Y. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3193288 · Cited by 442
DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion
Zhao, Z., Bai, H., Zhu, Y., Zhang, J., Xu, S., Zhang, Y., Zhang, K., Meng, D., Timofte, R., & Van Gool, L. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00742 · Cited by 431
Edgenext: Efficiently amalgamated CNN-transformer architecture for mobile vision applications
Maaz, M., Shaker, A., Cholakkal, H., Khan, S., Zamir, S. W., Anwer, R. M., & Shahbaz Khan, F. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-25082-8_1 · Cited by 407
Infrared and visible image fusion with ResNet and zero-phase component analysis
Li, H., Wu, X.-J., & Durrani, T. S. (2019) · Infrared Physics & Technology · doi:10.1016/j.infrared.2019.103039 · Cited by 360
Self-citationMulti-focus image fusion: A Survey of the state of the art
Liu, Y., Wang, L., Cheng, J., Li, C., & Chen, X. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.013 · Cited by 320
Multispectral and hyperspectral image fusion in remote sensing: A survey
Vivone, G. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.08.032 · Cited by 288
Dual Cross-Attention Learning for Fine-Grained Visual Categorization and Object Re-Identification
Zhu, H., Ke, W., Li, D., Liu, J., Tian, L., & Shan, Y. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00465 · Cited by 277
GAN review: Models and medical image fusion applications
Zhou, T., Li, Q., Lu, H., Cheng, Q., & Zhang, X. (2023) · Information Fusion · doi:10.1016/j.inffus.2022.10.017 · Cited by 268
Efficient and Model-Based Infrared and Visible Image Fusion via Algorithm Unrolling
Zhao, Z., Xu, S., Zhang, J., Liang, C., Zhang, C., & Liu, J. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3075745 · Cited by 242
Wang, D., Liu, J., Liu, R., & Fan, X. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101828 · Cited by 223
1991 IEEE International Conference on Consumer Electronics
(1991) · doi:10.1109/icce.1991 · Cited by 212
Deep Learning-based Multi-focus Image Fusion: A Survey and A Comparative Study
Zhang, X. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3078906 · Cited by 206
Qu, L., Liu, S., Wang, M., & Song, Z. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20109 · Cited by 179
Review of pixel-level remote sensing image fusion based on deep learning
Wang, Z., Ma, Y., & Zhang, Y. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.09.008 · Cited by 166
Bayesian fusion for infrared and visible images
Zhao, Z., Xu, S., Zhang, C., Liu, J., & Zhang, J. (2020) · Signal Processing · doi:10.1016/j.sigpro.2020.107734 · Cited by 164
Rao, Y., Wu, D., Han, M., Wang, T., Yang, Y., Lei, T., Zhou, C., Bai, H., & Xing, L. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.12.007 · Cited by 161
A Joint Cross-Attention Model for Audio-Visual Fusion in Dimensional Emotion Recognition
Praveen, R. G., de Melo, W. C., Ullah, N., Aslam, M. H., Zeeshan, O., Denorme, T., Pedersoli, M., Koerich, A. L., Bacon, S., Cardinal, P., et al. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00278 · Cited by 159
TCCFusion: An infrared and visible image fusion method based on transformer and cross correlation
Tang, W., He, F., & Liu, Y. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2022.109295 · Cited by 148
Semantics lead all: Towards unified image registration and fusion from a semantic perspective
Xie, H., Zhang, Y., Qiu, J., Zhai, X., Liu, X., Yang, Y., Zhao, S., Luo, Y., & Zhong, J. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101835 · Cited by 146
Transformer Based Conditional GAN for Multimodal Image Fusion
Zhang, J., Jiao, L., Ma, W., Liu, F., Liu, X., Li, L., Chen, P., & Yang, S. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3243659 · Cited by 116
Multi-feature, multi-modal, and multi-source social event detection: A comprehensive survey
Afyouni, I., Aghbari, Z. A., & Razack, R. A. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.10.013 · Cited by 106
Shape-Former: Bridging CNN and Transformer via ShapeConv for multimodal image matching
Chen, J., Chen, X., Chen, S., Liu, Y., Rao, Y., Yang, Y., Wang, H., & Wu, D. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.10.030 · Cited by 82
A multimodal hyper-fusion transformer for remote sensing image classification
Ma, M., Ma, W., Jiao, L., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.03.005 · Cited by 56
Jha, A., Bose, S., & Banerjee, B. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00629 · Cited by 50
Zhou, Q., Ye, S., Wen, M., Huang, Z., Ding, M., & Zhang, X. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-07635-1 · Cited by 47
MFF-Net: Towards Efficient Monocular Depth Completion With Multi-Modal Feature Fusion
Liu, L., Song, X., Sun, J., Lyu, X., Li, L., Liu, Y., & Zhang, L. (2023) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2023.3234776 · Cited by 42
Yadav, G., & Yadav, D. K. (2023) · Image and Vision Computing · doi:10.1016/j.imavis.2023.104693 · Cited by 25
Deep Cross Spectral Stereo Matching Using Multi-Spectral Image Fusion
Liang, X., & Jung, C. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3155202 · Cited by 18
Metadata
- Authors
- Li, H., & Wu, X.-J.
- Venue
- Information Fusion
- Publisher
- Elsevier BV
- Published
- 2024-03
- Volume
- 103
- Pages
- 102147
- ISSN
- 1566-2535 (ISSN-L)
- OpenAlex
- W4388706355
- Cited by
- 460
- References
- 60
- FWCI
- 50.80
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.