The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
DATFuse: Infrared and Visible Image Fusion via Dual Attention Transformer
Abstract
The fusion of infrared and visible images aims to generate a composite image that can simultaneously contain the thermal radiation information of an infrared image and the plentiful texture details of a visible image to detect targets under various weather conditions with a high spatial resolution of scenes. Previous deep fusion models were generally based on convolutional operations, resulting in a limited ability to represent long-range context information. In this paper, we propose a novel end-to-end model for infrared and visible image fusion via a dual attention Transformer termed DATFuse. To accurately examine the significant areas of the source images, a dual attention residual module (DARM) is designed for important feature extraction. To further model long-range dependencies, a Transformer module (TRM) is devised for global complementary information preservation. Moreover, a loss function that consists of three terms, namely, pixel loss, gradient loss, and structural loss, is designed to train the proposed model in an unsupervised manner. This can avoid manually designing complicated activity-level measurement and fusion strategies in traditional image fusion methods. Extensive experiments on public datasets reveal that our DATFuse outperforms other representative state-of-the-art approaches in both qualitative and quantitative assessments. The proposed model is also extended to address other infrared and visible image fusion tasks without fine-tuning, and the promising results demonstrate that it has good generalization ability. The source code is available athttps://github.com/tthinking/DATFuse.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., Torr, P. H. S., et al. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00681 · Cited by 4,008
FusionGAN: A generative adversarial network for infrared and visible image fusion
Ma, J., Yu, W., Liang, P., Li, C., & Jiang, J. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.09.004 · Cited by 2,267
Deep Learning for Person Re-Identification: A Survey and Outlook
Ye, M., Shen, J., Lin, G., Xiang, T., Shao, L., & Hoi, S. C. H. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3054775 · Cited by 2,259
U2Fusion: A Unified Unsupervised Image Fusion Network
Xu, H., Ma, J., Jiang, J., Guo, X., & Ling, H. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3012548 · Cited by 2,127
DenseFuse: A Fusion Approach to Infrared and Visible Images
Li, H., & Wu, X.-J. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2887342 · Cited by 2,032
Deep Learning for Image Super-Resolution: A Survey
Wang, Z., Chen, J., & Hoi, S. C. H. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2982166 · Cited by 1,969
Image Fusion With Guided Filtering
Li, S., Kang, X., & Hu, J. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2244222 · Cited by 1,781
A general framework for image fusion based on multi-scale transform and sparse representation
Liu, Y., Liu, S., & Wang, Z. (2015) · Information Fusion · doi:10.1016/j.inffus.2014.09.004 · Cited by 1,431
Self-citationA wavelet-based image fusion tutorial
Pajares, G., & Manuel de la Cruz, J. (2004) · Pattern Recognition · doi:10.1016/j.patcog.2004.03.010 · Cited by 1,328
Pixel-level image fusion: A survey of the state of the art
Li, S., Kang, X., Fang, L., Hu, J., & Yin, H. (2017) · Information Fusion · doi:10.1016/j.inffus.2016.05.004 · Cited by 1,276
RFN-Nest: An end-to-end residual fusion network for infrared and visible images
Li, H., Wu, X.-J., & Kittler, J. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.02.023 · Cited by 1,173
PIAFusion: A progressive infrared and visible image fusion network based on illumination aware
Tang, L., Yuan, J., Zhang, H., Jiang, X., & Ma, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.007 · Cited by 1,161
Image Fusion With Convolutional Sparse Representation
Liu, Y., Chen, X., Ward, R. K., & Jane Wang, Z. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2618776 · Cited by 986
Self-citationMa, J., Zhou, Z., Wang, B., & Zong, H. (2017) · Infrared Physics & Technology · doi:10.1016/j.infrared.2017.02.005 · Cited by 773
Deep learning for pixel-level image fusion: Recent advances and future prospects
Liu, Y., Chen, X., Wang, Z., Wang, Z. J., Ward, R. K., & Wang, X. (2018) · Information Fusion · doi:10.1016/j.inffus.2017.10.007 · Cited by 736
Self-citationMDLatLRR: A Novel Decomposition Method for Infrared and Visible Image Fusion
Li, H., Wu, X.-J., & Kittler, J. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2975984 · Cited by 627
Ma, J., Zhang, H., Shao, Z., Liang, P., & Xu, H. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2020.3038013 · Cited by 618
Deep Visual Attention Prediction
Wang, W., & Shen, J. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2787612 · Cited by 585
Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-identification
Ye, M., Shen, J., J. Crandall, D., Shao, L., & Luo, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58520-4_14 · Cited by 523
Comments on ‘Information measure for performance of image fusion’
Hossny, M., Nahavandi, S., & Creighton, D. (2008) · Electronics Letters · doi:10.1049/el:20081754 · Cited by 505
YDTR: Infrared and Visible Image Fusion via Y-Shape Dynamic Transformer
Tang, W., He, F., & Liu, Y. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3192661 · Cited by 485
Self-citationInfrared and visible image fusion with convolutional neural networks
Liu, Y., Chen, X., Cheng, J., Peng, H., & Wang, Z. (2018) · International Journal of Wavelets, Multiresolution and Information Processing · doi:10.1142/s0219691318500182 · Cited by 479
Self-citationMulti-spectral SIFT for scene category recognition
Brown, M., & Susstrunk, S. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995637 · Cited by 469
A novel similarity based quality metric for image fusion
Yang, C., Zhang, J.-Q., Wang, X.-R., & Liu, X. (2008) · Information Fusion · doi:10.1016/j.inffus.2006.09.001 · Cited by 468
MATR: Multimodal Medical Image Fusion via Multiscale Adaptive Transformer
Tang, W., He, F., Liu, Y., & Duan, Y. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3193288 · Cited by 442
Self-citationLearning a Deep Multi-Scale Feature Ensemble and an Edge-Attention Guidance for Image Fusion
Liu, J., Fan, X., Jiang, J., Liu, R., & Luo, Z. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3056725 · Cited by 431
A novel multi-modality image fusion method based on image decomposition and sparse representation
Zhu, Z., Yin, H., Chai, Y., Li, Y., & Qi, G. (2018) · Information Sciences · doi:10.1016/j.ins.2017.09.010 · Cited by 408
Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform
Yang, L., Guo, B. L., & Ni, W. (2008) · Neurocomputing · doi:10.1016/j.neucom.2008.02.025 · Cited by 379
Pixel-based and region-based image fusion schemes using ICA bases
Mitianoudis, N., & Stathaki, T. (2007) · Information Fusion · doi:10.1016/j.inffus.2005.09.001 · Cited by 343
ECFFNet: Effective and Consistent Feature Fusion Network for RGB-T Salient Object Detection
Zhou, W., Guo, Q., Lei, J., Yu, L., & Hwang, J.-N. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3077058 · Cited by 340
Early Convolutions Help Transformers See Better
Xiao, T., Singh, M., Mintun, E., Darrell, T. J., Dollár, P., & Girshick, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.14881 · Cited by 339
Visible-Infrared Person Re-Identification via Homogeneous Augmented Tri-Modal Learning
Ye, M., Shen, J., & Shao, L. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.3001665 · Cited by 305
Classification Saliency-Based Rule for Visible and Infrared Image Fusion
Xu, H., Zhang, H., & Ma, J. (2021) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2021.3100986 · Cited by 294
A human perception inspired quality metric for image fusion based on regional information
Chen, H., & Varshney, P. K. (2007) · Information Fusion · doi:10.1016/j.inffus.2005.10.001 · Cited by 286
Spatiotemporal Co-Attention Recurrent Neural Networks for Human-Skeleton Motion Prediction
Shu, X., Zhang, L., Qi, G.-J., Liu, W., & Tang, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3050918 · Cited by 276
Efficient and Model-Based Infrared and Visible Image Fusion via Algorithm Unrolling
Zhao, Z., Xu, S., Zhang, J., Liang, C., Zhang, C., & Liu, J. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3075745 · Cited by 242
Coherence Constrained Graph LSTM for Group Activity Recognition
Tang, J., Shu, X., Yan, R., & Zhang, L. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2928540 · Cited by 238
Image fusion metric based on mutual information and Tsallis entropy
Cvejic, N., Canagarajah, C. N., & Bull, D. R. (2006) · Electronics Letters · doi:10.1049/el:20060693 · Cited by 230
Zhao, J., & Liu, Z. (2007) · Cited by 219
Vs, V., Jose Valanarasu, J. M., Oza, P., & Patel, V. M. (2022) · 2022 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip46576.2022.9897280 · Cited by 205
Dictionary learning method for joint sparse representation-based image fusion
Zhang, Q., Fu, Y., Li, H., & Zou, J. (2013) · Optical Engineering · doi:10.1117/1.oe.52.5.057006 · Cited by 182
Infrared and Visible Image Fusion via Texture Conditional Generative Adversarial Network
Yang, Y., Liu, J., Huang, S., Wan, W., Wen, W., & Guan, J. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3054584 · Cited by 176
Exposure Fusion Using Boosting Laplacian Pyramid
Shen, J., Zhao, Y., Yan, S., & Li, X. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2290435 · Cited by 175
Kong, W., Lei, Y., & Zhao, H. (2014) · Infrared Physics & Technology · doi:10.1016/j.infrared.2014.07.019 · Cited by 173
UNFusion: A Unified Multi-Scale Densely Connected Network for Infrared and Visible Image Fusion
Wang, Z., Wang, J., Wu, Y., Xu, J., & Zhang, X. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3109895 · Cited by 169
Visual Object Tracking by Hierarchical Attention Siamese Network
Shen, J., Tang, X., Dong, X., & Shao, L. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2936503 · Cited by 167
Revisiting Feature Fusion for RGB-T Salient Object Detection
Zhang, Q., Xiao, T., Huang, N., Zhang, D., & Han, J. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3014663 · Cited by 164
The multiscale directional bilateral filter and its application to multisensor image fusion
Hu, J., & Li, S. (2012) · Information Fusion · doi:10.1016/j.inffus.2011.01.002 · Cited by 163
SwinFuse: A Residual Swin Transformer Fusion Network for Infrared and Visible Images
Wang, Z., Chen, Y., Shao, W., Li, H., & Zhang, L. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3191664 · Cited by 162
Multilingual evaluation of pre-processing for BERT-based sentiment analysis of tweets
Pota, M., Ventura, M., Fujita, H., & Esposito, M. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115119 · Cited by 140
Yin, J., Shen, J., Gao, X., Crandall, D., & Yang, R. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3125981 · Cited by 102
PPT Fusion: Pyramid Patch Transformerfor a Case Study in Image FusionPreprint
Fu, Y., Xu, T., Wu, X., & Kittler, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.13967 · Cited by 96
Learning Local-Global Multi-Graph Descriptors for RGB-T Object Tracking
Li, C., Zhu, C., Zhang, J., Luo, B., Wu, X., & Tang, J. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2874312 · Cited by 84
Multi-Source Information Exchange Encoding With PCNN for Medical Image Fusion
Nie, R., Cao, J., Zhou, D., et al. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.2998696 · Cited by 77
Person Re-Identification by Context-Aware Part Attention and Multi-Head Collaborative Learning
Wu, D., Ye, M., Lin, G., Gao, X., & Shen, J. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3075894 · Cited by 75
BERT syntactic transfer: A computational experiment on Italian, French and English languages
Guarasci, R., Silvestri, S., Pietro, G. D., Fujita, H., & Esposito, M. (2021) · Computer Speech & Language · doi:10.1016/j.csl.2021.101261 · Cited by 70
Effective Pan-Sharpening by Multiscale Invertible Neural Network and Heterogeneous Task Distilling
Zhou, M., Huang, J., Fu, X., Zhao, F., & Hong, D. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3199210 · Cited by 61
A New Multi-Focus Image Fusion Algorithm and Its Efficient Implementation
Liu, S., Chen, J., & Rahardja, S. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2901809 · Cited by 56
Green Fluorescent Protein and Phase Contrast Image Fusion Via Detail Preserving Cross Network
Tang, W., Liu, Y., Cheng, J., Li, C., & Chen, X. (2021) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2021.3083965 · Cited by 47
Self-citationEntropy-Based Image Fusion with Joint Sparse Representation and Rolling Guidance Filter
Yudan, L., Yang, X., Zhang, R., Albertini, M. K., Çelik, T., & Jeon, G. (2020) · Entropy · doi:10.3390/e22010118 · Cited by 44
Visible-Infrared Image Fusion Based on Early Visual Information Processing Mechanisms
Tan, M.-J., Gao, S., Xu, W., & Han, S. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3047935 · Cited by 34
Pan-Sharpening Based on Transformer With Redundancy Reduction
Zhang, K., Li, Z., Zhang, F., Wan, W., & Sun, J. (2022) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2022.3186985 · Cited by 34
Infrared and visible image fusion via octave Gaussian pyramid framework
Yan, L., Hao, Q., Cao, J., et al. (2021) · Scientific Reports · doi:10.1038/s41598-020-80189-1 · Cited by 32
Green Fluorescent Protein and Phase-Contrast Image Fusion via Generative Adversarial Networks
Tang, W., Liu, Y., Zhang, C., Cheng, J., Peng, H., & Chen, X. (2019) · Computational and Mathematical Methods in Medicine · doi:10.1155/2019/5450373 · Cited by 31
Self-citationNovel infrared and visible image fusion method based on independent component analysis
Yin, L., Wang, F., Luo, X., & Liu, F. (2014) · Frontiers of Computer Science · doi:10.1007/s11704-014-2328-2 · Cited by 26
Tang, W., Liu, Y., Cheng, J., Li, C., Peng, H., & Chen, X. (2020) · Microscopy Research and Technique · doi:10.1002/jemt.23514 · Cited by 24
Self-citationAttentive Feature Refinement Network for Single Rainy Image Restoration
Wang, G., Sun, C., & Sowmya, A. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3064229 · Cited by 20
FUSION OF INFRARED AND VISIBLE IMAGES BASED ON THE SECOND GENERATION CURVELET TRANSFORM
Fu, M., & Zhao, C. (2009) · JOURNAL OF INFRARED AND MILLIMETER WAVES · doi:10.3724/sp.j.1010.2009.00254 · Cited by 17
Green Fluorescent Protein and Phase Contrast Image Fusion via Dual Attention Residual Network
Tang, W., Wang, L., & Liu, Y. (2021) · IEEE International Conference on Medical Imaging Physics and Engineering (ICMIPE) · doi:10.1109/icmipe53131.2021.9698938 · Cited by 10
Self-citation
Cited by
CrossFuse: A novel cross attention mechanism based infrared and visible image fusion approach
Li, H., & Wu, X.-J. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102147 · Cited by 460
Metadata
- Authors
- Tang, W., He, F., Liu, Y., Duan, Y., & Si, T.
- Venue
- IEEE Transactions on Circuits and Systems for Video Technology
- Published
- 2023-07
- Volume
- 33
- Issue
- 7
- Pages
- 3159-3172
- ISSN
- 1051-8215 (ISSN-L)
- 1558-2205
- OpenAlex
- W4313555022
- Cited by
- 571
- References
- 77
- FWCI
- 63.06
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.