Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption
Open accessThe DOI matches its Crossref record; 183 of the 183 references it cites were found in a registry or an index.
Abstract
Infrared-visible image fusion (IVIF) is a fundamental and critical task in the field of computer vision. Its aim is to integrate the unique characteristics of both infrared and visible spectra into a holistic representation. Since 2018, growing amount and diversity IVIF approaches step into a deep-learning era, encompassing introduced a broad spectrum of networks or loss functions for improving visual enhancement. As research deepens and practical demands grow, several intricate issues like data compatibility, perception accuracy, and efficiency cannot be ignored. Regrettably, there is a lack of recent surveys that comprehensively introduce and organize this expanding domain of knowledge. Given the current rapid development, this paper aims to fill the existing gap by providing a comprehensive survey that covers a wide array of aspects. Initially, we introduce a multi-dimensional framework to elucidate the prevalent learning-based IVIF methodologies, spanning topics from basic visual enhancement strategies to data compatibility, task adaptability, and further extensions. Subsequently, we delve into a profound analysis of these new approaches, offering a detailed lookup table to clarify their core ideas. Last but not the least, We also summarize performance comparisons quantitatively and qualitatively, covering registration, fusion and follow-up high-level tasks. Beyond delving into the technical nuances of these learning-based fusion approaches, we also explore potential future directions and open issues that warrant further exploration by the community.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 1×
“Ma et al. [95] present the SwinFusion, leveraging the swin transformer [96] to construct two kinds of attention modules.”
Self-citationGenerative adversarial networks
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020) · Communications of the ACM · doi:10.1145/3422622 · Cited by 18,384
Restormer: Efficient Transformer for High-Resolution Image Restoration
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., & Yang, M.-H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00564 · Cited by 4,143
DARTS: Differentiable architecture search
Liu, H., Simonyan, K., & Yang, Y. (2018) · arXiv · doi:10.48550/arxiv.1806.09055 · Cited by 3,248
Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures
Wang, Z., & Bovik, A. C. (2009) · IEEE Signal Processing Magazine · doi:10.1109/msp.2008.930649 · Cited by 3,175
FusionGAN: A generative adversarial network for infrared and visible image fusion
Ma, J., Yu, W., Liang, P., Li, C., & Jiang, J. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.09.004 · Cited by 2,267 · cited 2×
“Single discrimination: Ma et al. first propose the FusionGAN method [65], which contains a generator, aiming to preserve the infrared intensities and textural details, and one discriminator, guaranteeing the textural details in the visible images.”
U2Fusion: A Unified Unsupervised Image Fusion Network
Xu, H., Ma, J., Jiang, J., Guo, X., & Ling, H. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3012548 · Cited by 2,127 · cited 1×
“Xu et al. [162] innova-tively use information theory for input weighting, enhancing loss optimization and task unification.”
Image quality measures and their performance
Eskicioglu, A. M., & Fisher, P. S. (1995) · IEEE Transactions on Communications · doi:10.1109/26.477498 · Cited by 2,117 · cited 1×
“Spatial Frequency (SF) [185] evaluates detail and texture sharpness, indicating richer edge and texture information with higher values.”
DenseFuse: A Fusion Approach to Infrared and Visible Images
Li, H., & Wu, X.-J. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2887342 · Cited by 2,032
Objective image fusion performance measure
Xydeas, C. S., & Petrović, V. (2000) · Electronics Letters · doi:10.1049/el:20000267 · Cited by 1,996 · cited 1×
“Gradient-BasedFusionPerformance (Q AB/F ) [183] assesses edge detail preservation, with scores near 1 indicating effective edge retention.”
Infrared and visible image fusion methods and applications: A survey
Ma, J., Ma, Y., & Li, C. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.02.004 · Cited by 1,732 · cited 2×
“Although there are existing surveys [8], [9] in the literature concerning infrared and visible image fusion, our work stands out distinctly.”
In-fibre Bragg grating sensors
Rao, Y.-J. (1997) · Measurement Science and Technology · doi:10.1088/0957-0233/8/4/002 · Cited by 1,606
IFCNN: A general image fusion framework based on convolutional neural network
Zhang, Y., Liu, Y., Sun, P., Yan, H., Zhao, X., & Zhang, L. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.07.011 · Cited by 1,550
SwinFusion: Cross-domain Long-range Learning for General Image Fusion via Swin Transformer
Ma, J., Tang, L., Fan, F., Huang, J., Mei, X., & Ma, Y. (2022) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2022.105686 · Cited by 1,526 · cited 2×
“Ma et al. [95] present the SwinFusion, leveraging the swin transformer [96] to construct two kinds of attention modules.”
A general framework for image fusion based on multi-scale transform and sparse representation
Liu, Y., Liu, S., & Wang, Z. (2015) · Information Fusion · doi:10.1016/j.inffus.2014.09.004 · Cited by 1,431 · cited 1×
“…advantages of generating a fused image, which can enhance information representation, reduce noise, and simultaneously satisfy observational requirements while better supporting wide practical applications, such as remote sensing, military surveillance, and autonomous driving, to name a few [7].”
Information measure for performance of image fusion
Qu, G., Zhang, D., & Yan, P. (2002) · Electronics Letters · doi:10.1049/el:20020212 · Cited by 1,427 · cited 1×
“Mutual Information (MI) [181] measures the information transferred from source to fused images.”
Integrated microwave photonics
Marpaung, D., Yao, J., & Capmany, J. (2019) · Nature Photonics · doi:10.1038/s41566-018-0310-5 · Cited by 1,393
Ma, J., Xu, H., Jiang, J., Mei, X., & Zhang, X.-P. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2977573 · Cited by 1,343 · cited 1×
“…[65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two discrim-inators to balance the typical modality information.”
Liu, J., Fan, X., Huang, Z., Wu, G., Liu, R., Zhong, W., & Luo, Z. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00571 · Cited by 1,336 · cited 3×
“…[65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two discrim-inators to balance the typical modality information.”
Self-citationRFN-Nest: An end-to-end residual fusion network for infrared and visible images
Li, H., Wu, X.-J., & Kittler, J. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.02.023 · Cited by 1,173 · cited 3×
“Network Architecture: RFN-Nest [21] distinguishes itself with a novel network architecture, introducing a Residual Fusion Network (RFN) that targets the enhancement of deep feature-based image fusion, focusing particularly on the sophistication of network design to preserve intricate details.”
PIAFusion: A progressive infrared and visible image fusion network based on illumination aware
Tang, L., Yuan, J., Zhang, H., Jiang, X., & Ma, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.007 · Cited by 1,161
Tang, L., Yuan, J., & Ma, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.12.004 · Cited by 1,136
A new image fusion performance metric based on visual information fidelity
Han, Y., Cai, Y., Cao, Y., & Xu, X. (2013) · Information Fusion · doi:10.1016/j.inffus.2011.08.002 · Cited by 1,071 · cited 1×
“Visual Information Fidelity (VIF) [182] assesses fusion fidelity aligned with the Authorized licensed use limited to the terms of the applicable license agreement with IEEE.”
Microwave Ablation: Principles and Applications
Simon, C. J., Dupuy, D. E., & Mayo-Smith, W. W. (2005) · RadioGraphics · doi:10.1148/rg.25si055501 · Cited by 1,023 · cited 1×
“Beyond the realm of visible light, there exist numerous other spectral types, including ultraviolet, infrared, microwaves, and X-rays, each with its distinct characteristics and applications [1], [2].”
CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion
Zhao, Z., Bai, H., Zhang, J., Zhang, Y., Xu, S., Lin, Z., Timofte, R., & Van Gool, L. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00572 · Cited by 948
LLVIP: A Visible-infrared Paired Dataset for Low-light Vision
Jia, X., Zhu, C., Li, M., Tang, W., & Zhou, W. (2021) · 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw54120.2021.00389 · Cited by 896
Li, H., Wu, X.-J., & Durrani, T. (2020) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2020.3005230 · Cited by 798
SDNet: A Versatile Squeeze-and-Decomposition Network for Real-Time Image Fusion
Zhang, H., & Ma, J. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01501-8 · Cited by 747
Assessment of image fusion procedures using entropy, image quality, and multispectral classification
Van Aardt, J. (2008) · Journal of Applied Remote Sensing · doi:10.1117/1.2945910 · Cited by 747
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers
Zhang, J., Liu, H., Yang, K., Hu, X., Liu, R., & Stiefelhagen, R. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3300537 · Cited by 731 · cited 1×
“Encoder-based fusion methods aggregate features early, as shown in works like [118], [119], [120], [121], while decoder-based fusion combines features during reconstruction, as demonstrated in [122], [123], [124], [125].”
Self-citationHyperspectral and Multispectral Data Fusion: A comparative review of the recent literature
Yokoya, N., Grohnfeldt, C., & Chanussot, J. (2017) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2016.2637824 · Cited by 712
MFNet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes
Ha, Q., Watanabe, K., Karasawa, T., Ushiku, Y., & Harada, T. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206396 · Cited by 693 · cited 1×
“Encoder-based fusion methods aggregate features early, as shown in works like [118], [119], [120], [121], while decoder-based fusion combines features during reconstruction, as demonstrated in [122], [123], [124], [125].”
Ma, J., Zhang, H., Shao, Z., Liang, P., & Xu, H. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2020.3038013 · Cited by 618 · cited 2×
“Ma et al. propose GANMcC [66], introducing a more balanced training with multi-classification for visible and infrared images.”
Cui, G., Feng, H., Xu, Z., Li, Q., & Chen, Y. (2015) · Optics Communications · doi:10.1016/j.optcom.2014.12.032 · Cited by 612 · cited 1×
“Average Gradient (AG) [187] measures texture features and details, with higher AG values indicating enhanced fusion performance.”
Zhang, H., Xu, H., Xiao, Y., Guo, X., & Ma, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6975 · Cited by 589
FusionDN: A Unified Densely Connected Network for Image Fusion
Xu, H., Ma, J., Le, Z., Jiang, J., & Guo, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6936 · Cited by 587 · cited 2×
“Xu et al. [166] employ NR-IQA and entropy for weight adjustment, ensuring stable qualityandreducinggroundtruthreliance.”
DATFuse: Infrared and Visible Image Fusion via Dual Attention Transformer
Tang, W., He, F., Liu, Y., Duan, Y., & Si, T. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3234340 · Cited by 571
Rtfnet: Rgb-thermal fusion network for semantic segmentation of urban scenes
Sun, Y., Zuo, W., & Liu, M. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2904733 · Cited by 546 · cited 1×
“Encoder-based fusion methods aggregate features early, as shown in works like [118], [119], [120], [121], while decoder-based fusion combines features during reconstruction, as demonstrated in [122], [123], [124], [125].”
LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images
Li, H., Xu, T., Wu, X.-J., Lu, J., & Kittler, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3268209 · Cited by 493 · cited 3×
“Atthe pixel level, L1 and MSE loss functions [19], [36], [156] assess image similarity through direct pixel comparisons.”
SuperFusion: A Versatile Image Registration and Fusion Network with Semantic Awareness
Tang, L., Deng, Y., Ma, Y., Huang, J., & Ma, J. (2022) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2022.106082 · Cited by 488 · cited 4×
“However, few efforts have been focus on this issues [114], [153], [155], [156], [160], [190], [191].”
InfluentialYDTR: Infrared and Visible Image Fusion via Y-Shape Dynamic Transformer
Tang, W., He, F., & Liu, Y. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3192661 · Cited by 485 · cited 2×
“At the evaluative metrics level, using less common metrics like Spatial Frequency (SF) [97] as loss functions emphasizes the frequency characteristics and visual effects of images, thus achieving effective fusion while maintaining visual comfort.”
STDFusionNet: An Infrared and Visible Image Fusion Network Based on Salient Target Detection
Ma, J., Tang, L., Xu, M., Zhang, H., & Xiao, G. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3075747 · Cited by 483
Infrared and visible image fusion with convolutional neural networks
Liu, Y., Chen, X., Cheng, J., Peng, H., & Wang, Z. (2018) · International Journal of Wavelets, Multiresolution and Information Processing · doi:10.1142/s0219691318500182 · Cited by 479 · cited 1×
“2) CNN-Based Approaches: In CNN-based image fusion algorithms [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], the process generally involves three main steps: feature extraction, fusion, and image reconstruction, as shown in Fig.”
The TNO Multiband Image Data Collection
Toet, A. (2017) · Data in Brief · doi:10.1016/j.dib.2017.09.038 · Cited by 476
Infrared and visible image fusion via detail preserving adversarial learning
Ma, J., Liang, P., Yu, W., Chen, C., Guo, X., Wu, J., & Jiang, J. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.07.005 · Cited by 456 · cited 1×
“Subse-quently, the same team [84] further improves the FusionGAN with two loss functions.”
MATR: Multimodal Medical Image Fusion via Multiscale Adaptive Transformer
Tang, W., He, F., Liu, Y., & Duan, Y. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3193288 · Cited by 442
DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion
Zhao, Z., Bai, H., Zhu, Y., Zhang, J., Xu, S., Zhang, Y., Zhang, K., Meng, D., Timofte, R., & Van Gool, L. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00742 · Cited by 431 · cited 1×
“Recursive Architectures: Emphasizing iterative information enhancement, this group uses recursive designs [156] and diffusion models [168] to progressively refine fusion quality, suitable for handling sequential data by capitalizing on prior information for ongoing image improvement.”
Learning a Deep Multi-Scale Feature Ensemble and an Edge-Attention Guidance for Image Fusion
Liu, J., Fan, X., Jiang, J., Liu, R., & Luo, Z. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3056725 · Cited by 431 · cited 3×
“Addressing the intricacies of infrared-visible image fusion, Liu et al. [22] advance the field with its innovative network structure that incorporates multi-scale feature learning and an edge-guided attention mechanism, optimizing the architecture to enhance detail definition while reducing noise.”
Self-citationLi, J., Huo, H., Li, C., Wang, R., & Feng, Q. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2997127 · Cited by 415 · cited 1×
“…[65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two discrim-inators to balance the typical modality information.”
Image fusion techniques for remote sensing applications
Simone, G., Farina, A., Morabito, F. C., Serpico, S. B., & Bruzzone, L. (2002) · Information Fusion · doi:10.1016/s1566-2535(01)00056-2 · Cited by 413 · cited 1×
“Consequently,multi-spectralimagefusiontechnologies haveemerged,aimingtosynthesizeandrefinethediverseimage datafromdifferentsensorscapturingthesameenvironment[4], [5].”
Visible and Infrared Image Fusion Using Deep Learning
Zhang, X., & Demiris, Y. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3261282 · Cited by 392 · cited 1×
“Although there are existing surveys [8], [9] in the literature concerning infrared and visible image fusion, our work stands out distinctly.”
Tang, L., Zhang, H., Xu, H., & Ma, J. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101870 · Cited by 385 · cited 2×
“PSFusion [115] introduces progressive semantic injection at the feature level, considering the semantic needs for fusion.”
A-ViT: Adaptive Tokens for Efficient Vision Transformer
Yin, H., Vahdat, A., Alvarez, J. M., Mallya, A., Kautz, J., & Molchanov, P. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01054 · Cited by 360 · cited 1×
“Tang et al. present a y-shape transformer YDTR [97] to construct the encoding and decoding branches by the combination of CNN and transformers, cascading vision transformer blocks [98].”
GMNet: Graded-Feature Multilabel-Learning Network for RGB-Thermal Urban Scene Semantic Segmentation
Zhou, W., Liu, J., Lei, J., Yu, L., & Hwang, J.-N. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3109518 · Cited by 344 · cited 1×
“Encoder-based fusion methods aggregate features early, as shown in works like [118], [119], [120], [121], while decoder-based fusion combines features during reconstruction, as demonstrated in [122], [123], [124], [125].”
Rao, D., Xu, T., & Wu, X. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3273451 · Cited by 343
DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusion
Zhao, Z., Xu, S., Zhang, C., Liu, J., Zhang, J., & Li, P. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/135 · Cited by 341
Self-citationCross-modality interactive attention network for multispectral pedestrian detection
Zhang, L., Liu, Z., Zhang, S., Yang, X., Qiao, H., Huang, K., & Hussain, A. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.09.015 · Cited by 320 · cited 1×
“These methods enhance detection by leveraging cross-modality interactions through attention mechanisms, iterative refinement of spectral features, probabilistic ensembling of detections, and aligning features between modalities for improved precision [109], [110], [111], [112].”
Self-citationWang, D., Liu, J., Fan, X., & Liu, R. (2022) · Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/487 · Cited by 318
Self-citationLiu, J., Liu, Z., Wu, G., Ma, L., Liu, R., Zhong, W., Luo, Z., & Fan, X. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00745 · Cited by 316 · cited 2×
“Lately, Liu et al. proposed SegMiF [116], which leverages dual-task correlation to enhance both segmenta-tionandfusionperformance,introducinghierarchicalinteractive attentionforfine-grainedtaskmappingandcollectingthelargest full-time benchmark for these tasks.”
Self-citationDif-Fusion: Toward High Color Fidelity in Infrared and Visible Image Fusion With Diffusion Models
Yue, J., Fang, L., Xia, S., Deng, Y., & Ma, J. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3322046 · Cited by 304 · cited 1×
“Yueetal.introduce Dif-Fusion[52],utilizingdiffusionmodelstoconstructmulti-channeldistributions,enablingdirectgenerationofchromatic fusedimageswithhighcolorfidelity.”
Multispectral Fusion for Object Detection with Cyclic Fuse-and-Refine Blocks
Zhang, H., Fromont, E., Lefevre, S., & Avignon, B. (2020) · 2020 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip40778.2020.9191080 · Cited by 297 · cited 1×
“These methods enhance detection by leveraging cross-modality interactions through attention mechanisms, iterative refinement of spectral features, probabilistic ensembling of detections, and aligning features between modalities for improved precision [109], [110], [111], [112].”
Classification Saliency-Based Rule for Visible and Infrared Image Fusion
Xu, H., Zhang, H., & Ma, J. (2021) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2021.3100986 · Cited by 294 · cited 1×
“1) AE-Based Approaches: Auto-Encoder (AE)-based meth-ods [10], [11], [12], [13], [14], [15], [16], [17], [18] consist of two steps.”
MURF: Mutually Reinforcing Multi-Modal Image Registration and Fusion
Xu, H., Yuan, J., & Ma, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3283682 · Cited by 288 · cited 3×
“However, few efforts have been focus on this issues [114], [153], [155], [156], [160], [190], [191].”
Weakly Aligned Cross-Modal Learning for Multispectral Pedestrian Detection
Zhang, L., Zhu, X., Chen, X., Yang, X., Lei, Z., & Liu, Z. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00523 · Cited by 271 · cited 1×
“These methods enhance detection by leveraging cross-modality interactions through attention mechanisms, iterative refinement of spectral features, probabilistic ensembling of detections, and aligning features between modalities for improved precision [109], [110], [111], [112].”
Self-citationMetaFusion: Infrared and Visible Image Fusion via Meta-Feature Embedding from Object Detection
Zhao, W., Xie, S., Zhao, F., He, Y., & Lu, H. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01341 · Cited by 259 · cited 2×
“Zhao et al. propose MetaFusion [106], which leverages meta-feature embedding from object detection to align semantic and fusion features, enabling effective joint learning.”
VIF-Net: An Unsupervised Framework for Infrared and Visible Image Fusion
Hou, R., Zhou, D., Nie, R., Liu, D., Xiong, L., Guo, Y., & Yu, C. (2020) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2020.2965304 · Cited by 257 · cited 1×
“2) CNN-Based Approaches: In CNN-based image fusion algorithms [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], the process generally involves three main steps: feature extraction, fusion, and image reconstruction, as shown in Fig.”
Jian, L., Yang, X., Liu, Z., Jeon, G., Gao, M., & Chisholm, D. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2020.3022438 · Cited by 251 · cited 2×
“1) AE-Based Approaches: Auto-Encoder (AE)-based meth-ods [10], [11], [12], [13], [14], [15], [16], [17], [18] consist of two steps.”
Self-citationVIFB: A Visible and Infrared Image Fusion Benchmark
Zhang, X., Ye, P., & Xiao, G. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00060 · Cited by 250
RGB-T Salient Object Detection via Fusing Multi-Level CNN Features
Zhang, Q., Huang, N., Yao, L., Zhang, D., Shan, C., & Han, J. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2959253 · Cited by 247
Liu, J., Lin, R., Wu, G., Liu, R., Luo, Z., & Fan, X. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-023-01952-1 · Cited by 244 · cited 2×
“[167] leveragecontrastivelearningforlossrefinement,boostingcross-modalconsistencyandinformationretention.”
Self-citationEquivariant Multi-Modality Image Fusion
Zhao, Z., Bai, H., Zhang, J., Zhang, Y., Zhang, K., Xu, S., Chen, D., Timofte, R., & Van Gool, L. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02448 · Cited by 243 · cited 1×
“Zhaoetal.propose EMMA[170],aself-supervisedlearningparadigmthatrefines conventionalfusionlossbyincorporatingapseudo-sensing moduleandsensingloss,effectivelysimulatingtheperceptual imagingprocess.”
Efficient and Model-Based Infrared and Visible Image Fusion via Algorithm Unrolling
Zhao, Z., Xu, S., Zhang, J., Liang, C., Zhang, C., & Liu, J. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3075745 · Cited by 242 · cited 3×
“Zhao et al. [24] employ algorithm unrolling for image fusion, targeting low/high-frequency information across modalities.”
Self-citationMulti-Interactive Dual-Decoder for RGB-Thermal Salient Object Detection
Tu, Z., Li, Z., Li, C., Lang, Y., & Tang, J. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3087412 · Cited by 241 · cited 1×
“Salient Object Detection [144], [145], [146], [147], [148] leverages two-stream frameworks and fusion modules to combine complementary cues from RGB-T data for more accurate object delineation.”
Zhang, Y., Zhang, L., Bai, X., & Zhang, L. (2017) · Infrared Physics & Technology · doi:10.1016/j.infrared.2017.05.007 · Cited by 236 · cited 1×
“2) CNN-Based Approaches: In CNN-based image fusion algorithms [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], the process generally involves three main steps: feature extraction, fusion, and image reconstruction, as shown in Fig.”
RXDNFuse: A aggregated residual dense network for infrared and visible image fusion
Long, Y., Jia, H., Zhong, Y., Jiang, Y., & Jia, Y. (2021) · Information Fusion · doi:10.1016/j.inffus.2020.11.009 · Cited by 233
Zhang, Q., Zhao, S., Luo, Y., Zhang, D., Huang, N., & Han, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00266 · Cited by 225 · cited 1×
“Encoder-based fusion methods aggregate features early, as shown in works like [118], [119], [120], [121], while decoder-based fusion combines features during reconstruction, as demonstrated in [122], [123], [124], [125].”
Multimodal Object Detection via Probabilistic Ensembling
Chen, Y.-T., Shi, J., Ye, Z., Mertz, C., Ramanan, D., & Kong, S. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20077-9_9 · Cited by 225 · cited 1×
“These methods enhance detection by leveraging cross-modality interactions through attention mechanisms, iterative refinement of spectral features, probabilistic ensembling of detections, and aligning features between modalities for improved precision [109], [110], [111], [112].”
Wang, D., Liu, J., Liu, R., & Fan, X. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101828 · Cited by 223
Self-citationZhang, H., Yuan, J., Tian, X., & Ma, J. (2021) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2021.3119954 · Cited by 215
Multi-Modal Fusion for End-to-End RGB-T Tracking
Zhang, L., Danelljan, M., Gonzalez-Garcia, A., van de Weijer, J., & Shahbaz Khan, F. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00278 · Cited by 212 · cited 1×
“Object Tracking [128], [129], [130], [131], [132], [133], [134], [135], [136] utilizes pixel-wise, feature-level, and decision-based fusion of RGB-T modalities to enhance tracking robustness and reliability.”
Attribute-Based Progressive Fusion Network for RGBT Tracking
Xiao, Y., Yang, M., Li, C., Liu, L., & Tang, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20187 · Cited by 208
Cross-Modal Transformers for Infrared and Visible Image Fusion
Park, S., Vien, A. G., & Lee, C. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3289170 · Cited by 206 · cited 1×
“Depth Estimation [54], [101], [149], [150], [151] is enhanced by spectral transfer and style transfer methods to handle varying illumination conditions and improve cross-modality matching.”
Vs, V., Jose Valanarasu, J. M., Oza, P., & Patel, V. M. (2022) · 2022 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip46576.2022.9897280 · Cited by 205 · cited 2×
“VS et al. [94] pioneer the use of Transformers for image fusion, introducing a multi-scale auto-encoder and a spatio-transformer fusion strategy to aggregate local and global feature information.”
Multi-modal Gated Mixture of Local-to-Global Experts for Dynamic Image Fusion
Cao, B., Sun, Y., Zhu, P., & Hu, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02153 · Cited by 201 · cited 1×
“Cao et al. propose the MoE-Fusion [108], integrating a mixture of local-global experts to dynamically extract effective features of respective modalities.”
Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion
Yi, X., Xu, H., Zhang, H., Tang, L., & Ma, J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02552 · Cited by 195
RGB-T Semantic Segmentation With Location, Activation, and Sharpening
Li, G., Wang, Y., Liu, Z., Zhang, X., & Zeng, D. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3208833 · Cited by 193 · cited 1×
“Encoder-based fusion methods aggregate features early, as shown in works like [118], [119], [120], [121], while decoder-based fusion combines features during reconstruction, as demonstrated in [122], [123], [124], [125].”
Self-citationRFNet: Unsupervised Network for Mutually Reinforcing Multi-modal Image Registration and Fusion
Xu, H., Ma, J., Yuan, J., Le, Z., & Liu, W. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01906 · Cited by 187 · cited 2×
“Due to the lack of direct supervision between cross-modality images, existing methods [152], [153], [154] utilize MTN to transform images from one modality to another, thereby generating corresponding pseudo-labels.”
DRF: Disentangled Representation for Visible and Infrared Image Fusion
Xu, H., Wang, X., & Ma, J. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3056645 · Cited by 187
Multispectral Object Detection for Autonomous Vehicles
Takumi, K., Watanabe, K., Ha, Q., Tejero-De-Pablos, A., Ushiku, Y., & Harada, T. (2017) · Proceedings of the on Thematic Workshops of ACM Multimedia 2017 · doi:10.1145/3126686.3126727 · Cited by 186
Li, H., Cen, Y., Liu, Y., Chen, X., & Yu, Z. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3069339 · Cited by 186
Efficient Context-Guided Stacked Refinement Network for RGB-T Salient Object Detection
Huo, F., Zhu, X., Zhang, L., Liu, Q., & Shu, Y. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3102268 · Cited by 186
SiamCDA: Complementarity- and Distractor-Aware RGB-T Tracking Based on Siamese Network
Zhang, T., Liu, X., Zhang, Q., & Han, J. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3072207 · Cited by 186 · cited 1×
“Object Tracking [128], [129], [130], [131], [132], [133], [134], [135], [136] utilizes pixel-wise, feature-level, and decision-based fusion of RGB-T modalities to enhance tracking robustness and reliability.”
Zhou, H., Wu, W., Zhang, Y., Ma, J., & Ling, H. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3129609 · Cited by 185 · cited 1×
“…[65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two discrim-inators to balance the typical modality information.”
Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation
Arar, M., Ginger, Y., Danon, D., Bermano, A. H., & Cohen-Or, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01342 · Cited by 175
ReCoNet: Recurrent Correction Network for Fast and Efficient Multi-modality Image Fusion
Huang, Z., Liu, J., Fan, X., Liu, R., Zhong, W., & Luo, Z. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19797-0_31 · Cited by 174 · cited 6×
“Wesurvey7misalignedmulti-modalityimagefusionmethods including UMFusion [153], SuperFusion [114], ReCoNet [156], RFVIF [190], SemLA [160], MURF [155], and IMF [191], and evaluate their fusion performance with slight deformation using 9 widely-used metrics.”
InfluentialSelf-citationMRFS: Mutually Reinforcing Image Fusion and Segmentation
Zhang, H., Zuo, X., Jiang, J., Guo, C., & Ma, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02547 · Cited by 174 · cited 1×
“Zhang et al. proposes MRFS [117], a coupled learning framework that integrates image fusion and segmentation through mutual reinforcement, achieving enhanced visual quality and more accurate segmentation results.”
CGTF: Convolution-Guided Transformer for Infrared and Visible Image Fusion
Li, J., Zhu, J., Li, C., Chen, X., & Yang, B. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3175055 · Cited by 171 · cited 1×
“4) Transformer-Based Approaches: Benefiting from the strong representation capabilities and long-range dependencies based on self-attention mechanisms, researchers are looking at ways to leverage this mechanism into infrared-visible image fusion [88], [89], [90], [91], [92], [93].”
Fusion from Decomposition: A Self-Supervised Decomposition Approach for Image Fusion
Liang, P., Jiang, J., Liu, X., & Ma, J. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19797-0_41 · Cited by 170
UNFusion: A Unified Multi-Scale Densely Connected Network for Infrared and Visible Image Fusion
Wang, Z., Wang, J., Wu, Y., Xu, J., & Zhang, X. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3109895 · Cited by 169
Learning a Graph Neural Network with Cross Modality Interaction for Image Fusion
Li, J., Chen, J., Liu, J., & Ma, H. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612135 · Cited by 168 · cited 2×
“Further, complex image feature-based loss functions, such as the maximization of gradients or edge extraction [51], [95], [100], [115], [153] and including perceptual and contrastive losses [167], provide deeper insights and solutions.”
Self-citationDetFusion: A Detection-driven Infrared and Visible Image Fusion Network
Sun, Y., Cao, B., Zhu, P., & Hu, Q. (2022) · Proceedings of the 30th ACM International Conference on Multimedia · doi:10.1145/3503161.3547902 · Cited by 168 · cited 2×
“Multi-Batch Architectures: Given the modality diversity in infrared-visible fusion, these architectures [22], [23], [107] incorporate specialized structures to address the varied inputs, using distinct network modules and parameters to optimize the fusion of disparate modal information effectively.”
A Task-Guided, Implicitly-Searched and Meta-Initialized Deep Model for Image Fusion
Liu, R., Liu, Z., Liu, J., Fan, X., & Luo, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3382308 · Cited by 166 · cited 3×
“Recently,animplicitly-searchstrategy[56]isproposedwith sufficientconvergenceoffusion,showingremarkableperfor-mancescomparedwithexistingmethods[57],[58].”
Self-citationA Bilevel Integrated Model With Data-Driven Layer Ensemble for Multi-Modality Image Fusion
Liu, R., Liu, J., Jiang, Z., Fan, X., & Luo, Z. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3043125 · Cited by 165 · cited 3×
“Liu et al. [37] advance structural optimization in multi-modal fusion by segregating base and detail layers processing, employing a bilevel optimization to refine texture and detail representation while ensuring structural integrity in the fused output.”
Self-citationSwinFuse: A Residual Swin Transformer Fusion Network for Infrared and Visible Images
Wang, Z., Chen, Y., Shao, W., Li, H., & Zhang, L. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3191664 · Cited by 162
Li, C., Liu, L., Lu, A., Ji, Q., & Tang, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58542-6_14 · Cited by 162
Rao, Y., Wu, D., Han, M., Wang, T., Yang, Y., Lei, T., Zhou, C., Bai, H., & Xing, L. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.12.007 · Cited by 161 · cited 2×
“…[65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two discrim-inators to balance the typical modality information.”
Liu, L., Chen, J., Wu, H., Li, G., Li, C., & Lin, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00479 · Cited by 158 · cited 1×
“Crowd Counting [137], [138], [139], [140], [141], [142], [143] benefits from multi-modality feature fusion to improve density estimation and crowd prediction.”
RGBT Tracking by Trident Fusion Network
Zhu, Y., Li, C., Tang, J., Luo, B., & Wang, L. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3067997 · Cited by 155 · cited 1×
“Object Tracking [128], [129], [130], [131], [132], [133], [134], [135], [136] utilizes pixel-wise, feature-level, and decision-based fusion of RGB-T modalities to enhance tracking robustness and reliability.”
Probing Synergistic High-Order Interaction in Infrared and Visible Image Fusion
Zheng, N., Zhou, M., Huang, J., Hou, J., Li, H., Xu, Y., & Zhao, F. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02492 · Cited by 154
Xu, H., Gong, M., Tian, X. S., Huang, J., & Ma, J. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103407 · Cited by 151 · cited 1×
“Structures: Early works focusing on network structures concentrated on the utilization and fine-tuning of architectures like residual networks [28], [40], [41], [42], [43], dense networks [41], [44], [45], [46], [47], [48], [49], [50], and U-Net, which will not be elaborated here.”
TCCFusion: An infrared and visible image fusion method based on transformer and cross correlation
Tang, W., He, F., & Liu, Y. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2022.109295 · Cited by 149 · cited 1×
“Furthermore, Tang et al. [99] presents Transformer-basedapproacheseffectivelycapturelong-range dependencies,crucialforunderstandingcross-modalityrela-tionships,whileself-attentionhelpsretainglobalcontextinim-agefusion.”
Wang, Z., Wu, Y., Wang, J., Xu, J., & Shao, W. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3139654 · Cited by 147
Edge-Aware Guidance Fusion Network for RGB–Thermal Scene Parsing
Zhou, W., Dong, S., Xu, C., & Qian, Y. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20269 · Cited by 146
Semantics lead all: Towards unified image registration and fusion from a semantic perspective
Xie, H., Zhang, Y., Qiu, J., Zhai, X., Liu, X., Yang, Y., Zhao, S., Luo, Y., & Zhong, J. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101835 · Cited by 146 · cited 2×
“However, few efforts have been focus on this issues [114], [153], [155], [156], [160], [190], [191].”
FreqGAN: Infrared and Visible Image Fusion via Unified Frequency Adversarial Learning
Wang, Z., Zhang, Z., Qi, W., Yang, F., & Xu, J. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3460172 · Cited by 143 · cited 1×
“Further, they developed FreqGAN [87], incorporating dual frequency-constrained discriminators that dynamically adjust the weights for each frequency band.”
PromptFusion: Harmonized Semantic Prompt Learning for Infrared and Visible Image Fusion
Liu, J., Li, X., Wang, Z., Jiang, Z., Zhong, W., Fan, W., & Xu, B. (2024) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2024.124878 · Cited by 140
Self-citationImage fusion based on generative adversarial network consistent with perception
Fu, Y., Wu, X., & Durrani, T. S. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.02.019 · Cited by 138 · cited 1×
“For instance, Fu et al. [85] apply densely connected blocks to construct the generator for preserving the textural details of fused images.”
Exploring fusion strategies for accurate RGBT visual object tracking
Tang, Z., Xu, T., Li, H., Wu, X., Zhu, X., & Kittler, J. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101881 · Cited by 135
Mitigating Modality Discrepancies for RGB-T Semantic Segmentation
Zhao, S., Liu, Y., Jiao, Q., Zhang, Q., & Han, J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3233089 · Cited by 130 · cited 1×
“Encoder-based fusion methods aggregate features early, as shown in works like [118], [119], [120], [121], while decoder-based fusion combines features during reconstruction, as demonstrated in [122], [123], [124], [125].”
Coupled GAN With Relativistic Discriminators for Infrared and Visible Images Fusion
Li, Q., Lu, L., Li, Z., Wu, W., Liu, Z., Jeon, G., & Yang, X. (2019) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2921803 · Cited by 127 · cited 1×
“5: i) Single discrimination [60], [61], [62], [63], [64], [65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two…”
Self-citationMultigrained Attention Network for Infrared and Visible Image Fusion
Li, J., Huo, H., Li, C., Wang, R.-H., Sui, C., & Liu, Z. (2020) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2020.3029360 · Cited by 126 · cited 1×
“…[65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two discrim-inators to balance the typical modality information.”
Self-citationDeep Learning Based Inter-modality Image Registration Supervised by Intra-modality Similarity
Cao, X., Yang, J., Wang, L., Xue, Z., Wang, Q., & Shen, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00919-9_7 · Cited by 123
Cross-Collaborative Fusion-Encoder Network for Robust RGB-Thermal Salient Object Detection
Liao, G., Gao, W., Li, G., Wang, J., & Kwong, S. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3184840 · Cited by 123 · cited 1×
“Salient Object Detection [144], [145], [146], [147], [148] leverages two-stream frameworks and fusion modules to combine complementary cues from RGB-T data for more accurate object delineation.”
Transformer Based Conditional GAN for Multimodal Image Fusion
Zhang, J., Jiao, L., Ma, W., Liu, F., Liu, X., Li, L., Chen, P., & Yang, S. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3243659 · Cited by 116
Xu, H., Liang, P., Yu, W., Jiang, J., & Ma, J. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/549 · Cited by 112 · cited 1×
“…[65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two discrim-inators to balance the typical modality information.”
Improving Misaligned Multi-Modality Image Fusion With One-Stage Progressive Dense Registration
Wang, D., Liu, J., Ma, L., Liu, R., & Fan, X. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3412743 · Cited by 108
Self-citationDEFNet: Dual-Branch Enhanced Feature Fusion Network for RGB-T Crowd Counting
Zhou, W., Pan, Y. H., Lei, J., Ye, L., & Yu, L. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3203385 · Cited by 106 · cited 1×
“Crowd Counting [137], [138], [139], [140], [141], [142], [143] benefits from multi-modality feature fusion to improve density estimation and crowd prediction.”
Gao, Y., Ma, S., & Liu, J. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3206807 · Cited by 105 · cited 1×
“…[65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two discrim-inators to balance the typical modality information.”
Self-citationLi, H., Zhao, J., Li, J., Yu, Z., & Lu, G. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.02.011 · Cited by 103 · cited 2×
“However, few efforts have been focus on this issues [114], [153], [155], [156], [160], [190], [191].”
Li, J., Huo, H., Liu, K., & Li, C. (2020) · Information Sciences · doi:10.1016/j.ins.2020.04.035 · Cited by 103 · cited 1×
“…[65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two discrim-inators to balance the typical modality information.”
Self-supervised feature adaption for infrared and visible image fusion
Zhao, F., Zhao, W., Yao, L., & Liu, Y. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.06.002 · Cited by 101 · cited 1×
“1) AE-Based Approaches: Auto-Encoder (AE)-based meth-ods [10], [11], [12], [13], [14], [15], [16], [17], [18] consist of two steps.”
A Dual-Branch Network for Infrared and Visible Image Fusion
Fu, Y., & Wu, X. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412293 · Cited by 101
Simultaneously Optimizing Perturbations and Positions for Black-Box Adversarial Patch Attacks
Wei, X., Guo, Y., Yu, J., & Zhang, B. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3231886 · Cited by 98 · cited 1×
“3) Attack: Adversarial attacks [172], [173], which add indistinguishable perturbations to images, are easy to fool the estimation of neural networks.”
Physically Adversarial Infrared Patches with Learnable Shapes and Locations
Wei, X., Yu, J., & Huang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01187 · Cited by 95
SMoA: Searching a Modality-Oriented Architecture for Infrared and Visible Image Fusion
Liu, J., Wu, Y., Huang, Z., Liu, R., & Fan, X. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3109818 · Cited by 93 · cited 1×
“As for the construction of super-net, SMoA [53] is proposed based on the auto-encoder paradigm, adequately representing the typical features based on two modality-specific encoders.”
Self-citationPAIF: Perception-Aware Infrared-Visible Image Fusion for Attack-Tolerant Semantic Segmentation
Liu, Z., Liu, J., Zhang, B., Ma, L., Fan, X., & Liu, R. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611928 · Cited by 90 · cited 2×
“While PAIFusion [161] has explored parameter perturbation in image fusion, effectively countering real-world physical and especially cross-modality attacks [193], [194] remains a formidable task.”
Self-citationBi-level Dynamic Learning for Jointly Multi-modality Image Fusion and Beyond
Liu, Z., Liu, J., Wu, G., Ma, L., Fan, X., & Liu, R. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/138 · Cited by 90
Self-citationInfrared and Visible Image Fusion via Interactive Compensatory Attention Adversarial Learning
Wang, Z., Shao, W., Chen, Y., Xu, J., & Zhang, X. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3228685 · Cited by 88
Searching a Hierarchically Aggregated Fusion Architecture for Fast Multi-Modality Image Fusion
Liu, R., Liu, Z., Liu, J., & Fan, X. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475299 · Cited by 87 · cited 3×
“To tackle blurred targets and detail loss, Liu et al. [54] develop a hierarchically aggregated fusion method, aiming for comprehensive target and detail representation.”
Self-citationUnified Adversarial Patch for Cross-modal Attacks in the Physical World
Wei, X., Huang, Y., Sun, Y., & Yu, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00410 · Cited by 73 · cited 1×
“While PAIFusion [161] has explored parameter perturbation in image fusion, effectively countering real-world physical and especially cross-modality attacks [193], [194] remains a formidable task.”
Deep Material-Aware Cross-Spectral Stereo Matching
Zhi, T., Pires, B. R., Hebert, M., & Narasimhan, S. G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00205 · Cited by 72 · cited 1×
“Depth Estimation [54], [101], [149], [150], [151] is enhanced by spectral transfer and style transfer methods to handle varying illumination conditions and improve cross-modality matching.”
Hou, J., Zhang, D., Wu, W., et al. (2021) · Entropy · doi:10.3390/e23030376 · Cited by 72 · cited 1×
“5: i) Single discrimination [60], [61], [62], [63], [64], [65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two…”
UIFGAN: An unsupervised continual-learning generative adversarial network for unified image fusion
Le, Z., Huang, J., Xu, H., Fan, F., Ma, Y., Mei, X., & Ma, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.07.013 · Cited by 71 · cited 1×
“…[65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two discrim-inators to balance the typical modality information.”
TAFNet: A Three-Stream Adaptive Fusion Network for RGB-T Crowd Counting
Tang, H., Wang, Y., & Chau, L. (2022) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas48785.2022.9937583 · Cited by 68 · cited 1×
“Crowd Counting [137], [138], [139], [140], [141], [142], [143] benefits from multi-modality feature fusion to improve density estimation and crowd prediction.”
DNDT: Infrared and Visible Image Fusion Via DenseNet and Dual-Transformer
Zhao, H., & Nie, R. (2021) · International Conference on Information Technology and Biomedical Engineering (ICITBE) · doi:10.1109/icitbe54178.2021.00025 · Cited by 64 · cited 2×
“4) Transformer-Based Approaches: Benefiting from the strong representation capabilities and long-range dependencies based on self-attention mechanisms, researchers are looking at ways to leverage this mechanism into infrared-visible image fusion [88], [89], [90], [91], [92], [93].”
Multimodal Crowd Counting with Mutual Attention Transformers
Wu, Z., Liu, L., Zhang, Y., Mao, M., Lin, L., & Li, G. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9859777 · Cited by 61 · cited 1×
“Crowd Counting [137], [138], [139], [140], [141], [142], [143] benefits from multi-modality feature fusion to improve density estimation and crowd prediction.”
MC 3 Net: Multimodality Cross-Guided Compensation Coordination Network for RGB-T Crowd Counting
Zhou, W., Yang, X., Lei, J., Yan, W., & Yu, L. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3321328 · Cited by 60 · cited 1×
“Crowd Counting [137], [138], [139], [140], [141], [142], [143] benefits from multi-modality feature fusion to improve density estimation and crowd prediction.”
The Effect of Pixel-Level Fusion on Object Tracking in Multi-Sensor Surveillance Video
Cvejic, N., Nikolov, S. G., Knowles, H. D., Łoza, A., Achim, A., Bull, D., & Canagarajah, C. N. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383433 · Cited by 60 · cited 1×
“Object Tracking [128], [129], [130], [131], [132], [133], [134], [135], [136] utilizes pixel-wise, feature-level, and decision-based fusion of RGB-T modalities to enhance tracking robustness and reliability.”
Transfuse: A Unified Transformer-Based Image Fusion Framework Using Self-Supervised LearningPreprint
Qu, L., Liu, S., Wang, M., Li, S., Yin, S., Qiao, Q., & Song, Z. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4130858 · Cited by 46
CAGNet: Coordinated attention guidance network for RGB-T crowd counting
Yang, X., Zhou, W., Yan, W., & Qian, X. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122753 · Cited by 44 · cited 1×
“Crowd Counting [137], [138], [139], [140], [141], [142], [143] benefits from multi-modality feature fusion to improve density estimation and crowd prediction.”
Triple-discriminator generative adversarial network for infrared and visible image fusion
Song, A., Duan, H., Pei, H., & Ding, L. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.02.025 · Cited by 44 · cited 1×
“…[65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two discrim-inators to balance the typical modality information.”
IPLF: A Novel Image Pair Learning Fusion Network for Infrared and Visible Image
Zhu, D., Zhan, W., Jiang, Y., Xu, X., & Guo, R. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3161733 · Cited by 43
IR-MSDNet: Infrared and Visible Image Fusion Based On Infrared Features and Multiscale Dense Network
Raza, A., Liu, J., Liu, Y., Liu, J., Zeng, L., Chen, X., Huo, H., & Fang, T. (2021) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2021.3065121 · Cited by 41 · cited 1×
“Structures: Early works focusing on network structures concentrated on the utilization and fine-tuning of architectures like residual networks [28], [40], [41], [42], [43], dense networks [41], [44], [45], [46], [47], [48], [49], [50], and U-Net, which will not be elaborated here.”
Self-citationDGLT-Fusion: A decoupled global–local infrared and visible image fusion transformer
Yang, X., Huo, H., Wang, R.-H., Li, C., Liu, X., & Li, J. (2022) · Infrared Physics & Technology · doi:10.1016/j.infrared.2022.104522 · Cited by 37 · cited 1×
“4) Transformer-Based Approaches: Benefiting from the strong representation capabilities and long-range dependencies based on self-attention mechanisms, researchers are looking at ways to leverage this mechanism into infrared-visible image fusion [88], [89], [90], [91], [92], [93].”
Infrared and visible image fusion with supervised convolutional neural network
An, W., & Wang, H. (2020) · Optik · doi:10.1016/j.ijleo.2020.165120 · Cited by 37 · cited 1×
“2) CNN-Based Approaches: In CNN-based image fusion algorithms [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], the process generally involves three main steps: feature extraction, fusion, and image reconstruction, as shown in Fig.”
MFST: Multi-Modal Feature Self-Adaptive Transformer for Infrared and Visible Image Fusion
Liu, X., Gao, H., Miao, Q., et al. (2022) · Remote Sensing · doi:10.3390/rs14133233 · Cited by 36 · cited 1×
“4) Transformer-Based Approaches: Benefiting from the strong representation capabilities and long-range dependencies based on self-attention mechanisms, researchers are looking at ways to leverage this mechanism into infrared-visible image fusion [88], [89], [90], [91], [92], [93].”
Infrared and Visible Image Fusion with a Generative Adversarial Network and a Residual Network
Xu, D., Wang, Y., Xu, S., Zhu, K., Zhang, N., & Zhang, X. (2020) · Applied Sciences · doi:10.3390/app10020554 · Cited by 35 · cited 1×
“5: i) Single discrimination [60], [61], [62], [63], [64], [65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two…”
Infrared and Visible Image Fusion through Details Preservation
Liu, Y., Dong, L., Ji, Y., & Xu, W. (2019) · Sensors · doi:10.3390/s19204556 · Cited by 34 · cited 1×
“2) CNN-Based Approaches: In CNN-based image fusion algorithms [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], the process generally involves three main steps: feature extraction, fusion, and image reconstruction, as shown in Fig.”
Feng, M., Song, K., Wang, Y., Liu, J., & Yan, Y. (2020) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2020.102881 · Cited by 34 · cited 1×
“Object Tracking [128], [129], [130], [131], [132], [133], [134], [135], [136] utilizes pixel-wise, feature-level, and decision-based fusion of RGB-T modalities to enhance tracking robustness and reliability.”
Self-citationLiu, J., Wu, G., Liu, Z., Ma, L., Liu, R., & Fan, X. (2024) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2024/123 · Cited by 33 · cited 1×
“Liu et al. [127] use detection and segmentation tasks to guide the automated search process of the loss function, freeing up manual effort and enabling the construction of fusion methods for perception tasks in the CAF framework.”
Self-citationXu, D., Zhang, N., Zhang, Y., Li, Z., Zhao, Z., & Wang, Y. (2022) · Infrared Physics & Technology · doi:10.1016/j.infrared.2022.104242 · Cited by 30 · cited 1×
“Structures: Early works focusing on network structures concentrated on the utilization and fine-tuning of architectures like residual networks [28], [40], [41], [42], [43], dense networks [41], [44], [45], [46], [47], [48], [49], [50], and U-Net, which will not be elaborated here.”
LBP-BEGAN: A generative adversarial network architecture for infrared and visible image fusion
Xu, J., Shi, X., Qin, S., Lu, K., Wang, H., & Ma, J. (2019) · Infrared Physics & Technology · doi:10.1016/j.infrared.2019.103144 · Cited by 29
Wang, M., Liu, X., & Jin, H. (2019) · Image and Vision Computing · doi:10.1016/j.imavis.2019.02.011 · Cited by 28 · cited 1×
“2) CNN-Based Approaches: In CNN-based image fusion algorithms [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], the process generally involves three main steps: feature extraction, fusion, and image reconstruction, as shown in Fig.”
Learn to Search a Lightweight Architecture for Target-Aware Infrared and Visible Image Fusion
Liu, J., Wu, Y., Wu, G., Liu, R., & Fan, X. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3180672 · Cited by 28 · cited 1×
“Furthermore, Liu et al. [55] propose a hardware latency-aware approach for crafting lightweight networks, re-ducingcomputationaldemandsandaidingpracticaldeployment.”
Self-citationThe Fusion of Unmatched Infrared and Visible Images Based on Generative Adversarial Networks
Zhao, Y., Fu, G., Wang, H., & Zhang, S. (2020) · Mathematical Problems in Engineering · doi:10.1155/2020/3739040 · Cited by 25 · cited 1×
“…[65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two discrim-inators to balance the typical modality information.”
Infrared and visible image fusion based on dilated residual attention network
Mustafa, H. T., Yang, J., Mustafa, H., & Zareapoor, M. (2020) · Optik · doi:10.1016/j.ijleo.2020.165409 · Cited by 23 · cited 3×
“Complex Stacked Networks: Methods in this category [13], [28], [41], [46], [88], [178] intertwine different aforementioned blocks, enhancing deep processing for superior fusion quality, and delving into the complementary aspects of various source images for finer fused results.”
Unsupervised densely attention network for infrared and visible image fusion
Li, Y., Wang, J., Miao, Z., & Wang, J. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09301-x · Cited by 22 · cited 1×
“Structures: Early works focusing on network structures concentrated on the utilization and fine-tuning of architectures like residual networks [28], [40], [41], [42], [43], dense networks [41], [44], [45], [46], [47], [48], [49], [50], and U-Net, which will not be elaborated here.”
FLGC-Fusion GAN: An Enhanced Fusion GAN Model by Importing Fully Learnable Group Convolution
Cao, Y., Sun, C., Tang, X., & Liu, R. (2020) · Mathematical Problems in Engineering · doi:10.1155/2020/6384831 · Cited by 22
Self-citationDeep Cross Spectral Stereo Matching Using Multi-Spectral Image Fusion
Liang, X., & Jung, C. (2022) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2022.3155202 · Cited by 20 · cited 1×
“Depth Estimation [54], [101], [149], [150], [151] is enhanced by spectral transfer and style transfer methods to handle varying illumination conditions and improve cross-modality matching.”
Fully convolutional network-based infrared and visible image fusion
Feng, Y., Lu, H., Bai, J., Cao, L., & Yin, H. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-019-08579-w · Cited by 20 · cited 1×
“2) CNN-Based Approaches: In CNN-based image fusion algorithms [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], the process generally involves three main steps: feature extraction, fusion, and image reconstruction, as shown in Fig.”
DenseNetFuse: a study of deep unsupervised DenseNet to infrared and visual image fusion
Pan, Y., Pi, D., Khan, I. A., Khan, Z. U., Chen, J., & Han, M. (2021) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-020-02820-3 · Cited by 19
A Multilevel Hybrid Transmission Network for Infrared and Visible Image Fusion
Li, Q., Han, G., Liu, P., Yang, H., Chen, D., Sun, X., Wu, J., & Liu, D. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3186048 · Cited by 17 · cited 1×
“2) CNN-Based Approaches: In CNN-based image fusion algorithms [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], the process generally involves three main steps: feature extraction, fusion, and image reconstruction, as shown in Fig.”
Advanced Driving Assistance Based on the Fusion of Infrared and Visible Images
Gu, Y., Wang, X., Zhang, C., & Li, B. (2021) · Entropy · doi:10.3390/e23020239 · Cited by 14 · cited 1×
“5: i) Single discrimination [60], [61], [62], [63], [64], [65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two…”
Multiple Task-Oriented Encoders for Unified Image Fusion
Li, Z., Liu, J., Liu, R., Fan, X., Luo, Z., & Gao, W. (2021) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme51207.2021.9428212 · Cited by 11 · cited 1×
“Structure: The structural evolution is characterized by innovative designs tailored for multi-modal data integration: Li et al. [171] introduce a framework with multiple task-specific encoders and a universal decoder, enhancing the precision of feature extraction and facilitating knowledge exchange…”
Self-citationTPFusion: Texture Preserving Fusion of Infrared and Visible Images via Dense Networks
Yang, Z., & Zeng, S. (2022) · Entropy · doi:10.3390/e24020294 · Cited by 11 · cited 1×
“Structures: Early works focusing on network structures concentrated on the utilization and fine-tuning of architectures like residual networks [28], [40], [41], [42], [43], dense networks [41], [44], [45], [46], [47], [48], [49], [50], and U-Net, which will not be elaborated here.”
General cross-modality registration framework for visible and infrared UAV target image registration
Luo, Y., Cha, H., Zuo, L., Cheng, P., & Zhao, Q. (2023) · Scientific Reports · doi:10.1038/s41598-023-39863-3 · Cited by 10
Multiscale channel attention network for infrared and visible image fusion
Zhu, J., Dou, Q., Jian, L., et al. (2020) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.6155 · Cited by 10 · cited 1×
“1) AE-Based Approaches: Auto-Encoder (AE)-based meth-ods [10], [11], [12], [13], [14], [15], [16], [17], [18] consist of two steps.”
Unsupervised Multi-Spectrum Stereo Depth Estimation for All-Day Vision
Guo, Y., Kong, H., & Gu, S. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3331387 · Cited by 10 · cited 1×
“Depth Estimation [54], [101], [149], [150], [151] is enhanced by spectral transfer and style transfer methods to handle varying illumination conditions and improve cross-modality matching.”
Multimodal Sensor Fusion Using Symmetric Skip Autoencoder Via an Adversarial Regulariser
Bhagat, S., Joshi, S. D., Lall, B., & Gupta, S. (2020) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2020.3035633 · Cited by 8 · cited 1×
“5: i) Single discrimination [60], [61], [62], [63], [64], [65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two…”
Li, J., Huo, H., Liu, K., Li, C., Li, S., & Yang, X. (2019) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2019.00322 · Cited by 8 · cited 2×
“…[65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two discrim-inators to balance the typical modality…”
Effective method for fusing infrared and visible images
Fu, Y., Wu, X., & Kittler, J. (2021) · Journal of Electronic Imaging · doi:10.1117/1.jei.30.3.033013 · Cited by 5 · cited 1×
“1) AE-Based Approaches: Auto-Encoder (AE)-based meth-ods [10], [11], [12], [13], [14], [15], [16], [17], [18] consist of two steps.”
Wang, J., Li, Y., & Miao, Z. (2021) · International Conference on Image and Graphics Processing (ICIGP) · doi:10.1145/3447587.3447603 · Cited by 4 · cited 1×
“5: i) Single discrimination [60], [61], [62], [63], [64], [65], [66], [67], [68], [69], [70] utilizes the original GAN to constrain the fused images as similar as one modality. ii) Dual discrimination [29], [71], [72], [73], [74], [75], [76], [77], [78], [79], [80], [81], [82], [83] utilizes two…”
Metadata
- Authors
- Liu, J., Wu, G., Liu, Z., Wang, D., Jiang, Z., Ma, L., Zhong, W., Fan, X., & Liu, R.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2025-04
- Volume
- 47
- Issue
- 4
- Pages
- 2349-2369
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W4405718380
- Cited by
- 218
- References
- 183
- FWCI
- 36.74
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.