DM-Fusion: Deep Model-Driven Network for Heterogeneous Image Fusion
The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
Abstract
Heterogeneous image fusion (HIF) is an enhancement technique for highlighting the discriminative information and textural detail from heterogeneous source images. Although various deep neural network-based HIF methods have been proposed, the most widely used single data-driven manner of the convolutional neural network always fails to give a guaranteed theoretical architecture and optimal convergence for the HIF problem. In this article, a deep model-driven neural network is designed for this HIF problem, which adaptively integrates the merits of model-based techniques for interpretability and deep learning-based methods for generalizability. Unlike the general network architecture as a black box, the proposed objective function is tailored to several domain knowledge network modules to model the compact and explainable deep model-driven HIF network termed DM-fusion. The proposed deep model-driven neural network shows the feasibility and effectiveness of three parts, the specific HIF model, an iterative parameter learning scheme, and data-driven network architecture. Furthermore, the task-driven loss function strategy is proposed to achieve feature enhancement and preservation. Numerous experiments on four fusion tasks and downstream applications illustrate the advancement of DM-fusion compared with the state-of-the-art (SOTA) methods both in fusion quality and efficiency. The source code will be available soon.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recently, the emergence of diffusion models14–17 has brought a paradigm shift in the image generation field, with their exceptional generative capabilities demonstrating tremendous potential in LLIE tasks.”
From Temporally consistent low-light face video enhancement via video-to-video conditional diffusion (2026) · cites this paper as [14]
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
A universal image quality index
Wang, Z., & Bovik, A. C. (2002) · IEEE Signal Processing Letters · doi:10.1109/97.995823 · Cited by 5,757
Deep Convolutional Neural Network for Inverse Problems in Imaging
Jin, K. H., McCann, M. T., Froustey, E., & Unser, M. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2713099 · Cited by 2,529 · cited 1×
“This kind of connection structure in the DM model is comparable to the residual network structure [21] or densely connected network structures [22], which have demonstrated superiority in extracting deep features.”
BackgroundBenchmarking Single-Image Dehazing and Beyond
Li, B., Ren, W., Fu, D., Tao, D., Feng, D., Zeng, W., & Wang, Z. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2867951 · Cited by 2,301
FusionGAN: A generative adversarial network for infrared and visible image fusion
Ma, J., Yu, W., Liang, P., Li, C., & Jiang, J. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.09.004 · Cited by 2,267
U2Fusion: A Unified Unsupervised Image Fusion Network
Xu, H., Ma, J., Jiang, J., Guo, X., & Ling, H. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3012548 · Cited by 2,127
DenseFuse: A Fusion Approach to Infrared and Visible Images
Li, H., & Wu, X.-J. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2887342 · Cited by 2,032
Objective image fusion performance measure
Xydeas, C. S., & Petrović, V. (2000) · Electronics Letters · doi:10.1049/el:20000267 · Cited by 1,996
Image Fusion With Guided Filtering
Li, S., Kang, X., & Hu, J. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2244222 · Cited by 1,781
IFCNN: A general image fusion framework based on convolutional neural network
Zhang, Y., Liu, Y., Sun, P., Yan, H., Zhao, X., & Zhang, L. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.07.011 · Cited by 1,550
Ma, J., Xu, H., Jiang, J., Mei, X., & Zhang, X.-P. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2977573 · Cited by 1,343
Liu, J., Fan, X., Huang, Z., Wu, G., Liu, R., Zhong, W., & Luo, Z. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00571 · Cited by 1,336
Multi-focus image fusion with a deep convolutional neural network
Liu, Y., Chen, X., Peng, H., & Wang, Z. (2017) · Information Fusion · doi:10.1016/j.inffus.2016.12.001 · Cited by 1,327
Pansharpening by Convolutional Neural Networks
Masi, G., Cozzolino, D., Verdoliva, L., & Scarpa, G. (2016) · Remote Sensing · doi:10.3390/rs8070594 · Cited by 1,152
Infrared and visible image fusion via gradient transfer and total variation minimization
Ma, J., Chen, C., Li, C., & Huang, J. (2016) · Information Fusion · doi:10.1016/j.inffus.2016.02.001 · Cited by 1,128
A new image fusion performance metric based on visual information fidelity
Han, Y., Cai, Y., Cao, Y., & Xu, X. (2013) · Information Fusion · doi:10.1016/j.inffus.2011.08.002 · Cited by 1,071
Deep ADMM-Net for compressive sensing MRI
Yang, Y., Sun, J. Z., Li, H., & Xu, Z. (2016) · neural information processing systems · Cited by 1,024
LLVIP: A Visible-infrared Paired Dataset for Low-light Vision
Jia, X., Zhu, C., Li, M., Tang, W., & Zhou, W. (2021) · 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw54120.2021.00389 · Cited by 896
Assessment of image fusion procedures using entropy, image quality, and multispectral classification
Van Aardt, J. (2008) · Journal of Applied Remote Sensing · doi:10.1117/1.2945910 · Cited by 747
DeepFuse: A Deep Unsupervised Approach for Exposure Fusion with Extreme Exposure Image Pairs
Prabhakar, K. R., Srikar, V. S., & Babu, R. V. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.505 · Cited by 726
Latent Low-Rank Representation for subspace segmentation and feature extraction
Liu, G., & Yan, S. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126422 · Cited by 679
MDLatLRR: A Novel Decomposition Method for Infrared and Visible Image Fusion
Li, H., Wu, X.-J., & Kittler, J. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2975984 · Cited by 627
Yin, M., Liu, X., Liu, Y., & Chen, X. (2019) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2018.2838778 · Cited by 594
Infrared and Visible Image Fusion using a Deep Learning Framework
Li, H., Wu, X.-J., & Kittler, J. (2018) · 2018 24th International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr.2018.8546006 · Cited by 534
Whole-brain activity mapping onto a zebrafish brain atlas
Randlett, O., Wee, C. L., Naumann, E. A., Nnaemeka, O., Schoppik, D., Fitzgerald, J. E., Portugues, R., Lacoste, A. M. B., Riegler, C., Engert, F., et al. (2015) · Nature Methods · doi:10.1038/nmeth.3581 · Cited by 528 · cited 3×
“In PMF and SMF, two different data modalities, i.e., PMF and SMF, are collected from the Whole Brain Atlas (WBA) dataset [41].”
MethodA new automated quality assessment algorithm for image fusion
Chen, Y., & Blum, R. S. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2007.12.002 · Cited by 516
Comments on ‘Information measure for performance of image fusion’
Hossny, M., Nahavandi, S., & Creighton, D. (2008) · Electronics Letters · doi:10.1049/el:20081754 · Cited by 505
The TNO Multiband Image Data Collection
Toet, A. (2017) · Data in Brief · doi:10.1016/j.dib.2017.09.038 · Cited by 476
The Seventh Visual Object Tracking VOT2019 Challenge Results
Kristan, M., Matas, J., Leonardis, A., Felsberg, M., Pflugfelder, R., Kamarainen, J.-K., Zajc, L. C., Drbohlav, O., Lukezic, A., Berg, A., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00276 · Cited by 461
Xu, T., Feng, Z.-H., Wu, X.-J., & Kittler, J. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2919201 · Cited by 412
Simultaneous image fusion and denoising with adaptive sparse representation
Liu, Y., & Wang, Z. (2015) · IET Image Processing · doi:10.1049/iet-ipr.2014.0311 · Cited by 351
Seeing What a GAN Cannot GeneratePreprint
(2020) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 347
Regularizing Hyperspectral and Multispectral Image Fusion by CNN Denoiser
Dian, R., Li, S., & Kang, X. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2980398 · Cited by 328
Multimodal medical image fusion review: Theoretical background and recent advances
Hermessi, H., Mourali, O., & Zagrouba, E. (2021) · Signal Processing · doi:10.1016/j.sigpro.2021.108036 · Cited by 301
VIFB: A Visible and Infrared Image Fusion Benchmark
Zhang, X., Ye, P., & Xiao, G. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00060 · Cited by 250
Efficient and Model-Based Infrared and Visible Image Fusion via Algorithm Unrolling
Zhao, Z., Xu, S., Zhang, J., Liang, C., Zhang, C., & Liu, J. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3075745 · Cited by 242
Joint Group Feature Selection and Discriminative Filter Learning for Robust Visual Object Tracking
Xu, T., Feng, Z.-H., Wu, X.-J., & Kittler, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00804 · Cited by 222
Koroleva, O. A., Tomlinson, M. L., Leader, D., Shaw, P., & Doonan, J. H. (2005) · The Plant Journal · doi:10.1111/j.1365-313x.2004.02281.x · Cited by 219
Zhao, J., & Liu, Z. (2007) · Cited by 219
Deep Learning-based Multi-focus Image Fusion: A Survey and A Comparative Study
Zhang, X. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3078906 · Cited by 206
Anatomical-Functional Image Fusion by Information of Interest in Local Laplacian Filtering Domain
Du, J., Li, W., & Xiao, B. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2745202 · Cited by 194
Unsupervised Deep Image Fusion With Structure Tensor Representations
Jung, H., Kim, Y., Jang, H., Ha, N., & Sohn, K. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2966075 · Cited by 186
Dictionary learning method for joint sparse representation-based image fusion
Zhang, Q., Fu, Y., Li, H., & Zou, J. (2013) · Optical Engineering · doi:10.1117/1.oe.52.5.057006 · Cited by 182
Adaptive Unfolding Total Variation Network for Low-Light Image Enhancement
Zheng, C., Shi, D., & Shi, W. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00440 · Cited by 174
A Neuro-Fuzzy Approach for Medical Image Fusion
Das, S., & Kundu, M. K. (2013) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2013.2282461 · Cited by 170
A novel similarity based quality metric for image fusion
Li, S., Hong, R., & Wu, X. (2008) · International Conference on Audio, Language and Image Processing · doi:10.1109/icalip.2008.4589989 · Cited by 123
VS-Net: Variable Splitting Network for Accelerated Parallel MRI Reconstruction
Duan, J., Schlemper, J., Qin, C., Ouyang, C., Bai, W., Biffi, C., Bello, G., Statton, B. K., O’Regan, D. P., & Rueckert, D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32251-9_78 · Cited by 96
Ripple-GAN: Lane Line Detection With Ripple Lane Line Detection Network and Wasserstein GAN
Zhang, Y., Lu, Z., Ma, D., Xue, J., & Liao, Q. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.2971728 · Cited by 81
Green Fluorescent Protein and Phase Contrast Image Fusion Via Detail Preserving Cross Network
Tang, W., Liu, Y., Cheng, J., Li, C., & Chen, X. (2021) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2021.3083965 · Cited by 50
Variational Osmosis for Non-Linear Image Fusion
Parisotto, S., Calatroni, L., Bugeau, A., Papadakis, N., & Schönlieb, C. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2983537 · Cited by 36
Tang, W., Liu, Y., Cheng, J., Li, C., Peng, H., & Chen, X. (2020) · Microscopy Research and Technique · doi:10.1002/jemt.23514 · Cited by 24
Deep Selective Fusion of Visible and Near-Infrared Images Using Unsupervised U-Net
Han, Q., & Jung, C. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3142780 · Cited by 21
Behavior Prediction for Unmanned Driving Based on Dual Fusions of Feature and Decision
Zhong, S., Meng, W., Gong, S., Xia, K., Fu, Y., Fu, Q., & Yin, H. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3037926 · Cited by 18
Liu, H.-X., Zhu, T., & Zhao, J. (2013) · Journal of Shanghai Jiaotong University (Science) · doi:10.1007/s12204-013-1437-7 · Cited by 16
Feng, P., Wang, J., Wei, B., & Mi, D. (2013) · Computational and Mathematical Methods in Medicine · doi:10.1155/2013/635040 · Cited by 15
A Multi-Stage Visible and Infrared Image Fusion Network Based on Attention Mechanism
Zheng, X., Yang, Q., Si, P., & Wu, Q. (2022) · Sensors · doi:10.3390/s22103651 · Cited by 13
Yin, J.-L., Chen, B., Peng, Y., & Hwang, H. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3088907 · Cited by 12
Research works printed but not linked: 1
- 10.1109/ICCV.2019.00460 · DOI · Paper page
Cited by
TFDet: Target-Aware Fusion for RGB-T Pedestrian Detection
Zhang, X., Zhang, X., Wang, J., Ying, J., Sheng, Z., Yu, H., Li, C., & Shen, H. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3443455 · Cited by 158
Diffusion Models in Low-Level Vision: A Survey
He, C., Shen, Y., Fang, C., Xiao, F., Tang, L., Zhang, Y., Zuo, W., Guo, Z., & Li, X. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3545047 · Cited by 208
Self-citationA degradation-aware guided fusion network for infrared and visible image
Wang, X., Guan, Z., Qian, W., Cao, J., Ma, R., & Bi, C. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102931 · Cited by 76
HQG-Net: Unpaired Medical Image Enhancement With High-Quality Guidance
He, C., Li, K., Xu, G., Yan, J., Tang, L., Zhang, Y., Wang, Y., & Li, X. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3315307 · Cited by 54
Self-citationA Survey of Camouflaged Object Detection and Beyond
Xiao, F., Hu, S., Shen, Y., Fang, C., Huang, J., Tang, L., Yang, Z., Li, X., & He, C. (2024) · CAAI Artificial Intelligence Research · doi:10.26599/air.2024.9150044 · Cited by 116
Self-citation
Metadata
- Authors
- Xu, G., He, C., Wang, H., Zhu, H., & Ding, W.
- Affiliations
- Xidian University (CN)
- Tsinghua Shenzhen International Graduate School (CN)
- Nanjing University of Posts and Telecommunications (CN)
- Nantong University (CN)
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2024-07
- Volume
- 35
- Issue
- 7
- Pages
- 10071-10085
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W4318141851
- Cited by
- 57
- References
- 58
- Citations by year
- 2026: 13 · 2025: 15 · 2024: 16
- FWCI
- 6.30
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Natural Science Foundation of China (62072256)
- Topics
- Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Enhancement Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- explainable deep learning, multi-source image fusion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.