The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
U2Fusion: A Unified Unsupervised Image Fusion Network
Abstract
This study proposes a novel unified and unsupervised end-to-end image fusion network, termed as U2Fusion, which is capable of solving different fusion problems, including multi-modal, multi-exposure, and multi-focus cases. Using feature extraction and information measurement, U2Fusion automatically estimates the importance of corresponding source images and comes up with adaptive information preservation degrees. Hence, different fusion tasks are unified in the same framework. Based on the adaptive degrees, a network is trained to preserve the adaptive similarity between the fusion result and source images. Therefore, the stumbling blocks in applying deep learning for image fusion, e.g., the requirement of ground-truth and specifically designed metrics, are greatly mitigated. By avoiding the loss of previous fusion capabilities when training a single model for different tasks sequentially, we obtain a unified model that is applicable to multiple fusion tasks. Moreover, a new aligned infrared and visible image dataset, RoadScene (available at https://github.com/hanna-xu/RoadScene), is released to provide a new option for benchmark evaluation. Qualitative and quantitative experimental results on three typical image fusion tasks validate the effectiveness and universality of U2Fusion. Our code is publicly available at https://github.com/hanna-xu/U2Fusion.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
Li, Z., & Hoiem, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2773081 · Cited by 4,202
FusionGAN: A generative adversarial network for infrared and visible image fusion
Ma, J., Yu, W., Liang, P., Li, C., & Jiang, J. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.09.004 · Cited by 2,267
Self-citationDenseFuse: A Fusion Approach to Infrared and Visible Images
Li, H., & Wu, X.-J. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2887342 · Cited by 2,032
Continual Learning Through Synaptic Intelligence
Zenke, F., Poole, B., & Ganguli, S. (2017) · arXiv · doi:10.48550/arxiv.1703.04200 · Cited by 1,784
Image Fusion With Guided Filtering
Li, S., Kang, X., & Hu, J. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2244222 · Cited by 1,781
Infrared and visible image fusion methods and applications: A survey
Ma, J., Ma, Y., & Li, C. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.02.004 · Cited by 1,732
Self-citationIFCNN: A general image fusion framework based on convolutional neural network
Zhang, Y., Liu, Y., Sun, P., Yan, H., Zhao, X., & Zhang, L. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.07.011 · Cited by 1,550
Ma, J., Xu, H., Jiang, J., Mei, X., & Zhang, X.-P. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2977573 · Cited by 1,343
Self-citationMulti-focus image fusion with a deep convolutional neural network
Liu, Y., Chen, X., Peng, H., & Wang, Z. (2017) · Information Fusion · doi:10.1016/j.inffus.2016.12.001 · Cited by 1,327
Pixel-level image fusion: A survey of the state of the art
Li, S., Kang, X., Fang, L., Hu, J., & Yin, H. (2017) · Information Fusion · doi:10.1016/j.inffus.2016.05.004 · Cited by 1,276
Learning a Deep Single Image Contrast Enhancer from Multi-Exposure Images
Cai, J., Gu, S., & Zhang, L. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2794218 · Cited by 1,234
Infrared and visible image fusion via gradient transfer and total variation minimization
Ma, J., Chen, C., Li, C., & Huang, J. (2016) · Information Fusion · doi:10.1016/j.inffus.2016.02.001 · Cited by 1,128
Self-citationA new image fusion performance metric based on visual information fidelity
Han, Y., Cai, Y., Cao, Y., & Xu, X. (2013) · Information Fusion · doi:10.1016/j.inffus.2011.08.002 · Cited by 1,071
Image Fusion With Convolutional Sparse Representation
Liu, Y., Chen, X., Ward, R. K., & Jane Wang, Z. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2618776 · Cited by 986
CNN: Single-label to Multi-label
Wei, Y., Xia, W., Huang, J., Ni, B., Dong, J., Zhao, Y., & Yan, S. (2014) · arXiv · doi:10.48550/arxiv.1406.5726 · Cited by 784
HCP: A Flexible CNN Framework for Multi-Label Image Classification
Wei, Y., Xia, W., Lin, M., Huang, J., Ni, B., Dong, J., Zhao, Y., & Yan, S. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2491929 · Cited by 784
Deep learning for pixel-level image fusion: Recent advances and future prospects
Liu, Y., Chen, X., Wang, Z., Wang, Z. J., Ward, R. K., & Wang, X. (2018) · Information Fusion · doi:10.1016/j.inffus.2017.10.007 · Cited by 736
DeepFuse: A Deep Unsupervised Approach for Exposure Fusion with Extreme Exposure Image Pairs
Prabhakar, K. R., Srikar, V. S., & Babu, R. V. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.505 · Cited by 726
Liu, Z., Blasch, E., Xue, Z., Zhao, J., Laganiere, R., & Wu, W. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.109 · Cited by 717
Yin, M., Liu, X., Liu, Y., & Chen, X. (2019) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2018.2838778 · Cited by 594
FusionDN: A Unified Densely Connected Network for Image Fusion
Xu, H., Ma, J., Le, Z., Jiang, J., & Guo, X. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6936 · Cited by 587
Self-citationA new image quality metric for image fusion: The sum of the correlations of differences
Aslantas, V., & Bendes, E. (2015) · AEU - International Journal of Electronics and Communications · doi:10.1016/j.aeue.2015.09.004 · Cited by 507
Zhang, Q., Liu, Y., Blum, R. S., Han, J., & Tao, D. (2018) · Information Fusion · doi:10.1016/j.inffus.2017.05.006 · Cited by 499
Multi-focus image fusion with dense SIFT
Liu, Y., Liu, S., & Wang, Z. (2015) · Information Fusion · doi:10.1016/j.inffus.2014.05.004 · Cited by 493
Zhou, Z., Wang, B., Li, S., & Dong, M. (2016) · Information Fusion · doi:10.1016/j.inffus.2015.11.003 · Cited by 475
Active and dynamic information fusion for facial expression understanding from image sequences
Zhang, Y., & Ji, Q. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.93 · Cited by 461
Infrared and visible image fusion via detail preserving adversarial learning
Ma, J., Liang, P., Yu, W., Chen, C., Guo, X., Wu, J., & Jiang, J. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.07.005 · Cited by 456
Self-citationA medical image fusion method based on convolutional neural networks
Liu, Y., Chen, X., Cheng, J., & Peng, H. (2017) · 2017 20th International Conference on Information Fusion (Fusion) · doi:10.23919/icif.2017.8009769 · Cited by 391
Zhu, Z., Zheng, M., Qi, G., Wang, D., & Xiang, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2898111 · Cited by 328
SESF-Fuse: an unsupervised deep model for multi-focus image fusion
Ma, B., Zhu, Y., Yin, X., Ban, X., Huang, H., & Mukeshimana, M. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05358-9 · Cited by 266
FuseGAN: Learning to Fuse Multi-Focus Image via Conditional Generative Adversarial Network
Guo, X., Nie, R., Cao, J., Zhou, D., Mei, L., & He, K. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2895292 · Cited by 227
Self-citationOn the Reconstruction of Face Images from Deep Face Templates
Mai, G., Cao, K., Yuen, P. C., & Jain, A. K. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2827389 · Cited by 220
Multi-Exposure and Multi-Focus Image Fusion in Gradient Domain
Paul, S., Sevcenco, I. S., & Agathoklis, P. (2016) · Journal of Circuits, Systems and Computers · doi:10.1142/s0218126616501231 · Cited by 211
Multi-Exposure Image Fusion by Optimizing A Structural Similarity Index
Ma, K., Duanmu, Z., Yeganeh, H., & Wang, Z. (2018) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2017.2786138 · Cited by 201
A Multi-Task Learning Framework for Emotion Recognition Using 2D Continuous Space
Xia, R., & Liu, Y. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2512598 · Cited by 183
Guided filter-based multi-focus image fusion through focus region detection
Qiu, X., Li, M., Zhang, L., & Yuan, X. (2019) · Signal Processing: Image Communication · doi:10.1016/j.image.2018.12.004 · Cited by 183
Multi-scale Single Image Dehazing Using Perceptual Pyramid Deep Network
Zhang, H., Sindagi, V., & Patel, V. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00135 · Cited by 179
Multi-Focus Image Fusion Based on Spatial Frequency in Discrete Cosine Transform Domain
Cao, L., Jin, L., Tao, H., et al. (2014) · IEEE Signal Processing Letters · doi:10.1109/lsp.2014.2354534 · Cited by 174
A Neuro-Fuzzy Approach for Medical Image Fusion
Das, S., & Kundu, M. K. (2013) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2013.2282461 · Cited by 170
Dense SIFT for ghost-free multi-exposure fusion
Liu, Y., & Wang, Z. (2015) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2015.06.021 · Cited by 166
Hedging Deep Features for Visual Tracking
Qi, Y., Zhang, S., Qin, L., Huang, Q., Yao, H., Lim, J., & Yang, M. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2828817 · Cited by 127
Xu, H., Liang, P., Yu, W., Jiang, J., & Ma, J. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/549 · Cited by 112
Self-citationMulti-Scale Fusion of Two Large-Exposure-Ratio Images
Yang, Y., Cao, W., Wu, S., & Li, Z. G. (2018) · IEEE Signal Processing Letters · doi:10.1109/lsp.2018.2877893 · Cited by 82
Continuous Learning without Forgetting for Person Re-Identification
Sugianto, N., Tjondronegoro, D., Sorwar, G., Chakraborty, P. R., & Yuwono, E. I. (2019) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2019.8909828 · Cited by 19
Cited by
SwinFusion: Cross-domain Long-range Learning for General Image Fusion via Swin Transformer
Ma, J., Tang, L., Fan, F., Huang, J., Mei, X., & Ma, Y. (2022) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2022.105686 · Cited by 1,526
Self-citationRFN-Nest: An end-to-end residual fusion network for infrared and visible images
Li, H., Wu, X.-J., & Kittler, J. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.02.023 · Cited by 1,173
PIAFusion: A progressive infrared and visible image fusion network based on illumination aware
Tang, L., Yuan, J., Zhang, H., Jiang, X., & Ma, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.007 · Cited by 1,161
Self-citationTang, L., Yuan, J., & Ma, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.12.004 · Cited by 1,136
Self-citationInfrared and Visible Image Fusion via Decoupling Network
Wang, X., Guan, Z., Yu, S., Cao, J., & Li, Y. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3216413 · Cited by 969
Metadata
- Authors
- Xu, H., Ma, J., Jiang, J., Guo, X., & Ling, H.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2022-01-01
- Volume
- 44
- Issue
- 1
- Pages
- 502-518
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W3046194589
- Cited by
- 2,127
- References
- 48
- FWCI
- 69.36
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.