The DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index.
Model-Informed Multistage Unsupervised Network for Hyperspectral Image Super-Resolution
Abstract
By fusing a low-resolution hyperspectral image (LrMSI) with an auxiliary high-resolution multispectral image (HrMSI), hyperspectral image super-resolution (HISR) can generate a high-resolution hyperspectral image (HrHSI) economically. Despite the promising performance achieved by deep learning (DL), there are still two challenges remaining to be solved. First, most DL-based methods heavily rely on large-scale training triplets, which reduces them to limited generalization and poor practicability in real-world scenarios. Second, existing methods pursue higher performance by designing complex structures from off-the-shelf components while ignoring inherent information from the degradation model, hence leading to insufficient integration of domain knowledge and lower interpretability. To address those drawbacks, we propose a model-informed multi-stage unsupervised network, M2U-Net for short, by leveraging both deep image prior (DIP) and degradation model information. Generally, M2U-Net is built with a three-stage scheme, i.e., degradation information learning (DIL), initialized image establishment (IIE), and deep image generation (DIG) stages. The first stage is to exploit the deep information of the degradation model via a tiny network whose parameters and outputs will serve as guidance for the following two stages. Instead of feeding uninformed noise as input for stage three, IIE stage aims to establish an initialized input with expressive HrHSI-relevant information by resorting to a spectral mapping learning network, thus facilitating the extraction of prior information and further magnifying the potential of DIP for high-quality reconstruction. Last, we propose a dual U-shape network as a powerful regularizer to capture image statistics, in which two U-Nets are coupled together by cross-attention guidance (CAG) module to separately achieve spatial feature extraction and final image generation. The CAG module can incorporate abundant spatial information into the reconstruction process and hence guide the network toward a more plausible generation. Extensive experiments demonstrate the effectiveness of our proposed M2U-Net in terms of quantitative evaluation and visual quality. The code will be available at https://github.com/JiaxinLiCAS.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Image Super-Resolution Using Deep Convolutional Networks
Dong, C., Loy, C. C., He, K., & Tang, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2439281 · Cited by 10,105
A universal image quality index
Wang, Z., & Bovik, A. C. (2002) · IEEE Signal Processing Letters · doi:10.1109/97.995823 · Cited by 5,757
Lempitsky, V., Vedaldi, A., & Ulyanov, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00984 · Cited by 1,916
Lucien Wald, A., Ranchin, T., & Mangolini, M. (1997) · HAL · Cited by 1,362
Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion
Yokoya, N., Yairi, T., & Iwasaki, A. (2012) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2011.2161320 · Cited by 1,180
Deep learning in multimodal remote sensing data fusion: A comprehensive review
Li, J., Hong, D., Gao, L., Yao, J., Zheng, K., Zhang, B., & Chanussot, J. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2022.102926 · Cited by 630
Self-citationFusing Hyperspectral and Multispectral Images via Coupled Sparse Tensor Factorization
Li, S., Dian, R., Fang, L., & Bioucas-Dias, J. M. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2836307 · Cited by 585
BockNet: Blind-Block Reconstruction Network With a Guard Window for Hyperspectral Anomaly Detection
Wang, D., Zhuang, L., Gao, L., Sun, X., Huang, M., & Plaza, A. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3335484 · Cited by 579
Self-citationHyperspectral Super-Resolution by Coupled Spectral Unmixing
Lanaras, C., Baltsavias, E., & Schindler, K. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.409 · Cited by 424
Learning a Low Tensor-Train Rank Representation for Hyperspectral Image Super-Resolution
Dian, R., Li, S., & Fang, L. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2885616 · Cited by 410
SSR-NET: Spatial–Spectral Reconstruction Network for Hyperspectral and Multispectral Image Fusion
Zhang, X., Huang, W., Wang, Q., & Li, X. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3018732 · Cited by 292
Multispectral and hyperspectral image fusion in remote sensing: A survey
Vivone, G. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.08.032 · Cited by 288
Sparse Spatio-spectral Representation for Hyperspectral Image Super-resolution
Akhtar, N., Shafait, F., & Mian, A. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_5 · Cited by 284
Hyper-Sharpening: A First Approach on SIM-GA Data
Selva, M., Aiazzi, B., Butera, F., Chiarantini, L., & Baronti, S. (2015) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2015.2440092 · Cited by 256
Hyperspectral Super-Resolution: A Coupled Tensor Factorization Approach
Kanatsoulis, C. I., Fu, X., Sidiropoulos, N. D., & Ma, W.-K. (2018) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2018.2876362 · Cited by 238
Li, J., Zheng, K., Li, Z., Gao, L., & Jia, X. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3300043 · Cited by 215
Self-citationZheng, K., Gao, L., Liao, W., Hong, D., Zhang, B., Cui, X., & Chanussot, J. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3006534 · Cited by 193
Self-citationMultiscale spatial–spectral transformer network for hyperspectral and multispectral image fusion
Jia, S., Min, Z., & Fu, X. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.03.011 · Cited by 187
Nonlocal Sparse Tensor Factorization for Semiblind Hyperspectral and Multispectral Image Fusion
Dian, R., Li, S., Fang, L., Lu, T., & Bioucas-Dias, J. M. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2951572 · Cited by 182
PSRT: Pyramid Shuffle-and-Reshuffle Transformer for Multispectral and Hyperspectral Image Fusion
Deng, S.-Q., Deng, L.-J., Wu, X., Ran, R., Hong, D., & Vivone, G. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3244750 · Cited by 170
Sliding Dual-Window-Inspired Reconstruction Network for Hyperspectral Anomaly Detection
Wang, D., Zhuang, L., Gao, L., Sun, X., Zhao, X., & Plaza, A. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3351179 · Cited by 168
Self-citationRan, R., Deng, L.-J., Jiang, T.-X., Hu, J.-F., Chanussot, J., & Vivone, G. (2023) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2023.3238200 · Cited by 167
Self-citationGao, L., Li, J., Zheng, K., & Jia, X. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3267890 · Cited by 166
Self-citationMulti-scale receptive fields: Graph attention neural network for hyperspectral image classification
Ding, Y., Zhang, Z., Zhao, X., Hong, D., Cai, W., Yang, N., & Wang, B. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.119858 · Cited by 164
Untrained Neural Network Priors for Inverse Imaging Problems: A Survey
Qayyum, A., Ilahi, I., Shamshad, F., Boussaïd, F., Bennamoun, M., & Qadir, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3204527 · Cited by 158
Spectral Response Function-Guided Deep Optimization-Driven Network for Spectral Super-Resolution
He, J., Li, J., Yuan, Q., Shen, H., & Zhang, L. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3056181 · Cited by 151
Self-citationZero-Shot Hyperspectral Sharpening
Dian, R., Guo, A., & Shutao Li, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3279050 · Cited by 150
Tensor Decompositions for Hyperspectral Data Processing in Remote Sensing: A comprehensive review
Wang, M., Hong, D., Han, Z., Li, J., Yao, J. T., Gao, L., Zhang, B., & Chanussot, J. (2023) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2022.3227063 · Cited by 146
Self-citationLearning a 3D-CNN and Transformer prior for hyperspectral image super-resolution
Ma, Q., Jiang, J., Liu, X., & Ma, J. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101907 · Cited by 140
Wang, D., Zhuang, L., Gao, L., Sun, X., Huang, M. J., & Plaza, A. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3276175 · Cited by 132
Self-citationGuided Deep Decoder: Unsupervised Image Pair Fusion
Uezato, T., Hong, D., Yokoya, N., & He, W. P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58539-6_6 · Cited by 128
Spectral super-resolution meets deep learning: Achievements and challenges
He, J., Yuan, Q., Li, J., Xiao, Y., Liu, D., Shen, H., & Zhang, L. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101812 · Cited by 126
Self-citationDiffusion model with disentangled modulations for sharpening multispectral and hyperspectral images
Cao, Z., Cao, S., Deng, L.-J., Wu, X., Hou, J., & Vivone, G. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102158 · Cited by 125
Multimodal Hyperspectral Unmixing: Insights From Attention Networks
Han, Z., Hong, D., Gao, L., Yao, J. T., Zhang, B., & Chanussot, J. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3155794 · Cited by 122
Self-citationModel Inspired Autoencoder for Unsupervised Hyperspectral Image Super-Resolution
Liu, J., Wu, Z., Xiao, L., & Wu, X. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3143156 · Cited by 118
Quantitative Quality Evaluation of Pansharpened Imagery: Consistency Versus Synthesis
Pálsson, F., Sveinsson, J. R., Ulfarsson, M. O., et al. (2015) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2015.2476513 · Cited by 95
Chen, Z., Wu, G., Gao, H., Ding, Y., Hong, D., & Zhang, B. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120828 · Cited by 94
Multi-Level and Multi-Scale Spatial and Spectral Fusion CNN for Hyperspectral Image Super-Resolution
Han, X., Zheng, Y., & Chen, Y. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00533 · Cited by 88
SENSE: Hyperspectral video object tracker via fusing material and motion cues
Chen, Y., Yuan, Q., Tang, Y., Xiao, Y., He, J., & Liu, Z. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102395 · Cited by 87
A Band Divide-and-Conquer Multispectral and Hyperspectral Image Fusion Method
Sun, W., Ren, K., Meng, X., Xiao, C., Yang, G., & Peng, J. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3046321 · Cited by 79
A Deep Framework for Hyperspectral Image Fusion between Different Satellites
Guo, A., Dian, R., & Shutao Li, A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3229433 · Cited by 77
Deep Unfolding Network for Spatiospectral Image Super-Resolution
Ma, Q., Jiang, J., Liu, X., & Ma, J. (2021) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2021.3136759 · Cited by 73
Multi-spectral and hyperspectral image fusion using 3-D wavelet transform
Zhang, Y., & He, M. (2007) · Journal of Electronics (China) · doi:10.1007/s11767-005-0232-5 · Cited by 68
Zhu, C., Deng, S., Zhou, Y., Deng, L.-J., & Wu, Q. F. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3332176 · Cited by 67
Hyperspectral and Multispectral Image Fusion Using Optimized Twin Dictionaries
Han, X., Yu, J. S., Xue, J., & Sun, W. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2968773 · Cited by 66
Spatio-temporal multi-level attention crop mapping method using time-series SAR imagery
Han, Z., Zhang, C., Gao, L., Zeng, Z., Zhang, B., & Atkinson, P. M. (2023) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2023.11.016 · Cited by 65
Self-citationHe, J., Yuan, Q., Li, J., Xiao, Y., & Zhang, L. (2023) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2023.09.003 · Cited by 64
Self-citationOrthogonal Subspace Unmixing to Address Spectral Variability for Hyperspectral Image
Ren, L., Hong, D., Gao, L., Sun, X., Huang, M. J., & Chanussot, J. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3236471 · Cited by 58
Self-citationLearning the external and internal priors for multispectral and hyperspectral image fusion
Shutao Li, A., Dian, R., & Liu, H. (2023) · Science China Information Sciences · doi:10.1007/s11432-022-3610-5 · Cited by 57
AutoNAS: Automatic Neural Architecture Search for Hyperspectral Unmixing
Han, Z., Hong, D., Gao, L., Zhang, B., Huang, M. J., & Chanussot, J. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3186480 · Cited by 54
Self-citationHyperspectral and Multispectral Image Fusion Using Cluster-Based Multi-Branch BP Neural Networks
Han, X., Yu, J. S., Luo, J., et al. (2019) · Remote Sensing · doi:10.3390/rs11101173 · Cited by 53
Fusion of Hyperspectral and Multispectral Images Accounting for Localized Inter-Image Changes
Fu, X., Jia, S., Xu, M., Zhou, J. W., & Li, Q. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3124240 · Cited by 53
Deep Self-Representation Learning Framework for Hyperspectral Anomaly Detection
Cheng, X., Zhang, M., Lin, S., Li, Y., & Wang, H. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3330225 · Cited by 53
KNLConv: Kernel-Space Non-Local Convolution for Hyperspectral Image Super-Resolution
Ran, R., Deng, L.-J., Zhang, T., Chang, J., Wu, X., & Tian, Q. C. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3382873 · Cited by 51
Unsupervised Deep Tensor Network for Hyperspectral–Multispectral Image Fusion
Yang, J., Xiao, L., Zhao, Y., & Chan, J. C.-W. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3266038 · Cited by 49
Two-Stream Isolation Forest Based on Deep Features for Hyperspectral Anomaly Detection
Cheng, X., Zhang, M., Lin, S., Zhou, K., Zhao, S., & Wang, H. (2023) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2023.3271899 · Cited by 47
Zhang, L., Nie, J., Wei, W., & Zhang, Y. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3361894 · Cited by 46
Li, J., Du, S., Song, R., Li, Y., & Du, Q. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3325682 · Cited by 45
Self-citationLiu, S., Miao, S., Su, J. D., et al. (2021) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2021.3097178 · Cited by 43
Learning Double Subspace Representation for Joint Hyperspectral Anomaly Detection and Noise Removal
Wang, M., Hong, D., Zhang, B., Ren, L., Yao, J. T., & Chanussot, J. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3261964 · Cited by 41
Self-citationHyperspectral Sparse Unmixing via Nonconvex Shrinkage Penalties
Ren, L., Hong, D., Gao, L., Sun, X., Huang, M. J., & Chanussot, J. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3232570 · Cited by 40
Self-citationToward Stable, Interpretable, and Lightweight Hyperspectral Super-Resolution
Guo, W., Xie, W., Jiang, K., Li, Y., Lei, J., & Fang, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02133 · Cited by 36
Bayesian Nonlocal Patch Tensor Factorization for Hyperspectral Image Super-Resolution
Ye, F., Wu, Z., Jia, X., Chanussot, J., Xu, Y., & Wei, Z. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3326687 · Cited by 35
Self-citationDeep Residual Network of Spectral and Spatial Fusion for Hyperspectral Image Super-Resolution
Han, X., & Chen, Y. (2019) · IEEE International Conference on Multimedia Big Data · doi:10.1109/bigmm.2019.00-13 · Cited by 33
Deep SURE for Unsupervised Remote Sensing Image Fusion
Nguyen, H. V., Ulfarsson, M. O., Sveinsson, J. R., & Mura, M. D. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3215902 · Cited by 32
Bidirectional Dilation Transformer for Multispectral and Hyperspectral Image Fusion
Deng, S., Deng, L.-J., Wu, X., et al. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/404 · Cited by 31
Multi-Task Interaction Learning for Spatiospectral Image Super-Resolution
Ma, Q., Jiang, J., Liu, X., & Ma, J. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3161834 · Cited by 27
Hyperspectral and Multispectral Image Fusion Via Self-Supervised Loss and Separable Loss
Gao, H., Shutao Li, A., & Dian, R. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3204769 · Cited by 25
Liu, S., Miao, S., Liu, S., Li, B., Hu, W., & Zhang, Y. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3271359 · Cited by 25
Zhu, C., Deng, S., Li, J., Zhang, Y., Gong, L., Gao, L., Ta, N., Chen, S., & Wu, Q. (2023) · International Journal of Digital Earth · doi:10.1080/17538947.2023.2253206 · Cited by 23
Self-citationUnsupervised Blind Spectral–Spatial Cross-Super-Resolution Network for HSI and MSI Fusion
Wu, H., Wu, S., Zhang, K., et al. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3362862 · Cited by 14
Wang, Y., Li, W., Liu, N., Gui, Y., & Tao, R. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3281355 · Cited by 13
Wu, H., Zhang, K. F., Wu, S., et al. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3320404 · Cited by 13
MMIF: Interpretable Hyperspectral and Multispectral Image Fusion via Maximum Mutual Information
Li, Y., Guo, W., Xie, W., Jiang, T., & Du, Q. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3343711 · Cited by 10
Metadata
- Authors
- Li, J., Zheng, K., Gao, L., Ni, L., Huang, M., & Chanussot, J.
- Venue
- IEEE Transactions on Geoscience and Remote Sensing
- Published
- 2024
- Volume
- 62
- Pages
- 1-17
- ISSN
- 0196-2892 (ISSN-L)
- 1558-0644
- OpenAlex
- W4394938955
- Cited by
- 265
- References
- 76
- FWCI
- 37.43
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.