The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Regularizing Hyperspectral and Multispectral Image Fusion by CNN Denoiser
Abstract
Hyperspectral image (HSI) and multispectral image (MSI) fusion, which fuses a low-spatial-resolution HSI (LR-HSI) with a higher resolution multispectral image (MSI), has become a common scheme to obtain high-resolution HSI (HR-HSI). This article presents a novel HSI and MSI fusion method (called as CNN-Fus), which is based on the subspace representation and convolutional neural network (CNN) denoiser, i.e., a well-trained CNN for gray image denoising. Our method only needs to train the CNN on the more accessible gray images and can be directly used for any HSI and MSI data sets without retraining. First, to exploit the high correlations among the spectral bands, we approximate the desired HR-HSI with the low-dimensional subspace multiplied by the coefficients, which can not only speed up the algorithm but also lead to more accurate recovery. Since the spectral information mainly exists in the LR-HSI, we learn the subspace from it via singular value decomposition. Due to the powerful learning performance and high speed of CNN, we use the well-trained CNN for gray image denoising to regularize the estimation of coefficients. Specifically, we plug the CNN denoiser into the alternating direction method of multipliers (ADMM) algorithm to estimate the coefficients. Experiments demonstrate that our method has superior performance over the state-of-the-art fusion methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
(2010) · International Conference on Machine Learning · Cited by 13,995
Image denoising by sparse 3-D transform-domain collaborative filtering
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.901238 · Cited by 9,283
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662206 · Cited by 8,983
A universal image quality index
Wang, Z., & Bovik, A. C. (2002) · IEEE Signal Processing Letters · doi:10.1109/97.995823 · Cited by 5,757
Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries
Elad, M., & Aharon, M. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.881969 · Cited by 5,321
Some Mathematical Notes on Three-Mode Factor Analysis
Tucker, L. R. (1966) · Psychometrika · doi:10.1007/bf02289464 · Cited by 3,492
FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising
Zhang, K., Zuo, W., & Zhang, L. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2839891 · Cited by 2,678
Vertex component analysis: a fast algorithm to unmix hyperspectral data
Nascimento, J. M. P., & Dias, J. M. B. (2005) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2005.844293 · Cited by 2,669
Weighted Nuclear Norm Minimization with Application to Image Denoising
Gu, S., Zhang, L., Zuo, W., & Feng, X. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.366 · Cited by 2,316 · cited 1×
“Many state-of-the-art algorithms have been proposed to solve the denoising problem, such as BM3D [43], K-SVD denoising method [48], weighted nuclear norm method [49], and deep CNN-based denoising method [15], [45].”
MethodDeep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art
Zhang, L., Zhang, L., & Du, B. (2016) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2016.2540798 · Cited by 2,208
Imaging Spectroscopy and the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS)
Green, R. O., Eastwood, M. L., Sarture, C. M., Chrien, T. G., Aronsson, M., Chippendale, B. J., Faust, J. A., Pavri, B. E., Chovit, C. J., Solis, M., et al. (1998) · Remote Sensing of Environment · doi:10.1016/s0034-4257(98)00064-9 · Cited by 1,851
Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm
Yuhas, R. H., Goetz, A. F. H., & Boardman, J. W. (1992) · NASA Technical Reports Server (NASA) · Cited by 1,259
Coupled Nonnegative Matrix Factorization Unmixing for Hyperspectral and Multispectral Data Fusion
Yokoya, N., Yairi, T., & Iwasaki, A. (2012) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2011.2161320 · Cited by 1,180
Plug-and-Play priors for model based reconstruction
Venkatakrishnan, S. V., Bouman, C. A., & Wohlberg, B. (2013) · 2013 IEEE Global Conference on Signal and Information Processing · doi:10.1109/globalsip.2013.6737048 · Cited by 1,178
FFDNet: Toward a fast and flexible solution for CNN-Based image denoising
Zhang, K., Zuo, W., & Zhang, L. (2018) · TUScholarShare (Temple University) · Cited by 916
A Convex Formulation for Hyperspectral Image Superresolution via Subspace-Based Regularization
Simoes, M., Bioucas-Dias, J., Almeida, L. B., & Chanussot, J. (2015) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2014.2375320 · Cited by 826
Hyperspectral Pansharpening: A Review
Loncan, L., de Almeida, L. B., Bioucas-Dias, J. M., Briottet, X., Chanussot, J., Dobigeon, N., Fabre, S., Liao, W., Licciardi, G. A., Simoes, M., et al. (2015) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2015.2440094 · Cited by 824
Weighted Nuclear Norm Minimization and Its Applications to Low Level Vision
Gu, S., Xie, Q., Meng, D., Zuo, W., Feng, X., & Zhang, L. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0930-5 · Cited by 816
Hyperspectral and Multispectral Data Fusion: A comparative review of the recent literature
Yokoya, N., Grohnfeldt, C., & Chanussot, J. (2017) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2016.2637824 · Cited by 712
Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation
Wei, Q., Bioucas-Dias, J., Dobigeon, N., & Tourneret, J.-Y. (2015) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2014.2381272 · Cited by 698
Fusing Hyperspectral and Multispectral Images via Coupled Sparse Tensor Factorization
Li, S., Dian, R., Fang, L., & Bioucas-Dias, J. M. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2836307 · Cited by 585
Self-citationHyperspectral Image Super-Resolution via Non-Negative Structured Sparse Representation
Dong, W., Fu, F., Shi, G., Cao, X., Wu, J., Li, G., & Li, X. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2542360 · Cited by 571
Multimodal Classification of Remote Sensing Images: A Review and Future Directions
Gomez-Chova, L., Tuia, D., Moser, G., & Camps-Valls, G. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2449668 · Cited by 508
Salient Band Selection for Hyperspectral Image Classification via Manifold Ranking
Wang, Q., Lin, J., & Yuan, Y. (2016) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2477537 · Cited by 483
Fast Fusion of Multi-Band Images Based on Solving a Sylvester Equation
Wei, Q., Dobigeon, N., & Tourneret, J.-Y. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2458572 · Cited by 474
Hyperspectral Anomaly Detection With Attribute and Edge-Preserving Filters
Kang, X., Zhang, X., Li, S., Li, K., Li, J., & Benediktsson, J. A. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2710145 · Cited by 469
Self-citationHyperspectral Super-Resolution by Coupled Spectral Unmixing
Lanaras, C., Baltsavias, E., & Schindler, K. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.409 · Cited by 424
Deep Hyperspectral Image Sharpening
Dian, R., Li, S., Guo, A., & Fang, L. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2798162 · Cited by 412
Self-citationHigh-resolution hyperspectral imaging via matrix factorization
Kawakami, R., Matsushita, Y., Wright, J., Ben-Ezra, M., Tai, Y.-W., & Ikeuchi, K. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995457 · Cited by 370
Fast Hyperspectral Image Denoising and Inpainting Based on Low-Rank and Sparse Representations
Zhuang, L., & Bioucas-Dias, J. M. (2018) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2018.2796570 · Cited by 363
Hyperspectral Image Super-Resolution via Non-local Sparse Tensor Factorization
Dian, R., Fang, L., & Li, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.411 · Cited by 330
Self-citationSparse Spatio-spectral Representation for Hyperspectral Image Super-resolution
Akhtar, N., Shafait, F., & Mian, A. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_5 · Cited by 284
MAP Estimation for Hyperspectral Image Resolution Enhancement Using an Auxiliary Sensor
Hardie, R. C., Eismann, M. T., & Wilson, G. L. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2004.829779 · Cited by 274
Exploiting Spectral and Spatial Information in Hyperspectral Urban Data With High Resolution
Dell'Acqua, F., Gamba, P., Ferrari, A., Palmason, J. A., Benediktsson, J. A., & Arnason, K. (2004) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2004.837009 · Cited by 267
Hyperspectral Super-Resolution: A Coupled Tensor Factorization Approach
Kanatsoulis, C. I., Fu, X., Sidiropoulos, N. D., & Ma, W.-K. (2018) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2018.2876362 · Cited by 238
Hyperspectral image unsupervised classification by robust manifold matrix factorization
Zhang, L., Zhang, L., Du, B., You, J., & Tao, D. (2019) · Information Sciences · doi:10.1016/j.ins.2019.02.008 · Cited by 237
Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network
Yang, J., Zhao, Y.-Q., & Chan, J. C.-W. (2018) · Remote Sensing · doi:10.3390/rs10050800 · Cited by 229
Weighted Low-Rank Tensor Recovery for Hyperspectral Image Restoration
Chang, Y., Yan, L., Zhao, X.-L., Fang, H., Zhang, Z., & Zhong, S. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2983102 · Cited by 224
Detection and Analysis of the Intestinal Ischemia Using Visible and Invisible Hyperspectral Imaging
Akbari, H., Kosugi, Y., Kojima, K., & Tanaka, N. (2010) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2010.2049110 · Cited by 196
Hyperspectral Super-Resolution of Locally Low Rank Images From Complementary Multisource Data
Veganzones, M. A., Simoes, M., Licciardi, G., Yokoya, N., Bioucas-Dias, J. M., & Chanussot, J. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2496263 · Cited by 180
Multiband Image Fusion Based on Spectral Unmixing
Wei, Q., Bioucas-Dias, J., Nicolas Dobigeon, B., Tourneret, J., Chen, M., & Godsill, S. (2016) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2598784 · Cited by 172
R-FUSE: Robust Fast Fusion of Multiband Images Based on Solving a Sylvester Equation
Wei, Q., Nicolas Dobigeon, B., Tourneret, J., Bioucas-Dias, J., & Godsill, S. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2608858 · Cited by 115
Self-Similarity Constrained Sparse Representation for Hyperspectral Image Super-Resolution
Han, X., Shi, B., & Zheng, Y. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2855418 · Cited by 111
A MAP-Based Approach for Hyperspectral Imagery Super-Resolution
Irmak, H., Akar, G. B., & Yüksel, S. E. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2814210 · Cited by 85
DOES multispectral / hyperspectral pansharpening improve the performance of anomaly detection?
Qu, Y., Qi, H., Ayhan, B., Kwan, C., & Kidd, R. D. (2017) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss.2017.8128408 · Cited by 83
A Super-Resolution and Fusion Approach to Enhancing Hyperspectral Images
Kwan, C., Choi, J. H., Chan, S. H., Zhou, J., & Budavari, B. (2018) · Remote Sensing · doi:10.3390/rs10091416 · Cited by 83
Bendoumi, M. A., He, M., & Mei, S. (2014) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2014.2298056 · Cited by 82
Hyperspectral and Multispectral Image Fusion Based on Local Low Rank and Coupled Spectral Unmixing
Zhou, Y., Feng, L., Hou, C., & Kung, S. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2718728 · Cited by 79
Zhong, P., & Wang, R. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2293061 · Cited by 72
Nonparametric Coupled Bayesian Dictionary and Classifier Learning for Hyperspectral Classification
Akhtar, N., & Mian, A. S. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2742528 · Cited by 61
Hyperspectral and Multispectral Image Fusion Using Cluster-Based Multi-Branch BP Neural Networks
Han, X., Yu, J. S., Luo, J., et al. (2019) · Remote Sensing · doi:10.3390/rs11101173 · Cited by 53
Ferraris, V., Dobigeon, N., Wei, Q., & Chabert, M. (2017) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2017.2692645 · Cited by 35
Cited by
Image fusion meets deep learning: A survey and perspective
Zhang, H., Xu, H., Tian, X., Jiang, J., & Ma, J. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.06.008 · Cited by 834
Deep learning in multimodal remote sensing data fusion: A comprehensive review
Li, J., Hong, D., Gao, L., Yao, J., Zheng, K., Zhang, B., & Chanussot, J. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2022.102926 · Cited by 630
Graph Information Aggregation Cross-Domain Few-Shot Learning for Hyperspectral Image Classification
Zhang, Y., Li, W., Zhang, M., Wang, S., Tao, R., & Du, Q. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3185795 · Cited by 340
SSR-NET: Spatial–Spectral Reconstruction Network for Hyperspectral and Multispectral Image Fusion
Zhang, X., Huang, W., Wang, Q., & Li, X. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3018732 · Cited by 292
EDiffSR: An Efficient Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution
Xiao, Y., Yuan, Q., Jiang, K., He, J., Jin, X., & Zhang, L. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3341437 · Cited by 280
Metadata
- Authors
- Dian, R., Li, S., & Kang, X.
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2021-03
- Volume
- 32
- Issue
- 3
- Pages
- 1124-1135
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W3016410830
- Cited by
- 328
- References
- 54
- FWCI
- 25.61
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.