The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network
Open accessAbstract
Enhancing the spatial resolution of hyperspectral image (HSI) is of significance for applications. Fusing HSI with a high resolution (HR) multispectral image (MSI) is an important technology for HSI enhancement. Inspired by the success of deep learning in image enhancement, in this paper, we propose a HSI-MSI fusion method by designing a deep convolutional neural network (CNN) with two branches which are devoted to features of HSI and MSI. In order to exploit spectral correlation and fuse the MSI, we extract the features from the spectrum of each pixel in low resolution HSI, and its corresponding spatial neighborhood in MSI, with the two CNN branches. The extracted features are then concatenated and fed to fully connected (FC) layers, where the information of HSI and MSI could be fully fused. The output of the FC layers is the spectrum of the expected HR HSI. In the experiment, we evaluate the proposed method on Airborne Visible Infrared Imaging Spectrometer (AVIRIS), and Environmental Mapping and Analysis Program (EnMAP) data. We also apply it to real Hyperion-Sentinel data fusion. The results on the simulated and the real data demonstrate that the proposed method is competitive with other state-of-the-art fusion methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Image Super-Resolution Using Deep Convolutional Networks
Dong, C., Loy, C. C., He, K., & Tang, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2439281 · Cited by 10,105
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662206 · Cited by 8,983
Accurate Image Super-Resolution Using Very Deep Convolutional Networks
Kim, J., Lee, J. K., & Lee, K. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.182 · Cited by 7,870
Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources
Zhu, X. X., Tuia, D., Mou, L., Xia, G.-S., Zhang, L., Xu, F., & Fraundorfer, F. (2017) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2017.2762307 · Cited by 3,400
Deep learning-based classification of hyperspectral data
Chen, Y., Lin, Z., Zhao, X., Wang, G., & Gu, Y. (2014) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2014.2329330 · Cited by 2,679
A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening
Yuan, Q., Wei, Y., Meng, X., Shen, H., & Zhang, L. (2018) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2018.2794888 · Cited by 649
Research works printed but not linked: 40
- Chang, C.I. (2016). Hyperspectral target detection. Real-Time Progressive Hyperspectral Image Processing, Springer. · doi:10.1007/978-1-4419-6187-7
- Yokoya, N., Chan, J.C.W., and Segl, K. (2016). Potential of resolution-enhanced hyperspectral data for mineral mapping using simulated EnMAP and Sentinel-2 images. Remote Sens., 8. · DOI · Paper page
- Yang, 2017, Learning and transferring deep joint spectral-spatial features for hyperspectral classification, IEEE Trans. Geosci. Remote Sens., 55, 4729, 10.1109/TGRS.2017.2698503 · DOI · Paper page
- Yokoya, 2017, Hyperspectral and multispectral data fusion: A comparative review of the recent literature, IEEE Geosci. Remote Sens. Mag., 5, 29, 10.1109/MGRS.2016.2637824 · DOI · Paper page
- Zhu, 2018, A novel multi-modality image fusion method based on image decomposition and sparse representation, Inf. Sci., 432, 516, 10.1016/j.ins.2017.09.010 · DOI · Paper page
- Yokoya, 2012, Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion, IEEE Trans. Geosci. Remote Sens., 50, 528, 10.1109/TGRS.2011.2161320 · DOI · Paper page
- Yokoya, 2013, Cross-calibration for data fusion of EO-1/Hyperion and Terra/ASTER, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 6, 419, 10.1109/JSTARS.2012.2208449 · doi:10.1109/jstars.2012.2208449
- Lanaras, C., Baltsavias, E., and Schindler, K. (2015, January 7–13). Hyperspectral super-resolution by coupled spectral unmixing. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile. · DOI · Paper page
- Huang, 2014, Spatial and spectral image fusion using sparse matrix factorization, IEEE Trans. Geosci. Remote Sens., 52, 1693, 10.1109/TGRS.2013.2253612 · DOI · Paper page
- Zhu, 2013, A sparse image fusion algorithm with application to pan-sharpening, IEEE Trans. Geosci. Remote Sens., 51, 2827, 10.1109/TGRS.2012.2213604 · DOI · Paper page
- Zhu, 2016, Exploiting joint sparsity for pansharpening: The J-SparseFI algorithm, IEEE Trans. Geosci. Remote Sens., 54, 2664, 10.1109/TGRS.2015.2504261 · doi:10.1109/tgrs.2015.2504261
- Akhtar, N., Shafait, F., and Mian, A. (2014, January 6–12). Sparse spatio-spectral representation for hyperspectral image super-resolution. Proceedings of the 13th European Conference on Computer Vision, Zurich, Switzerland. · DOI · Paper page
- Akhtar, N., Shafait, F., and Mian, A. (2015, January 7–12). Bayesian sparse representation for hyperspectral image super resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA. · DOI · Paper page
- Wei, 2015, Hyperspectral and multispectral image fusion based on a sparse representation, IEEE Trans. Geosci. Remote Sens., 53, 3658, 10.1109/TGRS.2014.2381272 · DOI · Paper page
- Almeida, 2015, A convex formulation for hyperspectral image superresolution via subspace-based regularization, IEEE Trans. Geosci. Remote Sens., 53, 3373, 10.1109/TGRS.2014.2375320 · DOI · Paper page
- Zhang, 2017, Multispectral and hyperspectral image fusion based on group spectral embedding and low-rank factorization, IEEE Trans. Geosci. Remote Sens., 55, 1363, 10.1109/TGRS.2016.2623626 · doi:10.1109/tgrs.2016.2623626
- Eismann, 2005, Hyperspectral resolution enhancement using high-resolution multispectral imagery with arbitrary response functions, IEEE Trans. Geosci. Remote Sens., 43, 455, 10.1109/TGRS.2004.837324 · DOI · Paper page
- Selva, 2015, Hyper-sharpening: A first approach on SIM-GA data, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 8, 3008, 10.1109/JSTARS.2015.2440092 · DOI · Paper page
- Kwan, 2017, Blind quality assessment of fused worldview-3 images by using the combinations of pansharpening and hypersharpening paradigms, IEEE Geosci. Remote Sens. Lett., 14, 1835, 10.1109/LGRS.2017.2737820 · DOI
- Loncan, 2015, Hyperspectral pansharpening: A review, IEEE Geosci. Remote Sens. Mag., 3, 27, 10.1109/MGRS.2015.2440094 · DOI · Paper page
- Zhang, 2016, Deep learning for remote sensing data: A technical tutorial on the state of the art, IEEE Geosci. Remote Sens. Mag., 4, 22, 10.1109/MGRS.2016.2540798 · DOI · Paper page
- Makantasis, K., Karantzalos, K., Doulamis, A., and Doulamis, N. (2015, January 26–31). Deep supervised learning for hyperspectral data classification through convolutional neural networks. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Milan, Italy. · DOI · Paper page
- Zhao, 2016, Spectral-spatial feature extraction for hyperspectral image classification: A dimension reduction and deep learning approach, IEEE Trans. Geosci. Remote Sens., 54, 4544, 10.1109/TGRS.2016.2543748 · DOI · Paper page
- Maltezos, 2017, Deep convolutional neural networks for building extraction from orthoimages and dense image matching point clouds, J. Appl. Remote Sens., 11, 42620, 10.1117/1.JRS.11.042620 · DOI
- Dong, C., Loy, C.C., and Tang, X. (2016, January 11–14). Accelerating the super-resolution convolutional neural network. Proceedings of the 14th European Conference on Computer Vision, Amsterdam, The Netherlands. · DOI · Paper page
- Zhao, Y., Wang, R., Dong, W., Jia, W., Yang, J., Liu, X., and Gao, W. (2017). GUN: Gradual upsampling network for single image super-resolution. Comput. Vis. Pattern Recognit. · DOI
- Mei, S., Yuan, X., Ji, J., Zhang, Y., Wan, S., and Du, Q. (2017). Hyperspectral Image Spatial Super-Resolution via 3D Full Convolutional Neural Network. Remote Sens., 9. · DOI · Paper page
- Wang, C., Liu, Y., and Bai, X. (2017). Deep Residual Convolutional Neural Network for Hyperspectral Image Super-Resolution. International Conference on Image and Graphics, Springer. · DOI
- Hu, 2017, Hyperspectral Image Super-Resolution by Spectral Difference Learning and Spatial Error Correction, IEEE Geosci. Remote Sens. Lett., 14, 1825, 10.1109/LGRS.2017.2737637 · DOI
- Masi, G., Cozzolino, D., Verdoliva, L., and Scarpa, G. (2016). Pansharpening by convolutional neural networks. Remote Sens., 8. · DOI · Paper page
- Wei, 2017, Boosting the Accuracy of multispectral image pansharpening by learning a deep residual network, IEEE Geosci. Remote Sens. Lett., 14, 1795, 10.1109/LGRS.2017.2736020 · DOI · Paper page
- Yang, J., Fu, X., Hu, Y., Huang, Y., Ding, X., and Paisley, J. (2017, January 22–29). PanNet: A deep network architecture for pan-sharpening. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Venice, Italy. · DOI · Paper page
- Bottou, L. (2012). Stochastic gradient descent tricks. Neural Networks: Tricks of the Trade, Springer. · DOI · Paper page
- Yang, J., Zhao, Y., Yi, C., and Chan, J.C.W. (2017). No-reference hyperspectral image quality assessment via quality-sensitive features learning. Remote Sens., 9. · DOI
- Zhang, 2011, FSIM: A feature similarity index for image quality assessment, IEEE Trans. Image Process., 20, 2378, 10.1109/TIP.2011.2109730 · DOI · Paper page
- Melgani, 2004, Classification of hyperspectral remote sensing images with support vector machines, IEEE Trans. Geosci. Remote Sens., 42, 1778, 10.1109/TGRS.2004.831865 · DOI · Paper page
- Chang, C.C., and Lin, C.J. (2011). LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol., 2. · DOI · Paper page
- Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. How transferable are features in deep neural networks? In Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal, QC, Canada, 8–13 December 2014; pp.
- Wang, 2018, Transferring deep object and scene representations for event recognition in still images, Int. J. Comput. Vis., 126, 390, 10.1007/s11263-017-1043-5 · DOI
- Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Darrell, T. (2014, January 3–7). Caffe: Convolutional architecture for fast feature embedding. Proceedings of the 22nd ACM International Conference on Multimedia, Orlando, FL, USA. · DOI · Paper page
Cited by
Deep learning in remote sensing applications: A meta-analysis and review
Ma, L., Liu, Y., Zhang, X., Ye, Y., Yin, G., & Johnson, B. A. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.04.015 · Cited by 2,456
Deep learning in multimodal remote sensing data fusion: A comprehensive review
Li, J., Hong, D., Gao, L., Yao, J., Zheng, K., Zhang, B., & Chanussot, J. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2022.102926 · Cited by 630
Deep Learning Meets Hyperspectral Image Analysis: A Multidisciplinary Review
Signoroni, A., Savardi, M., Baronio, A., & Benini, S. (2019) · Journal of Imaging · doi:10.3390/jimaging5050052 · Cited by 385
Regularizing Hyperspectral and Multispectral Image Fusion by CNN Denoiser
Dian, R., Li, S., & Kang, X. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2980398 · Cited by 328
Himeur, Y., Rimal, B., Tiwary, A., & Amira, A. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.06.003 · Cited by 288
Recent advances and new guidelines on hyperspectral and multispectral image fusion
Dian, R., Li, S., Sun, B., & Guo, A. (2021) · Information Fusion · doi:10.1016/j.inffus.2020.11.001 · Cited by 268
Metadata
- Authors
- Yang, J., Zhao, Y.-Q., & Chan, J. C.-W.
- Venue
- Remote Sensing
- Publisher
- MDPI AG
- Published
- 2018-05-21
- Volume
- 10
- Issue
- 5
- Pages
- 800
- ISSN
- 2072-4292 (ISSN-L)
- OpenAlex
- W2803825432
- Cited by
- 229
- References
- 12
- FWCI
- 17.22
- Type
- journal article
- Language
- en
- Open access
- gold
- Funders
- National Natural Science Foundation of China (61771391, 61371152)
- Fundamental Research Funds for the Central Universities (3102015ZY045)
- National Natural Science Foundation of China and South Korean National Research Foundation Joint Funded Cooperation Program (61511140292)
- China Scholarship Council (201506290120)
- Innovation Foundation of Doctor Dissertation of Northwestern Polytechnical University (CX201621)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.