Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 229

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

Open access
Yang, Jingxiang, Zhao, Yong-Qiang, Chan, Jonathan Cheung-Wai · 2018 · Remote Sensing

Abstract

Enhancing the spatial resolution of hyperspectral image (HSI) is of significance for applications. Fusing HSI with a high resolution (HR) multispectral image (MSI) is an important technology for HSI enhancement. Inspired by the success of deep learning in image enhancement, in this paper, we propose a HSI-MSI fusion method by designing a deep convolutional neural network (CNN) with two branches which are devoted to features of HSI and MSI. In order to exploit spectral correlation and fuse the MSI, we extract the features from the spectrum of each pixel in low resolution HSI, and its corresponding spatial neighborhood in MSI, with the two CNN branches. The extracted features are then concatenated and fed to fully connected (FC) layers, where the information of HSI and MSI could be fully fused. The output of the FC layers is the spectrum of the expected HR HSI. In the experiment, we evaluate the proposed method on Airborne Visible Infrared Imaging Spectrometer (AVIRIS), and Environmental Mapping and Analysis Program (EnMAP) data. We also apply it to real Hyperion-Sentinel data fusion. The results on the simulated and the real data demonstrate that the proposed method is competitive with other state-of-the-art fusion methods.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/rs10050800

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 12 references

Research works printed but not linked: 40

  1. Chang, C.I. (2016). Hyperspectral target detection. Real-Time Progressive Hyperspectral Image Processing, Springer. · doi:10.1007/978-1-4419-6187-7
  2. Yokoya, N., Chan, J.C.W., and Segl, K. (2016). Potential of resolution-enhanced hyperspectral data for mineral mapping using simulated EnMAP and Sentinel-2 images. Remote Sens., 8. · DOI · Paper page
  3. Yang, 2017, Learning and transferring deep joint spectral-spatial features for hyperspectral classification, IEEE Trans. Geosci. Remote Sens., 55, 4729, 10.1109/TGRS.2017.2698503 · DOI · Paper page
  4. Yokoya, 2017, Hyperspectral and multispectral data fusion: A comparative review of the recent literature, IEEE Geosci. Remote Sens. Mag., 5, 29, 10.1109/MGRS.2016.2637824 · DOI · Paper page
  5. Zhu, 2018, A novel multi-modality image fusion method based on image decomposition and sparse representation, Inf. Sci., 432, 516, 10.1016/j.ins.2017.09.010 · DOI · Paper page
  6. Yokoya, 2012, Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion, IEEE Trans. Geosci. Remote Sens., 50, 528, 10.1109/TGRS.2011.2161320 · DOI · Paper page
  7. Yokoya, 2013, Cross-calibration for data fusion of EO-1/Hyperion and Terra/ASTER, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 6, 419, 10.1109/JSTARS.2012.2208449 · doi:10.1109/jstars.2012.2208449
  8. Lanaras, C., Baltsavias, E., and Schindler, K. (2015, January 7–13). Hyperspectral super-resolution by coupled spectral unmixing. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile. · DOI · Paper page
  9. Huang, 2014, Spatial and spectral image fusion using sparse matrix factorization, IEEE Trans. Geosci. Remote Sens., 52, 1693, 10.1109/TGRS.2013.2253612 · DOI · Paper page
  10. Zhu, 2013, A sparse image fusion algorithm with application to pan-sharpening, IEEE Trans. Geosci. Remote Sens., 51, 2827, 10.1109/TGRS.2012.2213604 · DOI · Paper page
  11. Zhu, 2016, Exploiting joint sparsity for pansharpening: The J-SparseFI algorithm, IEEE Trans. Geosci. Remote Sens., 54, 2664, 10.1109/TGRS.2015.2504261 · doi:10.1109/tgrs.2015.2504261
  12. Akhtar, N., Shafait, F., and Mian, A. (2014, January 6–12). Sparse spatio-spectral representation for hyperspectral image super-resolution. Proceedings of the 13th European Conference on Computer Vision, Zurich, Switzerland. · DOI · Paper page
  13. Akhtar, N., Shafait, F., and Mian, A. (2015, January 7–12). Bayesian sparse representation for hyperspectral image super resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA. · DOI · Paper page
  14. Wei, 2015, Hyperspectral and multispectral image fusion based on a sparse representation, IEEE Trans. Geosci. Remote Sens., 53, 3658, 10.1109/TGRS.2014.2381272 · DOI · Paper page
  15. Almeida, 2015, A convex formulation for hyperspectral image superresolution via subspace-based regularization, IEEE Trans. Geosci. Remote Sens., 53, 3373, 10.1109/TGRS.2014.2375320 · DOI · Paper page
  16. Zhang, 2017, Multispectral and hyperspectral image fusion based on group spectral embedding and low-rank factorization, IEEE Trans. Geosci. Remote Sens., 55, 1363, 10.1109/TGRS.2016.2623626 · doi:10.1109/tgrs.2016.2623626
  17. Eismann, 2005, Hyperspectral resolution enhancement using high-resolution multispectral imagery with arbitrary response functions, IEEE Trans. Geosci. Remote Sens., 43, 455, 10.1109/TGRS.2004.837324 · DOI · Paper page
  18. Selva, 2015, Hyper-sharpening: A first approach on SIM-GA data, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 8, 3008, 10.1109/JSTARS.2015.2440092 · DOI · Paper page
  19. Kwan, 2017, Blind quality assessment of fused worldview-3 images by using the combinations of pansharpening and hypersharpening paradigms, IEEE Geosci. Remote Sens. Lett., 14, 1835, 10.1109/LGRS.2017.2737820 · DOI
  20. Loncan, 2015, Hyperspectral pansharpening: A review, IEEE Geosci. Remote Sens. Mag., 3, 27, 10.1109/MGRS.2015.2440094 · DOI · Paper page
  21. Zhang, 2016, Deep learning for remote sensing data: A technical tutorial on the state of the art, IEEE Geosci. Remote Sens. Mag., 4, 22, 10.1109/MGRS.2016.2540798 · DOI · Paper page
  22. Makantasis, K., Karantzalos, K., Doulamis, A., and Doulamis, N. (2015, January 26–31). Deep supervised learning for hyperspectral data classification through convolutional neural networks. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium, Milan, Italy. · DOI · Paper page
  23. Zhao, 2016, Spectral-spatial feature extraction for hyperspectral image classification: A dimension reduction and deep learning approach, IEEE Trans. Geosci. Remote Sens., 54, 4544, 10.1109/TGRS.2016.2543748 · DOI · Paper page
  24. Maltezos, 2017, Deep convolutional neural networks for building extraction from orthoimages and dense image matching point clouds, J. Appl. Remote Sens., 11, 42620, 10.1117/1.JRS.11.042620 · DOI
  25. Dong, C., Loy, C.C., and Tang, X. (2016, January 11–14). Accelerating the super-resolution convolutional neural network. Proceedings of the 14th European Conference on Computer Vision, Amsterdam, The Netherlands. · DOI · Paper page
  26. Zhao, Y., Wang, R., Dong, W., Jia, W., Yang, J., Liu, X., and Gao, W. (2017). GUN: Gradual upsampling network for single image super-resolution. Comput. Vis. Pattern Recognit. · DOI
  27. Mei, S., Yuan, X., Ji, J., Zhang, Y., Wan, S., and Du, Q. (2017). Hyperspectral Image Spatial Super-Resolution via 3D Full Convolutional Neural Network. Remote Sens., 9. · DOI · Paper page
  28. Wang, C., Liu, Y., and Bai, X. (2017). Deep Residual Convolutional Neural Network for Hyperspectral Image Super-Resolution. International Conference on Image and Graphics, Springer. · DOI
  29. Hu, 2017, Hyperspectral Image Super-Resolution by Spectral Difference Learning and Spatial Error Correction, IEEE Geosci. Remote Sens. Lett., 14, 1825, 10.1109/LGRS.2017.2737637 · DOI
  30. Masi, G., Cozzolino, D., Verdoliva, L., and Scarpa, G. (2016). Pansharpening by convolutional neural networks. Remote Sens., 8. · DOI · Paper page
  31. Wei, 2017, Boosting the Accuracy of multispectral image pansharpening by learning a deep residual network, IEEE Geosci. Remote Sens. Lett., 14, 1795, 10.1109/LGRS.2017.2736020 · DOI · Paper page
  32. Yang, J., Fu, X., Hu, Y., Huang, Y., Ding, X., and Paisley, J. (2017, January 22–29). PanNet: A deep network architecture for pan-sharpening. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Venice, Italy. · DOI · Paper page
  33. Bottou, L. (2012). Stochastic gradient descent tricks. Neural Networks: Tricks of the Trade, Springer. · DOI · Paper page
  34. Yang, J., Zhao, Y., Yi, C., and Chan, J.C.W. (2017). No-reference hyperspectral image quality assessment via quality-sensitive features learning. Remote Sens., 9. · DOI
  35. Zhang, 2011, FSIM: A feature similarity index for image quality assessment, IEEE Trans. Image Process., 20, 2378, 10.1109/TIP.2011.2109730 · DOI · Paper page
  36. Melgani, 2004, Classification of hyperspectral remote sensing images with support vector machines, IEEE Trans. Geosci. Remote Sens., 42, 1778, 10.1109/TGRS.2004.831865 · DOI · Paper page
  37. Chang, C.C., and Lin, C.J. (2011). LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol., 2. · DOI · Paper page
  38. Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. How transferable are features in deep neural networks? In Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal, QC, Canada, 8–13 December 2014; pp.
  39. Wang, 2018, Transferring deep object and scene representations for event recognition in still images, Int. J. Comput. Vis., 126, 390, 10.1007/s11263-017-1043-5 · DOI
  40. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Darrell, T. (2014, January 3–7). Caffe: Convolutional architecture for fast feature embedding. Proceedings of the 22nd ACM International Conference on Multimedia, Orlando, FL, USA. · DOI · Paper page

Cited by

See all 229

Metadata

Authors
Yang, J., Zhao, Y.-Q., & Chan, J. C.-W.
Venue
Remote Sensing
Publisher
MDPI AG
Published
2018-05-21
Volume
10
Issue
5
Pages
800
ISSN
2072-4292 (ISSN-L)
OpenAlex
W2803825432
Cited by
229
References
12
FWCI
17.22
Type
journal article
Language
en
Open access
gold
Funders
  • National Natural Science Foundation of China (61771391, 61371152)
  • Fundamental Research Funds for the Central Universities (3102015ZY045)
  • National Natural Science Foundation of China and South Korean National Research Foundation Joint Funded Cooperation Program (61511140292)
  • China Scholarship Council (201506290120)
  • Innovation Foundation of Doctor Dissertation of Northwestern Polytechnical University (CX201621)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network. https://app.cochat.ai/papers/10.3390/rs10050800
The paper
Yang, J., Zhao, Y.-Q., & Chan, J. C.-W. (2018). Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network. Remote Sensing. https://doi.org/10.3390/rs10050800

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.