Detail Injection-Based Deep Convolutional Neural Networks for Pansharpening
Open accessThe DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Abstract
The fusion of high spatial resolution panchromatic (PAN) data with simultaneously acquired multispectral (MS) data with the lower spatial resolution is a hot topic, which is often called pansharpening. In this article, we exploit the combination of machine learning techniques and fusion schemes introduced to address the pansharpening problem. In particular, deep convolutional neural networks (DCNNs) are proposed to solve this issue. The latter is combined first with the traditional component substitution and multiresolution analysis fusion schemes in order to estimate the nonlinear injection models that rule the combination of the upsampled low-resolution MS image with the extracted details exploiting the two philosophies. Furthermore, inspired by these two approaches, we also developed another DCNN for pansharpening. This is fed by the direct difference between the PAN image and the upsampled low-resolution MS image. Extensive experiments conducted both at reduced and full resolutions demonstrate that this latter convolutional neural network outperforms both the other detail injection-based proposals and several state-of-the-art pansharpening methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Image Super-Resolution Using Deep Convolutional Networks
Dong, C., Loy, C. C., He, K., & Tang, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2439281 · Cited by 10,105
The Laplacian pyramid as a compact image code
Burt, P., & Adelson, E. (1983) · IEEE Transactions on Communications · doi:10.1109/tcom.1983.1095851 · Cited by 6,077 · cited 1×
“(AWLP) [14], the “à-trous” wavelet transform [15], the Laplacian pyramid (LP) [16], the generalized Laplacian pyramid (GLP) [17], [18], the GLP with robust regression [19], and the GLP with full-scale regression (GLP-Reg) [20].”
MethodThe discrete wavelet transform: wedding the a trous and Mallat algorithms
Shensa, M. J. (1992) · IEEE Transactions on Signal Processing · doi:10.1109/78.157290 · Cited by 1,994
Lucien Wald, A., Ranchin, T., & Mangolini, M. (1997) · HAL · Cited by 1,362
A Critical Comparison Among Pansharpening Algorithms
Vivone, G., Alparone, L., Chanussot, J., Dalla Mura, M., Garzelli, A., Licciardi, G. A., Restaino, R., & Wald, L. (2015) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2014.2361734 · Cited by 1,303
Self-citationDiscrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm
Yuhas, R. H., Goetz, A. F. H., & Boardman, J. W. (1992) · NASA Technical Reports Server (NASA) · Cited by 1,259
Pansharpening by Convolutional Neural Networks
Masi, G., Cozzolino, D., Verdoliva, L., & Scarpa, G. (2016) · Remote Sensing · doi:10.3390/rs8070594 · Cited by 1,152
A wavelet transform method to merge Landsat TM and SPOT panchromatic data
Zhou, J., Civco, D. L., & Silander, J. A. (1998) · International Journal of Remote Sensing · doi:10.1080/014311698215973 · Cited by 979
Liu, J. G. (2000) · International Journal of Remote Sensing · doi:10.1080/014311600750037499 · Cited by 965
MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery
Aiazzi, B., Alparone, L., Baronti, S., Garzelli, A., & Selva, M. (2006) · Photogrammetric Engineering & Remote Sensing · doi:10.14358/pers.72.5.591 · Cited by 918
Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest
Alparone, L., Wald, L., Chanussot, J., Thomas, C., Gamba, P., & Bruce, L. M. (2007) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2007.904923 · Cited by 886
Self-citationAiazzi, B., Alparone, L., Baronti, S., & Garzelli, A. (2002) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2002.803623 · Cited by 855
Otazu, X., Gonzalez-Audicana, M., Fors, O., & Nunez, J. (2005) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2005.856106 · Cited by 850
PanNet: A Deep Network Architecture for Pan-Sharpening
Yang, J., Fu, X., Hu, Y., Huang, Y., Ding, X., & Paisley, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.193 · Cited by 802
A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening
Yuan, Q., Wei, Y., Meng, X., Shen, H., & Zhang, L. (2018) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2018.2794888 · Cited by 649
Thomas, C., Ranchin, T., Wald, L., & Chanussot, J. (2008) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2007.912448 · Cited by 632
Self-citationA New Adaptive Component-Substitution-Based Satellite Image Fusion by Using Partial Replacement
Choi, J., Yu, K., & Kim, Y. (2011) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2010.2051674 · Cited by 580
Optimal MMSE Pan Sharpening of Very High Resolution Multispectral Images
Garzelli, A., Nencini, F., & Capobianco, L. (2008) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2007.907604 · Cited by 570
Data Fusion. Definitions and Architectures - Fusion of Images of Different Spatial Resolutions
Wald, L. (2002) · HAL (Le Centre pour la Communication Scientifique Directe) · Cited by 520
Boosting the Accuracy of Multispectral Image Pansharpening by Learning a Deep Residual Network
Wei, Y., Yuan, Q., Shen, H., & Zhang, L. (2017) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2017.2736020 · Cited by 478
Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusion
Ma, J., Yu, W., Chen, C., Liang, P., Guo, X., & Jiang, J. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.04.006 · Cited by 471
A New Pan-Sharpening Method Using a Compressed Sensing Technique
Li, S., & Yang, B. (2011) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2010.2067219 · Cited by 444
A Sparse Image Fusion Algorithm With Application to Pan-Sharpening
Zhu, X. X., & Bamler, R. (2013) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2012.2213604 · Cited by 438
Target-Adaptive CNN-Based Pansharpening
Scarpa, G., Vitale, S., & Cozzolino, D. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2817393 · Cited by 383
Hypercomplex Quality Assessment of Multi/Hyperspectral Images
Garzelli, A., & Nencini, F. (2009) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2009.2022650 · Cited by 378
A New Pansharpening Algorithm Based on Total Variation
Palsson, F., Sveinsson, J. R., & Ulfarsson, M. O. (2014) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2013.2257669 · Cited by 373
A New Pan-Sharpening Method With Deep Neural Networks
Huang, W., Xiao, L., Wei, Z., Liu, H., & Tang, S. (2015) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2014.2376034 · Cited by 334
Spectral-Spatial Unified Networks for Hyperspectral Image Classification
Xu, Y., Zhang, L., Du, B., & Zhang, F. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2827407 · Cited by 316
Full Scale Regression-Based Injection Coefficients for Panchromatic Sharpening
Vivone, G., Restaino, R., & Chanussot, J. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2819501 · Cited by 306
Self-citationWu, C., Du, B., Cui, X., & Zhang, L. (2017) · Remote Sensing of Environment · doi:10.1016/j.rse.2017.07.009 · Cited by 302
Feature Learning Using Spatial-Spectral Hypergraph Discriminant Analysis for Hyperspectral Image
Luo, F., Du, B., Zhang, L., Zhang, L., & Tao, D. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2810806 · Cited by 301
Robust Band-Dependent Spatial-Detail Approaches for Panchromatic Sharpening
Vivone, G. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2906073 · Cited by 296
Self-citationMapping forest degradation in the Eastern Amazon from SPOT 4 through spectral mixture models
Souza, C. (2003) · Remote Sensing of Environment · doi:10.1016/j.rse.2002.08.002 · Cited by 294
Challenges and Opportunities of Multimodality and Data Fusion in Remote Sensing
Dalla Mura, M., Prasad, S., Pacifici, F., Gamba, P., Chanussot, J., & Benediktsson, J. A. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2462751 · Cited by 286
Self-citationPansharpening via Detail Injection Based Convolutional Neural Networks
He, L., Rao, Y., Li, J., Chanussot, J., Plaza, A., Zhu, J., & Li, B. (2019) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2019.2898574 · Cited by 255
Self-citationA Pansharpening Method Based on the Sparse Representation of Injected Details
Vicinanza, M. R., Restaino, R., Vivone, G., Dalla Mura, M., & Chanussot, J. (2015) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2014.2331291 · Cited by 218
Self-citationA New Pansharpening Method Based on Spatial and Spectral Sparsity Priors
He, X., Condat, L., Bioucas-Dias, J. M., Chanussot, J., & Xia, J. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2333661 · Cited by 175
Self-citationA Variational Approach for Pan-Sharpening
Fang, F., Li, F., Shen, C., & Zhang, G. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2258355 · Cited by 171
SIRF: Simultaneous Satellite Image Registration and Fusion in a Unified Framework
Chen, C., Li, Y., Liu, W., & Huang, J. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2456415 · Cited by 168
Deng, L.-J., Feng, M., & Tai, X. (2018) · Information Fusion · doi:10.1016/j.inffus.2018.11.014 · Cited by 155
Self-citationDeep Multiscale Detail Networks for Multiband Spectral Image Sharpening
Fu, X., Wang, W., Huang, Y., Ding, X., & Paisley, J. W. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2996498 · Cited by 155
Pansharpening Based on Semiblind Deconvolution
Vivone, G., Simões, M., Mura, M. D., Restaino, R., Bioucas‐Dias, J. M., Licciardi, G., & Chanussot, J. (2014) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2014.2351754 · Cited by 147
Self-citationHigh-Quality Bayesian Pansharpening
Wang, T., Fang, F., Li, F., & Zhang, G. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2866954 · Cited by 133
SIRF: Simultaneous Satellite Image Registration and Fusion in A Unified Framework
Chen, C., Member, S., Li, Y., Liu, W., & Huang, J. (2016) · Cited by 132
R-FUSE: Robust Fast Fusion of Multiband Images Based on Solving a Sylvester Equation
Wei, Q., Nicolas Dobigeon, B., Tourneret, J., Bioucas-Dias, J., & Godsill, S. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2608858 · Cited by 115
Deng, L.-J., Vivone, G., Guo, W., Mura, M. D., & Chanussot, J. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2839531 · Cited by 99
Self-citationPansharpening: Context-Based Generalized Laplacian Pyramids by Robust Regression
Vivone, G., Maranò, S., & Chanussot, J. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.2974806 · Cited by 97
Self-citationA residual convolutional neural network for pan-shaprening
Rao, Y., He, L., & Zhu, J. (2017) · International Workshop on Remote Sensing with Intelligent Processing (RSIP) · doi:10.1109/rsip.2017.7958807 · Cited by 74
Residual Encoder–Decoder Conditional Generative Adversarial Network for Pansharpening
Shao, Z., Lu, Z., Ran, M., Fang, L., Zhou, J., & Zhang, Y. (2019) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2019.2949745 · Cited by 73
HPGAN: Hyperspectral Pansharpening Using 3-D Generative Adversarial Networks
Xie, W., Cui, Y., Li, Y., Lei, J., Du, Q., & Li, J. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.2994238 · Cited by 69
Liu, L., Wang, J., Zhang, E., Li, B., Zhu, X., Zhang, Y., & Peng, J. (2020) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2020.2981695 · Cited by 41
Multi spectral image fusion by deep convolutional neural network and new spectral loss function
Eghbalian, S., & Ghassemian, H. (2018) · International Journal of Remote Sensing · doi:10.1080/01431161.2018.1452074 · Cited by 37
Cited by
Image fusion meets deep learning: A survey and perspective
Zhang, H., Xu, H., Tian, X., Jiang, J., & Ma, J. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.06.008 · Cited by 834
Deep learning in multimodal remote sensing data fusion: A comprehensive review
Li, J., Hong, D., Gao, L., Yao, J., Zheng, K., Zhang, B., & Chanussot, J. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2022.102926 · Cited by 630
Self-citationMachine Learning in Pansharpening: A benchmark, from shallow to deep networks
Deng, L.-J., Vivone, G., Paoletti, M. E., Scarpa, G., He, J., Zhang, Y., Chanussot, J., & Plaza, A. (2022) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2022.3187652 · Cited by 253
Self-citation
Metadata
- Authors
- Deng, L.-J., Vivone, G., Jin, C., & Chanussot, J.
- Venue
- IEEE Transactions on Geoscience and Remote Sensing
- Published
- 2021-08
- Volume
- 59
- Issue
- 8
- Pages
- 6995-7010
- ISSN
- 0196-2892 (ISSN-L)
- 1558-0644
- OpenAlex
- W3096904276
- Cited by
- 332
- References
- 53
- FWCI
- 19.17
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.