Coupled Temporal Variation Information Estimation and Resolution Enhancement for Remote Sensing Spatial–Temporal–Spectral Fusion

Abstract

Spatial-temporal-spectral fusion (STSF) of remote sensing imagery can produce data with the highest spatial and spectral resolution, only as well as fine temporal resolution, by integrating images with complementary information in both the temporal and spectral domains. Accuracy of temporal variation is an important guarantee for achieving fidelity fusion in STSF. However, current STSF methods estimate the temporal variation only by utilizing the temporal variation between observed multispectral image (MSI) and the relationship between MSI and hyperspectral image (HSI), which is difficult to obtain accurate temporal variation. To address this problem, this paper proposes a coupled temporal variation information estimation and resolution enhancement for remote sensing image spatial-temporal-spectral fusion (CTVRE-STSF). The temporal variation information estimation model estimates the temporal variation of the target image, while the resolution enhancement model provides additional constraints for estimating the temporal variation. For the temporal variation information reconstruction model, we build a temporal variation information estimation based on a generalized linear mixed model and use the temporal variation between MSIs. In addition, a resolution enhancement model is constructed to estimate the temporal variation of the target image by incorporating relevant prior knowledge. The introduction of the resolution enhancement model in the prior provides additional constraints on the estimation of the temporal variation high-dimensional information, thus facilitating the resolution improvement. Experimental results on two real datasets demonstrate the effectiveness and superiority of our proposed method over current state-of-the-art methods, especially in terms of spectral fidelity.