Fusion of Bi-Temporal Zhuhai-1 Orbita Hyperspectral and Multiseason Sentinel-2 Remote Sensing Imagery for Semantic Change Detection Based on Dual-Path 3DCNN-LSTM

Abstract

Remote sensing imagery plays a crucial role in monitoring Earth’s surface changes. With the advancement of computational power, convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have been increasingly applied in change detection tasks. This letter proposes a novel approach, the dual-path 3-D convolutional RNN, which integrates 3-D CNNs and long short-term memory (3DCNN-LSTM) to fuse bi-temporal Zhuhai-1 Orbita hyperspectral and multiseason Sentinel-2 remote sensing imagery for semantic change detection. The proposed method extracts joint spectral-spatial features using 3-D convolution layers. Subsequently, bi-temporal features and multiseason features are spliced as one time sequence and fed to one LSTM modules (referred to as Str1). In another strategy (Str2), multisource features are fed separately to two LSTM modules and fused in fully connected layers. Experiments demonstrate that Str1 and Str2 achieve better performance than Siamese convolutional multiple-layers RNN (SiamCRNN). Str1 obtained the overall accuracy (OA) of 99.12%, kappa coefficient (KC) of 98.77%, and ${F}1$ score of 98.91%, achieving improvements of 4.81%, 6.7%, and 6.51%, respectively, compared to SiamCRNN.