Multi-reference super-resolution reconstruction of remote sensing images based on hierarchical similarity mapping

Abstract

To make full use of the details from multi-reference images and improve the quality of super-resolution reconstruction of remote sensing images, a multi-reference super-resolution reconstruction of remote sensing images based on hierarchical similarity mapping is proposed. It is very important in both military and civilian fields. Firstly, one low resolution image and three reference images are used as the input of VGG network to extract their feature maps at 4 × , 2 × , and 1 × scales. These feature maps at each scale are respectively blocked and used as a set of inputs in subsequent operations. Specifically, the low resolution features are divided into $N_i$blocks, and each block is further divided into $N_c$sub-feature-blocks. And the $N_m$reference image features are divided into $N_r$ sub-feature-blocks. Then the $N_c$low-resolution sub-feature blocks are mapped for similarity with the reference features within the range of all reference sub-feature blocks, individual reference features, and all reference image features. The outputs of each layer are then iteratively mapped with the low-resolution features as inputs for next layers. Thus the final features include information from all the reference images and low-resolution image. Subsequently, an adaptive transfer module with multi-reference features and channel attention is used to match and transfer the information of each reference image, while achieving edge smoothing and noise filtering between different reference features. Finally, the quadruple super-resolution reconstruct result is got from the multi-scale feature fusion module and decoder. Experimental results show that our improvements can reconstruct better super-resolution results with more details for utilizing information of multi-reference images, which is superior to single image super-resolution methods and single reference super-resolution methods.