Semantic Super-Resolution Using a Transformer Model

Abstract

This paper proposes an effective method to improve the performance of SwinIR, a vision Transformer-based super-resolution neural network model, by introducing a Transformer decoder with learnable category queries. The decoder allows to extract semantic information of each dataset belonging to different categories (e.g., text and face); the semantic information can improve category-specific texture reconstruction in the process of super-resolution. Experiments were conducted using decoders of different architectures to analyze the performance of the proposed method. The experimental results confirm that the use of decoder can improve the quality of super-resolution images produced by SwinIR qualitatively and quantitatively, although improvements may vary depending on the depth of the decoder and how semantic information is applied.