Super Resolution with Sparse Gradient-Guided Attention for Suppressing Structural Distortion

Abstract

Generative adversarial network (GAN)-based methods recover perceptually pleasant details in super resolution (SR), but they pertain to structural distortions. Recent study alleviates such structural distortions by attaching a gradient branch to the generator. However, this method compromises the perceptual details. In this paper, we propose a sparse gradient-guided attention generative adversarial network (SGAGAN), which incorporates a modified residual-in-residual sparse block (MRRSB) in the gradient branch and gradient-guided self-attention (GSA) to suppress structural distortions. Compared to the most frequently used block in GAN-based SR methods, i.e., residual-in-residual dense block (RRDB), MRRSB reduces computational cost and avoids gradient redundancy. In addition, GSA emphasizes the highly correlated features in the generator by guiding sparse gradient. It captures the semantic information by connecting the global interdependencies of the sparse gradient features in the gradient branch and the features in the SR branch. Experimental results show that SGAGAN relieves the structural distortions and generates more realistic images compared to state-of-the-art SR methods. Qualitative and quantitative evaluations in the ablation study show that combining GSA and MRRSB together has a better perceptual quality than combining self-attention alone.