SqSFill : Joint spatial and spectral learning for high-fidelity image inpainting

Abstract

Image inpainting has made significant progress due to recent advances in deep learning. However, most generative inpainting networks face challenges such as producing blurry results that lack high-frequency details or introducing inconsistent structures. To address these issues, we propose a novel transformer-based approach, SqSFill, which exploits rich information in both spatial and spectral domains. Specifically, SqSFill incorporates Rectified Frequency Feature Extractor (RecFFE) in the early layers of the network to capture fine-grained details by leveraging frequency information, guided by frequency loss. Moreover, we design a Scout Attention Block with linear complexity to replace vanilla self-attention, thereby effectively capturing long-range dependencies with lower computational cost. By integrating the RecFFE and Scout Attention Block, SqSFill is able to generate both coherent structures and sharp textures. Extensive experiments demonstrate the proposed SqSFill achieves superior results, outperforming previous state-of-the-art approaches with fewer parameters.