Multi-Scale Transformer for Image Restoration

Abstract

Although Transformer-based image restoration methods have demonstrated impressive performance, existing Transformers still insufficiently exploit multiscale information. Previous non-Transformer-based studies have shown that incorporating multiscale features is crucial for improving restoration results. In this paper, we propose a multiscale Transformer (MST) that captures cross-scale attention among tokens, thereby effectively leveraging the multiscale patch recurrence prior of natural images. Furthermore, we introduce a channel-gate feed-forward network (CGFN) to enhance inter-channel information aggregation and reduce channel redundancy. To simultaneously utilise global, local and multiscale features, we design a multitype feature integration block (MFIB). Extensive experiments on both image super-resolution and HEVC compressed video artefact reduction demonstrate that the proposed MST achieves state-of-the-art performance. Ablation studies further verify the effectiveness of each proposed module.