Beneficial and flowing: Omni efficient feature aggregation network for image super-resolution

Abstract

Image super-resolution (SR) is a classic low-level vision task that reconstructs high-resolution (HR) images from low-resolution (LR) ones. Recent Transformer-based SR methods have achieved remarkable performance through modeling long-range dependencies. However, there are two critical challenges in existing approaches: (1) Invalid feature interaction and homogeneous aggregation scheme; (2) Network feature propagation blocking. These challenges imply that the potential of Self-Attention (SA) and Transformer architecture is still not fully exploited. To this end, we propose a novel Transformer model, Omni Efficient Aggregation Transformer (OEAT), boosting SR performance by mining and aggregating efficient information across the omni-dimensions and omni-stages. Specifically, we first design an Omni Efficient Aggregation Self-Attention (OEASA) with a local–global-channel feature interaction scheme, to aggregate heterogeneous features from multi-scales and multi-dimensions while facilitating information flow. Specifically, we design a global semantic SA based on content self-similarity in the spatial dimension and an adaptive sparse channel SA in the channel dimension, efficiently gathering the most useful feature for accurate reconstruction. Furthermore, we design a simple yet effective Omni Feature Fusion (OFF) to enhance global groups feature fusion and inter-layer adaptive feature aggregation, thus introducing more critical information for an accurate reconstruction. Extensive experiments demonstrate that our OEAT outperforms recent state-of-the-art methods both quantitatively and qualitatively.