Local Gaussian ensemble for arbitrary-scale image super-resolution

Abstract

In arbitrary-scale image super-resolution (SR), the local coordinate information is pivotal to enhancing performance through local ensemble. The previous method local implicit image function (LIIF) reconstructs pixels by using multi-layer perceptron (MLP), then refines each pixel by a weighted summation of nearby pixels (also called local ensemble), where the weight depends on the distances between the query pixel and the nearby pixels. Since the distances are fixed, so is the weighting mechanism, limiting the effectiveness of local ensemble. Furthermore, the weighted summation involves repeated reconstructions, increasing the computational cost. Orthogonal position encoding SR (OPE-SR) reduces pixel reconstruction complexity using orthogonal position encoding. However, it still relies on LIIF’s local ensemble method. Additionally, lacking scale information, OPE-SR demonstrates unstable performance across various datasets and scale factors. In this paper, we propose to conduct local ensemble in feature domain, and we present a new ensemble method, the local Gaussian ensemble (LGE), to utilize the local coordinate information more flexibly and efficiently. Specifically, we introduce learnable anisotropic 2D Gaussians for each query coordinate in the SR image, transforming normalized coordinates of nearby features into multiple Gaussian weights to effectively ensemble local features. Then a scale-aware deep MLP is applied only once for pixel reconstruction. Extensive experiments demonstrate that our LGE significantly reduces computational costs during both training and inference while delivering performance comparable to the existing local ensemble method. Moreover, our method consistently outperforms the existing parameter-free approach in terms of efficiency and stability across various benchmark datasets and scale factors.