Gaussian Splatting Based on Mamba Interaction for Arbitrary Scale Image Super Resolution

Abstract

Recently Gaussian Splatting has shown great potential in arbitrary scale super resolution over implicit neural representation with continuous feature expression ability and high rendering speed. But the Gaussian expression ability is constrained by fixed positions, the global Gaussian interaction based on self-attention improves the accuracy of Gaussian parameters but leads to large computational overhead. To address these problems, we propose GMSR, which introduces a set of Gaussian embeddings and initializes them based on window attention and encoded features, allowing them to interact locally and globally respectively. Specifically, based on the state space models, we learn the long-range dependencies of Gaussian embeddings within and across the windows, using the Hilbert scanning mechanism to maintain local continuity. To further emphasize key information, we calibrate the weights of embedded channels based on attention mechanism. Experimental results on three public datasets demonstrate that GMSR has achieved significant improvements in reconstruction effects and computational efficiency.