Video Super-Resolution via Effective Spatio-Temporal Alignment Network

Abstract

Extracting spatio-temporal cues from neighbouring frames is challenging in video super-resolution (VSR). Although deformable alignment-based VSR methods have shown promise in aligning neighbouring frames with the reference frame, most existing methods rely on one or a few traditional convolutions to estimate motion offsets for spatio-temporal alignment, restricting receptive field size and alignment accuracy. To address these limitations, we propose an effective spatio-temporal alignment network (ESTA-Net) for VSR. The core component of our method is the group convolution-based alignment module (GCBAM), which utilises cascaded group convolutions to learn offsets across both the original and downsampled resolutions. By employing group convolutions rather than traditional convolutions, GCBAM enables the deformable alignment to achieve a wider receptive field with lower computational cost, thereby improving the accuracy of offset estimation. Additionally, the bi-scale alignment strategy within GCBAM enhances robustness to complex and large-scale motions. Furthermore, we introduce an attention-based feature enhancement module (AFEM) to refine the aligned features, focusing on critical details to improve reconstruction quality. Extensive experiments on standard benchmarks show that our ESTA-Net achieves superior VSR performance against other advanced methods, while maintaining a good equilibrium between model size and performance.