Enhancing video frame interpolation with region of motion loss and self-attention mechanisms: A dual approach to address large, nonlinear motions

Abstract

Video frame interpolation is particularly challenging when dealing with large and non-linear object motions, often resulting in poor frame quality and motion artifacts. In this study, we introduce a novel dual-approach methodology for video frame interpolation that effectively addresses these complexities. Our method consists of two key components: a Region of Motion (RoM) loss and self-attention mechanisms. The RoM loss is designed to spotlight significant movements within frames. This is achieved by employing feature-matching techniques that assign tailored weights during the training process, ensuring that areas of intense motion are given priority. This is facilitated by the computation of optical flow, which identifies crucial feature points and highlights regions of significant motion for targeted enhancement. Our method incorporates self-attention mechanisms to maintain inter-frame continuity while emphasizing the unique attributes of individual frames. The self-attention scores reduce motion discrepancies and enhance the distinctiveness and texture quality of each frame. We validate the efficacy of our approach through extensive evaluations on benchmark datasets, including Vimeo-90K, Middlebury, UCF101, and SNU-Film.