Controllable Space-Time Video Super-Resolution via Enhanced Bidirectional Flow Warping

Abstract

Space-time video super-resolution targets to increase a given video's frame rate and resolution simultaneously. Al-though existing approaches have made great progress, most of them still suffer from the inaccurate approximation of large motions or fail to generate temporal consistent motion trajectory. To alleviate these problems, we carefully review the characteris-tics of different optical flow warping strategies, integrating and enhancing them to achieve more robust capabilities for handling extreme motions and time-modulated interpolation. Specifically, we utilize enhanced backward warping to perform alignment, mine space-time information across low resolution input frames, and propose an enhanced forward warping strategy to interpolate arbitrary intermediate frames. Furthermore, the proposed model can be trained end-to-end and produce intermediate results at any time by merely supervising the center moment. Experimental results show that the proposed algorithm performs favorably against the state-of-the-art methods in objective metrics and subjective visual effects.