Video frame interpolation based on invertible neural network for Internet of things

Abstract

Video frame interpolation focuses on directly synthesizing intermediate frames by utilizing inter-frame changes, relying heavily on large, high-frame-rate video datasets for supervised training, which imposes significant demands on bandwidth and computational resources in the Internet of Everything (IoE) environments. Since video frame rate down and up conversion are inverse processes, an Invertible Neural Network (INN) provides an efficient solution by ensuring lossless and symmetrical information transfer in forward and backward processes. This paper introduces a self-supervised video frame rate conversion method based on an INN to reconstruct missing intermediate frames. By leveraging an invertible coupling structure, the model encodes the spatio-temporal features of sparse input frames into a Gaussian distribution, which effectively simulates the frame rate downsampling process. Through the network's invertibility and lossless processing capabilities, the intermediate frames are then reconstructed through reverse inference. This approach captures missing information from a Gaussian prior, ensuring stability and realism in the generated frames. Extensive experiments on public video datasets show that the proposed method surpasses existing state-of-the-art algorithms in accuracy and efficiency, offering superior visual quality, faster processing speeds, and reduced model parameters, especially for high-frame-rate recovery.