A Deformable Convolutional Neural Network for Video Super-Resolution

Abstract

Convolutional Neural Networks used deep architectures to achieve deep feature extraction in video super-resolution. However, they suffered from challenges of rapid motion and complex scenes in video super-resolution. In this paper, we present a deformable convolutional neural network for video super-resolution (DVSRNet). DVSRNet mainly contains forward and backward feature propagation blocks (FPBs), feature enhancement blocks (FEBs), a feature fusion block (FFB), and a reconstruction block (RB). FPBs can leverage temporal sequence information to capture rich temporal dimensional information in video super-resolution. To restore detailed information, an optical flow technique guided a CNN to align the obtained structural information of different frames to reduce motion-induced blur and artifacts. To address deformable videos from motioned objects, two FEBs utilized deformable convolutions to adaptively correct misaligned objects to improve spatial continuity of videos. To improve reliability of obtained videos, an FFB is used to integrate relations of different video frames from forward and backward propagations. Finally, an RB via upsampling operations and a residual learning technique is used to construct high-quality videos. Experimental results demonstrate that our DVSRNet exhibits superior performance on multiple public datasets for video super-resolution. Its codes can be available at https://github.com/leyoukai/DVSRNet.