Enhanced Down/Up-Sampling-Based Video Coding Using the Residual Compensation

Down/up-sampling-based video coding is an effective strategy to cope with limited bandwidth for transmission. In recent years, CNN-based super-resolution methods have also been integrated into the above coding strategy. However, it raises a problem of how to balance the calculation and reconstruction quality. In order to alleviate this problem, we propose an enhanced down/up-sampling-based video coding scheme, in which residual data is used to compensate the unrecovered details of the reconstructed video. We first down-sample the video prior to encoding at the encoder and then obtain the residual data by subtracting the super-resolution reconstructed video frame from the original video frame. The residual data is encoded and transmitted together with the low-resolution video sequence. The decoder first reconstructs the encoded low-resolution video with CNN-based super resolution method, and then uses residual data to compensate the missing details of the reconstructed video. To further enhance the perceived quality, we use a quality enhancement network to enhance the quality-compensated video. The experimental results show that the proposed scheme achieves superior quality improvement compared with HEVC anchor and the general down/up-sampling schemes.