Adaptive conventional-learning video signal compression framework using texture fulfillment

With the explosive growth of various real-time video applications, it has been recognized that video compression is crucial for efficient data storage and transmission. In the low bit-rate scenario, the conventional video coding standards are possible to have small distortion but contain hand-crafted artifacts. Meanwhile, unlike conventional approaches, learning-based end-to-end techniques emphasize perceptual quality, which usually leads to relatively large distortion. To address this problem, this work proposes a new video compression framework with texture fulfillment (named ACLTF) by collaborating with conventional and learning-based video coding technologies. We separate and compress a video sequence to a small-portion key pack and a dominated non-key pack. On the encoder side, the key pack is compressed with low distortion and high texture information but a relatively low compression ratio by conventional learning. The non-key pack is highly compacted by applying semantic segment-based layered coding. On the decoder side, semantic-based self-enhancement and multi-frame enhancement are applied to transfer and interpolate the high-texture information from the key pack to the non-key pack. All the existing video coding systems are compatible with the proposed ACLTF. Experimental results verified that by applying ACLTF to the latest video coding standards (H.266/VVC, H.265/HEVC) and learning-based video coding, it significantly enhanced the compression results by 18.08%–47.57% BD rate over the standard HEVC in all-intra and improved by 6.08%–15.78% BD rate over the standard VVC in low delay.