End-to-End Neural Video Compression: A Review

Abstract

The pervasive presence of video content has spurred the development of advanced technologies to manage, process, and deliver high-quality content efficiently. Video compression is crucial in providing high-quality video services under limited network and storage capacities, traditionally achieved through hybrid codecs. However, as these frameworks reach a performance bottleneck with compression gains becoming harder to achieve with conventional methods, Deep Neural Networks (DNNs) offer a promising alternative. By leveraging DNNs’ nonlinear representation capacity, these networks can enhance compression efficiency and visual quality. Neural Video Coding (NVC) has recently received significant attention, with Neural Image Coding models surpassing traditional codecs in compression ratios. Therefore, this survey explores the state-of-the-art in NVC, examining recent works, frameworks, and the potential of this innovative approach to revolutionize video compression. We identify that NVC models have come a long way since the first proposals and currently are on par in compression efficiency with the latest hybrid codec, VVC. Still, many improvements are required to enable the practical usage of NVC, such as hardware-friendly development to enable faster inference and execution on mobile and energy-constrained devices.