Gradient degradation-aware rate control for VVC using Nash equilibrium

Abstract

The $\lambda$-domain rate control in Versatile Video Coding (VVC) achieves remarkable performance, enabling higher visual quality under stringent bit constraints. However, for high-resolution video, a significant portion of Coding Tree Units (CTUs) are the skipped block, and using the updated parameters from skip CTUs may lead to unreasonable bit allocation. This paper presents a novel rate control method for VVC that separately allocates bits to skip and non-skip CTUs. Skip CTUs’ bits can be calculated at once based on the frame’s target bits per pixel (bpp) without separate calculations. For non-skip CTUs, we first formulate the bit allocation problem as a Nash equilibrium problem inspired by the definition of the game theory. Subsequently, we design a utility function based on Gradient Magnitude Similarity Deviation (GMSD) to quantify the degradation in gradient information caused by encoding. The $\lambda$ parameter for bit allocation of non-skip CTUs is calculated accordingly. The proposed method is implemented in VTM 13.0, and experimental results confirm its effectiveness in enhancing visual quality and achieving significant reductions in bit rates.