Decoder-side Affine Model Refinement for Video Coding beyond VVC

Abstract

Decoder-side motion vector refinement (DMVR) was adopted into versatile video coding (VVC) and later was further improved in the enhance compression model (ECM) to improve the inter prediction accuracy by refining the motion vectors (MVs) derived from previously coded blocks in merge mode based on bilateral matching. However, DMVR can only be applied to blocks coded with translational motion. Affine motion compensation as supported by VVC can capture more complex motion and thus increases inter prediction accuracy, but DMVR is not applied to blocks coded with affine motion in VVC. In this paper, it is proposed to refine the affine model for affine merge coded blocks at the decoder side. The proposed method includes two steps, of which the first step refines the base MV and the second step refines the non-translation parameters. Experimental results show that by only applying base MV refinement, the proposed method achieves overall {0.15% (Y), 0.06% (U), 0.11% (V)} Bjontegaard delta bit-rate (BD-rate) reduction, and by applying both base MV and non-translation parameter refinement, the proposed method achieves overall {0.27% (Y), 0.15% (U), 0.19% (V)} BD-rate reduction in random access (RA) configuration. Due to the good trade-off between performance and complexity, the base MV refinement was adopted in ECM-7.0 and non-translation parameter refinement is currently being studied in exploration experiments (EE).