Fast Inter Mode Decision Algorithm Based on Residual Feature

〈Summary〉 H.264/AVC introduces the variable block size motion estimation (VBSME), which brings huge computational cost of the encoder. In this paper, a novel fast inter mode decision algorithm for H.264/AVC has been proposed. The proposed algorithm evaluates the modes based on residual feature. The residual is obtained after the motion search of P16 × 16 mode or P8 × 8 mode. And then basing on the extracted residual feature, the complexity and similarity are evaluated for the inter mode decision. According to the evaluation of similarity between different sub-blocks and the complexity of each sub-block, the most possible inter modes for current block is chosen to be conducted. In the worst case, the proposed whole scheme of inter mode decision algorithm only conducts 4 modes, which is much more effective than conducting all the 8 modes in conventional approach. The simulation results show that, comparing to JM14.1, on average, the proposed algorithm achieves 57.98% and 55.72% time-saving on CIF and 720p sequences respectively, with equivalent 0.219 dB PSNR drop and 5.55% bit rate increase for CIF and 0.107 dB PSNR drop and 3.53% bit rate increase for 720p. Compared to existing inter mode decision algorithm, proposed algorithm achieves 10.68% and 13.26% timing-reduction on CIF and 720p sequences respectively with less performance loss.