A perceptual preprocessor to segment video for motion estimation

Abstract

Summary form only given. The objective of motion estimation and motion compensation is to reduce the temporal redundancy between adjacent pictures in a video sequence. Motion estimation is usually performed by calculating an error metric, such as mean absolute error (MAE), for each block in the current frame over a displaced region in the previously reconstructed frame. The motion vector is attained as the displacement having the minimum error metric. Although this achieves minimum-MAE in the residual block, it does not necessarily result in the best perceptual quality since the MAE is not always a good indicator of video quality. In low bit rate video coding, the overhead in sending the motion vectors becomes a significant proportion of the total data rate. The minimum-MAE motion vector may not achieve the minimum joint entropy for coding the residual block and motion vector, and thus may not achieve the best compression efficiency. In this paper, we attack these problems by introducing a perceptual preprocessor which takes advantage of the insensitivity of the human visual system (HVS) to mild changes in pixel intensity in order to segment the video into regions according to the perceptibility of the picture changes. Our preprocessor can exploit the local psycho-perceptual properties of the HVS because it is designed to segment video in the spatio-temporal pixel domain. The associated computational complexity for the segmentation in the spatio-temporal pixel domain is very small. With the information of segmentation, we then determine which macroblocks require motion estimation.