GPU-based fast motion estimation for on-the-fly encoding of computer-generated video streams

Abstract

Motion estimation is known to be one of the most expensive tasks in video coding as it is usually performed through blind search-based methods. However, in the particular case of computer-generated video, the rendering stage provides useful information to speed up the process. In this paper, we propose a fast motion estimation algorithm, designed to run completely inside the GPU, to compute the optical flow required to estimate motion vectors at the same time as the graphical rendering process by using high-level information about the objects, viewpoints and effects that define each frame. The proposed method takes advantage of GPU parallelism and avoids bottlenecks in the CPU-GPU communication as the entire rendering and encoding process is performed completely inside the GPU. Avoiding search, motion estimation has very little overhead, negligible when compared with rendering and (the rest of the) video encoding costs while maintaining reasonably good quality. Performance evaluation is done with a CUDA implementation for MPEG-2 video, though results are valid for other formats, and it has been tested as part of the rendering and encoding engine of a real-world system that provides server-side visually-rich interactive applications to lightweight clients equipped with standard MPEG video decoders.