Real-time video object segmentation in H.264 compressed domain

In this study the authors proposed a real-time video object segmentation algorithm that works in the H.264 compressed domain. The algorithm utilises the motion information from the H.264 compressed bit stream to identify background motion model and moving objects. In order to preserve spatial and temporal continuity of objects, Markov random field (MRF) is used to model the foreground field. Quantised transform coefficients of the residual frame are also used to improve segmentation result. Experimental results show that the proposed algorithm can effectively extract moving objects from different kinds of sequences. The computation time of the segmentation process is merely about 16 ms per frame for CIF size frame, allowing the algorithm to be applied in real-time applications.