Segmentation of Video Sequences using Spatial-temporal Conditional Random Fields

Segmentation of video sequences requires the segmentations of consecutive frames to be consistent with each other. We propose to use a three dimensional Conditional Random Fields (CRF) to address this problem. A triple of consecutive image frames are treated as a small 3D volume to be segmented. Our spatial-temporal CRF model combines both local discriminative features and the conditional homogeneity of labeling variables in both the spatial and the temporal domain. After training the model parameters with a small set of training data, the optimal labeling is obtained through a probabilistic inference by Sum-product loopy belief propagation. We achieve accurate segmentation results on the standard video sequences, which demonstrates the promising capability of the proposed approach.