Action Segmentation via Robust Constraint Matrix Factorization Clustering Framework

Action understanding, which has been applied in a wide range of intelligent systems, has gained much attention for its better performance. However, the existing literature mainly focuses on supervised or semi-supervised frameworks, and effectively designing an unsupervised clustering method for action segmentation is still a challenging problem. In this paper, we propose a novel unsupervised clustering method for action segmentation based on robust structure constraint matrix factorization and the Ncut method by utilizing the similarity information among neighboring frames. Considering that the true neighboring frames are likely to share more similarity in action sequences, a useful structure constraint was designed to guide the action representation learning process. With the semi-nonnegative matrix factorization, more comprehensive low-dimensional representation of actions can be learned. Then, the similarity graph can be obtained from this new representation, and the final action segmentation results can be obtained by graph cut methods. Experiments on several real action datasets demonstrate that the proposed method outperforms state-of-the-art methods.