Hand tracking and segmentation via graph cuts and dynamic model in sign language videos

Abstract

In this paper, we propose a new method for hands tracking and segmentation based on augmented graph cuts and dynamic model in sign language videos. We focus on resolving three problems which are fast hand motion capture, hand over face and hand occlusions. At first, an effective dynamic model for state prediction is used. This dynamic model can correctly predict the location of hand which has a rapid movement and quick shape deformation. Then, new energy terms are augmented into the energy function in graph cuts. The additional terms are inspired by multi cues, such as color, motion and spatial-temporal information. Finally, we construct the graph and achieve the hand segmentation in successive frames using min-cut/max-flow algorithm. We evaluate our algorithm in a real American Sign Language video from Purdue ASL Database. Besides, our method can be easily extended to track objects with similar color.