Chinese sign language recognition based on video sequence appearance modeling

Abstract

According to the temporal characteristic and the spatial characteristic of video sequence, a novel recognition method of sign language spatio-temporal appearance modeling is introduced for the vision-based multi-features classifier of Chinese sign language recognition. The obvious advantage with such a novel approach is that we can exclude some skin-like object and tracking the moving recognized hand more precisely in the sign language video sequence. Experiments demonstrate that this new modeling method is feasible and robust. At first, dynamic sign language appearance modeling is done, and then classification method of SVMs for recognition is brought into use. Experimentation with 30 groups of the Chinese manual alphabet images is conducted and the results prove that this appearance modeling method is simple, efficient, and effective for characterizing hand gestures, and the SVMs method has excellent classification and generalization ability in solving learning problem with small training set of sample in sign language recognition. The experimentation shows that linear kernel function is suitable for sign language recognition, and the best recognition rate of 99.7% of letter ‘F’ image group is achieved.