A Visual Speech Feature to Indentify the Speaking States from Video

Abstract

The paper proposes a kind of visual speech feature for the speaking mouth images from the video combining clues of the shape and local teeth texture. The geometric feature we proposed based on the computing the Euclidian distant between each the feature point around the inner and outer lip. The local texture with G and B components as baseline is employed to calculate the color moment to describe the visibility of teeth. The weighted fusion is used to combine the two features. The k-mean algorithm is utilized to analyze the feature performance according to evaluate the clustering results. The results show that with G and B color component to derive the local texture to model the teeth visibility are better than the others and our feature has higher ability to perceive the visemes than the PCA and geometric feature only.