Temporal asymmetry in relations of acoustic and visual features of speech

Abstract

The fine temporal structure of relations of acoustic and visual features has been investigated to improve our speech to facial animation conversion system. Mutual information of acoustic and visual features has been calculated with different time shifts. Our result shows that the movement of feature points on the face of professional lip-speakers can precede the changes of acoustic parameters even by 100 milliseconds. Considering the measured time-shifts in synchrony in our system design the quality of our speech driven animations can be improved.