Automated lip synchronisation for human-computer interaction and special effect animation

Abstract

The research presents MARTI (Man-machine Animation Real-Time Interface) for the realisation of automated special effect animation and human computer interaction. MARTI introduces novel research in a number of engineering disciplines, which include speech recognition, facial modelling, and computer animation. This interdisciplinary research utilises the latest, hybrid connectionist/hidden Markov model system, providing very accurate phone recognition and timing for speaker independent continuous speech, and expands' knowledge from the animation industry in the development of accurate facial models and automated animation. The system uses simple vocal soundtracks of human speakers to provide lip synchronisation of computer graphical facial models, to realise the first natural interface and animation system capable of high performance for real users and real-world applications.