A hidden Markov model based visual speech synthesizer

Abstract

This paper describes a hidden Markov model (HMM) based visual synthesizer designed to assist persons with impaired hearing. This synthesizer builds on results in the area of audio-visual speech recognition. We describe how a correlation HMM can be used to integrate independent acoustic and visual HMMs for speech-to-visual synthesis. Our results show that an HMM correlating model can significantly improve synchronization errors versus techniques which compensate for rate differences through scaling.