Audiovisual articulatory inversion based on Gaussian Mixture Model (GMM)

Abstract

In this study, we examined articulatory inversion using audiovisual information based on Gaussian Mixture Model (GMM). In this method the joint distribution of the articulatory movement and audio (and/or visual) data are modelled via a mixture of Gaussians. The conditional expected value of the GMM is used as regression function between the audio (and/orvisual) and ar-ticulatory spaces. We also examined various fusion methods in order to combine acoustic and visual information in articula-tory inversion. The fusion methods improve the performance of articulatory inversion.