Structured modeling based on generalized variable parameter HMMs and speaker adaptation

Abstract

It is a challenging task that to handle ambient variable acoustic factors in automatic speech recognition (ASR) system. The ambient variable noise and the distinct acoustic factors among speakers are two key issues for recognition task. To solve these problems, we present a new framework for robust speech recognition based on structured modeling, using generalized variable parameter HMMs (GVP-HMMs) and unsupervised speaker adaptation (SA) to compensate the mismatch from environment and speaker variability. GVP-HMMs can explicitly approximate the continuous trajectory of Gaussian component mean, variance and linear transformation parameter with a polynomial function against the varying noise level. In recognition stage, MLLR transform captures general relationship between the original model set and the current speaker, which could help in removing the effects of unwanted speaker factors. The effectiveness of the proposed approach is confirmed by evaluation experiment on a medium vocabulary Mandarin recognition task.