Improving Automatic Evaluation of Mandarin Pronunciation with Speaker Adaptive Training (SAT) and MLLR Speaker Adaption

Abstract

Automatic pronunciation evaluation (APE) can be implemented with a speech recognition model trained by standard, "golden" speakers. The pronunciation accuracy is then measured with the Goodness of Pronunciation (GOP) as reported in our earlier work [1]. In this paper, we investigate two main strategies for improving the evaluation: speaker adaptive training (SAT) for reducing the speaker-specific characteristics in model training and MLLR-based speaker adaptation in evaluation for reducing mismatch between the trained model and a testing speaker. Overall, the proposed strategies improve the correlation between evaluations made by APE and human experts from 0.69 to 0.76, approaching the upper bound value of 0.78 among human expert evaluators. Additionally, APE also shows a consistency of 0.93 better than the consistency of 0.83 among human experts.