BiCAPT: Bidirectional Computer-Assisted Pronunciation Training with Normalizing Flows

Computer-Assisted Pronunciation Training (CAPT) plays an important role in language learning. So far, most existing CAPT methods are discriminative and focus on detecting where the mispronunciation is. Although learners receive feedback about their current pronunciation, they may still not be able to learn the correct pronunciation. Nevertheless, there has been little discussion about speech-based teaching in CAPT. To fill this gap, we propose a novel bidirectional CAPT method to detect mispronunciations and generate the corrected pronunciations simultaneously. This correction-based feedback can better preserve the speaking style to make the learning process more personalized. In addition, we propose to adopt normalizing flows to share the latent for these two mirrored discriminative-generative tasks, making the whole model more compact. Experiments show that our method is efficient for mispronunciation detection and can naturally correct the speech under different CAPT granularity requirements.