Robust Sequence-to-sequence Voice Conversion for Electrolaryngeal Speech Enhancement in Noisy and Reverberant Conditions.

Abstract

Electrolaryngeal (EL) speech, an artificial speech produced by an electrolarynx for laryngectomees, lacks essential phonetic features, and differs in temporal structure from normal speech, resulting in poor naturalness and intelligibility. To address this deficiency, sequence-to-sequence (seq2seq) voice conversion (VC) models have been applied in converting EL speech to normal speech (EL2SP), showing some promising performances. However, previous studies mostly focus on converting clean EL speech, thereby restricting the further applicability in real-world scenarios, especially when the EL speech is inevitably interfered with background noise and reverberation. In light of this, we suggest novel training techniques based on seq2seq VC to enhance the robustness of real-world EL2SP. We first pretrain a normal-to-normal seq2seq VC model based on a text-to-speech model. Then, a two-stage fine-tuning is conducted by effectively using pseudo noisy and reverberant EL speech data artificially generated from only a small amount of original clean data available. Several design options are investigated to figure out the effectiveness of our method. The significant improvements presented in experimental results indicate that our method can non-trivially handle both clean and noisy-reverberant EL speech, enhancing the robustness of EL2SP in real-world scenarios.