Research on the realization of multilingual speech synthesis and cross-lingual sound cloning in Tibetan

Abstract

Speech synthesis technology has achieved rapid development in recent years, and the speech synthesized has reached a very high level of intelligibility and naturalness. However, once the speech to synthesize is mixed with words from other languages, the quality of the speech will be greatly compromised. Imagine how great it would be if one can hear foreign place names pronounced in the corresponding language very smoothly when navigating. Given that most of us can only speak one or two foreign languages due to time constraints, it would make a big difference to speak a foreign language in your voice. Implementing it using the existing monolingual model has difficulty in collecting sound data from someone who speaks different languages at the same time. Using only monolingual corpora, our model can do a good job of cloning one person’s voice and realizing code-switching. The parameters of the encoder are generated by a separate network based on a specific language vector, the parameter generator module consists of several specific parameter generators, each of which takes a language vector as input to generate the parameters of a layer of an encoder in a given language and to complete the sound cloning, we use an adversarial speaker classifier to eliminate specific speaker information in model training and the information will be going back in the synthesis. Our model performs very well on code-switching task and can synthesize high-quality speech with high accuracy.