Voice conversion based on continuous frequency warping and magnitude scaling

In this paper, we present a novel spectrum mapping method — Continuous Frequency Warping and Magnitude Scaling (CFWMS) for voice conversion under the Joint Density Gaussian Mixture Model (JDGMM) framework. JDGMM is a mature clustering technique that models the joint probability density of speech signals from paired speakers. The conventional JDGMM-based approaches morph the spectral features via least square optimization. However, the speech quality is degenerated as the converted features are blurred by statistical smoothing and the uncorrelated conversion functions between adjacent frames cause noticeable distortion. To this end, CFWMS proposes a twofold frame-level conversion method — Frequency Warping and Magnitude Scaling (FWMS). FWMS directly operates on signals in the frequency domain without statistical smoothing. Moreover, a trajectory limitation strategy is introduced to renovate the discontinuities between adjacent frames. Note that the proposed solution does not require global information of sentences, making it feasible for low latency (e.g. real-time) applications. The experimental results show significantly improvements in terms of the speech quality and the perceptual identity.