Improve GAN-based Neural Vocoder using Truncated Pointwise Relativistic Least Square GAN

Abstract

Neural vocoders are widely utilized in modern text-to-speech (TTS) and voice conversion (VC) systems due to their high generation quality and fast inference speed. Recently, GAN-based neural vocoders have attracted great interest due to their lightweight and parallel structures, which enable them to generate a high-fidelity waveform in a real-time manner. Most existing GAN-based vocoders adopt the Least Square GAN (LSGAN) training framework. In this paper, we analyze the weaknesses of the LSGAN waveform synthesis framework and, inspired by Relativistic GAN, propose a simple yet effective variant of the LSGAN framework, named Truncated Pointwise Relativistic LSGAN (T-PRLSGAN). In this method, we consider the pointwise truism score distribution of real and fake wave segments and combine the Mean Squared error (MSE) loss with the proposed truncated pointwise relative discrepancy loss to increase the difficulty of the generator to fool the discriminator, leading to improved audio generation quality and stability. To demonstrate the effectiveness and generalization ability of our method, subjective and objective experiments have been conducted based on Avocodo, UnivNet, HiFiGAN, ParallelWaveGAN, and MelGAN vocoders, which show a consistent performance boost over those typical LSGAN-based vocoders. Moreover, our T-PRLSGAN can support multiple types of discriminators, i.e. multi-scale wave discriminator(MSD), multi-period discriminator(MPD), and multi-resolution spectrogram discriminator(MRD), without modifying their architecture or inference speed.