An analysis of the data efficiency in Tacotron2 speech synthesis system

Abstract

This paper introduces an evaluation of the amount of data required by the Tacotron2 speech synthesis model in order to achieve a good quality output synthesis. We evaluate the capabilities of the model to adapt to new speakers in very limited data scenarios. We use three Romanian speakers for which we gathered at most 5 minutes of speech, and use this data to fine tune a large pre-trained model over a few training epochs. We look at the performance of the system by evaluating the intelligibility, naturalness and speaker similarity measures, as well as performing an analysis of the trade-off between speech quality and overfitting of the network.The results show that the Tacotron2 network can replicate the identity of a speaker from as little as one speech sample. Also it inherently learns individual grapheme representations, such that if the training data is carefully selected to present all the common graphemes in the language, the adaptation data requirements can be significantly lowered.