Singing Fundamental Frequency Contour Generation Using Generalized Command-Response Model and Score-Conditional Variational Autoencoder

Abstract

This paper proposes a method for achieving physically motivated and interpretable control of fundamental frequency (F0) contour generation in singing aid systems for laryngectomees. Recently proposed variational autoencoder (VAE)-based method, VAE-SPACE, has successfully generated singing F0 contours from musical scores. However, VAE-SPACE can generate physically deviated F0 contours. Moreover, to represent fluctuations in F0 contours, VAE-SPACE requires manual adjustment of noise components used as the input with musical scores. To address these issues, the proposed method 1) introduces a generalized command-response (GCR) model to represent an F0 contour as an approximation of a physical F0 production mechanism, and 2) employs a conditional VAE (CVAE) to treat musical scores and the noise components separately. The experimental results reveal that the proposed method achieves comparable performance as VAE-SPACE without the manual adjustment of noise components and makes it possible to control F0 contours more intuitively by using the trained GCR model.