Music Source Separation with Generative Adversarial Network and Waveform Averaging

Abstract

The task of music source separation is to extract a target sound from mixed sound. A popular approach for this task uses a DNN which learns the relationship of the spectrum of mixed sound and one of separated sound. However, many DNN algorithms does not consider the clearness of the output sound, this tends to produce artifact in the output spectrum. We adopt a generative adversarial network (GAN) to improve the clearness of the separated sound. In addition, we propose data augmentation by pitch-shift. The performance of DNN strongly depends on the quantity of the dataset for train. In other words, the limited kinds of the training datasets gives poor knowledge for the unknown sound sources. Learning the pitch-shifted signal can compensate the kinds of training set and makes the network robust to estimate the sound spectrum with various pitches. Furthermore, we process the pitch-shifted signals and average them to reduce artifacts. This proposal is based on the idea that network once learned can also separate pitch-shifted sound sources not only original one. Compared with the conventional method, our method achieves to obtain well-separated signal with smaller artifacts.