Audio Compensation Network: to Improve the Quality of Low-Energy Audio in Visual Sound Separation

Abstract

Separating the single audio from the mixed audio has always been a task that researchers are trying to achieve. In visual audio, visual information can be used as an aid to audio separation, helping us to separate audio better. However, under the current method, the independent sound separated from the mixed video has the phenomenon that high energy audio causes serious interference to the low energy audio. We call it "over-complete separation" and "incomplete separation", which seriously affects our separation. This paper proposes a new separation network, which can compensate for the loss of low-energy audio in the separation process, improve the signal-to-noise ratio and loudness of low-energy audio, and achieve better results than state-of-the-art methods on our dataset.