SB-SENet: Diffusion model based on Schrödinger bridge for speech enhancement

Abstract

Score-based generative models and diffusion models are increasingly being applied in the field of speech enhancement, demonstrating remarkable performance. However, the lack of accurate structural information in mixed speech samples, which combine speech and Gaussian noise, still poses inference challenges and thereby affects speech quality. This paper introduces a novel generative model, SB-SENet, based on the Schrödinger bridge for speech enhancement. The Schrödinger bridge constructs the optimal transport path from the initial probability distribution to the target probability distribution by minimizing the Kullback-Leibler divergence cost function. This process is part of the entropy-regularized optimal path solution, aiming to approximate the noisy speech sample to the clean speech sample through probability distributions to obtain the predicted sample. Unlike diffusion models, which first learn the forward diffusion process from the noisy speech sample to a Gaussian distribution, SB-SENet directly learns the nonlinear diffusion process from the noisy speech sample to the clean speech sample, preserving more structural information about the initial sample. SB-SENet model utilizes a Transformer to capture unique features of time-series signals and a U-Net network to fuse multi-scale information. The loss function is constructed using a score-based generative framework, incorporating phase loss, magnitude loss, and metric loss to gradually reduce the difference between the predicted sample and the clean speech sample. Experimental results show that, in terms of speech quality, the SB-SENet model proposed in this paper achieves a PESQ score of 3.79 on the Voicebank+DEMAND dataset and a PESQ score of 3.65 on the DNS Challenge dataset, achieving state-of-the-art performance compared to recent speech enhancement models. We put the demo sample on a website https://svcodec.github.io/SB-SENet.github.io/.