FNSE-SBGAN: Far-field speech enhancement with Schrödinger bridge and generative adversarial networks

Abstract

The prevailing strategy for neural speech enhancement employs purely-supervised deep learning with simulated pairs of far-field noisy-reverberant speech (i.e., mixtures) and clean speech. However, trained models frequently reveal restricted generalizability to real-recorded mixtures. This limitation is primarily due to the inherent discrepancies between simulated and real-world acoustic environments. To address this issue, this study investigates training enhancement models directly on real mixtures. Specifically, we revisit the single-channel far-field to near-field speech enhancement (FNSE) task, focusing on real-world data characterized by low signal-to-noise ratio (SNR), high reverberation, and mid-to-high-frequency attenuation. We propose FNSE-SBGAN, which presents a unique solution by merging a Schrödinger Bridge (SB)-based diffusion model with generative adversarial networks (GANs). The framework is specifically crafted to directly enhance real-world far-field speech signals. Our experimental results demonstrate FNSE-SBAN's exceptional performance across multiple metrics. It notably decreases the character error rate (CER) by up to 14.58% compared to far-field signals while preserving superior subjective quality, establishing a new benchmark for real-world far-field speech enhancement. Additionally, we introduce an evaluation framework leveraging matrix rank analysis in the time-frequency domain, providing systematic insights into model performance and revealing the strengths and weaknesses of different generative methods.