Design of single-channel speech enhancement algorithm in noisy acoustic environments

Abstract

In speech enhancement, Transformers and Self-Attention-based denoising networks are widely used and perform well, and speech enhancement serves as a valuable front-end for speech recognition. However, existing dual-branch architectures lack sufficient natural speech phase extraction due to the phase spectrum’s sensitivity and easy compensation, and traditional dilated convolution architectures are unsuitable for resource-constrained devices, creating an urgent need for lightweight alternatives. Thus, this paper proposes TFEM-PHASEN-MINI, a discrete dual-branch phase extraction architecture based on the Base and Detail Feature Modules. It uses DilatedReparamBlock to replace the Dense Encoder’s dilated convolution module, balancing computational efficiency and performance by fusing Convolutional Neural Networks and Transformers. It also designs a time-frequency feature extraction module to verify integrating speech recognition modules into speech enhancement, and adds a Phase Enhancement Module to address insufficient phase-spectrum speech phoneme feature extraction (caused by magnitude spectrum over-compensation) via parallel phase estimation. On the VoiceBank+DEMAND dataset, it achieves scores of 3.44, 4.72, 4.18, 17.13, 2.10, and 0.96 for PESQ, CSIG, COVL, FWSSNR, CEPS, and STOI, respectively. On the DNS-Challenge dataset, it attains scores of 3.20 and 3.57 for WB-PESQ and NB-PESQ, respectively. On the EARS-WHAM testset and its blind testset, it improves the metrics of PESQ, CSIG, CBAK, COVL, SSNR, FWSSNR, CEPS, and STOI by 0.56, 1.00, 0.94, 0.83, 8.42, 5.26, 0.21, and 0.15 respectively, and achieves non-intrusive metrics (Overall Quality of 3.80, Noisiness of 4.18, Discontinuity of 4.32, Coloration of 3.85, Loudness of 3.45), showing optimal generalization. Though it has relatively lower CBAK and SSNR on the VoiceBank+DEMAND dataset, it remains overall advanced. Computational complexity and device inference tests verify the balance between its computational efficiency and accuracy.