Speech Enhancement Using Variational Autoencoders

Abstract

This paper discusses the experimental details of speech enhancement using variational autoencoders (VAE). A joint VAE architecture is formulated, and a training protocol that strikes a balance between speech enhancement and VAE correctness is defined. Extended short-term objective intelligibility (ESTOI) is used to measure the intelligibility of enhanced speech. The proposed approach is implemented using MFCC and STFT features on a benchmark dataset and we report, on an average, 2 times improvement in ESTOI for enhanced speech using MFCC over STFT features across all noise levels. Further, the proposed approach using MFCC features shows significant improvement in denoising very noisy speech, as opposed to marginal improvement on relatively clean speech.