Two-Point Neurons for Efficient Multimodal Speech Enhancement

Abstract

Here we present a two-point neuron-inspired deep convolutional net (DCN) with 18 convolutional layers for multimodal speech enhancement (MM-SE) and compare it against conventional point neuron-inspired DCN in terms of Perceptual Evaluation of Speech Quality (PESQ) and Short-Time Objective Intelligibility (STOI). We show that the two-point neuron-driven DCN performs comparably to point-neurons driven DCN by using only ≈0.2% neurons at any time during training.