Noisy training for deep neural networks

Abstract

Deep neural networks (DNN) have gained remarkable success in speech recognition, partially attributed to its flexibility in learning complex patterns of speech signals. This flexibility, however, may lead to serious over-fitting and hence miserable performance degradation in adverse environments such as those with high ambient noises. We propose a noisy training approach to tackle this problem: by injecting noises into the training speech intentionally and randomly, more generalizable DNN models can be learned. This `noise injection' technique has been well-known to the neural computation community, however there is little knowledge if it would work for the DNN model which involves a highly complex objective function. The experiments presented in this paper confirm that the original assumptions of the noise injection approach largely holds when learning deep structures, and the noisy training may provide substantial performance improvement for DNN-based speech recognition.