Multi-resolution spectral input for convolutional neural network-based speech recognition

Abstract

The convolutional deep neural network component applied frequently in current speech recognizers is trained on a context of consecutive spectral feature vectors. Here, we investigate whether we can extend the time span of this input and reduce the number of spectral features at the same time by using a multi-resolution spectrum as input. In the proposed multi-resolution scheme, the network processes the nearby neighbors of the actual frame using the standard resolution, while it applies a gradually coarser resolution for more distant frames. Using this solution, we managed to extend the input of our network to a time context of 45 frames without increasing the number of input features, and we also achieved a relative error rate reduction of 3–4% compared to the conventional high-resolution representation. We report a phone error rate of 17.0% on the TIMIT core test set, which is competitive with the best scores published on this data set.