Query-by-Example on-Device Keyword Spotting using Convolutional Recurrent Neural Network and Connectionist Temporal Classification

Abstract

Keyword spotting (KWS) is an essential feature for speech-based applications on mobile devices. For the sake of reducing power consumption and improving robustness on substandard pronunciations of KWS systems, this paper proposes a query-by-example on-device keyword spotting system using Convolutional Recurrent Neural Network (CRNN) and Connectionist T emporal Classification (CTC). CRNN is to directly predict the phoneme posterior probabilities, and CTC is to calculate the scores for the output phoneme sequences. To reduce the computational costs, the CRNN-based model is then simplified, and a template generator is built for generating keyword templates based on Dynamic Time Wrapper (DTW). The proposed KWS system has low computational requirements and is well-suited for both enrollment and inference on lower-power devices. It has competitive performance in comparison with other query-byexample systems, and has achieved the standards of the commercial application level, even in the condition of noise or under far-field environment.