A hybrid continuous speech recognition system using segmental neural nets with hidden Markov models

The authors present the concept of a 'segmental neural net' (SNN) for phonetic modeling in continuous speech recognition (CSR) and demonstrate how than can be used with a multiple hypothesis (or N-Best) paradigm to combine different CSR systems. In particular, they have developed a system that combines the SNN with a hidden Markov model (HMM) system. They believe that this is the first system incorporating a neural network for which the performance has exceeded the state of the art in large-vocabulary, continuous speech recognition. To take advantage of the training and decoding speed of HMMs, the authors have developed a novel hybrid SNN/HMM system that combines the advantages of both types of approaches. In this hybrid system, use is made of the N-best paradigm to generate likely phonetic segmentations, which are then scored by the SNN. The HMM and SNN scores are then combined to optimize performance.<>