A Study on the Performance Evaluation of Machine Learning Models for Phoneme Classification

Abstract

This paper provides a comparative performance analysis of both shallow and deep machine learning classifiers for speech recognition task using frame-level phoneme classification. Phoneme recognition is still a fundamental and equally crucial initial step toward automatic speech recognition (ASR) systems. Often conventional classifiers perform exceptionally well on domain-specific ASR systems having a limited set of vocabulary and training data in contrast to deep learning approaches. It is thus imperative to evaluate performance of a system using deep artificial networks in terms of correctly recognizing atomic speech units, i.e., phonemes in this case with conventional state-of-the-art machine learning classifiers. Two deep learning models - DNN and LSTM with multiple configuration architectures by varying the number of layers and the number of neurons in each layer on the OLLO speech corpora along with six shallow machine learning classifiers for Filterbank acoustic features are thoroughly studied. Additionally, features with three and ten frames temporal context are computed and compared with no-context features for different models. The classifier's performance is evaluated in terms of precision, recall, and F1 score for 14 consonants and 10 vowels classes for 10 speakers with 4 different dialects. High classification accuracy of 93% and 95% F1 score is obtained with DNN and LSTM networks respectively on context-dependent features for 3-hidden layers containing 1024 nodes each. SVM surprisingly obtained even a higher classification score of 96.13% and a misclassification error of less than 5% for consonants and 4% for vowels.