Classification of Social Signals Using Deep LSTM-based Recurrent Neural Networks

Non-linguistic speech cues aid expression of various emotions in human communication. In this work, we demonstrate the application of deep long short-term memory (LSTM) recurrent neural networks for frame-wise detection and classification of laughter and filler vocalizations in speech data. Further, we propose a novel approach to perform classification by incorporating cluster information as an additional feature wherein the clusters in the dataset are extracted via a k-means clustering algorithm. Extensive simulation results demonstrate that the proposed approach achieves significant improvement over the conventional LSTM-based classification methods. Also, the performance of deep LSTM models obtained by stacking LSTMs, is studied. Lastly, for classification of the temporally correlated speech data considered in this work, a comparison with popular machine learning-based techniques validates the superiority of the proposed LSTM-based scheme.