Multimodal Classification of Emotions in Latin Music

Abstract

In this study we classified the songs of the Latin Music Mood Database (LMMD) according to their emotion using two approaches: single-step classification, which consists of classifying the songs by emotion, valence, arousal and quadrant; and multistep classification, which consists of using the predictions of the best valence and arousal classifiers to classify quadrants and the best valence, arousal and quadrant predictions as features to classify emotions. Our hypothesis is that breaking the emotion classification in smaller problems would reduce complexity and improve results. Our best single-step emotion and valence classifiers used multimodal sets of features extracted from lyrics and audio. Our best arousal classifier used features extracted from lyrics and SMOTE to mitigate the dataset imbalance. The proposed multistep emotion classifier, which uses the predictions of a multistep quadrant classifier, improved the single-step classifier performance, reaching 0.605 of mean f-measure. These results show that using valence, arousal, and consequently, quadrant information can improve the prediction of specific emotions.