The power of words: Enhancing music mood estimation with textual input of lyrics

Music mood estimation (MME) is a key technology in mood-based music recommendation. While mainstream MME research nowadays relies on audio music analysis, exploring the significance of lyrics text in predicting song emotion is gaining attention in recent years. One major impediment to MME research is the lack of a clearly labeled and publicly available dataset annotating the emotion ratings of lyrics text and audio separately. In light of this, we compiled a dataset of 600 pop songs (iPop) from the mood ratings of 246 participants who experienced three different song sessions, lyrics text (L), audio music track (M), and the combination of lyrics text and audio music track (C). We then applied statistical analysis to estimate how lyrics text and audio contribute to a song's overall valence-arousal (V-A) mood ratings. Our results show that lyrics text are not only a valid measure for estimating a song's mood ratings but also provide supplementary information that can improve audio-only MME systems. Furthermore, a detailed examination suggests that lyrics text (L) ratings are better estimators of the overall mood ratings of a song (C) in cases where L and M ratings conflict. We then construct a MME system that employs both features extracted from lyrics text and audio music track and validate the conclusions acquired in our statistical analysis. In estimating either V or A rating, the model with lyrics text plus audio track features performs better than only the model with only lyrics text or audio track features. These results validate the statement acquired by the statistical analysis.