Emotion Detection System for Malayalam Text using Deep Learning and Transformers

Abstract

Recent advances in Natural Language Processing (NLP) have improved the performance of the systems that perform tasks, such as Emotion Detection (ED), Information Retrieval, Translation, etc., in resource-rich languages like English and Chinese. But similar advancements have not been made in Malayalam due to the dearth of annotated datasets. Because of its rich morphology, free word order and agglutinative character, data preparation in Malayalam is highly challenging. In this paper, we employ traditional Machine Learning (ML) techniques such as support vector machines (SVM) and multilayer perceptrons (MLP), and recent deep learning methods such as Recurrent Neural Networks (RNN) and advanced transformer-based methodologies to train an emotion detection system. This work stands out since all the previous attempts to extract emotions from Malayalam text have relied on lexicons, which are inappropriate for handling large amounts of data. By tweaking the hyperparameters, we enhanced the transformer-based model known as MuRIL to obtain an accuracy of 79%, which is then compared with the only state-of-the-art (SOTA) model. We found that the proposed techniques surpass the SOTA system available for detecting emotions in Malayalam reported so far.