Exploring Large Language Models’ Emotion Detection Abilities: Use Cases From the Middle East

Abstract

Emotion detection is a critical component in allowing machines to understand and respond to human emotions. In this paper, we explore the potential of pre-trained transformer-based language models, namely, GPT3.5 and RoBERTa for emotion detection in natural language processing. Specifically, we focus on examining the quality of emotion detection in LLMs and their potential as automatic labeling generators to improve accuracy. The emotional response to two significant events, the murder of Zhina (Mahsa) Amini in Iran and the earthquake in Turkey and Syria, is analyzed. We observe that GPT’s generative nature hinders its performance in fine-grained emotion classification, whereas RoBERTa’s fine-tuning abilities and extensive pre-training specifically for emotions enable more accurate predictions within a limited set of emotional labels.