Overcoming Transformer Fine-Tuning process to improve Twitter Sentiment Analysis for Spanish Dialects

Abstract

Is there an effective Spanish Sentiment Analysis algorithm? The aim of this paper is to answer this question. The task is challenging because there are several dialects for the Spanish Language. Thus, identically written words could have several meanings and polarities regarding Spanish speaking countries. To tackle this multidialect issue we rely on a transfer learning approach. To do so, we train a BERT language model to “transfer” general features of the Spanish language. Then, we fine-tune the language model to specific dialects. BERT is also used to generate contextual data augmentation aimed to prevent overfitting. Finally, we build the polarity classifier and propose a fine-tuning step using groups of layers. Our design choices allow us to achieve state-of-the-art results regarding multidialect benchmark datasets.