A Novel Hybrid Network for Arabic Sentiment Analysis using fine-tuned AraBERT model

Abstract

The pre-trained word embedding models become widely used in Natural Language Processing (NLP), but they disregard the context and sense of the text. We study in this paper, the capacity of pre-trained BERT model (Bidirectional Encoder Representations from Transformers) for the Arabic language to classify Arabic tweets using a hybrid network of two famous models;Bidirectional Long Short Term Memory (BiLSTM) and Gated Recurrent Unit (GRU) inspired by the great achievement of deep learning algorithms. In this context, we finetuned the Arabic BERT (AraBERT) parameters and we used it on three merged datasets to impart its knowledge for the Arabic sentiment analysis. For that, we lead the experiments by comparing the AraBERT model in one hand in the word embedding phase, with a statics pretrained word embeddings method namely AraVec and FastText, and on another hand in the classification phase, we compared the hybrid model with convolutional neural network (CNN), long short-term memory (LSTM), BiLSTM, and GRU, which are prevalently preferred in sentiment analysis. The results demonstrate that the fine-tuned AraBERT model, combined with the hybrid network, achieved peak performance with up to 94% accuracy.