BERT Multilingual and Capsule Network for Arabic Sentiment Analysis

Abstract

Sentiment Analysis (SA) is one of the fast-growing research tasks in Natural Language Processing (NLP), which aims to identify the attitude of online users or communities regarding to a specific topic, service, or product. It helps to evaluate the overall tonality of a document to make a good decision in the right direction. Arabic SA still growing slowly compared to English and Chinese languages because of some challenges including lack of corpora, Arabic language complexity, and the spread of many local Arabic dialects. The current study presents a work-in-progress for Arabic SA based on capsule network and Google BERT multi-languages.BERT Google language model achieved state-of-the-art results for several NLP tasks, that aim to pre-train deep bidirectional (right and left) representations on the context in all layers. Recently, capsule networks become one of the most successful techniques in image processing and NLP, it is consists of a group of neurons that represents different features of the same entity. Each layer in a capsule network contains many capsules that encode spatial information and the likelihood of existing input data. Not like traditional neural techniques, capsule networks allow the model to capture the likeliness of each feature and its variants, which affects positively the quality of the final predicted features. The proposed model was trained based on a small dataset. Although, the results were encouraging and competitive compared to the state-of-the-art Arabic SA models.