Commonsense Validation for Arabic Sentences using Deep Learning

Abstract

Arabic language processing is one of the largest challenges faced by researchers due to the complex nature of the Arabic language traits. With the recent advancements in deep learning applications and the vast amounts of data available, it is possible to secure human-like performance in several tasks. One of these tasks is distinguishing the common-sense in sentences in different languages in general, and in Arabic in specific. The Arabic commonsense validation is still a growing area in tackling the Arabic language understanding domain. In this paper, a set of multilingual pre-trained transformer models were employed. These models were trained and tested using a dataset of the commonsense understanding in the Arabic field. The dataset contains 12 thousand pairs of sentences. The two sentences, in every pair, are very similar in syntax; however, one of these sentences is logical (makes sense) and the other is illogical. BERT, XLM-MLM and XLM-Roberta models were fine-tuned to discover the logical sentences in the sentence pairs. The best result was achieved using XLM-Roberta with an accuracy equals to 81.2%.