Word Embedding in Small Corpora: A Case Study in Quran

Abstract

Text is a complex set of words to carry the meaning and representations of words is the first step to perform linguistic processing and text comprehension. So far, many researches have been done on the semantic representations of words using neural networks in various areas of natural language processing using large text corpus from general domain. In the meantime, some efforts have been done to apply deep learning methods to represent the words of small corpus supporting the hypothesis that the bigger corpora doesn't necessarily provide better results in words representation. In this research the capability of word2vec for learning semantic representation of words in small corpus is investigated. Here, we consider Skip-gram and CBOW learning models with different values of hyper parameters. Two new data sets have been created to evaluate the model's performance on the small domain-specific Quranic corpus. First and second datasets are used to test the words categorization and word pairwise similarity respectively. Our results demonstrate that the best performance for skip-gram occurs with 30 numbers of iterations when the dimension is set to 7.