Multiclass Imbalanced Classification of Quranic Verses Using Deep Learning Approach

Abstract

This paper uses deep learning approach for the classification of Quranic verses. The dataset has an imbalance, hence first it is balanced by oversampling. This paper aims to classify the verses using Bidirectional Encoder Representation from Transformers (BERT) word embedding by considering the context of words. BERT reads a word with all its neighboring words and assigns representations accordingly. Furthermore, to ensure that the classifier remembers the most important part of the input sequence, deep learning classifiers with Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) are used for classification. As the BERT word embeddings of the text data are created, they are fed to 3 Neural Network (NN) models i.e. NN with LSTM which achieved f1-score and accuracy of 0.85 for uncased, and f1-score of 0.82 and accuracy of 0.83 for cased embedding; NN with GRU which achieved F1-score and accuracy of 0.89 for uncased and 0.90 for cased embedding; and fine-tuned BERT model which achieved F1-scores of 0.93 and accuracy of 0.98 for both base-uncased and base-cased embedding.