Developing a news classifier for Greek using BERT

Abstract

Text categorization is a significant task in the re-search field of text mining, which has recently benefited from deep neural network algorithms and advanced learning techniques that extract language models from large textual corpora. These Pre-Trained Language Models are the main components of state-of-the-art solutions in many natural language processing and text-mining tasks can be very generic, trained in generic text corpora, or domain-specific when they employ large corpora from specific application domains (e.g. social media, news, sciences, etc.). When only generic language models are available the overall performance in the task can be improved by adapting or fine-tuning the model used for the task, e.g. the classifier. Although multilingual language models are reported in the literature, such models are usually language-specific. This work presents a news article classifier, which has been trained on a small corpus and employs a Greek version of BERT language model. Comparison with existing machine learning-based classifiers shows that the proposed method outperforms well-known methods in text classification. In addition, the proposed approach allows the continuous training of the classifier through user-provided feedback on falsely classified articles.