FinSentiment: Predicting Financial Sentiment Through Transfer Learning

Abstract

There is an increasing interest in financial text mining tasks. Significant progress has been made by using deep learning-based models on a generic corpus, which also shows reasonable results on financial text mining tasks such as financial sentiment analysis. However, financial sentiment analysis is still demanding work because of the insufficiency of labeled data for the financial domain and its specialized language. General-purpose deep learning methods are not as effective mainly due to specialized language used in the financial context. In this study, we focus on enhancing the performance of financial text mining tasks by improving the existing pretrained language models via NLP transfer learning. Pretrained language models demand a small quantity of labeled samples, and they could be enhanced to a greater extent by training them on domain-specific corpora instead. We propose an enhanced model FinSentiment, which incorporates enhanced versions of a number of recently proposed pretrained models, such as BERT, XLNet, RoBERTa, GPT, Llama, and T5, to better perform across NLP tasks in financial domain by training these models on financial domain corpora. The corresponding finance-specific models in FinSentiment are called Fin-BERT, Fin-XLNet, Fin-RoBERTa, Fin-GPT, Fin-Llama, and Fin-T5, respectively. We also propose variants of these models jointly trained over financial domain and general corpora. Our finance-specific FinSentiment models, in general, show the best performance across three financial sentiment analysis datasets, even when only a subpart of these models is fine-tuned with a smaller training set. Our results exhibit enhancement for each tested performance criteria on the existing results for these datasets. Extensive experimental results demonstrate the effectiveness and robustness of especially RoBERTa pretrained on financial corpora. Overall, we show that NLP transfer learning techniques are favorable solutions to financial sentiment analysis tasks. Our source code has been deposited at https://github.com/seferlab/finsentiment.