Multi-level Adversarial Training for Stock Sentiment Prediction

Abstract

Stock sentiment prediction is a task to evaluate whether the investors are expecting or gaining a positive or negative return from a stock, which has a high correlation with investors’ sentiments towards the business. However, as the nature of social media, the textual information posted by ordinary people is usually noisy, inconsistent, and even grammatically incorrect, leading the model to generate unsatisfied predictions. In this paper, we improve the performance of stock sentiment prediction by applying and comparing adversarial training at multiple levels, including character, word, and sentence levels, with the utilization of three novel adversarial attack models: DeepWordBug, BAE, and Generative Adversarial Network (GAN). We also propose an effective pre-processing technique and a novel adversarial examples incorporation method to improve the prediction results. To make an objective evaluation, we select three backbone models: Embedding Bag, BERT, and RoBERTa-Twitter, and validate the models before and after adversarial training on the TweetFinSent dataset. Experimental results demonstrate remarkable improvements in the models after adversarial training, and the RoBERTa-Twitter model with word-level adversarial training performs optimally among the experimented models. We conclude that sentence-level and word-level adversarial training are the most appropriate for deep learning and pre-trained language models, respectively, and we further conduct ablation studies to highlight the usefulness of our data pre-processing and adversarial examples incorporation approaches and a case study to display the adversarial examples generated by the proposed adversarial attack models.