Applying Sentiment Analysis, Topic Modeling, and XGBoost to Classify Implied Volatility

Abstract

Implied volatility is an important indicator that shows the market participants’ expectations about the future fluctuations in the options market. This paper evaluates the question of whether the combination of topics and sentiment scores extracted from mainstream financial news could improve forecasting the directional changes of the expected implied volatility index in the next month (iv30call). We select six stocks from the Dow Jones list of companies and acquire over 190,000 news published between January 2019 and September 2019. By building text processing and topic modeling pipelines, we can examine (i) the role of daily mean and medium of sentiment scores; and (ii) the influence of topic models on the classification metrics. The results demonstrate that adding a topic model has a positive effect on the model’s accuracy, which reaches higher accuracy in classifying the iv30call of the next business day in five out of six companies. The outcome suggests that applying the mean of the daily sentiment scores improves the models’ accuracy compared to the daily median for the selected assets.