Predictive Regressions for Aggregate Stock Market Volatility with Machine Learning

We investigate whether machine learning techniques and a large set of financial and macroeconomic variables can be used to predict future S&P realized volatility. We evaluate the aggregate volatility predictions of regularization methods (Ridge, Lasso, and Elastic Net), tree-based methods (Random forest and Gradient boosting), and forecast combination methods. We find that the machine learning algorithms outperform autoregressive benchmark models, both statistically and economically, and that the tree-based methods perform the best. In addition to its past realizations, our analysis reveals that the main drivers of aggregate volatility are several financial and macroeconomic uncertainty proxies.