Forecasting Stock Market Volatility Using Housing Market Indicators: A Reinforcement Learning-Based Feature Selection Approach

Abstract

This study tackles the complex challenge of accurately predicting stock market volatility through indicators from the housing market. We propose a sophisticated Early Warning System (EWS) designed to forecast stock market instability by leveraging the predictive power of housing market bubbles. Current EWS methods often face significant hurdles, including model generalization, feature selection, and hyperparameter optimization challenges. To directly address these issues, our innovative approach utilizes a spatial attention-based Transductive Long Short-Term Memory (TLSTM) model combined with a Reinforcement Learning (RL) strategy, which is further enhanced by a novel scope loss function for refined feature selection and an Artificial Bee Colony (ABC) algorithm for hyperparameter optimization. The TLSTM model surpasses traditional LSTM models by effectively capturing subtle temporal shifts and prioritizing data points proximate to the test sample, thereby enhancing model generalization. The RL component actively refines feature selection through continuous data interaction, ensuring the model captures the most significant features and effectively mitigates the risk of overfitting. The introduction of the scope loss function strategically manages the trade-off between exploiting known data and exploring new patterns, thereby maintaining a healthy balance between accuracy and generalizability. Additionally, the customized ABC algorithm specifically optimizes hyperparameters to increase the adaptability and performance of the model under varying market conditions. We validated our EWS using data from the Korean market, achieving an impressive accuracy of 90.427%. This validation demonstrates the robust capability of the system to forecast market dynamics. Our study significantly contributes to financial analytics by providing deeper insights into the interactions between housing and stock markets, particularly during periods of market bubbles. This research not only enhances predictive accuracy but also aids in understanding complex market behaviors, thereby offering valuable tools for financial risk management and decision-making.