Coupling LSTM neural networks and state-space models through analytically tractable inference

Abstract

Long short-term memory (LSTM) neural networks and state-space models (SSMs) are effective tools for time series forecasting. Coupling these methods to exploit their advantages is not a trivial task because their respective inference procedures rely on different mechanisms. In this paper, we present formulations that allow for analytically tractable inference in Bayesian LSTMs and the probabilistic coupling between Bayesian LSTMs and SSMs. This is enabled by using analytical Gaussian inference as a single mechanism for inferring both the LSTM’s parameters as well as the posterior for the SSM’s hidden states. We show through several experimental comparisons that the resulting hybrid model retains the interpretability feature of SSMs, while exploiting the ability of LSTMs to learn complex seasonal patterns with minimal manual setups.