FMamba: Bi-Mamba Based on Fast Gated-Attention for Multivariate Time-Series Forecasting

Abstract

In multivariate time-series forecasting (MTSF), extracting both temporal features and intervariable correlations is crucial. While Transformer-based models excel at capturing variable correlations, their quadratic computational complexity creates significant overhead, particularly for inputs with numerous variables. Recently, Mamba, a selective state space model, has shown promise due to its linear complexity and strong temporal extraction capabilities. However, Mamba lacks the explicit capability to model intervariable dependencies found in Transformers, and its unidirectional nature limits its receptive field. To address these limitations while maintaining efficiency, we propose FMamba, a novel framework with global selective capabilities. FMamba integrates a bidirectional Mamba (Bi-Mamba) structure with Gated-Performers—an improved linear Transformer with a content-aware gating mechanism. Technically, Gated-Performers explicitly compute dependencies among input variables, while Bi-Mamba selectively models temporal dependencies. Experimental results on seven public datasets demonstrate that FMamba outperforms Mamba-based and Transformer-based baselines, reducing the mean squared error (MSE) by an average of 15.5% and 15.2%, respectively.