Enhancing nonlinear dependencies of Mamba via negative feedback for time series forecasting

Mamba is a rising model designed to distill complex patterns from historical data, providing predictive capabilities for time series forecasting tasks. Mamba’s similarity to linear-based models has been criticized due to its limited ability to capture nonlinear dependencies. In this work, we propose a novel model named Embedding Channel Attention Maclaurin Einstein Mamba (CME-Mamba¹.) based on Mamba framework, with both Embedding Channel Attention and Maclaurin mechanisms incorporated. To further address gradient vanishing issues, we integrate Einstein FFT algorithms, ensuring robust performance against abnormal behaviors of Mamba-based architectures. Extensive experiments conducted on 11 real-world datasets with different numbers of variates, domain focus and granularity, reveal that CME-Mamba achieves state-of-the-art performance in both MSE and MAE, while maintaining reasonable memory efficiency and low time cost. The robustness and credibility of all results are substantiated by a comprehensive convergence and stability analysis. Statistically, consolidated by the Friedman Nonparametric Test and the Wilcoxon Signed-Rank Test, CME-Mamba ranks the first place with significance over counterparts. In addition, in terms of time and memory analysis, CME-Mamba is among the top three models for time and memory efficiency. Despite this, our results further demonstrate that the main contributor is the Embedding Channel Attention Block, which greatly enhances nonlinear dependencies over datasets. The Einstein FFT Block effectively suppresses gradient vanishing occurrences and contributes considerably to performance improvements, driving CME-Mamba both stable and promising. Moreover, the Maclaurin Block based on negative feedback is asymptotically stable without additional gradient vanishing issues and pioneered in achieving synergies with other blocks and greatly enhances nonlinear dependencies. With enhanced nonlinear dependencies generated from the synergy effect of all the three blocks, CME-Mamba grows excellent to uncover complex paradigms and predict future states in various domains, especially improving the performance for periodic and high-variate situations, such as traffic flow management ($\approx +8\text{\%}$), electricity predictions($\approx +6\text{\%}$).