Hierarchical long and short-term preference modeling with denoising Mamba for sequential recommendation

Abstract

Recent advancements in Mamba-based models have shown promising potential for sequential recommendation due to their linear scalability. However, existing Mamba-based approaches still suffer from three key limitations: (1) insufficient capability in modeling short-term user preference transitions, (2) limited robustness to noise in long interaction sequences, and (3) insufficient exploitation of rich side information (e.g., item attributes). To address these challenges, we propose HLSDMRec, a hierarchical preference modeling model that integrates a denoised Mamba module for capturing robust long-term preferences and a Local LSTM module for learning fine-grained short-term preferences. HLSDMRec adopts a hierarchical dual-path architecture that jointly models item ID and side information sequences, extracting both long and short-term preferences from each. To ensure representation consistency, a hierarchical alignment module is applied and a motivation-aware gating mechanism adaptively fuses the extracted signals based on user intent. Experiments on four datasets, including Amazon Beauty (0.19M interactions), Sports (0.29M interactions), ML-1M (1M interactions), and ML-10M (10M interactions), demonstrate average improvements of 6.06% in HR@5, 4.75% in HR@10, 11.25% in NDCG@5, and 10.66% in NDCG@10 over the baseline models. The source code for our model is publicly available at https://github.com/rookie2568/hlsdmrec.