A Local context enhanced Consistency-aware Mamba-based Sequential Recommendation model

Abstract

Sequential recommendation (SR) focuses on capturing users’ interests from their historical behaviors. Transformer-based SR models have demonstrated promising performance by leveraging self-attention for sequential modeling. Recently, Mamba, a novel sequential model, has shown competitive performance compared to Transformers. In SR tasks, item representation learning involves both global and local context information. While several existing SR models attempt to address this integration, they suffer from inferior performance or computational inefficiency. Moreover, existing Mamba-based SR model appears to capture only the global context information. Given Mamba’s merits in enhancing model performance and efficiency, there is substantial potential to more effectively integrate both global and local context information within a Mamba-based framework. Additionally, consistency training, which is pivotal for enhancing model performance, remains underexplored in existing SR models.

To tackle these challenges, we propose a Local Context Enhanced Consistency-aware Mamba-based Sequential Recommendation Model (LC-Mamba). LC-Mamba captures both global and local context information to improve recommendation performance. Specifically, LC-Mamba leverages a GNN-based sequence encoder to extract information from local neighbors for each item (local context information) in a graph view, while utilizing a Mamba-based sequence encoder to capture dependencies between items in the sequence (global context information) in a sequential view. Furthermore, we introduce consistency training, including model-level and representation-level consistency, to further enhance performance. Specifically, we incorporate R-Drop regularization into the Mamba-based sequence encoder to mitigate the inconsistency between training and inference caused by random dropout (model-level consistency). Additionally, we leverage contrastive learning to enhance consistency between the item representations learned from the sequential and graph views (representation-level consistency). Extensive experiments on three widely used datasets illustrate that LC-Mamba outperforms baseline models in HR and NDCG, achieving up to a 31.03% improvement in NDCG. LC-Mamba can be applied to real-world applications such as e-commerce and content platforms.