Locally enhanced denoising self-attention networks and decoupled position encoding for sequential recommendation

Abstract

Most of the existing Transformer-based models have been shown to have great advantages in sequential recommendation by modeling temporal dynamics through the self-attention mechanism. Nevertheless, the original self-attention mechanism requires the equal weighting and computation of all interactions between each item and every other item. This method presents limitations in effectively capturing shifts in users’ local interests. In addition, this approach ignores the noise in user data, while absolute position encoding leads to inaccurate sequential relations between items. An innovative Locally Enhanced Denoising Self-Attention Network and Decoupled Position Encoding for Sequential Recommendation, named LEDADP, is presented to resolve these issues. Specifically, we use the noise filtering module to convert the original data into the frequency domain to reduce the noise and achieve the purpose of filtering the noise. We integrate convolution into self-attention for local interest transfer, and we provide a multi-scale local enhanced convolution module that models local dependencies taking into account various local preferences at several scales, collecting more detailed local semantic information. Furthermore, in order to more precisely depict the sequential link between items, we additionally employ decoupled position encoding. Extensive experiments conducted on three real-world datasets : Beauty, Toys, and ML-1M. The experimental results show that compared with the suboptimal model, the proposed model has respectively improved by 2.87%, 7.83% and 4.1% on Recall@5, and by 2.03%, 4.08% and 6.8% on NDCG@5, which proves the validity of the model.