UPRec: User-aware Pre-training for sequential Recommendation

Abstract

Recent years witness the success of pre-trained models to alleviate the data sparsity problem in recommender systems. However, existing pre-trained models for recommendation mainly focus on leveraging universal sequence patterns from user behavior sequences and item information, whereas ignore heterogeneous user information to capture personalized interests, which has been shown to contribute to the personalized recommendation. In this paper, we propose a simple yet effective model, called User-aware Pre-training for Recommendation (UPRec), which could flexibly encode heterogeneous user information into the sequential modeling of user behaviors. Specifically, UPRec first encodes the sequential behavior to generate user embeddings, and then jointly optimizes the model with the sequential objective and user-aware objective constructed from the user attributes and structured social graphs. Comprehensive experimental results on two real-world large-scale recommendation datasets demonstrate that UPRec can effectively enrich the user representations with user attributes and social relations and thus provide more appropriate recommendations for users.