Bayesian attention-based user behaviour modelling for click-through rate prediction

Abstract

Exploiting the hierarchical dependence behind user behaviour is critical for click-through rate (CRT) prediction in recommender systems. Existing methods apply attention mechanisms to obtain the weights of items; however, the authors argue that deterministic attention mechanisms cannot capture the hierarchical dependence between user behaviours because they treat each user behaviour as an independent individual and cannot accurately express users' flexible and changeable interests. To tackle this issue, the authors introduce the Bayesian attention to the CTR prediction model, which treats attention weights as data-dependent local random variables and learns their distribution by approximating their posterior distribution. Specifically, the prior knowledge is constructed into the attention weight distribution, and then the posterior inference is utilised to capture the implicit and flexible user intentions. Extensive experiments on public datasets demonstrate that our algorithm outperforms state-of-the-art algorithms. Empirical evidence shows that random attention weights can predict user intentions better than deterministic ones.