Position Awareness Modeling with Knowledge Distillation for CTR Prediction

Abstract

Click-through rate (CTR) Prediction is of great importance in real-world online ads systems. One challenge for the CTR prediction task is to capture the real interest of users from their clicked items, which is inherently influenced by presented positions of items, i.e., more front positions tend to obtain higher CTR values. Therefore, It is crucial to make CTR models aware of the exposed position of the items. A popular line of existing works focuses on explicitly model exposed position by result randomization which is expensive and inefficient, or by inverse propensity weighting (IPW) which relies heavily on the quality of the propensity estimation. Another common solution is modeling position as features during offline training and simply adopting fixed value or dropout tricks when serving. However, training-inference inconsistency can lead to sub-optimal performance. This work proposes a simple yet efficient knowledge distillation framework to model the impact of exposed position and leverage position information to improve CTR prediction. We demonstrate the performance of our proposed method on a real-world production dataset and online A/B tests, achieving significant improvements over competing baseline models. The proposed method has been deployed in the real world online ads systems of JD, serving main traffic of hundreds of millions of active users.