TBformer: Multi-Scale Transformer With Time-Behavior Attention for Multi-Modal Customer Churn Prediction

Abstract

In highly competitive market of Internet service platforms, identifying and retaining potential churners through customer churn prediction techniques is crucial for maintaining platform vitality. The sequences of interaction behaviors between customers and platforms are closely related to churn prediction results. However, existing methods focus only on capturing the temporal dependencies in dynamic behavior sequences while ignoring the correlations between different behaviors. Moreover, classical methods apply only to static data, while deep learning-based methods focus on dynamic data, neither leveraging the complementary information between static and dynamic data. To address these issues, we propose a multi-modal customer churn prediction model based on Transformer with multi-scale Time-Behavior attention, TBformer, which adaptively fuses static and dynamic data. Time-Behavior module can capture multi-scale temporal dependencies and behavioral correlations in behavioral time series across time and behavior dimensions. We perform behavior-independent multi-scale dynamic feature fusion through bidirectional connection paths. Furthermore, the multi-modal fusion module based on the attention mechanism adaptively controls the fusion weights of static and dynamic features to improve performance. Extensive experiments on two publicly available datasets, KKBox and KDD, and a private dataset, HOF, demonstrate that our TBformer achieves an average AUC of 91.2% (+2.47%), outperforming the state-of-the-art customer churn prediction methods.