Learning promotion policies with attention-based deep Q-networks

Abstract

In financial services, personalized promotion strategies are critical for sustaining customer engagement and driving asset growth. We present FAT-DQN, a deep reinforcement learning framework for off-line environments that models sequential decision-making as a Markov Decision Process (MDP), where promotional actions influence future changes in customer assets under management (AUM). FAT-DQN extends the standard Deep Q-Network (DQN) architecture with a multi-head self-attention mechanism over promotion–reward histories augmented by learnable temporal encodings, and applies Feature-wise Linear Modulation (FiLM) to incorporate customer-segment embeddings. To improve robustness, we employ per-customer reward normalization and evaluate policies with both ranking-based metrics and counterfactual off-policy estimators. Empirical results on real promotion logs show that FAT-DQN consistently outperforms baseline methods, yielding a higher mean NDCG@3 (0.7744) compared to Batch-Constrained deep Q-learning (BCQ, 0.7325) and DQN (0.6852). It further improves alignment between predicted and realized outcomes, achieving a Spearman correlation of 0.2584, compared to 0.1619 for BCQ and 0.1522 for DQN. Counterfactual evaluations further show that FAT-DQN delivers consistently strong off-policy estimates, confirming its robustness across evaluation settings. These findings demonstrate that attention-based architectures with modulation offer a more effective and interpretable alternative to standard reinforcement learning approaches for personalized promotion planning in financial services.