Spatio-Temporal Predictive Learning Using Crossover Attention for Communications and Networking Applications

Abstract

This paper investigates the spatio-temporal predictive learning problem, which is crucial in diverse applications such as MIMO channel prediction, mobile traffic analysis, and network slicing. To address this problem, the attention mechanism has been adopted by many existing models to predict future outputs. However, most of these models use a single-domain attention which captures input dependency structures only in the temporal domain. This limitation reduces their prediction accuracy in spatio-temporal predictive learning, where understanding both spatial and temporal dependencies is essential. To tackle this issue and enhance the prediction performance, we propose a novel crossover attention mechanism in this paper. The crossover attention can be understood as a learnable regression kernel which prioritizes the input sequence with both spatial and temporal similarities and extracts relevant information for generating the output of future time slots. Simulation results and ablation studies based on synthetic and realistic datasets show that the proposed crossover attention achieves considerable prediction accuracy improvement compared to the conventional attention layers.