Dynamic spectrum anti-jamming with distributed learning and transfer learning

Abstract

Physical-layer security issues in wireless systems have attracted great attention. In this paper, we investigate the spectrum anti-jamming (AJ) problem for data transmissions between devices. Considering fast-changing physical-layer jamming attacks in the time/frequency domain, frequency resources have to be configured for devices in advance with unknown jamming patterns (i.e. the time-frequency distribution of the jamming signals) to avoid jamming signals emitted by malicious devices. This process can be formulated as a Markov decision process and solved by reinforcement learning (RL). Unfortunately, state-of-the-art RL methods may put pressure on the system which has limited computing resources. As a result, we propose a novel RL, by integrating the asynchronous advantage actor-critic (A3C) approach with the kernel method to learn a flexible frequency pre-configuration policy. Moreover, in the presence of time-varying jamming patterns, the traditional AJ strategy can not adapt to the dynamic interference strategy. To handle this issue, we design a kernel-based feature transfer learning method to adjust the structure of the policy function online. Simulation results reveal that our proposed approach can significantly outperform various baselines, in terms of the average normalized throughput and the convergence speed of policy learning.