Multi-Agent Discrete Soft Actor-Critic Algorithm-Based Multi-User Collaborative Anti-Jamming Strategy

In multi-user adversarial scenarios involving external malicious jamming and internal co-channel interference, environmental instability and increased decision-making dimensions cause traditional deep reinforcement learning (DRL)-based anti-jamming schemes to suffer from insufficient exploration. Agents must choose policies from a large action set, leading to a significant decline in anti-jamming performance. To address these issues, this paper proposes a multi-agent discrete soft actor-critic (MA-DSAC) algorithm-based collaborative anti-jamming strategy, integrating frequency, power, and modulation-coding domains. This strategy first introduces a Markov game to model and analyze the multi-user anti-jamming problem. Next, the soft actor-critic (SAC) algorithm is discretized to handle the multi-dimensional discrete action space. Finally, through information exchange between communication transceivers and based on a centralized training with decentralized execution (CTDE) framework, it is extended to a multi-agent DRL algorithm to achieve efficient multi-user cooperative anti-jamming. Simulation results show that in various anti-jamming scenarios with both fixed-mode and intelligent jammers, the proposed anti-jamming strategy’s performance improves by more than 25% compared to traditional value-based DRL strategies, including independent deep Q-network (I-DQN) and multi-agent virtual exploration in deep Q-learning (MA-VEDQL). Furthermore, through information exchange between communication transceivers, the instability problem of multi-agent DRL is effectively alleviated, enabling the communication transceivers to balance competition and cooperation. Consequently, its anti-jamming performance improves by more than 6% compared to the independent DSAC (I-DSAC) strategy.