Intelligent Decision-Making System of Air Defense Resource Allocation via Hierarchical Reinforcement Learning

Abstract

Intelligent decision-making in air defense operations has attracted wide attention from researchers. Facing complex battlefield environments, existing decision-making algorithms fail to make targeted decisions according to the hierarchical decision-making characteristics of air defense operational command and control. What’s worse, in the process of problem-solving, these algorithms are beset by defects such as dimensional disaster and poor real-time performance. To address these problems, a new hierarchical reinforcement learning algorithm named Hierarchy Asynchronous Advantage Actor-Critic (H-A3C) is developed. This algorithm is designed to have a hierarchical decision-making framework considering the characteristics of air defense operations and employs the hierarchical reinforcement learning method for problem-solving. With a hierarchical decision-making capability similar to that of human commanders in decision-making, the developed algorithm produces many new policies during the learning process. The features of air situation information are extracted using the bidirectional-gated recurrent unit (Bi-GRU) network, and then the agent is trained using the H-A3C algorithm. In the training process, the multihead attention mechanism and the event-based reward mechanism are introduced to facilitate the training. In the end, the proposed H-A3C algorithm is verified in a digital battlefield environment, and the results prove its advantages over existing algorithms.