Research on space proximity pursuit-evasion interception decision-making based on deep reinforcement learning

Abstract

Aiming at the one-to-one pursuit-evasion problem in space, to successfully intercept the close-range evader with arbitrary counter-maneuver under relative motion between pursuer and evader at a close given range, this paper proposes a decision-making method for close-range pursuit-evasion interception based on Distributed Distributional Deep Determined Policy Gradient (D4PG). An improved nearest neighbor algorithm exploration mechanism including random constant and logarithmic constant is adopted, which reduces the learning burden of the algorithm and improves its convergence stability. A target network containing three value networks is constructed, and the loss function is calculated by selecting a value network with the minimum variance of probability distribution in the three networks, which enables the more accurate estimation of the Q-functions, and the operation speed and efficiency of the algorithm are effectively improved. Four typical escaping scenarios of arbitrary counter-maneuvering are performed as experimental verification to the simulation, and the results show the effectiveness and superiority of the proposed decision-making method for space proximity pursuit-evasion interception.