Variable reward function-driven strategies for impulsive orbital attack-defense games under multiple constraints and victory conditions

This paper investigates impulsive orbital attack-defense (AD) games under multiple constraints and victory conditions, involving three spacecraft: attacker, target, and defender. In the AD scenario, the attacker aims to breach the defender's interception to rendezvous with the target, while the defender seeks to protect the target by blocking or actively pursuing the attacker. Four different maneuvering constraints and five potential game outcomes are incorporated to more accurately model AD game problems and increase complexity, thereby reducing the effectiveness of traditional methods such as differential games and game-tree searches. To address these challenges, this study proposes a multi-agent deep reinforcement learning solution with variable reward functions. Two attack strategies, Direct attack (DA) and Bypass attack (BA), are developed for the attacker, each focusing on different mission priorities. Similarly, two defense strategies, Direct interdiction (DI) and Collinear interdiction (CI), are designed for the defender, each optimizing specific defensive actions through tailored reward functions. Each reward function incorporates both process rewards (e.g., distance and angle) and outcome rewards, derived from physical principles and validated via geometric analysis. Extensive simulations of four strategy confrontations demonstrate average defensive success rates of 75% for DI vs. DA, 40% for DI vs. BA, 80% for CI vs. DA, and 70% for CI vs. BA. Results indicate that CI outperforms DI for defenders, while BA outperforms DA for attackers. Moreover, defenders achieve their objectives more effectively under identical maneuvering capabilities. Trajectory evolution analyses further illustrate the effectiveness of the proposed variable reward function-driven strategies. These strategies and analyses offer valuable guidance for practical orbital defense scenarios and lay a foundation for future multi-agent game research.