Learning-efficient orbit decision via reinforcement learning with astrodynamics-informed action parameterization

Abstract

Satellite orbit decision-making requires a reliable approach in uncertain environments. Classical guidance methods can produce high-quality trajectories but rely on repeated numerical solutions of boundary-value problems, which limits their scalability for large-scale online decision-making. Deep reinforcement learning (RL) can in principle handle nonlinear high-dimensional dynamics, yet its interaction demand makes sample efficiency a central bottleneck for orbital decision-making under high-fidelity astrodynamics models. This paper proposes an RL framework for learning-efficient orbit decision based on an astrodynamics-informed parameterized action space. Each action consists of a discrete maneuver index and a continuous parameter vector. The discrete index selects a maneuver primitive from a library of classical orbital guidance methods. The continuous parameters unify the internal degrees of freedom of these primitives. This architecture with reparameterized continuous sampling and projection enforces maneuver feasibility and generates bounded physically meaningful commands. Comprehensive simulations demonstrate that the proposed astrodynamics-informed parameterization improves sample efficiency and final performance compared with unstructured Δv action spaces and normalized baselines.