Towards fully autonomous orbit management for low-earth orbit satellites based on neuro-evolutionary algorithms and deep reinforcement learning

Abstract

The recent advances in space technology are focusing on fully autonomous, real-time, long-term orbit management and mission planning for large-scale satellite constellations in Low-Earth Orbit (LEO). Thus, a pioneering approach for autonomous orbital station-keeping has been introduced using a model-free Deep Policy Gradient-based Reinforcement Learning (DPGRL) strategy explicitly tailored for LEO. Addressing the critical need for more efficient and self-regulating orbit management in LEO satellite constellations, this work explores the potential synergy between Deep Reinforcement Learning (DRL) and Neuro-Evolution of Augmenting Topology (NEAT) to optimize station-keeping strategies with the primary goal to empower satellite to autonomously maintain their orbit in the presence of external perturbations within an allowable tolerance margin, thereby significantly reducing operational costs while maintaining precise and consistent station-keeping throughout their life cycle. The study specifically tailors DPGRL algorithms for LEO satellites, considering low-thrust constraints for maneuvers and integrating dense reward schemes and domain-based reward shaping techniques. By showcasing the adaptability and scalability of the combined NEAT and DRL framework in diverse operational scenarios, this approach holds immense promise for revolutionizing autonomous orbit management, paving the way for more efficient and adaptable satellite operations while incorporating the physical constraints of satellite, such as thruster limitations.