Fuel-aware autonomous docking using RL-augmented MPC rewards for on-orbit refueling

Abstract

The operational lifespan of satellites is constrained by finite fuel reserves, limiting their maneuverability and mission duration. On-orbit refueling offers a transformative solution, extending satellite functionality, reducing costs, and enhancing sustainability. However, the precise execution of docking maneuvers remains a critical challenge, exacerbated by fuel sloshing effects in microgravity, which introduce unpredictable disturbances. This study proposes an integrated control framework combining Model Predictive Control (MPC) and Reinforcement Learning (RL) to ensure safe and efficient docking under these dynamic conditions. Initially, a Proximal Policy Optimization (PPO)-based RL control strategy is introduced, leveraging MPC for trajectory optimization. To further enhance adaptability in highly dynamic environments, Soft Actor-Critic (SAC) is incorporated, offering superior sample efficiency and robustness against stochastic disturbances. The proposed SAC-MPC framework effectively mitigates fuel sloshing effects by balancing computational efficiency with predictive accuracy. Experimental validation is conducted in the Zero-G Lab, emulating control scenarios with 3-DoF floating platforms, while high-fidelity numerical simulations extend the study to 6-DoF dynamics with realistic sloshing behavior modeled using OpenFOAM. Comparative results demonstrate that SAC-MPC outperforms conventional RL and MPC-based methods in docking success rate, precision, and control effort. This research establishes a robust foundation for autonomous satellite docking, contributing to the viability of on-orbit refueling missions and the future of sustainable space operations.