Hierarchical Deep Reinforcement Learning for cubesat guidance and control

Abstract

Advancements in Reinforcement Learning (RL) algorithms and technologies have opened up new possibilities for their use in autonomous spacecraft control. This work presents a novel Hierarchical Deep Reinforcement Learning (HDRL) agent which can autonomously achieve satellite rendezvous while maintaining attitude control. The HDRL agents presented are built on a Hierarchical Actor–Critic (HAC) framework and are compared against combined and distributed TD3 RL agents. The controller has demonstrated the ability to achieve satellite rendezvous while performing large-angle slew manoeuvres with pointing accuracies of less than five degrees and resisting environmental perturbations. To assess the controller’s feasibility a six-Degree-of-Freedom (6-DoF) spacecraft dynamics testing platform was designed and constructed. The platform is made up of a reaction wheel actuated mock CubeSat, a frictionless space environment setup for attitude testing and a robotic arm based rendezvous mission simulator.