A neural priority model for agile earth observation satellite scheduling using deep reinforcement learning

Abstract

The agile earth observation satellite scheduling problem (AEOSSP) is a time-dependent and complex combinatorial optimization challenge that has spurred extensive research for decades. Traditional methods have primarily relied on iterative searching processes to approximate near-optimal solutions, but their efficiency remains limited. To address this issue, we propose a Priority Construction Model (PCM) based on deep reinforcement learning (DRL), forming a learning-based, two-stage construction heuristic. The PCM integrates a Priority Construction Neural Network (PCNN) alongside a Backward-Slacken and Top-Insert (BS-TI) scheduling algorithm. In PCM, the PCNN sequences observation requests, while the BS-TI schedules each sequenced request in accordance with specific constraints, thus freeing the neural policy from the burden of complex constraint checking. Experimental results indicate that following a policy-gradient-based DRL training process, PCM outperforms the state-of-the-art AEOSSP iterative algorithm, achieving better average profits within an exceptionally short construction time in most scenarios. The model study further reveals that PCNN outperforms other DRL policies in terms of priority policy representation, while the PCM exhibits superior generalization capabilities across varying scales and distributions. Therefore, our proposed model presents a valuable reference solution that not only meets the large-scale and rapid response requirements of the AEOSSP but also holds potential for application in upcoming large constellations and emerging management paradigms. More importantly, we introduce a novel framework that separates the DRL optimization process from constraint management, lowering the entry barrier for applying DRL to complex problems. This makes the model adaptable to various optimization challenges in engineering and operations research, thus extending its applicability beyond the AEOSSP domain.