Optimizing time-dependent multi-agile Earth observation satellite scheduling problem using deep Q-learning and ensemble heuristics

Abstract

The Multi-Agile Earth Observation Satellite Scheduling Problem (MAEOSSP) aims to maximize the total observation profits of a multi-satellite system while satisfying temporal and resource constraints. This problem has gained prominence owing to the increasing complexity of task scheduling scenarios and challenges in satellite management and control. MAEOSSP typically consists of two decision-making stages: task allocation and single-satellite scheduling. However, its NP-hard characteristic renders traditional methods susceptible to problem instance variations and computationally intensive. To address these issues, we propose the TAM-ECH, which integrates a Task Allocation Model (TAM) and an Ensemble Construction Heuristic (ECH) to solve MAEOSSP. TAM utilizes a Deep Q-learning Network (DQN) to efficiently allocate tasks to each satellite, while ECH employs ensemble heuristics for fast and stable single-satellite scheduling. Experimental results demonstrate that TAM-ECH outperforms state-of-the-art algorithms in both optimization speed and quality. On average, it achieves a profit rate 6% higher than GRILS and 30% higher than A-ALNS, with reduced solving time. Further experiments validate TAM's efficiency in task allocation and ECH's ability to provide stable, high-quality foundational assistance. Therefore, TAM-ECH provides an efficient solution to MAEOSSP, demonstrating its potential for application in upcoming large constellations and new management modes.