RGB-D Based Multimodal Convolutional Neural Networks for Spacecraft Recognition

Spacecraft recognition is a significant component of space situational awareness (SSA), especially for applications such as active debris removal, on-orbit servicing, and satellite formation. The complexity of recognition in actual space imagery is caused by a large diversity in sensing conditions, including background noise, low signal-to-noise ratio, different orbital scenarios, and high contrast. This paper addresses the previous problem and proposes multimodal convolutional neural networks (CNNs) for spacecraft detection and classification. The proposed solution includes two models: 1) a pre-trained ResNet50 CNN connected to a support vector machine (SVM) classifier for classification of RGB images. 2) an end-to-end CNN for classification of depth images. The experiments conducted on a novel SPARK dataset was generated under a realistic space simulation environment and has 150k of RGB images and 150k of depth images with 11 categories. The results show high performance of the proposed solution in terms of accuracy (89 %), F1 score (87 %), and Perf metric (1.8).