Synthetic Data Generation to Mitigate the Low/No-Shot Problem in Machine Learning

Abstract

The low/no-shot problem refers to a lack of available data for training deep learning algorithms. In remote sensing, complete image data sets are rare and do not always include the targets of interest. We propose a method to rapidly generate highfidelity synthetic satellite imagery featuring targets of interest over a range of solar illuminations and platform geometries. Specifically, we used the Digital Imaging and Remote Sensing Image Generation model and a custom image simulator to produce synthetic imagery of C130 aircraft in place of real Worldview-3 imagery. Our synthetic imagery was supplemented with real Worldview-3 images to test the efficacy of training deep learning algorithms with synthetic data. We deliberately chose a challenging test case of distinguishing C130s from other aircraft, or neither. Results show a negligible improvement in automatic target classification when synthetic data is supplemented with a small amount of real imagery. However, training with synthetic data alone only achieves F1-scores in line with a random classifier, suggesting that there is still significant domain mismatch between the real and synthetic datasets.