Multi-label Aerial Image Classification Based on Image-Specific Concept Graphs

Abstract

Multi-label aerial image classification (MAIC) is a fundamental but challenging task for computer vision-based remote sensing applications. Existing MAIC models suffer from the insufficient semantic information of image and label representations. To this end, we integrate commonsense knowledge into the MAIC task and propose a novel Knowledge-augmented Concept Graph Learning (KCGL) framework. KCGL first collects relevant semantic concepts for each label from a commonsense knowledge graph ConceptNet. With the guidance of semantic concepts, an image decoupling module is employed to extract concept-specific image features from the input image. Then, KCGL constructs an individual concept graph for each image, in which nodes are corresponding to concept-specific image features and edges are their relations extracted from ConceptNet. Finally, the classification probability on each label is computed in the specific concept graph via a GCN-based encoder-decoder model. Experimental results prove that the proposed KCGL outperforms existing state-of-the-art MAIC models on two aerial image datasets.