Deep Active Learning for Image Hierarchical Classification by Introducing Dependencies and Constraints Between Classes

Abstract

Deep active learning (DeepAL) extends supervised deep learning to human-machine interactive scenarios with limited annotation budgets. Most existing DeepAL approaches for visual recognition fail to consider the intrinsic hierarchical structure and dependencies between labels. In this article, we propose a unified DeepAL framework for the aforementioned challenge, which fuses three tightly coupled techniques: 1) hierarchical dependency representation entropy (HDRE); 2) approximate class-balanced typical sampling (ACTS); and 3) local probability suppression loss. First, the HDRE provides the features of information entropy, interclass dependencies, and constraints effectively. It is used to determine the query priority of unlabeled samples. Second, the ACTS, embedded with the HDRE, is designed for querying, where the optimal sample query size of each class is derived. It excludes samples near the boundary by employing a well-designed hierarchical margin sampling. Third, the local probability suppression loss is a transfer-friendly loss function that enables the deep model to flatly fit data with a hierarchical structure. It compensates for hierarchical dependencies between classes using the local probability suppression constraint, modeling conditional and unconditional probabilities simultaneously. We conducted experiments on five public image datasets, and the results demonstrated the effectiveness of our approach.