Label tree based image classification using sum match kernel

The label tree-based classification is one of the most popular approaches for reducing the testing complexity to sublinear with the large number of classes. One of the popular approaches to generate the label tree is to apply recursively a spectral clustering algorithm to an affinity matrix for partition set of class labels into subsets, each subset corresponds to a child node of the tree. To obtain the affinity matrix from confusion matrix, a set of N binary one-versus-all classifiers is trained and applied on validation set, where N is number of classes. However, these approaches are not reliable when there are a large number of classes because it is too costly to train these classifiers. Furthermore, the affinity matrix could not reflect the real similarity among classes due to the classification accuracy can be low. In addition, the resulting label tree may not be balanced due to the objective function of spectral clustering penalizes unbalanced partitions. In this paper, to achieve better similarity measurement between classes and without using one-versus-all classifiers, we adopt the sum match kernel to get similarity matrix. Moreover, we propose a heuristic for learning balanced tree structure by adjusting the number of class labels in clusters after the spectral clustering is done. The experimental results on benchmark datasets SUN-397 and Caltech-256 show that the performance of the proposed approach outperforms significantly the other approaches.